<?xml version="1.0" standalone="yes"?>
<Paper uid="P85-1030">
  <Title>OF L MF MF MF ME Get Swed Core Core OF L</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
AT&amp;T Boll Laboratories
Abreact
</SectionTitle>
    <Paragraph position="0"> This paper will discuss how to determine word stress from spelling.</Paragraph>
    <Paragraph position="1"> Stress assignment is a well-established weak point for many speech synthesizers because stress dependencies cannot be determined locally.</Paragraph>
    <Paragraph position="2"> It is impossible to determine the stress of a word by looking through a five or six character window, as many speech synthesizers do. Well-known examples such as degrade / dbgradl, tion and tMegraph / telegraph5 demonstrate that stress dependencies can span over two and three syllables. This paper will pre~nt a principled framework for dealing with these long distance dependencies. Stress assignment will be formulated in terms of Waltz' style constraint propagation with four sources of constraints: (1) syllable weight. (2) part of speech. (3) morphology and (4) etymology. Syllable weight is perhaps the most interesting, and will be the main focus of this paper. Most of what follows has been implemented.</Paragraph>
    <Paragraph position="3"> I. Back~e,,sd A speech synthesizer is a machine that inputs a text stream and outputs an accoustic signal. One small piece of this problem will be discussed here: words -- phonemes. The resulting phonemes are then mapped into a sequence of Ipe dyads which are combined with duration and pitch information to produce speech.</Paragraph>
    <Paragraph position="4"> text -- intonation phrases -- words phonemes -- Ipc dyads + prosody -- accoustiC/ -~ There are two general approaches to word -- phonemes:</Paragraph>
  </Section>
  <Section position="2" start_page="0" end_page="246" type="metho">
    <SectionTitle>
* Dictionary Lookup
</SectionTitle>
    <Paragraph position="0"> * Letter to Sound (i.e.. sound the word out from basic principles) Both approaches have their advantages and disadvantages; the dictionary approach fails for unknown words (e.g.. proper nouns) and the letter to sound approach fails when the word doesn't follow the rules, which happens all too often in English. Most speech synthesizers adopt a hybrid strategy, using the dictionary when appropriate and letter to sound for the rest.</Paragraph>
    <Paragraph position="1"> Some people have suggested to me that modern speech synthesizers should do away with letter to sound rules now that memory prices are dropping so low that it ought to be practical these days to put every word of English into a tiny box. Actually memory prices are still a major factor in the cost of a machine. But more seriously, it is not possible to completely do away with letter to sound rules because it is not possible to enumerate all of the words of English. A typical college dictionary of 50,000 hcadwords will account for about 93% of a typical newspaper text. The bulk of the unknown words are proper flOUfl-q.</Paragraph>
    <Paragraph position="2"> The difficulty with pmpor nouns h demonstrated by the table below which compares the Brown Corpus with the surnames in the Kansas City Telephone Book. The table answers the question: how much of each corpus would be covered by a dictionary of n words? Thus the first line shows that a dictionary of 2000 words would cover 68% of the Brown Corpus, and a dictionary of 2000 names would cover only 46% of the Kansas City Telephone Book. It should be clear from the table that a dictionary of surnames must be much targar than a typical college dictionary ('20,000 entries). Moreover. it would be a lot of work to consu'u~ such a dictionary since there are no existing computer readable dictionaries for surnames.</Paragraph>
    <Paragraph position="3">  Actually, this table overestimates the effectivene~ of the dictionary, for practical applications. A fair test would not use the same corpus for both selecting the words to go into the dictionary and for testing the coverage. The scores reported here were computed post hoc, a classic statistical error, l tried a more fair test, where a dictionary of 43777 words (the entire Brown Corpus) was tested against a corpus of 10687 words selected from the AP news wire. The results showed 96% coverage, which is slightly lower (as expected) than the 99% figure reported in the table for a 40000 dictionary.</Paragraph>
    <Paragraph position="4"> For names, the facts are much more striking as demonstrated in the following table which teats name lists of various sizes against the Bell Laboratories phone book. (As above, the name lists were gathered from the Kansas City Telephone Book.)*  Note that the asymptote of 60% coverage is quickly reached after only about 5000-1000 words, su88estiog (a) that the dictionary appnxtch may only be suitable for the 5000 to 1000 mint frequent names because larger dictionaries yield only negligible improvements in performance, and (b) that the dictionary approach has an inherent limitation on coverage of about 60%. To increase the coverage beyond this, it is probably neceqsary to apply alternative methods such as letter to sound rules.</Paragraph>
    <Paragraph position="5"> Over the past year l have been developing a set of letter to sound rules as part of a larger speech synthesis project currently underway at Murray Hill. Only one small piece of my letter to sound rules, orthography ~ stress, will be discussed here. The output streu assignment is then used to condition a number of rules such as palatalization in the mapping from letters to phonemes.</Paragraph>
    <Paragraph position="6"> 2. we/ght as ~ i,termt~tm ~ of Relm~mmutm Intuitively, stre~s dependencies come in two flavors: (a) those that apply locally within a syllable, and (b) throe that apply globally between syllables. Syllable weight is an attempt to represent the local stress constraints. Syllables are marked either heavy or light, depending only on the local 'shape' (e.g., vowel length and number of Ix~t-vocalic consonants). Heavy syllables are more likely to be * Admittedly. this teat is somewhat unfair to the dictionary appma(c)h sinca: thu ethnic mzxture in gamuut City is very differeat from that found here at Bell t.aboflltot~ stressed than light syllables, though the actual outcome depends upon contextual constraints, such as the English main stress rule, which will be d~ shortly.</Paragraph>
    <Paragraph position="7"> The notion of weight is derived from Chomsky and Halle's notion of strong and weak clusters \[Chonuky and Halle\] (SPE). In phonological theory, weight is used as an intermediate level of representation between the input underlying phonological representation and the output stress aaignment. In a similar fashion, \[ will use weight as an intermediate level of representation between the input orthography and the output strew. The orthography -- stress problem will be split into two subproblems:  * Orthography -- Weight * Weight ~ Stress 3. What is Sy~ Weight:  Weight is a binary feature (Heavy or Light) assigned to each syllable. The final syllables of the verbs obey, maintain, erase, torment.</Paragraph>
    <Paragraph position="8"> collapse, and exhaust arc heavy because they end in a long vowel or two consonants, in constrast, the final syllables of develop, astonish. edit. consider, and promise are light because they end in a short vowel and at moat one consonant. More precisely, to compute the weight of a syllable from the underlying phonological representation, strip off. the final consonant and then pane the word into syllables (assigning C/omommts to the right when there is ambiguity).</Paragraph>
    <Paragraph position="9"> owK y Weight Rea.~oa heavy final syllable long vowel tor-men heavy final syllable closed syllable diy-ve-lo light final syllable open syllable &amp; short vowel Then. if the syllable is clo~ (i.e., ends in a consonant as in tor.men) or if the vowel is marked underiyingly long (as in ow.bey), the syllable is marked heavy. Otherwise, the syllable ends in an open short vowel and it is marked light. Determining syllable weight from the orthography is considerably more difficult than from the underlying phonological form. I will return to this question shortly.</Paragraph>
  </Section>
  <Section position="3" start_page="246" end_page="32020103" type="metho">
    <SectionTitle>
4. we/slt -- Stnm
</SectionTitle>
    <Paragraph position="0"> Global stress assignment rules apply off&amp;quot; the weight representation. For example, the main stress rule of English says that verbs have final stress if the final syllable is heavy syllable (e.g., obey), and penultimate stress if the final syllable light syllable (e.g., develop). The main stress rule works similarly for nouns, except that the final syllable is ignored (extrametrical \[Hayes\]). Thus, nouns have penultimate stress if the penultimate syllable is heavy (e.g, aroma) and antipenultimate stress if the penultimate syllable is light (e.g., cinema).</Paragraph>
    <Paragraph position="2"> heavy long vowel verr6nda heavy closed syllable cinema light open syllabic &amp; short vowel  Adjectives stress just like verbs except suffixes are ignored (extrametrical). Thus monomorphemic adjectives such as diacr~et, robfist and cbmmon stress just like verbs (the final syllable is stressed if it is heavy and otherwise the penultimate syllable is stress) whereas adjectives with single syllable suffixes such as -al, -oas. -ant, -ent and -ire follow the same pattern as regular nouns \[Hayes, p. 242\]. Stress Pattera of Suffixed Adjectives Light Penultimate Hury Peaaidmate Heavy Pmultimale municipal adjectival frat&amp;'nai magn~minous desirous trem~ndoas significant clairv6yant relfictant innocent complY, cent dep6'ndent primitive condficive exp~-nsive S. SWeat's WeiOt Table A large number of phonological studies (e.g., \[Chomsky and HalleL \[Liberman and PrineeL \[Hayes\]) outline a deterministic procedure for assigning stress from the weight representation and the number of extrametrical syllables (1 for nouns, 0 for verbs). A version of this procedure was implemented by Richard Sproat last summer. For efficiency purposes. Sproat's program was compiled into a table,, which associated each possible input with the appropriate stress pattern.</Paragraph>
    <Paragraph position="3">  etc.</Paragraph>
    <Paragraph position="4"> Note that the table is extremely small. Assuming that words have up N to N syllables and up to E extrametrical syllables, there are E~2 ~ possible inputs. For E - 2 and N - 8, the table has only 1020 entries, which is not unreasonable.</Paragraph>
    <Paragraph position="5"> 6. Amlolff with Walt-' Comtndat Prolmptiea Paradigm Recall that Waltz was the first to showed how contraints could be used effectively in his program that analyzed line drawings in order to separate the figure from the ground and to distinguish concave edges from convex ones. He first assigned each line a convex label (+), a concave label (-) or a boundary label (&lt;, &gt;), using only ~ocal information. If the local information was ambiguous, he would assign a line two or more labels. Waltz then took advantage of the constraints impmed where multiple lines come together at a common vertex. One would think th~ t there ought to be 42 ways to label a vertex of two lines and 4 '~ ways to label a vertex of three lines and so on. By this argument, there ought to be 208 ways to label a vertex. But Waltz noted that there were only 18 vetex labelings that were consistent with certain reasonable assumptions about the physical world. Because the inventory of possible labelings was so small, he could disambiguate lines with multiple assignments by checking the junctures at each end of the line to see which of the assignments were consistent with one of the 18 possible junctures. This simple test turned out to be extremely powerful.</Paragraph>
    <Paragraph position="6"> Sproat's weight table is very analogous with Waltz' list of vertex constraints; both define an inventory of global contextual constraints on a set of local labels (H and L syllables in this application, and +. -, &gt;, &lt; in Waltz application). Waltz' constraint propagation paradigm depends on a highly constrained inventory of junctures. Recall that only 18 of 208 possible junctures turned out to be grammatical. Similarly, in this application there are very strong grammatical constraints. According to Spmat's table, there are only 51 distinct output stress a.udgnmeats, a very small number considering that there are 1020 distinct inputs.</Paragraph>
    <Paragraph position="7">  The strength of these constraints will help make up for the fact that the mapping from orthography to weight is usually underdetermined, In terms of information theory, about half of the bits in the weight representation arc redundant since log 51 is about half of log 1020. This means that I only have to determine the weight for about half of the syllables in a word in order to assign stress.</Paragraph>
    <Paragraph position="8"> The redundancy of the weight representation can also been seen directly from Sproat's weight table as shown below For a one syllable noun, the weight is irrelevant. For a two syllable noun, the weight of the penultimate is irrelevant. For a three syllable noun, the weight of  the antipenultimate syllable is irrelevant if the penultimate is light. For a four syllable noun, the weight of the antipenultimate is irrelevant if the penultimate is light and the weight of the initial two syllables are irrelevant if the penultimate is heavy. These redundancies follow, of course, from general phonological prin~ples of stresa assignment.  For practical purposes, Sproat's table offers a complete solution to the weight -- stress subtask. All that remains to be solved is: orthography weight. Unfortunately, this problem is much more dif~cult and much less well understood. 1'11 start by discussing some easy _~_,-e~, and then introduce the pseudo-weight heuristic which helps in some o\[ the more di~icuit cas~. Fortunately, l don't need a complete solution to orthography ~ weight since weight ~ stress is so well constrained. In easy cases, it is pmsible m determine the weight directly for the orthography. For example, the weight of torment must be &amp;quot;HH&amp;quot; because both syllables arc cloud (even after stripping off the final consonant). Thus, the stress of torment is either &amp;quot;31&amp;quot; or &amp;quot;13&amp;quot; stress depending on whether is has 0 or I extrametricai final syllables:&amp;quot;</Paragraph>
    <Paragraph position="10"> However, meet cases are not this easy. Consider a word like record where the first syllable might be light if the first vowel is reduced or it might be heavy if the vowel is underlyingly long or if the first syllable includes the /k/. It seems like it is imix~sstble to say anything in a case like this. The weight, it appears is either &amp;quot;LH&amp;quot; or &amp;quot;HH'. Even with this ambiguity, there are only three distinct stress assignments:  01, 31, and 13.</Paragraph>
    <Paragraph position="11"> AaueUy, ~ practk~. ~ ~l~t det~mm~on is ~mp~aud by t0,,, Smm~5~ -crazy ted -ew m, lht be mmx~. New, for example, ths |the tdj~:tiw ~ den ~ m'~/ike the '.~ mrm~w bin:sum Uul sdjm:trmd e~ .~w ie mumuneuncaL (stress-from-weights &amp;quot;LH&amp;quot; 0) -- ('01 &amp;quot;) (strm.(rom.weights &amp;quot;HH&amp;quot; 0) -- ('31&amp;quot;) (sirra-from-weights &amp;quot;LH&amp;quot; I) -- ('13&amp;quot;) (streas-from-weights &amp;quot;HH&amp;quot; l) -- ('13&amp;quot;)  8. Pmdee-Wekdn In fact. it is possible now to use the stress to further constrain the weight. Note that if the first syllable of record is light it must also be unstressed and if it is heavy it also must be stressed. Thus, the third line above is inconsistent.</Paragraph>
    <Paragraph position="12"> I implement this additional constraint by assigning record a pseudo-weight of &amp;quot;'-H', where the &amp;quot;-.&amp;quot; sign indicates that the weight a~sigment is constrained to be the same as the stress assigment (either heavy &amp; stressed or not heavy &amp; not stressed), \[ can now determine the possible stress assignments of the p~eudo-weight &amp;quot;.-H&amp;quot; by filling in the &amp;quot;&amp;quot;&amp;quot; constraint with all possible bindings (H or L) and testing the results to make sure the constraint is met.</Paragraph>
    <Paragraph position="14"> Of the four logical inputs, the -- constraint excludes the third case which would assign the first syllable a stress but not a heavy weight.</Paragraph>
    <Paragraph position="15"> Thus, there are only three possible input/output relations meeting all  weight -, H; stress is unknown weight -- L; stress is unknown (weight - H) ~ (stress - O) weight - H; stress ~ 0 weight - L: stress - 0 (weight - H) =~ (stress - 0)  weight is unknown: stress is unknown The eoun should ~mbebly have the mm tO rtt~. tMm d~ nress \[3. t u~ that te exmtmaCricef syllabk Ms 3 ~eus if it is buy% and 0 Irns if it is UZ,~t. l&amp;quot;~e ~es8 of tM estrsme~L-sJ 8ylhd~hr is ~ diR'lcz~t ~ is.edict, as dilc~Jsetd ~ou\].  \[ have already given examples of the labels H, L and -. S and R are used in certain morphological analyses (see below), N is used for examples where Hayes would invoke his rule of Sonorant Destr-~ing (see below), and ? is not used except for demonstrating the program. The procedure that assigns pseudo-weight to orthography is roughly as outlined below, ignoring morphology, etymological and more special cases than \[ wish to admit.</Paragraph>
    <Paragraph position="16">  1. Tokenize the orthography so that digraphs such as th. gh. wh, ae. ai, ei, etc., are single units.</Paragraph>
    <Paragraph position="17"> 2. Parse the string of tokens into syllables (assigning =onsonants to the right when the location of the syllable boundary is ambiguous).</Paragraph>
    <Paragraph position="18"> 3. Strip off the final consonant.</Paragraph>
    <Paragraph position="19"> 4. For each syllable a. Silent e, Vocalic y and Syllabic Sonorants (e.g., .le. -er. -re) are assigned no weight.</Paragraph>
    <Paragraph position="20"> b. Digraphs that are usually realized as long vowels (e.g.. oi) are marked H.</Paragraph>
    <Paragraph position="21"> c. Syllables ending with sonorant consonants are marked N;  other closed syllables are marked H.</Paragraph>
    <Paragraph position="22"> d. Open syllables are marked -.</Paragraph>
    <Paragraph position="23"> In practice. I have observed that there are remarkably few stress assignments meeting all of the constraints. After analyzing over 20.000 words, there were no more than 4 possible stress assigments for any particular combinatton of pseudo-weight and number of extrametrical number of syllables. Most observed combinations had a unique stre~ assignment, and the average (by observed combination with no frequency normalization) has 1.5 solutions. In short, the constraints are extremely powerful; words like record with multiple stress patterns are the exception rather than the rule.</Paragraph>
  </Section>
class="xml-element"></Paper>