<?xml version="1.0" standalone="yes"?>
<Paper uid="P81-1004">
  <Title>PERFORMANCE COMPARISON OF COMPONENT ALGORITHMS FOR THE PHONEMICIZATION OF ORTHOGRAPHY</Title>
  <Section position="3" start_page="19" end_page="137" type="metho">
    <SectionTitle>
STUDY ONE
</SectionTitle>
    <Paragraph position="0"> Study One reports a comparison of two routines for translating orthographic letters into segmental phonemes: Hunnicutt@TSI and NRL@DEC.</Paragraph>
    <Paragraph position="1"> Hunnicutt@TSI is the affix stripper and letter to sound rules as dlacribed in AJCL Microfiche 57, and implemented in MACRO-11 in Telesensory Systems' TTS-X prototype text-to-speech system. Hunnlcutt's system was modified only slightly in translation, and about 20 rules were added. The system starts from the right end of the word and identifies as many suffixes as it can from a table of about 140 suffixes, proceeding toward the beginning of the word until either the remainder (pseudo-root) of the word has no vowel or fewer than three letters, or no more suffixes can be matched. Next, a similar proceedure works from the beginning of the word, matching as many prefixes as it can from * a table of about 40 prefixes. Finally, the pseudo-root of the word is scanned left to right twice, once translating the consonants, and next translating the vowels.</Paragraph>
    <Paragraph position="2"> NRL@DEC is a system implemented by Martin Minnow at Digital Equtptment Corp. The whole system is somewhat more elaborate that the original NRL system, but the letter to sound module and its mode of operation are basically as described by Elovitz et alla, with 20 or 30 rules added. The NRL rules include about 60 very common whole words, as well as about 25 rules that handle various environments for three prefixes and fifteen suffixes.</Paragraph>
    <Paragraph position="3"> A set of 865 words was processed both by the Hunnlcutt@TSI affix stripper and letter to sound rules, and by the NRL@DEC letter to sound rules including the affix rules and the word fragments. The 865 words comprised approximately every fiftieth word of the Brown Corpus (Kucera &amp; Francis, 1967) in frequency order, starting from about the 400th most frequent word: &amp;quot;position.&amp;quot; The lexicon of the TSI system was disabled, and none of the whole words in the NRL rules was in the set of 865. Since the output from both subsystems was tapped before stress assignment, vowel reduction, and any allophonics were performed, the criterion of correctness was &amp;quot;does this phonemlcization represent any acceptable pronunciation of the spelled word, assuming one can assign stress correctly and then reduce vowels ~ppropriately.&amp;quot; Thus, a phonemlcization consistent wlth any possible word class for that spelling, or any 'regular' regional pronunciation was to be accepted.</Paragraph>
    <Paragraph position="4"> Three judges (two phonetlcians and a phonologist) were given printed copies of the two resulting phonemic transcriptions; both were in fairly transparent broad phonemic form. The judges chose among three possible responses to each word: 1 = correct; .5 = close or questionable; and 0 = wrong. Cross judge consistency can be seen from the bimodal distribution of summed scores in Figure I.</Paragraph>
    <Paragraph position="6"> Another, more diagnostic way to view the results is to present the number of words that fall into each cell of a 2X2 grid formed by the Hunnlcutt@TSl rating vs. the NRL@DEC rating, as shown in Figure 2. Figure 2 omits the 26 words that had a summed score of 1.5 for either of the two letter to sound systems.</Paragraph>
    <Paragraph position="7">  If the rule sets were equivalent, the grid would have zeroes in cells b and c. If one rule set were a super-set of the other, you would get a zero in cell b or cell c, but not both. Most of the 553 words in cell d are regular, or else are common exceptions (like &amp;quot;built&amp;quot;). Most of the 127 words in cell a are obviously exceptional (e.g. &amp;quot;minute, honor, one, two&amp;quot;).</Paragraph>
    <Paragraph position="8"> Examination of the 159 words distributed between cells b and c yields the payoff. Of the 69 words that Hunnicutt@TSI got right and NRL@DEC missed, nearly half are correct by virtue of the extensive affix stripping in Hunnicutt's algorithm. Among these 69 words in cell c are &amp;quot;mobile, naval, wallace, likened, coworkers, &amp; reenacted.&amp;quot; Of the 90 words that NRL@DEC got right and Hunnicutt@TSl got wrong, only about 15 are definitely due to NRL's word fragment rules.</Paragraph>
    <Paragraph position="9"> Six of the 90 words are in cell d just because NRL does not Strip suffixes the way that Hunnicutt's rules do. These six words are &amp;quot;november, visited, preferably, presidency, september, &amp; oven.&amp;quot; In general, both algorithms get about 25% wrong on this lexically flat sample of 865 word types. About 15~ of the words are incorectly phonemicized by both subsystems.</Paragraph>
    <Paragraph position="10"> This might suggest that 15~ wrong may be a state of the art performance level for segmental phonemicization of word types by sets of 400 rules.</Paragraph>
  </Section>
  <Section position="4" start_page="137" end_page="137" type="metho">
    <SectionTitle>
STUDY TWO
</SectionTitle>
    <Paragraph position="0"> Study Two compared the performance of two algorithms for assignment of lexical stress to words. Both of the algorithms were coded in MACRO-It and ran in different versions of TSl's TTS-X prototype text-to-speech system. The first algorithm is Hunnlcutt's lexical stresser, which is described in detail in AjCL Microfiche 57.</Paragraph>
    <Paragraph position="1"> Hunnicutt's algorithm is an adaptation of Halle's cyclic stress rules for English. The adaptations include adjustments for the less specified input to the rules (e.g. the part of speech of the root is unknown), and the number of stress levels specified in the output is reduced, presumably because the Klatt synthesizer it was designed to drive only used two stress levels. Hunnicutt also added stress rules that depended on the occurance of certain classes of suffixes. Hunnicutt's rules require several pointers and a suffix table, they sometimes pass through a word several times in the manner of Chomsky &amp; Halle's (1968) rules, and they occupy about 3K bytes of executable code in their TSI version.</Paragraph>
    <Paragraph position="2"> The second algorithm is a simplified version of a stress rule proposed in Hill &amp; Nessly (1973). We will refer to this rule as Nessly's default, since it is the default case of Nessly's full stress algorithm. Nessly's default stress is quite similar to Latin stress and to the &amp;quot;first approximation&amp;quot; stress rule discussed twoard the beginning of Chomsky &amp; Halle's chapter three (1968, pp.69-77). The main differences between Nessly's default rule and Chomsky &amp; Halle's &amp;quot;first approximation&amp;quot; are: (I) No word class information is used in Nessly's default, so verbs are stressed as nouns.</Paragraph>
    <Paragraph position="3"> and (2) What constitutes a &amp;quot;strong cluster&amp;quot; (which contains a tense vowel or a closed syllable end) is different. Nessly's default is indifferent to vowel length or tensity.</Paragraph>
    <Paragraph position="4"> Nessly's default rule can be outlined as follows: If(number of syllables : I) stress it.</Paragraph>
    <Paragraph position="5"> if(number of syllables : 2) stress left syllable.</Paragraph>
    <Paragraph position="6"> else skip the last syllable.</Paragraph>
    <Paragraph position="7"> * if(next-to-last is closed) stress it.</Paragraph>
    <Paragraph position="8"> else stress third from last.</Paragraph>
    <Paragraph position="9"> (place alternating 2nd stresses on syllables to the left.) The MACRO-It version of this rule requires about 150 bytes of executable code, and accepts one pointer to the last vowel in the word. It passes through the word once, right to left, and it does very well assigning correct stresses (in caps) to &amp;quot;LUminant&amp;quot; vs. &amp;quot;maLIGnant,&amp;quot; for example.</Paragraph>
    <Paragraph position="10"> For testing the stress algorithms, a sample of 430 words was selected. These 430 words were all the items of five or more characters that had frequencies of 40 ppm through 34 ppm (inclusive) in the Brown corpus. The segmental phonemicization was done by Hunnicutt's rules in TSI's TTS-X prototype. The automatically produced segmental phonemicizations that the stress algorithms operated on were rejected only if they did not have the correct number of syllables.</Paragraph>
    <Paragraph position="11"> Thirteen of the 430 words were phonemicized with the wrong number of syllables. Another 54, or 13~, of the 430 were one syllable words, which were allways assigned correct  stress. Stress assignments were judged by the first author. The results on the remaining 417 words of the sample were: Correct Wrong Hunnicutt/Halle 308 109 Nessly default 303 114 So, on these words, the two algorithms perform at about the same level of accuracy, which is about 252 wrong on a lexlcal sample.</Paragraph>
  </Section>
class="xml-element"></Paper>