<?xml version="1.0" standalone="yes"?>
<Paper uid="C00-1061">
  <Title>English-to-Korean Transliteration using Multiple Unbounded Overlapping Phoneme Chunks</Title>
  <Section position="6" start_page="421" end_page="423" type="evalu">
    <SectionTitle>
5 Experiments
</SectionTitle>
    <Paragraph position="0"> Experiments were done in two points of view: the accuracy test and the variation coverage test.</Paragraph>
    <Section position="1" start_page="421" end_page="421" type="sub_section">
      <SectionTitle>
5.1 Test Sets
</SectionTitle>
      <Paragraph position="0"> We use two data sets for an accuracy test. Test Set I is consists of 1.,650 English and Korean word pairs that aligned in a phoneme unit. It was made by (Lee and Choi, 1998) and tested by many methods. To compare our method with other methods, we use this data set. We use same training data (1,500 words) and test data (150 words). Test Set II is consists of 7,185 English and Korean word paii's. We use Test Set H to show the relation between the size of training data and the accuracy. We use 90% of total size as training data and 10% as test data. For a variation coverage test, we use Test Set III that is extracted from KTSET 2.0. Test Set HI is consists of 2,391 English words and their transliterations. An English word has 1.14 various transliterations in average.</Paragraph>
    </Section>
    <Section position="2" start_page="421" end_page="421" type="sub_section">
      <SectionTitle>
5.2 Evaluation functions
</SectionTitle>
      <Paragraph position="0"> Accuracy was measured by the percentage of the number of correct transliterations divided by the number of generated transliterations. We (:all it as word accuracy(W.A.). We use one more measure, called character accuracy(C.A.) that measures the character edit distance between a correct word and a generated word.</Paragraph>
      <Paragraph position="1"> no. of correct words</Paragraph>
      <Paragraph position="3"> where L is the length of the original string, and i, d, mid s are the number of insertion, deletion and substitution respectively. If the dividend is negative (when L &lt; (i + d + s)), we consider it as zero(Hall and Dowling, 1980).</Paragraph>
      <Paragraph position="4"> For the real usage test, we used variation coverage (V.C.) that considers various usages. We evaluated both tbr the term frequency (tf) and document frequency (d J), where tfis the number of term appearance in the documents and df is the number of documents that contain the term.</Paragraph>
      <Paragraph position="5"> If we set the usage tf (or d./) of the transliterations to 1 tbr each transliteration, we can calculate the transliteration coverage tbr the unique word types, single .frequency(.sf).</Paragraph>
      <Paragraph position="6"> V.C. = {if, df, s f} of found words (7) {t.f, 4f, &lt;f} of ,sed o,'ds</Paragraph>
    </Section>
    <Section position="3" start_page="421" end_page="421" type="sub_section">
      <SectionTitle>
5.3 Accuracy tests
</SectionTitle>
      <Paragraph position="0"> We compare our result \[PCa, PUp\] a with the simple statistical intbrmation based model(Lee and Choi, 1998) \[ST\], the Maxinmm Entropy based model(Tae-il Kim, 2000) \[MEM\], the Neural Network model(Jung-Jae Kim, 1999) INN\] and the Decision %'ee based model(Kang, 1999)\[DT\]. Table 3 shows the result of E-K transliteration and back-transliteration test with Test ,get L  Fig. 7, 8 show the results of our proposed method with the size of training data, Test Set II. We compare our result with the decision tree based method.</Paragraph>
    </Section>
    <Section position="4" start_page="421" end_page="422" type="sub_section">
      <SectionTitle>
Set H
</SectionTitle>
      <Paragraph position="0"> aPC stands for phoneme chunks based method and a and b stands for aligned by an alphabet unit and a</Paragraph>
    </Section>
    <Section position="5" start_page="422" end_page="422" type="sub_section">
      <SectionTitle>
5.4 Variation coverage tests
</SectionTitle>
      <Paragraph position="0"> To (:oml)~re our result(PCp) with (Lee and ()hoi, 1998), we tr~fincd our lnethods with the training data of Test Set L In ST, (Lee mid Choi, 1998) use 20 high rank results, but we j tlst llSe 5 results. TM)le 5 shows the (:overage  Fig. 9 shows the increase of (:overage with the number of outputs.</Paragraph>
    </Section>
    <Section position="6" start_page="422" end_page="423" type="sub_section">
      <SectionTitle>
5.5 Discussion
</SectionTitle>
      <Paragraph position="0"> We summarize the, information length ~md the kind of infonnation(Tnble 6). The results of experimenLs and information usage show theft MEM combines w~rious informal;ion better than DT and NN. ST does not list &amp; previous inlmt (el-l) but use ~ previous output(t,:i_~) to calculate the current outlml?s probability like  the lowest aecm'acy. It means that surrmmding alphal)ei;s give more informed;ion than t)revious outlmL. In other words, E-K trmlslii;e.ration is not the all)h~bet-per-alphabet or phonenle-pert)honeme (:lassific~tion problem. A previous outI)ut does not give, enough information for cllrrent ltnit's dismnbiguat;ion. An input mill mid an OUtlmt unit shouht be exl:ende(t. E-K transliteration is a (:hunk-l)er-chunk classification prot)lenL We restri(:t the length of infiwm~tion, to see the influence of' phoneme-chunk size. Pig. 10 shows the results.</Paragraph>
      <Paragraph position="1">  With the same length of information, we get the higher C.A. and W.A. than other methods. It means previous outputs give good information and our chunk-based nmthod is a good combining method. It also suggests that we can restrict the max size of chunk in a permissible size.</Paragraph>
      <Paragraph position="2"> PCa gets a higher accuracy than PCp. It is clue to the number of possible phoneme sequences. A transliteration network that consists of phoneme nnit has more nodes than a transliteration network that consists of alphabet unit. With small training data, despite of the loss due to the phoneme sequences ambiguity a phoneme gives more intbrmation than an alphabet. When the infbrmation is enough, PCa outpertbrms Pep.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>