<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2011">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics A High-Accurate Chinese-English NE Backward Translation System Combining Both Lexical Information and Web Statistics</Title>
  <Section position="6" start_page="86" end_page="87" type="evalu">
    <SectionTitle>
4 Experiments
</SectionTitle>
    <Paragraph position="0"> To evaluate the performance of our system, 15 common users are invited to provide 100 foreign NEs per user. These users are asked to simulate a scenario of using web search machine to perform cross-lingual information retrieval. The proportion of different types of NEs is roughly conformed to the real distribution, except for creation titles. We gathers a larger proportion of creation titles than other types of NEs, since the ways of translating creation titles is less regular and we may use them to test how much help could the web statistics provide.</Paragraph>
    <Paragraph position="1"> After removing duplicate entries provided by users, finally we obtain 1,119 nouns. Among them 7 are not NEs, 65 are originated from Oriental languages (Chinese, Japanese, and Korean), and the rest 1,047 foreign NEs are our main experimental subjects. Among these 1,047 names there are 455 personal names, 264 location names, 117 organization names, 196 creation titles, and 15 other types of NEs.</Paragraph>
    <Paragraph position="2"> Table 2 and Figure 5 show the performance of the system with different types of NEs. We could observe that the translating performance is best with location names. It is within our expectation, since location names are one of the most limited NE types. Human usually provide loca-tion names in a very limited range, and thus there are less location names having ambiguous translations and less rare location names in the test data. Besides, because most location names are purely transliterated, it can give us some clues about the performance of our phonetic model.</Paragraph>
    <Paragraph position="3"> Our system performs worst with creation titles.</Paragraph>
    <Paragraph position="4"> One reason is that the naming and translating style of creation titles are less formulated. Many titles are not translated by lexical information, but by semantic information or else. For example, &amp;quot;Mr. &amp; Mrs. Smith&amp;quot; is translated into &amp;quot;Shi Mi Si Ren Wu (Smiths' Mission)&amp;quot; by the content of the creation it denotes. Another reason is that many titles are not originated from English, such as &amp;quot;le Nozze di Figaro&amp;quot;. It results the C-E bilingual dictionary cannot be used in recognizing word sense similarity. A more serious problem with titles is that titles generally consist of more single words than other types of NEs. Therefore, in the returned snippets by Google, the correct translation is often cut off. It would results a great bias in estimating statistical scores.</Paragraph>
    <Paragraph position="5"> Table 3 compares the result of different feature combinations. It considers only foreign NEs in the test data. From the result we could conclude that both statistical and lexical features are helpful for translation finding, while the inverse search are the key of our system to achieve a good performance.</Paragraph>
    <Paragraph position="6">  From the result we could also find that our system has a high recall of 94.7% while considering top 4 candidates. If we only count in the given NEs with their correct translation appearing in the returned snippets, the recall would go to 96.8%. This achievement may be not yet good enough for computer-driven applications, but it is certainly a good performance for user querying.</Paragraph>
  </Section>
class="xml-element"></Paper>