<?xml version="1.0" standalone="yes"?>
<Paper uid="P05-3025">
  <Title>Interactively Exploring a Machine Translation Model</Title>
  <Section position="4" start_page="97" end_page="97" type="metho">
    <SectionTitle>
2 Translation Framework
</SectionTitle>
    <Paragraph position="0"> It is useful at this point to give a brief description of the syntax-based framework that we work with, which is based on translating Chinese sentences into English syntax trees. Galley et al. (2004) describe how to learn hundreds of millions of treetransformation rules from a parsed, aligned Chinese/English corpus, and Galley et al. (submitted) describe probability estimators for those rules. We decode a new Chinese sentence with a method similar to parsing, where we apply learned rules to build up a complete English tree hypothesis from the Chinese string.</Paragraph>
    <Paragraph position="1"> The rule extractor learns rules for many situations.</Paragraph>
    <Paragraph position="2"> Some are simple phrase-to-phrase rules such as:</Paragraph>
    <Paragraph position="4"> This rule should be read as follows: replace the Chinese word 0 with the noun phrase &amp;quot;the police&amp;quot;.</Paragraph>
    <Paragraph position="5"> Others rules can take existing tree fragments and build upon them. For example, the rule</Paragraph>
    <Paragraph position="7"> takes three parts of a sentence, a noun phrase (x0), a verb phrase (x1), and a period (x2) and ties them together to build a complete sentence. Rules also can involve phrase re-ordering, as in</Paragraph>
    <Paragraph position="9"> This rule builds an English noun phrase out of an adjective (x0) and a noun (x1), but in the Chinese, the order is reversed. Multilevel rules can tie several of these concepts together; the rule</Paragraph>
    <Paragraph position="11"> takes a Chinese word and two English constituents -- x1, a noun phrase, and x0, a past-participle verb -- and translates them into a phrase of the form &amp;quot;was [verb] by [noun-phrase]&amp;quot;. Notice that the order of the constituents has been reversed in the resulting English phrase, and that English function words have been generated.</Paragraph>
    <Paragraph position="12"> The decoder builds up a translation from the Chinese sentence into an English tree by applying these rules. It follows the decoding-as-parsing idea exemplified by Wu (1996) and Yamada and Knight (2002). For example, the Chinese verb phrase 0 (literally, &amp;quot;[passive] police kill&amp;quot;) can be translated to English via four rules (see</Paragraph>
  </Section>
  <Section position="5" start_page="97" end_page="99" type="metho">
    <SectionTitle>
3 DerivTool
</SectionTitle>
    <Paragraph position="0"> In order to test whether good translations can be generated with rules learned by Galley et al. (2004), we created DerivTool as an environment for interactively using these rules as a decoder would. A user starts with a Chinese sentence and applies rules one after another, building up a translation from Chinese to English. After finishing the translation, the user can save the trace of rule-applications (the derivation tree) for later analysis.</Paragraph>
    <Paragraph position="1"> We now outline the typical procedure for a user to translate a sentence with DerivTool. To start, the user loads a set of sentences to translate and chooses a particular one to work with. The tool then presents the user with a window split halfway up. The top  half is the workspace where the user builds a translation. It initially displays only the Chinese sentence, with each word as a separate node. The bottom half presents a set of tabbed panels which allow the user to select rules to build up the translation. See Figure 1 for a picture of the interface showing a completed derivation tree.</Paragraph>
    <Paragraph position="2"> The most immediately useful panel is called Selecting Template, which shows a grid of possible English phrasal translations for Chinese phrases from the sentence. This phrase grid contains both phrases learned in our extracted rules (e.g., &amp;quot;the police&amp;quot; from earlier) and phrases learned by the phrase-based translation system (Och and Ney, 2004)1. The user presses a grid button to choose a phrase to include in the translation. At this point, a frequency1The phrase-based system serves as a sparring partner. We display its best decoding in the center of the screen. Note that in Figure 1 its output lacks an auxiliary verb and an article. ordered list of rules will appear; these rules translate the Chinese phrase into the button-selected English phrase, and the user specifies which one to use.</Paragraph>
    <Paragraph position="3"> Often there will be more than one rule (e.g., may translate via the rule VBD(killed) - or VBN(killed) - ), and sometimes there are no rules available. When there are no rules, the buttons are marked in red, telling us that the phrase-based system has access to this phrasal translation but our learned syntactic rules did not capture it. Other buttons are marked green to represent translations from the specialized number/name/date system, and others are blue, indicating the phrases in the phrase-based decoder's best output. A purple button indicates both red and blue, i.e., the phrase was chosen by the phrase-based decoder but is unavailable in our syntactic framework. This is a bad combination, showing us where rule learning is weak. The  remaining buttons are gray.</Paragraph>
    <Paragraph position="4"> Once the user has chosen the phrasal rules required for translating the sentence, the next step is to stitch these phrases together into a complete English syntax tree using more general rules. These are found in another panel called Searching. This panel allows a user to select a set of adjacent, top-level nodes in the tree and find a rule that will connect them together. It is commonly used for building up larger constituents from smaller ones. For example, if one has a noun-phrase, a verb-phrase, and a period, the user can search for the rule that connects them and builds an &amp;quot;S&amp;quot; on top, completing the sentence. The results of a search are presented in a list, again ordered by frequency.</Paragraph>
    <Paragraph position="5"> A few more features to note are: 1) loading and saving your work at any point, 2) adding free-form notes to the document (e.g. &amp;quot;I couldn't find a rule that...&amp;quot;), and 3) manually typing rules if one cannot be found by the above methods. This allows us to see deficiencies in the framework.</Paragraph>
  </Section>
  <Section position="6" start_page="99" end_page="99" type="metho">
    <SectionTitle>
4 How DerivTool Helps
</SectionTitle>
    <Paragraph position="0"> First, DerivTool has given us confidence that our syntax-based framework can work, and that the rules we are learning are good. We have been able to manually build a good translation for each sentence we tried, both for short and long sentences. In fact, there are multiple good ways to translate sentences using these rules, because different DerivTool users translate sentences differently. Ordering rules by frequency and/or probability helps us determine if the rules we want are also frequent and favored by our model.</Paragraph>
    <Paragraph position="1"> DerivTool has also helped us to find problems with the framework and to see clearly how to fix them. For example, in one of our first sentences we realized that there was no rule for translating a date -- likewise for numbers, names, currency values, and times of day. Our phrase-based system solves these problems with a specialized date/name/number translator. Through the process of manually typing syntactic transformation rules for dates and numbers in DerivTool, it became clear that our current date/name/number translator did not provide enough information to create such syntactic rules automatically. This sparked a new area of research before we had a fully-functional decoder.</Paragraph>
    <Paragraph position="2"> We also found that multi-word noun phrases, such as &amp;quot;Israeli Prime Minister Sharon&amp;quot; and &amp;quot;the French Ambassador's visit&amp;quot; were often parsed in a way that did not allow us to learn good translation rules.</Paragraph>
    <Paragraph position="3"> The flat structure of the constituents in the syntax tree makes it difficult to learn rules that are general enough to be useful. Phrases with possessives also gave particular difficulty due to the awkward multilevel structure of the parser's output. We are researching solutions to these problems involving restructuring the syntax trees before training.</Paragraph>
    <Paragraph position="4"> Finally, our tool has helped us find bugs in our system. We found many cases where rules we wanted to use were unexpectedly absent. We eventually traced these bugs to our rule extraction system. Our decoder would have simply worked around this problem, producing less desirable translations, but DerivTool allowed us to quickly spot the missing rules.</Paragraph>
  </Section>
class="xml-element"></Paper>