<?xml version="1.0" standalone="yes"?>
<Paper uid="P01-1030">
  <Title>Fast Decoding and Optimal Decoding for Machine Translation</Title>
  <Section position="8" start_page="0" end_page="0" type="concl">
    <SectionTitle>
7 Experiments and Discussion
</SectionTitle>
    <Paragraph position="0"> In our experiments we used a test collection of 505 sentences, uniformly distributed across the lengths 6, 8, 10, 15, and 20. We evaluated all decoders with respect to (1) speed, (2) search optimality, and (3) translation accuracy. The last two factors may not always coincide, as Model 4 is an imperfect model of the translation process--i.e., there is no guarantee that a numerically optimal decoding is actually a good translation.</Paragraph>
    <Paragraph position="1"> Suppose a decoder outputs a6 a8 , while the optimal decoding turns out to be a6 . Then we consider six possible outcomes:  ther is a perfect translation.</Paragraph>
    <Paragraph position="2"> Here, &amp;quot;perfect&amp;quot; refers to a human-judged translation that transmits all of the meaning of the source sentence using flawless target-language syntax.</Paragraph>
    <Paragraph position="3"> We have found it very useful to have several decoders on hand. It is only through IP decoder output, for example, that we can know the stack decoder is returning optimal solutions for so many sentences (see Table 1). The IP and stack decoders enabled us to quickly locate bugs in the greedy decoder, and to implement extensions to the basic greedy search that can find better solutions. (We came up with the greedy operations discussed in Section 5 by carefully analyzing error logs of the kind shown in Table 1). The results in Table 1 also enable us to prioritize the items on our research agenda. Since the majority of the translation errors can be attributed to the language and translation models we use (see column PME in Table 1), it is clear that significant improvement in translation quality will come from better sent decoder time search translation length type (sec/sent) errors errors (semantic NE PME DSE FSE HSE CE  trigram language model. Greedya191 and greedya13 are greedy decoders optimized for speed.</Paragraph>
    <Paragraph position="4"> models.</Paragraph>
    <Paragraph position="5"> The results in Table 2, obtained with decoders that use a trigram language model, show that our greedy decoding algorithm is a viable alternative to the traditional stack decoding algorithm. Even when the greedy decoder uses an optimized-forspeed set of operations in which at most one word is translated, moved, or inserted at a time and at most 3-word-long segments are swapped--which is labeled &amp;quot;greedya191 &amp;quot; in Table 2--the translation accuracy is affected only slightly. In contrast, the translation speed increases with at least one order of magnitude. Depending on the application of interest, one may choose to use a slow decoder that provides optimal results or a fast, greedy decoder that provides non-optimal, but acceptable results. One may also run the greedy decoder using a time threshold, as any instance of anytime algorithm. When the threshold is set to one second per sentence (the greedya13 label in Table 1), the performance is affected only slightly.</Paragraph>
    <Paragraph position="6"> Acknowledgments. This work was supported by DARPA-ITO grant N66001-00-1-9814.</Paragraph>
  </Section>
class="xml-element"></Paper>