<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1079">
  <Title>Extending the BLEU MT Evaluation Method with Frequency Weightings</Title>
  <Section position="3" start_page="0" end_page="0" type="concl">
    <SectionTitle>
5. Conclusion and future work
</SectionTitle>
    <Paragraph position="0"> The results for weighted N-gram models have a significantly higher correlation with human intuitive judgements about translation Adequacy and Fluency than the baseline N-gram evaluation measures which are used in the BLEU MT evaluation toolkit. This shows that they are a promising direction of research. Future work will apply our approach to evaluating MT into languages other than English, extending the experiment to a larger number of MT systems built on different architectures and to larger corpora.</Paragraph>
    <Paragraph position="1"> However, the results of the experiment may also have implications for MT development: significance weights may be used to rank the relative &amp;quot;importance&amp;quot; of translation equivalents. At present all MT architectures (knowledge-based, example-based, and statistical) treat all translation equivalents equally, so MT systems cannot dynamically prioritise rule applications, and translations of the central concepts in texts are often lost among excessively literal translations of less important concepts and function words.</Paragraph>
    <Paragraph position="2"> For example, for statistical MT significance weights of lexical items may indicate which words have to be introduced into the target text using the translation model for source and target languages, and which need to be brought there by the language model for the target corpora. Similar ideas may be useful for the Example-based and Rule-based MT architectures. The general idea is that different pieces of information expressed in the source text are not equally important for translation: MT systems that have no means for prioritising this information often introduce excessive information noise into the target text by literally translating structural information, etymology of proper names, collocations that are unacceptable in the target language, etc. This information noise often obscures important translation equivalents and prevents the users from focusing on the relevant bits. MT quality may benefit from filtering out this excessive information as much as from frequently recommended extension of knowledge sources for MT systems. The significance weights may schedule the priority for retrieving translation equivalents and motivate application of compensation strategies in translation, e.g., adding or deleting implicitly inferable information in the target text, using non-literal strategies, such as transposition or modulation (Vinay and Darbelnet, 1995). Such weights may allow MT systems to make an approximate distinction between salient words which require proper translation equivalents and structural material both in the source and in the target texts. Exploring applicability of this idea to various MT architectures is another direction for future research.</Paragraph>
  </Section>
class="xml-element"></Paper>