<?xml version="1.0" standalone="yes"?>
<Paper uid="J93-2003">
  <Title>The Mathematics of Statistical Machine Translation: Parameter Estimation</Title>
  <Section position="2" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
1. Introduction
</SectionTitle>
    <Paragraph position="0"> The growing availability of bilingual, machine-readable texts has stimulated interest in methods for extracting linguistically valuable information from such texts. For example, a number of recent papers deal with the problem of automatically obtaining pairs of aligned sentences from parallel corpora (Warwick and Russell 1990; Brown, Lai, and Mercer 1991; Gale and Church 1991b; Kay 1991). Brown et al. (1990) assert, and Brown, Lai, and Mercer (1991) and Gale and Church (1991b) both show, that it is possible to obtain such aligned pairs of sentences without inspecting the words that the sentences contain. Brown, Lai, and Mercer base their algorithm on the number of words that the sentences contain, while Gale and Church base a similar algorithm on the number of characters that the sentences contain. The lesson to be learned from these two efforts is that simple, statistical methods can be surprisingly successful in achieving linguistically interesting goals. Here, we address a natural extension of that work: matching up the words within pairs of aligned sentences.</Paragraph>
    <Paragraph position="1"> In recent papers, Brown et al. (1988, 1990) propose a statistical approach to machine translation from French to English. In the latter of these papers, they sketch an algorithm for estimating the probability that an English word will be translated into any particular French word and show that such probabilities, once estimated, can be used together with a statistical model of the translation process to align the words in an English sentence with the words in its French translation (see their Figure 3).</Paragraph>
    <Paragraph position="2"> * IBM T.J. Watson Research Center, Yorktown Heights, NY 10598 (~) 1993 Association for Computational Linguistics Computational Linguistics Volume 19, Number 2 Pairs of sentences with words aligned in this way offer a valuable resource for work in bilingual lexicography and machine translation.</Paragraph>
    <Paragraph position="3"> Section 2 is a synopsis of our statistical approach to machine translation. Following this synopsis, we develop some terminology and notation for describing the word-by-word alignment of pairs of sentences. In Section 4 we describe our series of models of the translation process and give an informal discussion of the algorithms by which we estimate their parameters from data. We have written Section 4 with two aims in mind: first, to provide the interested reader with sufficient detail to reproduce our results, and second, to hold the mathematics at the level of college calculus. A few more difficult parts of the discussion have been postponed to the Appendix.</Paragraph>
    <Paragraph position="4"> In Section 5, we present results obtained by estimating the parameters for these models from a large collection of aligned pairs of sentences from the Canadian Hansard data (Brown, Lai, and Mercer 1991). For a number of English words, we show translation probabilities that give convincing evidence of the power of statistical methods to extract linguistically interesting correlations from large corpora. We also show automatically derived word-by-word alignments for several sentences.</Paragraph>
    <Paragraph position="5"> In Section 6, we discuss some shortcomings of our models and propose modifications to address some of them. In the final section, we discuss the significance of our work and the possibility of extending it to other pairs of languages.</Paragraph>
    <Paragraph position="6"> Finally, we include two appendices: one to summarize notation and one to collect the formulae for the various models that we describe and to fill an occasional gap in our development.</Paragraph>
  </Section>
class="xml-element"></Paper>