<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0809">
  <Title>Word Alignment for Languages with Scarce Resources</Title>
  <Section position="4" start_page="67" end_page="68" type="metho">
    <SectionTitle>
4 Participating Systems
</SectionTitle>
    <Paragraph position="0"> Ten teams from around the world participated in the word alignment shared task. Table 1 lists the names of the participating systems, the corresponding institutions, and references to papers in this volume that provide detailed descriptions of the systems and additional analysis of their results.</Paragraph>
    <Paragraph position="1"> Seven teams participated in the Romanian-English subtask, four teams participated in the English-Inuktitut subtask, and two teams participated in the English-Hindi subtask. There were no restrictions placed on the number of submissions each team could make. This resulted in a total of 50 submissions from the ten teams, where 37 sets of results were submitted for the Romanian-English subtask, 10 for the English-Inuktitut subtask, and 3 for the English-Hindi subtask. Of the 50 total submissions, there were 45 in the Limited resources subtask, and 5 in the Unlimited resources subtask. Tables 2, 4 and 6 show all of the submissions for each team in the three subtasks, and provide a brief description of their approaches.</Paragraph>
    <Paragraph position="2"> Results for all participating systems, including precision, recall, F-measure, and alignment error rate are listed in Tables 3, 5 and 7. Ranked results for all systems are plotted in Figures 2, 3 and 4. In the graphs, systems are ordered based on their AER scores. System names are preceded by a marker to indicate the system type: L stands for Limited Resources, and U stands for Unlimited Resources.</Paragraph>
    <Paragraph position="3"> While each participating system was unique, there were a few unifying themes. Several teams had approaches that relied (to varying degrees) on an IBM model of statistical machine translation (Brown et al., 1993), with different improvements brought by different teams, consisting of new submodels, improvements in the HMM model, model combination for optimal alignment, etc. Se-veral teams used symmetrization metrics, as introduced in (Och and Ney, 2003) (union, intersection, refined), most of the times applied on the alignments produced for the two directions source-target and target-source, but also as a way to combine different word alignment systems.</Paragraph>
    <Paragraph position="4"> Significant improvements with respect to baseline word alignment systems were observed when the vocabulary was reduced using simple stemming techniques, which seems to be a particularly effective technique given the data sparseness problems associated with the relatively small amounts of training data. In the unlimited resources subtask, systems made use of bilingual dictionaries, human-contributed word alignments, or syntactic constraints derived from a dependency parse tree applied on the English side of the corpus.</Paragraph>
    <Paragraph position="5"> When only small amounts of parallel corpora were available (i.e. the English-Hindi subtask), the use of additional resources resulted in absolute improvements of up to 20% as compared to the case when the word alignment systems were based exclusively on the parallel texts. Interestingly, this was not the case for the language pairs that had larger training corpora (i.e. Romanian-English, English-Inuktitut), where the limited resources systems seemed to lead to comparable or sometime even better results than those that relied on unlimited resources. This suggests  that the use of additional resources does not seem to contribute to improvements in word alignment quality when enough parallel corpora are available, but they can make a big difference when only small amounts of parallel texts are available.</Paragraph>
    <Paragraph position="6"> Finally, in a comparison across language pairs, the best results are obtained in the English-Inuktitut task, followed by Romanian-English, and by English-Hindi, which corresponds to the ordering of the sizes of the training data sets. This is not surprising since, like many other NLP tasks, word alignment seems to highly benefit from large amounts of training data, and thus better results are obtained when larger training data sets are available.</Paragraph>
  </Section>
class="xml-element"></Paper>