<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1045">
  <Title>Improving Word Alignment Quality using Morpho-syntactic Information</Title>
  <Section position="7" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
6 Experimental Results
</SectionTitle>
    <Paragraph position="0"> We performed our experiments on the Verbmobil corpus. The Verbmobil task (W. Wahlster, editor, 2000) is a speech translation task in the domain of appointment scheduling, travel planning and hotel reservation. The corpus statistics is shown in Table 1. The number of sure and possible alignments in the manual reference is given as well. We also used a small training corpus consisting of only 500 sentences randomly chosen from the main corpus.</Paragraph>
    <Paragraph position="1"> We carried out the training scheme 14H5334365 using the toolkit GIZA++.</Paragraph>
    <Paragraph position="2"> The scheme is defined according to the number of iterations for each model. For example, 43 means three iterations of the model IBM-4. We trained the IBM-1 and HMM model using hierarchical lexicon counts, and the parameters of the other models were also indirectly improved thanks to the refined parameters of the initial models.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
6.1 Evaluation Method
</SectionTitle>
      <Paragraph position="0"> We use the evaluation criterion described in (Och and Ney, 2000). The obtained word alignment is compared to a reference alignment produced by human experts. The annotation scheme explicitly takes into account the ambiguity of the word alignment. The unambiguous alignmentsare annotated as surealignments(S) and the ambiguous ones as possible alignments (P). The set of possible alignments P is used especially for idiomatic expressions, free translations and missing function words. The set S is subset of the set P (S P).</Paragraph>
      <Paragraph position="1"> The quality of an alignment A is computed as appropriately redefined precision and recall measures. Additionally, we use the alignment error rate (AER) which is derived from the well-known F-measure.</Paragraph>
      <Paragraph position="3"> Thus, a recall error can only occur if a S(ure) alignment is not found and a precision error can only occur if a found alignment is not even P(ossible).</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
6.2 Alignment Quality Results
</SectionTitle>
      <Paragraph position="0"> Table 2 shows the alignment quality for the two corpus sizes of the Verbmobil task. Results are presented for the Viterbi alignments from both translation directions (German!English and English!German) as well as for combination of those two alignments.</Paragraph>
      <Paragraph position="1"> The table shows the baseline AER for different training schemes and the corresponding AER when the hierarchical counts are used. We see that there is a consistent decrease in AER for all training schemes, especially for the small training corpus. It can be also seen that greater improvements are yielded for the simpler models. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>