<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-0301">
  <Title>al: A word alignment system with limited language</Title>
  <Section position="3" start_page="1" end_page="1" type="intro">
    <SectionTitle>
3 Evaluation Measures
</SectionTitle>
    <Paragraph position="0"> Evaluations were performed with respect to four different measures. Three of them - precision, recall, and F-measure - represent traditional measures in Information Retrieval, and were also frequently used in previous word alignment literature. The fourth measure was originally introduced by (Och and Ney, 2000), and proposes the notion of quality of word alignment.</Paragraph>
    <Paragraph position="1"> Given an alignment BT, and a gold standard alignment BZ, each such alignment set eventually consisting of two  Each word alignment submission was evaluated in terms of the above measures. Moreover, we conducted two sets of evaluations for each submission: AF NULL-Align, where each word was enforced to belong to at least one alignment; if a word did not belong to any alignment, a NULL Probable alignment was assigned by default. This set of evaluations pertains to full coverage word alignments.</Paragraph>
    <Paragraph position="2"> AF NO-NULL-Align, where all NULL alignments were removed from both submission file and gold stan- null We conducted therefore 14 evaluations for each submission file: AER, Sure/Probable Precision, Sure/Probable Recall, and Sure/Probable F-measure, with a different figure determined for NULL-Align and NO-NULL-Align alignments.</Paragraph>
  </Section>
class="xml-element"></Paper>