<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0809">
  <Title>Word Alignment for Languages with Scarce Resources</Title>
  <Section position="3" start_page="66" end_page="67" type="intro">
    <SectionTitle>
3 Evaluation Measures
</SectionTitle>
    <Paragraph position="0"> Evaluations were performed with respect to four different measures. Three of them - precision, recall, and F-measure - represent traditional measures in Information Retrieval, and were also frequently used in previous word alignment literature. The fourth measure was originally introduced by (Och and Ney, 2000), and proposes the notion of quality of word alignment.</Paragraph>
    <Paragraph position="1"> Given an alignment a0 , and a gold standard alignment a1 , each such alignment set eventually consisting of two sets a0a3a2 , a0a5a4 , and a1a6a2 , a1a6a4 corresponding to Sure and Probable alignments, the following measures are defined (where a7 is the alignment type, and can be set to either S or P).</Paragraph>
    <Paragraph position="3"> Each word alignment submission was evaluated in terms of the above measures. Given numerous (constructive) debates held during the previous word alignment evaluation, which questioned the informativeness of the NULL alignment evaluations, we decided</Paragraph>
  </Section>
class="xml-element"></Paper>