<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0818">
  <Title>LIHLA: Shared task system description</Title>
  <Section position="7" start_page="111" end_page="113" type="evalu">
    <SectionTitle>
3 Experiments
</SectionTitle>
    <Paragraph position="0"> In this section we present the experiments carried out with LIHLA for the &amp;quot;Shared task on word alignment&amp;quot; in the Workshop on Building and Using Parallel Texts during ACL2005. Systems participating in this shared task were provided with training data (consisting of sentence-aligned parallel texts) for three pairs of languages: English-Inuktitut,  Romanian-English and English-Hindi. Furthermore, the systems would choose to participate in one or both subtasks of &amp;quot;limited resources&amp;quot; (where systems were allowed to use only the resources provided) and &amp;quot;unlimited resources&amp;quot; (where systems were allowed to use any resources in addition to those provided). The system described in this paper, LIHLA, participated in the subtask of limited resources aligning English-Inuktitut and Romanian-English test sets.</Paragraph>
    <Paragraph position="1"> The training sets --composed of 338,343 English-Inuktitut aligned sentences (omission cases were excluded from the whole set of 340,526 pairs) and 48,478 Romanian-English aligned ones-- were used to build the bilingual lexicons. Then, without changing any default parameter (threshold for LCSR, maximum number of iterations, etc.), LIHLA aligned the 75 English-Inuktitut and the 203 Romanian-English parallel sentences on test sets.</Paragraph>
    <Paragraph position="2"> The whole alignment process (bilingual lexicon generation and alignment itself) did not take more than 17 minutes for English-Inuktitut (3 iterations per sentence, on average) and 7 minutes for Romanian-English (4 iterations per sentence, on average). The evaluation was run with respect to precision, recall, F-measure, and alignment error rate (AER) considering sure and probable alignments but not NULL ones (Mihalcea and Pedersen, 2003). Tables  1 and 2 present metric values for English-Inuktitut and Romanian-English alignments, respectively, as provided by the organization of the shared task.</Paragraph>
    <Paragraph position="3">  The results obtained in these experiments were not so good as those achieved by LIHLA on the language pairs for which it was developed, that is, 92.48% of precision and 88.32% of recall on Portuguese-Spanish parallel texts and 84.35% of precision and 76.39% of recall on Portuguese-English ones.3 The poor performance in the English-Inuktikut task may be partly due to the fact that Inuktikut is a polysynthetic language, that is, one in which, unlike in English, words are formed by long strings of concatenated morphemes. This makes it difficult for NATools to build reasonable dictionaries and lead to a predominance of n : 1 alignments, which are harder to determine --this fact can be confirmed by the better precision of LIHLA when probable alignments were considered (see table 1). The performance in the English-Romanian task, not very far from the English-Portuguese task used to tune up the parameters of the algorithm, is harder to explain without further analysis.</Paragraph>
    <Paragraph position="4"> The difference in precision and recall between the two language pairs is due to the fact that on the English-Inuktitut reference corpus in addition to sure alignments the probable ones were also annotated while in Romanian-English only sure alignments are found. This indicates that evaluating alignment systems is not a simple task since their performance depends not only on the language pairs and the quality of parallel corpora (constant criteria in this shared task) but also the way the reference corpus is built.</Paragraph>
    <Paragraph position="5"> So, at this moment, it would be unfair to blame the worse performance of LIHLA on its alignment methodology since it has been applied to the new language pairs without changing any of its default parameters. Maybe a simple optimization of parameters for each pair of languages could bring better results and also the impact of size and quality of training and reference corpora used in these experiments should be investigated. Then, the only conclusion that can be taken at this moment is that LIHLA, with its heuristics and/or default parameters, can not be indistinctly applied to any pair of languages.</Paragraph>
    <Paragraph position="6"> Despite of its performance, LIHLA has some  advantages when compared to other lexical alignment methods found in the literature, such as: it does not need to be trained for a new pair of languages (as in Och and Ney (2000)) and neither does it require pre-processing steps to handle texts (as in G'omez Guinovart and Sacau Fontenla (2004)).</Paragraph>
    <Paragraph position="7"> Furthermore, the whole alignment process (bilingual lexical generation and alignment itself) has proved to be very fast as mentioned previously.</Paragraph>
  </Section>
class="xml-element"></Paper>