<?xml version="1.0" standalone="yes"?>
<Paper uid="W01-1410">
  <Title>Machine Translation with Grammar Association: Some Improvements and the Loco C Model</Title>
  <Section position="7" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
5 Experimental results
</SectionTitle>
    <Paragraph position="0"> In a first series of experiments, we were interested in knowing whether or not our proposals actually improve Grammar Association state of the art. To this end, a simple artificial Machine Translation task was employed. The corpus consists of pairs of sentences describing two-dimensional scenes with circles, squares and triangles in Spanish and English (some examples can be found in Figure 4, where the task is referred to as MLA Task). There are a102a64a103 words in the Spanish vocabulary and a102a64a104 in  the English one.</Paragraph>
    <Paragraph position="1"> Let us begin considering English-to-Spanish translation, with a105a99a106 ,a106a64a106a64a106 pairs for training the systems and a102a14a106a64a106 different ones for testing purposes. We carefully implemented the original Grammar Association system described in (Vidal et al., 1993), tuned empirically a couple of smoothing parameters, trained the models and, finally, obtained an a119a21a120 a100 a104a122a121 of correct translations.9 Then, we studied the impact of: (1) sorting, as proposed in Section 3, the set of sentences presented to ECGI; (2) making language models deterministic and minimum; (3) constraining the best translation search to those sentences whose lengths have been seen, in the training set, related to the length of the input sentence. As shown in Table 1, all the proposed measures were beneficial and we got a final a103a64a103 a100 a104a122a121 of correct translations (that is, just one translation was wrong). Hence, we decided to apply those measures to all our Grammar Association systems and, in particular, to our Loco C one. This system, after tuning some minor parameters (for instance, the number of re-estimation iterations for the model was fixed to a104a14a106a64a106 ), got a</Paragraph>
    <Paragraph position="3"> Then, in order to further compare our two systems (which will be referred to as IOGA, for Improved Original Grammar Association, and simply Loco C) without more manual tuning, both were tested with a105 ,a106a64a106a64a106 new sentence pairs: in  In a second series of experiments, we wanted to compare our best system, Loco C, with Re-ConTra, the recurrent connectionist system described in (Casta~no and Casacuberta, 1997), where a a103a64a119  Since the MLA Task is an artificial task where each language can be exactly modelled by an acyclic finite-state automaton, we decided to use those exact automata in our systems in order to measure the impact of perfect language modelling. In this case, Loco C reached perfect re- null conclusion to this second series of experiments, we can point out that our systems are quite sensitive to the quality of language models and, also, that Loco C is a very good association model. Our last series of experiments were carried out on a different, more complex task (but artificial too). It was extracted from the task defined for the first phase of the EUTRANS project (Amengual et al., 1996) and covers just a small subset of the situations tourists can face when leaving hotels (some examples can be found in Figure 4, where the task is referred to as Simplified Tourist Task). There are a105a113a133a14a119 words in the Spanish vocabulary and a105a80a120a122a106 in the English one. We defined a standard scenario in which Spanish-to-English translation must be performed on a105 ,a106a64a106a64a106 sentences after training the corresponding models with a104 ,a106a64a106a64a106 pairs.</Paragraph>
    <Paragraph position="4"> In that scenario, Loco C achieved an a119a14a106</Paragraph>
    <Paragraph position="6"> of correct translations, where errors are mainly due to lack of coverage in the language models, especially in the input one: only a119a64a104 a100 a133a64a121 of the Spanish sentences in the test set could be correctly parsed with the inferred model, so we decided to apply word categories to improve the generalization capabilities of ECGI as exemplified in Section 3. Using automatic categorization (Martin et al., 1995) for extracting a133a14a104 Spanish word classes and a104a14a106 English ones, the resulting language models achieved perfect coverage and the Loco C system performance increased to a103a64a119  the same system in the absence of categorization.</Paragraph>
  </Section>
class="xml-element"></Paper>