<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-0207">
  <Title>Generation from Lexical Conceptual Structures</Title>
  <Section position="7" start_page="57" end_page="57" type="evalu">
    <SectionTitle>
6 Evaluation
</SectionTitle>
    <Paragraph position="0"> So far most of the evaluation has been fairly small-scale and fairly subjective, generating English sentences from CLCSs produced from about 80 sentences. Evaluation in this case is difficult, because the ultimate criteria is translation quality, which can, itself, be difficult to judge, but, moreover, it can be hard to attribute specific deficits to the analysis phase, the lexical resources, or the generation system proper. So far results have been mostly adequate, even for large and fairly complex sentences, taking less than 1 minute for generation up to inputs of about 1 megabyte input CLCS files. Ambiguity and complexity beyond that level tends to overtax the generation system.</Paragraph>
    <Paragraph position="1"> For the most part, the over-generation strategy of Nitrogen, coupled with the bigram preferences works very well. There are still some difficulties, however.</Paragraph>
    <Paragraph position="2"> One major one is that, especially with its bias for shorter sentences, fluency is given preference over translation fidelity. Thus, if there are options of whether or not to express some optional information, this will tend to be left out. Also, bigrams are obviously inadequate for capturing long-distance dependencies, and so, if things like agreement are not carefully controlled in the symbolic component, they will be incorrect in some cases.</Paragraph>
    <Paragraph position="3"> The generation component has also been used on a broader scale, generating thousands of simple sentences - at least one for each verb in the English LCS lexicon, creating sentence templates to be used in a Cross-Language information retrieval system (Levow et al., 2000).</Paragraph>
  </Section>
class="xml-element"></Paper>