<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0406">
  <Title>Dual Use of Linguistic Resources: Evaluation of MT Systems and Language Learners</Title>
  <Section position="3" start_page="33" end_page="33" type="intro">
    <SectionTitle>
2. Developing Linguistic
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="33" end_page="33" type="sub_section">
      <SectionTitle>
Resources for Evaluation
</SectionTitle>
      <Paragraph position="0"> In order to assess how accurately and consistently MT systems handle spatial language and how effectively second language learners are being taught about spatial language, we followed these steps in constructing a spatial expressions evaluation dataset. We first built a master list of English prepositions from several sources (Lindstromberg, 1998; Websters, website) and then created a sublist of only spatial prepositions, based on the judgments of three native English speakers, two of whom were linguistically trained and one who was not.</Paragraph>
      <Paragraph position="1"> Second, we constructed English sentences where the spatial PP was systematically composed into different syntactic positions, as shown in Figure 1. This enabled us to examine the range of Path/Place-type ambiguity in the resulting spatial expressions. The spatial prepositions were placed in contexts where only one place or path interpretation was feasible, as well as in contexts where the reading was ambiguous. For example, PP's with the preposition across were composed in (i) a verb's subcategorized argument for the verbs live and dance, as in 'he lived/danced \[PP across the street\]', (ii) as a verb's non-subcategorized argument for the verbs scare and sneeze, as in 'he scared the child \[PP across the street\]' and 'he sneezed the cards \[PP across the table\]', and (iii) as an adjunct outside the VP for the verb eat, as in 'he ate dinner \[PP across the street\]'.</Paragraph>
      <Paragraph position="2"> subcat Arg - nonsubcat Arg adjunct &amp;quot;He lives &amp;quot;He sneezed the &amp;quot;He ate across the cards across the dinner street.&amp;quot; table.&amp;quot; across the &amp;quot;He danced &amp;quot;He scared the child street.&amp;quot; across the street.&amp;quot; across the street.&amp;quot; Figure 1, a row of ESE data set, with preposition across Once this English Spatial Expression (ESE) dataset was constructed, our third step was to elicit translations of a subset of these sentences into French. Our translator was a native French speaker, fluent in English, with a PhD in linguistics from a university in the U.S. Our translator's extensive training in linguistics made it possible for us to be quite specific about the English ambiguities present in the sentences that we needed him to translate.</Paragraph>
      <Paragraph position="3"> Our efforts have yielded the following datasets: an English preposition list, an English spatial preposition list, a dataset of English spatial expressions sorted by their spatial preposition and syntactic structure, and a dataset of high-quality French translations of a proper subset of the ESE dataset.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>