<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2043">
  <Title>An Empirical Assessment of Semantic Interpretation</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Semantic interpretation has been an actively investigated issue on the research agenda of the logic-based paradigm of NLP in the late eighties (e.g., Charniak and Goldman (1988), Moore (1989), Pereira and Pollack (1991)). With the emergence of empirical methodologies in the early nineties, attention has almost completely shifted away from this topic. Since then, semantic issues have mainly been dealt with under a lexical perspective, viz. in terms of the resolution of lexico-semantie ambiguities (e.g., Schfitze (1998), Pedersen and Bruce (1998)) and the generation of lexical hierarchies from large text corpora (e.g., Li and Abe (1996), Hirakawa et al. (1996)) massively using statistical techniques.</Paragraph>
    <Paragraph position="1"> The research on semantic interpretation that was conducted in the pre-empiricist age of NLP was mainly driven by an interest in logical formalisms as carriers for appropriate semantic representations of NL utterances. With this representational bias, computational matters -- how can semantic representation structures be properly derived from parse trees for a large variety of linguistic phenomena? -became a secondary issue. In particular, this research lacked entirely quantitative data reflecting the accuracy of the proposed semantic interpretation mechanisms on real-world language data.</Paragraph>
    <Paragraph position="2"> One might be tempted to argue that recent evaluation efforts within the field of information extraction (IE) systems (Chinchor et al., 1993) are going to remedy this shortcoming. Given, however, the fixed number of knowledge templates and the restricted types of entities, locations, and events they encode as target information to be extracted, one readily realizes that such an evaluation framework provides, at best, a considerably biased, overly selective test environment for judging the understanding potential of text analysis systems which are not tuned for this special application.</Paragraph>
    <Paragraph position="3"> On the other hand, the IE experiments clearly indicate the need for a quantitative assessment of the interpretative performance of natural language understanding systems. We will focus on this challenge and propose such a general evaluation framework.</Paragraph>
    <Paragraph position="4"> We first outline the model of semantic interpretation underlying our approach and then focus on 'its empirical assessment for two basic syntactic structures of the German language, viz. genitives and auxiliary constructions, in two domains.</Paragraph>
  </Section>
class="xml-element"></Paper>