<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1017">
  <Title>Recent Improvements and Benchmark Results for Paramax ATIS System</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1. INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> In recent work on the Paramax spoken language understanding system we have focused on domain-independent capabilities whose relevance extends beyond the ATIS application. These include non-monotonic reasoning, implicit reference resolution, and the ability to create a simple paraphrase of a query for feedback to the user.</Paragraph>
    <Paragraph position="1"> Although these capabilities were motivated by ATIS data, they are likely to be required for processing data from other domains. We describe these improvements in the following section. In addition, we discuss our results on the ATIS benchmark tests. Because test data vary from test to test, it can be difficult to know whether fluctuations in performance are real, or are due to idiosyncracies in the particular test data selected. We have experimented with an evaluation paradigm in which systems used in earlier tests run the test data from the current test, thus controlling for the effect of variations in the test data. We suggest that this paradigm could provide a valuable supplement to the official tests.</Paragraph>
  </Section>
class="xml-element"></Paper>