<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-0107">
  <Title>A Measure of Semantic Complexity for Natural Language Systems</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Quantification of task difficulty has been applied to many areas in artificial intelligence, including information retrieval (Bagga, 1997) (Bagga and Biermann, 1997), machine learning (Niyogi, 1996), parsing and grammatical formalisms(G. Edward Barton et al., 1987), and language learning in general (Ristad, 1993). In addition to providing a way of comparing systems, these measures quantify task complexity before a system is built. The goal of this paper is to measure the complexity of domains for dialog processing. With a standard measure of complexity, domains can be compared and analyzed without having to build the dialog system first. This measure would be an indication of the cost, amount of code, accuracy, reliability, and execution time of the finished dialog system specified by the domain.</Paragraph>
    <Paragraph position="1"> The hope is to have a single number or pair of numbers that correlates strongly with these standard measures.</Paragraph>
    <Paragraph position="2"> * Supported by the Defense Advanced Research Projects Agency, CPoF project, Grant F30602-99-C-0060 Specifically, if domain D1 has complexity C1 and domain D2 has complexity C2 where C2 &gt; C1, then we would expect D2 to have a greater cost of software, more lines of code, less accuracy, less reliability, and longer execution time.</Paragraph>
    <Paragraph position="3"> Section 2 will describe the difference in semantic and syntactic complexity and explain why we consider each separately. In section 3 we define the terms in the complexity analysis, which is explained in section 4. Sections 5 and 6 discuss how to compute information measures that are needed in the complexity analysis, and in sections 7 and 8 we present future work and conclude.</Paragraph>
  </Section>
class="xml-element"></Paper>