<?xml version="1.0" standalone="yes"?>
<Paper uid="H91-1017">
  <Title>Interface Bugs Ignored Test Set System % Correct % Incorrect % No Answer Total Error</Title>
  <Section position="11" start_page="109" end_page="109" type="concl">
    <SectionTitle>
SUMMARY
</SectionTitle>
    <Paragraph position="0"> To summarize, SOUL was designed to deal with ambiguous, unanswerable, illegal and context removable utterances. The approach taken was to create an extensive semantic and pragmatic knowledge base for use in abductive reasoning and constraint satisfaction. The resulting system performs fine grained analysis of an input utterance when criteria for activating the postprocessor is met. It was hypothesized that the SOUL processor would contribute more significantly in difficult processing / interpretation situations, while the case-frame parser, itself semantically based, would be sufficient for more restricted test sets. This is shown by comparing error rates of PHOENIX alone and PHOENIX coupled with SOUL across the two conditions of restricted and unrestricted utterance sets. Test Sets 1 and 4 (DARPA June 1990 and February 1991) are highly constrained, while Test Sets 2 and 3 are completely unconstrained. As hypothesized, the SOUL system contributes significantly more to reducing error rates and enhancing accuracy when applied to the more difficult, unrestaScted data. When processing unrestricted input, as required in real world applications, the,addition of a semantic and pragmatic postprocessor for performing fine grained analyses results in significant improvements in accuracy.</Paragraph>
    <Paragraph position="1"> However, it would be expected that given a complete dialog and all the context knowledge a system can capitalize upon, or even a greater amount of context, SOUL would perform better than it does with limited context D1 or no-context Class A utterances even if they were constrained.</Paragraph>
    <Paragraph position="2"> The second question posed was whether a knowledge base alone would enable detection of contextually dependent utterances where the appficable context is unavailable. Results of Test Set 3 indicate reasonable detection abilities (70%).</Paragraph>
    <Paragraph position="3"> In summary, semantic and pragmatic knowledge can be effeefively used to enhance a system's accuracy of interpreta~tion rates. This effect holds even in isolated utterance processing tasks, which provide a great deal less data than can be derived from a complete dialog. In the absence of dialog, the accuracy improvements are more marked in more difficult processing conditions than when processing constrained, relatively straight forward utterances. null</Paragraph>
  </Section>
class="xml-element"></Paper>