<?xml version="1.0" standalone="yes"?>
<Paper uid="H89-2011">
  <Title>Francois-Michel Lang and Lynette Hirschman, Improved Portability and Parsing through Interactive Acquisition of Semantic Information. Proc. Second Conf. Applied Natural</Title>
  <Section position="3" start_page="72" end_page="73" type="ackno">
    <SectionTitle>
4. Discussion
</SectionTitle>
    <Paragraph position="0"> The effect of removing preference semantics would have been greater were it not for the presence of other mechanisms included in our system to enhance robustness. One of these is the arrangment of the semantic models in a hierarchy, so that if a model for a specific noun or verb fails to match, an attempt will be made to match a more general model. Another is a 'longest parse' mechanism which, if no analysis can be obtained for the entire sentence, takes the longest substring, starting with the first word, for which an analysis was obtained.</Paragraph>
    <Paragraph position="1"> We may expect that as one robustness mechanism is removed, others will play a larger role. We can see this effect between preference semantics and the longest parse mechanism. When running with preference semantics, the system resorts to the longest parse heuristic 42 times (246 other sentences got full parses); when preference semantics is disabled (i.e., selection is strictly enforced), the system used longest parse 83 times (68 others got full parses). This effect can be understood as follows: if the sentence contains a modifier which does not fit the semantic model, preference semantics will incorporate it into the sentence analysis with a penalty. If preference semantics is disabled and the modifier is near the end of the sentence, we may be able to obtain an analysis of the text up to the beginning of the modifier as a complete sentence or sentence fragment; this analysis will be returned by the longest parse heuristic.</Paragraph>
    <Paragraph position="2"> If both preference semantics and the longest parse mechanism are disabled, we are left with only 68 sentences which can be analyzed. The task performance plummets accordingly: only 43 (33%) of the events are correctly identified. These results can be summarized in a table: 2 For 246 sentences, we obtained a parse of the entire sentence; for an additional 42, a parse of a substring of the sentence. See section 4 for further discussion.</Paragraph>
    <Paragraph position="3">  However, in preparing this paper we have found and corrected a small error in the selection mechanism, and rerun all the experiments with this correction. This has resulted in small changes in some of the figures reported.</Paragraph>
    <Paragraph position="4">  heuristics used preference semantics and longest parse longest parse neither full sent. parses substring parses  The specific numbers presented here are not especially significant, since they reflect the incompleteness of the semantic model at the time of our evaluation. Our semantic model was constructed entirely by hand; for future evaluations, we hope that larger text samples coupled with more automated procedures for model acquisition (as described, for example, in (Gfishman 1986) and (Lang 1988)) will allow us to provide broader model coverage within similar time constraints. However, even with the best tools significant gaps will be unavoidable in a model for a large domain. This paper has indicated how, under these circumstances, relatively simple mechanisms can be used to boost the performance of text understanding systems.</Paragraph>
    <Paragraph position="5"> Acknowledgement. This research was supported by the Defense Advanced Research Projects Agency under contract N00014-85-K-0163 from the Office of Naval Research.</Paragraph>
  </Section>
class="xml-element"></Paper>