<?xml version="1.0" standalone="yes"?>
<Paper uid="H89-1051">
  <Title>PORTING PUNDIT TO THE RESOURCE MANAGEMENT DOMAIN</Title>
  <Section position="3" start_page="277" end_page="278" type="metho">
    <SectionTitle>
THE PORT
</SectionTitle>
    <Paragraph position="0"> As mentioned above, in this initial experiment, we undertook only the syntactic processing of the Resource Management training and test corpus. In the PUNDIT system, the syntactic stage consists of the generation of a detailed surface parse tree and the construction of a regularized Intermediate Syntactic Representation or ISR. The ISR uses an operator/argument notation to represent the regularized syntax. The regularization includes insertion of omitted constituents in relative clause constructions or as a result of various raising and equi operations. In addition, we performed some limited experiments running with selection, with provides a shallow (selection-based) semantic filtering during parsing \[Lang1988\].</Paragraph>
    <Paragraph position="1"> The tasks associated with the port are summarized below, with estimates of the time in person-weeks (PW). The total elapsed time was 1.5 months; the total port time was 10 person-weeks.</Paragraph>
    <Paragraph position="2"> Steps in Porting PUNDIT</Paragraph>
    <Paragraph position="4"> The final lexicon consisted of approximately 1100 words; this number is greater than the usually quoted vocabulary size for the resource management corpus, due to the inclusion of a number of multl-word expressions in our lexicon, particularly for handling geographic names (Bering Straights, Gulf of Tonkin). Of these, approximately 450 words were already in our general lexicon (which is still quite small, some 5000 words). We entered the remaining 650 words. This total number represents a mix of general English entries (some 150 words), ship names (200), numbers ( about 50, handled by the shapes component for productive expressions), place names (150), and some domain-specific entries (approximately 100), which were kept separate from the general English lexicon (e.g., hfdj~.</Paragraph>
    <Paragraph position="5">  3. A change to 4. A change to 5. A change to area8.</Paragraph>
    <Paragraph position="6"> 6. A change to</Paragraph>
  </Section>
  <Section position="4" start_page="278" end_page="278" type="metho">
    <SectionTitle>
SYNTAX
</SectionTitle>
    <Paragraph position="0"> Changes to the syntax focused on adding coverage, but not removing any definitions. It even turned out that our treatment of fragmentary or incomplete sentences \[Linebarger1988\] was needed to run the resource management corpus, for sentences such as The Kirk's distance from Hornei'. A few months prior to the beginning of the Resource Management port, we had added a comprehensive treatment of wh-expressions \[Hirschman1988\], which includes both relative clauses and question forms; at the same time, we had also added a treatment of imperatives. The fact that the grammar already contained these constructions made the port possible.</Paragraph>
    <Paragraph position="1"> There were only some ten constructions that were missing from the grammar. Of these, the most significant was a detailed treatment of the comparative. Fortunately, most of these could be handled (syntactically) by treating the comparative than operator as a right adjunct to the word being modified, e.g., than 1~ knots is a right-modifier of greater in speed greater than 1~ knots. This required only that than be treated in the lexicon as a preposition. This certainly does not represent an adequate treatment of the comparative, and indeed, certain complex comparative constructions were not covered by this minimal treatment, for example Is Puffer's position nearer to OSGP than Queenfish's location isf'.</Paragraph>
    <Paragraph position="2"> Other additions to the grammar included:  1. A treatment for what if questions, based on the existing treatment of wh-expressions.</Paragraph>
    <Paragraph position="3"> 2. A treatment for prepositionless time expressions, e.g., Monday or September ~, etc.</Paragraph>
    <Paragraph position="4">  allow determiners to have left modifiers, as in half the fuel or only these.</Paragraph>
    <Paragraph position="5"> allow adjectives to have a certain class of left modifiers, as in last three minutes.</Paragraph>
    <Paragraph position="6"> allow multiple right noun adjuncts, as in problems for Fanning that affect mission allow a preposed nominal argument to an adjective, as in harpoon capable.</Paragraph>
    <Paragraph position="7">  7. A change to allow fraction expressions (e.g., two thirds).</Paragraph>
    <Paragraph position="8"> 8. Domain specific changes to handle degree expressions and the particular forms of dates encountered in the corpus.</Paragraph>
    <Paragraph position="9"> These changes, coupled with a few changes to the restrictions, were sufficient to cover a very substantial portion of the corpus. Constructions that we did not cover (but which would require only modest grammar extensions to cover) include: 1. or + comparative as a right-modlfier of comparative adjectives, e.g., m5 or lower. 2. Certain combinations of right noun adjuncts, e.g., cruisers that are in the Indian Ocean that went to c~ August twenty.</Paragraph>
    <Paragraph position="10"> 3. Questions containing the form how + adjective (how bad) and how + adverb (how soon). This hole accounted for a substantial portion of of the incorrectly parsed sentences.</Paragraph>
  </Section>
  <Section position="5" start_page="278" end_page="279" type="metho">
    <SectionTitle>
SELECTION
</SectionTitle>
    <Paragraph position="0"> One way to constrain the search space that results from a broad-coverage grammar and lexlcon is to apply semantic constraints. Although we did not perform a deep semantic analysis, we did apply shallow semantic (selectlonal) constraints, to filter out semantically anomalous parses, in a second experiment. This procedure used PUNDIT's Selection Pattern Query and Response (SPQR) component ~Lang1988\]. We first used SPQR in acquisition mode, to collect semantic patterns, These patterns were then used to constrain search in parsing the test sentences.</Paragraph>
    <Paragraph position="1"> The acquisition procedure queries the &amp;quot;domain expert&amp;quot; during parsing, whenever it finds a new pattern, such as a new subject-verb-object pattern, or a new adjective-noun pattern. The expert declares that the pattern is valid, allowing parsing to continue, or that the pattern is invalid, which causes backtracking to find a different analysis (and associated pattern). Information about valid and invalid patterns is stored in a pattern database; as the parser generates each phrase, it checks  the pattern database to see whether the expert has ruled on this pattern; if the user has already classified the pattern, then the user need not be queried again. Thus the system &amp;quot;learns&amp;quot; as it parses more sentences. Following the acquisition (or training) phase, the system can be run in one of two modes: allowing any unknown pattern to succeed (which will overgenerate, assuming that the set of patterns is incomplete), or forcing unknown patterns to fail, which will undergenerate.</Paragraph>
    <Paragraph position="2"> To try to obtain maximum coverage of patterns, we generalized the patterns to semantic elaaa patterns, rather than patterns of actual words. For example, the subject-verb-object word pattern \[Yorktown, decrease, speed/, can be generalized (using the taxonomy provided by the knowledge base) to the semantic class pattern (the suf~x _G stands for concept): /platform_ C,~ha,~ge_ C, tra,~,ie~t_,hlp_aUribute_ C/.</Paragraph>
    <Paragraph position="3"> Previous experience had shown that use of word-level selectlonal patterns reduced the search by 20%, and the number of parses by a factor of three. We had hoped to achieve greater generality by use of the generalized semantic class patterns. However, due to time constraints, we were only able to process the first 100 training sentences, from which we collected some 450 patterns. This turned out (not surprisingly) to be far too small a set to generate any useful constraints in parsing. We therefore plan to complete our pattern collection on the full training set and rerun our experiment. This should provide us with a good measure of two things: the amount of pruning provided by application of shallow semantic constraints; and the amount of data that is required to obtain a complete set of patterns.</Paragraph>
  </Section>
  <Section position="6" start_page="279" end_page="279" type="metho">
    <SectionTitle>
THE KNOWLEDGE BASE
</SectionTitle>
    <Paragraph position="0"> Our experiment with generalization of semantic patterns required the use of a class hierarchy residing in a knowledge base. To support selection, we constructed a first pass at a knowledge base for the resource management domain. The KB contained some 750 concepts. One interesting observation that resulted from this exercise was that the semantic classes required for selection are not necessarily those classes that a knowledge engineer would develop as part of a domain model. In particular, certain words may exhibit similar distribution linguistically (e.g., average and maximum) but may not necessarily be collected under a single concept to permit easy generalization. For this reason, we may move to a more data-drlven paradigm for building the knowledge base in our subsequent experiments.</Paragraph>
  </Section>
  <Section position="7" start_page="279" end_page="280" type="metho">
    <SectionTitle>
THE METHODOLOGY
</SectionTitle>
    <Paragraph position="0"> As previously stated, we added domaln-independent rules to the grammar, and domain-independent entries to the lexicon, to cover the major constructions observed in the resource management corpus. We then trained on a (subset of) this corpus. The training involved parsing the first 200 sentences and examining and fixing parsing problems in these 200 sentences. We were able to collect semantic patterns only for the first 100 sentences. In both cases, this represents only a small fraction of the available training data (791 sentences). The sentences (training and test) were run on PUNDIT, under Quintus Prolog 2.2 on a Sun 3/60 with 8 MB of memory.</Paragraph>
    <Paragraph position="1"> Because PUNDIT normally produces many parses, especially when run without selectlonal eonstraints, we allowed the system to run to a maximum of 15 parses per sentence. We report several results below, for purposes of comparison with other groups presenting parsing results. The first result is the number of sentences obtaining a parse. We believe that this is not a meaningful figure, however, since it is possible for a sentence to obtain a parse, but never to obtain a correct parse. For  this reason, we report a second result: the number of sentences obtaining a correct parse within the first 15 parses. In some cases, the system obtained a parse, but did NOT obtain the correct parse within the first 15 parses. In this case, we report it a NOT GETTING A CORRECT PARSE.</Paragraph>
    <Paragraph position="2"> Our criteria for counting a parse correct were generally very stringent, and also required obtaining the correct regularized syntactic expression (or ISR). Our criteria included, for example: correct scoping of modifiers under conjunction; correct attachment of prepositional phrase and relative clause modifiers; and correct analysis of complex verb objects.</Paragraph>
  </Section>
class="xml-element"></Paper>