<?xml version="1.0" standalone="yes"?>
<Paper uid="C88-2163">
  <Title>Default Reasoning in Natural Language Processing lid ZERNIK Artificial Intelligence Program GE, Research and Development Center</Title>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
L Introduction
</SectionTitle>
    <Paragraph position="0"> Ironically, atlding knowlexlge to a computational system does not always extend its power. When equipped with comprehensive linguistic knowledge the same system might fare worse than when equipped with impoverished linguistic knowledge, since additional rules might impair simple cases. For example consider the following two sentences: (l) The ,-hitd sold his parents' dearest ornmnent.</Paragraph>
    <Paragraph position="1"> (2) &amp;quot;file child sold to a stranger was found alive.</Paragraph>
    <Paragraph position="2"> Linguistic systems must handle both sentences: sentence (1) which appears unambiguous~ artd requh'es only basic English ga~ammar, and sentence (2) whose interpretation is obscured by its g~axrmaatie conslruefion. However, it is crucial that rules required for h,'tndling sentence (2) will not complicate the analysis of sentence (l) by contfibnting sptwious ambiguity and computational ovedlead.</Paragraph>
    <Paragraph position="3"> This linguistic behavior is problematic for computer parsers.</Paragraph>
    <Paragraph position="4"> Sentence (2) involves tile garden path phenomenon, in which the reader is let to a standard interpretation which is retracted towards the eJJd of the sentence. On the other hand, sentence (1) is read &amp;quot;linearly&amp;quot; without any such deviations. For a computer parser, it is important to show (a) how the irdtial interpretation is flipped in parsing sentence (2), and (b), that for sentence (1) no such flipping occurs. However, this behavior cannot be be captured by PROLOG-based parsers* \[Dah177,PereiraS0\] for two reasons: (a) representation: all linguistic rules ate viewed as equal citizens, (b) computation: PROLOG backtracking can search for any solution, yet it cannot retract partial solutions.</Paragraph>
    <Paragraph position="5"> Thus, this difference in parsing complexity must yet be manifested in computational temas.</Paragraph>
    <Paragraph position="6"> Default reasoning \[MeCarthyS0\] and Truth Maintenance Systems (TMS's) were introduced to cope with the representational and computational limitations of first-order logic \[Doyle79, Reiter87, Goodwin87, Brown87\]. First, TMS's distinguish between default cases, and cases presenting deviations from the norm. Second, TMS's ~dlow the retraction of default assumptions during tile reasoning process itself. In this paper we explain how existiug parsing mechanisms based on unification fail in capturing important linguistic behavior. Furthermore, by implementing NMG (Non-Monotonic Grammar), we demonstrate how by monitoring a logic parser, a TMS can significantly enhance the parser's perfonnance without adding auy excessive overhead.</Paragraph>
  </Section>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2. The Theoretical Issues
</SectionTitle>
    <Paragraph position="0"> Four theoretical issues must be considered when parsing text: ambiguity, non monotonicity, context dependency, and knowledge gaps.</Paragraph>
    <Paragraph position="1"> (a) Ambiguity: Consider the following sentence: Mary was tired, so John shnwed her home.</Paragraph>
    <Paragraph position="2"> Even for this simple sentence which appears unambiguous to a human reader, a parser must span a variety of interpretations stemming from several linguistic levels: Semantics: Who is &amp;quot;John&amp;quot;: is it John Ma~erg or is it John Kepler (where both exist in the context)? Syntax: What part of speech is &amp;quot;her&amp;quot;: is it an objective pronoun, i.e. he showed Mary home or is it a possessive, i.e. he showed Mary's home? Lexicon: What is &amp;quot;to show&amp;quot;, and what is the intended memting: he showed the house to a potential buyer, or he showed her die way to her home? Role-Binding: Does &amp;quot;show&amp;quot; (meaning make-visible) take 1 or 2 objects? he showed &lt;her&gt;, or he showed &lt;her&gt; &lt;home&gt;? Context: What is the purpose of the act: selling Mary's house to potential buyers, or making sure she arrives safely home? *We allude here to PROLOG as a parsing mechanism and not to the programming language PROLOG which is computationally universal. 80\] A parser is required to negotiate this set of conflicting/cooperating clues and to deposit in a database a hypothesis about the meaning of the utterance. This problem is even aggravated when imposing left-to-right order on parsing. At each point, the parser must deposit a hypothesis based on a partial set of clues, a hypothesis which might be later retracted.  (b) Non-Monotonicity: Garden path sentences highlight the problem: (3) The old man's glasses were filled with sherry.</Paragraph>
    <Paragraph position="3"> (4) I saw the Grand Canyon flying to New York.</Paragraph>
    <Paragraph position="4"> (5) The book sold yesterday was cheap.</Paragraph>
    <Paragraph position="5"> (6) The horse raced past the barn fell.</Paragraph>
    <Paragraph position="6"> In each one of these sentences, the initial hypothesis is retracted when an assumed default rule is violated: (3) A semantic assumption--gl~ses stand for looking glasses-must be retracted at the end of the sentence-since glasses turn out to actually mean drinking containers. (4) A syntactic assumption dictates the default sentence structure (S -&gt; VP, NP). Thus the Grand Canyon is taken as the agent flying to NY. This interpretation is in conflict with existing world knowledge (since canyons cannot fly). (5) A word sold is assumed to take the active voice. Back-tracking occurs due to knowledge of the selling act (books do not normally sell days).</Paragraph>
    <Paragraph position="7"> (6) There are two default assumptions which fail: (1) raced is  taken by default to be active voice, and (2) the clause raced past the barn is taken as the verb phrase of the sentence. Both assumptions fail ~,hen the word fell is encountered, and raced is found to be the past participle in the unusual sense of &amp;quot;being driven&amp;quot;.</Paragraph>
    <Paragraph position="8"> Among these examples, some are impossible even for human listeners to comprehend; in others, the flipping of the meaning is hardly noticeable. How can a parsing mechanism reflect these degrees of difficulty?  (c) Context Dependency: Compare the interpretations of following pairs of sentences: (7) John wanted a new car. He took it up with his dad. (8) This is his new Porsche. He took it up with his dad.  The syntactic analysis was driven by the semantic context established by prior discourse: in (7), up is taken as an integral part of the entire lexicai phrase to take it up with somebody, where in (8) it serves as a general adverb. This is due to the different interpretation of it: in (7) John discussed &amp;quot;an issue&amp;quot;, while in (8) he probably drove &amp;quot;the car&amp;quot; itself up the hill accompanied by his dad. How can a parser reflect such context sensitivity? (d) Lexical Gaps: Linguistic systems cannot be assumed complete, and so text processing must take place even in the presence of lexicai gaps \[Zeruik87, Zernik88\]. Consider the following sentence which includes a lexical unknown, the word plend.  (9) John plended Mary to come over.</Paragraph>
    <Paragraph position="9"> How can a parser make even a partial sense out of this sentence? Can a parser use default lexicon rules in parsing such unknowns? 3. Truth Maintenance: The Architecture of NMG What is a truth maintenance sytem and how can it be tailored for  use in the linguistic domain? NMG is a system for employing truth maintenance in text comprehension. NMG's architecture is simple: it includes a parser (PAR), and a Truth Maintenance System (TMS). PAR is a DCG-style parser \[Pereira80\], implemented as a forward-chaining theorem prover. PAR produces a parse tree in the form of a logieai dependency-net. This dependency-net is fed to a TMS whose output is an IN/OUT labelling presenting the currently believed interpretation. The basic computationai paradigm is described by an example in parsing the sentence below: (10) The baby sold by his parents was found alive.</Paragraph>
    <Paragraph position="10"> Text interpretation can be pursued in two possible ways: either by propagating all possible grammatic interpretations of that utterance, as in logic parsers. This might introduce spurious interpretations for simple text. Or alternatively, by initially trying a default interpretation, and only when this default interpretation fails try out other interpretations. This is the approach taken by NMG. There are two interesting stages in the computation, according to the text already received:  (10a) The baby sold (10b) by his parents was found alive.</Paragraph>
    <Paragraph position="11"> One basic default assumption was committed at stage (10a): o Unless otherwise observed, the verb sold assumes the active voice past tense.</Paragraph>
    <Paragraph position="12">  The rules related to this assumption are given in NMG as follows: null</Paragraph>
    <Paragraph position="14"> NMG extends DCG's Horu-clause notation by allowing an &amp;quot;unless&amp;quot; term, marked by the symbol O. Such an &amp;quot;unless&amp;quot; term appears in (rl), the default case for sold. Deviations can be established as aggregation of cases. (R2) presents the first deviation.</Paragraph>
    <Paragraph position="15"> When sold is preceded by the particle was, it is taken as a passive voice. (R3) presents the second deviation, when a by-clanse follows sold. Other deviations may be added on as the linguistic system is enhanced. The diagram below describes two snapshots in the computation of sentence (10).</Paragraph>
    <Paragraph position="17"> Notation: ovals in this scheme stand for facts; AND gates stand for rules; dark ovals are IN; light ovals are OUT. Consider part (a) which describes the parse of (10a): The dependency-net construtted by PAR is based on instantiated linguistic rules. There are three new instantiated facts: NP the noun phrase, V the verb, and S the entire sentence (some short cuts were made in drawing the parse tree due to space limitations). The associated IN/OUT labelling is produced by the TMS, and so far all the facts have been labelk~ IN. In particular, the output of the default rule (rl) is labelled IN, since its inhibitive port (marked by an inverter) is labeled OUT: no deviation of (rl) has yet been observed.</Paragraph>
    <Paragraph position="18"> Part (b) describes the parse after the rest of the sentence (10b) has been read. (Notice that the dependencies of part (a) have been copied over for reference purposes, although in the model itself dependencies are not recalculated or copied). Reading  (10b) causes the withdrawal of the previous interpretation and the construction of a new one. However, since new words were only added on, how in this scheme, could anything be withdrawn? In stage (b) too there are two orthogonal activities: (1) PAR constructs new dependencies: The by-clause following  sold justifies the inhibitive port of (rl); the same by-clause also justifies an alternative role for sold 0/P), as a passive voice verb; this fact plus the by-clause itself add up to a relative clause (Re); RC joins the old noun-phrase (NP) to form a composite noun-phrase (CNP); CNP now joins a new verb-phrase O/P) in forming a new ~ntence (S). Throughout this process no dependencies trove been modified or retracted; new dependencies were only added on.</Paragraph>
    <Paragraph position="19"> (2) The TMS relabels the network. First, the old interpretation is ruled out: since the inhibitive input of (rl) is now labeled IN, the output of (rl) becomes OUT, and so does the initial interpretation S. The rest of the new facts are labelled IN. Thus, the non-monotonic effect is accomplished by relabeling nodes, and not by retracting dependencies.</Paragraph>
    <Paragraph position="20"> A PROLOG-based parser, at stage (b), must undo all its prior parsing inferences, retract the fact it has deposited in the database, and start processing from scratch, this time ruling out the incorrect default assumption. Using a TMS, a parser can recover by simply relabeling the parts of the parse which depend on the above assumptions, and proceod gracefully thereafter.</Paragraph>
    <Paragraph position="21"> Problems which are nattmdly addreued by non-~e inferenee are pervasive in the linguiltic~ domain, wheze palming proceeds left-to-fight, and they are epitomized by garden path sentences.</Paragraph>
    <Paragraph position="22">  4. NMG: A Process Model  Non-monotonic reasoning is not confined only to garden path sentences. We show here an example of an apparently simple sentence, for which interpretations are asserted and retraced dynamically. This example also demonstrates the role of default reasoning in lexical access and in context interaction, wbem the context is the semantic structure yielded by prior discourse.</Paragraph>
    <Paragraph position="23"> Consider the initial interpretation constructed after reading the following utterance: (lla) John needed a new battery. He took it This text yields an initial hypothesis: &amp;quot;John ptransed a battery&amp;quot;. This hypothesis is based on two default roles: (1) Lexieai access: unless otherwise observed, a generic word such as take indexes the generic meaning ptrans (physical transfer) \[Schank771 (2) Context interaction: Unless otherwise observed, it refers to the last physical object referred to in the discourse \[Sidner79\] (here it refers to the battery). However, as reading proceeds, these hypotheses are retracted: (l Ib) John needed a new battery. He took it up with his dad At this point, a more specific lexical entry is accessed: &amp;quot;X take Y up with Z&amp;quot; in the sense of &amp;quot;raising an issue&amp;quot; \[Wilks75, Wilensky80\]. The referent for it is switched now from the battery itself to &amp;quot;John's goal of getting a battery&amp;quot;, due to selectional restrictions in the new lexical phrase. However, the initial interpretation is recovered when reading continues: (tic) John needed a new battery.</Paragraph>
    <Paragraph position="24"> He took it up with his dad from the basement.</Paragraph>
    <Paragraph position="25"> At this point, the additional clauses in the sentence are used to augment the initial hypothesis which had been previously abandoned. Notice that the initial hypothesis is recovered simply by marlding it IN and not by computing it from scratch.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="803" type="metho">
    <SectionTitle>
5. Logic Programming: From CFG through DCG to NMG
</SectionTitle>
    <Paragraph position="0"> Logic programming \[Colmerauer73, Kowalski79, Dah177, Pereira80\] has mechanized many declarative linguistic systems \[Kay79, Bresnan82, Shieber87\], and provided a new computational paradigm. Definite-Clause Grammars (DCG) in particular, have exploited the advantages of Context-Free Grammars (CFG) by a simple extension. In DOG, a non-terminai may be any PROLOG term, rather than simply an atom as in CFG. The following example demonstrates how one particular rule has evolved from CFG to DCG.</Paragraph>
    <Paragraph position="1"> CFG: sentence --&gt; noun-phrase, verb-phrase DCG: sentence(s(NP,VP)) :- noun-phrase(NP,N), verb-phrase(NP,N) This extension has two features: (a) maintaining agreement rules-the argument N maintains the number in both the noun and the verb; (b) co0structing semantic denotations-the arguments NP and VP contain the denotations of the constituents, from which the denotation of the entire sentence (s(NP, VP)) is constructed. null  Logic progrmnming has assumed a central role in language processing for two reasons: (a) It allowed the expression of declarative linguistic rules, and (b) Direct application of PROLOG operationalized grammars, using unification and backtracking as the mechanism. PROLOG also enabled other gaTtrmnars (beside DCG) such as transformational grammar \[Chomsky81\] and case grammm&amp;quot; \[Fillmore68\], to be emulated. However, the direct application of PROLOG has presented three limitations: (a) Parsing was driven by syntax, and the semantic interpretation was a by-product. (b) While PROLOG itself can be extended to express default (through the notion of negation as .failure \[Clark77\]), PROLOG does not have an explicit notion of dependency so that a parser can diagnose what went wrong with the parse. (c) PROLOG itself does not facilitate default reasoning which can resolve lexical gaps.</Paragraph>
    <Paragraph position="2"> Therefore, we have introduced NMG; a logic parser which enhances DCG's capabilities in three' ways: non-monotonic rea.soning, refinement and retraction, and diagnostic reasoning. (a) Non-Monotonic Reasoning: NMG enables the parser to gracefully con'ect its parse tree by identifying parts of the reasoning strncture which depend on retracted assumptions. Non-monotonicity is pervasive in language processing due to the serial nature of language.</Paragraph>
    <Paragraph position="3"> (b) Retraction and Refinement: A main objective in text processing, required for left-to-right parsing, has been parsing by refinement. In reading a sentence, a parser should not deposit a single final meaning when a &amp;quot;full stop&amp;quot; is encountered. Rather, an initial concept must be asserted as early as possible, an as~rtion which must be refined as further words are provided at the input, ltowever, in fulfilling this o,bjective, existing models \[Hirst86, Lytinen84, Jacobs87\] have not dealt with the possibility that the entire hypothesis might be retracted, and replaced by a second hypothesis. NMG enables a parser to both refine an existing hypothesis, mad to retract the entire hypothesis from the database, if contradictory evidence has been received.</Paragraph>
    <Paragraph position="4">  (c) Diagnostic Reasoning: Consider a pair of operational  modes, given in the diagram below: text dependencies labelin~ text dependencies labellnq While in (1), the system operates in an &amp;quot;open loop&amp;quot;, and the TMS basically monitors which hypothesis is cmTently IN, in (2) the information produced by the TMS can be used in reasoning about the parse itself. This is important in learning and in parsing ill-fo~rned text. We describe this mode in a later repom</Paragraph>
  </Section>
  <Section position="5" start_page="803" end_page="803" type="metho">
    <SectionTitle>
6. Conelusions
</SectionTitle>
    <Paragraph position="0"> We have presented NMG, a mechanism which can potentially enhance all logic parsers. NMG's advantages are emphasized in parsing complex sentences in which hypotheses are being retracted. However, its main advantage is in avoiding spurious activity when parsing simple sentences. Thus we have accomplished an objective laid down by Allan Kay: &amp;quot;Easy things should be easy; hard things should be possible&amp;quot;.</Paragraph>
  </Section>
class="xml-element"></Paper>