File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/03/n03-1022_metho.xml
Size: 19,834 bytes
Last Modified: 2025-10-06 14:08:08
<?xml version="1.0" standalone="yes"?> <Paper uid="N03-1022"> <Title>COGEX: A Logic Prover for Question Answering</Title> <Section position="3" start_page="0" end_page="0" type="metho"> <SectionTitle> 2 Integration of Logic Prover into a QA System </SectionTitle> <Paragraph position="0"> The QA system includes traditional modules such as question processing, document retrieval, answer extraction, built in ontologies, as well as many tools such as syntactic parser, name entity recognizer, word sense disambiguation (Moldovan and Noviscki 2002), logic representation of text (Moldovan and Rus 2001) and others.</Paragraph> <Paragraph position="1"> The Logic Prover is integrated in this rich NLP environment and augments the QA system operation.</Paragraph> <Paragraph position="2"> As shown in Figure 1, the inputs to COGEX consist of logic representations of questions, potential answer paragraphs, world knowledge and lexical information. The term Answer Logic Form (ALF) refers to the candidate answers in logic form. Candidate answers returned by the Answer Extraction module are classified as open text due to the unpredictable nature of their grammatical structure. The term Question Logic Form (QLF) refers to the questions posed to the Question Answering system represented in logic form.</Paragraph> <Paragraph position="3"> The prover also needs world knowledge axioms supplied by the WordNet glosses transformed into logic representations. Additionally there are many other axioms representing equivalence classes of linguistic patterns, called NLP axioms. All these are described below.</Paragraph> <Paragraph position="4"> The Axiom Builder converts the Logic Forms for the question, the glosses, and its candidate answers into axioms. Based on the parse tree patterns in the question and answers, other NLP axioms are built to supplement the existing general NLP axioms. Once the axioms are complete and loaded, justification of the answer begins.</Paragraph> <Paragraph position="5"> If a proof fails, the relaxation module is invoked. The purpose of this module is twofold: (1) to compensate for errors in the text parsing and Logic Form transformation phase, such as prepositional attachments and subject/object detection in verbs, (2) to detect correct answers when the NLP and XWN (Extended WordNet) axioms fail to provide all the necessary inferences. During the relaxation, arguments to predicates in the question are incrementally uncoupled, the proof score is reduced, and the justification is re-attempted. The loop between the Justification and the Relaxation modules continues until the proof succeeds, or the proof score is below a predefined threshold. When all the candidate answers are processed, the candidate answers are ranked based on their proof scores, with the output from COGEX being the ranked answers and the answer justifications.</Paragraph> </Section> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> 3 Logic Representation of Text </SectionTitle> <Paragraph position="0"> A text logic form (LF) is an intermediary step between syntactic parse and the deep semantic form. The LF codification acknowledges syntax-based relationships such as: (1) syntactic subjects, (2) syntactic objects, (3) prepositional attachments, (4) complex nominals, and (5) adjectival/adverbial adjuncts. Our approach is to derive the LF directly from the output of the syntactic parser which already resolves structural and syntactic ambiguities.</Paragraph> <Paragraph position="1"> Essentially there is a one to one mapping of the words of the text into the predicates in the logic form. The predicate names consist of the base form of the word concatenated with the part of speech of the word. Each noun has an argument that is used to represent it in other predicates. One of the most important features of the Logic Form representation is the fixed-slot allocation mechanism of the verb predicates (Hobbs 1993). This allows for the Logic Prover to see the difference between the role of the subjects and objects in a sentence that is not answerable in a keyword based situation.</Paragraph> <Paragraph position="2"> Logic Forms are derived from the grammar rules found in the parse tree of a sentence. There are far too many grammar rules in the English language to efficiently and realistically implement them all. We have observed that the top ten most frequently used grammar rules cover 90% of the cases for WordNet glosses. This is referred to as the 10-90 rule (Moldovan and Rus 2001). Below we provide a sample sentence and its corresponding LF representation.</Paragraph> <Paragraph position="3"> Example: Heavy selling of Standard & Poor's 500-stock index futures in Chicago relentlessly beat stocks downward.</Paragraph> </Section> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> 4 World Knowledge Axioms </SectionTitle> <Paragraph position="0"> Logic representation of WordNet glosses A major problem in QA is that often an answer is expressed in words different from the question keywords. World knowledge is necessary to conceptually link questions and answers. WordNet glosses contain a source of world knowledge. To be useful in automated reasoning, the glosses need to be transformed into logic forms.</Paragraph> <Paragraph position="1"> Taking the same approach as for open text, we have parsed and represented in logic forms more than 50,000 WordNet glosses. For example, the gloss definition of concept sport NN#1 is an active diversion requiring physical exertion and competition, which yields the logic representation:</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> Lexical Chains </SectionTitle> <Paragraph position="0"> A much improved source of world knowledge is obtained when the gloss words are semantically disambiguated (Moldovan and Noviscki 2002). By doing this, the connectivity between synsets is dramatically increased. Lexical chains can be established between synsets in different hierarchies. These are sequences of semantically related words that link two concepts.</Paragraph> <Paragraph position="1"> Lexical chains improve the performance of question answering systems in two ways: (1) increase the document retrieval recall and (2) improve the answer extraction by providing the much needed world knowledge axioms that link question keywords with answers concepts.</Paragraph> <Paragraph position="2"> We developed software that automatically provides connecting paths between any two WordNet synsets a0a2a1 and a0a4a3 up to a certain distance (Moldovan and Noviscki 2002). The meaning of these paths is that the concepts along a path are topically related. The path may contain any of the WordNet relations augmented with a GLOSS relation which indicates that a certain concept is present in a synset gloss.</Paragraph> </Section> <Section position="2" start_page="0" end_page="0" type="sub_section"> <SectionTitle> Examples </SectionTitle> <Paragraph position="0"> Below we provide some relevant lexical chains that link a few selected TREC 2002 questions with their answers.</Paragraph> <Paragraph position="1"> Q1394: What country did the game of croquet originate in ? Answer: Croquet is a 15th-century French sport that has largely been dominated by older, wealthier people who play at exclusive clubs.</Paragraph> </Section> </Section> <Section position="6" start_page="0" end_page="0" type="metho"> <SectionTitle> 5 NLP Axioms </SectionTitle> <Paragraph position="0"> In additions to world knowledge axioms, a QA Logic Prover needs linguistic knowledge. This is what distinguishes an NLP prover from a traditional mathematical prover. General axioms that reflect equivalence classes of linguistic patterns need to be created and instantiated when invoked. We call these NLP axioms and present below some examples together with questions that call them.</Paragraph> <Paragraph position="1"> Complex nominals and coordinated conjunctions A question may refer to a subject/object by its full proper name, and the answer will refer to the subject/object in an abbreviated form. For example in the correct candidate answer for the question, &quot;Which company created the Internet browser Mosaic?&quot;, Internet browser Mosaic is referred to as Mosaic.</Paragraph> <Paragraph position="2"> Using abduction, an axiom is built such that the head noun of the complex nominal in the question implies the remaining nouns in the complex nominal: all x1 (mosaic nn(x1) a5 internet nn(x1) & browser nn(x1)) An additional axiom is built such that all the nouns in the complex nominal imply a complex nominal:</Paragraph> <Paragraph position="4"> So as not to restrict the ordering of the nouns in the noun phrase from which the complex nominal is built, the same argument is used for each of the noun predicates in the complex nominal. Similar to the above issue, a question may refer to the subject/object in an abbreviated form, while the answer will refer to the subject/object in its full, proper form. For example in the correct candidate answer for the question, &quot;When was Microsoft established?&quot;, Microsoft is referred to as Microsoft Corp.</Paragraph> <Paragraph position="5"> An axiom is built such that each noun of the complex nominal takes on the identifying argument of the complex nominal:</Paragraph> <Paragraph position="7"> Similar axioms are used for coordinated conjunctions detected in the answer and the question. These are considered weak axioms, and any proof that uses them will be penalized by being given a lower score than those that do not.</Paragraph> <Paragraph position="8"> A candidate answer for a question may use an apposition to describe the subject/object of the answer. The question may refer to the subject/object by this apposition. For example in the question, &quot;Name the designer of the shoe that spawned millions of plastic imitations , known as jellies&quot;, the candidate answer, &quot;..Italian Andrea Pfister , designer of the 1979 &quot; bird cage &quot; shoe that spawned millions of plastic imitations, known as &quot; jellies ...&quot; uses an apposition to describe the designer.</Paragraph> <Paragraph position="9"> An axiom is built to link the head of the noun phrases in the apposition such that they share the same argument:</Paragraph> <Paragraph position="11"> A question/answer substitutes the use of a possesive by using an of or by preposition. For example, in the question, &quot;What was the length of the Wright brothers' first flight?&quot;, the candidate answer, &quot;Flying machines , which got off the ground with a 120 - foot flight by the Wright brothers in 1903...&quot; implies ownership using the preposition by to connect the Wright brothers to flight.</Paragraph> <Paragraph position="12"> An axiom is built to connect by to the possessive: all x1 x2 (by in(x1,x2) a5 pos(x1,x2)) Equivalence classes for prepositions Prepositions can be grouped into equivalence classes depending on the context of the question, which is determined by the expected answer type. In location seeking questions the prepositions at and in are often interchangeable. Similarly for in and into, and from and of. In date seeking questions in and of have interchangeable meanings as do at and in. For example, in the question, &quot;What body of water does the Colorado River flow into?&quot;, the candidate answer, &quot;...the Colorado River flowed in the Salton trough about 130 miles east of San Diego&quot;, the preposition in and into in the answer take in the same meaning.</Paragraph> <Paragraph position="13"> An axiom is built to link in to into: all x1 x2 (in in(x1,x2) a5 into in(x1,x2)) Part of relations in location questions A location seeking question may have a candidate answer that identifies a location by referring to a part of the location. For example, in the question, &quot;Where is Devil 's Tower?&quot;, the answer, &quot;American Indians won another court battle over their right to worship without interference at Devils Tower National Monument in the northeast corner of Wyoming&quot;, identifies Wyoming as the location of Devil 's Tower by referring to the part of Wyoming in which it lies. An axiom is built to connect Wyoming to its part: all x1 x2 x3 (corner nn(x1) & of in(x1,x2) & wyoming nn(x2) a5 wyoming nn(x1) ) Attribute of relations in quantity seeking questions A question seeking a quantity may have a candidate answer that implies quantity of subject by prefixing the quantity to the subject. For example in the question &quot;What is the height of the tallest redwood?&quot; the answer is &quot;329 feet Mother of Forest's Big Basin tallest redwood..&quot; An axiom is built to connect the quantity to its subject, redwood:</Paragraph> <Paragraph position="15"> This is a weak axiom since the proximity of redwood to quantity in the answer text is not guaranteed. As mentioned for the complex nominal and coordinated conjunction axioms, any proof that uses these axioms should be penalized and ranked lower than those that do not. Note that for this axiom to be effective, an axiom linking the heads of the apposition is built:</Paragraph> <Paragraph position="17"/> </Section> <Section position="7" start_page="0" end_page="0" type="metho"> <SectionTitle> 6 Control Strategy </SectionTitle> <Paragraph position="0"> Axiom partitioning mechanism The search strategy used is the Set of Support Strategy, which partitions the axioms used during the course of a proof into those that have support and those that are considered auxiliary (Wos 1988). The axioms with support are placed in the Set of Support (SOS) list and are intended to guide the proof. The auxiliary axioms are placed in the Usable list and are used to help the SOS infer new clauses. This strategy restricts the search such that a new clause is inferred if and only if one of its parent clauses come from the Set of Support. The axioms that are placed in the SOS are the candidate answers, the question negated (to invoke the proof by contradiction), and axioms related to linking named entities to answer types.</Paragraph> <Paragraph position="1"> Axioms placed in the Usable list are: (1) Extended WordNet axioms, (2) NLP axioms, and (3) axioms based on outside world knowledge, such as people and organizations. null Inference rules The inference rule sets are based on hyperresolution and paramodulation. Hyperresolution is an inference rule that does multiple binary resolution steps in one, where binary resolution is an inference mechanism that looks for a positive literal in one clause and negative form of that same literal in another clause such that the two literals can be canceled, resulting in a newly inferred clause.</Paragraph> <Paragraph position="2"> Paramodulation introduces the notion of equality substitution so that axioms representing equality in the proof do not need to be explicitly included in the axiom lists. Additionally, similar to hyperresolution, paramodulation combines multiple substitution steps into one.</Paragraph> <Paragraph position="3"> All modern theorem provers use hyperresolution and paramodulation inference rules since they allow for a more compact and efficient proof by condensing multiple steps into one.</Paragraph> <Paragraph position="4"> COGEX will continue trying to find a proof until the Set of Support becomes empty, a refutation is found, or the proof score drops below a predefined threshold.</Paragraph> <Paragraph position="5"> Two techniques have been implemented in COGEX to deal with incomplete proofs: 1. Count the number of unifications/resolutions with terms in the question along the longest search path in the proof attempts, and 2. Relax the question logic form by incrementally un null coupling arguments in the predicates, and/or removing prepositions or modifiers that are not crucial to the meaning of the text.</Paragraph> <Paragraph position="6"> For example in question, &quot;How far is Yaroslavl from Moscow?&quot; a candidate answer is &quot;.. Yaroslavl, a city 250 miles north of Moscow.&quot; By dropping the from predicate in the question makes the proof succeed for the candidate answer.</Paragraph> </Section> <Section position="8" start_page="0" end_page="0" type="metho"> <SectionTitle> 7 An example </SectionTitle> <Paragraph position="0"> The following example illustrates how all these pieces are put together to generate answer proofs.</Paragraph> <Paragraph position="1"> In particular, a program called Mosaic , developed by the National Center for Supercomputing Applications ( NCSA ) at the University of Illinois at Urbana - Champaign , is gaining popularity as an easy to use point and click interface for searching portions of the Internet. The question contained the verb create while the answer contains the verb develop. In order to prove that this answer is in fact correct, we need to detect and use a lexical chain between develop and create. WordNet supplies us with that chain such that develop a6 make and make a6 create Using WordNet glosses, this chain is transformed into two axioms: Furthermore, the question asks about the Internet browser Mosiac, while the candidate answer refers to Mosaic. To provide the knowledge that the Internet browser Mosaic refers to the same thing as Mosaic, the head of the complex nominal, Internet browser Mosaic, implies its remaining components.</Paragraph> <Paragraph position="2"> (6) all x1 (mosaic nn(x1) a5 internet nn(x1) & browser nn(x1)).</Paragraph> <Paragraph position="3"> (7) all x1 x2 x3 x4 (mosaic nn(x1) & internet nn(x1) &</Paragraph> <Paragraph position="5"> The next step is to build the Set of Support Axiom(s) for the Question. The question is negated to invoke the proof by contradiction -(exists e1 x2 x3 x5 x6 ( organization at(x2) & company nn(x2) & create vb(e1,x2,x6) & internet nn(x3) & browser(x4) & mosaic nn(x5) & nn nnc(x6,x3,x4,x5))).</Paragraph> <Paragraph position="6"> Next, link the answer type term, its modifiers, and any prepositional attachments to the answer type as a substitute for more refined named entity recognition.</Paragraph> <Paragraph position="7"> all x1 ( organization at(x1)a5 company nn(x1)).</Paragraph> <Paragraph position="8"> It remains to create axioms for the ALF of the candidate answer and to start the proof.</Paragraph> <Paragraph position="9"> organization at($c96).</Paragraph> <Paragraph position="10"> 373 [hyper,372,74] company nn($c96).</Paragraph> <Paragraph position="11"> 374 [hyper,373,294,372,356,336,335,299,337] $F.</Paragraph> <Paragraph position="12"> The numbers on the left hand side of the proof summary indicate the step number in the search, not the step number in the proof. Through step 332 we see that COGEX has selected all the axioms it needs to prove that the candidate answer is correct for the question posed to the QA system. Steps 335 through 374 show hyperresolutions that result in all the terms of the question being derived in their positive form so the proof by contradiction succeeds, which is indicated by the $F in the final step and the hyperresolution of all the derived terms with the negated question from step 1 of the proof. The success of this proof boosts the candidate answer to the first position.</Paragraph> <Paragraph position="13"> When the proof fails, we devised a way to incrementally relax some of the conditions that hinder the completion of the proof. This relaxation process puts weights on the proof such that proofs weaker than a predefined threshold are not accepted.</Paragraph> </Section> class="xml-element"></Paper>