<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-1023">
  <Title>A Question Answering System Supported by Information Extraction*</Title>
  <Section position="2" start_page="0" end_page="166" type="metho">
    <SectionTitle>
QA Track Specifications (www.research.att.com/
</SectionTitle>
    <Paragraph position="0"> -singhal/qa-track-spec.txt) in the TREC community illustrates this point.</Paragraph>
    <Paragraph position="1"> Current information retrieval systems allow us to locate documents that might contain the pertinent information, but most of them leave it to the user to extract the useful information from a ranked list. This leaves the (often unwilling) user with a relatively large amount of text to consume. There is an urgent need for tools that would reduce the amount of text one might have to read in order to obtain the desired information. This track aims at doing exactly that for a special (and popular) class of information seeking behavior: QUESTION ANSWERING. People have questions and they need answers, not documents. Automatic question answering will definitely be a significant advance in the state-of-art information retrieval technology. Kupiec (1993) presented a QA system MURAX using an on-line encyclopedia. This system used the technology of robust shallow parsing but suffered from the lack of basic information extraction support. In fact, the most siginifcant IE advance, namely the NE (Named Entity) technology, occured after Kupiec (1993), thanks to the MUC program (MUC-7 1998).</Paragraph>
    <Paragraph position="2"> High-level IE technology beyond NE has not been in the stage of possible application until recently.</Paragraph>
    <Paragraph position="3"> AskJeeves launched a QA portal (www.askjeeves.com). It is equipped with a fairly sophisticated natural language question parser, but it does not provide direct answers to the asked questions. Instead, it directs the user to the relevant web pages, just as the traditional search engine does. In this sense, AskJeeves has only done half of the job for QA.</Paragraph>
    <Paragraph position="4"> We believe that QA is an ideal test bed for demonstrating the power of IE. There is a natural co-operation between IE and IR; we regard QA as one major intelligence which IE can offer IR. * This work was supported in part by the SBIR grants F30602-98-C-0043 and F30602-99-C-0102 from Air Force Research Laboratory (AFRL)/IFED.</Paragraph>
    <Paragraph position="5">  An important question then is, what type of IE can support IR in QA and how well does it support it? This forms the major topic of this paper. We structure the remaining part of the paper as follows. In Section 1, we first give an overview of the underlying IE technology which our organization has been developing. Section 2 discusses the QA system. Section 3 describes the limitation of the current system. Finally, in Section 4, we propose a more sophisticated QA system supported by three levels of IE.</Paragraph>
  </Section>
  <Section position="3" start_page="166" end_page="167" type="metho">
    <SectionTitle>
1 Overview of Textract IE
</SectionTitle>
    <Paragraph position="0"> The last decade has seen great advance and interest in the area of IE. In the US, the DARPA sponsored Tipster Text Program \[Grishman 1997\] and the Message Understanding Conferences (MUC) \[MUC-7 1998\] have been the driving force for developing this technology. In fact, the MUC specifications for various IE tasks have become de facto standards in the IE research community. It is therefore necessary to present our IE effort in the context of the MUC program.</Paragraph>
    <Paragraph position="1"> MUC divides IE into distinct tasks, namely, NE (Named Entity), TE (Template Element), TR (Template Relation), CO (Co-reference), and ST (Scenario Templates) \[Chinchor &amp; Marsh 1998\]. Our proposal for three levels of IE is modelled after the MUC standards using MUC-style representation.</Paragraph>
    <Paragraph position="2"> However, we have modified the MUC IE task definitions in order to make them more useful and more practical. More precisely, we propose a hierarchical, 3-level architecture for developing a kernel IE system which is domain-independent throughout.</Paragraph>
    <Paragraph position="3"> The core of this system is a state-of-the-art NE tagger \[Srihari 1998\], named Textract 1.0. The Textract NE tagger has achieved speed and accuracy comparable to that of the few deployed NE systems, such as NetOwl \[Krupka &amp; Hausman 1998\] and Nymble \[Bikel et al 1997\]. It is to be noted that in our definition of NE, we significantly expanded the type of information to be extracted. In addition to all the MUC defined NE types (person, organization, location, time, date, money and percent), the following types/sub-types of information are also identified by the TextractNE module:  government agency, school (belonging to the type organization) and military person, religious person (belonging to person) are also identified. These new sub-types provide a better foundation for defining multiple relationships between the identified entities and for supporting question answering functionality. For example, the key to a question processor is to identify the asking point (who, what, when, where, etc.). In many cases, the asking point corresponds to an NE beyond the MUC definition, e.g. the how+adjective questions: how long (duration or length), how far (length), how often (frequency), how old (age), etc.</Paragraph>
    <Paragraph position="4"> Level-2 IE, or CE (Correlated Entity), is concerned with extracting pre-defined multiple relationships between the entities. Consider the person entity as an example; the TextractCE prototype is capable of extracting the key relationships such as age, gender, affiliation, position, birthtime, birth__place, spouse, parents, children, where.from, address, phone, fax, email, descriptors. As seen, the information in the CE represents a mini-CV or profile of the entity. In general, the CE template integrates and greatly enriches the information contained in MUC TE and TR.</Paragraph>
    <Paragraph position="5"> The final goal of our IE effort is to further extract open-ended general events (GE, or level 3 IE) for information like who did what (to whom) when (or how often) and where. By general events, we refer to argument structures centering around verb notions plus the associated information of time/frequency and location. We show an example of our defined GE extracted from the text below: Julian Hill, a research chemist whose accidental discovery of a tough, taffylike compound revolutionized everyday life after it proved its worth in warfare and courtship,  died on Sunday in Hockessin, Del.</Paragraph>
    <Paragraph position="7"> The core of the system consists of three kernel IE modules and six linguistic modules.</Paragraph>
    <Paragraph position="8"> The multi-level linguistic modules serve as an underlying support system for different levels of IE. The IE results are stored in a database which is the basis for IE-related applications like QA, BR (Browsing, threading and visualization) and AS (Automatic Summarization). The approach to IE taken here, consists of a unique blend of machine learning and FST (finite state transducer) rule-based system \[Roche &amp; Schabes 1997\]. By combining machine learning with an FST rule-based system, we are able to exploit the best of both paradigms while overcoming their respective weaknesses \[Srihari 1998, Li &amp; Srihari 2000\].</Paragraph>
  </Section>
  <Section position="4" start_page="167" end_page="168" type="metho">
    <SectionTitle>
2 NE-Supported QA
</SectionTitle>
    <Paragraph position="0"> This section presents the QA system based on Named Entity tagging. Out of the 200 questions that comprised the TREC-8 QA track competition, over 80% asked for an NE, e.g. who (PERSON), when (TIME\[ DATE), where (LOCATION), how far (LENGTH). Therefore, the NE tagger has been proven to be very helpful. Of course, the NE of the targeted type is only necessary but not complete in answering such questions because NE by nature only extracts isolated individual entities from the text.</Paragraph>
    <Paragraph position="1"> Nevertheless, using even crude methods like &amp;quot;the nearest NE to the queried key words&amp;quot; or &amp;quot;the NE and its related key words within the same line (or same paragraph, etc.)&amp;quot;, in most cases, the QA system was able to extract text portions which contained answers in the top five list.</Paragraph>
    <Paragraph position="2"> Figure 2 illustrates the system design of TextractQA Prototype. There are two components for the QA prototype: Question Processor and Text Processor. The Text Matcher module links the two processing results and tries to find answers to the processed question.</Paragraph>
    <Paragraph position="3"> Matching is based on keywords, plus the NE type and their common location within a same sentence.</Paragraph>
    <Paragraph position="5"> The general algorithm for question answering is as follows:</Paragraph>
    <Section position="1" start_page="168" end_page="168" type="sub_section">
      <SectionTitle>
2.1 Question Processing
</SectionTitle>
      <Paragraph position="0"> The Question Processing results are a list of keywords plus the information for asking point.</Paragraph>
      <Paragraph position="1"> For example, the question: \[2\] Who won the 1998 Nobel Peace Prize? contains the following keywords: won, 1998, Nobel, Peace, Prize. The asking point Who refers to the NE type person. The output before question expansion is a simple 2-feature template as shown below:</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="168" end_page="168" type="metho">
    <SectionTitle>
\[3\] asking_point: PERSON
</SectionTitle>
    <Paragraph position="0"> FBI, word, processor } The question processor scans the question to search for question words (wh-words) and maps them into corresponding NE types/sub-types or pre-defined notions like REASON.</Paragraph>
    <Paragraph position="1"> We adopt two sets of pattern matching rules for this purpose: (i) structure based pattern matching rules; (ii) simple key word based pattern matching rules (regarded as default rules). It is fairly easy to exhaust the second set of rules as interrogative question words/phrases form a closed set. In comparison, the development of the first set of rules are continuously being fine-tuned and expanded. This strategy of using two set of rules leads to the robustness of the question processor.</Paragraph>
    <Paragraph position="2"> The first set of rules are based on shallow parsing results of the questions, using Cymfony FST based Shallow Parser. This parser identifies basic syntactic constructions like BaseNP (Basic  Rule \[6\] checks the head word of the NP. It covers cases like VG\[Name\] NP\[a country\] that VG\[is developing\] NP\[a magnetic levitation railway system\]. Rule \[7\] works for cases like VG\[Name\] NP\[the first private citizen\] VG\[to fly\] PP\[in space\] as citizen belongs to the word class person_w. Rule \[9\] is a catch-all rule: if the NP is not of class person (person_w) or organization (org_w), then the asking point is a proper name (default NE), often realized in English in capitalized string of words. Examples include Name a film that has won the Golden Bear in the Berlin Film Festival.</Paragraph>
    <Paragraph position="3"> The word lists org_w and person_w are currently manually maintained based on inspection of large volumes of text. An effort is underway to automate the learning of such word lists by utilizing machine learning techniques. We used the following pattern transformations to expand our ruleset:  Rule \[10\] extracts the asking point from cases like NP\[What\] Aux\[is\] NP\[the largest country\] PP\[in the world\]. Rule \[11\] covers the following questions: NP\[What costume designer\] VG\[decided\] that NP\[Michael  As seen, shallow parsing helps us to capture a variety of natural language question expressions. However, there are cases where some simple key word based pattern matching would be enough to capture the asking point. That is our second set of rules. These rules are used when the first set of rules has failed to produce results. The following is a sample of such rules:  In the stage of question expansion, the template in \[4\] would be expanded to the template shown in \[29\]: \[29\] asking_point: key_word: {because{because of\] due to{thanks to{since I in order{to VB} { asklaskslasked\[asking, David,Koresh,FBI, word, processor} The last item in the asking._point list attempts to find an infinitive by checking the word to followed by a verb (with the part-of-speech tag VB). As we know, infinitive verb phrases are often used in English to explain a reason for some action.</Paragraph>
    <Section position="1" start_page="168" end_page="168" type="sub_section">
      <SectionTitle>
2.2 Text Processing
</SectionTitle>
      <Paragraph position="0"> On the text processing side, we first send the question directly to a search engine in order to narrow down the document pool to the first n, say 200, documents for IE processing. Currently, this includes tokenization, POS tagging and NE tagging. Future plans include several levels of parsing as well; these are required to support CE and GE extraction. It should be noted that all these operations are extremely robust and fast, features necessary for large volume text indexing. Parsing is accomplished through cascaded finite state transducer grammars.</Paragraph>
    </Section>
    <Section position="2" start_page="168" end_page="168" type="sub_section">
      <SectionTitle>
2.3 Text Matching
</SectionTitle>
      <Paragraph position="0"> The Text Matcher attempts to match the question template with the processed documents for both the asking point and the key words. There is a preliminary ranking standard built-in the matcher in order to find the most probable answers. The primary rank is a count of how many unique keywords are contained within a sentence. The secondary ranking is based on the order that the keywords appear in the sentence compared to their order in the question. The third ranking is based on whether there is an exact match or a variant match for the key verb.</Paragraph>
      <Paragraph position="1"> In the TREC-8 QA track competition, Cymfony QA accuracy was 66.0%. Considering we have only used NE technology to support QA in this run, 66.0% is a very encouraging result.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>