File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/92/p92-1028_metho.xml
Size: 20,897 bytes
Last Modified: 2025-10-06 14:13:12
<?xml version="1.0" standalone="yes"?> <Paper uid="P92-1028"> <Title>CORPUS-BASED ACQUISITION OF RELATIVE PRONOUN DISAMBIGUATION HEURISTICS</Title> <Section position="4" start_page="0" end_page="217" type="metho"> <SectionTitle> 2 DISAMBIGUATING RELATIVE PRONOUNS </SectionTitle> <Paragraph position="0"> Accurate disambiguation of relative pronouns is important for any natural language processing system that hopes to process real world texts. It is especially a concern for corpora where the sentences tend to be long and information-packed. Unfortunately, to understand a sentence containing a relative pronoun, an NLP system must solve two difficult problems: the system has to locate the antecedent of the relative pronoun and then determine the antecedent's implicit position in the embedded clause. Although finding the gap in the embedded clause is an equally difficult problem, the work we describe here focuses on locating the relative pronoun antecedent.1 This task may at first seem relatively simple: the antecedent of a relative pronoun is just the most recent constituent that is a human. This is the case for sentences S1-$7 in Figure 1, for example.</Paragraph> <Paragraph position="1"> However, this strategy assumes that the NLP system produces a perfect syntactic and semantic parse of the clause preceding the relative pronoun, including prepositional phrase attachment (e.g., $3, $4, and $7) and interpretation of conjunctions (e.g., $4, $5, and $6) and appositives (e.g., $6). In $5, for example, the antecedent is the entire conjunction of phrases (i.e., &quot;Jim, Terry, and Shawn&quot;), not just the most recent human (i.e., &quot;Shawn&quot;). In $6, either s1. Tony saw the boy who won the award.</Paragraph> <Paragraph position="2"> $2. The boy who gave me the book had red hair.</Paragraph> <Paragraph position="3"> $3. Tony ate dinner with the men from Detroit who sold computers.</Paragraph> <Paragraph position="4"> $4. I spoke to the woman with the black shirt and green hat over in the far comer of the room whc wanted a second interview.</Paragraph> <Paragraph position="5"> SS. I'd like to thank Jim. Terry, and Shawn, who provided the desserts.</Paragraph> <Paragraph position="6"> $6. I'd like to thank our sponsors, GE andNSF, who provide financial support.</Paragraph> <Paragraph position="7"> ST. The woman from Philadelphia who played soccer was my sister.</Paragraph> <Paragraph position="8"> $8. The awards for the children who pass the test are in the drawer.</Paragraph> <Paragraph position="9"> $9. We wondered who stole the watch.</Paragraph> <Paragraph position="10"> S10. We talked with the woman and the man who</Paragraph> <Section position="1" start_page="216" end_page="216" type="sub_section"> <SectionTitle> Pronoun Antecedents </SectionTitle> <Paragraph position="0"> &quot;our sponsors&quot; or its appositive &quot;GE and NSF&quot; is a semantically valid antecedent. Because pp-attachment and interpretation of conjunctions and appositives remain difficult for current systems, it is often unreasonable to expect reliable parser output for clauses containing those constructs.</Paragraph> <Paragraph position="1"> Moreover, the parser must access both syntactic and semantic knowledge in finding the antecedent of a relative pronoun. The syntactic structure of the clause preceding &quot;who&quot; in $7 and $8, for example, is identical (NP-PP) but the antecedent in each case is different. In $7, the antecedent is the subject, &quot;the woman;&quot; in $9, it is the prepositional phrase 1For a solution to the gap-finding problem that is consistent with the simplified parsing strategy presented below, see (Cardie & Lehnert, 1991).</Paragraph> <Paragraph position="2"> modifier, &quot;the children.&quot; Even if we assume a perfect parse, there can be additional complications. In some cases the antecedent is not the most recent constituent, but is a modifier of that constituent (e.g., $8). Sometimes there is no apparent antecedent at all (e.g., $9). Other times the antecedent is truly ambiguous without seeing more of the surrounding context (e.g., S10).</Paragraph> <Paragraph position="3"> As a direct result of these difficulties, NLP system builders have found the manual coding of rules that find relative pronoun antecedents to be very hard. In addition, the resulting heuristics are prone to errors of omission and may not generalize to new contexts.</Paragraph> <Paragraph position="4"> For example, the UMass/MUC-3 system 2 began with 19 rules for finding the antecedents of relative pronouns. These rules included both structural and semantic knowledge and were based on approximately 50 instances of relative pronouns. As counter-examples were identified, new rules were added (approximately 10) and existing rules changed. Over time, however, we became increasingly reluctant to modify the rule set because the global effects of local rule changes were difficult to measure. Moreover, the original rules were based on sentences that UMass/MUC-3 had found to contain important information. As a result, the rules tended to work well for relative pronoun disambiguation in sentences of this class (93% correct for one test set of 50 texts), but did not generalize to sentences outside of the class (78% correct on the same test set of 50 texts).</Paragraph> </Section> <Section position="2" start_page="216" end_page="217" type="sub_section"> <SectionTitle> 2.1 CURRENT APPROACHES </SectionTitle> <Paragraph position="0"> Although descriptions of NLP systems do not usually include the algorithms used to find relative pronoun antecedents, current high-coverage parsers seem to employ one of 3 approaches for relative pronoun disambiguation. Systems that use a formal syntactic grammar often directly encode information for relative pronoun disambiguation in the grammar.</Paragraph> <Paragraph position="1"> Alternatively, a syntactic filter is applied to the parse tree and any noun phrases for which coreference with the relative pronoun is syntactically legal (or, in some cases, illegal) are passed to a semantic component which determines the antecedent using inference or preference rules (see (Correa, 1988), (Hobbs, 1986), (Ingria, & Stallard, 1989), (Lappin, & McCord, 1990)). The third approach employs hand-coded disambiguation heuristics that rely mainly on 2UMass/MUC-3 is a version of the CIRCUS parser (Lehnert, 1990) developed for the MUC-3 performance evaluation. See (Lehnert et. al., 1991) for a description of UMass/MUC-3. MUC-3 is the semantic knowledge but also include syntactic constraints (e.g., UMass/MUC-3).</Paragraph> <Paragraph position="2"> However, there are problems with all 3 approaches in that 1) the grammar must be designed to find relative pronoun antecedents for all possible syntactic contexts; 2) the grammar and/or inference rules require tuning for new corpora; and 3) in most cases, the approach unreasonably assumes a completely correct parse of the clause preceding the relative pronoun. In the remainder of the paper, we present an automated approach for deriving relative pronoun disambigu_a6on rules. This approach avoids the problems associated with the manual encoding of heuristics and grammars and automatically tailors the disambiguation decisions to the syntactic and semantic profile of the corpus. Moreover, the technique requires only a very simple parser because input to the clustering system that creates the disambiguation heuristics presumes neither pp-attachment nor interpretation of conjunctions and appositives.</Paragraph> </Section> </Section> <Section position="5" start_page="217" end_page="217" type="metho"> <SectionTitle> 3 AN AUTOMATED APPROACH </SectionTitle> <Paragraph position="0"> Our method for deriving relative pronoun disambiguation heuristics consists of the following steps: 1. Select from a subset of the corpus all sentences containing a particular relative pronoun. (For the remainder of the paper, we will focus on the relative pronoun &quot;who.&quot;) 2. For each instance of the relative pronoun in the selected sentences, a. parse the portion of the sentence that precedes it into low-level syntactic constituents b. use the results of the parse to create a training instance that represents the disambiguation decision for this occurrence of the relative pronoun.</Paragraph> <Paragraph position="1"> 3. Provide the training instances as input to an existing conceptual clustering system.</Paragraph> <Paragraph position="2"> During the training phase outlined above, the clustering system creates a hierarchy of relative pronoun disambiguation decisions that replace the hand-coded heuristics. Then, for each new occurrence of the wh-word encountered after training, we retrieve the most similar disambiguation decision from the hierarchy using a representation of the clause preceding the wh-word as the probe. Finally, the antecedent of the retrieved decision guides the selection of the antecedent for the new occurrence of the relative pronoun. Each step of the training and testing phases will be explained further in the sections that follow.</Paragraph> <Section position="1" start_page="217" end_page="217" type="sub_section"> <SectionTitle> 3.1 SELECTING SENTENCES </SectionTitle> <Paragraph position="0"/> </Section> </Section> <Section position="6" start_page="217" end_page="218" type="metho"> <SectionTitle> FROM THE CORPUS </SectionTitle> <Paragraph position="0"> For the relative pronoun disambiguation task, we used the MUC-3 corpus of 1500 articles that range from a single paragraph to over one page in length.</Paragraph> <Paragraph position="1"> In theory, each article describes one or more terrorist incidents in Latin America. In practice, however, about half of the texts are actually irrelevant to the MUC task. The MUC-3 articles consist of a variety of text types including newspaper articles, TV news reports, radio broadcasts, rebel communiques, speeches, and interviews. The corpus is relatively small - it contains approximately 450,000 words and 18,750 sentences. In comparison, most corpus-based algorithms employ substantially larger corpora (e.g., 1 million words (de Marcken, 1990), 2.5 million words (Brent, 1991), 6 million words (Hindle, 1990), 13 million words (Hindle, & Rooth, 1991)).</Paragraph> <Paragraph position="2"> Relative pronoun processing is especially important for the MUC-3 corpus because approximately 25% of the sentences contain at least one relative pronoun. 3 In fact, the relative pronoun &quot;who&quot; occurs in approximately 1 out of every 10 sentences. In the experiment described below, we use 100 texts containing 176 instances of the relative pronoun &quot;who&quot; for training. To extract sentences containing a specific relative pronoun, we simply search the selected articles for instances of the relative pronoun and use a preprocessor to locate sentence boundaries.</Paragraph> <Section position="1" start_page="217" end_page="218" type="sub_section"> <SectionTitle> 3.2 PARSING REQUIREMENTS </SectionTitle> <Paragraph position="0"> Next, UMass/MUC-3 parses each of the selected sentences. Whenever the relative pronoun &quot;who&quot; is recognized, the syntactic analyzer returns a list of the low-level constituents of the preceding clause prior to any attachment decisions (see Figure 2).</Paragraph> <Paragraph position="1"> UMass/MUC-3 has a simple, deterministic, stackoriented syntactic analyzer based on the McEli parser (Schank, & Riesbeck, 1981). It employs lexicallyindexed local syntactic knowledge to segment incoming text into noun phrases, prepositional phrases, and verb phrases, ignoring all unexpected constructs and unknown words. 4 Each constituent syntactic classes, only noun phrases, prepositional phrases, and verb phrases become part of the training instance.</Paragraph> <Paragraph position="2"> Sources in downtown Lima report that the police last night detained Juan returned by the parser (except the verb) is tagged with the semantic classification that best describes the phrase's head noun. For the MUC-3 corpus, we use a set of 7 semantic features to categorize each noun in the lexicon: human, proper-name, location, entity, physical-target, organization, and weapon. In addition, clause boundaries are detected using a method described in (Cardie, & Lehnert, 1991).</Paragraph> <Paragraph position="3"> It should be noted that all difficult parsing decisions are delayed for subsequent processing components. For the task of relative pronoun disambiguation, this means that the conceptual clustering system, not the parser, is responsible for recognizing all phrases that comprise a conjunction of antecedents and for specifying at least one of the semantically valid antecedents in the case of appositives. In addition, pp-attachment is more easily postponed until after the relative pronoun antecedent has been located. Consider the sentence &quot;I ate with the men from the restaurant in the club.&quot; Depending on the context, &quot;in the club&quot; modifies either &quot;ate&quot; or &quot;the restaurant.&quot; If we know that &quot;the men&quot; is the antecedent of a relative pronoun, however (e.g., &quot;I ate with the men from the restaurant in the club, who offered me the job&quot;), it is probably the case that &quot;in the club&quot; modifies &quot;the men.&quot; Finally, because the MUC-3 domain is sufficiently narrow in scope, lexical disambiguation problems are infrequent. Given this rather simplistic view of syntax, we have found that a small set of syntactic predictions covers the wide variety of constructs in the MUC-3 corpus.</Paragraph> </Section> </Section> <Section position="7" start_page="218" end_page="219" type="metho"> <SectionTitle> 3.3 CREATING THE TRAINING INSTANCES </SectionTitle> <Paragraph position="0"> Output from the syntactic analyzer is used to generate a training instance for each occurrence of the relative pronoun in the selected sentences. A training instance represents a single disambiguation decision and includes one attribute-value pair for every low-level syntactic constituent in the preceding clause.</Paragraph> <Paragraph position="1"> The attributes of a training instance describe the syntactic class of the constituent as well as its position with respect to the relative pronoun. The value associated with an attribute is the semantic feature of the phrase's head noun. (For verb phrases, we currently note only their presence or absence using the values t and nil, respectively.) Consider the training instances in Figure 3. In S 1, for example, &quot;of the 76th district court&quot; is represented with the attribute ppl because it is a prepositional phrase and is in the first position to the left of &quot;who.&quot; Its value is &quot;physical-target&quot; because &quot;court&quot; is classified as a physical-target in the lexicon. The subject and verb constituents (e.g., &quot;her DAS bodyguard&quot; in $3 and &quot;detained&quot; in $2) retain their traditional s and v labels, however -- no positional information is included for those attributes.</Paragraph> <Paragraph position="2"> In addition to the constituent attribute-value pairs, a training instance contains an attribute-value pair that represents the correct antecedent. As shown in Figure 3, the value of the antecedent attribute is a list of the syntactic constituents that contain the antecedent (or (none) if the relative pronoun has no anteceden0. In S 1, for example, the antecedent of &quot;who&quot; is &quot;the judge.&quot; Because this phrase is located in the subject position, the value of the antecedent attribute is (s). Sometimes, however, the antecedent is actually a conjunction of phrases. In these cases, we represent the antecedent as a list of the constituents associated with each element of the conjunction. Look, for example, at the antecedent in $2. Because &quot;who&quot; refers to the conjunction &quot;Juan Bautista and Rogoberto Matute,&quot; and because those phrases occur as rip1 and rip2, the value of the antecedent attribute is (np2 npl). $3 shows yet another variation of the antecedent attribute-value pair. In this example, an appositive creates three equivalent antecedents: 1) &quot;Dagoberto Rodriguez&quot; (rip1), 2) &quot;her DAS bodyguard&quot; m (s), and 3) &quot;her DAS bodyguard, Dagoberto Rodriguez&quot; -- (s npl).</Paragraph> <Paragraph position="3"> UMass/MUC-3 automatically generates the training instances as a side effect of parsing. Only the desired antecedent is specified by a human supervisor via a menu-driven interface that displays the antecedent options.</Paragraph> </Section> <Section position="8" start_page="219" end_page="220" type="metho"> <SectionTitle> 3.4 BUILDING THE HIERARCHY OF DISAMBIGUATION HEURISTICS </SectionTitle> <Paragraph position="0"> As the training instances become available they are input to an existing conceptual clustering system called COBWEB (Fisher, 1987). 5 COBWEB employs an evaluation metric called category utility (Gluck, & Corter, 1985) to incrementally discover a classification hierarchy that covers the training instances. 6 It is this hierarchy that replaces the hand-coded disambiguation heuristics. While the details of COBWEB are not necessary, it is important to know that nodes in the hierarchy represent concepts that increase in generality as they approach the root of the tree. Given a new instance to classify, COBWEB appropriate classes as well as the the concepts for each class when given a set of examples that have not been preclassified by a teacher. Our unorthodox use of COBWEB to perform supervised learning is prompted by plans to use the resulting hierarchy for tasks other than relative pronoun disambiguation.</Paragraph> <Paragraph position="1"> retrieves the most specific concept that adequately describes the instance.</Paragraph> </Section> <Section position="9" start_page="220" end_page="220" type="metho"> <SectionTitle> 3.5 USING THE DISAMBIGUATION HEURISTICS HIERARCHY </SectionTitle> <Paragraph position="0"> After training, the resulting hierarchy of relative pronoun disambiguation decisions supplies the antecedent of the wh-word in new contexts. Given a novel sentence containing &quot;who,&quot; UMass/MUC-3 generates a set of attribute-value pairs that represent the clause preceding the wh-word. This probe is just a training instance without the antecedent attribute-value pair. Given the probe, COBWEB retrieves from the hierarchy the individual instance or abstract class that is most similar and the antecedent of the retrieved example guides selection of the antecedent for the novel case. We currently use the following selection heuristics to 1) choose an antex~ent for the novel sentence that is consistent with the context of the probe; or to 2) modify the retrieved antecedent so that it is applicable in the current context: 1. Choose the first option whose constituents are all present in the probe.</Paragraph> <Paragraph position="1"> 2. Otherwise, choose the first option that contains at least one constituent present in the probe and ignore those constituents in the retrieved antex~ent that are missing from the probe.</Paragraph> <Paragraph position="2"> 3. Otherwise, replace the np constituents in the retrieved antecedent that are missing from the probe with pp constituents (and vice versa), and try 1 and 2 again.</Paragraph> <Paragraph position="3"> In S 1 of Figure 4, for example, the first selection heuristic applies. The retrieved instance specifies the np2 constituent as the location of the antecedent and the probe has rip2 as one of its constituents.</Paragraph> <Paragraph position="4"> Therefore, UMass/MUC-3 infers that the antecedent of &quot;who&quot; for the current sentence is &quot;the hardliners,&quot; i.e., the contents of the np2 syntactic constituent. In $2, however, the retrieved concept specifies an antecedent from five constituents, only two of which are actually present in the probe. Therefore, we ignore the missing constituents pp5, rip4, and pp3, and look to just np2 and rip1 for the antecedent. For $3, selection heuristics 1 and 2 fail because the probe contains no pp2 constituent. However, if we replace pp2 with np2 in the retrieved antecedent, then heuristic 1 applies and &quot;a specialist&quot; is chosen as the antecedent.</Paragraph> <Paragraph position="5"> Sl: \[It\] \[encourages\] \[the military men\] \[,\] \[and\] \[the hardliners\] \[in ARENA\] who...</Paragraph> </Section> class="xml-element"></Paper>