<?xml version="1.0" standalone="yes"?>
<Paper uid="P91-1030">
  <Title>STRUCTURAL AMBIGUITY AND LEXICAL RELATIONS</Title>
  <Section position="1" start_page="0" end_page="235" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> We propose that ambiguous prepositional phrase attachment can be resolved on the basis of the relative strength of association of the preposition with noun and verb, estimated on the basis of word distribution in a large corpus. This work suggests that a distributional approach can be effective in resolving parsing problems that apparently call for complex reasoning.</Paragraph>
    <Paragraph position="1"> Introduction Prepositional phrase attachment is the canonical case of structural ambiguity, as in the time worn example, (1) I saw the man with the telescope The existence of such ambiguity raises problems for understanding and for language models. It looks like it might require extremely complex computation to determine what attaches to what. Indeed, one recent proposal suggests that resolving attachment ambiguity requires the construction of a discourse model in which the entities referred to in a text must be reasoned about (Altmann and Steedman 1988). Of course, if attachment ambiguity demands reference to semantics and discourse models, there is little hope in the near term of building computational models for unrestricted text to resolve the ambiguity.</Paragraph>
    <Paragraph position="2"> Structure based ambiguity resolution There have been several structure-based proposals about ambiguity resolution in the literature; they are particularly attractive because they are simple and don't demand calculations in the semantic or discourse domains. The two main ones are: * Right Association - a constituent tends to attach to another constituent immediately to its right (Kimball 1973).</Paragraph>
    <Paragraph position="3"> * Minimal Attachment - a constituent tends to attach so as to involve the fewest additional syntactic nodes (Frazier 1978).</Paragraph>
    <Paragraph position="4"> For the particular case we are concerned with, attachment of a prepositional phrase in a verb + object context as in sentence (1), these two principles - at least in the version of syntax that Frazier assumes - make opposite predictions: Right Association predicts noun attachment, while Minimal Attachment predicts verb attachment.</Paragraph>
    <Paragraph position="5"> Psycholinguistic work on structure-based strategies is primarily concerned with modeling the time course of parsing and disambiguation, and proponents of this approach explicitly acknowledge that other information enters into determining a final parse. Still, one can ask what information is relevant to determining a final parse, and it seems that in this domain structure-based disambiguation is not a very good predictor. A recent study of attachment of prepositional phrases in a sample of written responses to a &amp;quot;Wizard of Oz&amp;quot; travel information experiment shows that neither Right Association nor Minimal Attachment account for more than 55% of the cases (Whittemore et al.</Paragraph>
    <Paragraph position="6"> 1990). And experiments by Taraban and McClelland (1988) show that the structural models are not in fact good predictors of people's behavior in resolving ambiguity.</Paragraph>
    <Paragraph position="7"> Resolving ambiguity through lexical associations Whittemore et al. (1990) found lexical preferences to be the key to resolving attachment ambiguity.</Paragraph>
    <Paragraph position="8"> Similarly, Taraban and McClelland found lexical content was key in explaining people's behavior.</Paragraph>
    <Paragraph position="9"> Various previous proposals for guiding attachment disambiguation by the lexical content of specific  words have appeared (e.g. Ford, Bresnan, and Kaplan 1982; Marcus 1980). Unfortunately, it is not clear where the necessary information about lexical preferences is to be found. In the Whittemore et al. study, the judgement of attachment preferences had to be made by hand for exactly the cases that their study covered; no precompiled list of lexical preferences was available. Thus, we are posed with the problem: how can we get a good list of lexical preferences.</Paragraph>
    <Paragraph position="10"> Our proposal is to use cooccurrence of with prepositions in text as an indicator of lexical preference. Thus, for example, the preposition to occurs frequently in the context send NP --, i.e., after the object of the verb send, and this is evidence of a lexical association of the verb send with to. Similarly, from occurs frequently in the context withdrawal --, and this is evidence of a lexical association of the noun withdrawal with the preposition from. Of course, this kind of association is, unlike lexical selection, a symmetric notion.</Paragraph>
    <Paragraph position="11"> Cooccurrence provides no indication of whether the verb is selecting the preposition or vice versa. We will treat the association as a property of the pair of words. It is a separate matter, which we unfortunately cannot pursue here, to assign the association to a particular linguistic licensing relation. The suggestion which we want to explore is that the association revealed by textual distribution - whether its source is a complementation relation, a modification relation, or something else - gives us information needed to resolve the prepositional attachment.</Paragraph>
    <Section position="1" start_page="229" end_page="231" type="sub_section">
      <SectionTitle>
Discovering Lexical Associa-
</SectionTitle>
      <Paragraph position="0"> tion in Text A 13 million word sample of Associated Press new stories from 1989 were automatically parsed by the Fidditch parser (Hindle 1983), using Church's part of speech analyzer as a preprocessor (Church 1988). From the syntactic analysis provided by the parser for each sentence, we extracted a table containing all the heads of all noun phrases. For each noun phrase head, we recorded the following preposition if any occurred (ignoring whether or not the parser attached the preposition to the noun phrase), and the preceding verb if the noun phrase was the object of that verb. Thus, we generated a table with entries including those shown in Table 1.</Paragraph>
      <Paragraph position="1"> In Table 1, example (a) represents a passivized instance of the verb blame followed by the prepo- null phrase whose head is money; this noun phrase is not an object of any verb, but is followed by the preposition for. Example (c) represents an instance of a noun phrase with head noun development which neither has a following preposition nor is the object of a verb. Example (d) is an instance of a noun phrase with head government, which is the object of the verb control but is followed by no preposition. Example (j) represents an instance of the ambiguity we are concerned with resolving: a noun phrase (head is concession), which is the object of a verb (grant), followed by a preposition (to).</Paragraph>
      <Paragraph position="2"> From the 13 million word sample, 2,661,872 noun phrases were identified. Of these, 467,920 were recognized as the object of a verb, and 753,843 were followed by a preposition. Of the noun phrase objects identified, 223,666 were ambiguous verb-noun-preposition triples.</Paragraph>
      <Paragraph position="3"> Estimating attachment preferences null Of course, the table of verbs, nouns and prepositions does not directly tell us what the strength lexical associations are. There are three potential sources of noise in the model. First, the parser in some cases gives us false analyses. Second, when a preposition follows a noun phrase (or verb), it may or may not be structurally related to that noun phrase (or verb). (In our terms, it may attach to that noun phrase or it may attach somewhere else). And finally, even if we get accurate attachment information, it may be that fre- null quency of cooccurrence is not a good indication of strength of attachment. We will proceed to build the model of lexical association strength, aware of these sources of noise.</Paragraph>
      <Paragraph position="4"> We want to use the verb-noun-preposition table to derive a table of bigrams, where the first term is a noun or verb, and the second term is an associated preposition (or no preposition). To do this we need to try to assign each preposition that occurs either to the noun or to the verb that it occurs with. In some cases it is fairly certain that the preposition attaches to the noun or the verb; in other cases, it is far less certain. Our approach is to assign the clear cases first, then to use these to decide the unclear cases that can be decided, and finally to arbitrarily assign the remaining cases. The procedure for assigning prepositions in our  sample to noun or verb is as follows: 1. No Preposition - if there is no preposition, the noun or verb is simply counted with the null preposition. (cases (c-h) in Table 1).</Paragraph>
      <Paragraph position="5"> 2. Sure Verb Attach 1 - preposition is attached to the verb if the noun phrase head is a pronoun. (i in Table 1) 3. Sure Verb Attach 2 - preposition is attached to the verb if the verb is passivized (unless the preposition is by. The instances of by following a passive verb were left unassigned.) (a in Table 1) 4. Sure Noun Attach - preposition is attached to the noun, if the noun phrase occurs in a context where no verb could license the prepositional phrase (i.e., the noun phrase is in sub-ject or pre-verbal position.) (b, if pre-verbal) 5. Ambiguous Attach 1 - Using the table of attachment so far, if a t-score for the ambiguity (see below) is greater than 2.1 or less than -2.1, then assign the preposition according to the t-score. Iterate through the ambiguous triples until all such attachments are done. (j and k may be assigned) 6. Ambiguous Attach 2 - for the remaining ambiguous triples, split the attachment between the noun and the verb, assigning .5 to the noun and .5 to the verb. (j and k may be assigned) 7. Unsure Attach - for the remaining pairs (all  of which are either attached to the preceding noun or to some unknown element), assign them to the noun. (b, if following a verb) This procedure gives us a table of bigrams representing our guess about what prepositions associate with what nouns or verbs, made on the basis of the distribution of verbs nouns and prepositions in our corpus.</Paragraph>
      <Paragraph position="6"> The procedure for guessing attachment null Given the table of bigrams, derived as described above, we can define a simple procedure for determining the attachment for an instance of verb-noun-preposition ambiguity. Consider the example of sentence (2), where we have to choose the attachment given verb send, noun soldier, and preposition into.</Paragraph>
      <Paragraph position="7"> (2) Moscow sent more than 100,000 soldiers into Afganistan ...</Paragraph>
      <Paragraph position="8"> The idea is to contrast the probability with which into occurs with the noun soldier (P(into \[ soldier)) with the probability with which into occurs with the verb send (P(into \[ send)). A t-score is an appropriate way to make this contrast (see Church et al. to appear). In general, we want to calculate the contrast between the conditional probability of seeing a particular preposition given a noun with the conditional probability of seeing that preposition given a verb.</Paragraph>
      <Paragraph position="10"> We use the &amp;quot;Expected Likelihood Estimate&amp;quot; (Church et al., to appear) to estimate the probabilities, in order to adjust for small frequencies; that is, given a noun and verb, we simply add 1/2 to all bigram frequency counts involving a preposition that occurs with either the noun or the verb, and then recompute the unigram frequencies. This method leaves the order of t-scores nearly intact, though their magnitude is inflated by about 30%.</Paragraph>
      <Paragraph position="11"> To compensate for this, the 1.65 threshold for significance at the 95% level should be adjusted up to about 2.15.</Paragraph>
      <Paragraph position="12"> Consider how we determine attachment for sentence (2). We use a t-score derived from the adjusted frequencies in our corpus to decide whether the prepositional phrase into Afganistan is attached to the verb (root) send/V or to the noun (root) soldier/N. In our corpus, soldier/N has an adjusted frequency of 1488.5, and send/V has an adjusted frequency of 1706.5; soldier/N occurred in 32 distinct preposition contexts, and send/Via  60 distinct preposition contexts; f(send/V into) = 84, f(soidier/N into) = 1.5.</Paragraph>
      <Paragraph position="13"> From this we calculate the t-score as follows: 1</Paragraph>
      <Paragraph position="15"> This figure of-8.81 represents a significant association of the preposition into with the verb send, and on this basis, the procedure would (correctly) decide that into should attach to send rather than to soldier. Of the 84 send/V into bigrams, 10 were assigned by steps 2 and 3 ('sure attachements').</Paragraph>
    </Section>
    <Section position="2" start_page="231" end_page="232" type="sub_section">
      <SectionTitle>
Testing Attachment Prefer-
</SectionTitle>
      <Paragraph position="0"> ence To evaluate the performance of this procedure, first the two authors graded a set of verb-noun-preposition triples as follows. From the AP new stories, we randomly selected 1000 test sentences in which the parser identified an ambiguous verb-noun-preposition triple. (These sentences were selected from stories included in the 13 million word sample, but the particular sentences were excluded from the calculation of lexical associations.) For every such triple, each author made a judgement of the correct attachment on the basis of the three words alone (forced choice - preposition attaches to noun or verb). This task is in essence the one that we will give the computer - i.e., to judge the attachment without any more information than the preposition and the head of the two possible attachment sites, the noun and the verb. This gave us two sets of judgements to compare the algorithm's performance to.</Paragraph>
      <Paragraph position="1"> a V is the number of distinct preposition contexts for either soldier/N or send/V; in this c~se V = 70. Since 70 bigram frequencies f(soldier/N p) are incremented by 1/2, the unigram frequency for soldier/N is incremented by 70/2.</Paragraph>
      <Paragraph position="2"> Judging correct attachment We also wanted a standard of correctness for these test sentences. To derive this standard, we together judged the attachment for the 1000 triples a second time, this time using the full sentence context.</Paragraph>
      <Paragraph position="3"> It turned out to be a surprisingly difficult task to assign attachment preferences for the test sample. Of course, many decisions were straightforward; sometimes it is clear that a prepositional phrase is and argument of a noun or verb. But more than 10% of the sentences seemed problematic to at least one author. There are several kinds of constructions where the attachment decision is not clear theoretically. These include idioms (3-4), light verb constructions (5), small clauses (6).</Paragraph>
      <Paragraph position="4">  (3) But over time, misery has given way to mending.</Paragraph>
      <Paragraph position="5"> (4) The meeting will take place in Quanrico null (5) Bush has said he would not make cuts in Social Security (6) Sides said Francke kept a .38-caliber  revolver in his car's glove compartment We chose always to assign light verb constructions to noun attachment and small clauses to verb attachment.</Paragraph>
      <Paragraph position="6"> Another source of difficulty arose from cases where there seemed to be a systematic ambiguity in attachment.</Paragraph>
      <Paragraph position="7">  (7) ...known to frequent the same bars in one neighborhood.</Paragraph>
      <Paragraph position="8"> (8) Inaugural officials reportedly were trying to arrange a reunion for Bush and his old submarine buddies ...</Paragraph>
      <Paragraph position="9"> (9) We have not signed a settlement  agreement with them Sentence (7) shows a systematic locative ambiguity: if you frequent a bar and the bar is in a place, the frequenting event is arguably in the same place. Sentence (8) shows a systematic benefactive ambiguity: if you arrange something for someone, then the thing arranged is also for them. The ambiguity in (9) arises from the fact that if someone is one of the joint agents in the signing of an agreement, that person is likely to be a party to the agreement. In general, we call an attachment systematically ambiguous when, given our understanding of the semantics, situations which  make the interpretation of one of the attachments true always (or at least usually) also validate the interpretation of the other attachment.</Paragraph>
      <Paragraph position="10"> It seems to us that this difficulty in assigning attachment decisions is an important fact that deserves further exploration. If it is difficult to decide what licenses a prepositional phrase a significant proportion of the time, then we need to develop language models that appropriately capture this vagueness. For our present purpose, we decided to force an attachment choice in all cases, in some cases making the choice on the bases of an unanalyzed intuition.</Paragraph>
      <Paragraph position="11"> In addition to the problematic cases, a significant number (120) of the 1000 triples identified automatically as instances of the verb-objectpreposition configuration turned out in fact to be other constructions. These misidentifications were mostly due to parsing errors, and in part due to our underspecifying for the parser exactly what configuration to identify. Examples of these misidentifications include: identifying the subject of the complement clause of say as its object, as in (10), which was identified as (say ministers from); misparsing two constituents as a single object noun phrase, as in (11), which was identified as (make subject to); and counting non-object noun phrases as the object as in (12), identified as (get hell out_oJ).</Paragraph>
      <Paragraph position="12">  (10) Ortega also said deputy foreign ministers from the five governments would meet Tuesday in Managua ....</Paragraph>
      <Paragraph position="13"> (11) Congress made a deliberate choice to make this commission subject to the open meeting requirements ...</Paragraph>
      <Paragraph position="14"> (12) Student Union, get the hell out of China!  Of course these errors are folded into the calculation of associations. No doubt our bigram model would be better if we could eliminate these items, but many of them represent parsing errors that cannot readily be identified by the parser, so we proceed with these errors included in the bigrams. After agreeing on the 'correct' attachment for the sample of 1000 triples, we are left with 880 verb-noun-preposition triples (having discarded the 120 parsing errors). Of these, 586 are noun attachments and 294 verb attachments.</Paragraph>
    </Section>
    <Section position="3" start_page="232" end_page="232" type="sub_section">
      <SectionTitle>
Evaluating performance
</SectionTitle>
      <Paragraph position="0"> First, consider how the simple structural attachment preference schemas perform at predicting the Judge 1 I i i i i 4.9 i LA 557 323 85.4 65.9 78.3  human judges and the lexical association procedure (LA).</Paragraph>
      <Paragraph position="1"> outcome in our test set. Right Association, which predicts noun attachment, does better, since in our sample there are more noun attachments, but it still has an error rate of 33%. Minimal Attach. meat, interpreted to mean verb attachment, has the complementary error rate of 67%. Obviously, neither of these procedures is particularly impressive. null Now consider the performance of our attachment procedure for the 880 standard test sentences. Table 2 shows the performance for the two human judges and for the lexical association attachment procedure.</Paragraph>
      <Paragraph position="2"> First, we note that the task of judging attachment on the basis of verb, noun and preposition alone is not easy. The human judges had overall error rates of 10-15%. (Of course this is considerably better than always choosing noun attachment.) The lexical association procedure based on t-scores is somewhat worse than the human judges, with an error rate of 22%, but this also is an improvement over simply choosing the nearest attachment site.</Paragraph>
      <Paragraph position="3"> If we restrict the lexical association procedure to choose attachment only in cases where its confidence is greater than about 95% (i.e., where t is greater than 2.1), we get attachment judgements on 607 of the 880 test sentences, with an overall error rate of 15% (Table 3). On these same sentences, the human judges also showed slight improvement. null</Paragraph>
    </Section>
    <Section position="4" start_page="232" end_page="233" type="sub_section">
      <SectionTitle>
Underlying Relations
</SectionTitle>
      <Paragraph position="0"> Our model takes frequency of cooccurrence as evidence of an underlying relationship, but makes no attempt to determine what sort of relationship is involved. It is interesting to see what kinds of relationships the model is identifying. To investigate this we categorized the 880 triples ac- null human judges and the lexical association procedure (LA) for test triples where t &gt; 2.1 cording to the nature of the relationship underlying the attachment. In many cases, the decision was difficult. Even the argument/adjunct distinction showed many gray cases between clear participants in an action (arguments) and clear temporal modifiers (adjuncts). We made rough best guesses to partition the cases into the following categories: argument, adjunct, idiom, small clause, locative ambiguity, systematic ambiguity, light verb. With this set of categories, 84 of the 880 cases remained so problematic that we assigned them to category other.</Paragraph>
      <Paragraph position="1"> Table 4 shows the performance of the lexical attachment procedure for these classes of relations. Even granting the roughness of the categorization, some clear patterns emerge. Our approach is quite successful at attaching arguments correctly; this represents some confirmation that the associations derived from the AP sample are indeed the kind of associations previous research has suggested are relevant to determining attachment. The procedure does better on arguments than on adjuncts, and in fact performs rather poorly on adjuncts of verbs (chiefly time and manner phrases). The remaining cases are all hard in some way, and the performance tends to be worse on these cases, showing clearly for a more elaborated model.</Paragraph>
    </Section>
    <Section position="5" start_page="233" end_page="233" type="sub_section">
      <SectionTitle>
Sense Conflations
</SectionTitle>
      <Paragraph position="0"> The initial steps of our procedure constructed a table of frequencies with entries f(z,p), where z is a noun or verb root string, and p is a preposition string. These primitives might be too coarse, in that they do not distinguish different senses of a preposition, noun, or verb. For instance, the temporM use of in in the phrase in December is identified with a locative use in Teheran. As a result, the procedure LA necessarily makes the same attach- null procedure by underlying relationship ment prediction for in December and in Teheran occurring in the same context. For instance, LA identifies the tuple reopen embassy in as an NP attachment (t-score 5.02). This is certainly incorrect for (13), though not for (14). 2  (13) Britain reopened the embassy in December null (14) Britain reopened its embassy in</Paragraph>
    </Section>
    <Section position="6" start_page="233" end_page="235" type="sub_section">
      <SectionTitle>
Teheran
</SectionTitle>
      <Paragraph position="0"> Similarly, the scalar sense of drop exemplified in (15) sponsors a preposition to, while the sense represented in drop the idea does not. Identifying the two senses may be the reason that LA makes no attachment choice for drop resistance to (derived from (16)), where the score is -0.18.</Paragraph>
      <Paragraph position="1"> (15) exports are expected to drop a further 1.5 percent to 810,000 (16) persuade Israeli leaders to drop their resistance to talks with the PLO We experimented with the first problem by substituting an abstract preposition in,MONTH for all occurrences of in with a month name as an object. While the tuple reopen embassy in~oMONTH was correctly pushed in the direction of a verb attachment (-1.34), in other cases errors were introduced, and there was no compelling general improvement in performance. In tuples of the form drop/grow/increase percent inJ~MONTH , derived from examples such as (16), the preposition was incorrectly attached to the noun percent.</Paragraph>
      <Paragraph position="2"> 2(13) is a phrase from our corpus, while (14) is a constructed example.</Paragraph>
      <Paragraph position="3">  (16) Output at mines and oil wells  dropped 1.8 percent in February (17) ,1.8 percent was dropped by output at mines and oil wells We suspect that this reveals a problem with our estimation procedure, not for instance a paucity of data. Part of the problem may be the fact that adverbial noun phrase headed by percent in (16) does not passivize or pronominalize, so that there are no sure verb attachment cases directly corresponding to these uses of scalar motion verbs. Comparison with a Dictionary The idea that lexical preference is a key factor in resolving structural ambiguity leads us naturally to ask whether existing dictionaries can provide useful information for disambiguation. There are reasons to anticipate difficulties in this regard. Typically, dictionaries have concentrated on the 'interesting' phenomena of English, tending to ignore mundane lexical associations. However, the Collins Cobuild English Language Dictionary (Sinclair et al. 1987) seems particularly appropriate for comparing with the AP sample for several reasons: it was compiled on the basis of a large text corpus, and thus may be less subject to idiosyncrasy than more arbitrarily constructed works; and it provides, in a separate field, a direct indication of prepositions typically associated with many nouns and verbs. Nevertheless, even for Cobuild, we expect to find more concentration on, for example, idioms and closely bound arguments, and less attention to the adjunct relations which play a significant role in determining attachment preferences.</Paragraph>
      <Paragraph position="4"> From a machine-readable version of the dictionary, we extracted a list of 1535 nouns associated with a particular preposition, and of 1193 verbs associated with a particular preposition after an object noun phrase. These 2728 associations are many fewer than the number of associations found in the AP sample. (see Table 5.) Of course, most of the preposition association pairs from the AP sample end up being nonsignificant; of the 88,860 pairs, fewer than half (40,869) occur with a frequency greater than 1, and only 8337 have a t-score greater than 1.65. So our sample gives about three times as many significant preposition associations as the COBUILD dictionary. Note however, as Table 5 shows, the overlap is remarkably good, considering the large space of possible bigrams. (In our bigram table  COBUILD and the AP sample there are over 20,000 nouns, over 5000 verbs, and over 90 prepositions.) On the other hand, the lack of overlap for so many cases - assuming that the dictionary and the significant bigrams actually record important preposition associations - indicates that 1) our sample is too small, and 2) the dictionary coverage is widely scattered.</Paragraph>
      <Paragraph position="5"> First, we note that the dictionary chooses attachments in 182 cases of the 880 test sentences. Seven of these are cases where the dictionary finds an association between the preposition and both the noun and the verb. In these cases, of course, the dictionary provides no information to help in choosing the correct attachment.</Paragraph>
      <Paragraph position="6"> Looking at the 175 cases where the dictionary finds one and only one association for the preposition, we can ask how well it does in predicting the correct attachment. Here the results are no better than our human judges or than our bigram procedure. Of the 175 cases, in 25 cases the dictionary finds a verb association when the correct association is with the noun. In 3 cases, the dictionary finds a noun association when the correct association is with the verb. Thus, overall, the dictionary is 86% correct.</Paragraph>
      <Paragraph position="7"> It is somewhat unfair to use a dictionary as a source of disambiguation information; there is no reason to expect that a dictionary to provide information on all significant associations; it may record only associations that are interesting for some reason (perhaps because they are semantically unpredictable.) Table 6 shows a small sample of verb-preposition associations from the AP sam-</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>