<?xml version="1.0" standalone="yes"?>
<Paper uid="W95-0103">
  <Title>Prepositional Phrase Attachment through a Backed-Off Model</Title>
  <Section position="3" start_page="0" end_page="28" type="intro">
    <SectionTitle>
2 Background
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.1 Training and Test Data
</SectionTitle>
      <Paragraph position="0"> The training and test data were supplied by IBM, being identical to that used in \[RRR94\]. Examples of verb phrases eontMning a (v np pp) sequence had been taken fl'om the Wall Street Journal Treebank \[MSM93\]. For each such VP the head verb, first head noun, preposition and second head noun were extracted, along with the attachment decision (1 for noun attachment, 0 for verb). For example the verb phrase: ((joined (the board)) (as a nonexecutive director)) would give the quintuple: 0 joined board as director The elements of this quintuple will from here on be referred to as the random variables A, V, N1, P, and N2. In the above verb phrase A = 0, V = joined, N1 = board, P = as, and N2 = director. The data consisted of training and test files of 20801 and 3097 quintuples respectively. In addition, a development set of 4039 quintuples was also supplied. This set was used during development of the attachment algorithm, ensuring that there was no implicit training of the method on the test set itself.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="28" type="sub_section">
      <SectionTitle>
2.2 Outline of the Problem
</SectionTitle>
      <Paragraph position="0"> A PP-attachment algorithm must take each quadruple (V = v, N1 = nl, P = p, N2 = n2) in test data and decide whether the attachment variable A = 0 or 1. The accuracy of the algorithm is then the percentage of attachments it gets 'correct' on test data, using the A values taken from the treebank as the reference set.</Paragraph>
      <Paragraph position="1"> The probability of the attachment variable A being 1 or 0 (signifying noun or verb attachment respectively) is a probability, p, which is conditional on the values of the words in the quadruple. In general a probabilistic algorithm will make an estimate, 15, of this probability: 15(A= llV=v, Nl=nl, P=p, N2=n2) For brevity this estimate will be referred to from here on as: p(l\[v, nl,p, n2)  The decision can then be made using the test: ~(llv, nl,p, n2 ) &gt;= 0.5 If this is true the attachment is made to the noun, !f not then it is made to the verb.</Paragraph>
    </Section>
    <Section position="3" start_page="28" end_page="28" type="sub_section">
      <SectionTitle>
2.3 Lower and Upper Bounds on Performance
</SectionTitle>
      <Paragraph position="0"> When evaluating an algorithm it is useful to have an idea of the lower and upper bounds on its performance. Some key results are summarised in the table below. All results in this section are on the IBM training and test data, with the exception of the two 'average human' results.</Paragraph>
    </Section>
    <Section position="4" start_page="28" end_page="28" type="sub_section">
      <SectionTitle>
Method Percentage Accuracy
</SectionTitle>
      <Paragraph position="0"> Always noun attachment 59.0 Most likely for each preposition 72.2 Average Human (4 head words only) 88.2 Average Human (whole sentence) 93.2 'Always noun attachment' means attach to the noun regardless of (v,nl,p,n2). 'Most likely for each preposition' means use the attachment seen most often in training data for the preposition seen in the test quadruple. The human performance results are taken from \[RRR94\], and are the average performance of 3 treebanking experts on a set of 300 randomly selected test events from the WSJ corpus, first looking at the four head words alone, then using the whole sentence. A reasonable lower bound seems to be 72.2% as scored by the 'Most likely for each preposition' method. An approximate upper bound is 88.2% - it seems unreasonable to expect an algorithm to perform much better than a human.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>