<?xml version="1.0" standalone="yes"?>
<Paper uid="P99-1048">
  <Title>Corpus-Based Identification of Non-Anaphoric Noun Phrases</Title>
  <Section position="7" start_page="377" end_page="378" type="evalu">
    <SectionTitle>
6 Evaluation
</SectionTitle>
    <Paragraph position="0"> We expected the $1, EHP, and DO methods to increase coverage. First, we evaluated each method independently (on top of the syntactic heuristics). The results appear in rows 2-4 of Figure 7. Each method increased recall to between 61-69%, but decreased precision to 8487%. All of these methods produced a substantial gain in recall at some cost in precision. Next, we tried combining the methods to make sure that they were not identifying exactly the same set of existential NPs. When we combined the S1 and EHP heuristics, recall increased to 80% with precision dropping only slightly to 82%. When we combined all three methods (S1, EHP, and DO), recall increased to 82% without any corresponding loss of precision. These experiments show that these heuristics substantially increase recall and are identifying different sets of existential NPs.</Paragraph>
    <Paragraph position="1"> Finally, we tested our vaccine algorithm to see if it could increase precision without sacrificing much recall. We experimented with two variations: Va used an upper definite probability threshold of 70% and ~ used an upper definite probability threshold of 50%. Both variations used a lower definite probability threshold of 25%. The results are shown in rows 7-8 of Figure 7. Both vaccine variations increased precision by several percentage points with only a slight drop in recall.</Paragraph>
    <Paragraph position="2"> In previous work, the system developed by Vieria &amp; Poesio achieved 74% recall and 85% precision for identifying &amp;quot;larger situation and unfamiliar use&amp;quot; NPs. This set of NPs does not correspond exactly to our definition of existential NPs because we consider associative NPs  to be existential and they do not. Even so, our results are slightly better than their previous results. A more equitable comparison is to measure our system's performance on only the independent existential noun phrases. Using this measure, our algorithm achieved 81.8% recall with 85.6% precision using Va, and achieved 82.9% recall with 83.5% precision using Vb.</Paragraph>
  </Section>
class="xml-element"></Paper>