<?xml version="1.0" standalone="yes"?>
<Paper uid="H93-1046">
  <Title>MEASURES AND MODELS FOR PHRASE RECOGNITION</Title>
  <Section position="7" start_page="235" end_page="235" type="concl">
    <SectionTitle>
4. CONCLUSION
</SectionTitle>
    <Paragraph position="0"> To summarize, I have approached the problem of parsing English as a problem of statistically approximating the Penn Treebank. For the purposes of parsing, English is a function from sentences to parse-trees, and the Treebank provides a (sufficiently representative) sample from the extension of that function. A parsing model approximates Treebank annotation. Our basic measure of the goodness of the approximation is the amount of additional information we must provide in order to specify the Treebank parse, given the probabilities assigned by the parser. I have presented a series of models to &amp;quot;calibrate&amp;quot; the measure, showing what kind of performance is achievable using obvious kinds of information.</Paragraph>
    <Paragraph position="1"> An impetus for this work is the success of parsers like Fidditch and Cass, which are able to greatly reduce search, and increase parsing speed, by using highly reliable patterns for recognizing phrases. The limitation of such work is the impracticality of constructing reliable patterns by hand, past a certain point. One hindrance to automatic acquisition of reliable patterns has been the lack of a framework for evaluating such parsers at a fine grain, and exploring which kinds of information contribute most to parsing accuracy.</Paragraph>
    <Paragraph position="2"> In the current work, I have presented a framework for fine-grained evaluation of parsing models. It does not assume stochastic context-free grammars, and it quantifies parsers' performance at parsing, rather than at a more indirectly related task like word prediction.</Paragraph>
  </Section>
class="xml-element"></Paper>