<?xml version="1.0" standalone="yes"?>
<Paper uid="P95-1037">
  <Title>Statistical Decision-Tree Models for Parsing*</Title>
  <Section position="2" start_page="0" end_page="276" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Parsing a natural language sentence can be viewed as making a sequence of disambiguation decisions: determining the part-of-speech of the words, choosing between possible constituent structures, and selecting labels for the constituents. Traditionally, disambiguation problems in parsing have been addressed by enumerating possibilities and explicitly declaring knowledge which might aid the disambiguation process. However, these approaches have proved too brittle for most interesting natural language problems. null This work addresses the problem of automatically discovering the disambiguation criteria for all of the decisions made during the parsing process, given the set of possible features which can act as disambiguators. The candidate disambiguators are the words in the sentence, relationships among the words, and relationships among constituents already constructed in the parsing process.</Paragraph>
    <Paragraph position="1"> Since most natural language rules are not absolute, the disambiguation criteria discovered in this work are never applied deterministically. Instead, all decisions are pursued non-deterministically according to the probability of each choice. These probabilities are estimated using statistical decision tree models. The probability of a complete parse tree (T) of a sentence (S) is the product of each decision (dl) conditioned on all previous decisions:</Paragraph>
    <Paragraph position="3"> Each decision sequence constructs a unique parse, and the parser selects the parse whose decision sequence yields the highest cumulative probability. By combining a stack decoder search with a breadth-first algorithm with probabilistic pruning, it is possible to identify the highest-probability parse for any sentence using a reasonable amount of memory and time.</Paragraph>
    <Paragraph position="4">  The claim of this work is that statistics from a large corpus of parsed sentences combined with information-theoretic classification and training algorithms can produce an accurate natural language parser without the aid of a complicated knowledge base or grammar. This claim is justified by constructing a parser, called SPATTER (Statistical PATTErn Recognizer), based on very limited lingnistic information, and comparing its performance to a state-of-the-art grammar-based parser on a common task. It remains to be shown that an accurate broad-coverage parser can improve the performance of a text processing application. This will be the subject of future experiments.</Paragraph>
    <Paragraph position="5"> One of the important points of this work is that statistical models of natural language should not be restricted to simple, context-insensitive models. In a problem like parsing, where long-distance lexical information is crucial to disambiguate interpretations accurately, local models like probabilistic context-free grammars are inadequate. This work illustrates that existing decision-tree technology can be used to construct and estimate models which selectively choose elements of the context which contribute to disambignation decisions, and which have few enough parameters to be trained using existing resources.</Paragraph>
    <Paragraph position="6"> I begin by describing decision-tree modeling, showing that decision-tree models are equivalent to interpolated n-gram models. Then I briefly describe the training and parsing procedures used in SPAT-TER. Finally, I present some results of experiments comparing SPATTER with a grammarian's rule-based statistical parser, along with more recent resuits showing SPATTER applied to the Wall Street Journal domain.</Paragraph>
  </Section>
class="xml-element"></Paper>