<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1025">
  <Title>Probabilistic Prediction and Picky Chart Parsing*</Title>
  <Section position="2" start_page="0" end_page="128" type="intro">
    <SectionTitle>
1. Introduction
</SectionTitle>
    <Paragraph position="0"> Two important concerns in natural language parsing which encourage the use of probabilistic analysis are efficiency and accuracy. An accurate parser which has a domain or language model so detailed that it takes hours to process a single sentence, while perhaps interesting, is no more useful than a simple instantaneous parser which is always wrong. Probabilistic modelling of the grammar of a language has been proposed as a potential solution to the accuracy problem, disambiguating grammatical parses generated by an ambiguous grammar. However, little attention has been paid to the repercussions of probabilistic parsing on the computational complexity and average-case performance of existing parsing algorithms. Effective probabilistic models of grammar which take into account contextual information (e. g. \[10\] \[2\]) cannot take advantage of the O(n 3) behavior of CKY-like parsing algorithms. If they are to use existing grammatical formalisms, these models must use algorithms that are worst-case exponential.</Paragraph>
    <Paragraph position="1"> When natural language parsers are incorporated into natural language understanding systems, another significant issue arises: robustness. As a component of a language processing system, a parser's task is to analyze correctly all inputs which can be understood by the system, not just those which are precisely grammatical. Or, one might say, the grammar of natural lan*Special thanks to Jerry Hobbs and Bob Moore at SRI for providing access to their computers, and to Salim Roukos, Peter Brown, and Vincent and Steven Della Pietra at IBM for their instructive lessons on probabillstic modelling of natural language. guage includes fragments, run-ons, split infinitives, and other disfluencies which would receive red marks on a high school English paper. At the same time, meaningless sequences of words and other uninterpretable inputs should nol be analyzed as though they are acceptable.</Paragraph>
    <Paragraph position="2"> Robust processing of natural language is an ideal application of probabilistic methods, since probability theory provides a well-behaved measure of expectation within a given language.</Paragraph>
    <Paragraph position="3"> This paper proposes an agenda-based probabilistic chart parsing algorithm which is both robust and efficient. The algorithm, Picky 1, is considered robust because it will potentially generate all constituents produced by a pure bottom-up parser and rank these constituents by likelihood. The efficiency of the algorithm is achieved through a technique called probabilistic prediction, which helps the algorithm avoid worst-case behavior. Probabilistic prediction is a trainable technique for modelling where edges are likely to occur in the chart-parsing process. 2 Once predicted edges are added to the chart using probabilistic prediction, they are processed in a style similar to agenda-based chart parsing algorithms. By limiting the edges in the chart to those which are predicted by this model, the parser can process a sentence while generating only the most likely constituents given the input.</Paragraph>
    <Paragraph position="4"> The Picky parsing algorithm is divided into three phases, where the goal of each phase is to minimize the set of rule predictions in the chart to only those necessary to generate an analysis of the input sentence. When a phase completes without producing an analysis of the input, the next phase expands the set of rules which it can use and applies these new rules to the chart from the previous phase. The proposed algorithm is still exponential in the worst-case, but only exhibits worst-case behavior on sentences which are completely outside the domain of the training material (i. e. contain multiple occurrences of grammatical structures rarely seen or unseen in training). In this work, the efficiency of various  algorithms and effectiveness of models is determined by a comparison of the number of rule predicts, rule advancing operations (the basic operation in chart parsing), and complete constituents detected by the parser.</Paragraph>
    <Paragraph position="5"> The results of experiments using this parsing algorithm are quite promising. On a corpus of 300 randomly selected test sentences, Picky parses these sentences with 89% first parse accuracy, and up to 92% accuracy within the first three parses. Further, sentences which are parsed completely by the probabilistic prediction technique, in phases I and II, have a 97% first parse accuracy. The algorithm is extremely efficient, with less than a 1.6:1 ratio of constituents recognized to constituents in the final parse for sentences parsed by phases I and II. The performance decreases for sentence outside the training corpus that are parsed in phase III.</Paragraph>
    <Paragraph position="6"> This paper will present the Picky parsing algorithm, describing the both the original features of the parser and those adapted from previous work. Then, along with accuracy and efficiency results, the paper will report an analysis of the interaction between the phases of the parsing algorithm and the probabilistic models of parsing and prediction.</Paragraph>
  </Section>
class="xml-element"></Paper>