<?xml version="1.0" standalone="yes"?>
<Paper uid="H91-1042">
  <Title>Statistical Agenda Parsing</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
DELPHI AGENDA PARSING
</SectionTitle>
    <Paragraph position="0"> Most techniques for search splice reduction involve careful tuning of the grammar or the parsing mechanism. This is very labor intensive and can place limits on the grammatical coverage of the system (Abney 1990). Our approach is to use an automated statistical technique for ranking rules based on their use in parsing a training set with the same grammar (under the control of an all-paths GHR parser without human supervision).</Paragraph>
    <Paragraph position="1"> This approach also allows us to include grammatical rules that are of use only rarely, or in specialized domains, and to learn how applicable they are to a body of sentences. To take into account general linguistic tendencies, we augment the statistical ranking by a small number of general agenda ordering strategies.</Paragraph>
    <Paragraph position="2"> The DELPHI agenda mechanism is based on three &amp;quot;scbedulable&amp;quot; action types:  1. the insertion of a term into the chart, 2. the insertion of a dotted rule into the chart, and 3. the (conditional) &amp;quot;pair extension&amp;quot; of a dotted rule by a  term.</Paragraph>
    <Paragraph position="3"> In principle one would like to order those actions in terms of the probability that they lead to a final parse. The initial implementation of the agenda mechanism uses an approximation to this ordering.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
USE OF STATISTICAL MEASURES
</SectionTitle>
    <Paragraph position="0"> There are two types of measures that one might estimate to help the agenda parsing mechanism. They are (1) category expansion probabilities and (2) rule success probabilities.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="222" type="metho">
    <SectionTitle>
Category Expansion Probabilities
</SectionTitle>
    <Paragraph position="0"> Category expansion probabilities are perhaps the more obvious of the two measures. The goal is to determine the probability that a given syntactic category (e.g., NP) is expanded by a given grammar rule in a valid parse.</Paragraph>
    <Paragraph position="1"> These probabilities allow one to estimate the probability that a given tree is the expansion of a given category. Bayes' rule may be used to calculate the relative probabilities of various parse trees for a specified input string.</Paragraph>
    <Section position="1" start_page="0" end_page="222" type="sub_section">
      <SectionTitle>
Rule Success Probabilities
</SectionTitle>
      <Paragraph position="0"> Using rule success probabilities, the goal is to determine the probability that a term inserted into the chart by a particular rule will be part of a Fmal parse.</Paragraph>
    </Section>
    <Section position="2" start_page="222" end_page="222" type="sub_section">
      <SectionTitle>
Training
</SectionTitle>
      <Paragraph position="0"> In order to train the agenda mechanism, a set of sentences is parsed using the all-paths GHR parser and their charts are analyzed.</Paragraph>
      <Paragraph position="1"> For each rule (R) in the grammar we determine three numbers:  1. NT(R), the number of terms in the charts based on that rule. 2. NDR(R), the number of dotted rules initiated in the chart based on that rule.</Paragraph>
      <Paragraph position="2"> 3. NGT(R), the number of &amp;quot;good terms&amp;quot; based on that rule,  ones that are constituents of an ACCEPTABLE parse (i.e., ones leading to executable database commands for ATIS). For each category C in the grammar, we calculate one number: 4. NGT(C), the number of terms with that category which are constituents in an acceptable parse.</Paragraph>
      <Paragraph position="3"> The ratio NGT(R)/NT(R) is an estimate of the probability that a term based on R will appear in the final parse, and NGT(R)\]NDR(R) is an estimate of the probability that the initiation of a dotted rule based on R will lead to a good term. (Note that in DELPHI, each word sense is treated as if it were a separate grammar rule, and so this mechanism takes into account the relative likdihood of various word senses in the training set.) If C(R) is the category produced by the rule R, then the category expansion probability of R is NGT(R)/NGT(C(R)).</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="222" end_page="222" type="metho">
    <SectionTitle>
Preliminary Results for Different Measures
</SectionTitle>
    <Paragraph position="0"> Using rule success probabilities leads to substantial reduction (a factor of more than 3) in chart size. In general, one might expect that better estimates of such probabilities, based on category expansion probabilities in the tree below the term, would lead to improved results, even though these estimates require somewhat more computation than rule success probabilities alone.</Paragraph>
    <Paragraph position="1"> We have compared the use of category expansion probabilities with the use of rule success probabilities in several variations of the agenda mechanism, and have found that rule success probabilities produce superior results, although the reasons for this are not entirely clear.</Paragraph>
    <Paragraph position="2"> An experiment using category expansion probabilities alone led to larger charts than produced by the use of rule success probabilities in isolation. Combining category expansion probabilities with rule success probabilities appeared to be no better than just using the rule success probabilities.</Paragraph>
  </Section>
  <Section position="7" start_page="222" end_page="223" type="metho">
    <SectionTitle>
AGENDA STRUCTURES
</SectionTitle>
    <Paragraph position="0"> The structure of the agenda mechanism appears to be as important as the statistical measures used to order agenda items. Experience with probabilistic agendas in speech processing would suggest an approach in which all information relevant to ordering is combined into a single numeric measure and used to order a single queue. In principal, this allows different measures to interact and for strength in one measure to make up for weakness in another.</Paragraph>
    <Paragraph position="1"> We experimented with this approach in a system which had a single agenda in which all three of the schedulable action types described above were placed. The statistical measures described above were combined in a weighted fashion with priorities based on the size of the constituents, the position of the right hand end of the constituent and the action type. A number of experiments were run, giving different weightings to the different parameters, but all of these experiments led to charts that were 20% to 40% larger than the alternative structured agenda described below.</Paragraph>
    <Paragraph position="2"> The structured-agenda approach involves the creation of a 2-dimansional array of agendas, as illustrated in figure 1.</Paragraph>
    <Paragraph position="3">  Each cell of the array consists of a single type of action, e.g. term insertion, and all of the actions in the list Ai in a cell have the same rightmost end. Within the cell, the actions in the list Ai are ordered by probability estimates.</Paragraph>
    <Paragraph position="4"> For each step, the first non-empty cell (starting with A1 and going in the order shown in figure 1) is chosen, and the first item on its agenda is run. This has the effect of reinforcing progress to the right through the input string, of choosing the most appropriate action for such motion at each step, and favoring close attachment of modifiers.</Paragraph>
  </Section>
class="xml-element"></Paper>