<?xml version="1.0" standalone="yes"?>
<Paper uid="P94-1011">
  <Title>PRECISE N-GRAM PROBABILITIES FROM STOCHASTIC CONTEXT-FREE GRAMMARS</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> Probabilistic language modeling with n-gram grammars (particularly bigram and trigram) has proven extremely useful for such tasks as automated speech recognition, part-of-speech tagging, and word-sense disambiguation, and lead to simple, efficient algorithms. Unfortunately, working with these grammars can be problematic for several reasons: they have large numbers of parameters, so reliable estimation requires a very large training corpus and/or sophisticated smoothing techniques (Church and Gale, 1991); it is very hard to directly model linguistic knowledge (and thus these grammars are practically incomprehensible to human inspection); and the models are not easily extensible, i.e., if a new word is added to the vocabulary, none of the information contained in an existing n-gram will tell anything about the n-grams containing the new item. Stochastic context-free grammars (SCFGs), on the other hand, are not as susceptible to these problems: they have many fewer parameters (so can be reasonably trained with smaller corpora); they capture linguistic generalizations, and are easily understood and written, by linguists; and they can be extended straight-forwardly based on the underlying linguistic knowledge.</Paragraph>
    <Paragraph position="1"> In this paper, we present a technique for computing an n-gram grammar from an existing SCFG--an attempt to get the best of both worlds. Besides developing the mathematics involved in the computation, we also discuss efficiency and implementation issues, and briefly report on our experience confirming its practical feasibility and utility.</Paragraph>
    <Paragraph position="2"> The technique of compiling higher-level grammatical models into lower-level ones has precedents: Zue et al. (1991) report building a word-pair grammar from more elaborate language models to achieve good coverage, by random generation of sentences. In our own group, the current approach was predated by an alternative one that essentially relied on approximating bigram probabilities through Monte-Carlo sampling from SCFGs.</Paragraph>
  </Section>
class="xml-element"></Paper>