<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1028">
  <Title>PARAMETER ESTIMATION FOR CONSTRAINED CONTEXT-FREE LANGUAGE MODELS</Title>
  <Section position="3" start_page="0" end_page="7962" type="intro">
    <SectionTitle>
1. INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> This paper introduces the idea of N-gram constrained context-free language models. This class of language models merges two prevalent ideas in language modeling: N-grams and context-free grammars. In N-gram language models, the underlying probability distributions are Markov chains on the word string. N-gram models have advantages in their simplicity. Both parameter estimation and sampling from the distribution are simple tasks. A disadvantage of these models is their weak modeling of linguistic structure.</Paragraph>
    <Paragraph position="1"> Context-free language models are instances of random branching processes. The major advantage of this class of models is its ability to capture linguistic structure.</Paragraph>
    <Paragraph position="2"> In the following section, notation for stochastic context-free language models and the probability of a word string under this model are presented. Section 3 reviews a parameter estimation algorithm for SCF language models.</Paragraph>
    <Paragraph position="3"> Section 4 introduces the bigram-constrained context-free language model. This language model is seen to be a Markov random field. In Section 5, a random sampling algorithm is stated. In Section 6, the problem of parameter estimation in the constrained context-free language model is addressed.</Paragraph>
    <Paragraph position="4"> *Division of Applied Mathematics, Brown University, Providence, Rhode Island 02904 tBell Communications Research, Morristown, New Jersey</Paragraph>
  </Section>
class="xml-element"></Paper>