<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-1119">
  <Title>Automatic Acquisition of Language Model based on Head-Dependent Relation between Words</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Language modeling is to associate a priori probability to a sentence. It is a key part of many natural language applications such as speech recognition and statistical machine translation.</Paragraph>
    <Paragraph position="1"> Previous works for language modeling can be broadly divided into two approaches; one is n-gram-based and the other is grammar-based.</Paragraph>
    <Paragraph position="2"> N-gram model estimates the probability of a sentence as the product of the probability of each word in the sentence. It assumes that probability of the nth word is dependent on the previous n- 1 words. The n-gram probabilities are estimated by simply counting the n-gram frequencies in a training corpus. In some cases, class (or part of speech) n-grams are used instead of word n-grams(Brown et al., 1992; Chang and Chen, 1996). N-gram model has been widely used so far, but it has always been clear that n-gram can not represent long distance dependencies.</Paragraph>
    <Paragraph position="3"> In contrast with n-gram model, grammar-based approach assigns syntactic structures to a sentence and computes the probability of the sentence using the probabilities of the structures. Long distance dependencies can be represented well by means of the structures. The approach usually makes use of phrase structure grammars such as probabilistic context-free grammar and recursive transition network(Lari and Young, 1991; Sneff, 1992; Chen, 1996). In the approach, however, a sentence which is not accepted by the grammar is assigned zero probability. Thus, the grammar must have broad-coverage so that any sentence will get non-zero probability. But acquisition of such a robust grammar has been known to be very difficult.</Paragraph>
    <Paragraph position="4"> Due to the difficulty, some works try to use an integrated model of grammar and n-gram compensating each other(McCandless, 1994; Meteer and Rohlicek, 1993). Given a robust grammar, grammar-based language modeling is expected to be more powerful and compact in model size than n-gram-based one.</Paragraph>
    <Paragraph position="5"> In this paper we present a language modeling based on a kind of simple dependency grammar. The grammar consists of head-dependent relations between words and can be learned automatically from a raw corpus using the reestimation algorithm which is also introduced in this paper. Based on the dependencies, a sentence is analyzed and assigned syntactic structures by which long distance dependences are represented. Because the model can be thought of as a linguistic bi-gram model, the smoothing functions of n-gram models can be applied to it.</Paragraph>
    <Paragraph position="6"> Thus, the model can be robust, adapt easily to new domains, and be effective.</Paragraph>
    <Paragraph position="7"> The paper is organized as follows. We introduce some definitions and notations for the dependency grammar and the reestimation algorithm in section 2, and explain the algorithm in section 3. In section 4, we show the experimental results for the suggested model compared to n-gram models. Finally, section 5 concludes this paper.</Paragraph>
  </Section>
class="xml-element"></Paper>