<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1227">
  <Title>A Method of Incorporating Bigram Constraints into an LR Table and Its Effectiveness in Natural Language Processing i</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> In natural language processing, stochastic language models are commonly used for lexical and syntactic disambiguation (Fujisaki et al., 1991; Franz, 1996).</Paragraph>
    <Paragraph position="1"> Stochastic language models are also helpful in reducing the complexity of speech and language processing by way of providing probabilistic linguistic constraints (Lee, 1989).</Paragraph>
    <Paragraph position="2"> N-gram language models (Jelinek, 1990), including bigram and trigram models, are the most commonly used method of applying local probabilistic constraints. However, context-free grammars (CFGs) produce more global linguistic constraints than N-gram models. It seems better to combine both local and global constraints and use them both concurrently in natural language processing. The reason why N-gram models are preferred over CFGs is that N-gram constraints are easily acquired from a given corpus. However, the larger N is, the more serious the problem of data sparseness becomes.</Paragraph>
    <Paragraph position="3"> CFGs are commonly employed in syntactic parsing as global linguistic constraints, since many eificient parsing algorithms are available. GLR (Generalized LR) is one such parsing algorithm that uses an LR table, into which CFG constraints are pre-compiled in advance (Knuth, 1965; Tomita, 1986).</Paragraph>
    <Paragraph position="4"> Therefore if we can incorporate N-gram constraints into an LR table, we can make concurrent use of both local and global linguistic constraints in GLR parsing.</Paragraph>
    <Paragraph position="5"> In the following section, we will propose a method that incorporates bigram constraints into an LR table. The advantages of the method are summarized as follows: First, it is expected that this method produces a lower perplexity than that for a simple bigram language model, since it is possible to utilize both local (bigram) and global (CFG) constraints in the LR table. We will evidence this reduction in perplexity by considering states in an LR table for the case of GLR parsing.</Paragraph>
    <Paragraph position="6"> Second, bigram constraints are easily acquired from smaller-sized corpora. Accordingly, data sparseness is not likely to arise.</Paragraph>
    <Paragraph position="7"> Third, the separation of local and global constraints makes it easy to describe CFG rules, since CFG writers need not take into'account tedious descriptions of local connection constraints within th  Figure 1 represents a situation in which ai and bj are adjacent to each other, where a~ belongs to Setl (i = 1,..-,I) and bj belongs to Setj (j = 1,..., J). Set~ l(Tana~ et al., 1997) reported that the separate description of local and global constraints reduced the CFG rules to one sixth of their original number.</Paragraph>
    <Paragraph position="8"> Imai and Tanaka 225 A Method of Incorporating Bigram Constraints Hirold Ima/ and Hozumi Tanaka (1998) A Method of Incorporating Bigram Constraints into an LR Table and Its Effectiveness in Natural Language Processing. In D.M.W. Powers (ed.) NeMLaP3/CoNLL98: New Methods in Language</Paragraph>
    <Paragraph position="10"> and Sets are defined by last1 (Y) and first1 ( Z) (Aho et al., 1986), respectively. If a E Setz and b E Sets happen not to be able to occur in this order, it becomes a non-trivial task to express this adjacency restriction within the framework of a CFG.</Paragraph>
    <Paragraph position="11"> One solution to this problem is to introduce a new nonterminal symbol Ai for each a~ and a nonterminal symbol Bj for each hi. We then add rules of the form A --* Ai and Ai &amp;quot;* ai, and B ~ Bj and B i --* bj.</Paragraph>
    <Paragraph position="12"> As a result of this rule expansion, the order of the number of rules will become I x J in the worst case.</Paragraph>
    <Paragraph position="13"> The introduction of such new nonterminal symbols leads to an increase in grammar rules, which not only makes the LR table very large in size, but also diminishes efficiency of the GLR parsing method.</Paragraph>
    <Paragraph position="14"> The second solution is to augment X ~ Y Z with a procedure that checks the connection between a~ and bj. This solution can avoid the problem of the expansion of CFG rules, but we have to take care of the information flow from the bottom leaves to the upper nodes in the tree, Y, Z, and X.</Paragraph>
    <Paragraph position="15"> Neither the first nor the second solution are preferable, in terms of both efficiency of GLR parsing and description of CFG rules. Additionally, it is a much easier task to describe local connection constraints between two adjacent terminal symbols by way of a connection matrix such as in Figure 2, than to express these constraints within the CFG.</Paragraph>
    <Paragraph position="16"> The connection matrix in Figure 2 is defined as:</Paragraph>
    <Paragraph position="18"> The best solution seems to be to develop a method that can combine both a CFG and a connection matrix, avoiding the expansion of CFG rules. Consequently, the size of the LR table will become smaller and we will get better GLR parsing performance.</Paragraph>
    <Paragraph position="19"> In the following section, we will propose one such method. Note that we are considering connections between preterminals rather than words. Thus, we will have Connect(ai, bj) = 0 in the preterminal connection matrix similarly to the case of words.</Paragraph>
    <Paragraph position="21"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 Relation between the LR Table and
Connection Matrix
</SectionTitle>
      <Paragraph position="0"> First we discuss the relation between the LR table and a connection matrix. The action part of an LR table consists of lookahead symbols and states. Let a shift action sh m be in state l with the lookahead symbol a. After the GLR parser executes action sh m, the symbol a is pushed onto the top of the stack and the GLR parser shifts to state m. Suppose there is an action A in state m with lookahead b (see Figure 3). The action A is executable if Connect(a,b) ~ 0 (b can follow a), whereas if Connect(a, b) = 0 (b cannot follow a), the action A in state m with lookahead b is not executable and we can remove it from the LR table as an invalid action. Removing such invalid actions enables us to incorporate connection constraints into the LR table in addition to the implicit CFG constraints.</Paragraph>
      <Paragraph position="1"> In section 3.2, we will propose a method that integrates both bigram and CFG constraints into an LR table. After this integration process, we obtain a table called a bigram LR table.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>