<?xml version="1.0" standalone="yes"?>
<Paper uid="P92-1018">
  <Title>Linear Context-Free Rewriting Systems and Deterministic Tree-Walking Transducers*</Title>
  <Section position="2" start_page="0" end_page="141" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> We show that the class of string languages generated by linear context-free rewriting systems is equal to the class of output languages of deterministic tree-walking transducers. From equivalences that have previously been established we know that this class of languages is also equal to the string languages generated by context-free hypergraph grammars, multicomponent tree-adjoining grammars, and multiple context-free grammars and to the class of yields of images of the regular tree languages under finite-copying top-down tree transducers.</Paragraph>
    <Paragraph position="1"> Introduction In \[9\] a comparison was made of the generative capacity of a number of grammar formalisms. Several were found to share a number of characteristics (described below) and the class of such formalisms was called linear context-free rewriting systems. This paper shows how the class of string languages generated by linear context-free rewriting systems relates to a number of other systems that have been studied by formal language theorists. In particular, we show that the class of string languages generated by linear context-free rewriting systems is equal to the class of output languages of deterministic tree-walking transducers \[1\]. A number of other equivalences have already been established. In \[10\] it was shown that linear context-free rewriting systems and multicomponent tree adjoining grammars \[6\] generate the same string languages. The multiple context-free grammars of \[7\] are equivalent to linear context-free systems. This follows *I would like to thank Joost Engelfriet for drawing my attention to context-free hypergraph grammars and their relationship to deterministic tree-walking automata.</Paragraph>
    <Paragraph position="2"> from the fact that multiple context-free grammars are exactly that subclass of the linear context-free rewriting systems in which the objects generated by the grammar are tuples of strings. The class of output languages of deterministic tree-walking transducers is known to be equal to the class of yields of images of the regular tree languages under finite-copying top-down tree transducers \[4\] and in \[3\] it was shown that it also equal to the string languages generated by context-free hypergraph grammars \[2, 5\].</Paragraph>
    <Paragraph position="3"> We therefore have a number of Characterizations of the same class of languages and results that have been established for the class of languages associated with one system carry over to the others. This is particularly fruitful in this case since the output languages of deterministic tree-walking transducers have been well studied (see \[4\]).</Paragraph>
    <Paragraph position="4"> In the remainder of the paper we describe linear context-free rewriting systems and deterministic tree-walking transducers and outline the equivalence proof. We then describe context-free hypergraph grammars and observe that they are a context-free rewriting system. null</Paragraph>
    <Section position="1" start_page="0" end_page="141" type="sub_section">
      <SectionTitle>
Linear Context-Free Rewriting Systems
</SectionTitle>
      <Paragraph position="0"> Linear context-free rewriting systems arose from the observation that a number of grammatical formalisms share two properties.</Paragraph>
      <Paragraph position="1">  1. Their derivation tree sets can be generated by a context-free grammar.</Paragraph>
      <Paragraph position="2"> 2. Their composition operations are size-preserving,  i.e., when two or more substructures are combined only a bounded amount of structure is added or deleted.</Paragraph>
      <Paragraph position="3">  Examples of formalisms that. satisfy these conditions are head grammars \[8\], tree adjoining grammars \[6\], multicomponent tree adjoining grammars \[6\] and context-free hypergraph grammars. It was shown \[9\] that a system satisfying the above conditions generates languages that are semilinear and can be recognized in polynomial time. The definition of linear context-free rewriting systems is deliberately not specific about the kinds of structures being manipulated. In the case of head grammars these are pairs of strings whereas tree adjoining grammars manipulate trees and context-free hypergraph grammars manipulate graphs.</Paragraph>
      <Paragraph position="4"> In \[9\] size-preserving operations are defined for arbitrary structures in terms of properties of the corresponding functions over the terminal yield of the structures involved. The yield is taken to be a tuple of terminal strings. We call the function associated with a composition operation the yield function of that operation. The yield function of Of of a composition operation f gives the yield of the structure f(cl,ldots, cn) based on the yield of the structures el, * *., am.</Paragraph>
      <Paragraph position="5"> Let ~ be an alphabet of terminal symbols, f is an n-ary linear regular operation over tuples of strings in ~ if it can be defined with an equation of the form f((xl,1,..., xl,k,),..., (ran,l,..., xn,k,,)) ---- (tl,...,tk) where each k i &gt; O, n &gt;_ 0 and each ti is a string of variables (x's) and symbols in ~ and where the equation is regular (all the variables appearing on one side appear on the other) and linear (the variables appear only once on the left and right).</Paragraph>
      <Paragraph position="6"> For example, the operations of head grammars can be define with the equations1:</Paragraph>
      <Paragraph position="8"> Thus, we have wrap( (ab, ca), (ac, bc) ) = (abac, bcca) concl( (ab, ca), (ac, bc) ) = (ab, caaebc) conc2( (ab, ca), (ac, be)) = (abcaac, be) A generalized context-free grammar (gcfg) \[8\] is denoted G = (VN, S, F, P) where  S is a distinguished member of VN, F is a finite set of function symbols and P is a finite set of productions of the form A --+ f(A1,..., A,) where n &gt; 0, f C F, and A, AI,...,Am C VN. With a grammatical formalism we associate an interpretation function m that maps symbols in F onto the formalism's composition operations. For example, in a typical head grammar the set F might include { W, el, C2} where re(W) = wrap, m(Cl) = concl and re(C2) = conc2.</Paragraph>
      <Paragraph position="9"> A formalism is a linear context-free rewriting system (lefts) if every grammar can be expressed as a gcfg and its interpretation function m maps symbols onto operations whose yield functions are linear regular operations.</Paragraph>
      <Paragraph position="10"> In order to simplify the remaining discussion we assume that m maps directly onto the yield functions themselves.</Paragraph>
      <Paragraph position="11"> The language L(G) generated by a gcfg G =</Paragraph>
      <Paragraph position="13"> We denote the class of all languages generated by lefrs as LCFRL.</Paragraph>
      <Paragraph position="14"> Deterministic Tree-Walking Transducers null A deterministic tree-walking transducer is an automaton whose inputs are derivation trees of some context-free grammar. The automaton moves around the tree starting at the root. At each point in the computation, depending on the label of the current node and the state of the finite state control, the automaton moves  up, down or stays at the current node and outputs a string. The computation ends when the machine tries to move to the parent of the root node.</Paragraph>
      <Paragraph position="15"> We denote a deterministic tree-walking transducer (dtwt) by M - (Q, G, A, 6, q0, F) where Q is a finite set of states, G = (VN, VT, S, P) is a context-free grammar without e-rules, A is a finite set of output symbols,</Paragraph>
      <Paragraph position="17"> A configuration of M is a 4-tuple (q, 7, r/, w) where q E Q is the current state, 7 is the derivation tree of G under consideration, r/is a node in 7 or T (where 1&amp;quot; can be thought of as the parent of the root ofT), and w E A* is the output string produced up to that point in the computation. We have (q, 7, r/, w) \['-M (qt, &amp;quot;\[, r/,, WW/) if the label of r/is X, ~f(q, X) = (q', d, w') such that when d = stay then T/' = r/, when d = d(i) then 7/' is the ith child of r/(if it exists), and when d = up then r/' is the parent of r/(T if r/is the root of 7).</Paragraph>
      <Paragraph position="18"> The output language OUT(M) of M is the set of strings: {weA*I (q0,7, r/r, e) b~/ (q f, 7, T, w), ql E F and 7 is a derivation tree of G with root r/r } where F-~ is the reflexive transitive closure of \['-M&amp;quot; We denote the class of all languages OUT(M) where M is a dtwt as OUT(DTWT).</Paragraph>
      <Paragraph position="19"> Consider the dtwt</Paragraph>
      <Paragraph position="21"> It can be seen that OUT(M) = { anbnc'~d '~ In &gt; 1 }.</Paragraph>
      <Paragraph position="22"> Equivalence In this section we outline a two part proof that</Paragraph>
      <Paragraph position="24"> Consider a dtwt M = (Q, E, G, A, 6, qo, F) where G = (VN, VT, S, P). For convenience we assume that M is a dtwt without stay moves (see Lemma 5.1 in \[3\] for proof that this can be done).</Paragraph>
      <Paragraph position="25"> Given a derivation tree of G, and a node r/in this tree, we record the strings contributed to the output between the first and last visit to nodes in the subtree rooted at r/. These contributed terminal strings can be viewed as a k tuple where k is the number of times that the transducer enters and then leaves the subtree.</Paragraph>
      <Paragraph position="26"> For each production X --* X1 ...Xn in P and each</Paragraph>
      <Paragraph position="28"> computations of M that start in state p at a node labelled X that has been expanded using the production X --* X1...Xn. The node labelled A may be visited several times, but each time the machine must be in a different state (otherwise, being deterministic, it would loop indefinitely). The sequence of visits is recorded as a string of states. The component of the rule that is underlined indicates which of the children or parent is currently being visited. The call C((X, a, C/) -~ (Xl, al, il)... (Xn, an, in)) is made when a computation is being simulated in which the node labelled A has been visited \]a\[ times (\[a\[ denotes the length of a) such that on the ith visit the machine was in the state indicated by the ith symbol in a. al,..., an are used in a similar way to encode the state of the machine during visits to each child node. C/ is a string of terms that is used to encode the output produced between the first and last visit to the subtree rooted at the node labelled A. Ultimately, it has the form .tl ....-tk. where each ti encodes the composition of the ith component of the tuple. The notation used for each ti is identical to that used in the equations used to define lefts composition operations given earlier, i.e., each ti is a string of output symbols and x's. il,...,in are used to encode the number of times that a given child has been visited from above. This gives the number of times the sub-tree rooted at that node has been visited and, hence, encodes which component of the tuple was completed most recently. Thus, for each j, 1 _&lt; j _&lt; n, the simulation has moved from the parent to the jth child ij  times. This number is used to determine which component of the tuple derived from the jth node should contribute to the parent's current component. When a move is made from the parent node to the jth child we add the variable xj,/~+x to the term currently being constructed for the parent node. In other words, the next component of the parent output is the ij + l th component of its jth child.</Paragraph>
      <Paragraph position="29"> The call C((X, a, C/) --* (X1, oq, ix)... (Xj, aj, ij)... (Xn, an, in)) sumulates the machine visiting the jth child of a node expanded using the rule X --~ X1 ... Xn.</Paragraph>
      <Paragraph position="30"> From M the gcfg G' is constructed such that G' =</Paragraph>
      <Paragraph position="32"> non-repeating a e Q*} and the procedure C determines P' and F where for each production A -~ X1 ... Xn in P and each p E Q we call C((A,p, .) --* (Xx, c, 0)...(X~, e, 0)) In addition, for each a E VT and each p E Q we call C((a, p, .)) -~ C is defined as follows.</Paragraph>
      <Paragraph position="33"> Case 1.</Paragraph>
      <Paragraph position="34"> C((X, ap, C/) ---* (X1, oq, ix)... (Xn, an, in)) Note that if n = 0 then X E VT, otherwise, X E VN. If 6(p, X) = (q, up, w) then (X, ap) --~ f((Xl, ~1),..., (Xn, otn)) E P' for a new function f E F where re(f) is defined by f((xl,...,mix),..., (xl,.:., mi,)) '= (tl,...,tk) where Cw. = 41 &amp;quot;...&amp;quot; tk'. (note that when ij = 0 for some j then (Xl,..., xij) will appear as e), in addition, for each p' in Q that does not appear in ap call C((X, o~pp', ew.) ---* (Xl, ~1, i1)... (Xn, O~n, in)) Note that * has been placed after ew. This indicates that we have finished with the current component of the tuple.</Paragraph>
      <Paragraph position="36"> Note that if Xj E VT then it is not possible for the machine to move down the tree any further.</Paragraph>
      <Paragraph position="37"> Case 2.</Paragraph>
      <Paragraph position="38"> c((x, C/) --.</Paragraph>
      <Paragraph position="39"> (X1, (~1, il)... (Xj, ajp, ij)... (Xn, an, in)) If 6(p, Xj) = (q, up, w) then call (Xl, al, il)... (Xj, ajp, ij)... (Xn, an, in)) Note that C/ will end with xj,ii and the ijth compoent of the yield at As. will end in w.</Paragraph>
      <Paragraph position="40"> Otherwise, if 6(p, Xj) = (q,d(k), w) then if Xj E VN for each p' in Q and not in aiP call c((x, a, C/) --.</Paragraph>
      <Paragraph position="41"> (Xl, ot\], it)... (Xj, %pp', ij) . .. (Xn, an, in)) This simulates the next visit to this node (which must be from below) in the (guessed) state p'.</Paragraph>
      <Paragraph position="42"> In addition to the productions added by C, include in P~ the production S ~ ---. ( S, qootq! ) for each qi E F and a E Q* such that aootqi is non-repeating and /f(q, S) = (qI, up, w) for some w where q is the last symbol in q0a.</Paragraph>
      <Paragraph position="43"> A complete proof would establish that the following equivalence holds.</Paragraph>
      <Paragraph position="44"> (Aa) ~ (wt,...,w,) if and only if there is a derivation tree 7 of G with root ~?r labelled A such that a = at...an for some al,...,an E Q+ and for each i (1 &lt; i &lt; n) 7, 7, f, where ai = pia\[ = a\['qi for some c~, a~' E Q*. Consider the application of this construction to example the dtwt given earlier. The grammar contains the following productions (where productions containing useless nonterminals have been omitted). (S, qoqlq3) --~ A((A, qoqlq2q3))</Paragraph>
      <Paragraph position="46"> Consider the gcfg G -- (VN, S, F, P) and mapping m that interprets the symbols in F. Without loss of generality we assume that no nonterminal appears more than once on the right of a production and that for each A E VN there is some rank(A) = k such that only k-tuples are derived from A.</Paragraph>
      <Paragraph position="47"> We define a dtwt M = (Q, ~, G ~, liT, 6, qo, F) where G ~ is a context-free grammar that generates derivation trees of G in the following way. A derivation involving the use of a production zr will he represented by a tree whose root is labelled by zr = A --* f(A1,..., Am) with n subtrees encoding the derivations from A1,..., An.</Paragraph>
      <Paragraph position="48"> The roots of these subtrees will be labelled by the n productions used to rewrite the A1,...,An. Let</Paragraph>
      <Paragraph position="50"> The dtwt M walks around a derivation tree 7 of G' in such a way that it outputs the yield of 7. Each subtree of 7 rooted at a node ~/labelled by the production ~r will be visited on k = rank(lhsOr)) occasions by M. During the ith visit to the subtree M will output the ith component of the tuple. We therefore include in Q k states { 1,...,k} that are used to keep track of which tuple is being considered. This will generally involve visiting children of y as determined by the equation used to define function used in 7r. Additional states in Q are used to keep track of these visits as follows. When the lth child of T/ has finished its ruth component, M will move back up to y in state (Az,m). Since no nonterminal appears twice on the right of a production it is possible for M to determine the value of l from At while at y.</Paragraph>
      <Paragraph position="51"> For each production ~r = A --* f(A1,...,An) E P where f is interpreted as the function defined by the equation f((xX,1,.-.,Xl,kl),.-.,(Xnj,..-,Xn,k,))= (tl,...,tk) we include the following components in the definition of 6.</Paragraph>
      <Paragraph position="52"> For each i (1 &lt; i &lt; k) * if ti = wxl,mC/, where w is a possibly empty terminal string then let</Paragraph>
      <Paragraph position="54"> where (q, move, w) is determined as follows. For some unique I we know that B is the lth nonterminal on the right-hand side of 7r. There is a unique ti such that ti = C/lXZ,mwC/2 where w is a possibly empty string of terminals.</Paragraph>
      <Paragraph position="55"> Case 1:C/2 is empty In this case the ith component of the current node is complete. Thus, q = (lhs(r), i) and move = up.</Paragraph>
      <Paragraph position="56"> Case 2:C/2 begins with the variable xv,m, In this case the machine M must find the m'th component of the/'th child. Thus, q = m' and move = d(l'). It should be clear that the start state q0 should be 1 and the set of final states F = { (S, rank(S)) }. A complete proof would involve verifying that the following equivalence holds.</Paragraph>
      <Paragraph position="57"> (Aa) ~ (wl,...,Wn) if and only if there is a derivation tree 7 of G' with root ~r labelled 7r such that lhs(lr) = A and for each i (1 &lt; i &lt; n) (i, 7, ~/r, e) t-~4 ((A, i), 7, t, w~) We apply the construction to the grammar produced in the illustration of the first construction. First, we name the productions of the grammar</Paragraph>
      <Paragraph position="59"> The construction gives a machine in which the function 5 is defined as follows.</Paragraph>
      <Paragraph position="60">  di(1, rl) = (1, d(1), e) &amp;(1, ~r2) = (1, d(1), a) 5(2, ~2) = (2, d(1), e) 5(1, 7rz) = (1, d(1), a) 5(2, r3) = (2, d(1), c) 5(1, ~r4) = ((e, 1), up, e) 5(2, ~,) = fie, 2), up, ~)</Paragraph>
      <Paragraph position="62"> The context-free grammar whose derivation trees are to be transduced has the following productions.</Paragraph>
      <Paragraph position="63"> &amp;quot;;l'l &amp;quot;~ 71&amp;quot;2 7l'1 -&amp;quot;+ 7r3 We denote a hypergraph as a five tuple H ( V, E, ~, incident, label) where V is a finite set of nodes, E is a finite set of edges, E is a finite set of edge labels,</Paragraph>
      <Paragraph position="65"> A string can be encoded with a string hypergraph \[5\]. The string bcaab is encoded with the following graph.</Paragraph>
      <Paragraph position="66"> 71&amp;quot;2 ~ 71&amp;quot;2 71&amp;quot;2 ~ 71&amp;quot;3 71&amp;quot;3 ~ 7i'4 Context-Free Hypergraph Grammars In this section we describe context-free hypergraph gramars since they are an example of a lcfrs involving the manipulation of graphs, zThe class of string languages generated by context-free hypergraph grammars is equal to OUT(DTWT) \[3\] and the above result shows that they are also equal to LCFRS.</Paragraph>
      <Paragraph position="67"> A directed hypergraph is similar to a standard graph except that its (hyper)edges need not simply go from one node to another but may be incident with any number of nodes. If an edge is incident with n nodes then it is a n-edge. The n nodes that are incident to some edge are linearly ordered. For example, in the figure below, dots denote nodes and labelled square boxes are edges. The edge labelled a is a 3edge, the edge labelled b is a 2-edge and the edge labelled c is a 1-edge. When the number of nodes incident to an edge exceeds 2, numbered tentacles are used to indicate the nodes that are incident to the edge. The numbers associated with the tentacles coming from an edge indicate the linear order of the nodes that are incident to that edge. 2-edges are shown in the standard way and 1-edges can be used as a way of associating labels with nodes as shown.</Paragraph>
      <Paragraph position="68"> @  b c a a b We denote a context-free hypergraph grammar (cfhg) as four tuple G = (VN, VT, S, P) where VN is a finite nonterminal alphabet, VT is a finite terminal alphabet, S E VN is the initial nonterminal and P is a finite set of productions e -* H where H = (V, E, VN O VT, incident, label) is a hypergraph and e E E is a nonterminal edge in H, i.e., label(e) E VN. Consider the application of a production e --* H to a graph H ~ at a node e p in H ~ with the same nonterminal label as e. The resulting graph is obtained from H ~ by replacing e ~ by the graph H with e removed from it. This involves merging of nodes. In particular, the ith node incident with e is merged with the ith node incident with e ~. We require that all edges with the same label have the same number of incident nodes. A derivation begins with a graph containing a single edge labelled S and no edges. A derivation is completed when there are no nonterminal nodes in the graph.</Paragraph>
      <Paragraph position="69"> The string language associated with a cfhg G is denoted STR(G). The class of languages generated by all cfhg is denoted STR(CFHG).</Paragraph>
      <Paragraph position="70"> Due to lack of space, rather than a complete formal definition of cfhg derivations, we present an illustrative example. Consider the three productions shown below. Note that the edge on the left-hand-side of the production is indicated with a double box.</Paragraph>
      <Paragraph position="71">  Below we show the steps in a derivation of the string aabbccdd involving these productions. Note that the set of graphs derived corresponds to the string lan- null It is clear from their definition that cfhg satisfy the conditions for being a lcfrs given earlier. As has been observed \[3\] it is possible to represent the set of derivations of a given cfhg with a set of trees that can be generated by a context-free grammar. The composition operation of cfhg in which a node is replaced by a graph is clearly size-preserving since it does not involve duplication or deletion of an unbounded number of nodes or edges.</Paragraph>
      <Paragraph position="72"> Additional Remarks We end by elaborating on the relationship between lcfrs, dtwt and cfhg in terms of the following complexity measures.</Paragraph>
      <Paragraph position="73"> * The maximum of rank(A) nonterminals A of a gcfg. Let LCFRLk be the class of languages generated by gcfg of some lcfrs whose nonterminals have rank k or less, i.e., derive at most k tuples. * The crossing number of a dtwt M. This is the maximum number of times that it visits any given subtree of an input tree. Let OUT(DTWTk) be the class of languages output by dtwt whose crossing number does not exceed k.</Paragraph>
      <Paragraph position="74"> * The maximum number of tentacles of the nonterminals of a cfhg. Let STR(CFI-IGk) be the class of languages associated with cfhg whose nonterminals have at most k tentacles.</Paragraph>
      <Paragraph position="75"> It has been shown (Theorem 6.1 in \[3\]) that</Paragraph>
      <Paragraph position="77"/>
    </Section>
  </Section>
class="xml-element"></Paper>