File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/95/p95-1023_metho.xml
Size: 22,967 bytes
Last Modified: 2025-10-06 14:14:05
<?xml version="1.0" standalone="yes"?> <Paper uid="P95-1023"> <Title>TAL Recognition in O(M(n2)) Time</Title> <Section position="4" start_page="166" end_page="167" type="metho"> <SectionTitle> 3 Context Free recognition in </SectionTitle> <Paragraph position="0"/> <Paragraph position="2"> N is a set of Nonterminals {A1, A2, .., Ak}, is a finite set of terminals, P is a finite set of productions, A1 is the start symbol is assumed to be in the Chomsky Normal Form. Valiant (1975) shows how the recognition problem can be reduced to the problem of finding Transitive Closure and how Transitive Closure can be reduced to Matrix Multiplication.</Paragraph> <Paragraph position="3"> Given an input string aza2 .... an E ~*, the recursive algorithm makes use of an (n+l)x (n+l) upper triangular matrix b defined by</Paragraph> <Paragraph position="5"> and proceeds to find the transitive closure b + of this matrix. (If b + is the transitive closure, then Ak E b. +. C/:~ Ak-~ ai .... aj-1) $,J Instead of finding the transitive closure by the customary method based on recursively splitting into disjoint parts, a more complex procedure based on 'splitting with overlaps' is used. The extra cost involved in such a strategy can be made almost negligible. The algorithm is based on the following lemma Lemma : Let b be an n x n upper triangular matrix, and suppose that for any r > n/e, the transitive closure of the partitions \[1 < i,j < r\] and \[n- r < i,j < n\] are known. Then the closure of b can be computed by I. performing a single matrix multiplication, and 2. finding the closure of a 2(n - r) x 2(n - r) upper triangular matrix of which the closure of the partitions\[1 < i,j < n- r\] and \[n- r < i,j < 2(n - r)\] are known.</Paragraph> <Paragraph position="6"> Proof: See (Valiant, 1975)for details The idea behind (Valiant, 1975) is based on visualizing Ak E b+j as spanning a tree rooted at the node Ak with l~aves ai through aj-1 and internal nodes as nonterminals generated from Ak according to the productions in P. Having done this, the following observation is made : Given an input string al...a, and 2 distinct symbol positions, i and j, and a nonterminal Ak such that Ak E b + ., where i' < i,j' > j, then 3 a non-I P3 terminal A k, which is a descendent of Ak in the b + . where tree rooted at Ak, such that A k, E i d' i&quot; < i, j&quot; > j and A k, has two children Ak~ and Ak2 such thatAk~ Eb +, andAk2 Eb +..withi<s<j.</Paragraph> <Paragraph position="7"> A k, can be thought of as a minimal node in this sense.(The descendent relation is both reflexive and transitive) Thus, given a string al...a, of length n, (say r = 2/3), the following steps are done :</Paragraph> <Paragraph position="9"> 1. Find the closure of the first 2/3 ,i.e. all nodes spanning trees which are within the first 2/3 .</Paragraph> <Paragraph position="10"> 2. Find the closure of the last 2/3 , i.e. all nodes spanning trees which are within the last 2/3.</Paragraph> <Paragraph position="11"> 3. Do a composition operation (i.e. matrix multiplication) on the nodes got as a result of Step 1 with nodes got as a result of Step 2.</Paragraph> <Paragraph position="12"> 4. Reduce problem size to az...an/zal+2n/3...an and find closure of this input.</Paragraph> <Paragraph position="13"> The point to note is that in step 3, we can get rid of the mid 1/3 and focus on the remaining problem size.</Paragraph> <Paragraph position="14"> This approach does not work for TALs because of the presence of the adjunction operation.</Paragraph> <Paragraph position="15"> Firstly, the data structure used, i.e. the 2dimensional matrix with the given representation, is not sufficient as adjunction does not operate on contiguous strings. Suppose a node in a tree dominates a frontier which has the substring aiaj to the left of the foot node and akat to the right of the footnode. These substrings need not be a contiguous part of the input; in fact, when this tree is used for adjunction then a string is inserted between these two suhstrings. Thus in order to represent a node, we need to use a matrix of higher dimension, namely dimension 4, to characterize the substring that appears to the left of the footnode and the substring that appears to the right of the footnode.</Paragraph> <Paragraph position="16"> Secondly, the observation we made about an entry</Paragraph> <Paragraph position="18"> of adjunction.</Paragraph> <Paragraph position="19"> Thirdly, the technique of getting rid of the mid 1/3 and focusing on the reduced problem size alone, does not work as shown in figure 3: Suppose 3' is a derived tree in which 3 a node rn on which adjunction was done by an auxiliary tree ft. Even if we are able to identify the derived tree 71 rooted at m, we have to first identify fl before we can check for adjunction, fl need not be realised as a result of the composition operation involving the nodes from the first and last 2/3's ,(say r =2/3). Thus, if we discard the mid 1/3, we will not be able to infer that the adjunction had indeed taken place at node m.</Paragraph> </Section> <Section position="5" start_page="167" end_page="168" type="metho"> <SectionTitle> 4 Notations </SectionTitle> <Paragraph position="0"> Before we introduce the algorithm, we state the notations that will be used.</Paragraph> <Paragraph position="1"> We will be making use of a 4-dimensional matrix A of size (n + 1) x (n + 1) x (n + 1) x (n + 1), where n is the size of the input string.</Paragraph> <Paragraph position="2"> (Vijayashanker, Joshi, 1986) Given a TAG G and an input string aza2..an, n > 1, the entries in A will be nodes of the trees of G. We say, that a node m (= < 0, node index, label >) E A(i,j, k, l) iff m is a node in a derived tree 7 and the subtree of 7 rooted at m has a yield given by either ai+l...ajXak+l...al (where X is the footnode of r/, j < k) or ai+l .... az (when j = k).</Paragraph> <Paragraph position="3"> If a node m E A(i,j,k,l}, we will refer to m as spanning a tree (i,j,k,l).</Paragraph> <Paragraph position="4"> When we refer to a node m being realised as a result of composition of two nodes ml and rnP, we mean that 3 an elementary tree in which m is the parent of ml and m2.</Paragraph> <Paragraph position="5"> A Grown Auxiliary Tree is defined to be either a tree resulting from an adjunction involving two auxiliary trees or a tree resulting from an adjunction involving an auxiliary tree and a grown auxiliary tree.</Paragraph> <Paragraph position="6"> Given a node m spanning a tree (i,j,k,l), we define the last operation to create this tree as follows : if the tree (i,j,k,l) was created in a series of operations, which also involved an adjunction by an auxiliary tree (or a grown auxiliary tree) (i, Jl, kz, l) onto the node m, then we say that the last operation to create this tree is an adjunction operation; else the last operation to create the tree (i,j,k,l) is a composition.</Paragraph> <Paragraph position="7"> The concept of last operation is useful in modelling the steps required, in a bottom-up fashion, to create</Paragraph> <Paragraph position="9"/> </Section> <Section position="6" start_page="168" end_page="169" type="metho"> <SectionTitle> 5 Algorithm </SectionTitle> <Paragraph position="0"> Given that the set of initial and auxiliary trees can have leaf nodes labelled with e, we do some preprocessing on the TAG G to obtain an Association List (ASSOC LIST) for each node. ASSOC LIST (m), where m is a node, will be useful in obtaining chains of nodes in elementary trees which have children la- null 1. If m is a leaf then quit 2. If m has children ml and me both yielding the empty string at their frontiers (i.e. m spans a subtree yielding e) then</Paragraph> <Paragraph position="2"> 3. If m has children m1 and me, with only me yielding the empty string at its frontier, then</Paragraph> <Paragraph position="4"> We initially fill A(i,i+l,i+l,i+l) with all nodes from Smt,Vml, where S,~1 = {ml} O ASSOC LIST (ml), ml being a node with the same label as the input hi+l, for 0 < i < n-1. We also fill A(i,i,j,j), i < j, with nodes from S,~2, Vm2, where Sin2 = {me) tJ ASSOC LIST (me), me being a foot node. All entries A(i,i,i,i), 0 < i < n, are filled with nodes from Sraa,Vm3, where S,n3 = { m3} U ASSOC LIST (mS), m3 having label C/.</Paragraph> <Paragraph position="5"> Following is the main procedure, Compute Nodes, which takes as input a sequence rlr2 ..... rp of symbol positions (not necessarily contiguous). The procedure outputs all nodes spanning trees (i,j,k,O, with {i, 1} E {rl,r2 ..... ~'ip } and {j,k} E {rl,r I Jr Z,...,rp}. The procedure is initially called with the sequence 012..n corresponding to the input string aa ..... an.</Paragraph> <Paragraph position="6"> The matrix A is updated with every call to this procedure and it is updated with the nodes just realised and also with the nodes in the ASSOC LISTs of the nodes just realised.</Paragraph> <Paragraph position="7"> Procedure Compute Nodes ( rl r2 ..... rp ) Begin 1. Ifp = 2, then a. Compose all nodes E A(rl,j, k, re) with all nodes E A(re,re, re, re), rt < j < k < re.</Paragraph> <Paragraph position="8"> Update A .</Paragraph> <Paragraph position="9"> b. Compose all nodes E A(rl,rl,rl,rx) with all nodes E A(rt, j, k, r2), rt < j < k < re.</Paragraph> <Paragraph position="10"> Update A .</Paragraph> <Paragraph position="11"> e. Check for adjunctions involving nodes realised from steps a and b. Update A .</Paragraph> <Paragraph position="12"> d. Return 2. Compute Nodes ( rlr2 ..... rep/a ).</Paragraph> <Paragraph position="13"> 3. Compute Nodes ( rl+p/z ..... rp ).</Paragraph> <Paragraph position="14"> 4. a. Compose nodes realised from step 2 with nodes realised from step 3.</Paragraph> <Paragraph position="15"> b. Update A.</Paragraph> <Paragraph position="16"> 5. a. Check for all possible adjunctions involving the nodes realised as a result of step 4.</Paragraph> <Paragraph position="17"> b. Update A.</Paragraph> <Paragraph position="18"> 6. Compute Nodes ( rlre...rp/arl+2p/a...r p )</Paragraph> <Section position="1" start_page="169" end_page="169" type="sub_section"> <SectionTitle> End </SectionTitle> <Paragraph position="0"> Steps la,lb and 4a can be carried out in the following manner : Consider the composition of node ml with node me. For step 4a, there are two cases to take care of. Case 1 If node ml in a derived tree is the ancestor of the foot node, and node me is its right sibling, such that ml 6 A(i, j, k, l) and m2 E A(l, r, r, s), then their parent, say node m should belong to A(i,j,k,s). This composition of ml with me can be reduced to a boolean matrix multiplication in the following way: (We use a technique similar to the one used in (Rajasekaran, 1995)) Construct two boolean matrices B1, of size ((n 4- 1)2p/3) x (p/3) and Be, of size</Paragraph> <Paragraph position="2"> Clearly the dot product of the ijk th row of B1 with the s th column of Be is a 1 iff m E A(i, j, k, s).</Paragraph> <Paragraph position="3"> Thus, update A(i,j,k, s) with {m} U ASSOC LIST (m).</Paragraph> <Paragraph position="4"> Case 2 If node me in a derived tree is the ancestor of the foot node, and node ml is its left sibling, such that ml E A(i,j,j,l) and m2 E A(l,p, q, r), then their parent, say node m should belong to A(i,p,q,s). This can also be handled similar to the manner described for case 1. Update A(i,p,q,s) with {m} U ASSOC LIST (m).</Paragraph> <Paragraph position="5"> Notice that Case 1 also covers step la and Case 2 also covers step lb.</Paragraph> <Paragraph position="6"> Step 5a and Step lc can be carried out in the following manner : We know that if a node m E A(i,j,k,i), and the root ml of an auxiliary tree E A(r, i, i, s), then adjoining the tree 7/, rooted at ml, onto the node m, results in the node m spanning a tree (rj,k,s), i.e. m E A(r, j, k, s).</Paragraph> <Paragraph position="7"> We can essentially use the previous technique of reducing to boolean matrix multiplication. Construct two matrices C1 and Ce of sizes (p2/9) x (n + 1) 2 and (n + 1) 2 x (n + 1) 2, respectively, as follows : Cl(ii, jk) = 1 iff 3ml, root of an auxiliary tree E A(i, j, k, l), with same label as m and</Paragraph> <Paragraph position="9"> Note that inC2 0<q<r<s<t<n.</Paragraph> <Paragraph position="10"> Clearly the dot product of the ii th row of C1 with the rs th column of Ce is a 1 iff m E A(i,r,s,l). Thus, update A(i, r, s, l) with {m} U ASSOC LIST (m).</Paragraph> <Paragraph position="11"> The input string ala2...an is in the language generated by the TAG G iff 3 a node labelled S in some A(O,j,j,n), 0 <_ j < n.</Paragraph> </Section> </Section> <Section position="7" start_page="169" end_page="169" type="metho"> <SectionTitle> 6 Complexity </SectionTitle> <Paragraph position="0"> Steps la, lb and 4a can be computed in O(neM(p)).</Paragraph> <Paragraph position="1"> Steps 5a and le can be computed in O((ne/pe)eM(pg)).</Paragraph> <Paragraph position="2"> If T(p) is the time taken by the procedure Compute Nodes, for an input of size p, then</Paragraph> <Paragraph position="4"> where n is the initial size of the input string.</Paragraph> <Paragraph position="5"> Solving the recurrence relation, we get T(n) O(M(ne)). null</Paragraph> </Section> <Section position="8" start_page="169" end_page="171" type="metho"> <SectionTitle> 7 Proof of Correctness </SectionTitle> <Paragraph position="0"> We will show the proof of correctness of the algorithm by induction on the length of the sequence of symbol positions.</Paragraph> <Paragraph position="1"> But first, we make an observation, given any two symbol positions (r~, rt), rt > r~ 4-1 , and a node m spanning a tree (i,j, k, l) such that i < rs and i _> rt with j and k in any of the possible combinations as shown in figure 4.</Paragraph> <Paragraph position="2"> 3 a node m' which is a descendent of the node m in the tree (i,j,k,l) and which either E ASSOC LIST(ml) or is the same as ml, with ml having one of the two properties mentioned below : 1. ml spans a tree (il,jl, kl, 11) such that the last operation to create this tree was a composition operation involving two nodes me and m3 with me spanning (ix, J2, k2, 12) and m3 spanning (12,j3, ks, ix). (with (r, < l~. < rt), 01 <- r,), (rt < !1) and either (j2 = kz,j3 = jl,k3 = kl) or (j2 = jl,k2 = kl,j3 = k3) ) 2. ml spans a tree (il,jl, kl, ll) such that the last operation to create this tree was an adjunction by an auxiliary tree (or a grown auxiliary tree) (il, j2, ke, Ix), rooted at node me, onto the node ml spanning the tree (je,jl, kl, k2) such that node me has either the property mentioned in (1) or belongs to the ASSOC LIST of a node</Paragraph> <Paragraph position="4"> of j and k being considered which has the property mentioned in (1). (The labels of ml and me being the same) Any node satisfying the above observation will be called a minimal node w.r.t, the symbol positions (r,, r0.</Paragraph> <Paragraph position="5"> The minimM nodes can be identified in the following manner. If the node m spans (i,j, k, l) such that the last operation to create this tree is a composition of the form in figure ha, then m tO ASSOC LIST(m) is minimal. Else, if it is as shown in figure 5b, we can concentrate on the tree spanned by node ml and repeat the process. But, if the last operation to create (i, j, k, 1) was an adjunction as shown in figure 5c, we can concentrate on the tree (il, j, k, 11) initially spanned by node m. If the only adjunction was by an auxiliary tree, on node m spanning tree (Q,j,k, lx) as shown in figure 5d, then the set of minimal nodes will include both m and the root ml of the auxiliary, tree and the nodes in their respective ASSOC LISTs. But if the adjunction was by a grown auxiliary tree as shown in figure he, then the minimal nodes include the roots of/31,/32, ..,/3s, 7 and the node m.</Paragraph> <Paragraph position="6"> Given a sequence < rl,r2,..,rp >, we call (rq,r~+l) a gap, iff rq+l C/ rq + 1. Identifying minimal nodes w.r.t, every new gap created, will serve our purpose in determining all the nodes spanning trees (i, j, k, 1), with {i, l} e {rl, r2, .., rp}.</Paragraph> <Paragraph position="7"> Theorem : Given an increasing sequence < rl, r2, .., rp > of symbol positions and given a. V gaps (rq, rq+l), all nodes spanning trees (i,j,k,l} with rq < i < j < k < l < rq+l b. V gaps (rq, rq+l), all nodes spanning trees (i,j,k,l) such that either rq < i < rq+l or rq < l < rq+l c. V gaps (rq,rq+l) , all the minimal nodes for the gap such that these nodes span trees (i,j,k,l) with {i,l} E { rl,r2,..,rp } and i <_ 1 in addition to the initialization information, the algorithm computes all the nodes spanning trees For length = 1, it is trivial as this information is already known as a result of initialization.</Paragraph> <Paragraph position="8"> For length = 2, there are two cases to consider : 1. r2 = rl + 1, in which case a composition involving nodes from A(rl, rl, rl, rl) with nodes from A(rl, r2, r2, r2) and a composition involving nodes from A(rl, r2, r2, r2) with nodes from A(r2, r2, r2, r2), followed by a check for adjunction involving nodes realised from the previous two compositions, will be sufficient. Note that since there is only one symbol from the input (namely, ar~), and because an auxiliary tree has at least one label from ~, thus, checking for one adjunction is sufficient as there can be at most one adjunction.</Paragraph> <Paragraph position="9"> 2. r2 ~ rl + 1, implies that (rl,r2) is a gap.</Paragraph> <Paragraph position="10"> Thus, in addition to the information given as per the theorem, a composition involving nodes from A(rl, j, k, r2) with nodes from A(r2,r2, r2,r2) and a composition involving nodes from A(rl,rl,rl,rl) with nodes from A(rl, j, k, r2), (rl < j < k < r2), followed by an adjunction involving nodes realised as a result of the previous two compositions will be sufficient as the only adjunction to take care of involves the adjunction of some auxiliary tree onto a node m which yields e, and m E A(rl, rl, rl, rl) or m E A(r2,r2,r2, r2).</Paragraph> <Paragraph position="11"> Induction hypothesis : V increasing sequence < rl,r2, ..,r~ > of symbol positions of length < p, (i.e q < p), the algorithm, given the information as required by the theorem, computes all nodes spanning trees (i,j,k,l) such that {i, l} e { rl, r2, .., rq } and i < j < k < I. Induction : Given an increasing sequence < rl, r~, .., rp, rp+l > of symbol positions together with the information required as per parts a,b,c of the theorem, the algorithm proceeds as follows: null 1. By the induction hypothesis, the algorithm correctly computes all nodes spanning trees (i,j,k,i) within the first 2/3, i.e, {i,l} E { rt, r2, .., r2(p+D/3 } and i < l . By the hypothesis, it also computes all nodes (i',j,k',l')within the last 2/3, i.e, { i ~, ! ~ } E {rl+(p+l)/3, .., rp+z} and i' < i'.</Paragraph> <Paragraph position="12"> 2. The composition step involving the nodes from the first and last 2/3 of the sequence < rl, r2, .., rp, rp+i >, followed by the adjunction step captures all nodes m such that either a. m spans a tree (i,j,k,l)such that the last operation to create this tree was a composition operation on two nodes ml and m2 with ml spanning (i,j',k;l'} and me spanning null (i;j&quot;,k&quot;,l). (with i E { rl, r2, .., r(p+l)/3 }, i E { rl+(p+l)/3,..,r2(p+D/3 } and I E ! ri+2(p+z)/3, .., rp+z }, and either (j' = k, j&quot; =j, k&quot; = k) or (j' =j, k'= k,j&quot; = k') ). b. m spans a tree O,J, k,l) such that the last operation to create this tree was an adjunction by an auxiliary or grown auxiliary tree (i,j',k',l), rooted at node mI, onto the node m spanning the tree (j',j,k,k') such that node ml has either the property mentioned in (1) or it belongs to the ASSOC LIST of a node which has the property mentioned in (1). (The labels of m and ml being the same) Note that, in addition to the nodes m captured from a or b, we will also be realising nodes E</Paragraph> </Section> <Section position="9" start_page="171" end_page="172" type="metho"> <SectionTitle> ASSOC LIST (m). </SectionTitle> <Paragraph position="0"> The nodes captured as a result of 2 are the minimal nodes with respect to the gap (r(p+l)/a, rl+2(p+l)/3) with the additional property that the trees (i,j,k,l) they span are such that i E { rl, r2, .., r(p+l)\]3 } and l E { rl+2(p+l)\]3, .., rp+l }. Before we can apply the hypothesis on the sequence < rx, r2, .., r(p+t)/3, rl+2(p+l)\[3, ..rp+l >, we have to make sure that the conditions in parts a,b,c of the theorem are met for the new gap (r(p+1)/3, rl+2(p+l)/3). It is easy to see that conditions for parts a and b are met for this gap. We have also seen that as a result of step 2, all the minimal nodes w.r.t the gap (r(p+x)/3 , rl+2(p+l)/3), with the desired property as required in part c have been computed. Thus applying the hypothesis on the sequence < rl, r2, .., r(p+l)\[3, rl+2(p+l)/3, ..rp+l >, the algorithm in the end correctly computes all the nodes spanning trees (ij,k,1) with {i,l} E {rl,r2,..,rp+x } andi<j<k<l. D</Paragraph> </Section> <Section position="10" start_page="172" end_page="172" type="metho"> <SectionTitle> 8 Implementation </SectionTitle> <Paragraph position="0"> The TAL recognizer given in this paper was implemented in Scheme on a SPARC station-10/30.</Paragraph> <Paragraph position="1"> Theoretical results in this paper and those in (Rajasekaran, 1995) clearly demonstrate that asymptotically fast algorithms can be obtained for TAL parsing with the help of matrix multiplication algorithms. The main objective of the implementation was to check if matrix multiplication techniques help in practice also to obtain efficient parsing algorithms. null The recognizer implemented two different algorithms for matrix multiplication, namely the trivial cubic time algorithm and an algorithm that exploits the sparsity of the matrices. The TAL recognizer that uses the cubic time algorithm has a run time comparable to that of Vijayashanker-\]oshi's algorithm. null Below is given a sample of a grammar tested and also the speed up using the sparse version over the ordinary version. The grammar used, generated the TAL anbnc n. This grammar is shown in figure 1.</Paragraph> <Paragraph position="2"> Interestingly, the sparse version is an order of magnitude faster than the ordinary version for strings of length greater than 7.</Paragraph> <Paragraph position="3"> The above implementation results suggest that even in practice better parsing algorithms can be obtained through the use of matrix multiplication techniques.</Paragraph> </Section> class="xml-element"></Paper>