File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/85/j85-4001_metho.xml
Size: 42,825 bytes
Last Modified: 2025-10-06 14:11:40
<?xml version="1.0" standalone="yes"?> <Paper uid="J85-4001"> <Title>ON THE COMPLEXITY OF ID/LP PARSING 1</Title> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> 3 THE ADVANTAGES OF SHIEBER'S ALGORITHM </SectionTitle> <Paragraph position="0"> The first question to ask is whether Shieber's algorithm saves anything. Is it faster to use Shieber's algorithm on a UCFG than to use Earley's algorithm on the corresponding expanded CFG? Consider the UCFG G1 that has only the single rule S -* abcde. The corresponding CFG Grl has 120 rules spelling out all the permutations of abcde: S -~ abcde, S -- abced, and so forth. If the string abcde is parsed using Shieber's algorithm directly on G1, the state sets of the parser remain small. 6 S O : \[S &quot;-&quot; { } &quot; {a,b,c,d,e}, O\] S 1 : \[S ~ {a} * {b,c,d,e}, O\] S z : \[S -*. {a,b} * {c,d,e}, O\] S 3 &quot; \[S ~ {a,b,c} * {d,e}, O\] 3 4 : \[S -~ {a,b,c,d} * {e}, 0\] S 5 : \[S ~ {a,b,c,d,e} * { }, O\] In contrast, consider what happens if the same string is parsed using Earley's algorithm on the expanded CFG with its 120 rules. As Figure 1 illustrates, the state sets of the Earley parser are much larger. In state set S~, the Earley parser uses 4! = 24 states to spell out all the possible orders in which the remaining symbols {b,c,d,e} could appear. Shieber's modified parser does not spell them out, but uses the single state \[S ~ {a} * {b,c,d,e}, O\] to summarize them all. Shieber's algorithm should thus be faster, since both parsers work by successively processing all of the states in the state sets.</Paragraph> <Paragraph position="1"> Similar examples show that the Shieber parser can enjoy an arbitrarily large advantage over the use of the Earley parser on the expanded CFG. Instead of multiplying out all surface appearances ahead of time to produce an expanded CFG, Shieber's algorithm works out the possibilities one step at a time, as needed. This can be an advantage because not all of the possibilities may arise with a particular input.</Paragraph> <Paragraph position="3"> enjoy a large advantage over the use of the Earley parser on the corresponding expanded CFG. After having processed the terminal a while parsing the string abcde as discussed in the text, the Shieber parser uses the single state shown in (a) to keep track of the same information for which the Earley parser uses the 24 states in (b).</Paragraph> </Section> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> 4 COMBINATORIAL EXPLOSION WITH SHIEBER'S ALGORITHM </SectionTitle> <Paragraph position="0"> The answer to the first question is yes, then: it can be more efficient to use Shieber's parser than to use the Earley parser on an expanded object grammar. The second question to ask is whether Shieber's parser always enjoys a large advantage. Does the algorithm blow up in difficult cases? In the presence of lexical ambiguity, Shieber's algorithm can suffer from combinatorial explosion. Consider the following UCFG, Gz, in which x is five-ways ambig- null string xxxxa according to this grammar? After the first three occurrences of x have been processed, the state set of Shieber's parser will reflect the possibility that any three of the phrases A, B, C, D, and E might have been encountered in the input and any two of them might 5 remain to be parsed. There will be (q) = 10 states reflecting progress through the rule expaffding S, in addition to 5 states reflecting phrase completion and 10 states reflecting phrase prediction (not shown): Computational Linguistics, Volume 11, Number 4, October-December 1985 207 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing In cases like this, Shieber's algorithm enumerates all of the combinations of k elements taken i at a time, where k is the rule length and i is the number of elements already processed. Thus it can be combinatorially explosive.</Paragraph> <Paragraph position="1"> It is important to note that even in this case Shieber's algorithm wins out over parsing the expanded CFG with Earley's algorithm. After the same input symbols have been processed, the state set of the Earley parser will reflect the same possibilities as the state set of the Shieber parser: any three of the required phrases might have been located, while any two of them might remain to be parsed. However, the Earley parser has a less concise representation to work with. In place of the state involv-</Paragraph> <Paragraph position="3"> and so forth. Instead of a total of 25 states, the Earley state set will contain 135 = 12 * 10 + 15 states. 7 In the above case, although the parser could not be sure of the eategorial identities of the phrases parsed, at least there was no uncertainty about the number of phrases and their extent. We can make matters even worse for the parser by introducing uncertainty in those areas as well. Let G3 be the result of replacing every x in Gz with the empty string e: Then an A, for instance, can be either an a or nothing. Before any input has been read, the first state set So in Shieber's parser must reflect the possibility that the correct parse may include any of the 25 = 32 possible subsets of A, B, C, D, and E as empty initial constituents. For example, S O must include \[S --- {A,B,C,D,E} * {}, 0\] because the input might turn out to be the null string.</Paragraph> <Paragraph position="4"> Similarly, it must include \[S -~ {A,C,E} * {B,D}, O\] because the input might turn out to be bd or db. Counting all possible subsets in addition to other states having to do with predictions, completions, and the parser's start symbol, there are 44 states in S O . (There are 338 states in the corresponding state when the expanded CFG Gt3 is used.)</Paragraph> </Section> <Section position="6" start_page="0" end_page="0" type="metho"> <SectionTitle> 5 THE SOURCE OF THE DIFFICULTY </SectionTitle> <Paragraph position="0"> Why is Shieber's algorithm potentially exponential in grammar size despite its &quot;close relation&quot; to Earley's algorithm, which has time complexity polynomial in grammar size? The answer lies in the size of the state space that each parser uses. Relative to grammar size, Shieber's algorithm~ involves a much larger bound than Earley's algorithm on the number of states in a state set. Since the main task of the Earley parser is to perform scan, predict, and complete operations on the states in each state set (Eariey 1970:97), an explosion in the size of the state sets will be fatal to any small runtime bound.</Paragraph> <Paragraph position="1"> Given a CFG Go, how many possible dotted rules are there? Resulting from each rule X -- A t ... Ak, there are k+ 1 possible dotted rules. Then the number of possible dotted rules is bounded by I G I, if this notation is taken to mean the number of symbols that it takes to write G down. An Earley state is a pair \[r,i\], where r is a dotted rule and i is an interword position ranging from 0 to the length n of the input string. Because of these limits, no state set in the Earley parser can contain more than O( I Q l'n) (distinct) states.</Paragraph> <Paragraph position="2"> The limited size of a state set allows an O( I Go 12 * n3) bound to be placed on the runtime of the Earley parser.</Paragraph> <Paragraph position="3"> Informally, the argument (due to Earley) runs as follows.</Paragraph> <Paragraph position="4"> The scan operation on a state can be done in constant time; the scan operations in a state set thus contribute no more than O( \[ Ga \[ &quot; n) computational steps. All of the predict operations in a state set taken together can add no more states than the number of rules in the grammar, bounded by I G al, since a nonterminal needs to be expanded only once in a state set regardless of how many times it is predicted; hence the predict operations need not take more than O( I G I &quot; n+ \[ G I ) = o( I G I * n) steps. Finally, there are the complete operations to be considered. A given completion can do no worse than advancing every state in the state set indicated by the return pointer. Therefore, any bound k on state set size leads to a bound of k z on the number of steps it takes to do all the completions in a state set. Here k = O( \[ G I'n), so the complete operations in a state set can take at most O( I G I z. n 2) steps. Overall, then, it takes no more than O( I G 12 * n 2) steps to process one state set and no more than O(IG a \[ z * n 3) steps for the Earley parser to process them all.</Paragraph> <Paragraph position="5"> In Shieber's parser, though, the state sets can grow much larger relative to grammar size. Given a UCFG Gb, how many possible dotted UCFG rules are there? Resulting from a rule X -~ A 1 ... A k, there are not k+l possible dotted rules tracking linear advancement, but 2 k possible dotted UCFG rules tracking accumulation of set elements.</Paragraph> <Paragraph position="6"> In the worst case, the grammar contains only one rule and k is on the order of I Gel; hence the number of possible dotted UCFG rules for the whole grammar is not bounded by I Gbl, but by 216b I. (The bound can be reached; recall that exponentially many dotted rules are created in the processing of G 3 from section 4.) 208 Computational Linguistics, Volume 11, Number 4, October-December ! 985 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing Informally speaking, the reason why Shieber's parser sometimes suffers from combinatorial explosion is that there are exponentially more possible ways to progress through an unordered rule expansion than an ordered one. When disambiguating information is scarce, the parser must keep track of all of them. In the more general task of parsing ID/LP grammars, the most tractable case occurs when constraint from the LP relation is strong enough to force a unique ordering for every rule expansion. Under such conditions, Shieber's parser reduces to Earley's. However, the case of strong constraint represents the best case computationally, rather than the worst case as Shieber ( 1983:14) claims.</Paragraph> </Section> <Section position="7" start_page="0" end_page="0" type="metho"> <SectionTitle> 6 ID/LP PARSING IS INHERENTLY DIFFICULT </SectionTitle> <Paragraph position="0"> The worst-case time complexity of Shieber's algorithm is exponential in grammar size rather than quadratic as Shieber (1983:15) believed. Did Shieber simply choose a poor algorithm, or is ID/LP parsing inherently difficult in the general case? In fact, the simpler problem of recognizing sentences according to a UCFG is NP-complete. 8 Consequently, unless ~ = ,./F~, no algorithm for ID/LP parsing can have a runtime bound that is polynomial in the size of the grammar and input.</Paragraph> <Paragraph position="1"> The proof of NP-completeness involves reducing the vertex cover problem (Garey and Johnson 1979:46) to the UCFG recognition problem. Through careful construction of the grammar and input string, it is possible to &quot;trick&quot; the parser into solving a known hard problem. The vertex cover problem involves finding a small set of vertices in a graph with the property that every edge of the graph has at least one endpoint in the set.</Paragraph> <Paragraph position="2"> Figure 2 shows a trivial example.</Paragraph> <Paragraph position="3"> To construct a grammar that encodes the question of whether the graph in Figure 2 has a vertex cover of size 2, first take the vertex names a, b, c, and d as the alphabet. Take START as the start symbol. Take H 1 through H 4 as special symbols, one per edge; also take U and D as special dummy symbols.</Paragraph> <Paragraph position="4"> Next, write the rules corresponding to the edges of the graph. Edge e a runs from a to c, so include the rules H~ -, a and H t -, c. Encode the other edges similarly.</Paragraph> <Paragraph position="5"> Rules expanding the dummy symbols are also needed.</Paragraph> <Paragraph position="6"> Dummy symbol D will be used to soak up excess input symbols, so D -, a through D -, d should be rules.</Paragraph> <Paragraph position="7"> Dummy symbol U will also be used to soak up excess input symbols, but U will be allowed to match only when there are four occurrences in a row of the same symbol (one occurrence for each edge). Take U ~ aaaa, U -, bbbb, U -, cccc, and U -~ dddd as the rules expanding U.</Paragraph> <Paragraph position="8"> Now, what does it take for the graph to have a vertex cover of size k = 2? One way to get a vertex cover is to go through the list of edges and underline one endpoint of each edge. If the vertex cover is to be of size 2, the underlining must be done in such a way that only two distinct vertices are ever touched in the process. Alternatively, since there are 4 vertices in all, the vertex cover will be of size 2 if there are 4-2=2 vertices left untouched in the underlining process. This method of finding a vertex cover can be translated into a UCFG rule as follows: START--,. HIH2H3H4UUDDDD That is, each H-symbol is supposed to match the name of one of the endpoints of the corresponding edge, in accordance with the rules expanding the H-symbols.</Paragraph> <Paragraph position="9"> Each U-symbol is supposed to correspond to a vertex that was left untouched by the H-matching, and the D-symbols are just there for bookkeeping. Figure 3 lists the complete grammar that encodes the vertex-cover problem of Figure 2.</Paragraph> <Paragraph position="11"> text transforms the vertex-cover problem of Figure 2 into this UCFG. A parse exists for the string aaaabbbbccccdddd iff the graph in the previous figure has a vertex cover of size < 2.</Paragraph> <Paragraph position="12"> To make all of this work properly, take o = aaaabbbbccccdddd as the input string to be parsed. (In general, for every vertex name x, include in o a contiguous run of occurrences of x, one occurrence for each edge in the graph.) The grammar encodes the underlining procedure by requiring each H-symbol to match one of its endpoints in o. Since the right-hand side of the START rule is unordered, the grammar allows an H-symbol to match anywhere in the input, hence to match any vertex name (subject to interference from other rules that have already matched). Furthermore, since there is one occur-Computational Linguistics, Volume 11, Number 4, October-December 1985 209 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing rence of each vertex name for every edge, all of the edges could conceivably be matched up with the same vertex; that is, it's impossible to run out of vertex-name occurrences. Consequently, the grammar will allow either endpoint of an edge to be &quot;underlined&quot;. The parser will have to figure out which endpoints to choose - in other words, which vertex cover to select. However, the grammar also requires two occurrences of U to match somewhere. U can only match four contiguous identical input symbols that have not been matched in any other way, and thus if the parser chooses a vertex cover that is too large, the U-symbols will not match and the parse will fail. The proper number of D-symbols is given by the length of the input string, minus the number of edges in the graph (to account for the /-/~-matches), minus k times the number of edges (to account for the U-matches): in this case, 16 - 4 - (2.4) = 4, as illustrated in the START rule.</Paragraph> <Paragraph position="13"> The net result of this construction is that in order to decide whether o is in the language generated by the UCFG, the parser must in effect search for a vertex cover of size 2 or less. 9 If a parse exists, an appropriate vertex cover can be read off from beneath the H-symbols in the parse tree; conversely, if an appropriate vertex cover exists, it indicates how to construct a parse. Figure 4 shows the parse tree that encodes a solution to the vertex-cover problem of Figure 2.</Paragraph> <Paragraph position="14"> The construction shows that vertex-cover problem is reducible to UCFG recognition. Furthermore, the construction of the grammar and input string can be carried out in polynomial time. Consequently, UCFG recognition and the more general task of 1D/LP parsing must be computationally difficult. For a more careful and detailed treatment of the reduction and its correctness, see the appendix.</Paragraph> </Section> <Section position="8" start_page="0" end_page="0" type="metho"> <SectionTitle> 7 COMPUTATIONAL IMPLICATIONS </SectionTitle> <Paragraph position="0"> The reduction of Vertex Cover shows that 'the ID/LP parsing problem is NP-complete. Unless ~ = ,./V~P, the time complexity of ID/LP parsing cannot be bounded by any polynomial in the size of the grammar and input, ldeg An immediate conclusion is that complexity analysis must be done carefully: despite its similarity to Earley's algorithm, Shieber's algorithm does not have complexity O( I G 12 . n3). For some choices of grammar and input, its internal structures undergo exponential growth. Other consequences also follow.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 7.1 PARSING THE OBJECT GRAMMAR </SectionTitle> <Paragraph position="0"> Even in the face of its combinatorially explosive worst-case behavior, Shieber's algorithm should not be immediately cast aside. Despite the fact that it sometimes blows up, it still has an advantage over the alternative of parsing the expanded object grammar. One interpretation of the NP-completeness result is that the general case of ID/LP parsing is inherently difficult; hence it should not be surprising that Shieber's algorithm for solving that problem can sometimes suffer from combinatorial explosion. More significant is the fact that parsing with the expanded CFG blows up in cases that should not be difficult. There is nothing inherently difficult about parsing the language that consists of all permutations of the string abcde, but while parsing that language the Earley parser can use 24 states or more to encode what the Shieber parser encodes in only one (section 3). To put the point another way, the significant fact is not that the Shieber parser can blow up; it is that the use of an expanded CFG blows up unnecessarily.</Paragraph> <Paragraph position="1"> aaaabbbbccccdddd according to this parse tree. The vertex cover {c,d} can be read off from the parse tree as the set of elements dominated by H-symbols.</Paragraph> <Paragraph position="2"> 210 Computational Linguistics, Volume 11, Number 4, October-December 1985 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing</Paragraph> </Section> <Section position="2" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 7.2 IS PRECOMPILATION POSSIBLE? </SectionTitle> <Paragraph position="0"> The present reduction of Vertex Cover to ID/LP Parsing involves constructing a grammar and input string that both depend on the problem to be solved. Consequently, the reduction does not rule out tlae possibility that through clever programming one might concentrate most of the computational difficulty of ID/LP parsing into a separate precompilation stage, dependent on the grammar but independent of the input. According to this optimistic scenario, the entire procedure of preprocessing the grammar and parsing the input string would be as difficult as any NP-complete problem, but after precompilation, the time required for parsing a particular input would be bounded by a polynomial in grammar size and sentence length.</Paragraph> <Paragraph position="1"> Regarding the case immediately at hand, Shieber's modified Earley algorithm has no precompilation step. 11 The complexity result implied by the reduction thus applies with full force; any possible precompilation phase has yet to be proposed. Moreover, it is by no means clear that a clever precompilation step is even possible; it depends on exactly how I GI and n enter into the complexity function for ID/LP parsing. If n enters as a factor multiplying an exponential, precompilation cannot help enough to ensure that the parsing phase will run in polynomial time.</Paragraph> <Paragraph position="2"> For example, suppose some parsing problem is known to require 2161 * n 3 steps for solution. 12 If one is willing to spend, say, 10 * 2161 steps in the precompilation phase, is it possible to reduce parsing-phase complexity to something like I GI 8 * n3? The answer is no. Since by hypothesis it takes at least 2161 * n 3 steps to solve the problem, there must be at least 2161 * n 3 -- 10 * 2161 steps left to perform after the precompilation phase. The parameter n is necessarily absent from the precompilation complexity, hence the term 2161 * n 3 will eventually dominate. null In a related vein, suppose the precompilation step is conversion from ID/LP to CFG form and the runtime step is the use of the Earley parser on the expanded CFG.</Paragraph> <Paragraph position="3"> Although the precompilation step does a potentially exponential amount of work in producing G p from G, another exponential factor still shows up at runtime because I G p I in the complexity bound I G r 1 2 &quot; n3 is exponentially larger than the original I G I *</Paragraph> </Section> </Section> <Section position="9" start_page="0" end_page="0" type="metho"> <SectionTitle> 7.3 POLYNOMINAL-TIME PARSING OF A FIXED GRAMMAR </SectionTitle> <Paragraph position="0"> As noted above, both grammar and input in the current vertex-cover reduction depend on the vertex-cover problem to be solved. The NP-completeness result would be strengthened if there were a reduction that used the same fixed grammar for all vertex-cover problems, for it would then be possible to prove that a precompilation phase would be of little avail. However, unless ~ = ,./t'~, it is impossible to design such a reduction. Since grammar size is not considered to be a parameter of a fixed-grammar parsing problem, the use of the Earley parser on the object grammar constitutes a polynomial-time algorithm for solving the fixed-grammar ID/LP parsing problem.</Paragraph> <Paragraph position="1"> Although ID/LP parsing for a fixed grammar can therefore be done in cubic time, that fact represents little more than an accounting trick. The object grammar G p corresponding to a practical ID/LP grammar would be huge, and if I G'I 2 * n 3 complexity is too slow, then it remains too slow when I G r 1 2 is regarded as a constant.</Paragraph> <Paragraph position="2"> The practical irrelevance of polynomial-time parsing for a fixed grammar sheds some light on another question that is sometimes asked. Can't we have our cake and eat it too by using the ID/LP grammar G directly when we want to see linguistic generalizations, but parsing the object grammar G ~ when we want efficient parsing? After all, the Earley algorithm runs in cubic time based on the length of the input string, and its dependence on grammar size is only I G r 1 2 Essentially, the answer is that using the object grammar doesn't help. The reduction shows that it's not always easy to process the ID/LP form of the grammar, but it is no easier to use the Earley algorithm on the expanded form. As the examples that have been presented clearly illustrate, both the Shieber parser and the Earley parser for a given language can end up with state sets that contain large numbers of elements. The object grammar does not promote efficient processing; the Shieber parser operating on the ID/LP grammar can often do better than the Earley parser operating on the object grammar, be.cause of its more concise representation (section 4).</Paragraph> <Paragraph position="3"> The Earley-algorithm grammar-size factor I G r 1 2 looks smaller than the Shieber factor 2161 until one recalls that G ~ can be exponentially larger than G. In other words, we can hide the factor 2161 inside I Grl, but that doesn't make it any smaller. Thus parsing is likely to take a great many steps even if we parse the object grammar, which might mistakenly be thought to be more efficient than direct parsing. If an algorithm runs too slowly, it doesn't make it faster if we cover up the exponential factor and make it a constant K, and it's unlikely that ID/LP parsing can be done quickly in the general case.</Paragraph> </Section> <Section position="10" start_page="0" end_page="0" type="metho"> <SectionTitle> 7.4 THE POWER OF THE UCFG FORMALISM </SectionTitle> <Paragraph position="0"> The Vertex Cover reduction also helps pin down the computational power of the UCFG formalism. As G 1 and Grl in section 3 illustrated, a UCFG (or an ID/LP grammar) can enjoy considerable brevity of expression compared to the equivalent CFG. The NP-completeness result illuminates this property in two ways. First, the result shows that this brevity of expression is sufficient to allow an instance of any problem in ,Tg~ to be stated in a UCFG that is only polynomiaUy larger than the original problem instance. In contrast, if an attempt is made to replicate the current reduction with a CFG rather than UCFG, the necessity of spelling out all the orders in which the H-, U-, and D-symbols might appear makes Computational Linguistics, Volume 11, Number 4, October-December 1985 211 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing the CFG more than polynomially larger than the problem instance. Consequently, the reduction fails to establish NP-completeness, which indeed does not hold. Second, the result shows that the increased expressive power does not come free; while the CFG recognition problem can be solved in time O( \[ G lZ'n 3) unless ~ = ,.,F~, the general UCFG recognition problem cannot be solved in polynomial time.</Paragraph> <Paragraph position="1"> The details of the reduction show how powerful a single UCFG rule can be. If the UCFG formalism is extended to permit ordinary CFG rules in addition to rules with unordered expansions, the grammar that expresses a vertex-cover problem needs only one UCFG rule, although that rule may need to be arbitrarily long. t3</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 7.5 THE ROLE OF CONSTRAINT </SectionTitle> <Paragraph position="0"> Finally, the discussion of section 5 illustrates the way in which the weakening of constraints can often make a problem computationally more difficult. It might erroneously be thought that weak constraints represent the best case in computational terms, for &quot;weak&quot; constraints sound easy to verify. However, oftentimes the weakening of constraint multiplies the number of possibilities that must be considered in the course of solving a problem. In the case at hand, the removal of constraints on the order in which constituents can appear causes the dependence of parsing complexity on grammar size to grow from \[ G 12 to 21 o l</Paragraph> </Section> </Section> <Section position="11" start_page="0" end_page="0" type="metho"> <SectionTitle> 8 LINGUISTIC IMPLICATIONS </SectionTitle> <Paragraph position="0"> The key factors that cause difficulty in ID/LP parsing are familiar to linguistic theory. GB-theory and GPSG both permit the existence of constituents that are empty on the surface, and thus in principle they both allow the kind of pathology illustrated by G 3 in section 4, subject to amelioration by additional constraints. Similarly, every current theory acknowledges lexical ambiguity, a key ingredient of the vertex-cover reduction. Though the reduction illuminates the power of certain mechanisms and formal devices, the direct implications of the NP-completeness result for grammatical theory are few.</Paragraph> <Paragraph position="1"> The reduction does expose the weakness of attempts to link context-free generative power directly to efficient parsability. Consider, for instance, Gazdar's (1981 : 155) claim that the use of a formalism with only context-free power can help explain the rapidity of human sentence processing: Suppose ... that the permitted class of generative grammars constituted a subset of those phrase structure grammars capable only of generating context-free languages. Such a move would have two important metatheoretical consequences, one having to do with learnability, the other with processability .... We would have the beginnings of an explanation for the obvious, but largely ignored, fact that humans process the utterances they hear very rapidly. Sentences of a context-free language are provably parsable in a time which is proportional to the cube of the length of the sentence or less.</Paragraph> <Paragraph position="2"> As the arguments and examples in this paper have illustrated, context-free generative power does not guarantee efficient parsability. Every ID/LP grammar technically generates a context-free language, but the potentially large size of the corresponding CFG means that we can't count on that fact to give us efficient parsing. Thus it is impossible to sustain this particular argument for the advantages of such formalisms as (early) GPSG over other linguistic theories; instead, GPSG and other modern theories seem to be (very roughly) in the same boat with respect to complexity. In such a situation, the linguistic merits of various theories are more important than complexity results. (See Berwick (1982), Berwick and Weinberg (1982, 1984), and Ristad (1985) for further discussion.) The reduction does not rule out the use of formalisms that decouple ID and LP constraints; note that Shieber's direct parsing algorithm wins out over the use of the object grammar. However, if we assume that natural languages are efficiently parsable (EP), then computational difficulties in parsing a formalism do indicate that the formalism itself does not tell the whole story. That is, they point out that the range of possible languages has been incorrectly characterized: the additional constraints that guarantee efficient parsability remain unstated.</Paragraph> <Paragraph position="3"> Since the general case of parsing ID/LP grammars is computationally difficult, if the linguistically relevant ID/LP grammars are to be efficiently parsable, there must be additional factors that guarantee a certain amount of constraint from some source, t4 (Constraints beyond the bare ID/LP formalism are required on linguistic grounds as well.) Note that the subset principle of language acquisition (cf. Berwiek and Weinberg 1984:233) would lead the language learner to initially hypothesize strong order constraints, to be weakened only in response to positive evidence.</Paragraph> <Paragraph position="4"> However, there are other potential ways to guarantee efficient parsability. It might turn out that the principles and parameters of the best grammatical theory permit languages that are not efficiently parsable in the worst case - just as grammatical theory permits sentences that are deeply center-embedded (Miller and Chomsky 1963). 15 In such a situation, difficult languages or sentences would not be expected to turn up in general use, precisely because they would be difficult to process. 16 The factors that guarantee efficient parsability would not be part of grammatical theory because they would result from extragrammatical factors, i.e. the resource limitations of the language-processing mechanisms. This &quot;easy way out&quot; is not automatically available, depending as it does on a detailed account of processing mechanisms.</Paragraph> <Paragraph position="5"> For example, in the Earley parser, the difficulty of parsing a construction can vary widely with the amount of lookahead used (if any). Like any other theory, an explanation based on resource limitations must make the right predictions about which constructions will be difficult to parse.</Paragraph> <Paragraph position="6"> 212 Computational Linguistics, Volume 11, Number 4, October-December 1985 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing In the same way, the language-acquisition procedure could potentially be the source of some constraints relevant to efficient parsability. Perhaps not all of the languages permitted by the principles and parameters of syntactic theory are accessible in the sense that they can potentially be constructed by the language-acquisition component. It is to be expected that language-acquisition mechanisms will be subject to various kinds of limitations just as all other mental mechanisms are. Again, however, concrete conclusions must await a detailed proposal.</Paragraph> </Section> <Section position="12" start_page="0" end_page="0" type="metho"> <SectionTitle> 9 APPENDIX </SectionTitle> <Paragraph position="0"> This appendix contains the details of a more careful reduction of the vertex-cover problem to the UCFG recognition problem. This version of the reduction establishes that the difficulty of UCFG recognition is not due either to the possibility of empty constituents (e-rules) or to the possibility of repeated symbols in rules (i.e., to the use of multisets rather than sets). Consequently, it is somewhat different from and more complex than the one sketched in the text.</Paragraph> <Paragraph position="1"> (a) A, A', A~, ... denote elements of N; (b) a, a', ai ... denote elements of E; (c) X, Y, X ~, Y, X, Y, .... denote elements of N U E; (d) o, u, u', ui, ... denote elements of E*; (e) a,/3, ~,, 4, ~, denote elements of (NU E)*.</Paragraph> <Paragraph position="2"> Definition G = (N, E, R, S) is e-free iff for every (,4, a) ~R, I~1 ~0.</Paragraph> <Paragraph position="3"> Definition: G = (N, E, R, S) is branching iff for some (,4, a>cR, lal >1.</Paragraph> <Paragraph position="4"> Definition: G = (N, E, R, S) is duplicate-free iff for every (,4, a) E R, a = Y1 --- Y and for all id&quot; ~ \[1,n\], Y = Yj iff i=j.</Paragraph> <Paragraph position="5"> Definition: G = (N, E, R, S) is simple iff it is e-free, duplicate-free, and branching.</Paragraph> <Paragraph position="6"> Note. The notion of a simple UCFG is introduced in order to help pin down the source of any computational difficulties associated with UCFGs. For example, since simple UCFGs are restricted to be duplicate-free, a difficulty that arises with simple UCFGs cannot result from the possibility that a symbol may occur more than once on the right-hand side of a rule.</Paragraph> <Paragraph position="7"> Definition: 4A~-->4a~P (by r) just in case (for some) r = G (A', Y1 &quot;&quot; Y) ~ R and for some permutation p of \[1,n\], A = A' and a = Yp<l)... Yp<n). If 4 c E*, also write 4A4,~lm 4aq~.</Paragraph> <Paragraph position="8"> G Definition: L(G) = {o E E*: S ~* o} Definition: An n-step derivation of q~ from 4 is a sequence (4o ..... 4,) such that 40, = 4, 4, = ~P, and for all i ~ \[0, n-l\], 4i ~ 4i+r If it is also true for all i that 4i => lm 4i+1, say that the derivation is leftmost.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 9.2 DEFINING THE COMPUTATIONAL PROBLEMS </SectionTitle> <Paragraph position="0"> Definition: A possible instance of the problem VERTEX COVER is a triple (V,E,k), where (V,E) is a finite graph with at least one edge and at least two vertices, k c N, and k < I V\[ .17 VERTEX COVER itself consists of all possible instances (V,E,k) such that for some 1f ___ V, I U I < k and for all edges e c E, at least one endpoint of e is in l/. (Figure 5 gives an example of a VERTEX</Paragraph> <Paragraph position="2"> COVER. The set I / = {v,x,z,'pf;} is a vertex cover of size k=3.</Paragraph> <Paragraph position="3"> Fact: VERTEX COVER is NP-complete. (Garey and Johnson 1979:46) Definition: A possible instance of the problem SIMPLE UCFG RECOGNITION is a pair (G, a), where G is a simple UCFG and o c Z*. SIMPLE UCFG Computational Linguistics, Volume 11, Number 4, October-December 1985 213 G. Edward Barton, Jr. On the Complexity of ID/LP Parsing RECOGNITION itself consists of all possible instances (G, o) such that o * L(G).</Paragraph> <Paragraph position="4"> Notation: Take \[\[. \[\[ to be any reasonable measure of the encoded input length for a computational problem; continue to use \[. \[ for set cardinality and strihg length. It is reasonable to require that if S is a set, k * H, and I s I > k, then II s \[I > II/~ II, that is, the encoding of numbers is better than unary. It is also reasonable to require that l\[ ( .... x .... ) II _ II x II.</Paragraph> </Section> </Section> <Section position="13" start_page="0" end_page="0" type="metho"> <SectionTitle> 9.3 THE UCFG RECOGNITION PROBLEM IS IN NP </SectionTitle> <Paragraph position="0"> Lemma 9.1: Let (if0 ..... q~) be a shortest leftmost derivation of ~ from ~0 in a branching e-free UCFG. If k > INI+I then Iq,~l > I%1.</Paragraph> <Paragraph position="1"> Proof. There exists some sequence of rules (,4 o, %) ... (Ak_ 1, a~_ 1) such that for all i * \[0, k-l\], ~ => lm q~i+l by (,4 i, %). Since G is e-free, I cPi+l \[ > I <Pi\[ always.</Paragraph> <Paragraph position="2"> Case 1. For some i, I%1 > 1. Then I~+11 > Iq~il.</Paragraph> <Paragraph position="3"> Hence \[ ~kl < \[ ~0 I.</Paragraph> <Paragraph position="4"> Case 2. For every i, I%1 = 1. Then there exist u, 3' such that for every i * \[0, k-2\], there is Ari * N such that ~+1 = u A'~ 3'. Suppose the A'~ are all distinct. Then IN\[ > k-l, hence INI+I > k, hence INI+I > I NI+I, which is impossible. Hence for some ij * \[0, k-2\], i < j, Ar~ = A~j. Hence ~+~ = q,j+~, since \[1,1\] has only one permutation. Then ~0 ..... ff~, q~j+l ..... q~k) is a leftmost derivation of ~ from ~0 and has length less than k, which is also impossible.</Paragraph> <Paragraph position="5"> Lemma 9.3: H = SIMPLE UCFG RECOGNITION is in the computational class ,/F~P.</Paragraph> <Paragraph position="6"> Proof. Let G = (N, X, R, S) be a simple UCFG and o * Y.*. Consider the following nondeterministic algorithm with input (G, o): Step 1. Write down ~0 = S.</Paragraph> <Paragraph position="7"> Step 2. Perform the following steps for i from 0 to \[ a \[ &quot; m-l, where m = \] NI +2.</Paragraph> <Paragraph position="8"> (a) Express ~i as ur4i7 ~ by finding the leftmost nontermihal, or loop if impossible.</Paragraph> <Paragraph position="9"> (b) Guess a rule (A~, Yt,l -.- Y,,k i) * R and a permutation p~ of \[ 1,k~\], or loop if there is no such rule. (c) Write down ~i+1 = u~ Y~, pi(1) &quot;'&quot; Y/, Pi(/q) 3`i&quot; (d) If t~i+l ---~ O then halt.</Paragraph> <Paragraph position="10"> Step 3. Loop.</Paragraph> <Paragraph position="11"> It should be apparent that the algorithm runs in time at worst polynomial in l\[ (G, o)II; note that the length of ~ increases by at most a constant amount on each iteration. Assume (G, o) * H. Then o has a leftmost derivation of length at most 1o1 &quot; m by Corollary 9.2; hence the nondeterministic algorithm will be able to guess it and will halt. Conversely, suppose the algorithm halts on input (G, o). On the iteration when the algorithm halts, the sequence (~0, ..., ff~+l) will constitute a leftmost derivation of o from S; hence o * L(G) and ( G, o) * H. Then there is a nondeterministic algorithm that runs in polynomial time and accepts exactly H. Hence</Paragraph> </Section> <Section position="14" start_page="0" end_page="0" type="metho"> <SectionTitle> H * JF@.D 9.4 THE UCFG RECOGNITION PROBLEM IS NP-COMPLETE </SectionTitle> <Paragraph position="0"> Lemma 9.4: Let (V,E,k) = (V,{ei} , k) be a possible instance of VERTEX COVER. Then it is possible to construct, in time polynomial in II VII, UEII, and k, a simple UCFG G(V,E,k) and a string o(V,E,k) such that (G(V,E,k), o(V,E,k)) ~ SIMPLE UCFG RECOGNITION iff (V,E,k) * VERTEX COVER.</Paragraph> <Paragraph position="1"> Proof. Construct G(V,E,k) as follows. Let the set N of nonterminals consist of the following symbols not in V: START, U, D, for i* \[1, IEI 1, U~ for i e \[1, I V I-k\], D, for i* \[1, \[El &quot;(k-l)\].</Paragraph> <Paragraph position="2"> II Nil will be at worst polynomial in IIE II, II vii, and k for a reasonable length measure. Define the terminal vocabulary Y to consist of subscripted symbols as follows: Z ={ai'a* V,i* \[1, \[E\[\]}.</Paragraph> <Paragraph position="3"> Designate START as the start symbol. Include the following as members of the rule set R: Take G(V,E,k) to be (N, E, R, START). (Figure 6 shows the results of applying this construction to the VERTEX COVER instance of Figure 5.) Let h : \[1, I V I \] -~ V be some standard enumeration of the elements of V. Construct o(V,E,k) as h(1) t ... h(1)lE I ... h(I VI)1 ... h(J VI)IEI thus o(V,E,k) will have length I EI &quot; I V I.</Paragraph> <Paragraph position="4"> It is easy to see that II (G(V,E,k), o(V,E,k))II will'be at worst polynomial in II Ell, II vii, and k for reasonable II&quot; II. It will also be possible to construct the grammar and string in polynomial time. Finally, note that given the definition of a possible instance of VERTEX COVER, the grammar will be branching, e-free, and duplicate-free, hence simple.</Paragraph> <Paragraph position="5"> Now suppose (V,E,k) * VERTEX COVER. Then there exist 1/ ~ V and f : E -~ 1/such that I I/I < k and for every e * E, f(e) is an endpoint of e. E is nonempty by hypothesis and I/ must hit every edge, hence \[l/\[ cannot be zero. Construct a parse tree for o(V,E,k) according to G(V,E,k) as follows.</Paragraph> <Paragraph position="6"> Step 1. Number the elements of V- 1/ as {x~ : i * \[1, I V-U I\]}. For each x i where i < I Vl-k, construct a node dominating the substring (xi) t ... (x i) I EI of o(V,E,k) and label it U. Then construct a node dominating only the U-node and label it U r Note that the available symbols U/ are numbered from 1 to IV I-k, so it is impossible to run out of U-symbols. Also, J U \[ < k and U---V, hence Iv-l/I = Ivl- I~l > Ivl-k, so all of the U-symbols will be used. Finally, note that U -~ a t ... ale I is a rule for any a * S and that U~ -- Uis a rule for any U r Step 2. For each ei E E, construct a node dominating the (unique) occurrence of f(e)~ ~ o(V,E,k) and label it H. Step 2 cannot conflict with step 1 because f(e) * V t, hence f(ea) C/ V - V'. Different parts of step 2 cannot conflict with each other because each one affects a symbol with a different subscript. Also note that f(e) is an endpoint of ei and that//~ -- a~ is a rule for any e~ e E and a an endpoint of e r Step 3. Number all occurrences of terminals in o(V,E,k) that were not attached in step 1 or step 2. For the ith such occurrence, construct a node dominating the occurrence and label it D. Then construct another node domi-</Paragraph> </Section> class="xml-element"></Paper>