<?xml version="1.0" standalone="yes"?>
<Paper uid="H89-2053">
  <Title>Joshi, Aravind K., 1985. How Much Context-Sensitivity is Necessary for Characterizing Structural Descriptions-- Tree Adjoining Grammars. In Dowty, D.; Karttunen, L.; and Zwicky, A. (editors), Natural Language Processing--</Title>
  <Section position="4" start_page="402" end_page="404" type="metho">
    <SectionTitle>
2 LEXICALIZED TAGS
</SectionTitle>
    <Paragraph position="0"> Not every grammar is in a 'lexicalized' form. 2 In the process of lexicalizing a grammar, we require that the 'lexicalized' grammar produce not only the same language as the original grammar, but also the same structures (or tree set).</Paragraph>
    <Paragraph position="1"> For example, a CFG, in general, will not be in a 'lexicalized' form. The domain of locality of CFGs can be easily extended by using a tree rewriting grammar (Schabes, Abeill~ and Joshi, 1988) that uses only substitution as a combining operation. This tree rewriting grammar consists of a set of trees that are not restricted to be of depth one (as in CFGs). Substitution can take place only on non-terminal nodes of the frontier of each tree. Substitution replaces a node marked for substitution by a tree rooted by the same label as the node (see Figure 1; the substitution node is marked by a down arrow ~.).</Paragraph>
    <Paragraph position="2"> However, in the general case, CFGs cannot be 'lexicalized', if only substitution is used. Furthermore, in general, there is not enough freedom to choose the anchor of each structure. This is important because we want the choice of the anchor for a given structure to be determined on purely linguistic grounds.</Paragraph>
    <Paragraph position="3"> If adjunction is used as an additional operation to combine these structures, CFGs can be lexicalized.</Paragraph>
    <Paragraph position="4"> Adjunction builds a new tree from an auxiliary tree fl and a tree ot . It inserts an auxiliary tree in another tree (see Figure 1). Adjunction is more powerful than substitution. It can weakly simulate substitution, but it also generates languages that could not be generated with substitution. 3  Substitution and adjunction enable us to lexicalize CFGs. The 'anchors' can be freely chosen (Schabes, Abeill~ and Joshi, 1988). The resulting system now falls in the class of mildly context-sensitive languages ~Notice the similarity of the definition of 'lexicalized' grammar with the ofltlne parsibillty constraint (Kaplan and Bresnan 1983). As consequences of our definition, each structure has at least one lexical item (its anchor) attached to it and all sentences are finitely ambiguous.</Paragraph>
    <Paragraph position="5">  (Joshi, 1985). Elementary structures of extended domain of locality combined with substitution and adjunction yield Lexicalized TAGs.</Paragraph>
    <Paragraph position="6"> TAGs were first introduced by Joshi, Levy and Takahashi (1975) and Joshi (1985). For more details on the original definition of TAGs, we refer the reader to Joshi (1985), Kroch and Joshi (1985), or Vijay-Shanker (1987). It is known that Tree Adjoining Languages (TALs) are mildly context sensitive. TALs properly contain context-free languages.</Paragraph>
    <Paragraph position="7"> TAGs with substitution and adjunction are naturally lexicalized. 4 A Lexicalized Tree Adjoining Grammar is a tree-based system that consists of two finite sets of trees: a set of initial trees, I and a set of auxiliary trees A (see Figure 2). The trees in I t3 A are called elementary trees. Each elementary tree is constrained to have at least one terminal symbol which acts as its anchor.</Paragraph>
    <Paragraph position="8">  The tree set of a TAG G, 7&amp;quot;(G) is defined to be the set of all derived trees starting from S-type initial trees in I. The string language generated by a TAG, PS(G), is defined to be the set of all terminal strings of the trees in 7-(G).</Paragraph>
    <Paragraph position="9"> By lexicalizing TAGs, we have associated lexical information to the 'production' system encoded by the TAG trees. We have therefore kept the computational advantages of 'production-like' formalisms (such as CFGs, TAGs) while allowing the possibility of linking them to lexical information. Formal properties of TAGs hold for Lexicalized TAGs.</Paragraph>
    <Paragraph position="10"> As first shown by Kroch and Joshi (1985), the properties of TAGs permit us to encapsulate diverse syntactic phenomena in a very natural way. TAG's extended domain of locality and its factoring recursion from local dependencies lead, among other things, to localizing the so-called unbounded dependencies. Abeill6 (1988a) uses the distinction between substitution and adjunction to capture the different extraction properties between sentential subjects and complements. Abeill6 (1988c) makes use of the extended domain of locality and lexicalization to account for NP island constraint violations in light verb constructions; in such cases, extraction out of NP is to be expected, without the use of reanalysis. The relevance of Lexicalized TAGs to idioms has been suggested by Abeill6 and Schabes (1989).</Paragraph>
    <Paragraph position="11"> We will now give some examples of structures that appear in a Lexicalized TAG lexicon.</Paragraph>
    <Paragraph position="12"> Some examples of initial trees are (for simplicity, we have omitted unification equations associated with the trees): 5 4In some earlier work of Joshi (1969, 1973), the use of the two operations 'adjoining' and 'replacement' (a restricted case of substitution) was investigated both mathematically and linguistically. However, these investigations dealt with string rewriting systems and not tree rewriting systems.</Paragraph>
    <Paragraph position="13"> 5The trees are simplified and the feature structures on the trees are not displayed. I is the mark for substitution nodes, * is the mark for the foot node of an auxiliary tree and NA stands for null adjunction constraint. This is the only adjunction constraint not indirectly stated by feature structures. We put indices on some non-terminals to express syntactic roles (0 for subject, 1 for first object, etc.). The index shown on the empty string (c) and the corresponding filler in the same tree is for the purpose of indicating the filler-gap dependency.</Paragraph>
    <Paragraph position="14">  In this approach, the argument structure is not just a list of arguments. It is the syntactic structure constructed with the lexical value of the predicate and with all the nodes of its arguments that eliminates the redundancy often noted between phrase structure rules and subcategorization frames. 6</Paragraph>
  </Section>
  <Section position="5" start_page="404" end_page="407" type="metho">
    <SectionTitle>
2.1 ORGANIZATION OF THE GRAMMAR
</SectionTitle>
    <Paragraph position="0"> A Lexicalized TAG is organized into two major parts: a lexicon and tree families, which are sets of trees. 7 TAG's factoring recursion from dependencies, the extended domain of locality of TAGs, and lexicalization of elementary trees make Lexicalized TAG an interesting framework for grammar writing. Abeill~ (1988b) discusses the writing of a Lexicalized TAG for French. Abeill~, Bishop, Cote and Schabes (1989) similarly discuss the writing of a Lexicalized TAG grammar for English.</Paragraph>
    <Paragraph position="1">  A tree family is essentially a set of sentential trees sharing the same argument structure abstracted from the lexical instantiation of the anchor (verb, predicative noun or adjective). Because of the extended domain of locality of Lexicalized TAG, the argument structure is not stated by a special mechanism but is implicitly stated in the topology of the trees in a tree family. Each tree in a family can be thought of as all possible syntactic 'transformations' of a given argument structure. Information (in the form of feature structures) that is valid independent of the value of the anchor is stated on the tree of the tree family. For example, the agreement between the subject and the main verb or auxiliary verb is stated on each tree of the tree family. Currently, the trees in a family are explicitly enumerated.</Paragraph>
    <Paragraph position="2">  The following trees, among others, compose the tree family of verbs taking one object (the family is named npOVnpl): s</Paragraph>
    <Paragraph position="4"> ompOVnpl is an initial tree corresponding to the declarative sentence, flROnpOVnpl is an auxiliary tree corresponding to a relative clause where the subject has been relativized, flRlnpOVnpl corresponds to the relative clause where the object has been relativized, o~ WOnpOVnpl is an initial tree corresponding to a wh-question on the subject, ot WlnpOVnpl corresponds to a wh-question on the object.</Paragraph>
    <Paragraph position="5">  The lexicon is the heart of the grammar. It associates a word with tree families or trees. Words are not associated with basic categories as in a CFG-based grammar, but with tree-structures corresponding to minimal linguistic structures. Multi-level dependencies can thus be stated in the lexicon.</Paragraph>
    <Paragraph position="6"> It also states some word-specific feature structure equations (such as the agreement value of a given verb) that have to be added to the ones already stated on the trees (such as the equality of the value of the subject and verb agreements).</Paragraph>
    <Paragraph position="7"> An example of a lexical entry follows:</Paragraph>
    <Paragraph position="9"> It should be emphasized that in our approach the category of a word is not a non-terminal symbol but a multi-level structure corresponding to minimal linguistic structures: sentences (for predicative verbs, nouns and adjectives) or phrases (NP for nouns, AP for adjectives, PP for prepositions yielding adverbial phrases).</Paragraph>
    <Section position="1" start_page="405" end_page="407" type="sub_section">
      <SectionTitle>
2.2 PARSING LEXICALIZED TAGs
</SectionTitle>
      <Paragraph position="0"> An Earley-type parser for TAGs has been developed by Schabes and Joshi (1988). It is a general TAG parser. It handles adjunction and substitution. It can take advantage of lexicalization. It uses the structures selected after the first pass to parse the sentence. The parser is able to use the non-local information given by the first step to filter out prediction and completion states.</Paragraph>
      <Paragraph position="1">  If an offline behavior is adopted, the Earley-type parser for TAGs can be used with no modification for parsing Lexicalized TAGs. First the trees corresponding to the input, string are selected and then the parser parses the input string with respect to this set of trees.</Paragraph>
      <Paragraph position="2"> However, Lexicalized TAGs simplify some cases of the algorithm. For example, since by definition each tree has at least one lexical item attached to it (its anchor), it will not be the case that a tree can be predicted  for substitution and completed in the same states set. Similarly, it will not be the case that an auxiliary tree can be left predicted for adjunction and right completed in the same states set.</Paragraph>
      <Paragraph position="3"> But most importantly the algorithm can be extended to take advantage of Lexicalized TAGs. Once the first pass has been performed, a subset of the grammar is selected. Each structure encodes the morphological value (and therefore the positions in the string) of its anchor. Identical structures with different anchor values are merged together (by identical structures we mean identical trees and identical information, such as feature structures, stated on those trees). 9 This enables us to use the anchor position information while processing efficiently the structures. For example, given the sentence The 1 men 2 who 3 saw 4 the 5 woman 6 who 7 saw 8 .John 9 are 10 happy n the following trees (among others) are selected after the first pass: ldeg</Paragraph>
      <Paragraph position="5"> The trees for men and for woman are distinguished since they carry different agreement feature structures (not shown in the figure).</Paragraph>
      <Paragraph position="6"> Notice that there is only one tree for the relative clauses introduced by saw but that its anchor position can be 4 or 8. Similarly for who and the.</Paragraph>
      <Paragraph position="7"> The anchor positions of each structure impose constraints on the way that the structures can be combined (the anchor positions must appear in increasing order in the combined structure). This helps the parser to filter out predictions or completions for adjunction or substitution. For example, the tree corresponding to men will not be predicted for substitution in any of the trees corresponding to saw since the anchor positions would not be in the right order.</Paragraph>
      <Paragraph position="8"> We have been evaluating the influence of the filtering of the grammar and the anchor position information on the behavior of the Earley-type parser. We have conducted experiments on a feature structure-based Lexicalized English TAG whose lexicon defines 200 entries associated with 130 different elementary trees (the trees are differentiated by their topology and their feature structures but not by their anchor value). Twenty five sentences of length ranging from 3 to 14 words were used to evaluate the parsing strategy. For each experiment, the number of trees given to the parser and the number of states were recorded.</Paragraph>
      <Paragraph position="9"> In the first experiment (referred to as one pass, OP), no first pass was performed. The entire grammar (i.e., the 130 trees) was used to parse each sentence. In the second experiment (referred to as two passes no anchor, NA), the two-pass strategy was used but the anchor positions were not used in the parser. And in the third experiment (referred to as two passes with anchor, A), the two-pass strategy was used and the information given by the anchor positions was used by the parser.</Paragraph>
      <Paragraph position="10"> The average behavior of the parser for each experiment is given in Figure 3. The first pass filtered on average 85% (always at least 75%) of the trees. The filtering of the grammar by itself decreased by 86% the number of states ((NA - OP)/OP). The additional use of the information given by the anchor positions further decreased by 50% ((A - NA)/NA) the number of states. The decrease given by the filtering of the grammar and by the information of the anchor positions is even bigger on the number of attempts to add a state (not reported in the table), n This set of experiments shows that the two-pass strategy increases the performance of the Earley-type parser for TAGs. The filtering of the grammar affects the parser the most. The information given by anchor 9Unlike our previous suggestions (Schabes, Abeill6 and Josh_i, 1988), we do not distinguish each structure by its anchor position since it increases unnecessarily the number of states of the Earley parser. By factoring recursion, the Earley parser enables us to process only once parts of a tree that are associated with several lexlcal items selecting the same tree. However, if termination is required for a pure top-down parser, it is necessary to distinguish each structure by its anchor position.  position in the first pass allows further improvement of the parser's performance (- 50% of the number of states on the set of experiments). The bottom-up non-local information given by the anchor positions improves the top-down component of the Earley-type parser.</Paragraph>
      <Paragraph position="11">  We performed our evaluation on a relatively small grammar and we did not evaluate the variations across grammars. The lexical degree of ambiguity of each word, the number of structures in the grammar, the number of lexical entries, and the length (and nature) of the input sentences are parameters to be considered. Although it might appear easy to conjecture the influence of these parameters, the actual experiments are difficult to perform since statistical data on these parameters are hard to obtain. We hope to perform some limited experiments along those lines.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>