File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/06/p06-1023_metho.xml
Size: 18,022 bytes
Last Modified: 2025-10-06 14:10:18
<?xml version="1.0" standalone="yes"?> <Paper uid="P06-1023"> <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Trace Prediction and Recovery With Unlexicalized PCFGs and Slash Features</Title> <Section position="4" start_page="177" end_page="179" type="metho"> <SectionTitle> 2 Feature Annotation </SectionTitle> <Paragraph position="0"> A context-free grammar which generates empty categories has to make sure that a filler exists for each trace and vice versa. A well-known technique which enforces this constraint is the GPSG-style percolation of a slash feature: All constituents on the direct path from the trace to the filler are annotated with a special feature which represents the category of the filler as shown in figure 2. In order to restore the original treebank an- null gory WHNP is linked to the trace node via percolation of a slash feature. The trace node is labeled with *T*.</Paragraph> <Paragraph position="1"> notation with co-reference indices from the representation with slash features, the parse tree has to be traversed starting at a trace node and following the nodes annotated with the respective filler category until the filler node is encountered. Normally, the filler node is a sister node of an ancestor node of the trace, i.e. the filler c-commands the trace node, but in case of clausal fillers it is also possible that the filler dominates the trace. An example is the sentence &quot;S-1 She had - he informed her *1 - kidney trouble&quot; whose parse tree is shown in figure 3.</Paragraph> <Paragraph position="2"> Besides the slash features, we used other features in order to improve the parsing accuracy of the PCFG, inspired by the work of Klein and Manning (2003). The most important ones of these features1 will now be described in detail. Section 4.3 shows the impact of these features on labeled bracketing accuracy and empty category prediction.</Paragraph> <Paragraph position="3"> VP feature VPs were annotated with a feature that distinguishes between finite, infinitive, toinfinitive, gerund, past participle, and passive VPs. S feature The S node feature distinguishes between imperatives, finite clauses, and several types of small clauses.</Paragraph> <Paragraph position="4"> Parent features Modifier categories like SBAR, PP, ADVP, RB and NP-ADV were annotated with a parent feature (cf. Johnson (1998)). The parent features distinguish between verbal (VP), adjectival (ADJP, WHADJP), adverbial (ADVP, WHADVP), nominal (NP, WHNP, QP), prepositional (PP) and other parents.</Paragraph> <Paragraph position="5"> PENN tags The PENN treebank annotation uses semantic tags to refine syntactic categories. Most parsers ignore this information. We preserved the tags ADV, CLR, DIR, EXT, IMP, LGS, LOC, MNR, NOM, PRD, PRP, SBJ and TMP in combination with selected categories.</Paragraph> <Paragraph position="6"> Auxiliary feature We added a feature to the part-of-speech tags of verbs in order to distinguish between be, do, have, and full verbs.</Paragraph> <Paragraph position="7"> Agreement feature Finite VPs are marked with 3s (n3s) if they are headed by a verb with part-of-speech VBZ (VBP).</Paragraph> <Paragraph position="8"> Genitive feature NP nodes which dominate a node of the category POS (possessive marker) are marked with a genitive flag.</Paragraph> <Paragraph position="9"> Base NPs NPs dominating a node of category NN, NNS, NNP, NNPS, DT, CD, JJ, JJR, JJS, PRP, RB, or EX are marked as base NPs.</Paragraph> <Paragraph position="10"> IN feature The part-of-speech tags of the 45 most frequent prepositions were lexicalized by adding the preposition as a feature. The new part-of-speech tag of the preposition &quot;by&quot; is &quot;IN/by&quot;. Irregular adverbs The part-of-speech tags of the adverbs &quot;as&quot;, &quot;so&quot;, &quot;about&quot;, and &quot;not&quot; were also lexicalized.</Paragraph> <Paragraph position="11"> Currency feature NP and QP nodes are marked with a currency flag if they dominate a node of category $, #, or SYM.</Paragraph> <Paragraph position="12"> Percent feature Nodes of the category NP or QP are marked with a percent flag if they dominate the subtree (NN %). Any node which immediately dominates the token %, is marked, as well.</Paragraph> <Paragraph position="13"> Punctuation feature Nodes which dominate sentential punctuation (.?!) are marked.</Paragraph> <Paragraph position="14"> DT feature Nodes of category DT are split into indefinite articles (a, an), definite articles (the), and demonstratives (this, that, those, these).</Paragraph> <Paragraph position="15"> WH feature The wh-tags (WDT, WP, WRB, WDT) of the words which, what, who, how, and that are also lexicalized.</Paragraph> <Paragraph position="16"> Colon feature The part-of-speech tag ':' was replaced with &quot;;&quot;, &quot;-&quot; or &quot;...&quot; if it dominated a corresponding token.</Paragraph> <Paragraph position="17"> DomV feature Nodes of a non-verbal syntactic category are marked with a feature if they dominate a node of category VP, SINV, S, SQ, SBAR, or SBARQ.</Paragraph> <Paragraph position="18"> Gap feature S nodes dominating an empty NP are marked with the feature gap.</Paragraph> <Paragraph position="19"> Subcategorization feature The part-of-speech tags of verbs are annotated with a feature which encodes the sequence of arguments. The encoding maps reflexive NPs to r, NP/NP-PRD/SBAR-NOM to n, ADJP-PRD to j, ADVP-PRD to a, PRT to t, PP/PP-DIR to p, SBAR/SBAR-CLR to b, S/fin to sf, S/ppres/gap to sg, S/to/gap to st, other S nodes to so, VP/ppres to vg, VP/ppast to vn, VP/pas to vp, VP/inf to vi, and other VPs to vo. A verb with an NP and a PP argument, for instance, is annotated with the feature np.</Paragraph> <Paragraph position="20"> Adjectives, adverbs, and nouns may also get a subcat feature which encodes a single argument using a less fine-grained encoding which maps PP to p, NP to n, S to s, and SBAR to b. A node of category NN or NNS e.g. is marked with a subcat feature if it is followed by an argument category unless the argument is a PP which is headed by the preposition of.</Paragraph> <Paragraph position="21"> RC feature In relative clauses with an empty relative pronoun of category WHADVP, we mark the SBAR node of the relative clause, the NP node to which it is attached, and its head child of category NN or NNS, if the head word is either way, ways, reason, reasons, day, days, time, moment, place, or position. This feature helps the parser to correctly insert WHADVP rather than WHNP.</Paragraph> <Paragraph position="22"> Figure 4 shows a sample tree.</Paragraph> <Paragraph position="23"> TMP features Each node on the path between an NP-TMP or PP-TMP node and its nominal head is labeled with the feature tmp. This feature helps the parser to identify temporal NPs and PPs.</Paragraph> <Paragraph position="24"> MNR and EXT features Similarly, each node on the path between an NP-EXT, NP-MNR or ADVP-TMP node and its head is labeled with the empty relative pronoun of category WHADVP feature ext or mnr.</Paragraph> <Paragraph position="25"> ADJP features Nodes of category ADJP which are dominated by an NP node are labeled with the feature &quot;post&quot; if they are in final position and the feature &quot;attr&quot; otherwise.</Paragraph> <Paragraph position="26"> JJ feature Nodes of category JJ which are dominated by an ADJP-PRD node are labeled with the feature &quot;prd&quot;.</Paragraph> <Paragraph position="27"> JJ-tmp feature JJ nodes which are dominated by an NP-TMP node and which themselves dominate one of the words &quot;last&quot;, &quot;next&quot;, &quot;late&quot;, &quot;previous&quot;, &quot;early&quot;, or &quot;past&quot; are labeled with tmp. QP feature If some node dominates an NP node followed by an NP-ADV node as in (NP (NP one dollar) (NP-ADV a day)), the first child NP node is labeled with the feature &quot;qp&quot;. If the parent is an NP node, it is also labeled with &quot;qp&quot;.</Paragraph> <Paragraph position="28"> NP-pp feature NP nodes which dominate a PP node are labeled with the feature pp. If this PP itself is headed by the preposition of, then it is annotated with the feature of.</Paragraph> <Paragraph position="29"> MWL feature In adverbial phrases which neither dominate an adverb nor another adverbial phrase, we lexicalize the part-of-speech tags of a small set of words like &quot;least&quot; (at least), &quot;kind&quot;, or &quot;sort&quot; which appear frequently in such adverbial phrases.</Paragraph> <Paragraph position="30"> Case feature Pronouns like he or him , but not ambiguous pronouns like it are marked with nom or acc, respectively.</Paragraph> <Paragraph position="31"> Expletives If a subject NP dominates an NP which consists of the pronoun it, and an S-trace in sentences like It is important to..., the dominated NP is marked with the feature expl.</Paragraph> <Paragraph position="32"> LST feature The parent nodes of LST nodes2 are marked with the feature lst.</Paragraph> <Paragraph position="33"> Complex conjunctions In SBAR constituents starting with an IN and an NN child node (usually indicating one of the two complex conjunctions &quot;in order to&quot; or &quot;in case of&quot;), we mark the NN child with the feature sbar.</Paragraph> <Paragraph position="34"> LGS feature The PENN treebank marks the logical subject of passive clauses which are realized by a by-PP with the semantic tag LGS. We move this tag to the dominating PP.</Paragraph> <Paragraph position="35"> OC feature Verbs are marked with an object control feature if they have an NP argument which dominates an NP filler and an S argument which dominates an NP trace. An example is the sentence She asked him to come.</Paragraph> <Paragraph position="36"> Corrections The part-of-speech tags of the PENN treebank are not always correct. Some of the errors (like the tag NNS in VP-initial position) can be identified and corrected automatically in the training data. Correcting tags did not always improve parsing accuracy, so it was done selectively. null The gap and domV features described above were also used by Klein and Manning (2003).</Paragraph> <Paragraph position="37"> All features were automatically added to the PENN treebank by means of an annotation program. Figure 5 shows an example of an annotated parse tree.</Paragraph> </Section> <Section position="5" start_page="179" end_page="180" type="metho"> <SectionTitle> 3 Parameter Smoothing </SectionTitle> <Paragraph position="0"> We extracted the grammar from sections 2-21 of the annotated version of the PENN treebank. In order to increase the coverage of the grammar, we selectively applied markovization to the grammar (cf. Klein and Manning (2003)) by replacing long infrequent rules with a set of binary rules.</Paragraph> <Paragraph position="1"> Markovization was only applied if none of the non-terminals on the right hand side of the rule had a slash feature in order to avoid negative effects on the slash feature percolation mechanism.</Paragraph> <Paragraph position="2"> The probabilities of the grammar rules were directly estimated with relative frequencies. No smoothing was applied, here. The lexical probabilities, on the other hand, were smoothed with the following technique which was adopted from Klein and Manning (2003). Each word is assigned to one of 216 word classes. The word classes are defined with regular expressions. Examples are the class [A-Za-z0-9-]+-oldwhich contains the word 20-year-old, the class [a-z][az]+ifies which contains clarifies, and a class which contains a list of capitalized adjectives like Advanced. The word classes are ordered. If a string is matched by the regular expressions of more than one word class, then it is assigned to the first of these word classes. For each word class, we compute part-of-speech probabilities with relative frequencies. The part-of-speech frequencies a0a2a1a4a3a6a5a8a7a8a9 of a word a3 are smoothed by adding the part-of-speech probabilitya10 a1a4a7a12a11a14a13a3a16a15a4a9 of the word class a13a3a16a15 according to equation 1 in order to obtain the smoothed frequency a17a0a2a1a4a3a6a5a8a7a8a9 . The part-of-speech probability of the word class is weighted by a parameter a18 whose value was set to 4 after testing on held-out data. The lexical probabilities are finally estimated from the smoothed frequencies according to equation 2.</Paragraph> </Section> <Section position="6" start_page="180" end_page="182" type="metho"> <SectionTitle> 4 Evaluation </SectionTitle> <Paragraph position="0"> In our experiments, we used the usual splitting of the PENN treebank into training data (sections 221), held-out data (section 22), and test data (section 23).</Paragraph> <Paragraph position="1"> The grammar extracted from the automatically annotated version of the training corpus contained 52,297 rules with 3,453 different non-terminals. Subtrees which dominated only empty categories were collapsed into a single empty element symbol. The parser skips over these symbols during parsing, but adds them to the output parse. Overall, there were 308 different empty element symbols in the grammar.</Paragraph> <Paragraph position="2"> Table 1 shows the labeled bracketing accuracy of the parser on the whole section 23 and compares it to the results reported in Klein and Manning (2003) for sentences with up to 100 words.</Paragraph> <Section position="1" start_page="180" end_page="181" type="sub_section"> <SectionTitle> 4.1 Empty Category Prediction </SectionTitle> <Paragraph position="0"> Table 2 reports the accuracy of the parser in the empty category (EC) prediction task for ECs occurring more than 6 times. Following Johnson (2001), an empty category was considered correct if the treebank parse contained an empty node of the same category at the same string position.</Paragraph> <Paragraph position="1"> Empty SBAR nodes which dominate an empty S node are treated as a single empty element and listed as SBAR-S in table 2.</Paragraph> <Paragraph position="2"> Frequent types of empty elements are recognized quite reliably. Exceptions are the traces of adverbial and prepositional phrases where the recall was only 65% and 48%, respectively, and empty relative pronouns of type WHNP and WHADVP with f-scores around 60%. A couple of empty relative pronouns of type WHADVP were mis-analyzed as WHNP which explains why the precision is higher than the recall for WHADVP, but vice versa for WHNP.</Paragraph> <Paragraph position="3"> prec. recall f-sc. freq.</Paragraph> <Paragraph position="4"> on section 23. The first column shows the type of the empty element and - except for empty complementizers and empty units - also the category. The last column shows the frequency in the test data. The accuracy of the pseudo attachment labels *RNR*, *ICH*, *EXP*, and *PPA* was generally low with a precision of 41%, recall of 21%, and f-score of 28%. Empty elements with a test corpus frequency below 8 were almost never generated by the parser.</Paragraph> </Section> <Section position="2" start_page="181" end_page="181" type="sub_section"> <SectionTitle> 4.2 Co-Indexation </SectionTitle> <Paragraph position="0"> Table 3 shows the accuracy of the parser on the co-indexation task. A co-indexation of a trace and a filler is represented by a 5-tuple consisting of the category and the string position of the trace, as well as the category, start and end position of the filler. A co-indexation is judged correct if the treebank parse contains the same 5-tuple.</Paragraph> <Paragraph position="1"> For NP3 and S4 traces of type '*T*', the co-indexation results are quite good with 85% and speech like the sentence &quot;That's true!&quot;, he said *T*. other categories and for NP traces of type '*',5 the parser shows high precision, but moderate recall.</Paragraph> <Paragraph position="2"> The recall of infrequent types of empty elements is again low, as in the recognition task.</Paragraph> <Paragraph position="3"> prec. rec. f-sc. freq.</Paragraph> <Paragraph position="4"> The first column shows the category and type of the trace. If the filler category of the filler is different from the category of the trace, it is added in front. The filler category is abbreviated to &quot;WH&quot; if the rest is identical to the trace category. The last column shows the frequency in the test data. In order to get an impression how often EC prediction errors resulted from misplacement rather than omission, we computed EC prediction accuracies without comparing the EC positions. We observed the largest f-score increase for ADVP *T* and PP *T*, where attachment ambiguities are likely, and for VP *?* which is infrequent.</Paragraph> </Section> <Section position="3" start_page="181" end_page="182" type="sub_section"> <SectionTitle> 4.3 Feature Evaluation </SectionTitle> <Paragraph position="0"> We ran a series of evaluations on held-out data in order to determine the impact of the different features which we described in section 2 on the parsing accuracy. In each run, we deleted one of the features and measured how the accuracy changed compared to the baseline system with all features.</Paragraph> <Paragraph position="1"> The results are shown in table 4.</Paragraph> <Paragraph position="2"> 5The trace type '*' combines two types of traces with different linguistic properties, namely empty objects of passive constructions which are co-indexed with the subject, and empty subjects of participial and infinitive clauses which are co-indexed with an NP of the matrix clause.</Paragraph> <Paragraph position="3"> for labeled bracketing, EC prediction, and co-indexation (CI) and the f-scores without the specified feature.</Paragraph> </Section> </Section> class="xml-element"></Paper>