<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2104">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics A Comparison of Alternative Parse Tre Paths for Labeling Semantic Roles</Title>
  <Section position="7" start_page="815" end_page="817" type="concl">
    <SectionTitle>
6 Discusion
</SectionTitle>
    <Paragraph position="0"> Anotated corpora of linguistic phenomena enable many new natural language processing applications and provide new means for tackling difficult research problems. Just as the Penn Treebank offers the posibility of developing systems capable of accurate syntactic parsing, corpora of semantic role annotations open up new posibilities for rich textual understanding and integrated inference.</Paragraph>
    <Paragraph position="1"> In this paper, we compared five encodings of parse tree paths based on two constituency parsers and a dependency parser. Despite our expectations that the semantic richness of dependency parses would yield paths that outperformed the others, we discovered that parse tree paths from Charniak's constituency parser performed the best overall. In applications where either precision or recall is the only concern, then Minipar-derived parse tree paths would yield the best results. We also found that the performance of all of these systems varied acros different argument types.</Paragraph>
    <Paragraph position="2"> In contrast to the performance results reported by Palmer et al. (205) and Gildea &amp; Jurafsky (202), our evaluation was based solely on parse tree path features. Even so, we were able to obtain reasonable levels of performance without the use of additional features or stochastic methods.</Paragraph>
    <Paragraph position="3"> Learning curves indicate that the greatest gains in performance can be garnered from the first 10 or so training examples. This result has implications for the development of large-scale corpora of semantically annotated text. Developers should distribute their effort in order to maximize the number of predicate-argument pairs with at least 10 annotations.</Paragraph>
    <Paragraph position="4"> An automated semantic role labeling system could be constructed using only the parse tree path features described in this paper, with estimated performance between our recall scores and our adjusted recall scores. There are several ways to improve on the random selection approach used in the adjusted recall calculation. For example, one could simply select the candidate answer with the most frequent parse tree path.</Paragraph>
    <Paragraph position="5"> The results presented in this paper help inform the design of future automated semantic role labeling systems that improve on the best-performing systems available today (Gildea &amp;  Jurafsky, 202; Moschiti et al., 205). We found that different parse tree paths encode different types of linguistic information, and exhibit different characteristics in the tradeoff between precision and recall. The best approaches in future systems wil intelligently capitalize on these differences in the face of varying amounts of training data.</Paragraph>
    <Paragraph position="6"> In our own future work, we are particularly interested in exploring the regularities that exist among parse tree paths for different predicates.</Paragraph>
    <Paragraph position="7"> By identifying these regularities, we believe that we wil be able to significantly reduce the total number of annotations necessary to develop lexical resources that have broad coverage over natural language.</Paragraph>
  </Section>
class="xml-element"></Paper>