<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-1008">
  <Title>Identifying Semantic Roles Using Combinatory Categorial Grammar</Title>
  <Section position="9" start_page="3" end_page="3" type="concl">
    <SectionTitle>
7 Conclusion
</SectionTitle>
    <Paragraph position="0"> Our CCG-based system for automatically labeling verb arguments with PropBank-style semantic roles outperforms a system using a traditional Treebank-based parser for core arguments, which comprise 75% of the role labels, but scores lower on adjunct-like roles such as temporals and locatives. The CCG parser returns predicate-argument structures that include long-range dependencies; therefore, it seems inherently better suited for this task. However, the performance of our CCG system is lowered by the fact that the syntactic analyses in its training corpus differ from those that underlie PropBank in important ways (in particular in the notion of heads and the complement-adjunct distinction). We would expect a higher performance for the CCG-based system if the analyses in CCGbank resembled more closely those in PropBank.</Paragraph>
    <Paragraph position="1"> Our results also indicate the importance of recovering long-range dependencies, either through the trace information in the Penn Treebank, or directly, as in the predicate-argument structures returned by the CCG parser. We speculate that much of the performance improvement we show could be obtained with traditional (ie. non-CCG-based) parsers if they were designed to recover more of the information present in the Penn Treebank, in particular the trace co-indexation. An interesting experiment would be the application of our role-labeling system to the output of the trace recovery system of Johnson (2002). Our results also have implications for parser evaluation, as the most frequently used constituent-based precision and recall measures do not evaluate how well long-range dependencies can be recovered from the output of a parser. Measures based on dependencies, such as those of Lin (1995) and Carroll et al. (1998), are likely to be more relevant to real-world applications of parsing.</Paragraph>
    <Paragraph position="2"> Acknowledgments This work was supported by the Institute for Research in Cognitive Science at the University of Pennsylvania, the Propbank project (DoD Grant MDA904-00C2136), an EPSRC studentship and grant GR/M96889, and NSF ITR grant 0205 456. We thank Mark Steedman, Martha Palmer and Alexandra Kinyon for their comments on this work.</Paragraph>
  </Section>
class="xml-element"></Paper>