<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1204">
  <Title>Deep Linguistic Analysis for the Accurate Identification of Predicate-Argument Relations</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Recently, deep linguistic analysis has successfully been applied to real-world texts. Several parsers have been implemented in various grammar formalisms and empirical evaluation has been reported: LFG (Riezler et al., 2002; Cahill et al., 2002; Burke et al., 2004), LTAG (Chiang, 2000), CCG (Hockenmaier and Steedman, 2002b; Clark et al., 2002; Hockenmaier, 2003), and HPSG (Miyao et al., 2003; Malouf and van Noord, 2004). However, their accuracy was still below the state-of-the-art PCFG parsers (Collins, 1999; Charniak, 2000) in terms of the PARSEVAL score. Since deep parsers can output deeper representation of the structure of a sentence, such as predicate argument structures, several studies reported the accuracy of predicate-argument relations using a treebank developed for each formalism. However, resources used for the evaluation were not available for other formalisms, and the results cannot be compared with each other.</Paragraph>
    <Paragraph position="1"> In this paper, we employ PropBank (Kingsbury and Palmer, 2002) for the evaluation of the accuracy of HPSG parsing. In the PropBank, semantic arguments of a predicate and their semantic roles are manually annotated. Since the PropBank has been developed independently of any grammar formalisms, the results are comparable with other published results using the same test data.</Paragraph>
    <Paragraph position="2"> Interestingly, several studies suggested that the identification of PropBank annotations would require linguistically-motivated features that can be obtained by deep linguistic analysis (Gildea and Hockenmaier, 2003; Chen and Rambow, 2003).</Paragraph>
    <Paragraph position="3"> They employed a CCG (Steedman, 2000) or LTAG (Schabes et al., 1988) parser to acquire syntactic/semantic structures, which would be passed to statistical classifier as features. That is, they used deep analysis as a preprocessor to obtain useful features for training a probabilistic model or statistical classifier of a semantic argument identifier. These results imply the superiority of deep linguistic analysis for this task.</Paragraph>
    <Paragraph position="4"> Although the statistical approach seems a reasonable way for developing an accurate identifier of PropBank annotations, this study aims at establishing a method of directly comparing the outputs of HPSG parsing with the PropBank annotation in order to explicitly demonstrate the availability of deep parsers. That is, we do not apply statistical model nor machine learning to the post-processing of the output of HPSG parsing. By eliminating the effect of post-processing, we can directly evaluate the accuracy of deep linguistic analysis.</Paragraph>
    <Paragraph position="5"> Section 2 introduces recent advances in deep linguistic analysis and the development of semantically annotated corpora. Section 3 describes the details of the implementation of an HPSG parser evaluated in this study. Section 4 discusses a problem in adopting PropBank for the performance evaluation of deep linguistic parsers and proposes its solution.</Paragraph>
    <Paragraph position="6"> Section 5 reports empirical evaluation of the accuracy of the HPSG parser.</Paragraph>
  </Section>
class="xml-element"></Paper>