<?xml version="1.0" standalone="yes"?>
<Paper uid="E06-1010">
  <Title>Constraints on Non-Projective Dependency Parsing</Title>
  <Section position="2" start_page="0" end_page="73" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Data-driven approaches to syntactic parsing has until quite recently been limited to representations that do not capture non-local dependencies. This is true regardless of whether representations are based on constituency, where such dependencies are traditionally represented by empty categories and coindexation to avoid explicitly discontinuous constituents, or on dependency, where it is more common to use a direct encoding of so-called non-projective dependencies.</Paragraph>
    <Paragraph position="1"> Whilethis&amp;quot;surface dependency approximation&amp;quot; (Levy and Manning, 2004) may be acceptable for certain applications of syntactic parsing, it is clearly not adequate as a basis for deep semantic interpretation, which explains the growing body of research devoted to different methods for correcting this approximation. Most of this work has so far focused either on post-processing to recover non-local dependencies from context-free parse trees (Johnson, 2002; Jijkoun and De Rijke, 2004; Levy and Manning, 2004; Campbell, 2004), or on incorporating nonlocal dependency information in nonterminal categories in constituency representations (Dienes and Dubey, 2003; Hockenmaier, 2003; Cahill et al., 2004) or in the categories used to label arcs in dependency representations (Nivre and Nilsson, 2005).</Paragraph>
    <Paragraph position="2"> By contrast, there is very little work on parsing methods that allow discontinuous constructions to be represented directly in the syntactic structure, whether by discontinuous constituent structures or by non-projective dependency structures. Notable exceptions are Plaehn (2000), where discontinuous phrase structure grammar parsing is explored, and McDonald et al. (2005b), where non-projective dependency structures are derived using spanning tree algorithms from graph theory.</Paragraph>
    <Paragraph position="3"> Onequestion that arises ifwewanttopursue the structure-based approach is how to constrain the class of permissible structures. On the one hand, we want to capture all the constructions that are found in natural languages, or at least to provide a much better approximation than before. On the other hand, it must still be possible for the parser not only to search the space of permissible structures in an efficient way but also to learn to select the most appropriate structure for a given sentence with sufficient accuracy. This is the usual tradeoff  between expressivity and complexity, where a less restricted class of permissible structures can capture more complex constructions, but where the enlarged search space makes parsing harder with respect to both accuracy and efficiency.</Paragraph>
    <Paragraph position="4"> Whereas extensions to context-free grammar have been studied quite extensively, there are very few corresponding results for dependency-based systems. Since Gaifman (1965) proved that his projective dependency grammar is weakly equivalent to context-free grammar, Neuhaus and Br&amp;quot;oker (1997) have shown that the recognition problem for a dependency grammar that can define arbitrary non-projective structures is NP complete, but there are no results for systems of intermediate complexity. The pseudo-projective grammar proposed by Kahane et al. (1998) can be parsed in polynomial time and captures non-local dependencies through a form of gap-threading, but the structures generated by the grammar are strictly projective. Moreover, the study of formal grammarsisonly partially relevant for research ondata-driven dependency parsing, where most systems are not grammar-based but rely on inductive inference from treebank data (Yamada and Matsumoto, 2003; Nivre et al., 2004; McDonald et al., 2005a).</Paragraph>
    <Paragraph position="5"> For example, despite the results of Neuhaus and Br&amp;quot;oker (1997), McDonald et al. (2005b) perform parsing with arbitrary non-projective dependency structures in O(n2) time.</Paragraph>
    <Paragraph position="6"> In this paper, we will therefore approach the problem from a slightly different angle. Instead of investigating formal dependency grammars and their complexity, we will impose a series of graph-theoretic constraints on dependency structures and see how these constraints affect expressivity and parsing efficiency. The approach is mainly experimental and we evaluate constraints using data from two dependency-based treebanks, the Prague Dependency Treebank (HajiVc et al., 2001) and the Danish Dependency Treebank (Kromann, 2003).</Paragraph>
    <Paragraph position="7"> Expressivity is investigated by examining how large a proportion of the structures found in the treebanks are parsable under different constraints, and efficiency is addressed by considering the number of potential dependency arcs that need to be processed when parsing these structures. This is a relevant metric for data-driven approaches, where parsing timeisoften dominated by thecomputation of model predictions or scores for such arcs. The parsing experiments are performed with a variant of Covington's algorithm for dependency parsing (Covington, 2001), using the treebank as an oracle in order to establish an upper bound on accuracy. However, the results are relevant for a larger class of algorithms that derive non-projective dependency graphs by treating every possible word pair as a potential dependency arc.</Paragraph>
    <Paragraph position="8"> The paper is structured as follows. In section 2 we define dependency graphs, and in section 3 we formulate a number of constraints that can be used to define different classes of dependency graphs, ranging from unrestricted non-projective to strictly projective. In section 4 we introduce the parsing algorithm used in the experiments, and in section 5 we describe the experimental setup. In section 6 we present the results of the experiments and discuss their implications for non-projective dependency parsing. We conclude in section 7.</Paragraph>
  </Section>
class="xml-element"></Paper>