<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1180">
  <Title>Wide-Coverage Semantic Representations from a CCG Parser</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Combinatory Categorial Grammar
</SectionTitle>
    <Paragraph position="0"> We assume familiarity with CCG (Steedman, 2000), an entirely type-driven lexicalized theory of grammar based on categorial grammar. CCG lexical entries pair a syntactic category (defining syntactic valency and directionality) with a semantic interpretation. For example, one of the categories for the verb cost can be written as follows, with a provisional Montague-style semantics expressed in terms of predicate-argument structure:1</Paragraph>
    <Paragraph position="2"> Combinatory rules project such lexical categoryinterpretation pairs onto derived categoryinterpretation pairs. The specific involvement in CCG of rules of functional composition (indexed a18 B and a19 B in derivations) and type-raising (indexed a18 T and a19 T) allows very free derivation of non-standard constituents. This results in semantic interpretations that support the &amp;quot;surface compositional&amp;quot; analysis of relativization and coordination, as in Figure 1 for the sentence It could cost taxpayers PS15 million to install and BPC residents 1 million a year to maintain.2  identifies constants with primes and uses concatenation a b to indicate application of a to b. Application is &amp;quot;left-associative,&amp;quot; so abc is equivalent to a21 aba22 c. The order of arguments in the predication is &amp;quot;wrapped&amp;quot;, consistent with the facts of reflexive binding.</Paragraph>
    <Paragraph position="3"> 2Some details of the derivation and of the semantics of noun phrases are suppressed, since these are developed be-While the proliferation of surface constituents allowed by CCG adds to derivational ambiguity (since the constituent taxpayers PS15 million to install is also allowed in the non-coordinate sentence It could cost taxpayers PS15 million to install), previous work has shown that standard techniques from the statistical parsing literature can be used for practical wide-coverage parsing with state-of-the-art performance.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 The Parser
</SectionTitle>
    <Paragraph position="0"> A number of statistical parsers have recently been developed for CCG (Clark et al., 2002; Hockenmaier and Steedman, 2002b; Clark and Curran, 2004b). All of these parsers use a grammar derived from CCGbank (Hockenmaier and Steedman, 2002a; Hockenmaier, 2003), a treebank of normal-form CCG derivations derived semi-automatically from the Penn Treebank. In this paper we use the Clark and Curran (2004b) parser, which uses a log-linear model of normal-form derivations to select an analysis.</Paragraph>
    <Paragraph position="1"> The parser takes a POS tagged sentence as input with a set of lexical categories assigned to each word. A CCG supertagger (Clark and Curran, 2004a) is used to assign the categories. The supertagger uses a log-linear model of the target word's context to decide which categories to assign. Clark and Curran (2004a) shows how dynamic use of the supertagger -- starting off with a small number of categories assigned to each word and gradually increasing the number until an analysis is found -- can lead to a highly efficient and robust parser.</Paragraph>
    <Paragraph position="2"> The lexical category set used by the parser consists of those category types which occur at least 10 times in sections 2-21 of CCGbank, which results in a set of 409 categories. Clark and Curran (2004a) demonstrates that this relatively small set has high coverage on unseen data and can be used to create low. Some categories and interpretations are split across lines to save space.</Paragraph>
    <Paragraph position="3"> a robust and accurate parser. The relevance of a relatively small category set is that, in order to obtain semantic representations for a particular formalism, only 409 categories have to be annotated.</Paragraph>
    <Paragraph position="4"> The parser uses the CKY chart-parsing algorithm from Steedman (2000). The combinatory rules used by the parser are functional application (forward and backward), generalised forward composition, backward composition, generalised backward-crossed composition, and type raising. There is also a coordination rule which conjoins categories of the same type.</Paragraph>
    <Paragraph position="5"> The parser also uses a number of unary type-changing rules (Hockenmaier and Steedman, 2002a) and punctuation rules taken from CCGbank.</Paragraph>
    <Paragraph position="6"> An example of a type-changing rule used by the parser is the following, which takes a passive form of a verb and creates a nominal modifier:</Paragraph>
    <Paragraph position="8"> This rule is used to create NPs such as the role played by Kim Cattrall. An example of a comma rule is the following:</Paragraph>
    <Paragraph position="10"> This rule takes a sentential modifier followed by a comma and returns a sentential modifier of the same type.</Paragraph>
    <Paragraph position="11"> Type-raising is applied to the categories NP, PP and Sa0 adja1a3a2 NP (adjectival phrase), and is implemented by adding the relevant set of type-raised categories to the chart whenever an NP, PP or Sa0 adja1a3a2 NP is present. The sets of type-raised categories are based on the most commonly used type-raising rule instantiations in sections 2-21 of CCGbank, and currently contain 8 type-raised categories for NP and 1 each for PP and Sa0 adja1a3a2 NP.</Paragraph>
    <Paragraph position="12"> For a given sentence, the automatically extracted grammar can produce a very large number of derivations. Clark and Curran (2003) and Clark and Curran (2004b) describe how a packed chart can be used to efficiently represent the derivation space, and also efficient algorithms for finding the most probable derivation. The parser uses a log-linear model over normal-form derivations.3 Features are defined in terms of the local trees in the derivation, including lexical head information and word-word dependencies. The normal-form derivations in CCGbank provide the gold standard training data.</Paragraph>
    <Paragraph position="13"> For a given sentence, the output of the parser is a set of syntactic dependencies corresponding to the 3A normal-form derivation is one which only uses type-raising and function composition when necessary.</Paragraph>
    <Paragraph position="14"> most probable derivation. However, for this paper the parser has been modified to simply output the derivation in the form shown in Figure 2, which is the input for the semantic component.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Building Semantic Representations
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Semantic Formalism
</SectionTitle>
      <Paragraph position="0"> Our method for constructing semantic representations can be used with many different semantic formalisms. In this paper we use formulas of first-order logic with a neo-Davidsonian analysis of events. We do not attempt to cover all semantic phenomena; for example, we do not currently deal with the resolution of pronouns and ellipsis; we do not give a proper analysis of tense and aspect; we do not distinguish between distributive and collective readings of plural noun phrases; and we do not handle quantifier scope ambiguities.</Paragraph>
      <Paragraph position="1"> The following first-order formula for the sentence A spokesman had no comment demonstrates the representation we use:</Paragraph>
      <Paragraph position="3"> The tool that we use to build semantic representations is based on the lambda calculus. It can be used to mark missing semantic information from natural language expressions in a principled way using l, an operator that binds variables ranging over various semantic types. For instance, a noun phrase like a spokesman can be given the l-expression lp.a7 x(spokesman(x)a8 (p@x)) where the @ denotes functional application, and the variable p marks the missing information provided by the verb phrase. This expression can be combined with the l-expression for lied, using functional application, yielding the following expression: null lp.a7 x(spokesman(x)a8 (p@x))@ ly.a7 e(lie(e)a8 agent(e,y)).</Paragraph>
      <Paragraph position="4"> b-conversion is the process of eliminating all occurrences of functional application by substituting the argument for the l-bound variables in the functor. b-conversion turns the previous expression into a first-order translation for A spokesman lied:</Paragraph>
      <Paragraph position="6"> The resulting semantic formalism is very similar to the type-theoretic language Ll (Dowty et al., 1981). However, we merely use the lambda-calculus as a tool for constructing semantic representations, rather as a formal tool for model-theoretic interpretation. As already mentioned, we can use the same method to obtain, for example, Discourse Representation Structures (Kuschert, 1999), or underspecified semantic representations (Bos, 2004) to deal with quantifier scope ambiguities. null</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Method and Algorithm
</SectionTitle>
      <Paragraph position="0"> The output of the parser is a tree representing a CCG derivation, where the leaves are lexical items and the nodes correspond to one of the CCG combinatory rules, a unary type-changing rule, a type-raising rule, or one of the additional miscellaneous rules discussed earlier. Mapping the CCG derivation into a semantic representation consists of the following tasks:  1. assigning semantic representations to the lexical items; 2. reformulating the combinatory rules in terms of functional application; 3. dealing with type-raising and type-changing rules; 4. applying b-conversion to the resulting tree  structure.</Paragraph>
      <Paragraph position="1"> Lexical items are ordered pairs consisting of the CCG category and a lemmatised wordform. This information is used to assign a l-expression to the leaf nodes in the tree. For most open-class lexical items we use the lemma to instantiate the lexical semantics, as illustrated by the following two examples (intransitive verbs and adjectives):</Paragraph>
      <Paragraph position="3"> For closed-class lexical items, the lexical semantics is spelled out for each lemma individually, as in the following two examples:</Paragraph>
      <Paragraph position="5"> The second task deals with the combinatory rules.</Paragraph>
      <Paragraph position="6"> The rules we currently use are forward and backward application (FAPP, BAPP), generalised forward composition (FCOMP), backward composition (BCOMP), and generalised backward-crossed composition (BCROSS).</Paragraph>
      <Paragraph position="8"> The type-raising and type-changing rules are dealt with by looking up the specific rule and replacing it with the resulting semantics. For instance, the rule that raises category NP to S[X]/(S[X]\NP) converts the semantics as follows:</Paragraph>
      <Paragraph position="10"> The following type-changing rule applies to the lexical semantics of categories of type N and converts them to NP:</Paragraph>
      <Paragraph position="12"> Tasks 1-3 are implemented using a recursive algorithm that traverses the derivation and returns a l-expression. Note that the punctuation rules used by the parser do not contribute to the compositional semantics and are therefore ignored.</Paragraph>
      <Paragraph position="13"> Task 4 reduces the l-expression to the target representation by applying b-conversion. In order to maintain correctness of this operation, the functor undergoes a-conversion (renaming all bound variables for new occurrences) before substitution takes place. b-conversion is implemented using the tools provided by Blackburn and Bos (2003).</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.3 Results
</SectionTitle>
      <Paragraph position="0"> There are a number of possible ways to evaluate the semantic representations output by our system. The first is to calculate the coverage -- that is, the percentage of syntactic parses which can be given some analysis by the semantic component. The second is to evaluate the accuracy of the semantic representations; the problem is that there is not yet an accepted evaluation metric which can be applied to such representations. null There is, however, an accepted way of evaluating the syntactic component of the system, namely to calculate precision and recall figures for labelled syntactic dependencies (Clark et al., 2002). Given  The school-board hearing at which she was dismissed was crowded with students and teachers that the CCG parser produces dependencies which are essentially predicate-argument dependencies, the accuracy of the syntactic component should be a good indication of the accuracy of the semantics, especially given the transparent interface between syntax and semantics used by our system. Hence we report coverage figures in this paper, and repeat figures for dependency recovery from an earlier paper. null We do not evaluate the accuracy of the system output directly, but we do have a way of checking the well-formedness of the semantic representations. (The well-formedness of the representation does not of course guarantee the correctness of the output.) If the semantic representation fails to bconvert, we know that there are type conflicts resulting from either: incorrect semantics assigned to some lexical entries; incorrect interpretation of one of the combinatory rules; or an inconsistency in the output of the syntactic component.</Paragraph>
      <Paragraph position="1"> We assigned lexical semantics to the 245 most frequent categories from the complete set of 409, and implemented 4 of the type-raising rules, and the 10 unary type-changing rules, used by the parser.</Paragraph>
      <Paragraph position="2"> We used section 00 from CCGbank for development purposes; section 23 (2,401 sentences) was used as the test set. The parser provides a syntactic analysis for 98.6% of the sentences in section 23. The accuracy of the parser is reported in Clark and Curran (2004b): 84.6% F-score over labelled dependencies for section 23. Of the sentences the parser analyses, 92.3% were assigned a semantic representation, all of which were well-formed. The output of the system for an example sentence is given in Figure 2.</Paragraph>
      <Paragraph position="3"> The reason for the lack of complete coverage is that we did not assign semantic representations to the complete set of lexical categories. In future work we will cover the complete set, but as a simple remedy we have implemented the following robustness strategy: we assign a semantic template to parts of the tree that could not be analysed. For example, the template for the NP category is lp.a7 x(p@x).</Paragraph>
      <Paragraph position="4"> This was done for the 10 most frequent categories and results in a coverage of 98.6%.</Paragraph>
      <Paragraph position="5"> Although we expect the accuracy of the semantic representations to mirror those of the syntactic component, and therefore be useful in NLP applications, there is still a small number of errors arising from different sources. First, some constructions are incorrectly analysed in CCGbank; for example, appositives in CCGbank are represented as coordinate constructions (Hockenmaier, 2003). Second, errors are introduced by the semantic construction component; for example, the non-head nouns in a noun-noun compound are currently treated as modifiers of the head noun, in the same way as adjectives. And finally, the parser introduces errors because of incomplete coverage of the lexicon, and mistakes due to the parsing model. We expect general improvements in statistical parsing technology will further improve the accuracy of the parser, and we will further develop the semantic component.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>