<?xml version="1.0" standalone="yes"?>
<Paper uid="J01-4002">
  <Title>Design and Enhanced Evaluation of a Robust Anaphor Resolution Algorithm</Title>
  <Section position="2" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
1. Introduction
</SectionTitle>
    <Paragraph position="0"> The interpretation of anaphoric expressions is known to be a difficult problem. In principle, a variety of constraints and preference heuristics, including factors that rely on semantic, pragmatic, and world knowledge, contribute to this task (Carbonell and Brown 1988). Robust, operational approaches to anaphor resolution on unrestricted discourse, however, are confined to strategies exploiting globally available evidence like morphosyntactic, syntactic, and surface information.</Paragraph>
    <Paragraph position="1"> Beginning with the pioneering work of Hobbs (1978), many practical approaches rely on the availability of syntactic surface structure by employing coindexing restrictions, salience criteria, and parallelism heuristics (e.g., Lappin and Leass 1994). However, even the assumption of the availability of a unique syntactic description is unrealistic since, in general, parsing involves the solution of difficult problems like at- null * DaimlerstraSe 32, D-60314 Frankfurt am Main. E-mail: roland@stuckardt.de (c) 2001 Association for Computational Linguistics  Computational Linguistics Volume 27, Number 4 tachment ambiguities, role uncertainty, and the instantiation of empty categories. Based on this observation, Kennedy and Boguraev (1996) have suggested an adaptation of Lappin and Leass's approach to the shallow analysis frontend of English Constraint Grammar (Karlsson et al. 1995), which provides a part-of-speech tagging comprising an assignment of syntactic function but no constituent structure. This information deficiency is partly overcome by the application of a regular filter that heuristically reconstructs parts of the constituent structure. An alternative solution, which is based on the possibly partial but potentially more comprehensive and reliable output of a conventional parser, has been suggested in Stuckardt (1997).</Paragraph>
    <Paragraph position="2"> In the present paper, an approach to robust anaphor resolution is developed that enhances the latter work. The coreference resolution algorithm ROSANA l is developed, the core of which consists of a set of rule patterns by means of which the verification of disjoint reference rules is generalized in order to make it applicable to deficient (fragmentary) syntactic descriptions. Based on this algorithm, the ROSANA system, which works on the partial syntactic descriptions generated by the robust FDG (Functional Dependency Grammar of English) parser of J~irvinen and Tapanainen (1997), is implemented. By a formal evaluation on two text corpora that differ with respect to genre and domain, it is proven that ROSANA achieves robust (truly operational) high-quality coreference resolution on unrestricted texts. An in-depth analysis shows that the robust implementation of syntactic disjoint reference is nearly optimal. Compared with approaches that rely on a combination of shallow preprocessing and heuristic syntactic disjoint reference, the largely nonheuristic disjoint reference algorithmization employed by ROSANA opens up the possibility for a slight improvement.</Paragraph>
    <Paragraph position="3"> The performance study of the ROSANA system crucially rests on an enhancement of the evaluation methodology for coreference resolution systems, the development of which constitutes the second major contribution of the paper. As a supplement to the coreference class scoring scheme that was developed for the CO-task evaluation of the Message Understanding Conferences (Vilain et al. 1996), two additional evaluation disciplines are defined that, on one hand, aim at supporting the developer of anaphor resolution systems, and, on the other hand, shed light on application aspects of pronoun interpretation. The evaluation of ROSANA according to the refined scoring scheme gives evidence that the interpretation quality may be improved by a genre-specific choice of the preference factors and their relative weights. This demonstrates the usefulness of enhancing the evaluation methodology for coreference resolution systems.</Paragraph>
    <Paragraph position="4"> The paper is organized as follows. In Section 2, the robustness issue of natural language processing is briefly discussed at a general level, and two models of robust anaphor resolution are introduced. In Section 3, by deriving a set of disjoint reference rule patterns for fragmentary syntax, the core component of a robust, operational anaphor resolution algorithm is developed. In Section 4, the ROSANA algorithm is designed, and an implementation, the ROSANA system, is described. In Section 5, an enhanced set of evaluation disciplines for coreference resolution systems is advocated for, and the respective formal measures are defined. In Section 6, the evaluation results of ROSANA are discussed. Finally, in section 7, ROSANA is compared with other approaches to anaphor resolution and, in particular, robust syntactic disjoint reference.</Paragraph>
  </Section>
class="xml-element"></Paper>