<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-3021">
  <Title>Compiling Boostexter Rules into a Finite-state Transducer</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Many problems in Natural Language Processing (NLP) can be modeled as classi cation tasks either at the word or at the sentence level. For example, part-of-speech tagging, named-entity identi cation supertagging1, word sense disambiguation are tasks that have been modeled as classi cation problems at the word level. In addition, there are problems that classify the entire sentence or document into one of a set of categories. These problems are loosely characterized as semantic classi cation and have been used in many practical applications including call routing and text classi cation.</Paragraph>
    <Paragraph position="1"> Most of these problems have been addressed in isolation assuming unambiguous (one-best) input.</Paragraph>
    <Paragraph position="2"> Typically, however, in NLP applications these modules are chained together with each module introducing some amount of error. In order to alleviate the errors introduced by a module, it is typical for a module to provide multiple weighted solutions (ideally as a packed representation) that serve as input to the next module. For example, a speech recognizer provides a lattice of possible recognition outputs that is to be annotated with part-of-speech and 1associating each word with a label that represents the syntactic information of the word given the context of the sentence. named-entities. Thus classi cation approaches need to be extended to be applicable on weighted packed representations of ambiguous input represented as a weighted lattice. The research direction we adopt here is to compile the model of a classi er into a weighted nite-state transducer (WFST) so that it can compose with the input lattice.</Paragraph>
    <Paragraph position="3"> Finite state models have been extensively applied to many aspects of language processing including, speech recognition (Pereira and Riley, 1997), phonology (Kaplan and Kay, 1994), morphology (Koskenniemi, 1984), chunking (Abney, 1991; Bangalore and Joshi, 1999), parsing (Roche, 1999; O azer, 1999) and machine translation (Vilar et al., 1999; Bangalore and Riccardi, 2000). Finite-state models are attractive mechanisms for language processing since they (a) provide an ef cient data structure for representing weighted ambiguous hypotheses (b) generally effective for decoding (c) associated with a calculus for composing models which allows for straightforward integration of constraints from various levels of speech and language processing.2 In this paper, we describe the compilation process for a particular classi er model into an WFST and validate the accuracy of the compilation process on a one-best input in a call-routing task. We view this as a rst step toward using a classi cation model on a lattice input. The outline of the paper is as follows. In Section 2, we review the classi cation approach to resolving ambiguity in NLP tasks and in Section 3 we discuss the boosting approach to classi cation. In Section 4 we describe the compilation of the boosting model into an WFST and validate the result of this compilation using a call-routing task.</Paragraph>
  </Section>
class="xml-element"></Paper>