<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-1304">
  <Title>Coaxing Confidences from an Old Friend: Probabilistic Classifications from Transformation Rule Lists</Title>
  <Section position="3" start_page="0" end_page="26" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> In natural language processing, a great amount of work has gone into the development of machine learning algorithms which extract useful linguistic information from resources such as dictionaries, newswire feeds, manually annotated corpora and web pages. Most of the effective methods can be roughly divided into rule-based and probabilistic algorithms. In general, the rule-based methods have the advantage of capturing the necessary information in a small and concise set of rules. In part-of-speech tagging, for example, rule-based and probabilistic methods achieve comparable accuracies, but rule-based methods capture the knowledge in a hundred or so simple rules, while the probabilistic methods have a very high--dimensional parameter space (millions of parameters).</Paragraph>
    <Paragraph position="1"> One of the main advantages of probabilistic methods, on the other hand, is that they include a measure of uncertainty in their output. This can take the form of a probability distribution over potential outputs, or it may be a ranked list of IA11 the experiments are performed on text chnnklng. The technique presented is general-purpose, however, and can be applied to many tasks for which transformation-based learning performs well, without changing the interrials of the learner.</Paragraph>
    <Paragraph position="2"> candidate outputs. These uncertainty measures are useful in situations where both the classification of an sample and the system's confidence in that classification are needed. An example of this is a situation in an ensemble system where ensemble members disagree and a decision must be made about how to resolve the disagreement.</Paragraph>
    <Paragraph position="3"> A similar situation arises in pipeline systems, such as a system which performs parsing on the output of a probabilistic part-of-speech tagging.</Paragraph>
    <Paragraph position="4"> Transformation-based learning (TBL) (Brill, 1995) is a successful rule-based machine learning algorithm in natural language processing. It has been applied to a wide variety of tasks, including part of speech tagging (Roche and Schabes, 1995; Brill, 1995), noun phrase chvnklng (Ramshaw and Marcus, 1999), parsing (Brill, 1996; Vilain and Day, 1996), spelling correction (Mangu and Brill, 1997), prepositional phrase attachment (Brill and Resnik, 1994), dialog act tagging (Samuel et al., 1998), segmentation and message understanding (Day et al., 1997), often achieving state-of-the-art performance with a small and easilyunderstandable list of rules.</Paragraph>
    <Paragraph position="5"> In this paper, we describe a novel method which enables a transformation-based classifier to generate a probability distribution on the class labels. Application of the method allows the transformation rule list to retain the robustness of the transformation-based algorithms, while benefitting from the advantages of a probabilistic classifter. The usefulness of the resulting probabilities is demonstrated by comparison with another state-of-the-art classifier, the C4.5 decision tree (Quinlan, 1993). The performance of our algorithm compares favorably across many dimensions: it obtains better perplexity and cross-entropy; an active learning algorithm using our system outperforms a similar algorithm using decision trees; and finally, our algorithm has better rejection curves than a similar decision tree. Section 2 presents the transformation based learning paradigm; Section 3 describes the algorithm for construction of the decision tree associated with the transformation based list; Section 4 describes the experiments in detail and Section 5 concludes the paper and outlines the future work.</Paragraph>
  </Section>
class="xml-element"></Paper>