<?xml version="1.0" standalone="yes"?>
<Paper uid="P02-1063">
  <Title>Revision Learning and its Application to Part-of-Speech Tagging</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Recently, corpus-based approaches have been widely studied in many natural language processing tasks, such as part-of-speech (POS) tagging, syntactic analysis, text categorization and word sense disambiguation. In corpus-based natural language processing, one important issue is to decide which learning model to use.</Paragraph>
    <Paragraph position="1"> Various learning models have been studied such as Hidden Markov models (HMMs) (Rabiner and Juang, 1993), decision trees (Breiman et al., 1984) and maximum entropy models (Berger et al., 1996). Recently, Support Vector Machines (SVMs) (Vapnik, 1998; Cortes and Vapnik, 1995) are getting to be used, which are supervised machine learning algorithm for binary classification. SVMs have good generalization performance and can handle a large number of features, and are applied to some tasks / Presently with Oki Electric Industry successfully (Joachims, 1998; Kudoh and Matsumoto, 2000). However, their computational cost is large and is a weakness of SVMs. In general, a trade-off between capacity and computational cost of learning models exists. For example, SVMs have relatively high generalization capacity, but have high computational cost. On the other hand, HMMs have lower computational cost, but have lower capacity and difficulty in handling data with a large number of features. Learning models with higher capacity may not be of practical use because of their prohibitive computational cost. This problem becomes more serious when a large amount of data is used.</Paragraph>
    <Paragraph position="2"> To solve this problem, we propose a revision learning method which combines a model with high generalization capacity and a model with small computational cost to achieve high performance with small computational cost. This method is based on the idea that processing the entire target task using a model with higher capacity is wasteful and costly, that is, if a large portion of the task can be processed easily using a model with small computational cost, it should be processed by such a model, and only difficult portion should be processed by the model with higher capacity.</Paragraph>
    <Paragraph position="3"> Revision learning can handle a general multi-class classification problem, which includes POS tagging, text categorization and many other tasks in natural language processing. We apply this method to English POS tagging and Japanese morphological analysis.</Paragraph>
    <Paragraph position="4"> This paper is organized as follows: Section 2 describes the general multi-class classification Computational Linguistics (ACL), Philadelphia, July 2002, pp. 497-504. Proceedings of the 40th Annual Meeting of the Association for problem and the one-versus-rest method which is known as one of the solutions for the problem. Section 3 introduces revision learning, and discusses how to combine learning models. Section 4 describes one way to conduct Japanese morphological analysis with revision learning.</Paragraph>
    <Paragraph position="5"> Section 5 shows experimental results of English POS tagging and Japanese morphological analysis with revision learning. Section 6 discusses related works, and Section 7 gives conclusion.</Paragraph>
  </Section>
class="xml-element"></Paper>