<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0822">
  <Title>Augmenting Ensemble Classification for Word Sense Disambiguation with a Kernel PCA Model</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Classifier combination has become a standard architecture for shared task evaluations in word sense disambiguation (WSD), named entity recognition, and similar problems that can naturally be cast as classification problems. Voting is the most common method of combination, having proven to be remarkably effective yet simple.</Paragraph>
    <Paragraph position="1"> A key problem in improving the accuracy of such ensemble classification systems is to find new voting models that (1) exhibit significantly different prediction biases from the models already voting, and yet (2) attain stand-alone classification accuracies that are as good or better. When either of these conditions is not met, adding the new voting model typically degrades the accuracy of the ensemble instead of helping it.</Paragraph>
    <Paragraph position="2"> In this work, we investigate the potential of one promising new disambiguation model with respect 1The author would like to thank the Hong Kong Research Grants Council (RGC) for supporting this research in part through research grants RGC6083/99E, RGC6256/00E, and DAG03/04.EG09.</Paragraph>
    <Paragraph position="3"> to augmenting our existing ensemble combining a maximum entropy model, a boosting model, and a na&amp;quot;ive Bayes model--a combination representing some of the best stand-alone WSD models currently known. The new WSD model, proposed by Wu et al. (2004), is a method for disambiguating word senses that exploits a nonlinear Kernel Principal Component Analysis (KPCA) technique.</Paragraph>
    <Paragraph position="4"> That the KPCA-based model could potentially be a good candidate for a new voting model is suggested by Wu et al.'s empirical results showing that it yielded higher accuracies on Senseval-2 data sets than other models that included maximum entropy, na&amp;quot;ive Bayes, and SVM based models.</Paragraph>
    <Paragraph position="5"> In the following sections, we begin with a description of the experimental setup, which utilizes a number of individual classifiers in a voting ensemble. We then describe the KPCA-based model to be added to the baseline ensemble. The accuracy results of the three submitted models are examined, and also the individual voting models are compared.</Paragraph>
    <Paragraph position="6"> Subsequently, we analyze the degree of difference in voting bias of the KPCA-based model from the others, and finally show that this does indeed usually lead to accuracy gains in the voting ensemble.</Paragraph>
  </Section>
class="xml-element"></Paper>