<?xml version="1.0" standalone="yes"?>
<Paper uid="P05-1050">
  <Title>Domain Kernels for Word Sense Disambiguation</Title>
  <Section position="7" start_page="406" end_page="407" type="evalu">
    <SectionTitle>
4 Evaluation and Discussion
</SectionTitle>
    <Paragraph position="0"> In this section we present the performance of our kernel-based algorithms for WSD. The objectives of these experiments are: to study the combination of different kernels, to understand the bene ts of plugging external information using domain models, to verify the portability of our methodology among different languages.</Paragraph>
    <Paragraph position="1"> 4The parameters p and l are optimized by cross-validation. The best results are obtained setting p = 2, l = 0.5 for KColl and l - 0 for KPoS.</Paragraph>
    <Section position="1" start_page="407" end_page="407" type="sub_section">
      <SectionTitle>
4.1 WSD tasks
</SectionTitle>
      <Paragraph position="0"> We conducted the experiments on four lexical sample tasks (English, Catalan, Italian and Spanish) of the Senseval-3 competition (Mihalcea and Edmonds, 2004). Table 2 describes the tasks by reporting the number of words to be disambiguated, the mean polysemy, and the dimension of training, test and unlabeled corpora. Note that the organizers of the English task did not provide any unlabeled material. So for English we used a domain model built from a portion of BNC corpus, while for Spanish, Italian and Catalan we acquired DMs from the unlabeled corpora made available by the organizers.</Paragraph>
    </Section>
    <Section position="2" start_page="407" end_page="407" type="sub_section">
      <SectionTitle>
4.2 Kernel Combination
</SectionTitle>
      <Paragraph position="0"> In this section we present an experiment to empirically study the kernel combination. The basic kernels (i.e. KBoW , KD, KColl and KPoS) have been compared to the combined ones (i.e. Kwsd and Kprimewsd) on the English lexical sample task.</Paragraph>
      <Paragraph position="1"> The results are reported in Table 3. The results show that combining kernels signi cantly improves the performance of the system.</Paragraph>
    </Section>
    <Section position="3" start_page="407" end_page="407" type="sub_section">
      <SectionTitle>
4.3 Portability and Performance
</SectionTitle>
      <Paragraph position="0"> We evaluated the performance of Kprimewsd and Kwsd on the lexical sample tasks described above. The results are showed in Table 4 and indicate that using DMs allowed Kprimewsd to signi cantly outperform Kwsd.</Paragraph>
      <Paragraph position="1"> In addition, Kprimewsd turns out the best systems for all the tested Senseval-3 tasks.</Paragraph>
      <Paragraph position="2"> Finally, the performance of Kprimewsd are higher than the human agreement for the English and Spanish tasks5.</Paragraph>
      <Paragraph position="3"> Note that, in order to guarantee an uniform application to any language, we do not use any syntactic information provided by a parser.</Paragraph>
    </Section>
    <Section position="4" start_page="407" end_page="407" type="sub_section">
      <SectionTitle>
4.4 Learning Curves
</SectionTitle>
      <Paragraph position="0"> The Figures 1, 2, 3 and 4 show the learning curves evaluated on Kprimewsd and Kwsd for all the lexical sample tasks.</Paragraph>
      <Paragraph position="1"> The learning curves indicate that Kprimewsd is far superior to Kwsd for all the tasks, even with few examples. The result is extremely promising, for it demonstrates that DMs allow to drastically reduce the amount of sense tagged data required for learning. It is worth noting, as reported in Table 5, that Kprimewsd achieves the same performance of Kwsd using about half of the training data.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>