<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0813">
  <Title>The Basque Country University system: English and Basque tasks</Title>
  <Section position="6" start_page="3" end_page="3" type="evalu">
    <SectionTitle>
5 Results and Conclusions
</SectionTitle>
    <Paragraph position="0"> Table 5 shows the performance obtained by our systems and the winning system in the Senseval-3 evaluation. We can see that we are very close to the best algorithms in both languages.</Paragraph>
    <Paragraph position="1"> The recall of our systems is 1.2%-1.9% lower than cross-validation for every system and task, which is not surprising when we change the setting. The combination of methods is useful for English, where we improve the recall in 0.3%, reaching 72.3%. The difference is statistically significant according to McNemar's test.</Paragraph>
    <Paragraph position="2"> However, the combination of methods does not improve the results in the the Basque task, where the SVM method alone provides better  Basque lexical tasks (recall).</Paragraph>
    <Paragraph position="3"> results (69.9% recall). In this case the difference is not significant applying McNemar's test.</Paragraph>
    <Paragraph position="4"> Our disambiguation procedure shows a similar behavior on the Senseval-2 and Senseval-3 data for English (both in cross-validation and in the testing part), where the ensemble works best, followed by the vector model. This did not apply to the Basque dataset, where some algorithms seem to perform below the expectations. For future work, we plan to study better the Basque feature set and include new features, such as domain tags.</Paragraph>
    <Paragraph position="5"> Overall, the ensemble of algorithms provides a more robust system for WSD, and is able to achieve state-of-the-art performance.</Paragraph>
    <Section position="1" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
6Acknowledgements
</SectionTitle>
      <Paragraph position="0"> We wish to thank both David Yarowsky's group, from Johns Hopkins University, and Gerard Escudero's group, from Universitat Politecnica de Catalunya, for providing us software for the acquisition of features. This research has been partially funded by the European Commission</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>