<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0849">
  <Title>Class-based Collocations for Word-Sense Disambiguation</Title>
  <Section position="4" start_page="2" end_page="2" type="evalu">
    <SectionTitle>
4 Results and Discussion
</SectionTitle>
    <Paragraph position="0"> Disambiguation is performed via a decision tree formulated using Weka's J4.8 classifier (Witten and Frank, 1999). For the system used in the competition, the decision tree was learned over the entire Senseval-3 training data and then applied to the test data. Table 1 shows the results of our system in the Senseval-3 competition.</Paragraph>
    <Paragraph position="1"> Table 2 shows the results of 10-fold cross-validation just over the Senseval-3 training data (using Naive Bayes rather than decision trees.) To illustrate the contribution of the three types</Paragraph>
    <Section position="1" start_page="2" end_page="2" type="sub_section">
      <SectionTitle>
Experiment Precision
</SectionTitle>
      <Paragraph position="0"> All values are averages, except #Words,which is the number of distinct word types classified.</Paragraph>
      <Paragraph position="1"> Baseline always uses the most-frequent sense.</Paragraph>
      <Paragraph position="2"> of class-based collocations, the table shows results separately for systems developed using a single feature type, as well as for all features in combination. In addition, the performance of these systems are shown with and without the use of the local features (Local), as well as with and without the use of standard word collocations (WordColl). As can be seen, the related-word and definition collocations perform better than hypernym collocations when used alone.</Paragraph>
      <Paragraph position="3"> However, hypernym collocations perform better when combined with other features. Future work will investigate ways of ameliorating such interactions. The best overall system  (HyperColl+WordColl+Local)usesthecombination of local-context features, word collocations, and hypernym collocations. The performance of this system compared to a more typical system for WSD (WordColl+Local)isstatistically significant at p&lt;.05, using a paired t-test.</Paragraph>
      <Paragraph position="4"> We analyzed the contributions of the various collocation types to determine their effectiveness. Table 3 shows performance statistics for each collocation type taken individually over the training data. Precision is based on the number of correct positive indicators versus the total number of positive indicators, whereas recall is the number correct over the total number of training instances (7706). This shows that hypernym collocations are nearly as effective as word collocations. We also analyzed the occurrence of unique positive indicators provided by the collocation types over the training data. Ta- null Total #Pos. is number of positive indicators for the collocation in the training data, and Total #Corr. is the number of these that are correct.</Paragraph>
      <Paragraph position="5">  Unique #Pos. is number of training instances with the feature as the only positive indicator, and Unique #Corr. is number of these correct.</Paragraph>
      <Paragraph position="6"> ble 4 shows how often each feature type is positive for a particular sense when all other features for the sense are negative. This occurs fairly often, suggesting that the different types of collocations are complementary and thus generally useful when combined for word-sense disambiguation. Both tables illustrate coverage problems for the definition and related word collocations, which will be addressed in future work.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>