<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0839">
  <Title>Complementarity of Lexical and Simple Syntactic Features: The SyntaLex Approach to SENSEVAL-3</Title>
  <Section position="4" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
4 Experiments and Discussion
</SectionTitle>
    <Paragraph position="0"> The SyntaLex systems are used to perform a series of word sense disambiguation experiments using lexical and syntactic features both individually and in combination. The C4.5 algorithm, as implemented by the J48 program in the Waikato Environ- null 2000) is used to learn bagged decision trees for each word to be disambiguated.</Paragraph>
    <Paragraph position="1"> Ten decision trees are learned for each task based on ten different samples of training instances. Each sample is created by drawing N instances, with replacement, from a training set consisting of N total instances. Given a test instance, weighted scores for each sense provided by each of the ten decision trees are summed. The sense with the highest score is chosen as the intended sense.</Paragraph>
    <Paragraph position="2"> A majority classifier which always chooses the most frequent sense of a word in the training data, achieves an accuracy of 56.5%. This result acts as a baseline to which our results may be compared. The decision trees learned by our system fall back on the most frequent sense in case the identified features are unable to disambiguate the target word. Thus, the classification of all test instances is attempted and we therefore report our results (Table 1) in terms of accuracies. The break down of the coarse and fine grained accuracies for nouns, verbs and adjectives is also depicted.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 SyntaLex-1: Part of Speech Features
(Narrow Context)
</SectionTitle>
      <Paragraph position="0"> SyntaLex-1 uses bagged decision trees to classify a target word based on its Part of Speech and that of its immediate neighbors. The nodes in the decision trees are features of form: Pa3a4a1 = a6 Taga7 ,</Paragraph>
      <Paragraph position="2"> sents any Part of Speech. Consider a sentence where the target word line is used in the plural form, has a personal pronoun preceding it and is not followed by a preposition. A decision tree based on such Part of Speech features as described above is likely to capture the intuitive notion that in such cases line is used in the line of text sense, as in, the actor forgot his lines or they read their lines slowly. Similarly, if the word following line is a preposition, the tree is likely to predict the product sense, as in, the line of clothes.</Paragraph>
      <Paragraph position="3"> The system achieves a fine grained accuracy of 62.4% and a coarse grained accuracy of 69.1%.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 SyntaLex-2: Part of Speech Features
(Broad Context)
</SectionTitle>
      <Paragraph position="0"> SyntaLex-2, like SyntaLex-1, uses bagged decision trees based on part of speech features for word sense disambiguation. However, it relies on the Part of Speech of words within a broader window around the target word. The Part of Speech of words in a sentence have local influence. The Part of Speech of words further away from the target word are not expected to be as strong indicators of intended sense as the immediate neighbors. However, inclusion of such features has been shown to improve accuracies (Mohammad and Pedersen, 2004).</Paragraph>
      <Paragraph position="1"> The nodes in the decision trees are features of the form: Pa3a5a2 = a6 Taga7 , Pa3a4a1 = a6 Taga7 , Pa0 = a6 Taga7 ,</Paragraph>
      <Paragraph position="3"> The system achieves a fine grained and coarse grained accuracy of 61.8% and 68.4%, respectively.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.3 SyntaLex-3: Ensemble of Lexical and
Simple Syntactic Features
</SectionTitle>
      <Paragraph position="0"> Prior research has shown that both lexical and syntactic features can individually achieve a reasonable quality of disambiguation. Further, some of the work (e.g., (McRoy, 1992), (Ng and Lee, 1996)) suggests that using both kinds of features may result in significantly higher accuracies as compared to individual results.</Paragraph>
      <Paragraph position="1"> SyntaLex-3 utilizes Part of Speech features and bigrams. Individual classifiers based on both kinds of features are learned. Given a test instance, both classifiers assign probabilities to every possible sense. The probabilities assigned to a particular sense are summed and the sense with the highest score is chosen as the desired sense. A narrow context of Part of Speech features is used for the syntactic decision tree that has features of the form: Pa3a4a1 =</Paragraph>
      <Paragraph position="3"> SyntaLex-3 achieves a fine grained accuracy of 64.6% and a coarse grained accuracy of 72.0%.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.4 SyntaLex-4: Combination of Lexical
and Simple Syntactic Features
</SectionTitle>
      <Paragraph position="0"> SyntaLex-4 also relies on a combination of PoS and bigram features but uses unified decision trees that can have either kind of feature at a particular node. In an ensemble, for a sense to be chosen as the intended one, both classifiers must assign reasonably high probabilities to it. A low score for a particular sense by any of the classifiers will likely entail its rejection. However, in certain instances, the context may be rich in useful disambiguating features of one kind but not of the other.</Paragraph>
      <Paragraph position="1"> A unified decision tree based on both kinds of features has the flexibility of choosing the intended sense based on one or both kinds of features and hence likely to be more successful. It must be noted though that throwing in a large number of features intensifies the data fragmentation problem of decision trees.</Paragraph>
      <Paragraph position="2"> SyntaLex-4achieves a fine grained and coarse grained accuracies of 63.3% and 71.1%, respectively. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>