<?xml version="1.0" standalone="yes"?>
<Paper uid="E06-2007">
  <Title>Selecting the &amp;quot;Right&amp;quot; Number of Senses Based on Clustering Criterion Functions</Title>
  <Section position="4" start_page="112" end_page="113" type="metho">
    <SectionTitle>
3 Experimental Results
</SectionTitle>
    <Paragraph position="0"> Weconductedexperimentswithwordsthathave2, 3, 4, and 6 actual senses. We used three words that had been manually sense tagged, including the 3 sense adjective hard, the 4 sense verb serve, and the 6 sense noun line. We also created 19 name conflations where sets of 2, 3, 4, and 6 names of persons, places, or organizations that are included in the English GigaWord corpus (and that are typically unambiguous) are replaced with a single name to create pseudo or false ambiguities. For example, we replaced all mentions of Bill Clinton and Tony Blair with a single name that can refer to either of them. In general the names we used in these sets are fairly well known and occur hundreds or even thousands of times.</Paragraph>
    <Paragraph position="1"> We clustered each word or name using four different configurations of our clustering approach, in order to determine how consistent the selected value of k is in the face of changing feature sets and context representations. The four configurations are first order feature vectors made up of unigrams that occurred 5 or more times, with and without singular value decomposition, and then second order feature vectors based on bigrams that occurred 5 or more times and had a log-likelihood score of 3.841 or greater, with and without singular value decomposition. Details on these approaches can be found in (Purandare and Pedersen, 2004).</Paragraph>
    <Paragraph position="2"> Thus, in total there are 22 words to be discriminated, 7 with 2 senses, 6 words with 3 senses, 6 with 4 senses, and 3 words with 6 senses. Four different configurations of clustering are run for each word, leading to a total of 88 experiments.</Paragraph>
    <Paragraph position="3"> The results are shown in Tables 1, 2, and 3. In these tables, the actual numbers of senses are in the columns, and the predicted number of senses are in the rows.</Paragraph>
    <Paragraph position="4"> We see that the predicted value of PK1 agreed  with the actual value in 15 cases, whereas PK3 agreed in 17 cases, and PK2 agreed in 22 cases.</Paragraph>
    <Paragraph position="5"> We observe that PK1 and PK3 also experienced considerable confusion, in that their predictions were in many cases several clusters off of the correct value. While PK2 made various mistakes, it was generally closer to the correct values, and had fewer spurious responses (very large or very small predictions). We note that the distribution of PK2's predictions were most like those of the actual senses.</Paragraph>
  </Section>
class="xml-element"></Paper>