<?xml version="1.0" standalone="yes"?>
<Paper uid="W01-0716">
  <Title>Learning to identify animate references</Title>
  <Section position="4" start_page="0" end_page="1" type="metho">
    <SectionTitle>
3 The method
</SectionTitle>
    <Paragraph position="0"> In this section a two step method used to classify words according to their animacy is presented. In Section 3.1, we present an automatic method for determining the animacy of senses from WordNet on the basis of an annotated corpus. Once the senses from WordNet have been classified, a classical machine learning technique uses this information to determine the animacy of a noun for which the sense is not known. This technique is presented in Section 3.2.</Paragraph>
    <Section position="1" start_page="0" end_page="1" type="sub_section">
      <SectionTitle>
3.1 The classification of the senses
</SectionTitle>
      <Paragraph position="0"> As previously mentioned, the unique beginners are too general to be satisfactorily classified as animate or inanimate. However, this does not  mean that it is not possible to uniquely classify more specific senses as animate or inanimate. In this section, we present a corpus-based method which classifies the synsets from WordNet according to their animacy.</Paragraph>
      <Paragraph position="1"> The NPs in a 52 file subset of the SEMCOR corpus were manually annotated with animacy information and then used by an automatic system to classify the nodes. These 52 files contain 2512 animate entities and 17514 inanimate entities. The system attempts to classify the senses from WordNet that explicitly appear in the corpus directly, on the basis of their frequency.  However, our goal is to design a procedure which is also able to classify senses that are not found in the corpus. To this end, we decided to use a bottom up procedure which starts by classifying the terminal nodes and then continues with more general nodes. The terminal nodes are classified using the information straight from the annotated files. When classifying a more general node, the following hypothesis is used: &amp;quot;if all the  Due to linguistic ambiguities and tagging errors, not all the senses at this level can be classified adequately in this way.</Paragraph>
      <Paragraph position="2"> hyponyms of a sense are animate, then the sense itself is animate&amp;quot;. However, this does not always hold because of annotation errors or rare uses of a sense and instead, a statistical measure must be used to test the animacy of a more general node. Several measures were considered and the most appropriate one seemed to be chi-square.</Paragraph>
      <Paragraph position="3"> Chi-square is a non-parametric test which can be used for estimating whether or not there is any difference between the frequencies of items in frequency tables (Oakes, 1998). The formula used to calculate chi-square is:</Paragraph>
      <Paragraph position="5"> where O is the observed number of cases and E the expected number of cases. If AV</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="1" end_page="3" type="metho">
    <SectionTitle>
BE
</SectionTitle>
    <Paragraph position="0"> is less than or equal to a critical level, we may conclude that the observed and expected values do not differ significantly.</Paragraph>
    <Paragraph position="1"> Each time that a more general node is to be classified, its hyponyms are considered. If all the hyponyms observed in the corpus  are annotated as either animate or inanimate (but not both), the  Either directly or indirectly via the hyponymy relations. Generalisation rejected.... for hypernym Def:(any living entity)  Generalisation accepted .... for hypernym Def:(the continuum of experience in which events pass from the future through the present to the past)  more general node is classified as its hyponyms are. However, for the aforementioned reasons, this rule does not apply in all cases. In the remaining cases the chi-square test is applied. For each more general node which is about to be classified, two hypotheses are tested: the first one considers the node animate and the second one inanimate. The system classifies the node according to which test is passed. If neither are passed, it means that the node is too general and it and all its hypernyms can equally refer to both animate and inanimate entities.</Paragraph>
    <Paragraph position="2"> For example, a more general node can have several hyponyms as shown in Figure 1. In that case, the hypernym has n hyponyms. We consider each sense to have two attributes: the number of times it has been annotated as animate (CPD2CX</Paragraph>
    <Paragraph position="4"> ). For more general nodes, these attributes are the sum of the number of animate/inanimate instances of its hyponyms. When the node is tested to determine whether or not it is animate, a contingency table like Table 1 is built. Given that we are testing to see if the more general node is animate or not, for each of its hyponyms, the total number of occurrences of a sense in the annotated corpus is the expected value (meaning that all the instances should be animate) and the number of times the hyponym is annotated as referring to an animate entity is the observed value. Formula 1 is used to compute chi-square, and the result is compared with the critical level obtained for n-1 degrees of freedom and a significance level of .05. If the test is passed, the more general node is classified as animate. In a similar way, more general nodes are tested for inanimacy. Figures 2 and 3 show two small examples in which the generalisation is rejected and accepted, respectively.</Paragraph>
    <Paragraph position="5"> In order to be a valid test of significance, chi-square usually requires expected frequencies to be 5 or more. If the contingency table is larger than two-by-two, some few exceptions are allowed as long as no expected frequency is less than one and no more than 20% of the expected frequencies are less than 5 (Sirkin, 1995). In our case it is not possible to have expected frequencies less than one because this would entail no presence in the corpus. If, when the test is applied, more than 20% of the senses have an expected frequency less than 5, the two similar senses with the lowest frequency are merged and the test is repeated.</Paragraph>
    <Paragraph position="6">  If no senses can be merged and still more than 20% of the expected frequencies are less than 5, the test is rejected.</Paragraph>
    <Section position="1" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
3.2 The classification of a word
</SectionTitle>
      <Paragraph position="0"> The classification described in the previous section is useful for determining the animacy of a sense, even for those which were not previously found in the annotated corpus, but which are hyponyms of a node that has been classified.</Paragraph>
      <Paragraph position="1"> However, nouns whose sense is unknown cannot be classified directly and therefore an additional level of processing is necessary. In this section, we show how TiMBL (Daelemans et al., 2000)  Two senses are considered similar if they both have the same attribute equal to zero.</Paragraph>
      <Paragraph position="2"> was used to determine the animacy of nouns.</Paragraph>
      <Paragraph position="3"> TiMBL is a program which implements several machine learning techniques. After trying the algorithms available in TiMBL with different configurations, the best results were obtained using instance-based learning with gain ratio as the weighting measure (Quinlan, 1993; Mitchell, 1997). In this type of learning, all the instances are stored without trying to infer anything from them. At the classification stage, the algorithm compares a previously unseen instance with all the data stored at the training stage. The most frequent class in the k nearest neighbours is assigned as the class to which that instance belongs. After experimentation, it was noticed that the best results were obtained when k=3.</Paragraph>
      <Paragraph position="4"> In our case the instances used in training and classification consist of the following information: AF The lemma of the noun which is to be classified.</Paragraph>
      <Paragraph position="5"> AF The number of animate and inanimate senses of the word. As we mentioned before, in the cases where the animacy of a sense is not known, it is inferred from its hypernyms.</Paragraph>
      <Paragraph position="6"> If this information cannot be found for any of a word's hypernyms, information on the unique beginners for the word's sense is used, in a manner similar to that used in (Evans and OrVasan, 2000).</Paragraph>
      <Paragraph position="7"> AF If the word is the head of a subject, the number of animate/inanimate senses of its verb. For those senses for which the classification is not known, an algorithm similar to the one described for nouns is employed. These values are 0 for heads of non-subjects.</Paragraph>
      <Paragraph position="8"> AF The ratio of the number of animate singular pronouns (e.g he or she) to inanimate singular pronouns (e.g. it) in the whole text. The output of this stage is a list of nouns classified according to their animacy.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>