<?xml version="1.0" standalone="yes"?>
<Paper uid="H01-1006">
  <Title>Answering What-Is Questions by Virtual Annotation</Title>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3. VIRTUAL ANNOTATION
</SectionTitle>
    <Paragraph position="0"> If we look up a word in a thesaurus such as WordNet [Miller et al., 1993]), we can discover its hypernym tree, but there is no indication which hypernym is the most appropriate to answer a What is question. For example, the hypernym hierarchy for &amp;quot;nematode&amp;quot; is shown in Table 1. The level numbering counts levels up from the starting term. The numbers in parentheses will be explained later.</Paragraph>
    <Paragraph position="1">  At first sight, the desirability of the hypernyms seems to decrease with increasing level number. However, if we examine &amp;quot;meerkat&amp;quot; we find the hierarchy in Table 2.</Paragraph>
    <Paragraph position="2"> We are leaving much unsaid here about the context of the question and what is known of the questioner, but it is not unreasonable to assert that the &amp;quot;best&amp;quot; answer to &amp;quot;What is a meerkat&amp;quot; is either &amp;quot;a mammal&amp;quot; (level 4) or &amp;quot;an animal&amp;quot; (level 7). How do we get an automatic system to pick the right candidate?  It seems very much that what we would choose intuitively as the best answer corresponds to Rosch et al.'s basic categories [Rosch et al., 1976]. According to psychological testing, these are categorization levels of intermediate specificity that people tend to use in unconstrained settings. If that is indeed true, then we can use online text as a source of evidence for this tendency. For example, we might find sentences such as &amp;quot;... meerkats and other Y ...&amp;quot;, where Y is one of its hypernyms, indicating that Y is in some sense the preferred descriptor.</Paragraph>
    <Paragraph position="3"> We count the co-occurrences of the target search term (e.g.</Paragraph>
    <Paragraph position="4"> &amp;quot;meerkat&amp;quot; or &amp;quot;nematode&amp;quot;) with each of its hypernyms (e.g. &amp;quot;animal&amp;quot;) in 2-sentence passages, in the TREC9 corpus. These counts are the parenthetical numbers in Tables 1 and 2. The absence of a numerical label there indicates zero co-occurrences. Intuitively, the larger the count, the better the corresponding term is as a descriptor.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 Hypernym Scoring and Selection
</SectionTitle>
      <Paragraph position="0"> Since our ultimate goal is to find passages describing the target term, discovering zero co-occurrences allows elimination of useless candidates. Of those remaining, we are drawn to those with the highest counts, but we would like to bias our system away from the higher levels. Calling a nematode a life-form is correct, but hardly helpful.</Paragraph>
      <Paragraph position="1"> The top levels of WordNet (or any ontology) are by definition very general, and therefore are unlikely to be of much use for purposes of definition. However, if none of the immediate parents of a term we are looking up co-occur in our text corpus, we clearly will be forced to use a more general term that does.</Paragraph>
      <Paragraph position="2"> We want to go further, though, in those cases where the immediate parents do occur, but in small numbers, and the very general parents occur with such high frequencies that our algorithm would select them. In those cases we introduce a tentative level ceiling to prevent higher-level terms from being chosen if there are suitable lower-level alternatives.</Paragraph>
      <Paragraph position="3"> We would like to use a weighting function that decreases monotonically with level distance. Mihalcea and Moldovan [1999], in an analogous context, use the logarithm of the number of terms in a given term's subtree to calculate weights, and they claim to have shown that this function is optimal. Since it is approximately true that the level population increases exponentially in an ontology, this suggests that a linear function of level number will perform just as well.</Paragraph>
      <Paragraph position="4"> Our first step is to generate a level-adapted count (LAC) by dividing the co-occurrence counts by the level number (we are only interested in levels 1 and greater). We then select the best hypernym(s) by using a fuzzy maximum calculation. We locate the one or more hypernyms with greatest LAC, and then also select any others with a LAC within a predefined threshold of it; in our experimentation we have found that a threshold value of 20% works well. Thus if, for example, a term has one hypernym at level 1 with a count of 30, and another at level 2 with a count of 50, and all other entries have much smaller counts, then since the LAC 25 is within 20% of the LAC 30, both of these hypernyms will be proposed.</Paragraph>
      <Paragraph position="5"> To prevent the highest levels from being selected if there is any alternative, we tentatively exclude them from consideration according to the following scheme: If the top of the tree is at level N, where N &lt;= 3, we set a tentative ceiling at N-1, otherwise if N&lt;=5, we set the ceiling at N-2, otherwise we set the ceiling at N-3. If no co-occurrences are found at or below this ceiling, then it is raised until a positive value is found, and the corresponding term is selected.</Paragraph>
      <Paragraph position="6"> If no hypernym at all co-occurs with the target term, then this approach is abandoned: the &amp;quot;What&amp;quot; in the question is replaced by &amp;quot;THING$&amp;quot; and normal procedures of Predictive Annotation are followed.</Paragraph>
      <Paragraph position="7"> When successful, the algorithm described above discovers one or more candidate hypernyms that are known to co-occur with the target term. There is a question, though, of what to do when the question term has more than one sense, and hence more than one ancestral line in WordNet. We face a choice of either selecting the hypernym(s) with the highest overall score as calculated by the algorithm described above, or collecting together the best hypernyms in each parental branch. After some experimentation we made the latter choice. One of the questions that benefitted from this was &amp;quot;What is sake&amp;quot;. WordNet has three senses for sake: good (in the sense of welfare), wine (the Japanese drink) and aim/end, with computed scores of 122, 29 and 87/99 respectively.</Paragraph>
      <Paragraph position="8"> It seems likely (from the phrasing of the question) that the &amp;quot;wine&amp;quot; sense is the desired one, but this would be missed entirely if only the top-scoring hypernyms were chosen.</Paragraph>
      <Paragraph position="9"> We now describe how we arrange for our Predictive Annotation system to find these answers. We do this by using these descriptors as virtual QA-Tokens; they are not part of the search engine index, but are tagged in the passages that the search engine returns at run time.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Integration
</SectionTitle>
      <Paragraph position="0"> Let us use H to represent either the single hypernym or a disjunction of the several hypernyms found through the WordNet analysis. The original question Q =  &amp;quot;What is (a/an) X&amp;quot; is converted to Q' = &amp;quot;DEFINE$ X H&amp;quot;  where DEFINE$ is a virtual QA-Token that was never seen at indexing time, does not annotate any text and does not occur in the index. The processed query Q' then will find passages that contain occurrences of both X and H; the token DEFINE$ will be ignored by the search engine. The top passages returned by the search engine are then passed to Answer Selection, which re-annotates the text. However, this time the virtual QA-Token DEFINE$ is introduced and the patterns it matches are defined to be the disjuncts in H. In this way, all occurrences of the proposed hypernyms of X in the search engine passages are found, and are scored and ranked in the regular fashion. The end result is that the top passages contain the target term and one of its most frequently co-occurring hypernyms in close proximity, and these hypernyms are selected as answers. When we use this technique of Virtual Annotation on the aforementioned questions, we get answer passages such as &amp;quot;Such genes have been found in nematode worms but not yet in higher animals.&amp;quot; and &amp;quot;South African golfer Butch Kruger had a good round going in the central Orange Free State trials, until a mongoose-like animal grabbed his ball with its mouth and dropped down its hole. Kruger wrote on his card: &amp;quot;Meerkat.&amp;quot;&amp;quot;</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>