<?xml version="1.0" standalone="yes"?>
<Paper uid="W97-0212">
  <Title>Sense Tagging in Action Combining Different Tests with Additive Weightings</Title>
  <Section position="4" start_page="0" end_page="75" type="metho">
    <SectionTitle>
3. ProcJ~ure
</SectionTitle>
    <Paragraph position="0"> Besides some simple tests for suffixes (for unknown words), capitalisation, register and frequency, the main tagging processes are the following:</Paragraph>
    <Section position="1" start_page="0" end_page="74" type="sub_section">
      <SectionTitle>
3.1 Multi-word unit tagger
</SectionTitle>
      <Paragraph position="0"> The CIDE database contains detailed information on both single words and multi-word units. For a word pair X Y (e.g. has been), the tagger is thus able to produce possible scores for X and Y as separate words, and for X Y as a multi-word unit throughout each</Paragraph>
      <Paragraph position="2"> tagging process. If a multi-word unit is found, it is given an initial additional score (a headstart over the words treated separately) proportional to the number of words in the unit minus 1, but this can easily be cancelled out by other scores.</Paragraph>
      <Paragraph position="3"> As a learner dictionary, CIDE contains much examples text. This examples text forms a convenient hand sense tagged corpus, though with only one word (the headword) sense tagged in each example. Much research has been devoted to using just collocation information for sense disambignation, even using contexts of as much as 50 words (Gale, Church and Yarowsky, 1992). We instead choose to look more at the immediate context around a word, by dividing collocation match weightings by the distance between the pair of collocating words, expecting subject domain tagging (see section 3.2) to deal with more long-range effects.</Paragraph>
    </Section>
    <Section position="2" start_page="74" end_page="74" type="sub_section">
      <SectionTitle>
3.2 Subject domain tagger
</SectionTitle>
      <Paragraph position="0"> Each entry in CIDE has been subject coded. A subject domain for the sentence is created by looking at the subject codes of each likely (from the tests so far) sense of every word in the sentence, and at any document information available about the subject domain of the article, e.g. a sports page. Then the subject codes of each sense of each word are compared with the subject domain for the sentence and the number of matches noted. The subject codes are arranged in a hierarchy, so for example, Christmas and Passover would match at some levels, despite not having exactly the same subject code. Long sentences can distort the results, so the weightings awarded to subject domain matches are divided by the number of words in the sentence.</Paragraph>
    </Section>
    <Section position="3" start_page="74" end_page="74" type="sub_section">
      <SectionTitle>
3.3 Part of speech tagger
</SectionTitle>
      <Paragraph position="0"> Our part-of-speech tagger is based on a series of rules, listing valid 'transition pair' sequences of grammatical tags. These pairs can be given weightings but the emphasis of the approach is on the list of valid pairs rather than the weightings assigned to each pair. Thus most valid pairs are given a standard weighting of 0.</Paragraph>
      <Paragraph position="1"> Six special intermediate tags have been created to reduce the number of tag pairs that need to be listed and to add 'partial parsing' to the process. These are: p\[ and p\] around noun phrases acting as subjects (i.e.</Paragraph>
      <Paragraph position="2"> expecting to be followed by a verb) p&lt; and p&gt; around noun phrases acting as objects p( and p) around adverbial or prepositional phrases, or sub-clauses Thus, for example, a determiner may only be preceded by p\[ or p&lt; or a pre-determiner. The p( and p) are a particularly powerful feature which enable intermediate phrases to be ignored. The tagger does not check for p) followed by the next tag, but rather looks back to what came innnediately before the preceding p( and then does the transition pair match on that. Atwell (1987) has termed these kind of brackets &amp;quot;hyperbrackets&amp;quot; and considers a very similar approach to that we are now adopting, choosing himself instead to add hyperhrackets to already tagged text to enhance it with parsing information, but thereby losing the benefit these hyperhrackets can assign to the part-of-speech tagging process itself, One example of the possible benefit is in trying to make the distinction between a preposition, which is generally followed by what we term an object noun phrase as it will not be followed by a verb, and a subordinating conjunction, which is generally followed by what we term a subject noun phrase as it will be followed by a verb.</Paragraph>
      <Paragraph position="3"> For a valid transition pair between two tags, the score is simply calculated by adding the maximum score (from the other tagging processes) for a sense that can have each grammatical tag to the transition pair weighting (usually 0). There are also some special features to cope with more long-range effects (e.g.</Paragraph>
      <Paragraph position="4"> singular nouns being followed by the 3ps form of the present simple, conjunctions tending to co-ordinate the same grammatical tags). Thus, all valid sequences can be given a score by adding up the relevant transition pair scores.</Paragraph>
      <Paragraph position="5"> Our method is more ambitious but intrinsically less efficient than Hidden Markov Model .approaches, although certain restrictions are applied to reduce the number of sequences to a manageable size (e.g. a limit on the number of nested brackets). More time also needs to be spent on rule development.</Paragraph>
    </Section>
    <Section position="4" start_page="74" end_page="75" type="sub_section">
      <SectionTitle>
3.4 Selectional preference pattern tagger
</SectionTitle>
      <Paragraph position="0"> The selectional preference pattern tagger checks verb complementation and selectional preferences, and also adjective selectional preferences. Lexicographers have specifically attached CIDE grammar cedes (which give verb complementation patterns) to selectionai preference patterns using a restricted list of about 40 selectional classes for nouns. The tagger translates these grammar codes into sequences of grammatical tags and super-segmental tags representing the possible sequences that may follow the verb, and then integrates these with the selectional preference patterns.</Paragraph>
      <Paragraph position="1"> It is these resulting patterns that the pattern tagger uses to test the syntactic and semantic veracity of the tag sequences produced by the part-of-speech tagger. If  the argument pattern (subject and objects) fail to match a tag sequence, this is considered a verb complementation pattern failure. When an argument is encountered, the class specified in the selectionai preference pattern is matched against the possible classes for the word. Selectional classes are hierarchical in structure like subject domain codes (see section 3.2), so allowance is made for near-matches. Adjective selectional preferences are matched in a similar but more simple way. Each adjective is coded with the possible clnss(es) of the nouns which it may modify.</Paragraph>
      <Paragraph position="2"> The adjective class is matched against the class of the noun which it modifies using much the same scoring system as for the verbs.</Paragraph>
      <Paragraph position="3"> Selectionai preference pattern matching has proved one of the most useful of all tests. A good example is the sentence: The head asked the pupil a question.</Paragraph>
      <Paragraph position="4"> Here, the CIDE database gives the possible selectionai classes for head as body part, state, object, human or device; for pupil as human or body part; for question as communication or abstract.</Paragraph>
      <Paragraph position="5"> The verb asked with two objects can only have the pattern human asked human communication. Thus, all the senses can be correctly assigned just by using selectional preferences.</Paragraph>
    </Section>
    <Section position="5" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
3.5 Refinement
</SectionTitle>
      <Paragraph position="0"> There are three main processes involved in refining the tagger's performance: * Refining the lexicographic data, or indeed adding whole new categories of lexicographic data (e.g. selectional preference patterns).</Paragraph>
      <Paragraph position="1"> * Writing new algorithms (&amp;quot;taggers&amp;quot;). * Analysing the interaction between different tests, and refining the weightings used for each.</Paragraph>
      <Paragraph position="2"> A hand-tagged corpus is of course very useful for performing the third of these processes in a rigorous manner. The next stage of our research is to use the test corpus (section 4) as a training corpus to fine-tune the weightings. The main weightings currently in use, which may be of interest to other researchers trying to combine different tests, are shown in the table.</Paragraph>
      <Paragraph position="3"> An example of how different taggers can interact is given by the following two sentences: He was fired with enthusiasm by his boss.</Paragraph>
      <Paragraph position="4"> He was fired by his boss with enthusiasm.</Paragraph>
      <Paragraph position="5"> The DISMISS sense of fired matches with boss at 3 levels of subject domain coding, thus scoring 30*3/8 = 11 for both sentences.</Paragraph>
      <Paragraph position="6"> The EXCITE sense of fired has with as a functional collocate and enthusiasm as an illustrative collocate in CIDE, and thus scores 20/1 + 10/2 = 25 for the fu-st sentence and 20/4 + 1015 = 7 for the second sentence. Thus, assuming no other taggers intervene, the sense tagger will make the best possible assignment for these two, admittedly rather ambiguous, examples.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="75" end_page="76" type="metho">
    <SectionTitle>
4. Results
</SectionTitle>
    <Paragraph position="0"> To test the tagging, we compared the results against a previously hand sense tagged corpus of 4000 words.</Paragraph>
    <Paragraph position="1">  Each of the 4000 words was manually assigned with just one sense tag and the tagging program likewise assigned precisely one sense tag to each word. The results are thus strictly determined by the number of matching taggings, with no ambiguous coding allowed. (These criteria are somewhat over-strict as in some cases more than one tag could be considered acceptable, e.g. where there are cross-references in the dictionary or where there is genuine ambiguity.) In calculating the results, prepositions were deliberately ignored because they have been heavily &amp;quot;split&amp;quot; in CIDE, far more so than in other dictionaries (L~ar 1996). Any attempt at distinguishing these senses would have to rely heavily on selectional preferences for prepositions, which are yet to be implemented within the tagging program.</Paragraph>
    <Paragraph position="2"> At the sense (CIDE guideword) level, with an average 5 senses per word, the sense tagger was correct 78% of the time. At the sub-sense level, with an average 19 senses per word, the sense tagger was correct 73% of the time.</Paragraph>
    <Paragraph position="3"> The part of speech tagging was also tested on the same texts to similarly strict criteria (i.e. no ambiguous coding allowed) and found to assign the correct part of speech 91% of the time. Three other part of speech taggers were run on the same texts for comparison.</Paragraph>
    <Paragraph position="4"> Two taggers developed from work done at Cambridge University under the ACQUILEX programme assigned 93% and 87% correctly, while the commercial Prospero Parser performed best, assigning 94% correctly.</Paragraph>
  </Section>
class="xml-element"></Paper>