<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-2606">
  <Title>Extended Lexical-Semantic Classi cation of English Verbs</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Levin's Classi cation
</SectionTitle>
    <Paragraph position="0"> Levin's classi cation (Levin, 1993) provides a summary of the variety of theoretical research done on lexical-semantic verb classi cation over the past decades. In this classi cation, verbs which display the same or similar set of diathesis alternations in the realization of their argument structure are assumed to share certain meaning components and are organized into a semantically coherent class. Although alternations are chosen as the primary means for identifying verb classes, additional properties related to subcategorization, morphology and extended meanings of verbs are taken into account as well.</Paragraph>
    <Paragraph position="1"> For instance, the Levin class of Break Verbs (class 45.1), which refers to actions that bring about a change in the material integrity of some entity, is characterized by its participation (1-3) or non-participation (4-6) in the following alternations and other constructions (7-8):  1. Causative/inchoative alternation: Tony broke the window a0 The window broke 2. Middle alternation: Tony broke the window a0 The window broke easily 3. Instrument subject alternation: Tony broke the window with the hammer a0 The hammer broke the window 4. *With/against alternation: Tony broke the cup against the wall a0 *Tony broke the wall with the cup 5. *Conative alternation: Tony broke the window a0 *Tony broke at the window 6. *Body-Part possessor ascension alternation: *Tony broke herself on the arm a0 Tony broke her arm 7. Unintentional interpretation available (some verbs): Re exive object: *Tony broke himself Body-part object: Tony broke his nger 8. Resultative phrase:  Tony broke the piggy bank open, Tony broke the glass to pieces Levin's taxonomy provides a classi cation of 3,024 verbs (4,186 senses) into 48 broad and 192 ne-grained classes according to their participation in 79 alternations involving NP and PP complements.</Paragraph>
    <Paragraph position="2"> Some extensions have recently been proposed to this resource. Dang et al. (1998) have supplemented the taxonomy with intersective classes: special classes for verbs which share membership of more than one Levin class because of regular polysemy. Bonnie Dorr (University of Maryland) has provided a reformulated and extended version of Levin's classi cation in her LCS database (http://www.umiacs.umd.edu/a1 bonnie/verbs-English.lcs). This resource groups 4,432 verbs (11,000 senses) into 466 Levin-based and 26 novel classes. The latter are Levin classes re ned according to verbal telicity patterns (Olsen et al., 1997), while the former are additional classes for non-Levin verbs which do not fall into any of the Levin classes due to their distinctive syntactic behaviour (Dorr, 1997).</Paragraph>
    <Paragraph position="3"> As a result of this work, the taxonomy has gained considerably in depth, but not to the same extent in breadth. Verbs taking ADJP, ADVP, ADL, particle, predicative, control and sentential complements are still largely excluded, except where they show interesting behaviour with respect to NP and PP complementation. As many of these verbs are highly frequent in language, NLP applications utilizing lexical-semantic classes would benet greatly from a linguistic resource which provides adequate classi cation of their senses. When extending Levin's classi cation with new classes, we particularly focussed on these verbs.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Creating Novel Classes
</SectionTitle>
    <Paragraph position="0"> Levin's original taxonomy was created by  1. selecting a set of diathesis alternations from linguistic resources, 2. classifying a large number of verbs according to their participation in these alternations, 3. grouping the verbs into semantic classes based on  their participation in sets of alternations. We adopted a different, faster approach. This involved  1. composing a set of diathesis alternations for verbs not covered comprehensively by Levin, 2. selecting a set of candidate lexical-semantic classes for these verbs from linguistic resources, 3. examining whether (sub)sets of verbs in each candi null date class could be related to each other via alternations and thus warrant creation of a new class. In what follows, we will describe these steps in detail.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 Novel Diathesis Alternations
</SectionTitle>
      <Paragraph position="0"> When constructing novel diathesis alternations, we took as a starting point the subcategorization classi cation of Briscoe (2000). This fairly comprehensive classi cation incorporates 163 different subcategorization frames (SCFs), a superset of those listed in the ANLT (Boguraev et al., 1987) and COMLEX Syntax dictionaries (Grishman et al., 1994). The SCFs de ne mappings from surface arguments to predicate-argument structure for bounded dependency constructions, but abstract over speci c particles and prepositions, as these can be trivially instantiated when the a frame is associated with a speci c verb. As most diathesis alternations are only semi-predictable on a verb-by-verb basis, a distinct SCF is de ned for every such construction, and thus all alternations can be represented as mappings between such SCFs.</Paragraph>
      <Paragraph position="1"> We considered possible alternations between pairs of SCFs in this classi cation, focusing in particular on those SCFs not covered by Levin. The identi cation of alternations was done manually, using criteria similar to Levin's: the SCFs alternating should preserve the sense in question, or modify it systematically.</Paragraph>
      <Paragraph position="2"> 106 new alternations were discovered using this method and grouped into different, partly overlapping categories. Table 1 shows some example alternations and their corresponding categories. The alternating patterns are indicated using an arrow (a2 ). The SCFs are marked using number codes whose detailed description can be found in (Briscoe, 2000) (e.g. SCF 53. refers to the COM-LEX subcategorization class NP-TO-INF-OC).</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Candidate Lexical-Semantic Classes
</SectionTitle>
      <Paragraph position="0"> Starting off from set of candidate classes accelerated the work considerably as it enabled building on extant linguistic research. Although a number of studies are available on verb classes not covered by Levin, many of these assume a classi cation system completely different to that of Levin's, and/or incorporate sense distinctions too ne-grained for easy integrations with Levin's classi cation. We therefore restricted our scope to a few classi cations of a suitable style and granularity:  The LCS database includes 26 classes for verbs which could not be mapped into any of the Levin classes due to their distinctive syntactic behaviour. These classes were originally created by an automatic verb classi cation algorithm described in (Dorr, 1997). Although they appear semantically meaningful, their syntactic-semantic properties have not been systematically studied in terms of diathesis alternations, and therefore re-examination is warranted.</Paragraph>
      <Paragraph position="1">  Rudanko (1996, 2000) provides a semantically motivated classi cation for verbs taking various types of sentential complements (including predicative and control constructions). His relatively ne-grained classes, organized into sets of independent taxonomies, have been created in a manner similar to Levin's. We took 43 of Rundanko's verb classes for consideration.</Paragraph>
      <Paragraph position="2">  Sager (1981) presents a small classi cation consisting of 13 classes, which groups verbs (mostly) on the basis of their syntactic alternations. While semantic properties are largely ignored, many of the classes appear distinctive also in terms of semantics.</Paragraph>
      <Paragraph position="3">  At least 20 (broad) Levin classes involve verb senses which take sentential complements. Because full treatment of these senses requires considering sentential complementation, we re-evaluated these classes using our method.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.3 Method for Creating Classes
</SectionTitle>
      <Paragraph position="0"> Each candidate class was evaluated as follows:  1. We extracted from its class description (where one was available) and/or from the COMLEX Syntax dictionary (Grishman et al., 1994) all the SCFs taken by its member verbs.</Paragraph>
      <Paragraph position="1"> 2. We extracted from Levin's taxonomy and from our novel list of 106 alternations all the alternations where these SCFs were involved.</Paragraph>
      <Paragraph position="2"> 3. Where one or several alternations where found  which captured the sense in question, and where the minimum of two member verbs were identi ed, a new verb class was created.</Paragraph>
      <Paragraph position="3"> Steps 1-2 were done automatically and step 3 manually. Identifying relevant alternations helped to identify additional SCFs, which in turn often led to the discovery of additional alternations. The SCFs and alternations discovered in this way were used to create the syntactic-semantic description of each novel class. For those candidate classes which had an insuf cient number of member verbs, new members were searched for in WordNet (Miller, 1990). Although WordNet classi es verbs on a purely semantic basis, the syntactic regularities studied by Levin are to some extent re ected Category Example Alternations Alternating SCFs Equi I advised Mary to go a3 I advised Mary 53 a3 24 He helped her bake the cake a3 He helped bake the cake 33 a3 142 Raising Julie strikes me as foolish a3 Julie strikes me as a fool 143 a3 29 He appeared to her to be ill a3 It appeared to her that he was ill 99 a3 12 Category He failed in attempting to climb a3 He failed in the climb 63 a3 87 switches I promised Mary to go a3 I promised Mary that I will go 54 a3 52 PP deletion Phil explained to him how to do it a3 Phil explained how to do it 90 a3 17 He contracted with him for the man to go a3 He contracted for the man to go 88 a3 15 P/C deletion I prefer for her to do it a3 I prefer her to do it 15 a3 53 They asked about what to do a3 They asked what to do 73 a3 116  by semantic relatedness as it is represented by Word-Net's particular structure (e.g. (Fellbaum, 1999)). New member verbs were frequently found among the synonyms, troponyms, hypernyms, coordinate terms and/or antonyms of the extant member verbs.</Paragraph>
      <Paragraph position="4"> For example, using this method, we gave the following description to one of the candidate classes of Rudanko (1996), which he describes syntactically with the single SCF 63 (see the below list) and semantically by stating that verbs in this class (e.g. succeed, manage, fail) have approximate meaning1 perform the act of or carry out the activity of : 20. SUCCEED VERBS SCF 22: John succeeded SCF 87: John succeeded in the climb SCF 63: John succeeded in attempting the climb SCF 112: John succeeded to climb Alternating SCFs: 22 a3 87, 87 a3 63, 22 a3 112 Some of the candidate classes, particularly those of Rudanko, proved too ne-grained to be helpful for a Levin type of classi cation, and were either combined with other classes or excluded from consideration. Some other classes, particularly the large ones in the LCS database, proved too coarse-grained after our method was applied, and were split down to subclasses.</Paragraph>
      <Paragraph position="5"> For example, the LCS class of Coerce Verbs (002) was divided into four subclasses according to the particular syntactic-semantic properties of the subsets of its member verbs. One of these subclasses was created for verbs such as force, induce, and seduce, which share the ap1Rudanko does not assign unique labels to his classes, and the descriptions he gives - when taken out of the context - cannot be used to uniquely identify the meaning involved in a speci c class. For details of this class, see his description in (Rudanko, 1996) page 28.</Paragraph>
      <Paragraph position="6"> proximate meaning of urge or force (a person) to an action . The sense gives rise to object equi SCFs and alternations: null</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2. FORCE VERBS
</SectionTitle>
    <Paragraph position="0"> New subclasses were also created for those Levin classes which did not adequately account for the variation among their member verbs. For example, a new class was created for those 37. Verbs of Communication which have an approximate meaning of make a proposal (e.g.</Paragraph>
    <Paragraph position="1"> suggest, recommend, propose). These verbs take a rather distinct set of SCFs and alternations, which differ from those taken by other communication verbs. This class is somewhat similar in meaning to Levin's 37.9 Advise Verbs. In fact, a subset of the verbs in 37.9 (e.g. advise, instruct) participate in alternations prototypical to this class (e.g. 104 a3 106) but not, for example, in the  ones involving PPs (e.g. 103 a3 116).</Paragraph>
    <Paragraph position="2"> 47. SUGGEST VERBS SCF 16: John suggested how she could do it SCF 17: John suggested how to do it SCF 24: John suggested it SCF 49: John suggested it to her SCF 89: John suggested to her how she could do it SCF 90: John suggested to her how to do it SCF 97: John suggested to her that she would do it SCF 98: John suggested to her that she do it SCF 101: John suggested to her what she could do SCF 103: John suggested to her what to do SCF 104: John suggested that she could do it SCF 106: John suggested that she do it SCF 114: John suggested what she could do SCF 116: John suggested what to do  Our work resulted in accepting, rejecting, combining and re ning the 102 candidate classes and - as a by-product - identifying 5 new classes not included in any of the resources we used. In the end, 57 new verb classes were formed, each associated with 2-45 member verbs. Those Levin or Dorr classes which were examined but found distinctive enough as they stand are not included in this count. However, their possible subclasses are, as well as any of the classes adapted from the resources of Rudanko or Sager. The new classes are listed in table 2, along with example verbs.</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Evaluation
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Task-Based Evaluation
</SectionTitle>
      <Paragraph position="0"> We performed an experiment in the context of automatic SCF acquisition to investigate whether the new classes can be used to support an important NLP task. The task is to associate classes to speci c verbs along with an estimate of the conditional probability of a SCF given a speci c verb. The resulting valency or subcategorization lexicon can be used by a (statistical) parser to recover predicate-argument structure.</Paragraph>
      <Paragraph position="1"> Our test data consisted of a total of 35 verbs from 12 new verb classes. The classes were chosen at random, subject to the constraint that their member verbs were frequent enough in corpus data. A minimum of 300 corpus occurrences per verb is required to yield a reliable SCF distribution for a polysemic verb with multiple SCFs (Korhonen, 2002). We took a sample of 20 million words of the British National Corpus (BNC) (Leech, 1992) and extracted all sentences containing an occurrence of one of the test verbs. After the extraction process, we retained  1. URGE ask, persuade 2. FORCE manipulate, pressure 3. ORDER command, require 4. WANT need, want 5. TRY attempt, try 6. WISH hope, expect 7. ENFORCE impose, risk 8. ALLOW allow, permit 9. ADMIT include, welcome 10. CONSUME spend, waste 11. PAY pay, spend 12. FORBID prohibit, ban 13. REFRAIN abstain, refrain 14. RELY bet, count 15. CONVERT convert, switch 16. SHIFT resort, return 17. ALLOW allow, permit 18. HELP aid, assist 19. COOPERATE collaborate, work 20. SUCCEED fail, manage 21. NEGLECT omit, fail 22. LIMIT restrict, restrain 23. APPROVE accept, object 24. ENQUIRE ask, consult 25. CONFESS acknowledge, reveal 26. INDICATE demonstrate, imply 27. DEDICATE devote, commit 28. FREE cure, relieve 29. SUSPECT accuse, condemn 30. WITHDRAW retreat, retire 31. COPE handle, deal 32. DISCOVER hear, learn 33. MIX pair, mix 34. CORRELATE coincide, alternate 35. CONSIDER imagine, remember 36. SEE notice, feel 37. LOVE like, hate 38. FOCUS focus, concentrate 39. CARE mind, worry 40. DISCUSS debate, argue 41. BATTLE ght, communicate 42. SETTLE agree, contract 43. SHOW demonstrate, quote 44. ALLOW allow, permit 45. EXPLAIN write, read 46. LECTURE comment, remark 47. SUGGEST propose, recommend 48. OCCUR happen, occur 49. MATTER count, weight 50. AVOID miss, boycott 51. HESITATE loiter, hesitate 52. BEGIN continue, resume 53. STOP terminate, nish 54. NEGLECT overlook, neglect 55. CHARGE commit, charge 56. REACH arrive, hit 57. ADOPT assume, adopt  1000 citations, on average, for each verb. Our method for SCF acquisition (Korhonen, 2002) involves rst using the system of Briscoe and Carroll (1997) to acquire a putative SCF distribution for each test verb from corpus data. This system employs a robust statistical parser (Briscoe and Carroll, 2002) which yields complete though shallow parses from the PoS tagged data. The parse contexts around verbs are passed to a comprehensive SCF classi er, which selects one of the 163 SCFs. The SCF distribution is then smoothed with the back-off distribution corresponding to the semantic class of the predominant sense of a verb. Although many of the test verbs are polysemic, we relied on the knowledge that the majority of English verbs have a single predominating sense in balanced corpus data (Korhonen and Preiss, 2003).</Paragraph>
      <Paragraph position="2"> The back-off estimates were obtained by the following method: (i) A few individual verbs were chosen from a new verb class whose predominant sense according to the WordNet frequency data belongs to this class, (ii) SCF distributions were built for these verbs by manually analysing c. 300 occurrences of each verb in the BNC, (iii) the resulting SCF distributions were merged. An empirically-determined threshold was nally set on the probability estimates from smoothing to reject noisy SCFs caused by errors during the statistical parsing phase. This method for SCF acquisition is highly sensitive to the accuracy of the lexical-semantic classes. Where a class adequately predicts the syntactic behaviour of the predominant sense of a test verb, signi cant improvement is seen in SCF acquisition, as accurate back-off estimates help to correct the acquired SCF distribution and deal with sparse data. Incorrect class assignments or choice of classes can, however, degrade performance. The SCFs were evaluated against manually analysed corpus data. This was obtained by annotating a maximum of 300 occurrences for each test verb in the BNC data. We calculated type precision (the percentage of SCF types that the system proposes which are correct), type recall (the percentage of SCF types in the gold standard that the system proposes) and Fa5 -measure2. To investigate how well the novel classes help to deal with sparse data, we recorded the total number of SCFs missing in the distributions, i.e. false negatives which did not even occur in the unthresholded distributions and were, therefore, never hypothesized by the parser and classi er. We also compared the similarity between the acquired unthresholded  and gold standard SCF distributions using several measures of distributional similarity: the Spearman rank correlation (RC), Kullback-Leibler distance (KL), Jensen-Shannon divergence (JS), cross entropy (CE), and intersection (IS)3.</Paragraph>
      <Paragraph position="3"> Table 3 shows average results for the 35 verbs with the the baseline system and for the system which employs the novel classes. We see that the performance improves when the novel classes are employed, according to all measures used. The method yields 8% absolute improvement in Fa5 -measure over the baseline method. The measures of distributional similarity show likewise improved performance. For example, the results with IS indicate that there is a large intersection between the acquired and gold standard SCFs when the method is used, and those with RC demonstrate that the method clearly improves the ranking of SCFs according to the conditional probability distributions of SCFs given each test verb. From the total of 193 gold standard SCFs unseen in the unsmoothed lexicon, only 115 are unseen after using the new classication. This demonstrates the usefulness of the novel classes in helping the system to deal with sparse data. While these results demonstrate clearly that the new classes can be used to support a critical NLP task, the improvement over the baseline is not as impressive as that reported in (Korhonen, 2002) where Levin's original classes are employed4. While it is possible that the new classes require further adjustment until optimal accuracy can be obtained, it is clear that many of our test verbs (and verbs in our new classes in general) are more polysemic on average and thus more 'dif cult' than those employed by Korhonen (2002). Our subcategorization acquisition method, based on predominant sense heuristics, is less adequate for these verbs rather, a method based on word sense disambiguation and the use of multi- null ple classes should be employed to establish the true upper bound on performance. Korhonen and Preiss (2003) have proposed such a method, but the method is not currently applicable to our test data.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Evaluation of Coverage
</SectionTitle>
      <Paragraph position="0"> Investigating the coverage of the current extended classication over the English verb lexicon is not straightforward because no fully suitable gold standard is available.</Paragraph>
      <Paragraph position="1"> We conducted a restricted evaluation against the comprehensive semantic classi cation of WordNet. As WordNet incorporates particularly ne-grained sense distinctions, some of its senses are too idiomatic or marginal for classi cation at this level of granularity. We aimed to identify and disregard these senses from our investigation.</Paragraph>
      <Paragraph position="2"> All the WordNet senses of 110 randomly chosen verbs were manually linked to classes in our extended classi cation (i.e. to Levin's, Dorr's or our new ones). From the total of 253 senses exempli ed in the data, 238 proved suitable (of right granularity) for our evaluation. From these, 21 were left unclassi ed because no class was found for them in the extended resource. After we evaluated these senses using the method described in section 3, only 7 of them turned out to warrant classes of their own which should be added to the extended classi cation.</Paragraph>
    </Section>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Discussion
</SectionTitle>
    <Paragraph position="0"> The evaluation reported in the previous section shows that the novel classes can used to support a NLP task and that the extended classi cation has good coverage over the English verb lexicon and thus constitutes a resource suitable for large-scale NLP use.</Paragraph>
    <Paragraph position="1"> Although the classes resulting from our work can be readily employed for NLP purposes, we plan, in the future, to further integrate them into Levin's taxonomy to yield a maximally useful resource for the research community. While some classes can simply be added to her taxonomy as new classes or subclasses of extant classes (e.g. our 47. SUGGEST VERBS can be added as a subclass to Levin's 37. Verbs of Communication), others will require modifying extant Levin classes. The latter classes are mostly those whose members classify more naturally in terms of their sentential rather than NP and PP complementation (e.g. ones related to Levin's 29. Verbs with Predicative Complements).</Paragraph>
    <Paragraph position="2"> This work will require resolving some con icts between our classi cation and Levin's. Because lexical-semantic classes are based on partial semantic descriptions manifested in alternations, it is clear that different, equally viable classi cation schemes can be constructed using the same data and methodology. One can grasp this easily by looking at intersective Levin classes (Dang et al., 1998), created by grouping together subsets of existing classes with overlapping members. Given that there is strong potential for cross-classi cation, we will aim to resolve any con icts by preferring those classes which show the best balance between the accuracy in capturing syntactic-semantic features and the ability to generalize to as many lexical items as possible.</Paragraph>
    <Paragraph position="3"> An issue which we did not address in the present work (as we worked on candidate classes), is the granularity of the classi cation. It is clear that the 'suitable' level of granularity varies from one NLP task to another. For example, tasks which require maximal accuracy from the classi cation are likely to bene t the most from ne-grained classes (e.g. re ned versions of Levin's classes (Green et al., 2001)), while tasks which rely more heavily on the capability of a classi cation to capture adequate generalizations over a set of lexical items bene t the most from broad classes. Therefore, to provide a general purpose classi cation suitable for various NLP use, we intend to re ne and organize our novel classes into taxonomies which incorporate different degrees of granularity.</Paragraph>
    <Paragraph position="4"> Finally, we plan to supplement the extended classi cation with additional novel information. In the absence of linguistic resources exemplifying further candidate classes we will search for additional novel classes, intersective classes and member verbs using automatic methods, such as clustering (e.g. (Brew and Schulte im Walde, 2002; Korhonen et al., 2003)). For example, clustering sense disambiguated subcategorization data (acquired e.g. from the SemCor corpus) should yield suitable (sense speci c) data to work with. We will also include in the classi cation statistical information concerning the relative likelihood of different classes, SCFs and alternations for verbs in corpus data, using e.g. the automatic methods proposed by McCarthy (2001) and Korhonen (2002).</Paragraph>
    <Paragraph position="5"> Such information can be highly useful for statistical NLP systems utilizing lexical-semantic classes.</Paragraph>
  </Section>
class="xml-element"></Paper>