<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2611">
  <Title>Towards Free-text Semantic Parsing: A Unified Framework Based on FrameNet, VerbNet and PropBank</Title>
  <Section position="3" start_page="78" end_page="79" type="metho">
    <SectionTitle>
2 Automatic semantic role detection on
FrameNet
</SectionTitle>
    <Paragraph position="0"> One of the goals of the FrameNet project is to design a linguistic ontology that can be used for automatic processing of semantic information.</Paragraph>
    <Paragraph position="1"> This hierarchy contains an extensive semantic analysis of verbs, nouns, adjectives and situations in which they are used, called frames. The basic assumption on which the frames are built is that each word evokes a particular situation with specific participants (Fillmore, 1968). The situations can be fairly simple depicting the entities involved and the roles they play or can be very complex and in this case they are called scenarios. The word that evokes a particular frame is called target word or predicate and can be an 1 A verb sense is an Intersective Levin class in which the verb is listed.</Paragraph>
    <Paragraph position="2"> adjective, noun or verb. The participant entities are defined using semantic roles and they are called frame elements.</Paragraph>
    <Paragraph position="3"> Several models have been developed for the automatic detection of the frame elements based on the FrameNet corpus (Gildea and Jurafsky, 2002; Thompson et al., 2003; Litkowski, 2004).</Paragraph>
    <Paragraph position="4"> While the algorithms used vary, almost all the previous studies divide the task into 1) the identification of the verb arguments to be labeled and 2) the tagging of each argument with a role.</Paragraph>
    <Paragraph position="5"> Also, most of the models agree on the core features as being: Predicate, Headword, Phrase Type, Governing Category, Position, Voice and Path. These are the initial features adopted by Gildea and Jurafsky (2002) (henceforth G&amp;J) for both frame element identification and role classification. null A difference among the previous machine-learning models is whether the frame information was used as gold feature. Of particular interest for us is the impact of the frame over unseen predicates and unseen words in general. The results obtained by G&amp;J are relevant in this sense; especially, the experiment that uses the frame to generalize from predicates seen in the training data to other predicates (i.e. when no data is available for a target word, G&amp;J use data from the corresponding frame). The overall performance induced by the frame usage increased.</Paragraph>
    <Paragraph position="6"> Other studies suggest that the frame is crucial when trying to eliminate the major sources of errors. In their error analysis, (Thompson et al., 2003) pinpoints that the verb arguments with headwords that are &amp;quot;rare&amp;quot; in a particular frame but not rare over the whole corpus are especially hard to classify. For these cases the frame is very important because it provides the context information needed to distinguish between different word senses.</Paragraph>
    <Paragraph position="7"> Overall, the experiments presented in G&amp;J's study correlated with the results obtained in the Senseval-3 competition show that the frame feature increases the performance and decreases the amount of annotated examples needed in training (i.e. frame usage improves the generalization ability of the learning algorithm). On the other hand the results obtained without the frame information are very poor.</Paragraph>
    <Paragraph position="8"> This behavior suggests that predicates in the same frame behave similarly in terms of their argument structure and that they differ with respect to other frames. From this perspective, having a broader verb knowledge base becomes of major importance for free-text semantic parsing.</Paragraph>
    <Paragraph position="9">  Unfortunately, the 321 frames that contain at least one verb predicate cover only a small fraction of the English verb lexicon and of possible domains. Also from these 321 frames only 100 were considered to have enough training data and were used in Senseval-3 (see Litkowski, 2004 for more details).</Paragraph>
    <Paragraph position="10"> Our approach for solving such problems involves the usage of a frame-like feature, namely the Intersective Levin class. We show that the Levin class is similar in many aspects to the frame and can replace it with almost no loss in performance. At the same time, Levin class provides better coverage as it can be learned also from other corpora (i.e. PropBank). We annotate FrameNet with Intersective Levin classes by using a mapping algorithm that exploits current theories of linking. Our extensive experimentation shows the validity of our technique and its effectiveness on corpora different from Frame-Net. The next section provides the theoretical support for the unified usage of FrameNet, VerbNet and PropBank, explaining why and how is possible to link them.</Paragraph>
  </Section>
  <Section position="4" start_page="79" end_page="80" type="metho">
    <SectionTitle>
3 Linking FrameNet to VerbNet and
PropBank
</SectionTitle>
    <Paragraph position="0"> In general, predicates belonging to the same FrameNet frame have a coherent syntactic behavior that is also different from predicates pertaining to other frames (G&amp;J). This finding is consistent with theories of linking that claim that the syntactic behavior of a verb can be predicted from its semantics (Levin 1993, Levin and Rappaport Hovav, 1996). This insight determined us to study the impact of using a feature based on Intersective Levin classes instead of the frame feature when classifying FrameNet semantic roles. The main advantage of using Levin classes comes from the fact that other resources like PropBank and the VerbNet lexicon contain this kind of information. Thus, we can train a Levin class classifier also on the PropBank corpus, considerably increasing the verb knowledge base at our disposal. Another advantage derives from the syntactic criteria that were applied in defining the Levin clusters. As shown later in this article, the syntactic nature of these classes makes them easier to classify than frames, when using only syntactic and lexical features.</Paragraph>
    <Paragraph position="1"> More precisely, the Levin clusters are formed according to diathesis alternation criteria which are variations in the way verbal arguments are grammatically expressed when a specific semantic phenomenon arises. For example, two different types of diathesis alternations are the following:  (a) Middle Alternation [Subject, Agent The butcher] cuts [Direct Object, Patient the meat]. [Subject, Patient The meat] cuts easily.</Paragraph>
    <Paragraph position="2"> (b) Causative/inchoative Alternation [Subject, Agent Janet] broke [Direct Object, Patient the cup]. [Subject, Patient The cup] broke.</Paragraph>
    <Paragraph position="3">  In both cases, what is alternating is the grammatical function that the Patient role takes when changing from the transitive use of the verb to the intransitive one. The semantic phenomenon accompanying these types of alternations is the change of focus from the entity performing the action to the theme of the event. Levin documented 79 alternations which constitute the building blocks for the verb classes. Although alternations are chosen as the primary means for identifying the classes, additional properties related to subcategorization, morphology and extended meanings of verbs are taken into account as well. Thus, from a syntactic point of view, the verbs in one Levin class have a regular behavior, different from the verbs pertaining to other classes. Also, the classes are semantically coherent and all verbs belonging to one class share the same participant roles.</Paragraph>
    <Paragraph position="4"> This constraint of having the same semantic roles is further ensured inside the VerbNet lexicon that is constructed based on a more refined version of the Levin classification called Intersective Levin classes (Dang et al., 1998). The lexicon provides a regular association between the syntactic and semantic properties of each of the described classes. It also provides information about the syntactic frames (alternations) in which the verbs participate and the set of possible semantic roles.</Paragraph>
    <Paragraph position="5"> One corpus associated with the VerbNet lexicon is PropBank. The annotation scheme of PropBank ensures that the verbs belonging to the same Levin class share similarly labeled arguments. Inside one Intersective Levin class, to one argument corresponds one semantic role numbered sequentially from Arg0 to Arg5. Higher numbered argument labels are less consistent and assigned per-verb basis.</Paragraph>
    <Paragraph position="6"> The Levin classes were constructed based on regularities exhibited at grammatical level and the resulting clusters were shown to be semantically coherent. As opposed, the FrameNet frames were build on semantic bases, by putting together verbs, nouns and adjectives that evoke the same situations. Although different in conception, the  FrameNet verb clusters and VerbNet verb clusters have common properties2: (1) Coherent syntactic behavior of verbs inside one cluster, (2) Different syntactic properties between any two distinct verb clusters, (3) Shared set of possible semantic roles for all verbs  pertaining to the same cluster.</Paragraph>
    <Paragraph position="7"> Having these insights, we have assigned a correspondent VerbNet class not to each verb predicate but rather to each frame. In doing this we have applied the simplifying assumption that a frame has a unique corresponding Levin class.</Paragraph>
    <Paragraph position="8"> Thus, we have created a one-to-many mapping between the Intersective Levin classes and the frames. In order to create a pair [?]FrameNet frame, VerbNet class[?], our mapping algorithm checks both the syntactic and semantic consistency by comparing the role frequency distributions on different syntactic positions for the two candidates. The algorithm is described in detail in the next section.</Paragraph>
  </Section>
  <Section position="5" start_page="80" end_page="82" type="metho">
    <SectionTitle>
4 Mapping FrameNet frames to
</SectionTitle>
    <Paragraph position="0"> VerbNet classes The mapping algorithm consists of three steps: (a) we link the frames and Intersective Levin verb classes that have the largest number of verbs in common and we create a set of pairs [?]FrameNet frame, VerbNet class[?] (see Figure 1); (b) we refine the pairs obtained in the previous step based on diathesis alternation criteria, i.e. the verbs pertaining to the FrameNet frame have to undergo the same diathesis alternation that characterize the corresponding VerbNet class (see Figure 2) and (c) we manually check and correct the resulting mapping. In the next sections we will explain in more detail each step of the mapping algorithm.</Paragraph>
    <Section position="1" start_page="80" end_page="80" type="sub_section">
      <SectionTitle>
4.1 Linking frames and Intersective Levin
</SectionTitle>
      <Paragraph position="0"> classes based on common verbs During the first phase of the algorithm, given a frame, we compute its intersection with each VerbNet class. We choose as candidate for the mapping the Intersective Levin class that has the largest number of verbs in common with the given frame (Figure 1, line (I)). If the size of the intersection between the FrameNet frame and the candidate VerbNet class is bigger than or equal 2 For FrameNet, properties 1 and 2 are true for most of the frames but not for all. See section 4.4 for more details.</Paragraph>
      <Paragraph position="1"> to 3 elements then we form a pair [?]FrameNet frame, VerbNet class[?] that qualifies for the second step of the algorithm.</Paragraph>
      <Paragraph position="2"> Only the frames that have more than three verb lexical units are candidates for this step (frames with less than 3 members cannot pass condition (II)). This excludes a number of 60 frames that will subsequently be mapped</Paragraph>
    </Section>
    <Section position="2" start_page="80" end_page="80" type="sub_section">
      <SectionTitle>
4.2 Refining the mapping based on verb
alternations
</SectionTitle>
      <Paragraph position="0"> In order to assign a VerbNet class to a frame, we have to check that the verbs belonging to that frame respect the diathesis alternation criteria used to define the VerbNet class. Thus, the pairs [?]FrameNet frame, VerbNet class[?] formed in step (I) of the mapping algorithm have to undergo a validation step that verifies the similarity between the enclosed FrameNet frame and VerbNet class. This validation process has several substeps. null First, we make use of the property (3) of the Levin classes and FrameNet frames presented in the previous section. According to this property, all verbs pertaining to one frame or Levin class have the same participant roles. Thus, a first test of compatibility between a frame and a Levin class is that they share the same participant roles. As FrameNet is annotated with frame-specific semantic roles we manually mapped these roles into the VerbNet set of thematic roles. Given a frame, we assigned thematic roles to all frame elements that are associated with verbal predicates. For example the roles Speaker, Addressee,  Second, we build a frequency distribution of VerbNet thematic roles on different syntactic position. Based on our observation and previous studies (Merlo and Stevenson, 2001), we assume that each Levin class has a distinct frequency distribution of roles on different grammatical slots. As we do not have matching grammatical function in FrameNet and VerbNet, we approximate that subjects and direct objects are more likely to appear on positions adjacent to the predicate, while indirect objects appear on more distant positions. The same intuition is used successfully by G&amp;J in the design of the Position feature.</Paragraph>
      <Paragraph position="1"> We will acquire from the corpus, for each thematic role thi, the frequencies with which it appears on an adjacent (ADJ) or distant (DST) position in a given frame or VerbNet class (i.e.</Paragraph>
      <Paragraph position="2"> #(thi, class, position)). Therefore, for each frame and class, we obtain two vectors with thematic role frequencies corresponding respectively to the adjacent and distant positions (see Figure 2). We compute a score for each pair [?]FrameNet frame, VerbNet class[?] using the normalized scalar product. We give a bigger weight to the adjacent dot product multiplying its score by 2/3 with respect to the distant dot product that is multiplied by 1/3. We do this to minimize the impact that adjunct roles like Temporal and Location (that appear mostly on the distant positions) could have on the final outcome.</Paragraph>
      <Paragraph position="3">  The above frequency vectors are computed for FrameNet directly from the corpus of predicate-argument structure examples associated with each frame. The examples associated with the VerbNet lexicon are extracted from the PropBank corpus. In order to do this we apply a preprocessing step in which each label ARG0..N is replaced with its corresponding thematic role given the Intersective Levin class of the predicate. We assign the same roles to the adjuncts all over PropBank as they are general for all verb classes. The only exception is ARGM-DIR that can correspond to Source, Goal or Path. We assign different roles to this adjunct based on the prepositions. We ignore some adjuncts like ARGM-ADV or ARGM-DIS because they cannot bear a thematic role.</Paragraph>
    </Section>
    <Section position="3" start_page="80" end_page="80" type="sub_section">
      <SectionTitle>
4.3 Mapping Results
</SectionTitle>
      <Paragraph position="0"> We found that only 133 VerbNet classes have correspondents among FrameNet frames. Also, from the frames mapped with an automatic score smaller than 0.5 points almost a half did not match any of the existing VerbNet classes3. A summary of the results is depicted in Table 1.</Paragraph>
      <Paragraph position="1"> The first column contains the automatic score provided by the mapping algorithm when comparing frames with Intersective Levin classes.</Paragraph>
      <Paragraph position="2"> The second column contains the number of frames for each score interval. The third column contains the percentage of frames, per each score interval, that did not have a corresponding VerbNet class and finally the forth column contains the accuracy of the mapping algorithm.</Paragraph>
    </Section>
    <Section position="4" start_page="80" end_page="82" type="sub_section">
      <SectionTitle>
4.4 Discussion
</SectionTitle>
      <Paragraph position="0"> In the literature, other studies compared the Levin classes to the FrameNet frames (Baker and Ruppenhofer, 2002). Their findings suggest that although the two set of clusters are roughly equivalent there are also several types of mistmaches: 1) Levin classes that are narrower than the corresponding frames, 2) Levin classes that are broader that the corresponding frames and 3) overlapping groupings. For our task, point 2 does not pose a problem. Points 1 and 3 however suggest that there are cases in which to one FrameNet frame corresponds more than one Levin class. By investigating such cases we noted that the mapping algorithm consistently assigns scores below 75% to cases that match problem 1 (two Levin classes inside one frame) and below 50% to cases that match problem 3 (more than two Levin classes inside one frame).</Paragraph>
      <Paragraph position="1"> Thus, in order to increase the accuracy of our results a first step should be to assign an 3 The automatic mapping can be improved by manually assigning the FrameNet frames of the pairs that receive a score lower than 0.5.</Paragraph>
      <Paragraph position="2">  Intersective Levin class to each of the verbs pertaining to frames with score lower than 0.75. Nevertheless the current results are encouraging as they show that the algorithm is achiving its purpose by successfully detecting syntactic incoherencies that can be subsequently corrected manually. Also, in the next section we will show that our current mapping achieves very good results, giving evidence for the effectivenes of the Levin class feature.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>