<?xml version="1.0" standalone="yes"?>
<Paper uid="N03-2022">
  <Title>Semantic Extraction with Wide-Coverage Lexical Resources</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3. FrameNet
</SectionTitle>
    <Paragraph position="0"> FrameNet (Baker et. al. 1998) is building a lexicon based on the theory of Frame Semantics.</Paragraph>
    <Paragraph position="1"> Frame Semantics suggests that the meanings of lexical items (lexical units (LU)) are best defined with respect to larger conceptual chunks, called Frames. Individual lexical units evoke specific frames and establish a binding pattern to specific slots or roles (frame elements (FE)) within the frame. The Berkeley FrameNet project describes the underlying frames for different lexical units, examines sentences related to the frames using a very large corpus, and records (annotates) the ways in which information from the associated frames are expressed in these sentences. The result is a database that contains a set of frames (related through hierarchy and composition), a set of frame elements for each frame, and a set of frame annotated sentences that covers the different patterns of usage for lexical units in the frame.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 FrameNet data as seed patterns for IE:
</SectionTitle>
      <Paragraph position="0"> Using the FrameNet annotated dataset, we compiled a set of IE patterns and also the lexicon for each of the lexical units in FrameNet.</Paragraph>
      <Paragraph position="1"> We filtered out all of the non-relevant terms in all frame element lexicons. We hypothesized that using a highly precise set of patterns along with precise lexicon should enable a promising IE performance. For our Information Extraction experiments, we used GATE (Cunningham et. al.</Paragraph>
      <Paragraph position="2"> 2002), an open source natural language engineering system. The component-based architecture of GATE enabled us to plug-in our FrameNet based lexicon and pattern set and run IE experiments on this system.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Initial Experiment:
</SectionTitle>
      <Paragraph position="0"> As a preliminary test, we compiled a set of 100 news stories from Yahoo News Service with topics related to Criminal Investigation. We also compiled a set of IE patterns and also the lexicon from the crime related frames (&amp;quot;Arrest&amp;quot;, &amp;quot;Detain&amp;quot;, &amp;quot;Arraign&amp;quot; and &amp;quot;Verdict&amp;quot;.) We ran the GATE system on this corpus with our FrameNet data. We evaluated the IE performance by human judgment and hand counting the semantic role assignments. The systems achieved an average of 55% Recall while the precision was 68.8%. The fairly high precision (given just the FrameNet annotations) is the result of a highly precise lexicon and pattern set, while we see the low recall as the result of the small coverage. That is the reason that employed WordNet to enlarge our lexicon.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4. Expanding the Lexicon
</SectionTitle>
    <Paragraph position="0"> In order to expand our lexicon for each of the frame elements, we used the human-built knowledge base (WordNet (Fellbaum 1998)) and its rich hierarchical structure.</Paragraph>
    <Paragraph position="1"> We built a graphical model of WordNet making some assumptions about the structure of the induced WordNet graph. For our initial experiments, we built a graph whose leaf was the enclosing category of the FrameNet annotated frame element. We then looked at an ancestor tree following the WordNet hypernym relation. This gave us a graphical model of the form shown in Figure 1 for the FrameNet frame element Suspect and WordNet category Thief.</Paragraph>
    <Paragraph position="2"> Figure 1 We then used the sum-product algorithm (Frey 1998) for statistical inference on the frame element induced graph (such as in Figure 1). We now illustrate our use of the algorithm to expand the FrameNet derived lexicon.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Statistical Inference
</SectionTitle>
      <Paragraph position="0"> We employed a statistical inference algorithm to find the relevant nodes of WordNet. For each of the frame elements, we took the terms in FrameNet FE annotations as ground truth which means that the relevance probability of the WordNet nodes for those terms is equal to 1. The Sum Product algorithm helps us find the relevance probability of higher level nodes as a lexical category for the frame element through a bottom up computation of the inter-node messages. For example the message between nodes 1 and 0 in the Figure 1 can be computed as:  We should note that based on the WordNet's hypernym relation, the conditional relevance probability of each parent node (given any child node) is equal to 1. Therefore the Sum Product inter-node messages are computed as:  and the probability of each WordNet node can be computed by a normalized interpolation of all of the incoming messages from the children nodes:</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Relevance of a WordNet Nodes
</SectionTitle>
      <Paragraph position="0"> Throughout our experiments with the training data, we discovered that some infrequent tail terms in the frame element lexicon that might not be filtered out by the statistical inference algorithm but still are frequently used in relevant text.</Paragraph>
      <Paragraph position="1"> Therefore, we defined the relevance metric for the WordNet nodes to achieve a larger coverage. We compiled a large corpus of text (News stories) and made a second smaller corpus from the original one which contains only sentences which are relevant to the IE task. For each of the WordNet nodes we defined the relevance of the node based on the proportion of the occurrence of the node in IE related Text (Orel) to the occurrence of the node in the general text (Ogen).</Paragraph>
      <Paragraph position="2">  Using this relevance metric, we evaluated all of the WordNet nodes for the training data (found in the previous step) and re-ranked and picked the top 'm' relevant nodes (m=5 for our reported experiment) and added them to the previous set of WordNet nodes.</Paragraph>
      <Paragraph position="3"> With a set of relevant WordNet nodes, we extended the lexicon for the IE system and re-ran our IE task on the same 100 Yahoo news stories that were used in the initial experiments. The average recall rose up to 76.4% this time with an average precision equal to 66%.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5. Frame Assignment
</SectionTitle>
    <Paragraph position="0"> Using FrameNet data with IE techniques shows promising results for semantic extraction. Our current efforts are geared toward extending the extraction program to include automatic frame assignment. For this task, we assume that that the frame is a latent class variable (whose domain is the set of lexical units) and the frame elements are variables whose domain is the expanded lexicon (FrameNet + WordNet). We assume that the frame elements are conditionally independent from each other, given the frame. For our initial experiments, we assume that each frame is an independent model and frame assignment is the task of selecting the Maximum A Posteriori (MAP) frame given the input and the priors of the frame. Figure 2 shows the graphical model exemplifying this assertion. With this model, we are able to estimate the overall joint distribution for each FrameNet frame, given the lexical items in the candidate sentence from the corpus. During training frame priors and model parameters )|( framefep are estimated from a large corpus using our SE machinery outlined in sections 3 and 4. While our initial results seem promising, the work is ongoing and we should have more results to report on this aspect of the work by the time of the conference.</Paragraph>
    <Paragraph position="1"> Figure 2</Paragraph>
  </Section>
class="xml-element"></Paper>