<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-1037">
  <Title>I Relations I Lexico-syntactic Patterns Examples H WordNet Relations</Title>
  <Section position="2" start_page="0" end_page="268" type="metho">
    <SectionTitle>
1 Desiderata for Automated
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="268" type="sub_section">
      <SectionTitle>
Knowledge Acquisition
</SectionTitle>
      <Paragraph position="0"> The need for knowledge The knowledge is infinite and no matter how large a knowledge base is, it is not possible to store all the concepts and procedures for all domains. Even if that were possible, the knowledge is generative and there are no guarantees that a system will have the latest information all the time. And yet, if we are to build common-sense knowledge processing systems in the future, it is necessary to have general-purpose and domain-specific knowledge that is up to date.</Paragraph>
      <Paragraph position="1"> Our inability to build large knowledge bases without much effort has impeded many ANLP developments.</Paragraph>
      <Paragraph position="2"> The most successful current Information Extraction systems rely on hand coded linguistic rules representing lexico-syntactic patterns capable of matching natural language expressions of events. Since the rules are hand-coded it is difficult to port systems across domains. Question answering, inference, summarization, and other applications can benefit from large linguistic knowledge bases.</Paragraph>
      <Paragraph position="3"> The basic idea A possible solution to the problem of rapid development of flexible knowledge bases is to design an automatic knowledge acquisition system that extracts knowledge from texts for the purpose of merging it with a core ontological knowledge base. The attempt to create a knowledge base manually is time consuming and error prone, even for small application domains, and we believe that automatic knowledge acquisition and classification is the only viable solution to large-scale, knowledge intensive applications. This paper presents an interactive method that acquires new concepts and connections associated with user-selected seed concepts, and adds them to the WordNet linguistic knowledge structure (Fellbaum 1998). The sources of the new knowledge are texts acquired from the Internet or other corpora. At the present time, our system works in a semi-automatic mode, in the sense that it acquires concepts and relations automatically, but their validation is done by the user.</Paragraph>
      <Paragraph position="4"> We believe that domain knowledge should not be acquired in a vacuum; it should expand an existent ontology with a skeletal structure built on consistent and acceptable principles. The method presented in this paper is applicable to any Machine Readable Dictionary. However, we chose WordNet because it is freely available and widely used.</Paragraph>
      <Paragraph position="5"> Related work This work was inspired in part by Marti Hearst's paper (Hearst 1998) where she discovers manually lexico-syntactic patterns for the HYPERNYMY relation in WordNet.</Paragraph>
      <Paragraph position="6"> Much of the work in pattern extraction from texts was done for improving the performance of Information Extraction systems. Research in this area was done by (Kim and Moldovan 1995) (Riloff 1996), (Soderland 1997) and others.</Paragraph>
      <Paragraph position="7"> The MindNet (Richardson 1998) project at Microsoft is an attempt to transform the Longman Dictionary of Contemporary English (LDOCE) into a form of knowledge base for text processing.</Paragraph>
      <Paragraph position="8"> Woods studied knowledge representation and classification for long time (Woods 1991), and more recently is trying to automate the construction of taxonomies by extracting concepts directly from texts (Woods 1997).</Paragraph>
      <Paragraph position="9"> The Knowledge Acquisition from Text (KAT) system is presented next. It consists of four parts: (1) discovery of new concepts, (2) discovery of new lexical patterns, (3) discovery of new relationships reflected by the lexical patterns, and (4) the classification and integration of the knowledge discovered with a WordNet - like knowledge base.</Paragraph>
    </Section>
  </Section>
  <Section position="3" start_page="268" end_page="272" type="metho">
    <SectionTitle>
2 KAT System
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="268" end_page="268" type="sub_section">
      <SectionTitle>
2.1 Discover new concepts
</SectionTitle>
      <Paragraph position="0"> Select seed concepts. New domain knowledge can be acquired around some seed concepts that a user considers important. In this paper we focus on the financial domain, and use: interest rate, stock market, inflation, economic growth, and employment as seed concepts. The knowledge we seek to acquire relates to one or more of these concepts, and consists of new concepts not defined in WordNet and new relations that link these concepts with other concepts, some of which are in WordNet.</Paragraph>
      <Paragraph position="1"> For example, from the sentence: When the US economy enters a boom, mortgage interest rates rise, the system discovers: (1) the new concept mortgage interest rate not defined in WordNet but related to the seed concept interest rate, and (2) the state of the US economy and the value of mortgage interest rate are in a DIRECT RELATIONSHIP.</Paragraph>
      <Paragraph position="2"> In WordNet, a concept is represented as a synset that contains words sharing the same meaning. In our experiments, we extend the seed words to their corresponding synset. For example, stock market is synonym with stock exchange and securities market, and we aim to learn concepts related to all these terms, not only to stock market.</Paragraph>
      <Paragraph position="3"> Extract sentences. Queries are formed with each seed concept to extract documents from the Internet and other possible sources. The documents retrieved are further processed such that only the sentences that contain the seed concepts are retained. This way, an arbitrarily large corpus .4 is formed of sentences containing the seed concepts. We limit the size of this corpus to 1000 sentences per seed concept. null Parse sentences. Each sentence in this corpus is first part-of-speech (POS) tagged then parsed. We use Brill's POS tagger and our own parser. The output of the POS tagger for the example above is:</Paragraph>
      <Paragraph position="5"> Extract new concepts. In this paper only noun concepts are considered. Since, most likely, one-word nouns are already defined in WordNet, the focus here is on compound nouns and nouns with modifiers that have meaning but are not in WordNet.</Paragraph>
      <Paragraph position="6"> The new concepts directly related to the seeds are extracted from the noun phrases (NPs) that contain the seeds. In the example above, we see that the seed belongs to the NP: mortgage interest rate.</Paragraph>
      <Paragraph position="7"> This way, a list of NPs containing the seeds is assembled automatically from the parsed texts. Every such NP is considered a potential new concept.</Paragraph>
      <Paragraph position="8"> This is only the &amp;quot;raw material&amp;quot; from which actual concepts are discovered.</Paragraph>
      <Paragraph position="9"> In some noun phrases the seed is the head noun, i.e. \[word, word,..see~, where word can be a noun or an adjective. For example, \[interest rate\] is in Word-Net, but \[short term nominal interest rate\] is not in WordNet. Most of the new concepts related to a seed are generated this way. In other cases the seed is not the head noun i.e. \[word, word,..seed, word, wor~. For example \[interest rate peg\], or \[international interest rate differentia~.</Paragraph>
      <Paragraph position="10"> The following procedures are used to discover concepts, and are applicable in both cases: Procedure 1.1. WordNet reduction. Search NP for words collocations that are defined in WordNet as concepts. Thus \[long term interest rate\] becomes \[long_term interest_rate\], \[prime interest rate\] becomes \[prime_interest_rate\], as all hyphenated concepts are in WordNet.</Paragraph>
      <Paragraph position="11"> Procedure 1.2. Dictionary reduction. For each NP, search further in other on-line dictionaries for more compound concepts, and if found, hyphenate the words. Many domain-specific dictionaries are available on-line. For example, \[mortgage interest_rate\] becomes \[mortgage_interest_rate\], since it is defined in the on-line dictionary OneLook Dictionaries (http://www.onelook.com).</Paragraph>
      <Paragraph position="12"> Procedure 1.3. User validation. Since currently we lack a formal definition of a concept, it is not possible to completely automate the discovery of concepts.</Paragraph>
      <Paragraph position="13"> The human inspects the list of noun phrases and decides whether to accept or decline each concept.</Paragraph>
    </Section>
    <Section position="2" start_page="268" end_page="269" type="sub_section">
      <SectionTitle>
2.2 Discover lexlco-syntactic patterns
</SectionTitle>
      <Paragraph position="0"> Texts represent a rich source of information from which in addition to concepts we can also discover relations between concepts. We are interested in discovering semantic relationships that link the concepts extracted above with other concepts, some of which may be in WordNet. The approach is to search for lexico-syntactic patterns comprising the concepts of interest. The semantic relations from WordNet are the first we search for, as it is only natural to add more of these relations to enhance the WordNet knowledge base. However, since the focus is on the acquisition of domain-specific knowledge, there are semantic relations between concepts other than the WordNet relations that are important. These new relations can be discovered automatically from the clauses and sentences in which the seeds occur.</Paragraph>
      <Paragraph position="1">  Pick a semantic relation R. These can be Word-Net semantic relations or any other relations defined by the user. So far, we have experimented with the WordNet HYPERNYMY (or so-called IS-A) relation, and three other relations. By inspecting a few sentences containing interest rate one can notice that INFLUENCE is a frequently used relation. The two other relations are CAUSE and EQUIVALENT.</Paragraph>
      <Paragraph position="2"> Pick a pair of concepts Ci, C# among which R holds. These may be any noun concepts. In the context of finance domain, some examples of concepts linked by the INFLUENCE relation are: interest rate INFLUENCES earnings, or credit worthiness INFLUENCES interest rate.</Paragraph>
      <Paragraph position="3"> Extract lexico-syntactic patterns Ci :P Cj.</Paragraph>
      <Paragraph position="4"> Search any corpus B, different from ,4 for all instances where Ci and Cj occur in the same sentence. Extract the lexico-syntactic patterns that link the two concepts. For example~ from the sentence : The graph indicates the impact on earnings from several different interest rate scenarios, the generally applicable pattern extracted is: impact on NP2 from NP1 This pattern corresponds unambiguously to the relation R we started with, namely INFLUENCE. Thus we conclude: INFLUENCE(NPI, NP2).</Paragraph>
      <Paragraph position="5"> Another example is: As the credit worthiness decreases, the interest rate increases. From this sentence we extract another lexical pattern that expresses the INFLUENCE relation: \[as NP1 vbl, NP2 vb$\] &amp; \[vbl and vb2 are antonyms\] This pattern is rather complex since it contains not only the lexical part but also the verb condition that needs to be satisfied.</Paragraph>
      <Paragraph position="6"> This procedure repeats for all relations R.</Paragraph>
    </Section>
    <Section position="3" start_page="269" end_page="269" type="sub_section">
      <SectionTitle>
2.3 Discover new relationships between
concepts
</SectionTitle>
      <Paragraph position="0"> Let us denote with Cs the seed-related concepts found with Procedures 1.1 through 1.3. We search now corpus ,4 for the occurrence of patterns ~ discovered above such that one of their two concepts is a concept Cs.</Paragraph>
      <Paragraph position="1"> Search corpus ,4 for a pattern ~. Using a lexico-syntactic pattern P, one at a time, search corpus ,4 for its occurrence. If found, search further whether or not one of the NPs is a seed-related concept Cs.</Paragraph>
      <Paragraph position="2"> Identify new concepts Cn. Part of the pattern 7 ~ are two noun phrases, one of which is Cs. The head noun from the other noun phrase is a concept Cn we are looking for. This may be a WordNet concept, and if it is not it will be added to the list of concepts discovered.</Paragraph>
      <Paragraph position="3"> Form relation R(Cs, Cn). Since each pattern 7 ~ is a linguistic expression of its corresponding semantic relation R, we conclude R(Cs,Cn) (this is interpreted &amp;quot;C8 is relation R Cn)'). These steps are repeated for all patterns.</Paragraph>
      <Paragraph position="4"> User intervention to accept or reject relationships is necessary mainly due to our system inability of handling coreference resolution and other complex linguistic phenomena.</Paragraph>
    </Section>
    <Section position="4" start_page="269" end_page="272" type="sub_section">
      <SectionTitle>
2.4 Knowledge classification and
</SectionTitle>
      <Paragraph position="0"> integration Next, a taxonomy needs to be created that is consistent with WordNet. In addition to creating a taxonomy, this step is also useful for validating the concepts acquired above. The classification is based on the subsumption principle (Schmolze and Lipkis 1983), (Woods 1991).</Paragraph>
      <Paragraph position="1"> This algorithm provides the overall steps for the classification of concepts within the context of Word-Net. Figure 1 shows the inputs of the Classification Algorithm and suggests that the classification is an iterative process. In addition to WordNet, the inputs consist of the corpus ,4, the sets of concepts Cs  and Cn, and the relationships 7~. Let's denote with C = Cs U Cn the union of the seed related concepts with the new concepts. All these concepts need to be classified.</Paragraph>
      <Paragraph position="2"> Wo,aN=l Cdeg~Tr~ A Co.=i= ~. Cdeg, V.=~tio.~=a~\[ I R \[ I t Knowledge Classification C/--k Algorithm '1 ..... ~i;\] Figure 1: The knowledge classification diagram Step 1. From the set of relationships 7&amp;quot;~ discovered in Part 3, pick all the HYPERNYMY relations. From the way these relations were developed, there are two possibilities: (1) A HYPERNYMY relation links a WordNet concept</Paragraph>
      <Paragraph position="4"> A HYPERNYMY relation links a concept Cs with a concept Cn.</Paragraph>
      <Paragraph position="5"> Concepts C~w are immediately linked to Word-Net and added to the knowledge base. The concepts from case (2) are also added to the knowledge base but they form at this point only some isolated islands since are not yet linked to the rest of the knowledge base.</Paragraph>
      <Paragraph position="6"> Step 2. Search corpus `4 for all the patterns associated with the HYPERNYMY relation that may link  concepts in the set Cn with any WordNet concepts. Altough concepts C, are not seed-based concepts, they are related to at least one Cs concept via a relationship (as found in Task 3). Here we seek to find HYPERNYMY links between them and WordNet concepts. If such C,~ concepts exist, denote them with C~w. The union Chw = C~w LJ C2w represents all concepts from the set C that are linked to WordNet without any further effort. We focus now on the rest of concepts, Cc -- C N Chw, that are not yet linked to any WordNet concepts.</Paragraph>
      <Paragraph position="7"> Step 3. Classify all concepts in set Ce using Procedures 4.1 through 4.5 below.</Paragraph>
      <Paragraph position="8"> Step 4. Repeat Step 3 for all the concepts in set Cc several times till no more changes occur. This reclassification is necessary since the insertion of a concept into the knowledge base may perturb the ordering of other surrounding concepts in the hierarchy. null Step 5. Add the rest of relationships 7~ other than the HYPERNYMY to the new knowledge base.</Paragraph>
      <Paragraph position="9"> The HYPERNYMY relations have already been used in the Classification Algorithm, but the other relations, i.e. INFLUENCE, CAUSE and EQUIVALENT need to be added to the knowledge base.</Paragraph>
      <Paragraph position="10"> Concept classification procedures Procedure 4.1. Classify a concept of the form \[word, head\] with respect to concept \[head\].</Paragraph>
      <Paragraph position="11"> It is assumed here that the \[head\] concept exists in WordNet simply because in many instances the &amp;quot;head&amp;quot; is the &amp;quot;seed&amp;quot; concept, and because frequently the head is a single word common noun usually defined in WordNet. In this procedure we consider only those head nouns that do not have any hyponyms since the other case when the head has other concepts under it is more complex and is treated by Procedure 4.4. Here &amp;quot;word&amp;quot; is a noun or an adjective. null The classification is based on the simple idea that a compound concept \[word, head\] is ontologically subsumed by concept \[head\]. For example, mortgage_interest_rate is a kind of interest_rate, thus linked by a relation nYPERNYMY(interest_rate, mortgage_interest_rate).</Paragraph>
      <Paragraph position="12"> Procedure 4.2. Classify a concept \[wordx, headx\] with respect to another concept \[words, head2\].</Paragraph>
      <Paragraph position="13"> For a relative classification of two such concepts, the ontological relations between headz and head2 and between word1 and words, if exist, are extended to the two concepts. We distinguish here three possibilities: null 1. heady subsumes heads and word1 subsumes word2. In this case \[wordz, headl\] subsumes \[word2, heads\]. The subsumption may not always be a direct connection; sometimes it may consist of a chain of subsumption relations since subsumption is (usually) a transitive relation (Woods 1991). An example is shown in Figure 2a; in WordNet, Asian_country subsumes Japan and interest_rate subsumes discount_rate.</Paragraph>
      <Paragraph position="14"> A particular case of this is when head1 is identical with head2.</Paragraph>
      <Paragraph position="15"> 2. Another case is when there is no direct subsumption relation in WordNet between word1 and words, and/or head1 and heads, but there are a common subsuming concepts, for each pair. When such concepts are found, pick the most specific common subsumer (MSCS) concepts of word1 and words, and of head1 and head2, respectively. Then form a concept \[MSCS(wordz, words), MSCS(headl, head2)\] and place \[word1 headz\] and \[words heads\] under it. This is exemplified in Figure 2b. In WordNet, country Subsumes Japan and Germany, and interest_rate subsumes discount_rate  and prime_interest_rate.</Paragraph>
      <Paragraph position="16"> 3. In all other cases, no subsumption relation is es null tablished between the two concepts. For example, we cannot say whether Asian_country discount_rate is more or less abstract then Japan interest_rate.</Paragraph>
      <Paragraph position="17"> Procedure 4.3. Classify concept \[word1 words head\]. Several poss!bilities exist: 1. When there is already a concept \[words head\] in the knowledge base under the \[head\], then place \[wordl words head\] under concept \[words head\].</Paragraph>
      <Paragraph position="18"> 2. When there is already a concept \[wordz head\] in the knowledge base under the \[head\], then place \[wordl word2 head\] under concept \[wordl head\].</Paragraph>
      <Paragraph position="19"> 3. When both cases 1 and 2 are true then place \[wordz word2 head\] under both concepts.</Paragraph>
      <Paragraph position="20">  its ~ concepts Since we do not deal here with the sentence semantics, it is not possible to completely determine the meaning of \[word1 word2 head\], as it may be either \[((word1 word2) head)\] or \[(word1 (words head))\] often depending on the sentence context.</Paragraph>
      <Paragraph position="21"> In the example of Figure 3 there is only one meaning, i.e. \[(automobile radio) components\]. However, in the case of ~erformance skiing equipment\] there are two valid interpretations, namely \[(performance skiing) equipment\] and ~erformance (skiing equipment)\]. null Procedure 4.4 Classify a concept \[word1, head\] with respect to a concept h/erarchy under the ~aead\]. The task here is to identify the most specific subsumer (MSS) from all the concepts under the head that subsumes \[wordx, head\]. By default, \[wordl head\] is placed under \[head\], however, since it may be more specific than other hyponyms of \[head\], a more complex classification analysis needs to be implemented. null In the previous work on knowledge classification it was assumed that the concepts were accompanied by rolesets and values (Schmolze and Lipkis 1983), (Woods 1991), and others. Knowledge classifiers are part of almost any knowledge representation system. However, the problem we face here is more difficult. While in build-by-hand knowledge representation systems, the relations and values defining concepts are readily available, here we have to extract them from text. Fortunately, one can take advantage of the glossary definitions that are associated with concepts in WordNet and other dictionaries. One approach is to identify a set of semantic relations into which the verbs used in the gloss definitions are mapped into for the purpose of working with a manageable set of relations that may describe the concepts restrictions. In WordNet these basic relations are already identified and it is easy to map every verb into such a semantic relation.</Paragraph>
      <Paragraph position="22"> As far as the newly discovered concepts are concerned, their defining relations need to be retrieved from texts. Human assistance is required, at least for now, to pinpoint the most characteristic relations that define a concept.</Paragraph>
      <Paragraph position="23"> Below is a two step algorithm that we envision for the relative classification of two concepts A and B. Let's us denote with ARaCa and BRbCb the relationships that define concepts A and B respectively. These are similar to rolesets and values.</Paragraph>
      <Paragraph position="24">  1. Extract relations (denoted by verbs) between concept and other gloss concepts.</Paragraph>
      <Paragraph position="25"> ARalC~I BRblCbl ARa2Ca2 BRb2Cb2 AR,~Cam B Rbn Cb,, 2. A subsumes B ff and only if (a) Relations Rai subsume Rbl, for 1 &lt; i &lt; m.</Paragraph>
      <Paragraph position="26"> (b) Col subsumes or is a meronym of Cbi.</Paragraph>
      <Paragraph position="27"> (c) Concept B has more relations than concept  A, i.e. m&lt;n.</Paragraph>
      <Paragraph position="28"> Example: In Figure 4 it is shown the classification of concept monetary policy that has been discovered. By default this concept is placed under policy. However in WordNet there is a hierarchy fiscal policy IS-A - economic policy - IS-A - policy. The question is where exactly to place monetary policy in this hierarchy. null The gloss of economic policy indicates that it is MADE BY Government, and that it CONTROLS economic growth- (here we simplified the explanation and used economy instead of economic growth). The gloss of fiscal policy leads to relations MADE BY Government, CONTROLS budget, and CONTROLS taxation. The concept money supply was found by Procedure 1.2 in several dictionaries, and its dictionary definition leads to relations MADE BY Federal Government, and CONTROLS money supply. In Word-Net Government subsumes Federal Government, and economy HAS PART money. All necessary conditions are satisfied for economic policy to subsume monetary policy. However, fiscal policy does not subsume monetary policy since monetary policy does not control budget or taxation, or any of their hyponyms. Procedure 4.5 Merge a structure of concepts with the rest of the knowledge base.</Paragraph>
      <Paragraph position="29"> It is possible that structures consisting of several inter-connected concepts are formed in isolation of the main knowledge base as a result of some procedures. The task here is to merge such structures with the main knowledge base such that the new knowledge base will be consistent with both the structure and the main knowledge base. This is done by  bridging whenever possible the structure concepts and the main knowledge base concepts. It is possible that as a result of this merging procedure, some HYPERNYMY relations either from the structure or the main knowledge base will be destroyed to keep the consistency. An example is shown in Figure 5. Example : The following HYPERNYMY relationships were discovered in Part 3: HYPERNYMY(financial market,capital market) HYPERNYMY(fInancial market,money market) HYPERNYMY(capital market,stock market) The structure obtained from these relationships along with a part of WordNet hierarchy is shown in Figure 5. An attempt is made to merge the new structure with WordNet. To these relations it corresponds a structure as shown in Figure 5. An attempt is made to merge this structure with Word-Net. Searching WordNet for all concepts in the structure we find money market and stock market in WordNet where as capital market and financial market are not. Figure 5 shows how the structure merges with WordNet and moreover how concepts that were unrelated in WordNet (i.e. stock market and money market) become connected through the new structure. It is also interesting to notice that the IS-A link in WordNet from money market to market is interrupted by the insertion of financial market in-between them.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>