<?xml version="1.0" standalone="yes"?>
<Paper uid="M91-1011">
  <Title>26 MAY KIDNAPPING TERRORIST ACT &amp;quot;COLOMBIAN ARMY OF NATIONAL LIBERATION&amp;quot; ARMY OF NATIONAL LIBERATION REPORTED AS FACT*</Title>
  <Section position="4" start_page="86" end_page="87" type="metho">
    <SectionTitle>
LIMITING FACTORS
</SectionTitle>
    <Paragraph position="0"> In LSI's case, these included all of the items on the NOSC list of possible limiting factors, i .e., time, people , cpu cycles, and knowledge (interpreted as limitations on system use of available knowledge), as discussed below.</Paragraph>
    <Paragraph position="1"> Time/People There were not enough person hours available to LSI under the given resource limitations to carry out th e full effort required for MUC-3 in the time allotted, as well as to perform on other in-house contracts . Although particular sentence types represented in the MUC-3 texts were no more complex than in previous projects suc h as the air activities work carried out for Rome Laboratory, there was a great deal more variety in the sentenc e types in the MUC-3 texts, and the time and resources available were substantially less than the 18 months an d $225K dedicated to the Rome effort .</Paragraph>
    <Paragraph position="2"> Cpu Cycle s The main problem here is that we were essentially developing and debugging a substantial number of line s of new code, requiring detailed tracing facilities to identify and fix bugs . In addition, a key component for MUC3 texts is LUX (Lexical Unexpected Inputs) and its associated module WAM (Word Acquisition Module ) which deal with items not present in the DBG lexicon, either because they are misspellings of existing entries , or because they are new words . LUX in particular consumes a lot of cpu cycles, but is absolutely critical fo r processing of texts containing many words which are unknown to the DBG lexicon .</Paragraph>
    <Paragraph position="3"> In order to complete Test 2 in some finite time frame, it was therefore necessary to limit the input to th e DBG message understanding system by utilizing some fairly draconian measures . Discourse analysis of the development corpus revealed that detecting the transitions from descriptions of one event to another in the text s was too complicated to attempt within the limited resources, so all messages labeled as relevant were arbitraril y truncated to 10 sentences . Since the intent was to exclude all but event reports, which typically describe th e event(s) of interest within the 10 sentence segment, it did not appear that much information loss would resul t from this measure. No attempt was made to determine the number of MUC-3 templates represented in the truncated text that began at the 11th sentence of all messages in the critical event directory . (For further discussion , see LSI's section in the paper on discourse processing within the MUC-3 context included in this proceedings .) The selection of relevant messages, which was performed using Logicon's Message Dissemination Syste m (LMDS), 4 was thus executed to exclude all potentially windy messages such as reports of political speeches , interviews, and clandestine radio broadcasts of political propaganda, as well as military attack events and dru g traffic related events not involving terrorist acts . Critical event and irrelevant message criteria were defined i n terms of LMDS profiles and used to filter the test message set into the groups shown in Table 2 . The LMDS 4. LMDS does shallow text analysis based on boolean combinations of key words/phrases with proximity and other criteria, which operat e as search logic specifications within particular user-defined zones of messages (e .g ., mediasource zone, with typical contents such as &amp;quot;Radio Venceremos&amp;quot;, &amp;quot;Television Peruana&amp;quot;, etc .).</Paragraph>
    <Paragraph position="4">  filtering was performed as a pre-processing run which took less than a minute for the total test set of 100 messages. In Table 2, false positives (in terms of the MUC-3 test key) in the critical event directory and false negatives in the &amp;quot;nohit&amp;quot; d irectory are indicated by Xs.</Paragraph>
    <Paragraph position="5"> Table 3 contains numbers of the messages which were partially processed, but produced no template outpu t because processing hung up in the parser. Per the instructions for running the test procedure, processing was res tarted with the next relevant message following these parser failures .</Paragraph>
    <Paragraph position="6"> Knowledge Availability vs. System Functionalit y The DBG system was fairly well primed with knowledge, as can be seen from the size of the lexicon and th e frames data bases cited previously. However, because the the GB parser was not completely functional (in fact, was still undergoing extensive debugging, as mentioned above), many attachments were not being made, resulting in a large number of partial trees which could not labeled with their thematic roles (i .e., agent, patient, etc.) . The consequences of these attachment failures propagated throughout the remainder of the processing components, resulting in predicate/argument functions unlabeled, unindexed, or missing in the functional parse, s o that the DBG templates were extremely sparse, as were the MUC-3 application templates. Figures 2 and 3 show partial output for Test 2 Message 100, which illustrates this point. Essentially because of the limited functionality of the GB parser at this stage of development, a great deal of the knowledge represented in the frame hierarchy and associated rules, as well as knowledge represented in the rules for filling the MUC-3 templates, wa s never exploited by the system .</Paragraph>
  </Section>
  <Section position="5" start_page="87" end_page="87" type="metho">
    <SectionTitle>
TRAINING
</SectionTitle>
    <Paragraph position="0"> It goes without saying that the development corpus was extremely useful in lexical, syntactic, semantic, an d discourse analysis for system development . We also found the treebank analysis of MUC-3 messages very useful for identifying the multitude of possible variations on a single syntactic theme . Due to the partially functional status of the evolving GB parser, we were unable to fully exploit the 1300 messages in testing for this phase of MUC-3, but were limited to a few messages that we used for regression testing. The MUC-3 corpus is a very valuable archive that we intend to utilize more fully in the next few months, as our parser stabilizes an d we can take advantage of the variety of texts represented in the MUC-3 collection .</Paragraph>
  </Section>
  <Section position="6" start_page="87" end_page="89" type="metho">
    <SectionTitle>
MODULE MOST OVERDUE FOR REWRITIN G
</SectionTitle>
    <Paragraph position="0"> Most of the components of the DBG system have been at least revised and extended, and in most cases , completely replaced, as part of our evolutionary design philosophy (see the system summary paper for a discussion). null As noted previously, however, one of the oldest modules in the system is the Lexical Unexpected Inputs (LUX) module, and its associated Word Acquisition Module (WAM) . These modules attempt to determine whether an unexpected lexical input (i .e., one which is not present in the lexicon) is erroneous (e .g., a misspelling of an entry actually present in the lexicon), or entirely new. In the first case, LUX goes through an elaborate procedure to determine whether a spelling error exists, which is corrected if a reasonable hypothesis fo r an association with a word in the lexicon can be found (e .g., in test 1, the form &amp;quot;kidapped&amp;quot; was corrected to &amp;quot;kidnapped&amp;quot;). If no correction can be made, the form is determined to be new, which requires WAM to provide a temporary grammatical category assignment so that the sentence containing the new word can be parsed .</Paragraph>
    <Paragraph position="1"> As noted previously, our lexicon included approximately 10,000 words ; however, the vocabulary in the MUC-3 development corpus is estimated at 20,000 words (see Hirschman paper in this proceedings) . Clearly, the LUX/WAM components were of inestimable value to us in processing the test sets ; it would have been impossible to run without them . On the other hand, because of the many new words encountered in the MUC- 3 texts, LUX and WAM had to be used many times on every message, and because these procedures are non optimized at present, the amount of time devoted to autonomous LUX/WAM processing was substantial .</Paragraph>
    <Paragraph position="2"> Another module that should be rewritten for higher efficiency is LXI, which handles lexical lookup and morphological processing; however, LUX/WAM are first on the list.</Paragraph>
  </Section>
  <Section position="7" start_page="89" end_page="90" type="metho">
    <SectionTitle>
APPLICATION OUTPUT
MUC TEMPLATES
0. MESSAGE ID
1. TEMPLATE ID
2. DATE OF INCIDENT
3. TYPE OF INCIDENT
4. CATEGORY OF INCIDENT
</SectionTitle>
    <Paragraph position="0"> 5. PERPETRATOR : ID OF INDIV(S ) 6. PERPETRATOR : ID OF ORG(S) 7. PERPETRATOR : CONFIDENCE 8. PHYSICAL TARGET : ID(S) 9. PHYSICAL TARGET : TOTAL NUM 10. PHYSICAL TARGET : TYPE(S) 11. HUMAN TARGET: ID(S) 12. HUMAN TARGET: TOTAL NUM 13. HUMAN TARGET : TYPES) 14. TARGET: FOREIGN NATION(S) 15. INSTRUMENT : TYPE(S) 16. LOCATION OF INCIDENT 17. EFFECT ON PHYSICAL TARGET(S ) 18. EFFECT ON HUMAN TARGET(S) 0 . MESSAGE ID 1. TEMPLATE ID 2. DATE OF INCIDENT 3. TYPE OF INCIDENT 4. CATEGORY OF INCIDENT 5. PERPETRATOR : ID OF INDIV(S) 6. PERPETRATOR: ID OF ORG(S) 7. PERPETRATOR: CONFIDENCE 8. PHYSICAL TARGET : ID(S) 9. PHYSICAL TARGET : TOTAL NUM 10. PHYSICAL TARGET : TYPE(S) 11. HUMAN TARGET: ID(S) 12. HUMAN TARGET: TOTAL NUM 13. HUMAN TARGET: TYPE(S) 14. TARGET: FOREIGN NATION(S )</Paragraph>
  </Section>
class="xml-element"></Paper>