<?xml version="1.0" standalone="yes"?>
<Paper uid="M91-1006">
  <Title>BBN PLUM: MUC-3 Test Results and Analysis</Title>
  <Section position="5" start_page="56" end_page="57" type="concl">
    <SectionTitle>
CONCLUSIONS
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="56" end_page="56" type="sub_section">
      <SectionTitle>
Successes
</SectionTitle>
      <Paragraph position="0"> PLUM has the following key features:  1. Fragment production based on the lexicon and local syntactic information 2. Partial understanding provided for each fragment found. 3. Event-based and template-based knowledge to find relations among entities when syntax/semantics canno t find them.</Paragraph>
      <Paragraph position="1"> 4. Statistical language models at multiple levels .</Paragraph>
      <Paragraph position="2">  These were the key to PLUM's performance in MUC-3 . All components of PLUM except the domain-specifi c knowledge bases seem transferable to other domains .</Paragraph>
    </Section>
    <Section position="2" start_page="56" end_page="57" type="sub_section">
      <SectionTitle>
Improvements Desired
</SectionTitle>
      <Paragraph position="0"> Coverage in both the semantics and discourse components can and should be increased . The fragmen t combining component should be tested and evaluated thoroughly, since it was not thoroughly tested in MUC-3 .</Paragraph>
      <Paragraph position="1"> Rather than a purely deterministic fragment finding algorithm as in MITFP, a fragment finding algorithm based o n probabilistic language models and local search might provide more accurate prediction of phrase boundaries an d phrase types.</Paragraph>
      <Paragraph position="2"> The template generator today is based on hand-crafted rules of thumb . Within the next two years we hope to develop and test an acquisition algorithm that would acquire most of the rules from examples in a new domain.</Paragraph>
    </Section>
    <Section position="3" start_page="57" end_page="57" type="sub_section">
      <SectionTitle>
Lessons Learned
</SectionTitle>
      <Paragraph position="0"> The degree of success obtained by marrying fragment processing/partial understanding with statistical techniques has been quite gratifying . The availability of 1300 messages with their desired templates was invaluable .</Paragraph>
      <Paragraph position="1"> Furthermore, the value of annotated text as in TREEBANK was great ; the provision of more data is warranted an d would be even better. It would also have been impossible to determine our progress over large sets of message s without the scoring program.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>