<?xml version="1.0" standalone="yes"?>
<Paper uid="M98-1011">
  <Title>NYU: Description of the Proteus#2FPET System as Used for MUC-7 ST</Title>
  <Section position="5" start_page="5" end_page="5" type="evalu">
    <SectionTitle>
PERFORMANCE ON THE LAUNCH SCENARIO
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="5" end_page="5" type="sub_section">
      <SectionTitle>
Scenario Patterns
</SectionTitle>
      <Paragraph position="0"> This section describes howthe Proteus#2FPET system was adapted to accommodatethe MUC-7 scenario.</Paragraph>
      <Paragraph position="1"> The scenario-speci#0Cc patterns were primarilyoftwotypes: those for launchevents #28#5CNASA launched a rocket.&amp;quot;, #5CTheterrorists #0Cred a missile.&amp;quot;#29 andthose for missions#28#5Cthe retrieval of a satellite&amp;quot;#29. Starting from patterns for simpleactive clauses, the system automaticallygenerated patterns for the syntactic variants, such as the passive, relative, andreduced relative clauses. The missions added informationregarding payloads and mission functions toalaunchevent, but did not directly generatealaunchevent. In some cases, the mission was syntactically tied to a particular launchevent #28#5C... launched theshuttle todeployasatellite&amp;quot;#29. If there was no direct connection, the post-processing inference rules attempted totie the mission toalaunchevent.</Paragraph>
      <Paragraph position="2"> Inference Rules Consider theevent in #0Cgure 4: the surface representation contains a generic #5CAgent&amp;quot; role. The agent can be of several types, e.g. it can be a launchvehicle, an organization, or evenalaunchsite, in case the agentisa country.Inthis case, the role is #0Clled by an organization, which, in principle, further admitsthe possibility of either the payload owner or thevehicle owner. The scenario speci#0Ccation mandates thatthefunction of the #5Cagent&amp;quot; be precisely speci#0Ced, although atthe surface it is underspeci#0Ced. In this case, thefunction can be determined on the basis of thesemantic class of the agent, andthe observation thatthe payload-owner slot is already occupied unambiguously by another organization entity.</Paragraph>
      <Paragraph position="3"> This type of computation is performed by scenario-speci#0Cc inference rules; in general, this determination can be quite complex. Translatingthe surface representations intothose mandated bythetask speci#0Ccation can involvemany-to-many relations, suchasones that exist between payloads andlaunchevents, where multiple payloads correspondtoasingle event, andmultiple launchevents are concerned withasingle payload.</Paragraph>
      <Paragraph position="4"> Onetechnique thatappeared fruitful in theLaunch scenario was extending our set of inference rules with heuristics. Often a slot in an event cannot be #0Clled, as when patterns fail to#0Cnd a syntactically suitable  The expert user can view thevariants whichthe system generates, andmakechanges to them directly.</Paragraph>
      <Paragraph position="5">  The tools can be used to acquire non-clausal patterns as well, e.g. patterns for noun groups and complex noun phrases, to extend an existing pattern library.</Paragraph>
      <Paragraph position="6">  candidate. Theidea was tomakeintelligent guesses about #0Cllers for these empty slots. For example, consider the #0Crst sentence of thewalk-through message: Xichang, China, Feb. 15 #28Bloomberg#29 #7B A Chinese rocket carrying an Intelsat satellite exploded as it was being launchedtoday, delivering a blow ::: Here we#0Cndtwo similar problems: concerningthelaunchdateandthelaunchsite. Our patterns recognize the correspondinglocativeandtemporal noun phrases, however, because neither stands in a direct syntactic relation tothemain launchevent clause #28here, headed bytheverb #5Cexplode&amp;quot;#29, they fail to #0Cll theappropriate slotsintheevent. We use a simple heuristic rule to recover from this problem: if thelauncheventhas an emptydate, andifthesentence contains a unique expression of the correct type #28i.e. date#29, use the expression to #0Cll the empty slot.</Paragraph>
      <Paragraph position="7"> Wehave experimented withavarietyofheuristics for several slots, including organizations for vehicle and payload owners andmanufacturers, dates andsites. At present, the contribution of these heuristics to our score accounts for just under 10#25 of the F-measure. It is also apparentthat someoftheheuristics actually overgenerate, though wehaveyet toanalyze their e#0Bect in detail.</Paragraph>
      <Paragraph position="8"> We believethattheoverall approach of example-based pattern acquisition is more appropriatethan automatic training from annotated corpora, as the amount of trainingdata for ST-level tasks is usually quite limited. Wehave foundpattern editingtool reasonably e#0Bective. However, we discovered thatmuchofthe task involved creation andtuning of post-processing rules andwedoyet not have support in thetool for this activity. This consumed a considerable part of the customization e#0Bort . This pointsto an importantproblem thatneeds to be addressed, especially for tasks where the structure of outputtemplates di#0Bers substantially from the structure of entities andevents as picked up bythe syntactic analysis.</Paragraph>
      <Paragraph position="9"> We did not speci#0Ccally focus on theTEtask within thelaunch scenario, and simplyused the same system wehad used for theSTtask. Table 5 is a summary of the scores of our system.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>