<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0211">
  <Title>Using Coreference Chains for Text Summarization</Title>
  <Section position="4" start_page="78" end_page="79" type="metho">
    <SectionTitle>
3 Coreference Chain Selection
</SectionTitle>
    <Paragraph position="0"> The summarisation mechanism is implemented as an additional module in the LaSIE system.</Paragraph>
    <Paragraph position="1"> It processes all the coreference chains built by the discourse interpreter and applies various criteria, described below, to select a 'best' chain.</Paragraph>
    <Paragraph position="2"> The set of sentences in which entries in the best chain occur are then identified, using their character positions, and concatenated together to act as a summary.</Paragraph>
    <Section position="1" start_page="78" end_page="79" type="sub_section">
      <SectionTitle>
3.1 Selection Criteria
</SectionTitle>
      <Paragraph position="0"> The summarisation module implements several selection criteria which can be applied either independently or in combination, as specified in parameters of the module. The current set of criteria is not fixed, however, and is simply intended to represent common intuitive heuristics as a starting point for further experimentation.</Paragraph>
      <Paragraph position="1"> Length of Chain This criteria simply prefers the chain containing the most entries, which represents the most frequently mentioned instance in a text, and so possibly the most important instance. In the case of several chains of equal maximum length, the spread is used in addition to select a single chain.</Paragraph>
      <Paragraph position="2">  Spread of Chain This criteria involves the calculation of the distance, in byte offsets, between the earliest and latest entry in each chain. The chain which spans the greatest portion of the original text is preferred, corresponding to the intuition that an instance mentioned throughout a text, rather than just being frequently mentioned in one specific section, may be the most important instance. If this criteria fails to select a single chain, the length criteria will be used in addition.</Paragraph>
      <Paragraph position="3"> Start of Chain A further intuition is that instances mentioned at the start of a text, or in a title if present, may be more important than those only introduced part way through. This criteria acts to require that the earliest entry in a chain is within either the title or the first paragraph of a text.</Paragraph>
      <Paragraph position="4"> Again, other criteria will be needed to select a single chain if several start in the first paragraph. This criteria may be more appropriate for particular text genres, such as newswires, than the previous two.</Paragraph>
    </Section>
    <Section position="2" start_page="79" end_page="79" type="sub_section">
      <SectionTitle>
3.2 Focus Chains
</SectionTitle>
      <Paragraph position="0"> An additional selection mechanism, which may be combined with the above criteria in several ways, is the 'reduction' of each coreference chain to what may be called a &amp;quot;focus chain&amp;quot;. This makes use of the focus registers built within the discourse interpreter to track changes in the main focus of each clause in the text. A focus chain is the subset of a coreference chain which contains only those mentions of an instance that occur as the focus of a clause. For example, an indefinite noun phrase occurring as a logical subject may be recorded in the focus registers, and so be retained in a focus chain, whereas a subsequent definite noun phrase embedded in an optional prepositional phrase will not be in focus and will be removed.</Paragraph>
      <Paragraph position="1"> The criteria listed above can all be applied to focus chains in exactly the same way as coreference chains. However, the point at which the coreference chains are reduced is significant: selecting the longest coreference chain and subsequently removing all non-focus entries before output may give different results than an initial selection of the longest focus chain. The former sequence of operations acts to filter the entries within the 'best' coreference chain, possibly to reduce the final length of a summary, whereas the latter selects on the basis of the frequency (in combination with the length criteria) or the position (with spread) of focus itself. Only the latter sequence is considered in any detail below, although the implementation does allow experimentation with the use of focus chains at several alternative stages.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="79" end_page="80" type="metho">
    <SectionTitle>
4 Example Output
</SectionTitle>
    <Paragraph position="0"> This section illustrates the kind of summaries produced by the different heuristics, using an example from the MUC-6 evaluation corpus of newswire articles (much reduced for inclusion here by omitting the eight final paragraphs):  and an expert in securities laws, is a leading candidate to be chairwoman of the Securities and Exchange Commission in the Clinton administration.</Paragraph>
    <Paragraph position="1"> Ms. Washington, ~ years old, would be the first woman and the first black to head the five-member commission that oversees the securities markets. Ms. Washington's candidacy is being championed by several powerful lawmakers including her boss, Chairman John Dingell (D., Mich.) of the House Energy and Commerce Committee. She currently is a counsel to the committee. Ms. Washington and Mr. Dingell have been considered allies of the securities exchanges, while banks and futures exchanges have often fought with them.</Paragraph>
    <Paragraph position="2"> A graduate of Harvard Law School, Ms. Washington worked as a lawyer for the corporate finance division of the SEC in the late 1970s. She has been a congressional staffer since 1979.</Paragraph>
    <Paragraph position="3"> Separately, Clinton transition o~cials said that Frank Newman, 50, vice chairman and chief financial o~eer of BankAmerica Corp., is expected to be nominated as assistant Treasury secretary for domestic finance. Mr. Newman, who would be giving up a job that pays $1 million a year, would oversee the Treasury's auctions of government securities as well as banking issues. He would report directly to Treasury Secretary-designate Lloyd Bentsen.</Paragraph>
    <Paragraph position="4"> (...) Using the 'length of chain' criteria selects the coreference chain for the person Consuela  Washington, and the following summary is produced: &lt;HL&gt; Economy: Washington, an Exchange Ally, Seems To Be Strong Candidate to Head SEC &lt;/HL&gt; Consuela Washington, a longtime House staffer and an expert in securities laws, is a leading candidate to be chairwoman of the Securities and Exchange Commission in the Clinton administration.</Paragraph>
    <Paragraph position="5"> Ms. Washington, 44 years old, would be the first woman and the first black to head the five-member commission that oversees the securities markets. Ms. Washington's candidacy is being championed by several powerful lawmakers including her boss, Chairman John Dingell (D., Mich.) of the House Energy and Commerce Committee. She currently is a counsel to the committee. Ms. Washington and Mr. Dingell have been considered allies of the securities exchanges, while banks and futures exchanges have often fought with them, A graduate of Harvard Law School, Ms. Washington worked as a lawyer for the corporate finance division o/the SEC in the late 1970s. She has been a congressional staffer since 1979.</Paragraph>
    <Paragraph position="6"> Using the 'spread of chain' selects the coreference chain for the person Clinton, used here as a noun modifier: Consuela Washington, a longtime House staffer and an expert in securities laws, is a leading candidate to be chairwoman of the Securities and Exchange Commission in the Clinton administration.</Paragraph>
    <Paragraph position="7"> Separately, Clinton transition officials said that Frank Newman, 50, vice chairman and chief financial officer of BankAmeriea Corp., is expected to be nominated as assistant Treasury secretary for domestic finance.</Paragraph>
    <Paragraph position="8"> Restricting the selection to focus chains only, the 'length of chain' criteria again chooses the Ms. Washington chain, but this does not include the elements occurring in phrases where Ms. Washington is not in focus. The summary is therefore reduced by omitting the non-focus mentions: &lt;HL&gt; Economy: Washington, an Exchange Ally, Seems To Be Strong Candidate to Head SEC &lt;/HL&gt; Consuela Washington, a longtime House staffer and an expert in securities laws, is a leading candidate to be chairwoman of the Securities and Exchange Commission in the Clinton administration.</Paragraph>
    <Paragraph position="9"> Ms. Washington, 44 years old, would be the first woman and the first black to head the five-member commission that oversees the securities markets. She currently is a counsel to the committee.</Paragraph>
    <Paragraph position="10"> A graduate o/Harvard Law School, Ms. Washington worked as a lawyer for the corporate finance division o/ the SEC in the late 1970s. She has been a congressional staffer since 1979.</Paragraph>
    <Paragraph position="11"> This same summary is also obtained when the 'spread of chain' criteria is applied to the focus chains. The Clinton chain selected above does not include any focus elements, and the Consuela Washington focus chain is the most spread as well as the longest.</Paragraph>
  </Section>
  <Section position="6" start_page="80" end_page="81" type="metho">
    <SectionTitle>
5 Evaluating Summaries
</SectionTitle>
    <Paragraph position="0"> Evaluating the merit of a summary is a difficult task. The most extensive effort to date to develop a framework for assessing summaries has been the TIPSTER SUMMAC evaluation exercise (Mani et al., 1998). In this section we first review the SUMMAC evaluation measures and propose how we might 'simulate' these without expensive human judges; then, we describe how we have carried out one of these simulated evaluations to evaluate the summarization technique described in the previous sections. null</Paragraph>
    <Section position="1" start_page="80" end_page="81" type="sub_section">
      <SectionTitle>
5.1 The SUMMAC Evaluation
</SectionTitle>
      <Paragraph position="0"> This exercise involved a number of different tasks and a number of different evaluation measures. The measures divided into extrinsic measures - those that ignore the content of the summary and assess it solely according to how useful it is in enabling an agent to perform some measurable task - and intrinsic measures - those that examine the content of the summary and attempt to pass some judgment on it directly.</Paragraph>
      <Paragraph position="1"> In brief, the four SUMMAC tasks were: 1. Ad Hoc Task The summariser is given a topic description and a set of documents and produces user-focused summaries based on the topic description. The summaries and the topic description (along with some source documents for control) are passed to a judge who reads the summaries passes relevance judgments on them with respect to the topic. These judgments are scored against 'true' relevance judgments previously established for the full documents with respect to the topic. This  is an extrinsic evaluation that measures the utility of a summarization system at generating user-focused summaries capable of supporting a relevance judgment.</Paragraph>
      <Paragraph position="2"> 2. Categorization Task The summariser is given a set of documents only and generates a summary of each. The summaries (again along with some source documents for control) along with five topic descriptions axe given to a judge who reads the summaries and categorizes them with respect to the five topics, or &amp;quot;none of the above&amp;quot;, if none is deemed appropriate. These categorization judgments are scored against 'true' categorization judgments previously established for the full documents with respect to the topics. This is an extrinsic evaluation that measures the utility of a summarization system at generating generic summaries capable of supporting a categorization judgment.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>