<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0404">
  <Title>Revisions that Improve Cohesion in Multi-document Summaries: A Preliminary Study</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Revision-based system architecture
</SectionTitle>
    <Paragraph position="0"> The proposed architecture of our system, which would implement the generate-and-revise approach to summarization, is depicted in Figure 1. Input to this system is a cluster of source documents related to the same topic. Next, sentence extraction takes place, in which important sentences in the articles are identified. The output of this module is an extract, which lists the sentences to be included in the summary.</Paragraph>
    <Paragraph position="1"> In the next stage, Cross-document Structure Theory (CST) relationships are established. Specific relationships between sentences are identified. Here, a CST-enhancement procedure [Zhang et al, 2002] may take place, ensuring that interdependent sentences appear together in a summary. Sentences may also be reordered in the summary with respect to their temporal relations, topic, or other criteria.</Paragraph>
    <Paragraph position="2"> The next stage in the process is the revision module. First, high level revision operators are chosen, with respect to the cohesion problems that need repair. Afterwards, the specific lexical items to be added, deleted or modified are chosen. The output of this module is the revised, enhanced summary.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 The MEAD summarizer
</SectionTitle>
      <Paragraph position="0"> The MEAD summarizer [Radev et al, 2000] [Radev et al 2002] is based on sentence extraction and uses a linear combination of three features to rank the sentences in the source documents. The first of the three features is the centroid score, which quantifies the centrality of a sentence to the overall cluster of documents. The second is the position score, which assigns higher scores to sentences that are closer to the beginning of the document. The third feature, length, gives a higher score to longer sentences. Using a linear combination of the three features, sentences are ranked by score and added to the summary until the desired length is attained.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Data and procedure
</SectionTitle>
    <Paragraph position="0"> We generated a corpus of summaries using the MEAD summarizer. The original documents come from three sources - DUC 2001, the Hong Kong News corpus, and the GA3-11 data set. One cluster of related news articles was chosen from each source. The DUC 2001 articles describe the 1991 eruption of Mount Pinatubo in the Philippines.</Paragraph>
    <Paragraph position="1"> This cluster, which is not typical of the DUC data, focuses on this single event and its subevents over a 2-week time period. Those taken from the HK corpus are about government initiatives surrounding the problem of drug rehabilitation. Due to the expense and labor involved in the generation and revision of multi-document summaries, we have used a subset of 15 summaries from our corpus in order to develop our revision taxonomy and to present some initial findings. Our future revision studies will employ a much larger set of data.</Paragraph>
    <Paragraph position="2"> The summaries were revised manually by the first author. This was a three-step process that involved identifying each problem, choosing an operator that could address the problem and then selecting the lexical items to which the operator should be applied. It is important to note that multiple lexical choices are possible in some cases.</Paragraph>
    <Paragraph position="3"> Since we were interested in identifying all types of cohesion problems as well as considering all possibilities for addressing these problems, the reviser was permitted to make any revision necessary in order to correct problems in the summaries. Obviously, a module that makes revisions automatically would be much more restricted in its set of revision operators. However, since a major goal for this paper was to establish a taxonomy of problems specific to multi-document summarization and to consider the complexities involved in making repairs in MDS, we did not place such restrictions on the reviser. Rather, she applied corrections to the summaries as to make them as intelligible as possible, given the sentences chosen by the summarizer.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Revision example
</SectionTitle>
      <Paragraph position="0"> &lt;DELETE-place stamp&gt; Cairo, Egypt - &lt;/DELETE&gt; The crash of a Gulf Air flight that killed 143 people in Bahrain &lt;ADD-time exp-day&gt;Wednesday &lt;/ADD&gt; is a disturbing deja vu for Egyptians: It is the second plane crash within a year to devastate this Arab country. Egypt, which lacks the oil wealth of the Gulf and has an economy struggling to revive from decades of socialist stagnation, has a long tradition of sending workers to the Gulf to fill everything from skilled to menial jobs. &lt;DELETE-place stamp&gt; Manama,  The above figure shows an example of a revised summary that was produced from three source articles from the GA3-11 corpus. The news stories were collected live from the web, and come from two different sources www.foxnews.com and www.abcnews.com. The revision operator used and the corresponding pragmatic concern precede the modified text in pointed brackets. This type of markup scheme was used because it enables us to use simple Perl scripts to move between the original and revised versions of the summaries.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Taxonomy of revision strategies
</SectionTitle>
    <Paragraph position="0"> Based on our corpus of revised summaries, we have identified five major categories of pragmatic concerns related to text cohesion in multi- null document summaries: 1) Discourse - Concerns the relationships between the sentences in a summary, as well as those between individual sentences and the overall summary.</Paragraph>
    <Paragraph position="1"> 2) Identification of entities - Involves the resolution of referential expressions such that each entity mentioned in a summary can easily be identified by the reader.</Paragraph>
    <Paragraph position="2"> 3) Temporal - Concerns the establishment of the correct temporal relationships between events.</Paragraph>
    <Paragraph position="3"> 4) Grammar - Concerns the correction of  grammatical problems, which may be the result of juxtaposing sentences from different sources, or due to the previous revisions that were made.</Paragraph>
    <Paragraph position="4"> 5) Location/setting - Involves establishing where each event in a summary takes place. Explanations of the specific pragmatic concerns in each category, as well as their corresponding operator(s), are detailed in the appendix. Overall, 160 revisions were made across the 15 summaries.  The majority (82%) of the revisions fall into the first three categories. This is not surprising, as in MDS, we expect to find many problems relating to discourse - such as abrupt topic shifts or redundant messages. Additionally, concerns relating to the identification of entities in the text are likely to occur when the sentence from the original document that introduced an entity is not included in the resulting summary, but sentences that make reference to the entity are included. Finally, it may not be clear when events described in a summary occurred. This could be because sentences which stated when the event occurred were left out of the summary or because the sentences include relative time expressions such as 'today' even though the stories were written at different times or on different days.</Paragraph>
    <Paragraph position="5"> Revisions relating to grammar or to establishing where an event occurred were less frequently used, accounting for only 12% and 6% of the total repairs, respectively. Sentences extracted from the original news stories are usually grammatical. However, problems related to grammar may arise from previous revisions. In our corpus, the place or setting of an event was typically obvious in the summary and rarely required repair.</Paragraph>
    <Paragraph position="6"> Next, we present the analysis of revisions within each of the five categories. We are interested in revising our summaries to be as coherent as possible, without having to implement complicated and knowledge-intensive discourse models. Therefore, we will discuss the feasibility of implementing the revisions in our taxonomy automatically. null</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.1 Discourse-related concerns in MDS
</SectionTitle>
      <Paragraph position="0"> It is intuitive that problems relating to discourse are abundant in our summaries and, at the same time, that such repairs would be the most difficult to make. The first obstacle is the detection of each of these concerns, which requires knowledge of the rhetorical relations of the sentences in the summary. null  In all the instances of topic shift and lack of purpose in our corpus, a phrase or an entire sentence was added to provide a transition or motivation for the troublesome sentence. Therefore, our module would require the ability to generate text, in order to repair these problems, which occur often in our summaries.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.2 Identification of entities in MDS
</SectionTitle>
      <Paragraph position="0"> Nine specific problems were found that concern the reader's ability to identify each entity mentioned in a summary. Most of these revisions could be made using rewrite rules. For example, if it can be determined that a definite article is used when a (non-proper noun) entity is mentioned for the first time, the misused definite article could be replaced with the corresponding indefinite article.</Paragraph>
      <Paragraph position="1"> The most frequent problem, underspecified entity, is the most difficult one to correct. This disfluency typically occurs where an entity is referred to by a proper noun or other noun phrase, such as the name of a person or organization, but has no title or further description. In such cases, the missing information may be found in the source document only.</Paragraph>
      <Paragraph position="2">  Therefore, to correct the underspecified entity problem, a revision module might require a knowledge source for the profiles of entities mentioned in a summary. When an entity is introduced for the first time in a summary, it should be associated with its description (such as a title and full name for a person).</Paragraph>
      <Paragraph position="3"> Discourse information would be useful for solving problems such as a bare anaphor or missing subject. In revising single-document summaries, [Mani et al, 1999] employed rules such as the referencing of pronouns with the most recently mentioned noun phrase. However, this might be inappropriate in MDS, where the use of multiple documents increases the number of possible entities with which an anaphor could be referenced.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.3 Temporal relationships in MDS
</SectionTitle>
      <Paragraph position="0"> An important aspect of revision in MDS is the establishment of the correct temporal relationships between the events described in a summary. We have identified five types of problems that fall into this category.</Paragraph>
      <Paragraph position="1">  The most frequent revision in this category for our multi-document summaries was temporal ordering. This is an important consideration for the summarization of news articles, which typically describe several events or a series of events in a given news story.</Paragraph>
      <Paragraph position="2"> A revision module might use metadata, including the time stamps of source documents, in addition to surface properties of sentences in addressing this problem. Temporal relations were typically established by adding a time expression to one or more sentences in a summary. Therefore, our module will require a dictionary of such expressions as well as a set of rules for assigning an appropriate expression to a given sentence. For example, if the time stamps of two source documents from which two adjacent summary sentences come indicate that they were written one day apart, an appropriate way to order them might be: add a time expression indicating the day to the first sentence, and a relative time expression such as 'the following day' to the second sentence. Our dictionary will require both relative and absolute time expressions at different levels of granularity (hour, day, etc.).</Paragraph>
      <Paragraph position="3"> Most of the temporal revisions in our corpus were made at points where sentences from different sources followed one another or when sentences from the same source were far apart in the original document. By using such clues, it is hoped that temporal relations problems in summaries can be corrected without knowledge of the discourse.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.4 Grammatical concerns in MDS
</SectionTitle>
      <Paragraph position="0"> The majority of grammatical problems in our corpus resulted from previous revisions performed on the text. For example, the addition of information to a sentence can result in it becoming too long.</Paragraph>
      <Paragraph position="1"> Such concerns can also occur because the grammar of one sentence, such as verb tense, does not match that of the next sentence.</Paragraph>
      <Paragraph position="2">  A revision module should be able to correct the above concerns using rules applied after other revisions are made and without any discourse knowledge.</Paragraph>
    </Section>
    <Section position="5" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.5 Location/setting concerns
</SectionTitle>
      <Paragraph position="0"> The least frequent type of revision made in our corpus related to establishing the correct locations of events in a summary. Occasionally, a sentence in a summary retains the place/source stamp that appears at the beginning of a news article. This appears ungrammatical unless the sentence is the first in the summary.</Paragraph>
      <Paragraph position="1">  In addition, such stamps might be inappropriate for a summary, since not all the sentences may share the same location. In order to promote cohesion in the summary, our module could move the stamp information into the body of the summary.</Paragraph>
      <Paragraph position="2"> Sentences could be missing location information altogether. In such cases, the revision module might require information from the source documents in order to repair this problem. Overall, the revisions related to establishing the location of events should not require knowledge of discourse in the summary. Adding location information can usually be performed with the addition of a prepositional phrase, usually at the beginning of the sentence. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>