<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1002">
  <Title>Extending Document Summarization to Information Graphics</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Summarization work has focused primarily on the written words in a document. However, graphics are an important part of many documents, and they often convey information that is not included elsewhere in the document. Thus as text summarization branches out, it is essential that it consider the summarization of graphical information in documents.</Paragraph>
    <Paragraph position="1"> Graph summarization has received some attention. (Yu et al., 2002) has used pattern recognition techniques to summarize interesting features of automatically generated graphs of time-series data from a gas turbine engine. (Futrelle and Nikolakis, 1995) developed a constraint grammar formalism for parsing vector-based visual displays and producing structured representations of the elements comprising the display. The goal of Futrelle's project is to produce a graphic that summarizes one or more graphics from a document (Futrelle, 1999).</Paragraph>
    <Paragraph position="2"> The summary graphic might be a simplification of a graphic or a merger of several graphics from the document, along with an appropriate summary caption. Thus the end result of summarization will itself be a graphic.</Paragraph>
    <Paragraph position="3"> Our project is concerned with information graphics (non-pictorial graphics such as bar charts or line graphs). Our current focus is on providing an initial summary of an information graphic, within a larger interactive natural language system that can respond to followup questions about the graphic.</Paragraph>
    <Paragraph position="4"> There are several useful applications for a system that can summarize information graphics. For digital libraries, the initial summary of the graphic will be used in conjunction with the document text/summary to provide a more complete representation of the content of the document to be used for searching and indexing. In the case of environments with low-bandwidth transmission and miniature viewing facilities, such as cellular telephones for accessing the web, the initial summary and followup capability will provide an alternative modality for access to the document.</Paragraph>
    <Paragraph position="5"> However, the most compelling application of the overall system is to provide effective access to information graphics for individuals with sight impairments. The rapidly growing Information Infrastructure has had a major impact on society and the development of technology. However, the growing reliance on visual information display paradigms obliges society to ensure that individuals with visual impairments can access and assimilate information resources as effectively as their sighted counterparts. The underlying hypothesis of our work is that alternative access to what the graphic looks like is not enough -- the user should be provided with the message and knowledge that one would gain from viewing the graphic in order to enable effective and efficient use of this information resource. Thus our system will present the user with an initial summary that includes the primary message that the graphic designer intended to convey, augmented with relevant interesting features of the graphic, and then interactively allow the user to access more detailed summaries of information contained in the graphic.</Paragraph>
    <Paragraph position="6"> As an example of the kinds of summaries that we envision, consider the information graphic in Figure 1. The graphic designer's communicative goal is ostensibly to convey the sharp increase in bankruptcies in 2001 compared with the previous decreasing trend. More detailed features that might be of interest include 1) that bankruptcies had been decreasing at a steady rate since 1998, 2) that bankruptcies had been decreasing slowly since 1998, 3) the percentage decrease each year, 4) the percentage increase in bankruptcies in 2001, 5) the absolute increase in bankruptcies in 2001, and 6) the total number of bankruptcies in 2001. Thus the initial summary of this graphic might be This graphic shows that although Delaware bankruptcy personal filings decreased slowly and steadily from 1998 to 2000, they rose sharply in 2001.</Paragraph>
    <Paragraph position="7"> Note that the proposed summary includes the hypothesized intended message of the graphic, along with the first two of the additional interesting features of the graphic. The selection of additional features to augment the summary is discussed further in Section 3.3. The system would then respond to user requests for additional information by presenting some or all of the other interesting features that had been identified, as discussed in Section 3.4.</Paragraph>
    <Paragraph position="8"> This paper provides an overview of our project.</Paragraph>
    <Paragraph position="9"> Section 2 discusses the essential role of intention recognition in graphics summarization. It argues not only that the intended message of the graphic designer must be inferred and included in a summary of a graphic, but also that the intended message significantly influences the additional propositions that should be included in the summary.</Paragraph>
    <Paragraph position="10"> Section 3 presents our approach to graph summarization. It discusses how we use a computer vision module to construct an XML representation that captures the components of the graphic and their relationship to one another, and how we use a Bayesian belief network to hypothesize the intentions of the graph designer. The paper then discusses our plans for constructing a summary that includes the graphic designer's intended message along with highly ranked additional propositions, and how the lesser ranked propositions will be used in an interactive natural language system that responds to the user's requests for further summaries of additional features of the graphic.</Paragraph>
  </Section>
class="xml-element"></Paper>