<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0906">
  <Title>Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 41-48, Ann Arbor, June 2005. c(c)2005 Association for Computational Linguistics Evaluating Summaries and Answers: Two Sides of the Same Coin?</Title>
  <Section position="6" start_page="46" end_page="46" type="concl">
    <SectionTitle>
5 Conclusion
</SectionTitle>
    <Paragraph position="0"> What's in store for the ongoing co-evolution of summarization and question answering? Currently, definition questions exercise a system's ability to integrate information from multiple documents. In the process, it needs to automatically recognize similar information units to avoid redundant information, much like in multi-document summarization. The other research direction in advanced question answering, integrationofreasoningcapabilitiestogenerate answers that cannot be directly extracted from text, remains more elusive for a variety of reasons.</Paragraph>
    <Paragraph position="1"> Finer-grained linguistic analysis at a large scale and sufficiently-rich domain ontologies to support potentially long inference chains are necessary prerequisites--both of which represent open research problems. Furthermore, it is unclear how exactly one would operationalize the evaluation of such capabilities. null Nevertheless, we believe that advanced reasoning capabilities based on detailed semantic analyses of text will receive much attention in the future. The recent flurry of work on semantic analysis, based on resources such as FrameNet (Baker et al., 1998) and PropBank (Kingsbury et al., 2002), provide the substrate for reasoning engines. Developments in the automatic construction, adaptation, and merging of ontologies will supply the knowledge necessary to draw inferences. In order to jump-start the knowledge acquisition process, we envision the development of domain-specific question answering systems, the lessons from which will be applied to systems that operate on broader domains. In terms of operationalizing evaluations for these advanced capabilities, the field has already made important first steps, e.g., the Pascal Recognising Textual Entailment Challenge.</Paragraph>
    <Paragraph position="2">  Whateffectwillthesedevelopmentshaveonsummarization research? We believe that future systems will employ more detailed linguistic analysis.</Paragraph>
    <Paragraph position="3"> As a simple example, the ability to reason about people's age based on their birthdates would undoubtedly be useful for answering particular types of questions, but may also play a role in redundancy detection, for example. In general, we anticipate a move towards more abstractive techniques in multidocumentsummarization. Fluent,cohesive,andtopical summaries cannot be generated solely using an extractive approach--sentences are at the wrong level of granularity, a source of problems ranging from dangling anaphoric references to verbose subordinate clauses. Only through more detailed linguistic analysis can information from multiple documents be truly synthesized. Already, there are hybrid approaches to multi-document summarization that employ natural language generation techniques (McKeown et al., 1999; Elson, 2004), and researchers have experimented with sentential operations to improve the discourse structure of summaries (Otterbacher et al., 2002).</Paragraph>
    <Paragraph position="4"> The primary purpose of this paper was to identify similarities between multi-document summarization and complex question answering, pointing out potentialsynergisticopportunitiesintheareaofsystem null evaluation. We hope that this is merely a small part of a sustained dialogue between researchers from these two largely independent communities. Answering complex questions and summarizing multiple documents are essentially opposite sides of the same coin, as they represent different approaches to  thecommonproblemofaddressingcomplexuserinformation needs.</Paragraph>
  </Section>
class="xml-element"></Paper>