<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0405">
  <Title>Feature-Based Segmentation of Narrative Documents</Title>
  <Section position="2" start_page="0" end_page="32" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Many long text documents, such as magazine articles, narrative books and news articles contain few section headings. The number of books in narrative style that are available in digital form is rapidly increasing through projects such as Project Gutenberg and the Million Book Project at Carnegie Mellon University. Access to these collections is becoming easier with directories such as the Online Books Page at the University of Pennsylvania.</Paragraph>
    <Paragraph position="1"> As text analysis and retrieval moves from retrieval of documents to retrieval of document passages, the ability to segment documents into smaller, coherent regions enables more precise retrieval of meaningful portions of text (Hearst, 1994) and improved question answering. Segmentation also has applications in other areas of information access, including document navigation (Choi, 2000), anaphora and ellipsis resolution, and text summarization (Kozima, 1993).</Paragraph>
    <Paragraph position="2"> Research projects on text segmentation have focused on broadcast news stories (Beeferman et al., 1999), expository texts (Hearst, 1994) and synthetic texts (Li and Yamanishi, 2000; Brants et al., 2002).</Paragraph>
    <Paragraph position="3"> Broadcast news stories contain cues that are indicative of a new story, such as coming up , or phrases that introduce a reporter, which are not applicable to written text. In expository texts and synthetic texts, there is repetition of terms within a topical segment, so that the similarity of blocks of text is a useful indicator of topic change. Synthetic texts are created by concatenating stories, and exhibit stronger topic changes than the subtopic changes within a document; consequently, algorithms based on the similarity of text blocks work well on these texts.</Paragraph>
    <Paragraph position="4"> In contrast to these earlier works, we present a method for segmenting narrative documents. In this domain there is little repetition of words and the segmentation cues are weaker than in broadcast news stories, resulting in poor performance from previous methods.</Paragraph>
    <Paragraph position="5"> We present a feature-based approach, where the features are more strongly engineered using linguistic knowledge than in earlier approaches. The key to most feature-based approaches, particularly in NLP tasks where there is a broad range of possible feature sources, is identifying appropriate features. Selecting features in this domain presents a number of interesting challenges. First, features used in previous methods are not suf cient for solving this problem.</Paragraph>
    <Paragraph position="6"> We explore a number of different sources of information for extracting features, many previously unused. Second, the sparse nature of text and the high  cost of obtaining training data requires generalization using outside resources. Finally, we incorporate features from non-traditional resources such as lexical chains where features must be extracted from the underlying knowledge representation.</Paragraph>
  </Section>
class="xml-element"></Paper>