<?xml version="1.0" standalone="yes"?>
<Paper uid="P05-1056">
  <Title>Using Conditional Random Fields For Sentence Boundary Detection In Speech</Title>
  <Section position="3" start_page="0" end_page="451" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Standard speech recognizers output an unstructured stream of words, in which the important structural features such as sentence boundaries are missing.</Paragraph>
    <Paragraph position="1"> Sentence segmentation information is crucial and assumed in most of the further processing steps that one would want to apply to such output: tagging and parsing, information extraction, summarization, among others.</Paragraph>
    <Section position="1" start_page="0" end_page="451" type="sub_section">
      <SectionTitle>
1.1 Sentence Segmentation Using HMM
</SectionTitle>
      <Paragraph position="0"> Most prior work on sentence segmentation (Shriberg et al., 2000; Gotoh and Renals, 2000; Christensen et al., 2001; Kim and Woodland, 2001; NIST-RT03F, 2003) have used an HMM approach, in which the word/tag sequences are modeled by N-gram language models (LMs) (Stolcke and Shriberg, 1996). Additional features (mostly related to speech prosody) are modeled as observation likelihoods attached to the N-gram states of the HMM (Shriberg et al., 2000). Figure 1 shows the graphical model representation of the variables involved in the HMM for this task. Note that the words appear in both the states1 and the observations, such that the word stream constrains the possible hidden states to matching words; the ambiguity in the task stems entirely from the choice of events. This architecture differs from the one typically used for sequence tagging (e.g., part-of-speech tagging), in which the &amp;quot;hidden&amp;quot; states represent only the events or tags. Empirical investigations have shown that omitting words in the states significantly degrades system performance for sentence boundary detection (Liu, 2004). The observation probabilities in the HMM, implemented using a decision tree classifier, capture the probabilities of generating the prosodic features</Paragraph>
      <Paragraph position="2"/>
    </Section>
  </Section>
class="xml-element"></Paper>