<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-1011">
  <Title>Sentence-Internal Prosody Does not Help Parsing the Way Punctuation Does</Title>
  <Section position="12" start_page="0" end_page="0" type="concl">
    <SectionTitle>
4 Discussion and Conclusion
</SectionTitle>
    <Paragraph position="0"> Simple statistical tests show that there is in fact a signi cant correlation between the location of opening and closing phrase boundaries and all of the prosodic pseudo-punctuation symbols described above, so there is no doubt that these do convey information about syntactic structure. However, adding the prosodic pseudo-punctuation symbols uniformly decreased parsing accuracy relative to input with no prosodic information. There are a number of reasons why this might be the case.</Paragraph>
    <Paragraph position="1"> While we investigated a wide range of prosodic features, it is possible that different prosodic features might improve parsing performance, and it would be interesting to see if improved prosodic feature extraction would improve parsing accuracy.</Paragraph>
    <Paragraph position="2"> We suspect that the decrease in accuracy is due to the fact that the addition of prosodic pseudo-punctuation symbols effectively excluded other sources of information from the parser's statistical models. For example, as mentioned earlier the parser uses a mixture of n-gram models to predict the sequence of categories on the right-hand side of syntactic rules, backing off ultimately to a distribution that includes just the head and the preceding sibling's category. Consider the effect of inserting a prosodic pseudo-punctuation symbol on such a model. The prosodic pseudo-punctuation symbol would replace the true preceding sibling's category in the model, thus possibly resulting in poorer over-all performance (note however that the parser also includes a higher-order backoff distribution in which the next category is predicted using the preceding two sibling's categories, so the true sibling's category would still have some predictive value).</Paragraph>
    <Paragraph position="3"> The basic point is that inserting additional information into the parse tree effectively splits the conditioning contexts, exacerbating the sparse data problems that are arguably the bane of all statistical parsers. Additional information only improves parsing accuracy if the information it conveys is sufcient to overcome the loss in accuracy incurred by the increase in data sparseness. It seems that punctuation carries suf cient information to overcome this loss, but that the prosodic categories we introduced do not.</Paragraph>
    <Paragraph position="4"> It could be that our results re ect the fact that we are parsing speech transcripts in which the words (and hence their parts of speech) are very reliably identi ed, whereas our prosodic features were automatically extracted directly from the speech signal and hence might be noisier. If the explanation proposed above is correct, it is perhaps not surprising that an accurate part of speech label would prove more useful in a conditioning context used by the parser than a noisy prosodic feature. Note that this would not be the case when parsing from speech recognizer output (since word identity would itself be uncertain), and it is possible that in such applications prosodic information would be more useful.</Paragraph>
    <Paragraph position="5"> Of course, there are many other ways prosodic information might be exploited in a parser, and one of those may yield improved parser performance.</Paragraph>
    <Paragraph position="6"> We chose to incorporate prosodic information into our parser in a way that was similar to the way that punctuation is annotated in the Penn treebanks because we assumed that punctuation carries information similar to prosody, and it had already been demonstrated that punctuation annotated in the Penn treebank fashion does systematically improve parsing accuracy.</Paragraph>
    <Paragraph position="7"> But the assumption that prosody conveys information about syntactic structure in the same way that punctuation does could be false. It could also be that even though prosody encodes information about syntactic structure, this information is encoded in a manner that is too complicated for our parser to utilize. For example, even though commas are often used to indicate pauses, pauses have many other functions in uent speech. Pauses of greater than 200 ms are often associated with planning problems, which might be correlated with syntactic structure in ways too complex for the parser to exploit. While not reported here, we tried various techniques to isolate different functions of pauses, such as excluding pauses of greater than 200 ms. However, all of these experiments produced results similar to those reported here.</Paragraph>
    <Paragraph position="8"> Finally, there is another possible reason why our assumption that prosody and punctuation are similar in their information content could be wrong. Our prosodic information was automatically extracted from the speech stream, while punctuation was produced by human annotators who presumably comprehended the utterances being annotated. Given this, it is perhaps no surprise that our automatically extracted prosodic annotations proved less useful than human-produced punctuation.</Paragraph>
  </Section>
class="xml-element"></Paper>