<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2125">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics An HMM-Based Approach to Automatic Phrasing for Mandarin Textto-Speech Synthesis</Title>
  <Section position="8" start_page="979" end_page="980" type="evalu">
    <SectionTitle>
4 Experimental Results
</SectionTitle>
    <Paragraph position="0"> Before reporting the experimental results, we first define the criterion of evaluation and the</Paragraph>
    <Section position="1" start_page="980" end_page="980" type="sub_section">
      <SectionTitle>
4.1 The evaluation method
</SectionTitle>
      <Paragraph position="0"> After analyzing the existing evaluation methods, we feel that the method proposed in (Taylor and Black, 1998) is appropriate for our application.</Paragraph>
      <Paragraph position="1"> By employing this method, we can examine each word pair in the test set. If the algorithm generated break fully matches the manually labeled break, it marks correct. Similarly, if there is no labeled break and the algorithm does not place a break, it also marks correct. Otherwise, an error arises. To emphasize the effectiveness of break prediction, we define the adjusted score, Sa, as follows.</Paragraph>
      <Paragraph position="2">  where S is the ratio of the number of correct word pairs to the total number of word pairs; B is the ratio of non-breaks to the number of word-pairs.</Paragraph>
    </Section>
    <Section position="2" start_page="980" end_page="980" type="sub_section">
      <SectionTitle>
4.2 The test corpora
</SectionTitle>
      <Paragraph position="0"> From the perspective of perception, multiple predictions of prosodic phrasing may be acceptable in many cases. At the labeling stage, three experts (E1, E2, E3) were requested to label 1,174 sentences independently. Experts first read the sentences silently. Then, they marked the breaks in sentences independently.</Paragraph>
      <Paragraph position="1"> Table 1 and 2 show their labeling differences in terms of S and Sa, respectively.</Paragraph>
      <Paragraph position="2"> Table 1 indicates that any two can achieve a consistency of roughly 87% among three experts.</Paragraph>
    </Section>
    <Section position="3" start_page="980" end_page="980" type="sub_section">
      <SectionTitle>
4.3 The results
</SectionTitle>
      <Paragraph position="0"> To evaluate the approaches mentioned above, we conducted a series of experiments. In all our experiments, we assume that no breaking is necessary for those sentences that are shorter than the average phrase length and remove them in the statistic computation. For the approaches based on HMM path, we further define that the initial and final words of a sentence can only assume two state values, namely, (phrase initial, separate) and (phrase final, separate), respectively. With this definition, we modify the approach HMM-Path to HMM-Path-I.</Paragraph>
      <Paragraph position="1"> Alternatively, to investigate acceptance, we also calculate the matching score between the approaches and any expert (We assume the prediction is acceptable if the predicted phrase sequence matches any of three phrase sequences labeled by the experts). By employing the preceding criterion, we achieve the results as shown in Table 3 and 4.</Paragraph>
      <Paragraph position="2"> A sentence consumes less than 0.3 ms on average for all the evaluated methods. So they are all computationally efficient. Alternatively, we compared the HMM-based approach base on word format and some POS-based ones on the same training set and test set. Overall, HMMpath-I can achieve high accuracy by about 10%.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>