<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-0204">
  <Title>PLASER: Pronunciation Learning via Automatic Speech Recognition</Title>
  <Section position="7" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
6 Evaluation
</SectionTitle>
    <Paragraph position="0"> A beta version of PLASER was tested by 900 students of Grade Seven and Eight over a period of about 3 months in twelve high schools. Both quantitative and qualitative evaluations were conducted to gauge the effectiveness of using PLASER to learn English pronunciation.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
6.1 Quantitative Evaluation
</SectionTitle>
      <Paragraph position="0"> A pronunciation test consisting of speaking 60 words was conducted once before a student even started to use PLASER and once after they finished the 3-month PLASER trial. The recordings from 210 students were successfully collected. Recordings were not obtained from the rest of students for various reasons: + Some schools did not have time to do the evaluation recordings due to schedule problems.</Paragraph>
      <Paragraph position="1"> + Some recordings were poorly administered; e.g.</Paragraph>
      <Paragraph position="2"> parts of utterances were missing in the files.</Paragraph>
      <Paragraph position="3"> + Some schools accidently erased or lost the recorded speech files in their computers.</Paragraph>
      <Paragraph position="4"> At the end, recordings from 210 students were found to be good enough for evaluation. Their recordings were transcribed and compared with the correct transcriptions to find their pronunciation accuracies. The two histograms in Fig. 3 summarize their pronunciation accuracies of the 60 words before and after they practiced with PLASER. Here are some detailed statistics: + 73% of the students had their pronunciation accuracy improved by an average of (absolute) 4.53%.</Paragraph>
      <Paragraph position="5"> + The remaining 27% of the students got worse for unknown reasons by an average of 2.68%.</Paragraph>
      <Paragraph position="6"> + Collectively we observe an obvious improvement: the mean accuracy after the use of PLASER is greater than that before using PLASER, and the mean difference is statistically significant at the 99% confidence level.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
6.2 Qualitative Evaluation
</SectionTitle>
      <Paragraph position="0"> In addition, a questionnaire survey was conducted to get comments and suggestions from teachers and students after they finished the study. Some figures are worth mentioning: null + 77% of the students believed that their pronunciation skill was improved after using PLASER while 91% of school teachers believed their students' pronunciation had improved.</Paragraph>
      <Paragraph position="1"> + 77% of the students like to use PLASER to learn English pronunciation.</Paragraph>
      <Paragraph position="2"> + 53% of the students preferred using PLASER to the traditional classroom teaching method to learn English pronunciation while 73% of the teachers would prefer their students using PLASER to self-learn pronunciation.</Paragraph>
      <Paragraph position="3">  + All teachers would recommend their students to use PLASER to learn English pronunciation.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>