<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-1034">
  <Title>Modelling User Satisfaction and Student Learning in a Spoken Dialogue Tutoring System with Generic, Tutoring, and User Affect Parameters</Title>
  <Section position="3" start_page="264" end_page="265" type="intro">
    <SectionTitle>
2 Spoken Dialogue Tutoring Corpora
ITSPOKE (Intelligent Tutoring SPOKEn dialogue
</SectionTitle>
    <Paragraph position="0"> system) (Litman et al., 2006) is a speech-enabled tutor built on top of the text-based Why2-Atlas conceptual physics tutor (VanLehn et al., 2002). In ITSPOKE, a student rst types an essay into a web-based interface answering a qualitative physics problem. ITSPOKE then analyzes the essay and engages the student in spoken dialogue to correct misconceptions and elicit more complete explanations.</Paragraph>
    <Paragraph position="1"> Student speech is digitized from the microphone input and sent to the Sphinx2 recognizer. Sphinx2's most probable transcription is then sent to Why2-Atlas for syntactic, semantic and dialogue analysis. Finally, the text response produced by Why2-Atlas is converted to speech as described below, then played in the student's headphones and displayed on the interface. After the dialogue, the student revises the essay, thereby ending the tutoring or causing another round of tutoring/essay revision.</Paragraph>
    <Paragraph position="2"> For this study, we used 3 ITSPOKE corpora, shown in Table 1.1 The SYN03 corpus was collected in 2003 for an evaluation comparing learning in typed and spoken human and computer tutoring (Litman et al., 2006). ITSPOKE's voice was synthesized with the Cepstral text-to-speech system, and its speech recognizer was trained from pilot ITSPOKE studies and Why2-Atlas evaluations. The PR05 and SYN05 corpora were collected in 2005, to evaluate the impact of tutor voice quality (Forbes-Riley et al., 2006). For these 2 corpora, ITSPOKE used an updated speech recognizer further trained on the SYN03 corpus. For the SYN05 corpus, ITSPOKE used the synthesized tutor voice from the SYN03 corpus; for the PR05 corpus, ITSPOKE used a pre-recorded tutor voice from a paid voice talent. Figure 1 gives an annotated (Section 3) PR05 excerpt (ASR shows what ITSPOKE heard).</Paragraph>
    <Paragraph position="3"> The same experimental procedure was used to collect all 3 ITSPOKE corpora: college students who had taken no college physics: 1) read a small document of background material, 2) took a pretest measuring initial physics knowledge, 3)  Subjects in the PR05 and SYN05 corpora also completed a survey probing user satisfaction after taking the posttest (SYN03 corpus subjects did not).</Paragraph>
    <Paragraph position="4"> Our survey, shown in Figure 2, is essentially the same as the one used in the DARPA Communicator multi-site evaluation (Walker et al., 2002). Although tailored lexically for a tutoring system, these statements are generally applicable to spoken dialogue systems. Students rated their degree of agreement with each statement on a scale of 1 to 5.</Paragraph>
    <Paragraph position="5"> PHYSICS PROBLEM: Suppose a lightweight car and a massive truck hit a patch of frictionless ice and have a head-on collision. Upon which vehicle is the impact force greater? Which vehicle undergoes the greater change in its motion? Defend your answers.</Paragraph>
    <Paragraph position="6"> ESSAY UNDER DISCUSSION: The truck exerts an equal and opposite reaction force on the car. The car has a greater acceleration. The car will have a greater change in its motion. . . . excerpt at 28.8 minutes into dialogue. . .</Paragraph>
    <Paragraph position="7"> TUTOR60: So, when you add forces together, besides the magnitudes of the forces, what must you also consider? STUDENT57: direct- direction (ASR: the truck in direction) [correct, uncertain, non-frustrated/angry] TUTOR61: Yes. [...] The vector sum of the forces acting on the body is also equal to the mass of the body multiplied by its acceleration. What is another term for the vector sum of forces acting on a body? STUDENT58: net force (ASR: net force) [correct, neutral, non-frustrated/angry] TUTOR62: That's super. Another term for the resultant force is the net force. In this relationship, what is the direction of the net force applied to a body and the direction of the body's acceleration? STUDENT59: same same same (ASR: seems same type seems) [correct, certain, frustrated/angry]  Task Ease: It was easy to learn from the tutor.</Paragraph>
    <Paragraph position="8"> TTS Performance: It was easy to understand the tutor.</Paragraph>
    <Paragraph position="9"> User Expertise: I knew what I could say or do at each point in  the conversations with the tutor.</Paragraph>
    <Paragraph position="10"> Expected Behavior: The tutor worked the way I expected it to. Future Use: Based on my experience using the tutor to learn physics, I would like to use such a tutor regularly.</Paragraph>
  </Section>
class="xml-element"></Paper>