<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-2326">
  <Title>Annotating Student Emotional States in Spoken Tutoring Dialogues</Title>
  <Section position="6" start_page="0" end_page="0" type="concl">
    <SectionTitle>
7 Conclusions and Current Directions
</SectionTitle>
    <Paragraph position="0"> In this paper we presented and analyzed our scheme for annotating student emotional states in spoken tutoring dialogues. Our scheme distinguishes three main (negative, neutral and positive) and three minor (weak negative, mixed, and weak positive) emotion classes.</Paragraph>
    <Paragraph position="1"> Our inter-annotator agreement is on par with prior emotion annotation in other types of corpora. We used consensus-labeling to resolve disagreements and increase our dataset. Through further annotation and the use of other inter-annotation metrics (Gwet, 2001), we will investigate how systematic disagreements can yield revisions to our annotation scheme that improve reliability. Our machine learning experiments have shown that our main emotion categories can be predicted with a high degree of accuracy. Although not presented here, F-Measures (a31 a32a34a33a36a35a38a37a40a39a36a41a34a42a43a42a25a33a23a44a46a45a16a37a40a39a6a47a49a48a40a47a51a50a6a52a35a38a37a6a39a36a41a34a42a43a42a25a53a54a44a46a45a34a37a6a39a6a47a51a48a6a47a51a50a6a52 ) for our experiments on agreed data ranged from 67%-86%; in future work we will more closely examine the tradeoff between recall and precision when predicting our annotations. Our experiments have also highlighted tradeoffs that can be made between coding reliability, predictive accuracy, and annotation scheme granularity.</Paragraph>
    <Paragraph position="2"> Finally, we presented initial results in annotating our ITSPOKE human-computer tutoring corpus, and discussed differences from our human-human annotations.</Paragraph>
    <Paragraph position="3"> This research on emotion annotation and prediction is a rst step towards extending the ITSPOKE computer tutoring dialogue system to predict and adapt to student emotional states. Our next goal is to label human tutor reactions to emotional student turns, in order to formulate adaptive strategies for ITSPOKE, and to determine which of our six prediction tasks best triggers adaptation.</Paragraph>
  </Section>
class="xml-element"></Paper>