<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-4015">
  <Title>First.Last@dfki.de</Title>
  <Section position="8" start_page="59" end_page="59" type="concl">
    <SectionTitle>
6 Experiments and Evaluation
</SectionTitle>
    <Paragraph position="0"> So far we conducted two WOZ data collection experiments and one evaluation experiment with a baseline version of the SAMMIE system. The SAMMIE-1 WOZ experiment involved only spoken interaction, SAMMIE-2 was multimodal, with speech and haptic input, and the subjects had to perform a primary driving task using a Lane Change simulator (Mattes, 2003) in a half of their experiment session. The wizard was simulating an MP3 player application with access to a large database of information (but not actual music) of more than 150,000 music albums (almost 1 million songs). In order to collect data with a variety of interaction strategies, we used multiple wizards and gave them freedom to decide about their response and its realization. In the multimodal setup in SAMMIE-2, the wizards could also freely decide between mono-modal and multimodal output.</Paragraph>
    <Paragraph position="1"> (See (Kruijff-Korbayov'a et al., 2005) for details.) We have just completed a user evaluation to explore the user-acceptance, usability, and performance of the baseline implementation of the  SAMMIE multimodal dialogue system. The users were asked to perform tasks which tested the system functionality. The evaluation analyzed the user's interaction with the baseline system and combined objective measurements like task completion (89%) and subjective ratings from the test subjects (80% positive).</Paragraph>
    <Paragraph position="2"> Acknowledgments This work has been carried out in the TALK project, funded by the EU 6th Framework Program, project No. IST-507802.</Paragraph>
  </Section>
class="xml-element"></Paper>