<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1012">
  <Title>User Expertise Modelling and Adaptivity in a Speech-based E-mail System</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Adaptive functionality in spoken dialogue systems is usually geared towards dealing with communication disfluencies and facilitating more natural interaction (e.g. Danieli and Gerbino, 1995; Litman and Pan, 1999; Krahmer et al, 1999; Walker et al, 2000). In the AthosMail system (Turunen et al., 2004), the focus has been on adaptivity that addresses the user's expertise levels with respect to a dialogue system's functionality, and allows adaptation to take place both online and between the sessions.</Paragraph>
    <Paragraph position="1"> The main idea is that while novice users need guidance, it would be inefficient and annoying for experienced users to be forced to listen to the same instructions every time they use the system. For instance, already (Smith, 1993) observed that it is safer for beginners to be closely guided by the system, while experienced users like to take the initiative which results in more efficient dialogues in terms of decreased average completion time and a decreased average number of utterances.</Paragraph>
    <Paragraph position="2"> However, being able to decide when to switch from guiding a novice to facilitating an expert requires the system to be able to keep track of the user's expertise level. Depending on the system, the migration from one end of the expertise scale to the other may take anything from one session to an extended period of time.</Paragraph>
    <Paragraph position="3"> In some systems (e.g. Chu-Carroll, 2000), user inexperience is countered with initiative shifts towards the system, so that in the extreme case, the system leads the user from one task state to the next. This is a natural direction if the application includes tasks that can be pictured as a sequence of choices, like choosing turns from a road map when navigating towards a particular place. Examples of such a task structure include travel reservation systems, where the requested information can be given when all the relevant parameters have been collected. If, on the other hand, the task structure is flat, system initiative may not be very useful, since nothing is gained by leading the user along paths that are only one or two steps long.</Paragraph>
    <Paragraph position="4"> Yankelovich (1996) points out that speech applications are like command line interfaces: the available commands and the limitations of the system are not readily visible, which presents an additional burden to the user trying to familiarize herself with the system. There are essentially four ways the user can learn to use a system: 1) by unaided trial and error, 2) by having a pre-use tutorial, 3) by trying to use the system and then asking for help when in trouble, or 4) by relying on advice the system gives when concluding the user is in trouble. Kamm, Litman &amp; Walker (1998) experimented with a pre-session tutorial for a spoken dialogue e-mail system and found it efficient in teaching the users what they can do; apparently this approach could be enhanced by adding items 3 and 4. However, users often lack enthusiasm towards tutorials and want to proceed straight to using the system.</Paragraph>
    <Paragraph position="5"> Yankelovich (1996) regards the system prompt design at the heart of the effective interface design which helps users to produce well-formed spoken input and simultaneously to become familiar with the functionality that is available. She introduced various prompt design techniques, e.g. tapering which means that the system shortens the prompts for users as they gain experience with the system, and incremental prompts, which means that when a prompt is met with silence (or a timeout occurs in a graphical interface), the repeated prompt will be incorporated with helpful hints or instructions. The system utterances are thus adapted online to mirror the perceived user expertise.</Paragraph>
    <Paragraph position="6"> The user model that keeps track of the perceived user expertise may be session-specific, but it could also store the information between sessions, depending on the application. A call service providing bus timetables may harmlessly assume that the user is always new to the system, but an e-mail system is personal and the user could presumably benefit from personalized adaptations.</Paragraph>
    <Paragraph position="7"> If the system stores user modelling information between sessions, there are two paths for adaptation: the adaptations take place between sessions on the basis of observations made during earlier sessions, or the system adapts online and the resulting parameters are then passed from one session to another by means of the user model information storage. A combination of the two is also possible, and this is the chosen path for AthosMail as disclosed in section 3.</Paragraph>
    <Paragraph position="8"> User expertise has long been the subject of user modelling in the related fields of text generation, question answering and tutorial systems. For example, Paris (1988) describes methods for taking the user's expertise level into account when designing how to tailor descriptions to the novice and expert users. Although the applications are somewhat different, we expect a fair amount of further inspiration to be forthcoming from this direction also.</Paragraph>
    <Paragraph position="9"> In this paper, we describe the AthosMail user expertise model, the Cooperativity Model, and discuss its effect on the system behaviour. The paper is organised as follows. In Section 2 we will first briefly introduce the AthosMail functionality which the user needs to familiarise herself with.</Paragraph>
    <Paragraph position="10"> Section 3 describes the user expertise model in more detail. We define the three expertise levels and the concept of DASEX (dialogue act specific explicitness), and present the parameters that are used to calculate the online, session-specific DASEX values as well as offline, between-thesessions DASEX values. We also list some of the system responses that correspond to the system's assumptions about the user expertise. In Section 4, we report on the evaluation of the system's adaptive responses and user errors. In Section 5, we provide conclusions and future work.</Paragraph>
  </Section>
class="xml-element"></Paper>