<?xml version="1.0" standalone="yes"?>
<Paper uid="E06-1045">
  <Title>Data-driven Generation of Emphatic Facial Displays</Title>
  <Section position="3" start_page="353" end_page="353" type="intro">
    <SectionTitle>
2 Choosing Non-Verbal Behaviour for
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="353" end_page="353" type="sub_section">
      <SectionTitle>
Embodied Conversational Agents
</SectionTitle>
      <Paragraph position="0"> Embodied Conversational Agents (ECAs) are computer interfaces that are represented as human bodies, and that use their face and body in a human-like way in conversations with the user (Cassell et al., 2000). The main benefit of ECAs is that they allow users to interact with a computer in the most natural possible setting: face-to-face conversation. However, to realise this advantage fully, the agent must produce high-quality output, both verbal and non-verbal. A number of previous systems have based the choice of non-verbal behaviours for an ECA on the behaviours of humans in conversational situations. The implementations vary as to how directly they use the human data.</Paragraph>
      <Paragraph position="1"> In some systems, motion specifications for the agent are created from scratch, using rules derived from studying human behaviour. For the REA agent (Cassell et al., 2001a), for example, gesturing behaviour was selected to perform particular communicative functions, using rules based on studies of typical North American non-verbal displays. Similarly, the Greta agent (de Carolis et al., 2002) selected its performative facial displays using hand-crafted rules to map from affective states to facial motions. Such implementations do not make direct use of any recorded human motions; this means that they generate average behaviours from a range of people, but it is difficult to adapt them to reproduce the behaviour of an individual.</Paragraph>
      <Paragraph position="2"> In contrast, other ECA implementations have selected non-verbal behaviour based directly on motion-capture recordings of humans. Stone et al.</Paragraph>
      <Paragraph position="3"> (2004), for example, recorded an actor performing scripted output in the domain of the target system.</Paragraph>
      <Paragraph position="4"> They then segmented the recordings into coherent phrases and annotated them with the relevant semantic and pragmatic information, and combined the segments at run-time to produce complete performance specifications that were then played back on the agent. Cunningham et al.</Paragraph>
      <Paragraph position="5"> (2004) and Shimodaira et al. (2005) used similar techniques to base the appearance and motions of their talking heads directly on recordings of human faces. This technique is able to produce more naturalistic output than the more rule-based systems described above; however, capturing the motion requires specialised hardware, and the agent must be implemented in such a way that it can exactly reproduce the human motions.</Paragraph>
      <Paragraph position="6"> A middle ground is to use a purely synthetic agent--one whose behaviour is controlled by high-level instructions, rather than based directly on human motions--but to create the instructions  forthatagentusingthedatafromanannotatedcorpus of human behaviour. Like a motion-capture implementation, this technique can also produce increased naturalism in the output and also allows choices to be based on the motions of a single performer if necessary. However, annotating a video corpus can be less technically demanding than capturing and directly re-using real motions, especially when the corpus and the number of features under consideration are small. This approach has been taken, for example, by Cassell et al. (2001b) to choose posture shifts for REA, and by Kipp (2004) to select gestures for agents, and it is also the approach that we adopt here.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>