<?xml version="1.0" standalone="yes"?>
<Paper uid="C88-2122">
  <Title>Generating Multimodal Output - Conditions, Advantages and Problems</Title>
  <Section position="3" start_page="0" end_page="584" type="intro">
    <SectionTitle>
1. Introduction
</SectionTitle>
    <Paragraph position="0"> In face-to-face communication, speech and communicative body movements are performed simultaneously. A prime example ot this mu/lirnoda//ly are detctic actions which specify elements of a shared visual world by the combination of deictic expressions ('this', 'there' etc.) and extralinguistic devices like pointing gestures. The advantages of this mu#imoda/ de/xis motivate the integration of extralinguistic means for referent specification into natural language (NL) dialog systems. Starting point of the following considerations is the system XTRA, an NL access system for expert systems, which is under development at the University of SaarbrLicken. In its current application domain, it assists the user in filling out a tax form which is visible on the screen. Elements of this form can be specified not only by (typed) verbal descriptions, but also by combining descriptions and simulated pointing gestures. Some problems of rnu#imoda/input and solutions in XTRA have already been treated in detail (cf. /AIIgayer, Reddig 86/, /AIIgayer et al. 88/,/Schmauks 86a, 87/).</Paragraph>
    <Paragraph position="1"> Mu#/moda/ou/put is no simple mirror image of multimodal input.</Paragraph>
    <Paragraph position="2"> Rather, it has to deal with different problems the investigation of which has been missing till now (for a first impression see/Reithinger 87a/).</Paragraph>
    <Paragraph position="3"> Because of the novelty of the task, one cannot claim to offer ultimate solutions. Instead, we wish to outline several approaches for the realization of multimodality, present our strategy and give reasons for the choice.</Paragraph>
    <Paragraph position="4"> In section 2, we present the means, conditions and advantages of multimodal deixis within natural communication situations. Topics of section 3 are the different strategies for realizing multimodality in NL dialog systems and some of the problems arising. Section 4 sketches the framework of the XTRA system and the types of gestures occuring in this domain. Section 5 presents the generation component POPEU, focussing on its global strategy for generating multimodal output.</Paragraph>
    <Paragraph position="5"> Subtopics are POPEL's architecture and its methods for simulating different types of pointing gestures. In section 6, some alternative strategies for generating multimodal output are briefly discussed.</Paragraph>
  </Section>
class="xml-element"></Paper>