<?xml version="1.0" standalone="yes"?>
<Paper uid="C00-1083">
  <Title>Taking Account of the User's View in 3D Multimodal Instruction Dialogue</Title>
  <Section position="3" start_page="572" end_page="573" type="intro">
    <SectionTitle>
2 Problems
</SectionTitle>
    <Paragraph position="0"> In a virtual emdromnent, the user can freely move a.round the world and select her/his own view. r\['he systelll C&amp;llllOt; predict where the user will stand and what; s/he observes in the virtual environment. This section describes two types of 1)roblems in generating instru(:tion dialogues ibr such virtual enviromnents. They arc caused l)y mismatches b(~,twe(;ll tile llSel'~S vi0,wl)oint ;m(1 the sta.te of th(; dialogue.</Paragraph>
    <Paragraph position="1"> First, the syStelll shouM check whether the user's view matches the tbcns of the next exchange when the systen~ tries to ('hange COllllllllni('ative goals. \]if a mismatch occurs, the system shouhl choose the instru(:tion (li~dogue content according to the user's view. Figure 1 a,n(1 2 m:e examl)les of observing a car's front suspension from (liff(',r(mt, points of view. In Figm'(', 1, the right; side of the steering system can 1)e seen, while Figure 2 shows the left side. If the system is not aware of the user's view, I;he system may talk about the left; tie rod end even though the user's view remains the right side (Figure 1).</Paragraph>
    <Paragraph position="2"> In such n (:ase, the system shouM chang(: its d(&gt; scril)tion or ask the user to change her/his view to |;11('. left; side. view (Figure 2) and r('.(-Olmnen(:e its instruction hi)out this part. Therefore, the system should be al)le to change the content of the dialogue according 1;o the user's view.</Paragraph>
    <Paragraph position="3"> In order to ac(:omplish this, the system shoul(1 lmve ;1. content selection nlechan.ism whi(:h incrementally (let:ides i;h('~ content while ('he(:king the llSef~s (;llrrellt vi(!w.</Paragraph>
    <Paragraph position="4"> Second, t;here could 1)e a case in which 1;21(; user chang(~s 1;he, topi(: as well as the vie\vl)oillt as interrupl;ing the. system's instru('t;ion, i n such a case, the (tia.h)gue~ system shouhl kee l) track of the user's viewpoint as ~ 1)art of the dialogue state nnd return to that viewpoint when resmning the (lia.logu(? after the interrupl;ing sul)(lialogue. Sul)l)ose that while the sys|;em is (',xl)lnining tlm right; t)i(; rod end, th('. user initially looks a,t the right side, (l&amp;quot;igure 1) hut then shifts her/his view to the left (Figure 2) and asks about the \]eft knu(-kle arm. After finishing a sub(lialogue about this arm, the syst(;nl tries to return to the dialogue al)out the interrupted topic. At this time, if the sysl;em resumed the dialogue using the current view (Figure 2), the view and the instruction would \])e(;olne mismatched. When resmning the interrupted dialogue, it would be less (:onfllsing to the user if the system retm:ned to the user's prior viewl)oint rather than selecting n new o11o. '\].'he user may be (:onfilsed if the dialogue is resulned but the observed state looks different.</Paragraph>
    <Paragraph position="5"> \,Ve address the ~fl)ove problems. In order to (:ope wit;h the first; problem, we present a content selection mechanism that incrementally expands the content plan of a multimodal dialogue while checking the user's view. To solve the second 1)roblem, we present a. dialogue nmnagemerit me(:\]mnism l;hat keel)s t:ra(-k of the user's viewpoint as a part of the diah)gue context and  uses this intbrmation in resuming the dialogue after interruptive subdialogues.</Paragraph>
  </Section>
class="xml-element"></Paper>