<?xml version="1.0" standalone="yes"?>
<Paper uid="C86-1085">
  <Title>Combining Deictic Gestures and Natural Language for Referent Identification</Title>
  <Section position="3" start_page="360" end_page="360" type="intro">
    <SectionTitle>
4. Discussion
</SectionTitle>
    <Paragraph position="0"> Our system demonstrates that spatial deixis is a valuable source of information for identifying referents which also can be investigated and utilized in natural language dialog systems with pictoral display. Three reasons sum up the advantages of using pointing gestures: They save the speaker the generation, and the hearer the analysis of complex referential descriptions and thus simplify the natural-language dialog; they often allow for reference in situations in which linguistic reference is simply not possible (think of referring to one out of a dozen similar objects); and they permit the speaker to be vague, imprecise, or ambiguous, and to use everyday terms instead of precise technical terms unknown to him/her.</Paragraph>
    <Paragraph position="1"> In natural-language dialog systems, deixis analysis can be combined well with standard methods for referent identification.</Paragraph>
    <Paragraph position="2"> Sonre of the identification processes (e.g., tests with case frame, descriptor and dialog menmry) are rather similar to the classical methods used ibr anaphora and ellipsis resolntion. Others, such as the generation and evaluation of candidates by the deixis analyzer, are typical with respect to this particular kind of conversational medium.</Paragraph>
    <Paragraph position="3"> It should be pointed out, however, that out' environment for spatial deixis is, in several ways, somewhat simpler than those occurring in person-to-person dialogs (cf. Schmauks 1986).</Paragraph>
    <Paragraph position="4"> The deictic fieM is only two.dimensional, and the objects that carl be pointed at are clem'ly separated from each other. Compared to real-life situations, the number of possible referents is relatively small. &amp;quot;Left&amp;quot; and &amp;quot;right&amp;quot; xrman the same thing for the user and the system (which is not the case, e.g., in face-tolace conversation), iIowever, this relative simplicity neeci not be a rh'awback. Instead, one might regard our environment as a study in vitro, eliminating a number of uncertainty t~tctors so that tile essential characteristics of spatial deixis become more salient.</Paragraph>
    <Paragraph position="5"> Another question is whether the deictic behavior ofsul~jects who use a poiming device is the same as that of subjects who touch the display with their fingers (and thus, whether deixis via a pointing device is a valid sinmlation of tactile deixis). One might argue, e.g., that people point more precisely with a mouse than with their lingers, or vice versa. We are currently conducting an inibnnal experiment to answer these questions. In any case, only the propagation functions are perhaps all~:cted t0y a change of tile deictic medium, whcreas the referent identiticalion processes will remain tile same.</Paragraph>
    <Paragraph position="6"> Attempts are currently being made to also integrate visual ancl conceptual salience in our model (cf. Clark et al. 1983). When a pointing gesture is ambiguotxs, it appears that regions set off by boM fi'ame or coloring, as well as regions containing important data tbr tile task domain are preferred. We expect this preference to be laken into account in the evahmtion processes of tile deixis analyzer. Another possible extension which wc wouM like to invesdgatc is in replacing the strategy described in section 3.1.1. by a certain form of incremental referent identification.</Paragraph>
    <Paragraph position="7"> There is strong empirical evidence (e.g. Goodman 1985) that people begin with referent identification immediately alter receiving initial information about it, instead of waiting tmtil the speaker's reti~rential act is terminated. Since all components described above are strictly separated, it appears basically possible to also use them in an incrmnental identification process. In one-processor systems, however, great care must be taken that the knowledge source first adressed does not block the system by generating too many candidates. Therefore, some process controlling will be necessary, either by ressource limitation or by taking into account the heuristics listed in section 3.1.1.</Paragraph>
  </Section>
class="xml-element"></Paper>