<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-1608">
  <Title>Incremental Generation of Multimodal Deixis Referring to Objects</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Pointing in Multimodal Deictic Expressions
</SectionTitle>
    <Paragraph position="0"> There is little doubt in the literature that pointing is tied up with reference as the following quotation from [Lyons, 1977, p. 654] shows: When we identify an object by pointing to it (and this notion, as we have seen, underlies the term 'deixis' and Peirce's term 'index': cf. 15.1), we do so by drawing the attention of the addressee to some spatio-temporal region in which the object is located.</Paragraph>
    <Paragraph position="1"> Pointing, then, is related to objects indicated and regions occupied. Lyons also emphasises that certain kinds of expressions, especially definite descriptions, are closely linked to pointing or demonstration (op. cit., p. 657): [. . . ] definite referring noun-phrases, as they have been analysed in this section, always contain a deictic element. It follows that reference by means of definite descriptions depends ultimately upon deixis, just as much as does reference by means of demonstratives and (as we saw in the previous section) personal pronouns.</Paragraph>
    <Paragraph position="2"> Pointing and definite descriptions therefore represent on the one hand different kinds of referring to objects (indicating their location or describing their properties). On the other hand they appear to be intimately connected. Lyons does not discuss how exactly pointing and verbal expression are related. Following [Rieser, 2004], we pursue a line of thought associated with Peirce, who maintains the idea of gestures being part of more complex signs [Peirce, 1965]. Transferring that to deixis we call such complex signs, which are composed of a pointing gesture and a definite description, complex demonstrations. In other words, complex demonstrations are definite descriptions to which pointings add content, either by specifying an object independently of the definite description (Lyons' attention being drawn to some object) or by narrowing down the description's restrictor (Lyons' spatio-temporal region). Below, we refer to these two possibilities as the respective functions of demonstration, see [Rieser, 2004] for discussion. If a pointing gesture uniquely singles out an object, it is said to have object-pointing function. If the gesture draws the attention of the addressee to a region making the objects inside it salient it is ascribed a region-pointing function.</Paragraph>
    <Paragraph position="3"> The distinction between object-pointing and region-pointing is closely connected with the observation that pointing gestures are inherently ambiguous, varying with the distance between pointing agent and referent. In the empirical data collected in our demonstration games we found object-pointing only in demonstrations to objects near to the demonstrating subject, while pointings to objects farther are accompanied by definite descriptions [L&amp;quot;ucking et al., 2004]. Two phenomena can be recognised (even though they are blurred by over-specification which we observe very often in complex demonstrations). First, pointing saves words; definite descriptions accompanied by a pointing gesture are shorter and less complex than definite descriptions without gesture.</Paragraph>
    <Paragraph position="4"> Secondly, length and complexity of the definite description in complex demonstrations depend on the distance between demonstrating subject and referent pointed to. Similar results can be found in literature, e.g. [Beun and Cremers, 2001; van der Sluis and Krahmer, 2004].</Paragraph>
    <Paragraph position="5"> These results indicate that the discriminative power of pointing gestures influences the construction of definite descriptions and that in order to determine the set of entities delimited by a pointing gesture the distance to the referent has to be accounted for. As a first approximation we model the topology of the region singled out by a pointing gesture as a cone anchored at the index finger tip and directed along the vector defined by the stretched index finger.</Paragraph>
    <Paragraph position="6"> It has to be stressed, however, that a cone is an idealisation of the pointing region. There are a lot of influencing parameters, which we can divide in perceivable parameters on the one hand (like spatial configuration of demonstrating agent, addressee, and referents as well as the clustering of the entities under demonstration) and dialogue parameters on the other hand. Determining the pointing cone in more detail is the issue of further empirical investigations currently undertaken. The concept of pointing cone we use is based on a set of parameters which guarantees that the cone's form and size can be adjusted as further findings become available.</Paragraph>
    <Paragraph position="7"> Observations we made in our corpus suggest that we have to acknowledge that each of the two referential functions of pointing, i.e. object-pointing and region-pointing, comes with a cone on its own. Therefore, the concept of pointing cone can be divided into two topologically different types for object- and for region-pointing respectively, with the former having a narrower angle than the latter. The cone of object-pointing represents the resolution of a pointing gesture visually perceivable to the dialogue participants, and therefore, defines the borderline up to which object-pointing can be conducted successfully. Preliminary findings [K&amp;quot;uhnlein and Stegmann, 2003] indicate an apex angle of this cone of about 12 to 24 degrees. In contrast, region-pointing draws the attention of the addressee to a wider region making the objects inside this region salient. The cone representing this region has to be modelled with a wider apex angle than the cone for object-pointing to ensure robust reference and to fit empirical findings concerning over-specification.</Paragraph>
  </Section>
class="xml-element"></Paper>