<?xml version="1.0" standalone="yes"?>
<Paper uid="A92-1008">
  <Title>Generating Spatial Descriptions for Cross-modal References</Title>
  <Section position="3" start_page="56" end_page="57" type="intro">
    <SectionTitle>
2 Object Localisation
</SectionTitle>
    <Paragraph position="0"> A lot of work has been done on 'object localisation' and its linguistic complelnent, 'spatial l)repositions'.</Paragraph>
    <Paragraph position="1"> Wunderlich/Herweg (\[Wunderlich, 1982\], \[Wunderlich and Herweg, forthcoming\]) and Herskovits (\[Herskovits, 1985\]) provide linguistic approaches to the semantics of spatial prepositions. NL-systems like NAOS (\[Neumann and Novak, 1986\]), HAM-RPM (\[Hahn el al., 1980\]), SWYSS (\[HuBmann and Schefe, 1984\]) and C'ITYTOUR (\[Andr~ et al., 1985\],\[Andr~ et al., 1986\]) address various issues regarding computational aspects. Schirra (\[Schirra, to appear 1992\]) and Habel/Pribbenow (\[Hal)el and Pribbenow, 1988\],\[Pribbenow, 1990\]) also incorporate relevant work from cognitive psychology.</Paragraph>
    <Paragraph position="2"> In our approach, we concentrate on the requirements for localising objects ill pictures. We assume that the user can see the picture containing the objects to be localised and we do not deal with the problem of anticipating possibly wrong visualisations of the user in the case he/she cannot see the picture. We do not deal with possible intrinsic orientations of depicted objects (c.f. \[Retz-Schlnidt, 1988\]) and assume the deictic reference frame of a common viewer (c.f. figure 5). Together with every localisation, we compute a so-called applicability degree from the intervall \[0..1\]. The applicability degree is not only used to generate linguistic hedges (c.f.</Paragraph>
    <Paragraph position="3"> \[Lakoff, 1972\]) as in SWYSS or CITYTOUR, but also for selecting the 'best' localisation from a set of alternatives. The localisations computed on our system are two-dimensional localisations in the sense that they are based on the 2D-projection of a picture aim not on its possible 3D-representation. In the rest of this section we will describe the localisation phenomena we take into account and introduce our terminology.</Paragraph>
    <Section position="1" start_page="56" end_page="56" type="sub_section">
      <SectionTitle>
2.1 Relative and absolute loealisations
</SectionTitle>
      <Paragraph position="0"> The objects shown in part A of figure 2 can be localised as follows:  (1) &amp;quot;Object A is on the right side of the picture.&amp;quot; (2) &amp;quot;Object B is ill the lower part of the picture.&amp;quot; (3) &amp;quot;Object A is to the right of Object B.&amp;quot; (4) &amp;quot;Object B is below Object A.&amp;quot; Sentences (1) and (2) are considered to contain absolute localisations: an object is localised by stating its absolute position in the picture. Sentences (3) and (4) are examples of relative localisations: an object  is localised by stating its position relative to another object. The object to be located will be called the primary object (LO for short). The object that serves as reference for locating the primary object is called reference object (REFO for short).</Paragraph>
      <Paragraph position="1"> How can we explain the similarity between absolute and relative localisations, between &amp;quot;on the right side of the picture&amp;quot; and &amp;quot;to the right of Object B&amp;quot;? Our hy1)othesis is: Absoh'lte localisations are specialisations of relative localisations in the sense that for absolute localisations the center of the picture functions as an implicit reference object.</Paragraph>
      <Paragraph position="2"> Part B of figure 2 shows how the absolute localisation of part A can be explained as a relative localisation by assuming a circle-shaped center: &amp;quot;Object A is on the right side of the picure.&amp;quot; is equivalent to &amp;quot;Object A is to the right of the center of the picture.&amp;quot;</Paragraph>
    </Section>
    <Section position="2" start_page="56" end_page="57" type="sub_section">
      <SectionTitle>
2.2 Elementary and composite localisations
</SectionTitle>
      <Paragraph position="0"> Whereas the unambiguous localisations of the objects in figure 2 could be achieved by naming either the horizontal (&amp;quot;on the right side&amp;quot;, &amp;quot;to the right of&amp;quot;) or vertical relation (&amp;quot;in the lower part&amp;quot;, &amp;quot;below&amp;quot;), figure 3 shows a situation in which it is necessary to give both the horizontal and vertical position of the object with respect to the reference object:  C as the object &amp;quot;to the right of&amp;quot; or &amp;quot;above&amp;quot; object A. But in part B, both descriptions would be ambiguous, because &amp;quot;to the right of&amp;quot; or &amp;quot;above&amp;quot; could refer to object D or B respectively. The only possibility to localise C unambiguously is to describe it as being &amp;quot;above and to the right&amp;quot; of A.</Paragraph>
      <Paragraph position="1"> Localisations where either the horizontal or vertical relation is given will be called elementary localisations. If both relations are stated together, we will call it a composite localisation.</Paragraph>
      <Paragraph position="2"> The localisation types introduced so far -- absolute vs.</Paragraph>
      <Paragraph position="3"> relative and elementary vs. composite -- are orthogonal.</Paragraph>
      <Paragraph position="4"> Therefore, an absolute or a relative localisation can be further subcategorized as being all elementary or a cornposite localisation.</Paragraph>
      <Paragraph position="5">  (;omposite localisations cannot always he applied, e.g., in figure 2 object B cannot be localised as &amp;quot;the object in the lower left part of the picture.&amp;quot; Criteria for the applicability of composite localisations will not he exalnined further in this paper as this would lead to more complex questions, e.g., whether an object can be localised at all. A detailed discussion of these prohlems is given in \[Wazinski, 1991\].</Paragraph>
    </Section>
    <Section position="3" start_page="57" end_page="57" type="sub_section">
      <SectionTitle>
2.3 The construction of the horizontal and
</SectionTitle>
      <Paragraph position="0"> vertical reference fi'alne One important feature of the localisation l)rocedures is the division of the horizontal and vertical reference frame into three parts. The reason for this are 'center'localisations as shown in figure 4:  In all pictures, object A can be localised as tile object &amp;quot;in the center of the 1)icture.&amp;quot; In order to integrate this observation with the elementary vs. composite distinction we divided the horizontal and vertical dimension into three parts: 'top', 'horizontal center' and 'bottom' and 'left', 'vertical center' and 'top' respectively (c.f. figure 5). Under these conditions the 'center'-localisation in the left part of figure 4 can be analysed as a composite ('vertical center','horizontal center')-localisation. For the picture in the middle it is an elementary 'vertical center'-localisation and for the right one an elementary 'horizontal center'-localisation. When transforming these different localisations into a surface string they all become the same: &amp;quot;in the center of the picture.&amp;quot; left vertical right center top horizontal center &amp;quot;to the right of A&amp;quot; by assuming that the 'center'-part of a composite localisation is a special part of a composite localisation that does not appear at the linguistic level.</Paragraph>
    </Section>
    <Section position="4" start_page="57" end_page="57" type="sub_section">
      <SectionTitle>
2.4 Corner Loealisations
</SectionTitle>
      <Paragraph position="0"> An additional localisation type that can be used to localise objects in pictures is the corner localisation: if an object is placed in one of the four corner regions of the picture it can be localised as, e.g., &amp;quot;the object in the left upper corner of the picture.&amp;quot; Tile difference between absolute composite localisations and corner localisations is illustrated in figure 7: While object B can be localised as being &amp;quot;in the lower right corner of the picture&amp;quot; it is not possible to use a corner localisation for A. In that case, only &amp;quot;in the left upper part of the picture&amp;quot; could be used. Therefore, we consider corner localisations to be more precise than absolute composite localisations, i.e., the applicability of a corner localisation implies the applicability of the corresponding absolute composite localisation but not vice versa.</Paragraph>
      <Paragraph position="1">  this partition scheme for relative localisations: B would usually be described as the object &amp;quot;to the right of A&amp;quot; and C as the object &amp;quot;above and to the right of A.&amp;quot; With respect to tile partition scheme a ('right', 'top')localisation can be applied to C and a ('right', 'horizontal center')-localisation to B, The former matches exactly with the surface string. The latter can be matched with</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>