<?xml version="1.0" standalone="yes"?>
<Paper uid="C94-2133">
  <Title>Multi-Modal Definite Clause Grammar</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> This paper describes tile first reported grammatical fi'amework for a multimodal interface. Specifically, the authors have developed MM-DC.G (Multi-Modal l)cfinite Clause Gra,nmar), an extension of I)CCI \[Pereira and Warren, 1980\] for lnultm3odal input processing.</Paragraph>
    <Paragraph position="1"> The major features of MM-DCG include capability to handle an arbitrary nn,nber of modes and temporal information in grammar rules.</Paragraph>
    <Paragraph position="2"> The motivation behind this research was two-foht.</Paragraph>
    <Paragraph position="3"> First, the extension to multimodal was found t.o be the minimum requirement \[br natural language interface systems to be insta.lled in real al~plications. We have developed natural language interface for relational database (RDB) \[Shimazu et. al., I9!)2; Arita et. al., 1992a; Arita et. al., 1992b\]. Spoken user queries are transformed into SQL specifications, and dispatched t.o RDBMS. The retrieved results are displayed at a computer terminal. The results include not only table forms but also picture images, like Figure 1. When users see picture images on the terminal, they naturally want to generate following queries by referring to such picture images. For example, they want to say, &amp;quot;Show me the interior of this one&amp;quot; or &amp;quot;Are there the same type of cars as this ear&amp;quot; while pointing at a specific picture on the display. If such multi-modal utterances be accept.able, the natural language interface will be more practical Figure I: Natural Language Interface Screen hnage enough to be used in many real world applications.</Paragraph>
    <Paragraph position="4"> Second, no st;udy has yet appeard on developing forreal grammatical fi'amework for multi-modal interfaces.</Paragraph>
    <Paragraph position="5"> Although there have been many researches on multi-modal systems, these systems are built as task-specific expert systems. The capability of such systems to process multi-moda.l inputs is too limited to interpret complex multi-modal expressions. This is mainly due to the fact that they have not developed their systems on fornml grammatical framework for multi-modal interfaces.</Paragraph>
    <Paragraph position="6"> MM-DCG is the first reported grammatical framework for a multimodal interface. Multi-modal input processing rules can be written in MM-I)CG simply and effectively. Rules in MM-I)CG are translated into Prolog predicates easily.</Paragraph>
  </Section>
class="xml-element"></Paper>