<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-4012">
  <Title>UI on the Fly: Generating a Multimodal User Interface</Title>
  <Section position="3" start_page="0" end_page="0" type="relat">
    <SectionTitle>
2 Related Work
</SectionTitle>
    <Paragraph position="0"> Since Bolt's (1980) Put-That-There system introduced cross-modal coordination in multimodal user input, various projects have investigated multimodal input and output methods. Users display a preference for the touch-screen in map-based positioning acts and object selection (Oviatt et al., 1997). WIP (Andr'e et al., 1993) and other systems (Feiner and McKeown, 1990; Roth and Hefley, 1993) generate static multimodal documents. In an interactive user interface, however, layout should remain consistent (Woods and Roth, 1988, perceived stability).</Paragraph>
    <Paragraph position="1"> SmartKom (Wahlster, 2002) is a recent effort that produces a multimodal user interface, using XML/XSLT techniques to render the output. These are deterministic, which makes soft constraints such as usability hard to implement. SUPPLE (Gajos and Weld, 2004) overcomes this problem in its model of the user and the expected workload for various interfaces, generating a unimodal (graphical) user interface without natural language generation elements. On the integration side, Johnston (1998) presents a unification-based grammar that recasts multi-modal signal fusion as a parsing problem.</Paragraph>
    <Paragraph position="2"> Our approach employs a non-deterministic grammar to derive variants which are evaluated with a comparatively simple user and situation model according to their utility (information conveyed) and the projected cognitive load imposed on the user. It also removes the requirement inherent in Johnston's system of explicitly defining rules to integrate multimodal information.</Paragraph>
    <Paragraph position="3"> In the following, we discuss the grammar formalism used to create output, as well as consistency and adaptation considerations.</Paragraph>
  </Section>
class="xml-element"></Paper>