<?xml version="1.0" standalone="yes"?>
<Paper uid="E89-1039">
  <Title>EMPIRICAL STUDIES OF DISCOURSE REPRESENTATIONS FOR NATURAL LANGUAGE INTERFACES</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
METHOD
</SectionTitle>
    <Paragraph position="0"> We have conducted a series of Wizard of Oz experiments. There are two important aspects to consider when developing the experimental situation. The first concerns the background system.</Paragraph>
    <Paragraph position="1"> It should in our opinion be something that could run on a computer using the technology of today or at least tomorrow m to ensure that the influence of the situation does not invalidate the use of data and results when developing a natural language interface. Great care should also be given to the design of the scenario, i.e. the task given to the subjects. Obviously, any simple task which only requires a few interactions between user and system will not give us much data to analyze. Our experience shows that one should either give the subjects a task for which there does not exist a single correct answer, but where the subjects own preferences determines what counts as a satisfying goal, or by having a task where there exists more than one way to achieve the goal.</Paragraph>
    <Paragraph position="2"> When conducting a Wizard of Oz experiment it is important to ensure that the subjects believe they are using a computer. To achieve this we have developed an experimental environment with a number of tools. The use of windows gives easy access to all relevant systems.The 'wizard' has at his disposal windows monitoring the user, the background system, an editor and windows with parsers or other modules developed for the current application. Menus with prestored (partial) answers guarantee a consistent, fast output with a 'computerized' quality (Dahlbtick and Jtnsson, 1986).</Paragraph>
    <Paragraph position="3"> Generalizability of results requires experiments with a variety of background systems, scenarios and many subjects. We have used five different scenarios for five background systems of varying complexityl; one library database used at our department and four simulated advisory systems: one student advisory system; one wine selection advisory system and two advisory-and-order systems m one for HIFI equipment and one for travel. We have collected dialogues from 21 subjects. Approximately half of them were students. The subjects' previous experience with computers were limited or nonexistent.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
THE DISCOURSE MODEL
</SectionTitle>
    <Paragraph position="0"> The collected corpus should be analyzed with an explicit formalism in mind. Our goal is not to develop a general discourse model, but instead to find the simplest possible usable model for natural language interface applications (or some subclass of such applications).</Paragraph>
    <Paragraph position="1"> The interface consists of three modules. One resembles a question-answering system without any dialogue handling capabilities. This will transform the user input into the appropriate query-language command or other background system input, given that enough information is available in the user's utterance. Another (linguistic context) module is used when the input does not contain enough information to form a  have recently conducted experiments using a combined graphical and NL calendar booking system. Since this communication situation differs from the others, we have excluded these data from the present analysis.</Paragraph>
    <Paragraph position="2"> - 292 command to the background system. This module uses the immediate linguistic context, i.e. the user's and the system's last utterance, and tries to complete the fragmentary input. Simple forms of indexicality will be handled here, e.g. ellipsis and pronouns that can be resolved by available surface structure linguistic information. The third module uses a case-frame like representation of the current discourse domain (task) 1. Here utterances whose interpretation requires background knowledge can be interpreted. One consequence of the use of this latter module is that it is necessary to specify the task structure of the discourse domain in advance of the analysis. This approach differs from linguistically oriented approaches to discourse analysis, where the task structure of the dialogue is found through the linguistic analysis.</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
ANALYSIS CATEGORIES
</SectionTitle>
    <Paragraph position="0"> We divide our utterances into four different categories (c.f. LineU, Gustavsson and Juvonen, 1988): 1) Initiative means that one of the participants initiates a query. 2) Response is when a participant responds to an initiative, such as an answer to a question. 3) Resp/Init is used when a new initiative is expressed in the same utterance as a response. Typical situations are when the system has found an answer and asks if the sub-ject wants to see it. The utterance type 4) Clarification is used in reply to a Response of type Clarification request and indicates what type of clarification is used. Jtnsson and Dahlb~tck (1988) describe and discuss the analysis categories in more detail.</Paragraph>
    <Paragraph position="1">  Initiatives are analyzed (&amp;quot;tagged&amp;quot;) for Context Dependence which concems the interpretation of an utterance. We tag an utterance Context Dependent if it cannot be interpreted without information in the immediate context. Every utterance that is complete enough to be interpreted without context is tagged Context Independent, regardless of the possible existence of a usable context in the previous utterance. Initiatives are tagged Task Dependent if background knowledge is required for their interpretation.</Paragraph>
    <Paragraph position="2"> 1 We use the term Task in this paper. The notion is similar to what we previously called Topic (Dahlback and JOnsson 1988, JOnsson and Dahlbltck 1988).</Paragraph>
    <Paragraph position="3"> Indexicality We tag our Context Dependent utterances for indexicality using three main categories: pronoun, ellipsis and definite description. It is important to note that there is a difference between these types, since they vary in their dependence of a specific theory or discourse representation model. What counts as a pronoun can be determined lexicaUy, and presents no major problem. But what counts as an ellipsis is dependent on the grammar used in the analysis, and to count a definite description as context dependent simply because there exists something in the previous text that could be seen as its antecedent seems somewhat dubious. In our opinion such an utterance should be called context dependent only if knowledge of the preceding linguistic context is necessary for finding its referent in the discourse representation, i.e. that the antecedent is necessary for determining the referent. And this is obviously dependent on the qualities of the discourse representation and the process working on it.</Paragraph>
    <Paragraph position="4"> Tagging a pronoun is usually straightforward, but there are some utterances which are ambiguous. For instance, the Swedish pronoun det (it) may act as an anaphoric pronoun or as a formal subject in various types of constructions, e.g.Traveh1:26 What does it cost? 2 \[Vad kostar det?\]. This is a question to a previous response suggesting a hotel to live in. The it in Travel: 1:26 can be interpreted either as pronoun referring to the hotel, or it can be a formal subject and then the utterance is elliptical. There are five utterances tagged ambiguous (all from the travel dialogues) and they are not included in the results.</Paragraph>
    <Paragraph position="5"> Definite descriptions are definite NP's or other definite referents like demonstratives, e.g. HiFi:l:5 What is the price for a complete hifi system with these models.\[Vad blir priset fi~r en komplett hifi-anldggning med dessa rood. eUer.l. Proper names are not tagged as definite descriptions.</Paragraph>
    <Paragraph position="6"> Ellipsis is a problematic category, cf. above.</Paragraph>
    <Paragraph position="7"> Our basic criterion is semantic incompleteness, thus one word phrases, except for some impera2All examples are from our corpus. The first field indicate the dialogue, the second subject and finally utterance number. The bold face does not occur in the dialogues. The corpus is in Swedish and translated into English striving for verbatim rather than idiomatic correctness.</Paragraph>
    <Paragraph position="8"> - 293 fives and expressives (Yes, Help, Thanks etc), are tagged ellipsis e.g. C line:4:5 prerequisites? \[fOrkunskaperl as response to a list of courses.</Paragraph>
    <Paragraph position="9"> We also use ellipsis for comparative constructions without expression of the comparative object e.g.</Paragraph>
    <Paragraph position="10"> Wines:4:9 Is there any cheaper white wine \[Finns det migot billigare vitt vin\].</Paragraph>
    <Paragraph position="11"> However, in spite of the fact that we have not used an explicit grammar, we have also regarded syntactic incompleteness as a ground for tagging an utterance elliptical. Certain questions like HiFi:3 :12 price sondek \[pris sondek\] are tagged elliptical for syntactic reasons. On the other hand imperative utterances like HiFi:3 :28 Order Sondek \[Best~ll Sondek\] are not tagged context dependent and thus not indexical at all. This might seem inconsequential, but is in fact a reflection of the characteristics of our assumed grammar.</Paragraph>
  </Section>
class="xml-element"></Paper>