<?xml version="1.0" standalone="yes"?>
<Paper uid="E06-1009">
  <Title>Information Presentation in Spoken Dialogue Systems</Title>
  <Section position="4" start_page="66" end_page="67" type="metho">
    <SectionTitle>
3 Our Approach
</SectionTitle>
    <Paragraph position="0"> Our work combines techniques from the UM and SR approaches. We exploit information from a user model to reduce dialogue duration by (1) selecting all options that are relevant to the user, and(2)introducingacontentstructuringalgorithm that supports stepwise refinement based on the ranking of attributes in the user model. In this way, we keep the benefits of user tailoring, while extending the approach to handle presentation of large numbers of options in an order that reflects user preferences. To address the problem of user confidence, wealsobrieflysummarizeoptionsthat the user model determines to be irrelevant (see section 4.3). Thus, we give users an overview of the whole option space, and thereby reduce the risk of leaving out options the user may wish to choose in a given situation.</Paragraph>
    <Paragraph position="1"> The integration of a user model with the clustering and structuring also alleviates the three problems we identified for the SR approach. When a  user model is available, it enables the system to determine which options and which attributes of options are likely to be of interest to the particular user. The system can then identify compelling options, and delete irrelevant options from the refinement structure, leading to shorter refinement paths. Furthermore, the user model allows the system to determine the tradeoffs among options.</Paragraph>
    <Paragraph position="2"> These tradeoffs can then be presented explicitly.</Paragraph>
    <Paragraph position="3"> Theusermodelalsoallowstheidentificationofthe attribute that is most relevant at each stage in the refinement process. Finally, the problem of summarizing a large number of diverseattributevalues can be tackled by adapting the cluster criterion to the user's interest.</Paragraph>
    <Paragraph position="4"> In our approach, information presentation is driven by the user model, the actual dialogue context and the available data. We allow for an arbitrarily large number of alternative options. These are structured so that the user can narrow in on one of them in successive steps. For this purpose, a static option tree is built. Because the structure of the option tree takes the user model into account, it allows the system to ask the user to make the most relevant decisions first. Moreover, the option tree is pruned using an algorithm that takes advantage of the tree structure, to avoid wasting time by suggesting irrelevant options to the user. The tradeoffs (e.g., cheap but indirect flights vs. direct but expensive flights) are presented to the user explicitly, so that the user won't have to &amp;quot;guess&amp;quot; or try out paths to find out what tradeoffs exist. Our hypothesis was that explicit presentation of trade-offs would lead to a more informed choice and decrease the risk that the user does not find the optimal option.</Paragraph>
  </Section>
  <Section position="5" start_page="67" end_page="69" type="metho">
    <SectionTitle>
4 Implementation
</SectionTitle>
    <Paragraph position="0"> Our approach was implemented within a spoken dialogue systemfor flight booking. While the content selection step is a new design, the content presentation part of the system is an adaptation and  extensionoftheworkongeneratingnaturalsounding tailored descriptions reported in (Moore et al., 2004).</Paragraph>
    <Section position="1" start_page="67" end_page="67" type="sub_section">
      <SectionTitle>
4.1 Clustering
</SectionTitle>
      <Paragraph position="0"> The clustering algorithm in our implementation is based on that reported in (Polifroni et al., 2003).</Paragraph>
      <Paragraph position="1"> The algorithm can be applied to any numerically ordered dataset. It sorts the data into bins that roughly correspond to small, medium and large values in the following way. The values of each attribute of the objects in the database (e.g., flights) are clustered using agglomerative group-average clustering. The algorithm begins by assigning each unique attribute value to its own bin, and successively merging adjacent bins whenever the difference between the means of the bins falls below a varying threshold. This continues until a stopping criterion (a target number of no more than three clusters in our current implementation) is met. The bins are then assigned predefined labels, e.g., cheap, average-price, expensive for the price attribute.</Paragraph>
      <Paragraph position="2"> Clustering attribute values with the above algorithm allows for database-dependent labelling. A PS300 flight gets the label cheap if it is a flight from Edinburgh to Los Angeles (because most other flights in the database are more costly) but expensive if it is from Edinburgh to Stuttgart (for which there are a lot of cheaper flights in the data base). Clustering also allows the construction of user valuation-sensitive clusters for categorial values, such as the attribute airline: They are clustered to a group of preferred airlines, dispreferred airlines and airlines the user does not-care about.</Paragraph>
    </Section>
    <Section position="2" start_page="67" end_page="68" type="sub_section">
      <SectionTitle>
4.2 Building up a Tree Structure
</SectionTitle>
      <Paragraph position="0"> The tree building algorithm works on the clusters producedbytheclusteringalgorithminsteadofthe original values. Options are arranged in a refinement tree structure, where the nodes of an option tree correspond to sets of options. The root of the tree contains all options and its children containcomplementarysubsetsoftheseoptions. Each child is homogeneous for a given attribute (e.g., if the parent set includes all direct flights, one child might include all direct cheap flights whereas another child includes all direct expensive flights).</Paragraph>
      <Paragraph position="1"> Leaf-nodes correspond either to a single option or to a set of options with very similar values for all attributes.</Paragraph>
      <Paragraph position="2"> This treestructuredetermines the dialogue flow.</Paragraph>
      <Paragraph position="3"> To minimize the need to explore several branches of the tree, the user is asked for the most essential criteria first, leaving less relevant criteria for later in the dialogue. Thus, the branching criterion for the first level of the tree is the attribute that has the highest weight according to the user model. For example, Figure 5 shows an option tree structure  for our &amp;quot;business&amp;quot; user model (Figure 4).</Paragraph>
      <Paragraph position="4"> The advantage of this ordering is that it minimizes the probability that the user needs to backtrack. If an irrelevant criterion had to be decided  onfirst,interestingtradeoffswouldriskbeingscattered across the different branches of the tree. A special case occurs when an attribute is homogeneous for all options in an option set. Then a unarynodeisinsertedregardlessofitsimportance.</Paragraph>
      <Paragraph position="5"> This special case allows for more efficient summarization, e.g., &amp;quot;There are no business class flights on KLM.&amp;quot; In the example of Figure 5, the attribute airline is inserted far up in the tree despite its low rank.</Paragraph>
      <Paragraph position="6"> The user is not forced to impose a total ordering on the attributes but may specify that two attributes, e.g., arrival-time and number-of-legs, are equally important to her.</Paragraph>
      <Paragraph position="7"> This partial ordering leads to several attributes having the same ranking. For equally ranked attributes, wefollowtheapproachtakenbyPolifroni et al. (2003). The algorithm selects the attribute that partitions the data into the smallest number of sub-clusters. For example, in the tree in Figure 5, number-of-legs, which creates two sub-clusters for the data set (direct and indirect), comes before arrival-time, which splits the set of economy class flights into three subsets.</Paragraph>
      <Paragraph position="8"> The tree building algorithm introduces one of the main differences between our structuring and Polifroni's refinement process. Polifroni et al.'s systemchoosestheattributethatpartitionsthedata into the smallest set of unique groups for summarization, whereas in our system, the algorithm takes the ranking of attributes in the user model into account.</Paragraph>
    </Section>
    <Section position="3" start_page="68" end_page="69" type="sub_section">
      <SectionTitle>
4.3 Pruning the Tree Structure
</SectionTitle>
      <Paragraph position="0"> To determine the relevance of options, we did not use the notion of compellingness (as was done in (Moore et al., 2004; Carenini and Moore, 2001)), but instead defined the weaker criterion of &amp;quot;dominance&amp;quot;. Dominant options are those for which there is no other option in the data set that is better on all attributes. A dominated option is in all respects equal to or worse than some other option in the relevant partition of the data base; it should not be of interest for any rational user. All dominant options represent some tradeoff, but depending on the user's interest, some of them are more interesting tradeoffs than others.</Paragraph>
      <Paragraph position="1"> Pruning dominated options is crucial to our structuring process. The algorithm uses information from the user model to prune all but the dominant options. Paths from the root to a given option are thereby shortened considerably, leading to a smaller average number of turns in our system compared to Polifroni et al.'s system.</Paragraph>
      <Paragraph position="2"> An important by-product of the pruning algorithm is the determination of attributes which make an option cluster compelling with respect to alternative clusters (e.g., for a cluster containing direct flights, as opposed to flights that require a connection, the justification would be #-of-legs). We call such an attribute the &amp;quot;justification&amp;quot; for a cluster, as it justifies its existence, i.e.,isthereasonitisnotprunedfromthetree. Justifications are used by the generation algorithm to present the tradeoffs between alternative options explicitly.</Paragraph>
      <Paragraph position="3"> Additionally, the reasons why options have been pruned from the tree are registered and provide information for the summarization of bad options in order to give the user a better overview of the option space (e.g., &amp;quot;All other flights are either indirect or arrive too late.&amp;quot;). To keep summaries about irrelevant options short, we back off to a default statement &amp;quot;or are undesirable in some other way.&amp;quot; if these options are very heterogeneous.</Paragraph>
    </Section>
    <Section position="4" start_page="69" end_page="69" type="sub_section">
      <SectionTitle>
4.4 Presenting Clusters
4.4.1 Turn Length
</SectionTitle>
      <Paragraph position="0"> In a spoken dialogue system, it is important not to mention too many facts in one turn in order to keep the memory load on the user manageable.</Paragraph>
      <Paragraph position="1"> Obviously, it is not possible to present all of the options and tradeoffs represented in the tree in a single turn. Therefore, it is necessary to split the tree into several smaller trees that can then be presentedoverseveralturns. Inthecurrentimplementation, aheuristiccut-offpoint(nodeeperthantwo branching nodes and their children, which corresponds to the nodes shown in Figure 5) is used.</Paragraph>
      <Paragraph position="2"> This procedure produces a small set of options to present in a turn and includes the most relevant advantages and disadvantages of an option. The next turn is determined by the user's choice indicating which of the options she would like to hear more about (for illustration see Figure 6).</Paragraph>
      <Paragraph position="3">  The identification of an option set is based on its justification. If an option is justified by several attributes, only one of them is chosen for identification. If one of the justifications is a contextually salient attribute, this one is preferred, leading to constructions like: &amp;quot;...you'd have to make a connection in Brussels. If you want to fly direct,...&amp;quot;). Otherwise, the cluster is identified by the highest ranked attribute e.g.,&amp;quot;There are four flights with availability in business class.&amp;quot;. If an option cluster has no compelling homogeneous attribute, but only a common negative homogeneous attribute, this situation is acknowledged: e.g., &amp;quot;If you're willing to travel economy / arrive later / accept a longer travel time, ...&amp;quot;.</Paragraph>
      <Paragraph position="4">  After the identification of a cluster, more information is given about the cluster. All positive homogeneous attributes are mentioned and contrasted against all average or negative attributes. An attribute that was used for identification of an option is not mentioned again in the elaboration. Inoppositiontoa singleflight, attributes may have different values for the entities within a set of flights. In that case, these attribute values need to be summarized.</Paragraph>
      <Paragraph position="5"> There are three main cases to be distinguished:  price, arrival-time etc. need to be summarized, as they may differ in their values even if they are in the same cluster. One way to summarize them is to use an expression that reflects their value range, e.g.</Paragraph>
      <Paragraph position="6"> &amp;quot;between x and y&amp;quot;. Another solution is to mention only the evaluation value, leading to sentences like &amp;quot;The two flights with shortest travel time&amp;quot; or &amp;quot;The cheapest flights.&amp;quot; 2. For discrete-valued attributes with a small number of possible values, e.g., number-of-legs and fare-class, summarization is not an issue, because when homogeneous for a cluster, the attribute values of its options are identical.</Paragraph>
      <Paragraph position="7"> 3. The third group are attributes with categorial values, e.g., &amp;quot;airline&amp;quot;. If there are no more than three different values, we summarize using quantifications like &amp;quot;none/all/both of them&amp;quot;, as done in (Polifroni et al., 2003).</Paragraph>
      <Paragraph position="8"> If the values are more diverse, the user model comes back into play to produce a tailored summary based on user preferences (e.g., liking KLM). For example, we would generate &amp;quot;None are on KLM.&amp;quot;, which takes into accounttheuser'spreferenceandisshorterthan null mentioning all airlines the flights are on.</Paragraph>
      <Paragraph position="9">  Anissuearisingfromsummarizationwithnegation is that the negated value has to be salient, otherwise the utterance might be irritating. For example, it would be better to say &amp;quot;These flights are not direct.&amp;quot; in a neutral context, but &amp;quot;You would not need to connect in London Heathrow.&amp;quot; if London Heathrow had already been mentioned.</Paragraph>
      <Paragraph position="10"> A sample dialogue produced by our system, whengiventhebusinessusermodel(seeFigure4), is shown in Figure 6.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="69" end_page="70" type="metho">
    <SectionTitle>
5 Evaluation
</SectionTitle>
    <Paragraph position="0"> A within-participants laboratory experiment was conducted in order to determine whether user model-based clustering leads to increased overall user satisfaction, a better overview of the available options, quicker accessibility to the optimal option and higher confidence of having heard all relevant options. The experiment furthermore assessed whether the options were presented in a way that users found understandable and recorded the time users took to read a dialogue turn.</Paragraph>
    <Paragraph position="1">  U: Hello. I'd like to book a flight from Edinburgh to Brussels for tomorrow arriving at around 3pm.</Paragraph>
    <Paragraph position="2"> S: There are four flights with availability in business class. None are on KLM. The only direct flight arrives at 5:30 pm, which is later than you requested. To arrive earlier, you'll have to make a connection.</Paragraph>
    <Paragraph position="3"> If you're willing to travel economy, there is a direct flight on BMI, arriving at 1:10 pm. There's also a KLM flight arriving at 1:50 pm, but it requires a connection in Amsterdam. All other economy flights take longer or arrive after 4 pm.</Paragraph>
    <Paragraph position="4"> U: Canyoutellmemoreaboutthebusinessclassflightsthat arrive earlier? S: The two flights with shortest travel time are both on British Airways and cost 308 pounds. The first takes 4 hours and 20 minutes and arrives at 1:20 pm. The other takes 4 hours and 30 minutes and arrives at 3:30pm.</Paragraph>
    <Paragraph position="5"> There's a cheaper flight on Lufthansa. It arrives at 3:10pm and costs just 270 pounds, but it takes 5 hours and 35 minutes. All other business class flights arrive very early or too late.</Paragraph>
    <Section position="1" start_page="70" end_page="70" type="sub_section">
      <SectionTitle>
5.1 Experimental Design
</SectionTitle>
      <Paragraph position="0"> Each of the 38 subjects who completed the experiment was presented with six dialogue pairs, the first of which was used for training and was thus not included in the analysis. Each dialogue pair consisted of one dialogue between a user and our system and one dialogue between the same user and a system designed as described in (Polifroni et al., 2003; Chung, 2004) (cf. Section 2.2). Some of the dialogues with our system were constructed manually based on the content selection and structuring step, because the generation component did not cover all linguistic constructions needed. The dialogues with the Chung system were designed manually, as this system is implemented for another domain. The order of the dialogues in a pair was randomized. The dialogues were provided as transcripts.</Paragraph>
      <Paragraph position="1"> After reading each dialogue transcript, participantswereaskedfourquestionsaboutthesystem's null responses. They provided their answers using Likert scales.</Paragraph>
      <Paragraph position="2">  1. Did the system give the information in a way that was easy to understand? 1: very hard to understand 7: very easy to understand 2. Did the system give you a good overview of the available options? 1: very poor overview 7: very good overview 3. Doyouthinktheremaybeflightsthatarebetteroptions for X1 that the system did not tell X1 about? 1X was instantiated by name of our example users. 1: I think that is very possible 7: I feel the system gave a good overview of all options that are relevant for X1.</Paragraph>
      <Paragraph position="3"> 4. How quickly did the system allow X1 to find the optimal flight? 1: slowly 3: quickly  After reading each pair of dialogues, the participants were also asked the forced choice question: &amp;quot;Which of the two systems would you recommend to a friend?&amp;quot; to assess user satisfaction.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>