<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1044">
  <Title>Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems</Title>
  <Section position="9" start_page="0" end_page="0" type="concl">
    <SectionTitle>
8 Conclusion
</SectionTitle>
    <Paragraph position="0"> We used a combination of acoustic confidence and pragmatic plausibility features (i.e. computed from dialogue context) to predict the quality of incoming recognition hypotheses to a multi-modal dialogue system. We classified hypotheses as accept, (clarify), reject, or ignore: functional categories that 7Following (Hinton, 1995), we leave out categories with expected frequencies &lt; 5 in the kh2 computation and reduce the degrees of freedom accordingly.</Paragraph>
    <Paragraph position="1"> can be used by a dialogue manager to decide appropriate system reactions. The approach is novel in combining machine learning with n-best processing for spoken dialogue systems using the Information State Update approach.</Paragraph>
    <Paragraph position="2"> Our best results, obtained using TiMBL with optimized parameters, show a 25% weighted f-score improvement over a baseline system that uses a &amp;quot;grammar-switching&amp;quot; approach to context-sensitive speech recognition, and are only 8% away from the optimal performance that can be achieved on the data. Clearly, this improvement would result in better dialogue system performance overall. Parameter optimization improved the classification results by 9% compared to using the learner with default settings, which shows the importance of such tuning.</Paragraph>
    <Paragraph position="3"> Future work points in two directions: first, integrating our methodology into working ISU-based dialogue systems and determining whether or not they improve in terms of standard dialogue evaluation metrics (e.g. task completion). The ISU approach is a particularly useful testbed for our methodology because it collects information pertaining to dialogue context in a central data structure from which it can be easily extracted. This avenue will be further explored in the TALK project8.</Paragraph>
    <Paragraph position="4"> Second, it will be interesting to investigate the impact of different dialogue and task features for classification and to introduce a distinction between &amp;quot;generic&amp;quot; features that are domain independent and &amp;quot;application-specific&amp;quot; features which reflect properties of individual systems and application scenarios.</Paragraph>
  </Section>
class="xml-element"></Paper>