<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1313">
  <Title>References</Title>
  <Section position="8" start_page="93" end_page="93" type="evalu">
    <SectionTitle>
6 Evaluation
</SectionTitle>
    <Paragraph position="0"> We conducted an evaluation of the Radiobot-CFF system in fully-automated, semi-automated, and human-controlled conditions. The system performed well in a number of measures; for example, Table 1 shows the scores for median time-tofire and task-completion rates. Additional measures and further details are available in (Robinson et al., 2006).</Paragraph>
    <Paragraph position="1">  Of particular relevance here, we performed an evaluation of the dialogue manager, using the evaluation corpus of 17 missions run on 8 sessions, a total of 408 FO utterances. We took transcribed recordings of the FO utterances, ran them through the Interpreter, and corrected them. For each session, we ran corrected Interpreter output through the Dialogue Manager to print out the values of the informational components at the end of every turn. We then corrected those, and compared the corrections to the uncorrected values to receive precision, accuracy, and f-scores of 0.99 each.2</Paragraph>
  </Section>
class="xml-element"></Paper>