<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1009">
  <Title>Human-Machine Problem Solving Using Spoken Language Systems (SLS): Factors Affecting Performance and User Satisfaction</Title>
  <Section position="3" start_page="49" end_page="50" type="metho">
    <SectionTitle>
2. DATA COLLECTION METHODS
2.1. Subjects
</SectionTitle>
    <Paragraph position="0"> Data from a total of 145 subjects were included in the analyses. Subsets of these data were chosen for inclusion in each analysis in order to counterbalance for gender and scenario. The majority of subjects were SRI employees recruited from an advertisement in an intemal newsletter; a small number were students from a nearby university, employees in a local research corporation, or members of a volunteer organization. Subjects were native speakers of English, ranged in age from 22 to 71 and had varying degrees of experience with travel planning and computers.</Paragraph>
    <Section position="1" start_page="49" end_page="49" type="sub_section">
      <SectionTitle>
2.2. Materials
</SectionTitle>
      <Paragraph position="0"> Four different travel-planning scenarios were used. One entailed arranging flights to two cities in three days; a second entailed finding two fares for the price of a first class fare; a third required coordinating the arrival times of three flights from different cities; and a fourth involved weighing factors such as fares and meals in order to choose between two flight times. Because the task demands of the scenarios were different, we controlled for scenario in the analyses.</Paragraph>
    </Section>
    <Section position="2" start_page="49" end_page="49" type="sub_section">
      <SectionTitle>
2.3. Apparatus
</SectionTitle>
      <Paragraph position="0"> The data were collected using two versions of SRI's SLS (with no human in the loop); the first study also included data collected in a Wizard of Oz setting (Bly et a.l. \[1\]). The basic characteristics of the DECIPHER speech recognition component are described in Murveit et al.\[7,9\], and the basic characteristics of the natural language understanding component are described in Jackson et al. \[4\]. Some subjects used the real-time hardware version of the DECIPHER system (Murveit and Weintraub \[8\]; Weintraub et al. \[12\]); others used the software version of the system, which was a modified version of SRI's benchmark system (as described in the references above) tuned using the pnming threshold to improve speed at the cost of introducing a small number of recognition errors.</Paragraph>
      <Paragraph position="1"> SRI's SLS technology was implemented in the air travel planning domain, a domain with which many people are familiar (see Price \[10\]). The underlying database was a relational version of an 11-city subset of the Official Airline Guide. Two DARPA/NIST standard microphones were used: the Sennheiser HMD-410 close-talking microphone and the Crown PCC-160 table-top microphone. Most data were collected with two channels; some of the early data were collected using only the Sennheiser microphone.</Paragraph>
      <Paragraph position="2"> When both microphones were used, recognition was based on the Sennheiser input.</Paragraph>
      <Paragraph position="3"> The interface presented the user with a screen showing a large button labeled &amp;quot;Click Here to Talk.&amp;quot; A mouse click on this button caused the system to capture speech starting a half second before the click; the system automatically determined when the speaker finished speaking based on silence duration set at a threshold of two seconds. The user could move to the context of previous questions via mouse clicks.</Paragraph>
      <Paragraph position="4"> Once the speech was processed, the screen displayed the recognized string of words, a &amp;quot;paraphrase&amp;quot; of the system's understanding of the request, and, where appropriate, a formatted table of data containing the answer to the query. In cases where the natural language component could not arrive at a reasonable answer, a message window appeared displaying one of a small number of error messages. A log file was automatically created, containing time-stamps marking each action by the user and by the system.</Paragraph>
    </Section>
    <Section position="3" start_page="49" end_page="50" type="sub_section">
      <SectionTitle>
2.4. Procedure
</SectionTitle>
      <Paragraph position="0"> Subjects were seated in a quiet room in front of a color monitor, and had use of a mouse and microphone(s) but no keyboard. They were given a short demonstration on how to use the system. Some of the subjects were given additional instructions explaining that, while they might have a tendency to enunciate more clearly in the face of recognition errors, they should try to speak naturally, since the system was not trained on overenunciated or separated speech.</Paragraph>
      <Paragraph position="1"> Once subjects were comfortable with the system, they were left alone in the room while they solved travel planning scenarios. After they finished as many scenarios as possible within an hour, they were asked to fill out a questionnaire and were given a choice of gift certificate for use at a local bookstore or a contribution to a charitable institution.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>