<?xml version="1.0" standalone="yes"?>
<Paper uid="H94-1083">
  <Title>Advanced Human-Computer Interface and Voice Processing Applications in Space</Title>
  <Section position="3" start_page="0" end_page="416" type="metho">
    <SectionTitle>
2. THE WORKPLACE: SPACE
</SectionTitle>
    <Paragraph position="0"> Any space flight represents some degree of risk and working in space, as in aviation, comports some hazards. Suddenly, at any time during a mission, a situation may occur that will threaten the life of the astronauts or radically alter the flight plan. Thus, critical to the success of the mission and security of the crew is the complex process of interaction between astronauts and their spacecraft, not only in routine operation, but also in unforseen, unplanned, and life-threatening situations.</Paragraph>
    <Section position="1" start_page="0" end_page="416" type="sub_section">
      <SectionTitle>
2.1. Environmental Factors
</SectionTitle>
      <Paragraph position="0"> The environment outside spacecraft is unforgiving. With surface temperatures ranging from -180 C in darkness and 440 C in sunlight, high radiation and no atmosphere, lower earth orbit is hostile to life.</Paragraph>
      <Paragraph position="1"> Yet, astronauts work in this environment, under high workload and high stress, sheltered inside protective vehicules or dressed in bulky spacesuits.</Paragraph>
      <Paragraph position="2"> To limit the risks of space walks, the ability to perform physical actions remotely is crucial. Aboard the shuttle, remote action is performed using of the Remote Manipulator System (RMS) 1 . And more than any other task performed in space, telerobotics introduces higher demands on the relationship between operators and machines\[2\].</Paragraph>
      <Paragraph position="3"> Microgravity is another important environmental factors which considerably influence the interface configuration of aerospace systems. Space travel not only generates significant levels of stress in the human organism, but transforms the entire operational conditions.</Paragraph>
      <Paragraph position="4"> For instanc e , to perform their work in space, &amp;quot;weightless&amp;quot; astronauts must hold themselves &amp;quot;down&amp;quot; one way or another. They use straps and foot restraints, or simply grasp hand holders to maintain their position.</Paragraph>
      <Paragraph position="5"> All the above considerations impose restrictions and introduce severe design requirements as follows:  operated robot arm that is used in a semi- autonomous mode during those space flights that require objects to be handled, captured, and released into space.</Paragraph>
      <Paragraph position="6">  spacecraft. Every procedure and piece of equipment undergoes thorough review before being rated flight eligible. For example, all critical shuttle controls, such as an emergency stop switch, are required to meet very stringent layout requirements. No floating object or particle may inadvertently activate . or damage a sensitive system.</Paragraph>
      <Paragraph position="7"> * Reliability/Accuracy/Redundancy: high tolerance to failure is a condition to safety. Operative systems in space must be at least two fault-tolerant, if not more in the ease of critical systems such as flight controls or envkonmental control and life support systems (ECLSS). Where applicable, error correction mechanisms must be implemented.</Paragraph>
      <Paragraph position="8"> * Accessibility: the crew's ability to execute tasks safely and efficiently is notably improved if controls are ergonomically placed, clearly marked, and readily available\[3\]. Indirect accessibility is also crucial, particularly where overriding of automated functions is required.</Paragraph>
      <Paragraph position="9"> * Feedback: in diffeult operational envkonments such as microgravity, precise system feedback becomes essential. Through visual, auditive and tactile means, feedback reinforces security procedures and lessen the monitoring workload, particularly for telerobotic tasks which must be performed with extreme caution.</Paragraph>
      <Paragraph position="10"> On the shuttle, for example, robot arm operations are executed by two astronauts, one manipulating the robot and the other assisting with secondary functions, camera controls and stares displays. Visual feedback, if not precisely obtained from camera views, is directly available from the four windows of the Shuttle's flight deck. On Space Station, dkect visual feed-back will only be available on rare occasions and thus other means of feedback will have to be developed and integrated with the HCI of the robotics control workstation to provide camera redundancy.</Paragraph>
      <Paragraph position="11"> * Commonality: system configuration consistent in type and quality for crew operations enhance efficacy and lowers training demands. Operating Space Station with an international crew, in particular, will necessitate very high commonality of functions to ensure safety.</Paragraph>
    </Section>
    <Section position="2" start_page="416" end_page="416" type="sub_section">
      <SectionTitle>
2.2. Technology Proliferation
</SectionTitle>
      <Paragraph position="0"> Environmental constraints are only one of the many factors influencing the HCI problem in space. Technological diversification is another. Recent advances have considerably increased the processing and information handling capability of computer systems, thus bringing additional operative complexity that must be absorbed by operators. In aircraft and spacecraft, despite notable efforts to integrate systems more efficiently, there is so much information, so many sources, datatypes, categories, variations, possibilities, layouts, scales, etc. that crew members no longer operate their system globally. Instead, they receive specific training or pair up to accomplish their tasks.</Paragraph>
      <Paragraph position="1"> The impact of technology proliferation is clearly seen on the Space Shuttle. As described in a NASA technical report, it is clear &amp;quot;that the Shuttle cockpit contains the most complicated assortment of D&amp;C (Displays and Controls) ever developed for an aerodynamic vehicule. For control, there are toggle, push button, thumbwheel, press-down and rotary switches; potienfiometers; keyboards; circuit breakers; and hand controllers. Display devices include circular and vertical meters, tape meters, mechanical talkbacks, annunciators, flight control meters, digital readouts and CRTs. There are more than 2100 D&amp;C devices in the orbiter cockpit.&amp;quot;\[3\] With the number and types of redundant subsystems continually increasing, the use of dedicated control devices is rapidly growing into a large, complex system difficult to update and interact with.</Paragraph>
      <Paragraph position="2"> These conditions have prompted reconsideration of the direction taken in aerospace system design.</Paragraph>
      <Paragraph position="3"> An obvious solution to the problem of the exploding cockpit and crew workload in a demanding environment is a greater level of automation of functions and the introduction of alternative interfaces.</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="416" end_page="416" type="metho">
    <SectionTitle>
3. ADVANCED INTERFACES
</SectionTitle>
    <Paragraph position="0"> The computer and operational systems used in space function under either autonomous or human control. Much of the configuration complexity is kept as transparent as possible to the users, to allow them to concentrate on the purpose of the interaction, rather than system design details. The current design approach focuses on means of simplifying operations wherever possible and facilitating operatormachine communication.</Paragraph>
    <Paragraph position="1"> The concept of a more integrated human-computer system is clearly pertinent in space application. Astronauts are functional components of space systems, not only as operators and controllers, but as contributors to the overall performance of the system. On Space Station, where the network of computers will control and monitor thousands of automated systems as well as provide an interface to the crew, the need for performance will be heightened, necessitating increased automation and expansion of the supervisory role of the crew members.</Paragraph>
    <Paragraph position="2"> However, the decision to automate certain aspects of aerospace mission operations demands a careful consideration of the potential human-computer relationship. The decision to use a machine for a particular set of functions will depend on many factors such as availability, appropriateness, cost, compatibility with existing systems, and more importantly, safety and efficiency.</Paragraph>
    <Paragraph position="3"> Since few, ff any, external resources and development systems will be available on a permanent space platform such as Space Station, great selectivity and perspicacity must be exercised when designing and building the human computer interface. Hence, the interest in investigating new forms of interfaces and input/output devices, such as voice command and automatic speech recognition.</Paragraph>
  </Section>
  <Section position="5" start_page="416" end_page="417" type="metho">
    <SectionTitle>
4. AUTOMATIC SPEECH IN SPACE
</SectionTitle>
    <Paragraph position="0"> Automatic recognition and understanding of speech is one of the very promising application of advanced information technology. As the most natural communication means for humans, speech is often argued as being the ultimate medium for human-machine interaction.</Paragraph>
    <Paragraph position="1"> On the other hand, with its hesitations and complexity of intention, spoken language is often thought as being inadequate and unsafe for accurate control and time critical tasks\[4\]. Unconvinced of the reliability of speech processing as a control technology, pilots and astronauts have traditionally been reluctant to accept voice interfaces.</Paragraph>
    <Paragraph position="2"> Yet within a domain-limited command vocabulary, voice control has already been identified as a likely choice for controlling multifunction systems, displays and control panels in a variety of environments. Requiring minimal training, information transfer via voice control offers the basis for more effective information processing,  particularly in situations where speakers are already busy performing some other tasks.</Paragraph>
    <Section position="1" start_page="417" end_page="417" type="sub_section">
      <SectionTitle>
4.1. Benefits of Speech Technology
</SectionTitle>
      <Paragraph position="0"> Motivations for using ASR in space are numerous. Traditionally, space operations have been accomplished via hardware devices, dedicated system switches, keyboards and display interfaces. In such context, ASR is seen as a complement to existing controls that should be used in conjunction with other interaction devices akeady bounded in terms of previously defined needs and capabilities.</Paragraph>
      <Paragraph position="1"> It is foreseeable that voice control and synthesis could be used as an added I/O channel to use the crew more efficiently during peak workload periods. In particular, ASR may serve to facilitate operations in such areas as simultaneous control and monitoring (when hands and eyes are busy), extravehicular activities (EVA) and information storage and retrieval.</Paragraph>
      <Paragraph position="2"> For some applications, voice commands combined with manual controis may allow more rapid task completion than would be possible with manual methods alone. As an example, a study conducted in the Manipulator Development Facility (MDF) of the NASA Johnson Space Center showed that voice control could be effectively used to perform the many switching camera functions associated with the closed-circuit television system supporting the RMS robot arm\[3\]. The study also revealed that identical tasks (berthing and deployment) were completed in virtually identical times using manual switching and voice controlled switching having recognition accuracy between 85 and 95 percent. Using more accurate, state-of-the-art ASR equipment should allow for marked improvement in the overall RMS operations.</Paragraph>
      <Paragraph position="3"> Interest in voice command and automatic speech recognition interfaces for space stems from the benefits it may bring to the demanding  visual attention from monitoring tasks.</Paragraph>
    </Section>
    <Section position="2" start_page="417" end_page="417" type="sub_section">
      <SectionTitle>
4.2. Disadvantages and Concerns
</SectionTitle>
      <Paragraph position="0"> As described in section 2, technical constraints and environmental factors impose significant implementation requirements on the use of ASR and voice technology in space. Other issues to be considered range from the technical choices (isolated word vs continuous speech, single vs multiple speakers, word based vs phoneme based), the recognizer training update and maintenance requirements, the magnitude of changes in voice characteristics while in microgravity, and the effect of the space suit (0.3 atmosphere, pure oxygen) upon maintenance of highly accurate recognition.</Paragraph>
      <Paragraph position="1"> Without a doubt, ASR system will require a very high recognition accuracy rate, possibly 99evaluations performed at NASA that astronauts will switch to habitual controls if latency, reliability and efficiency criteria are not met\[5\]. Also, safety and requirements will necessitate a high level of recognition feedback to the users, with interactive error correction and user query functions.</Paragraph>
      <Paragraph position="2"> Finally, on the international Space Station, the diversity of languages and accents may make ASR an even more difficult challenge to meet.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="417" end_page="418" type="metho">
    <SectionTitle>
5. APPLYING VOICE IN SPACE
</SectionTitle>
    <Paragraph position="0"> Interest in voice technology for space appfications is not new. NASA is actively pursuing applications of voice recognition and synthesis for its spacecraft and ground operations\[5, 6, 7, 8\]. Several testbeds have incorporated voice into their commanding scheme, but only a few experiments have been performed in operational environments.</Paragraph>
    <Paragraph position="1"> These experiments are summarized below, followed by an outline of future applications.</Paragraph>
    <Section position="1" start_page="417" end_page="417" type="sub_section">
      <SectionTitle>
5.1. A Bit of History
Ground Test
</SectionTitle>
      <Paragraph position="0"> On the Shuttle, most Extra-Vehicular Activities (EVA) are performed for a specific tasks and rehearsed many times before the mission.</Paragraph>
      <Paragraph position="1"> To aid in these operations, cuff-mounted checklists have served as a useful reminder of procedures to follow. The problem with cuff checklists is that the wrist is not always in the best position for reading and at least one hand is required to turn the pages. Furthermore, information is limited to 3.25x4.5 inch pages which require a restraint to keep in position. For the longer missions on Space Station where EVA tasks will be less predictable, cuff checklist will be inadequate.</Paragraph>
      <Paragraph position="2"> In 1986 and 1988, a voice-activated/voice-synthesis system was developed in conjunction with a prototype space suit to provide an alternate information system. Equipped with a voice- controlled head-mounted display, the suit was evaluated on the ground in a series of neutral buoyancy tests\[9\].</Paragraph>
      <Paragraph position="3"> The voice system was termed an improvement over the cuffchecklist, allowing both hands on the job while moving through procedures, but astronauts commented that the system created a lot of disruptive &amp;quot;chatter&amp;quot; on the channel and interfered with communications.</Paragraph>
    </Section>
    <Section position="2" start_page="417" end_page="417" type="sub_section">
      <SectionTitle>
Voice Recording Test
</SectionTitle>
      <Paragraph position="0"> In 1990, direct digital recordings of an astronaut's voice were performed on the ground before a mission, in flight during the mission and on the ground upon return. A selected vocabulary was used and templates were made. After analysis, significant acoustic differences were noted. No conclusions were drawn, however, as to whether microgravity was the cause of these changes in voice production, since the discrepancy was mostly blamed on a substantial difference between recording environments.</Paragraph>
    </Section>
    <Section position="3" start_page="417" end_page="418" type="sub_section">
      <SectionTitle>
Shuttle Flight Test
</SectionTitle>
      <Paragraph position="0"> To date, only one experiment using voice recognition technology has ever been used aboard a spacecraft. The voice command system (VCS) experiment flew on board Space Shuttle Discovery STS-41 in October 1990 and allowed astronauts Bill Shepherd and Bruce Melnick to control the closed-circuit television (CCTV) cameras and monitors by voice inputs\[10\]. The voice command system had the capability to control the CCTV camera selection, and camera functions such as pan, tilt, focus, iris and zoom. The VCS paralleled the manual controls and provided both audible and visual feedback.</Paragraph>
      <Paragraph position="1"> The system was speaker dependent with templates of the voice of the two astronauts previously made on the ground. The recognizer  had limited continuous recognition and syntactic capabilities.</Paragraph>
      <Paragraph position="2"> The VCS intended to collect baseline data on the effect of microgravity on speech production and recognition. In addition, the experiment was meant to show the operational effectiveness of controlling a spacecraft subsystem using voice input. Analysis of the data showed little variation between the microgravity and ground-based templates of the astronauts voices. According to the investigators, astronauts were pleased with the tests and stated that voice control was a useful tool for performing secondary tasks on the Shuttle.</Paragraph>
      <Paragraph position="3"> Recent evaluations of different modes of camera control performed at NASA and to which the author participated have shown however that non-hardware controls will only be considered as sufficiently safe to be used in space if reliability can be proven, redundancy possible and efficiency significantly optimized. Moreover, as mentioned, experience has shown that crew members readily revert to the primary control system to which they are used to if an alternative system is not sufficiently accurate.</Paragraph>
    </Section>
    <Section position="4" start_page="418" end_page="418" type="sub_section">
      <SectionTitle>
5.2. Application Potential
</SectionTitle>
      <Paragraph position="0"> Spoken language communication with the control and monitoring subsystems onboard the Shuttle or Space Station is a convenience that could be provided through automatic speech recognition applications. Although ASR could not be the primary means of controlling critical actinns\[7\], it could be used to backup the primary controller and as an alternative I/O medium for the crew.</Paragraph>
      <Paragraph position="1"> Speech could also be used to query the status of a particular subsystem or database. Reference manuals could be called up and paged through. ASR could also be used for &amp;quot;hands-free&amp;quot; maintenance reporting, allowing the crew to attend to more important work and spend less time generating written reports.</Paragraph>
      <Paragraph position="2"> Other possible applications would be the use of speech to overcome reduced manual dexterity caused by astronauts having to wear bulky space suit and gloves during ascent and reentry. Voice interfaces could be used to allow a diversity in the number of tasks to be performed as flexible as the size of the recognizer's vocabulary.</Paragraph>
      <Paragraph position="3"> Of particular interest are the high workload and adverse condition situations (G-load, noise, stress) where dkect voice input could make a significant contribution to overall efficiency.</Paragraph>
      <Paragraph position="4"> Another promising application for voice technology in space is during EVA activities, where voice control would allow astronauts on space walks to perform interactive queries and/or remote manipulator control, while busy performing some other maintenance or repair task or even simply, busy holding themselves down.</Paragraph>
    </Section>
    <Section position="5" start_page="418" end_page="418" type="sub_section">
      <SectionTitle>
5.3. Space Station
</SectionTitle>
      <Paragraph position="0"> The proposed Space Station may also benefit from some form of automatic voice interaction to reduce transaction time between crew members and their multitasking, multi-panel workstations. Space-based crew are expected to interact with highly automated systems and to perform these interactions with often little prior training, or on an infrequent or sporadic basis. These activities will characterize a new role for astronauts, that of supervisory control.</Paragraph>
      <Paragraph position="1"> For instance, current plans for the Space Station involve the use of a significant robotic workforce for assembly, servicing and maintenance tasks. Generically referred to as the Mobile Servicing System, this workforce will be operated from a multi- purpose control workstation. Equipped with three display devices, the workstation will include one keyboard, one cursor control device and a dedicated hardware switching panel. The HCI aspects of the workstation are currently under designed in Canada and the proposed configuration has already raised several major issues centering around how crewmembers will interact with multiscreen systems. As there will be times in which users will be performing up to four simultaneous tasks using the robotics workstation, designers are now looking at alternative methods of interaction, including voice-activated features.</Paragraph>
      <Paragraph position="2"> Finally, as technology progresses, ASR might be used in conjunction with voice synthesis and natural language techniques to provide technical advice or even language translation to assist with communication between the international crew members.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>