File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/concl/80/c80-1027_concl.xml
Size: 15,939 bytes
Last Modified: 2025-10-06 13:55:51
<?xml version="1.0" standalone="yes"?> <Paper uid="C80-1027"> <Title>LINGUISTIC ANALYSIS OF NATURAL LANGUAGE COMMUNICATION WITH COMPUTERS</Title> <Section position="6" start_page="197" end_page="200" type="concl"> <SectionTitle> 5. Special Strate~ies~ Learning, Persistence of </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="197" end_page="200" type="sub_section"> <SectionTitle> Errors </SectionTitle> <Paragraph position="0"> A number of interesting strategies with respect to the use of language were observed. The discussion here is Just lllnstcative, but the annotation off the protocols shows that they were quite frequently employed. They are pretty self explanatory. (a) Paraphrase: e.g., H: &quot;What do the DKS usually hold?&quot; C: &quot;Input error, please re-enter request&quot; H: &quot;What are the primary uses of the DKS?&quot;. Similarly: &quot;Now long \[s the Anchorage?&quot; &quot;What Is the length of the Anchorage?&quot;. (b) Success: this usually involves repetitious structure of a sequence of requests, e.g., &quot;What relations are there?&quot; &quot;what shlp classes are there?&quot; &quot;Describe the AL.&quot; &quot;Describe the DKS.&quot; &quot;Describe water.&quot; &quot;Describe &quot; tank. * (c) Simplification of Sentence Structure: e.g., H: &quot;What \[s the maximum stow helght,bale cube,square foot capacity and top stack number of the cube of the PL?&quot; C: &quot;Input error...&quot; H: &quot;what is the maximum stow height and bale cube of the PL?&quot; C: &quot;40 72&quot; H: &quot;What ls the square foot capacity and top stack number oF. the PL?&quot; C: &quot;36 0&quot;. This example illustrates also the strategy of suspecting, and therefore changing, syntax. The subject made mistakes in punctuatlon by not using spaces, but changed syntax instead. That was quite a common strategy. Another example follows. (d) Mistrust of Syntax: e.g., &quot;What is known about the hatch of each deck of Alamo?&quot; &quot;What is known about hatches?&quot; &quot;What is the deck of each hatch?&quot; &quot;What is known ahout hatch?&quot; Each of these questions resulted in an ecror message, because the word &quot;hatch&quot; was not In the vocal)ul ary. But the subject persistently changes syntax and morphology rather than vocabulary. In the following example, the subject misspelled the word &quot;height&quot;, but still changes syntax first: &quot;The heigth of the Alamo/39?&quot; &quot;What is the height of the well deck of the Alamo/39?&quot; (e) Grammar Test in~: this was not very frequent, but interesting: &quot;What are the primary uses of each deck of the Alamo?&quot; &quot;What ls the primary use of each deck of the Alamo?&quot; (f) Data Add ltion: one subject added a new class: &quot;item:=class&quot;, and then made all items to be loaded members of this class: &quot;Water is an item.&quot; &quot;Conwire is an item.&quot;..., thus enabling himself to deal with all Items: &quot;Llst dimensions of items.&quot; Learning is evident throughout the proto~:ols. It ranges from a simple correction of a misspelling or punctuatlon error to changing the syntax, for Instance to obtain more specific answers; e.g., &quot;What ls it's beam?&quot; (repeatd three times) &quot;What is its beam?&quot;, H: &quot;What ls the minimum clearance of each deck?&quot; C: &quot;There are 224 lines ...&quot; H: &quot;What is the minimum clearance of each deck of the Alamo?&quot; Pertstence of errors was occasionally found, as in the next to last example, and the ffollowing dialogue -- which shows a variety of errors and strategies such as success, mistrusting syntax, breaking the input into shorter ones, adding a quant\[fier instead of correcting punctuation. This sequence also illustrates some of the errors analyzed in the next subsection (6).</Paragraph> <Paragraph position="1"> C: &quot;Input err...&quot;(lack of space) H: &quot;List C,D of sandbag&quot; C: &quot;Input err. . .&quot;(wrong punctuation, space) H: &quot;List C of sandbag, lub oil SAE50, fuel oil~ gen set PU670G.&quot; C: &quot;Input err. . .&quot; (lack of conjunction before last NP) FI: &quot;C of sandbag and lub ell SAE50.&quot; C: &quot;Input err...&quot;(wrong punctuation) H: &quot;C of sandbag.&quot; C: &quot;Input err.. .&quot;(same) H: &quot;C of sandbag?&quot; C: &quot;Class IV&quot;(good answer) H: &quot;C of lub ell SAE50,fuel oil,gem set PU670G?&quot; C: &quot;Input err. ..&quot;(lack of spaces and conJ unc tion) H: &quot;C of lub ell SAE50?&quot; C: &quot;Class lll&quot;(good answer) H: &quot;C of gem set PU670G?&quot; C: &quot;Class ll&quot;(good answer) H: &quot;List D of sandbag, lub oil SAE50,fuel oil, gen set PU670G.&quot; C: &quot;Input err. . .&quot;(lack of spaces and conJ unc t ton) H: &quot;List D of sandbag,lub oil SAEb0, fuel oil, gem set PU670G?&quot; C: &quot;Input err...&quot;(lack of space and conjunction) H: &quot;List D of each sandbag, lub oil SAE50, fuel oil, and gen set PU670(}.&quot; C: &quot;...&quot;(good answer) 6. Analysls of Errors Errors are all of those inputs that are not parsed, resulting in error messages. Not included in this analysis however are phatics and false starts, already covered. The message: &quot;Input error, please re-enter request&quot; is issued in all cases except when the system requests a response and the user fails to give one that can be ~inderstood, in which case the message is: &quot;Your response is not understandable. Try again.&quot;, and in the case of bugs, where it reads: &quot;Your request could not be completed as entered. Please rephrase and try again.&quot; The total of 446 errors were classified into eight categories: (a) Vocabulary ' Errors: arising from the lack of a word in the language, e.g., &quot;big&quot; in &quot;Is the Mobile a big ship?&quot;, &quot;feet&quot; in &quot;List the decks of each shlp with square feet capacity less than 70.&quot;. This being by fat the largest category, the importance of the semantic facto~ ls clear. (b) Punctuation: involves sentence final marks, commas and spaces; they are well illustrated in Figure 2.</Paragraph> <Paragraph position="2"> (c) Synta___x: the low incidence of these errors is surprising; formal style, repetitiousness of structure, expediency in problem solving may al\[ be factors. Errors involving conjunctions or preposistlons are typical. Some difficult to categorize, nonparsed inputs were also included here, such as: &quot;What is known?&quot;. In some cases, there ave vocabulary errors but the syntax could not have been handled either, typically: &quot;On what decks of the Alamo may cargo be stowed?&quot;, &quot;stow&quot; and &quot;may&quot; being not known. This input was immediately paraphrased as &quot;What is the primary use of each deck of the Alamo?&quot; and handled correctly; so one may wonder what ts \[nvolved in cases which could not be reasonably expected to be handled. (d) Spelling: the only interesting observation is that some subjects noticed these errors immediately, others not for a while. (e) Transmission: terminal and phone line errors. (f) Definition Format: all errors in framing definitions are included here, whether vocabulary, punctuation or format. (g) Lack of Response: to &quot;There are xx lines in this answer. How many do you want?&quot; One sub-ject tried 6 careful requests before catching on. (h) BuR: the actual number of bugs encountered was very low. In a very few cases they resulted in termination of the session.</Paragraph> <Paragraph position="3"> In general, errors were far fewer and far different from what I expected. The high intellectual level of the subjects cannot account for that, since it was more than counterbalanced by lack of familiarity with the system and lack of knowledge of the task. What should be done about errors~ and indeed what we are doing, is discussed in Section V.</Paragraph> <Paragraph position="4"> V. Habitability and Naturalness of Human-Computer Interaction: Sqme Conclusions The purpose of the experiments was to learn more about dialogues with the view to enhancing interaction with computers. What have we learned, and what are we doing? First, our guiding eonvlc t tons have been confirmed : English, especially if angmented to suit specific tasks, is a natural and useful medium.</Paragraph> <Paragraph position="5"> The Job of improving it is open-ended; English for the computer will never be all of English, since English is in reality not one language, but a variety of languages, among some of which all speakers choose freely, and many belong to specialists.</Paragraph> <Paragraph position="6"> Our task is to build as good a system as our understanding permits, observe the results of its use in actual tasks, and then with increased understanding continue to Improve.</Paragraph> <Paragraph position="7"> The REI. System served well in the experiments; its rapid response time was well worth achieving \[f for this purpose alone. But it is no longer a research tool. We are now building the POL (Problem Oriented Language) System. 15 Wqlat we have learned from the experiments \[s having a major influence on its design. Advances in parallel to our own are changing the human-computer relationship, and POL reflects these too. 7,16-19 Unlike REL, POL is programmed in a high-level language and thus more amenable for the research tasks that lie ahead.</Paragraph> <Paragraph position="8"> System breadth and depth in Pet rlck's sense 8 and rapid response time remain our major concerns. Whatever improvements are introduced have to meet these requirements. Experiments leave no doubt as to their essentiality. Intelligent system response to the user, using his knowledge base, and support for building that knowledge base using the facile capabilities of English, are two major areas where changes are made.</Paragraph> <Paragraph position="9"> Much is being done in the response to errors. REL was particularly weak in this area as Figure 2 on errors shows. Punctuation rules were too stringent, these can easily be relaxed and so designed as to almost entirely remove this source of error. For example, final punctuation can in almost all cases be added or corrected, and any ambiguities clarified gracefully. Even in REL &quot;List .... &quot; and &quot;List ... ?&quot; are accepted, to the relief of users.</Paragraph> <Paragraph position="10"> Defaulted responses and responses that add additional information should be acccepted; for example, lack of response to &quot;There are 203 lines...&quot; caused 16 errors in the protocols, yet in the POL design it is handled by defaulting.</Paragraph> <Paragraph position="11"> Identification of words not in the w)cabu-Lary and spelling correction did not exist in REL, resulting in a great deal of frustration.</Paragraph> <Paragraph position="12"> The two are related, and together accounted for 50 percent of errors. A problem here is the time inherent \[n spelling correction, however the new lexicon methods introduced in POL show promise of solving this problem. Syntactic and semantic means are used, as well as lexical, to identify intended usage, and echo is used to inform the user of the correction that is made; If the intent is not clearly identifiable, the user is infom~ed, ILsting the troublesome words.</Paragraph> <Paragraph position="13"> The users should be encouraged and guided to avail themselves of the wide range of definitional capabilities. This is a primary way for users to directly build knowledge into the system. Def\[nition guides and help sequences are available in POL to this end. A major aspect of definitions is multiple defining of terms. To illustrate from one application of REL, the notion of &quot;net sales&quot; was defined in five ways; th~is one could ask for &quot;net sales of d\[odes&quot;,&quot;net sales of the Eastern Sales Region&quot;, &quot;net sales of salesman Jones,&quot;; the internal ambiguity was always clarified in context. However, the stat\[stlcs from the experiments showing that of the ,q3 attempts at def\[nitlon 30 were not sucessful point to needed improvements in making this capability available. I feel especially that the incorporation of verbs which are introduced by definitional paraphrase and which were used in other REL applications enhances naturalness, even though the experiments showed a preference for be-verbs.</Paragraph> <Paragraph position="14"> The area of pronouns and ellipsis in general is, of course, very important. Pronouns worked to a certain extent in REL and they have been throughly revised for the POL Syste~ profitiug from the work of Grosz, 14 $\[dner and Robinson. 16 This area, however, will require much additional effort if we are to recognize the wide range of fragments - terse question, added information, and terse information. Some forms were handled by REL, e.g., &quot;Dimensions of eonwire?&quot;. However forms such as: &quot;How about ...&quot; and &quot;Those of ...&quot; need to be added. Added Information might be handled in such a sequence as &quot;Consider John, Joan, Betty and Bob. John and Bob are males. Joan and Betty are females.</Paragraph> <Paragraph position="15"> All are doctors.&quot; or &quot;What is the longest tanker? Only Norwegian.&quot; Terse information and dialogue connectors may also be considered, for instance: &quot;List the dimensions of vehicles.&quot;, and, following the answer, &quot;And pallets.&quot; Although I have only touched upon it briefly here, the prompt sequence in loading shlps was an effective tool whose usefullness was strongly supported by the experiments. The setting up ,of such abbreviated means of communication by the user, as well as their use, will be supported in POL.</Paragraph> <Paragraph position="16"> Finally, what about phatics? Should they be part of the computer's language? One is led by their wide use in face-to-face to include phatic messages from the computer, as is done in some of the other natural language systems.</Paragraph> <Paragraph position="17"> &quot;Welcome,&quot; &quot;Okay,&quot; &quot;Thank you&quot; are already in wide use. More of that nature would not hurt, within reason. Some inputs from the computer would undoubtedly be appreciated, such as: &quot;Be patient, l'm working on it.&quot; if the computation is long or response delayed.</Paragraph> <Paragraph position="18"> Is the recognition of users&quot; phatlcs and response to them desirable? Fillmore 21 pointed out that politeness can be carried too far, as In the sequence: A: &quot;You have lovely eyes.&quot; B: &quot;Thank you.&quot; A: &quot;You are welcome.&quot; Chafe 22 seems to be ready to see more human-like behavior on the part of the computer, even using variations in typing speed as a means of introducing a form of intonation and emphasis. We are currently investigating phatlcs, but while it could be interesting to observe users&quot; reactions in this respect, naturalness may be more highly enhanced in other areas. And so, not knowing how to respond, swearing is likely to remain ignored by the forever imperfect computer. null</Paragraph> </Section> </Section> class="xml-element"></Paper>