File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/92/a92-1009_metho.xml
Size: 22,199 bytes
Last Modified: 2025-10-06 14:12:55
<?xml version="1.0" standalone="yes"?> <Paper uid="A92-1009"> <Title>Automatic Generation of On-Line Documentation in the IDAS Project*</Title> <Section position="4" start_page="65" end_page="68" type="metho"> <SectionTitle> 3 KR and NLG </SectionTitle> <Paragraph position="0"> The fundamental purpose of IDAS's knowledge repre.</Paragraph> <Paragraph position="1"> sentation (KR) and natural-language generation (NLG components is to represent domain information in a mot, efficient form than thousands of canned text responses For example, a component's model number will typicall2 appear in at least 30 different query responses (i.e., ques. tion space points); representing it in the knowledge bas, and using NLG to produce text from the knowledge bas, allows the documenter to enter (and update) this infor mation only once, instead of 30 times.</Paragraph> <Paragraph position="2"> Many of the theoretically interesting aspects of IDAS' KR and NLG systems are discussed elsewhere, e.g., \[Re iter and Mellish, 1992; Reiter and Dale, 1992\]. Here, w, present a brief overview of the KR and NLG systems and then discuss three design decisions that we mad during the course of development: allowing canned tex and other 'cheats'; generating short and focused replies and stressing authorability instead of deep reasoning in content-determination. These decisions were not part of the original IDAS design, but rather were made as a result of experience in developing prototypes and interacting with our industrial collaborators; hence, they are 'lessons' we have learned that may be of interest to other researchers and developers working on similar applications-oriented projects.</Paragraph> <Section position="1" start_page="66" end_page="66" type="sub_section"> <SectionTitle> 3.1 Knowledge Representation </SectionTitle> <Paragraph position="0"> The IDAS knowledge-representation system uses a KL-ONE type taxonomy \[Brachman and Schmolze, 1985\] to represent domain entities (e.g., companies, ATE components, user actions) and linguistic knowledge (grammatical units, lexieal definitions, etc.). The knowledge-base is supported by a KL-ONE-like automatic classitier; a superclass-to-subclass attribute inheritance system, based on Touretzky's minimal inferential distance principle \[Touretzky, 1986\]; and a graphical browse/edit tool. We currently use a small demonstration knowledge base that contains about 200 classes that represent domain entities (e.g., the company Raeal, the component Counter-timer, and the user-action Clean), and 50 roles that represent domain attributes (e.g., colour and manufacturer). The knowledge-base will, of course, need to be substantially enlarged before it is of much use to real users.</Paragraph> <Paragraph position="1"> The knowledge-base also contains user-expertise and user-task models. The user-expertise models overlay the class taxonomy, and specify what words a user knows and what primitive actions he can execute; they are in some ways similar to the user-models used in the FN system \[Reiter, 1990\]. The task models do not contain any structure themselves, but affect which content-determination rule is chosen (Section 3.2), and hence the system's decision as to what information the response should communicate to the user. The current prototype contains 3 user-expertise models and 6 task models. More expertise and task models will probably be added with time, but we expect our final system to have at most tens of such models, not hundreds; our objective is provide expertise and task models that are a reasonable fit to most circumstances, not to be able to cover all possible users performing all possible actions.</Paragraph> <Paragraph position="2"> An authoring tool for the knowledge base is currently being developed by one of our industrial collaborators.</Paragraph> <Paragraph position="3"> Such a tool, which we hope will be directly usable by technical authors and domain experts, is of course vital to the ultimate success of the project.</Paragraph> </Section> <Section position="2" start_page="66" end_page="66" type="sub_section"> <SectionTitle> 3.2 Natural Language Generation </SectionTitle> <Paragraph position="0"> Natural-language generation is performed in IDAS in three stages: Content Determination: The basic-question, component, and user-task components of the question-space tuple are used to pick a content-determination rule, which specifies which information from the domain knowledge base should be communicated to the user.</Paragraph> <Paragraph position="1"> Text Planning: The KB information is turned into an SPL term, in a process which is sensitive to the user-expertise and discourse components of the question-space tuple. This process involves, for example, generating referring expressions and choosing open-class lexical items.</Paragraph> <Paragraph position="2"> Surface Realization: The SPL term is converted into a surface form, i.e., a set of words with formatting and hypertext annotations. Except for its hypertextrelated abilities, the IDAS surface-generation system has a similar functionality to a subset of the PENMAN system \[Penman Natural Language Group, 1989\].</Paragraph> <Paragraph position="3"> IDAS's NL generation system is only designed to be able to generate small pieces of text (a few sentences, a paragraph at most). This is because IDAS's hyper-text system should enable users to dynamically select the paragraphs they wish to read, i.e., perform their own high-level text planning \[Levine el al., 1991\], thereby eliminating the need for the generation system to perform such planning.</Paragraph> </Section> <Section position="3" start_page="66" end_page="67" type="sub_section"> <SectionTitle> 3.3 Design Decisions </SectionTitle> <Paragraph position="0"> We decided fairly early on not to put a great deal of effort into 'properly' handling rarely-occurring special cases, but instead to support canned text and other 'cheats' as a way of handling these cases. If a particular response is difficult for our KB system to represent or our generation system to generate, we simply enter canned text for this response, or (preferably) generate as much of the response as possible with straightforward applications of our knowledge-based techniques, and then add canned text annotations to convey things it would be difficult to generate.</Paragraph> <Paragraph position="1"> For example, one instructional action in the knowledge base is &quot;mount the ITA against the test head with the four lugs of the ITA resting in the four hooked receptacles of the test head&quot;. This is currently represented as a Mount action where the aetee is the ITA, the location is the Test-head, and the manner is the canned text.</Paragraph> <Paragraph position="2"> &quot;with the four lugs of the ITA resting in the four hooked receptacles of the test head&quot;. The system could be augmented to 'properly' represent this manner modifier, but we felt development efforts could more productively be spent elsewhere, and hence have left this modifier as canned text. Since IDAS's domain KB is only used for generating text and does not have to support general domain reasoning, the decision on whcn to use canned text can be made on such engineering grounds.</Paragraph> <Paragraph position="3"> We believe that 'cheating' in this and other manners (e.g., by only supporting a small number of user task and expertise models) is unavoidable, given our goal of building a usable NL generation system with non-trivial domain coverage. As in so many other fields, there is something like a 90%-10% law in operation; properly handling the 10% of special and unusual cases would require 90% of the development effort. With current-day NL generation technology, it is difficult enough to do a good job on the 90% of common cases; spending ten times this effort to handle the remaining 10% of unusual cases would not be justifiable, since we would get a much better usability payoff by spending a fraction of this effort in improving the handling of common cases.</Paragraph> <Paragraph position="4"> When the project started, our industrial collaborators gave us an initial list of sample responses they wanted us to try to generate. These responses were fairly general, a.nd subsequent discussions revealed that using more context-specific responses was preferable both for our collaborators and for us. Our collaborators preferred such responses because they were more likely to give users the information they really needed, while we found that the context-specific responses were in many ways easier to generate than the original responses; this was largely because they tended to have simpler linguistic and rhetorical structures.</Paragraph> <Paragraph position="5"> For example, the original human-generated response for the query &quot;What is the ARINC-429 Interface&quot; was &quot;The ARINC-429 interface is a serial Avionics bus interface for communicating with a UUT fitted with this bus interface. ARINC 429 is a single source, multi-sink unidirectional data transmission standard. It is used to interface digital avionics equipment in commercial applications, but is also seen in military equipment where there is a commonality with commercial equipment.&quot; In our current prototype, if this query was asked by a user with a Skilled expertise level who was engaged in a Replace-Part task, the response would be &quot;It is a Racal 10500-130 ARINC-429 interface with part number RIL-523.&quot; The second response is intended to inform the user of the interface's manufacturer, model number, and part number, since that presumably is the information someone performing a Replace-Part task most needs to know. Hypertext follow-ups enable the user to get the location of the ARINC-429 interface and a list of its subcomponents; these also might be important pieces of information for a Replace-Part task.</Paragraph> <Paragraph position="6"> The second response thus gives the user the information he most needs to know to perform his task, and uses hypertext follow-ups to enable him to obtain other possibly important pieces of information; it does not give him a general description of the ARINC standard, as was present in the original human-generated response.</Paragraph> <Paragraph position="7"> This information is not directly relevant to the Replace-Part task, and could be as much of a distraction as a help to a maintenance technician; 4 it also would be difficult to represent and generate this text in the current IDAS architecture, except as canned text (e.g., it is difficult to represent the concept of 'military equipment that has a commonality with civilian equipment' in the IDAS knowledge base).</Paragraph> <Paragraph position="8"> Short, specific, and targeted responses were thus felt to be both more useful and in many ways easier to generate. There is a danger that such responses might be inappropriate, if one of the contextual factors (task, expertise, discourse) is incorrect. We will investigate this in more detail when we perform user-evaluation trials; our hope is that users will be able to use IDAS's hypertext follow-up capabilities to obtain the information they need if an inappropriate response is generated.</Paragraph> </Section> <Section position="4" start_page="67" end_page="68" type="sub_section"> <SectionTitle> 3.3.3 Content Determination </SectionTitle> <Paragraph position="0"> IDAS uses a much simpler content-determination system than other generation systems with somewhat similar goals (e.g., COMET \[McKeown et al., 1990\] and WIP \[Wahlster et al., 1991\]). Instead of using planning \[Moore and Paris, 1989\] or schemas \[McKeown, 1985\] to determine what to communicate, IDAS's content-determination system is based on rules (created by domain experts) of the form 'if a user asks question Q about a component of type C in the context of task T, he should be told facts F'. 5 In other words, it is intended to sup- null appropriate basic-question, component, and task role fillers, and attached data that indicates the facts to be communicated; content-determination is done by classifying the current question-space point into the rule taxonomy, and inheriting the attached data \[Reiter and Mellish, 1992\].</Paragraph> <Paragraph position="1"> port easy authorability, instead of reasoning from basic principles. We felt'this was the most appropriate way in which to achieve IDAS's goal of fairly broad, but not necessarily deep, domain coverage.</Paragraph> <Paragraph position="2"> One drawback of the lack of plan-based content-determination is that IDAS can not in general answer Why questions about its suggested actions, in the manner described by \[Moore and Swartout, 1990\]. Indeed, because IDAS does not have access to an underlying domain reasoning system (such as the EES system used by Moore and Swartout), it can only respond to a Why question if a 'purpose' plan for the relevant object or action has been explicitly added to the knowledge base by a domain expert.</Paragraph> </Section> </Section> <Section position="5" start_page="68" end_page="70" type="metho"> <SectionTitle> 4 Hypertext </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="68" end_page="68" type="sub_section"> <SectionTitle> 4.1 Hypertext in IDAS </SectionTitle> <Paragraph position="0"> IDAS's hypertext interface allows users to issue new queries by mouse-clicking on pieces of text. Hypertext links are automatically added to referring expressions and action descriptions; clicking on a referring expression pops up a menu of basic questions that can be asked about the referred-to component in the current context, while clicking on an action issues a request for IDAS to explain this action in more detail (i.e., issues a How-do-I-perform question for this action).</Paragraph> <Paragraph position="1"> P~esponses can also contain hyperschema follow-up buttons. These are specified by the content-determination rules' (Section 3.3.3), i.e., by rules of the form 'if a user asks question Q about a component of type C in the context of task T, he should be given the opportunity to ask follow-up question F'. These follow-up questions were originally intended to implement a variant of McKeown's schema system \[MeKeown, 1985\] where the user, instead of the system, decided which ATN arc to traverse; hence the name hyperschema. The mechanism is quite general, however, and can be used to add any useful follow-up question to a hypertext node.</Paragraph> <Paragraph position="2"> The current IDAS system also adds a special MENU button to all nodes, which allows users to explicitly modify their question-space coordinates in any way they choose; this button is primarily a development aid, and may not appear in the final system.</Paragraph> <Paragraph position="3"> Users can utilize the hypertext interface for many purposes, including: * Elaboration: If a user wants further information about an object or action mentioned in the text, he can obtain it by clicking on the textual description of that cntity.</Paragraph> <Paragraph position="4"> * High-level text planning: Ilyt~erschemas and the other follow-up mechanisnas allow users to dynamically specify which paragraphs they are interested in reading; this effectively means they can perform their own high-level text planning (Section 3.2).</Paragraph> <Paragraph position="5"> * Browsing: The hypertext interface provides some support for general browsing, which may be necessary if a user is not entirely sure which question he should ask. We may add more support for browsing in the future, such as a hypertext-based structural browser similar to the one proposed for the IMAD system \[Hayes and Pepper, 1989\].</Paragraph> <Paragraph position="6"> IDAS's hypertext interface is in some ways similar to the one presented by \[Moore and Swartout, 1990\], although it has a broader scope; Moore and Swartout used hypertext primarily to enable users to ask for clarifications of explanations, while IDAS uses hypertext as a general input mechanism which users can use to pose any question the system is capable of answering.</Paragraph> <Paragraph position="7"> The current IDAS prototype does not do anything interesting in modeling the discourse structure of hyper-text dialogues; it simply assumes that each hypertext node corresponds to a separate closed focus-space \[Grosz and Sidner, 1986\], and hence that an object introduced in one node cannot be referred to in another node unless it is re-introduced. We suspect this may be an overly conservative approach, and hope to do more research in the future on the relationship between the focus spaces of different hypertext nodes.</Paragraph> </Section> <Section position="2" start_page="68" end_page="69" type="sub_section"> <SectionTitle> 4.2 Example </SectionTitle> <Paragraph position="0"> Figure 2 shows some example IDAS texts produced by the various follow-up mechanisms. The initial query was What-are-its-parts, asked about the complete ATE by a Skilled expertise person performing an Operations task; this produces the text shown in Response 1. The underlined part names (which are in fact referring expressions) are all mousable, as is ATE in the title question and the buttons on the bottom line. Response 2 was produced by clicking on test head in Response 1, and selecting What-is-it from a pop-up menu of basic questions; this response was generated using the same user-task, userexpertise, and discourse-in-focus question-space components as Response 1. 6 The hyperschema follow-ups for (What-is-it, ?Component, Operations) are (Where-isit, ?Component, Operations) and (How-do-I-performthe-task, ?Component, Operations), so WHERE and USE 7 follow-up buttons are added to the response, s The MENU ~As mentioned above, IDAS currently assumes that discourse focus-space changes within one hypertext node do not have any effect on other nodes.</Paragraph> <Paragraph position="1"> Zltow-do-I-use-it is the interpretation of Itow-do-lperform-the-ta~k under an Operations user-task ~Other questions, e.g., What-are-its-parts, C/Ill be asked by clicking on test head in the title question, alld selecting button was described above; it allows the user to explicitly specify a new point in question space. Response 3 was obtained by clicking on WrlERE; it answers 'Where is the test head'.</Paragraph> <Paragraph position="2"> Response 4 comes from clicking on the USE button in Response 2; 9 it is a response to 'How do I use the test head'. In this response the underlined nouns test head, ITA mechanism, and ITA are all linked to pop-up menus of basic questions about these components, while the verbs unlock, mount, and lock are all linked to Howdo-I-perform queries for the relevant action. Clicking on unlock produces Response 5, which presents a step-by-step decomposition of the action of unlocking the ITA mechanism. Response 6 was obtained by clicking on lever in Response 5, and selecting What-is-it from the pop-up menu.</Paragraph> </Section> <Section position="3" start_page="69" end_page="70" type="sub_section"> <SectionTitle> 4.3 Hypertext vs NL Understanding </SectionTitle> <Paragraph position="0"> From IDAS's perspective, hypertext is a technique for enabling users to specify an input (i.e., a question-space point) to the NL generation system. As such, it is natural to compare it to other input mechanisms, particularly natural language text input. The advantages of hyper-text over NL input systems include: * Implementation: A hypertext interface is easier to implement than a NL input system; indeed, we have found that generating hypertext is only marginally from the pop-up menu.</Paragraph> <Paragraph position="1"> 9An identical response would have been obtained by clicking on USE in Response 3, since Responses 2 and 3 have the same task, expertise, and discourse question-space conlponents.</Paragraph> <Paragraph position="2"> more difficult than generating conventional text (i appropriate graphics support software is available) Implementing an NL input system, in contrast, is major undertaking.</Paragraph> <Paragraph position="3"> . Deictic References: As \[Moore and Swartout, 1990 point out, a hypertext interface makes many (al though not all) kinds of references trivial for a user he simply points to the phrase that describes th, object or action he wants more information about The user does not have to construct a complex re ferring expression (e.g., &quot;the board I was told to re move in the second sentence&quot;), and the system doe'. not have to try to resolve such complex references. * Transparency of Capabilities: NL understandinl systems are in general only capable of answering subset of the questions the user is able to pose, an( the user may become confused if he is not aware o the boundaries of this subset \[Tennant et al., 1983\] This problem does not arise with hypertext, whet, the user is only allowed to issue questions that th, system can answer.</Paragraph> <Paragraph position="4"> Perhaps the primary disadvantage of hypertext sys tems is their lack of flexibility; hypertext systems typi cally limit the user to pointing to a single entity and ask ing one of a small number of questions about it, while NI input systems allow queries to be stated using the ful power of English or other human languages. While thi is a severe, perhaps crippling, drawback in many applica tions, we believe it is less of a problem in IDAS, becaus, IDAS is only capable of responding to a small numbe of basic questions about entities in any case. Hypertext clicking can in fact be used in IDAS to pose almost all questions that IDAS is capable of answering; if the user is allowed to use the MENU button, then any answerable query can be issued through the hypertext interface. Accordingly, IDAS does not currently include an NL understanding component, and there are no plans for adding one; we believe that hypertext mechanisms will provide a sufficient query input mechanism for IDAS.</Paragraph> </Section> </Section> class="xml-element"></Paper>