<?xml version="1.0" standalone="yes"?>
<Paper uid="P84-1020">
  <Title>LIMITED DOMAIN SYSTEMS FOR LANGUAGE TEACHING</Title>
  <Section position="3" start_page="0" end_page="84" type="metho">
    <SectionTitle>
ERROR DETECTION AND REPORTING
</SectionTitle>
    <Paragraph position="0"> Of course, such an application carries its own special requirements. By definition, a language learner interacting with such a system is likely to be giving it input which is ill-formed in some way quite often. It is not a feature of most NL systems that they respond usefully in this situation: in a language tuition context, an efficient method for detecting and diagnosing errors is essential.</Paragraph>
    <Paragraph position="1"> The problem has of course not gone unnoticed.</Paragraph>
    <Paragraph position="2"> Hayes and Mouradian (1981), Kwasny and Sondheimer (1981) - among others - have presented techniques for allowing a parser to succeed even with ill-formed or partial input. The ATN based framework of the latter also generates descriptions of the linguistic requirements which have had to be violated in order for the parse to succeed. Such descriptions might well form the basis for a useful interaction between system and learner. However, the work most directly related to that reported here, and an influence on it, is that by Weischedel et al (1978) and Weischedei and Black (1980), (see also Hendr~x (1977). They also describe ATN based systems, this time  specifically intended for use in language tutoring programs. The earlier paper describes two techniques \[or handling errors: encoding likely errors directly into the network, so that the ungrammatical sentences are treated like grammatical ones, except that error messages are printed; and using 'failable' predicates on arcs for such things as errors of agreement. The disadvantages of such a system are obvious: the grammar writer has to predict in advance likely mistakes and a/low for them in designing the ATN. Unpredicted errors cannot be handled.</Paragraph>
    <Paragraph position="3"> The later paper describes a generalisation of these techniques, with two new features: condition-action pairs on selected states of the ATN for generating reports (1980:100) and the use of a 'longest path' heuristic (lOI) for deciding between alternative failed parsings. Although impressive in its coverage, We~schedel and Black report two major problems with the system: the difficulty of locating precisely where in a sentence the parser failed, and the difficulty of generating appropriate responses for the user.</Paragraph>
    <Paragraph position="4"> Those derived from relaxed predicates for the meanings of states were often fairly technical: some helpful examples of usage were given in some cases, but these had to be prestored and indexed by particular lexical items (103).</Paragraph>
    <Paragraph position="5"> The problem of accurately locating ungrammaticality is one that is extremely difficult, but arguably made more difficult than it need be by adopting the ATN framework for grammatical description. The ATN formalism is simply too rich: a successful parse in general depends not only on having traversed the network and consumed all the inpul but on having various registers appropriately filled. Since the registers may be inspected at different points this makes it difficult to provide an algorithmic method of locating ungrammaticality.</Paragraph>
    <Paragraph position="6"> The problem of generating error reports and helpful responses for the learner is also made more difficult than it need be if this is conceived of as something extra which needs to be added to a system already capable of dealing with we/l-formed input. This is because there is a perfectly straightforward sense in which this problem has already been solved if the system contains an adequate grammar. Such a grammar, by explicitly c~aracterising well-formedness, automatically provides an implicit characterisation of how far actual inputs deviate from expected inputs. It also contains all the grammatical information necessary for providing the user with examples of correct usage. These two types of information ought to be sufficient to generate appropriate reports.</Paragraph>
  </Section>
  <Section position="4" start_page="84" end_page="85" type="metho">
    <SectionTitle>
THE SYSTEM
</SectionTitle>
    <Paragraph position="0"> The syntactic theory underlying the present system is that of Generalised Phrase Structure Grammar, of the vintage described in Gazdar (1982). This is a more constrained grammatical formalism than that of an ATN, and hence it was possible to develop a relatively simple procedure for almost always accurately locating ungrammaticality, and also for automatically generating error reports of varying degrees of complexity, as well as examples of correct usage.</Paragraph>
    <Paragraph position="1"> All this is done using no information over and above what is already encoded in the grammar: nothing need be anticipated or pre-stored.</Paragraph>
    <Paragraph position="2"> Briefly, on the GPSG theory, the syntactic description of a language consists of two parts: a basic context-free grammar generating simple canonical structures, and a set of metarules, which generate rules for more complex structures from the basic rules. The result of applying the metarules to the basic rules is a large CFG.</Paragraph>
    <Paragraph position="3"> The system contains a suite of pre-compilation programs which manipulate a GPSG into the form used by the parser. First, the metarules are applied, producing a large, simple, CFG. The metarule expansion routine is in fact only fully defined for a subset of the metarules permitted by the theory. Roughly speaking, only metarules which do not contain variables which could be instantiated more than one way on any given rule application will be accepted. This is not a theoretically motivated restriction but simply a short cut to enable a straighforward pattern matching production system already available in Pop-ll to be transferred wholesale. A set of filters can be specified for the output by the same means if required.</Paragraph>
    <Paragraph position="4"> Next, the resulting CFG is compiled into an equivalent RTN, and finally this RTN is optimised and reduced, using a variant of a standard algorithm for ordinary transition networks (Aho and Uilman (1977:101). The intention behind this extensive preprocessing, apart from increased efficiency, is that the eventual system could be tailored by teachers for their own purposes. All that would be needed is the ability to write GPS grammars, or simple CF grammars, with no knowledge needed of the internal workings of the system.</Paragraph>
    <Paragraph position="5"> To give an example of the effect of this pre-processing, the grammar used by the system in the interchanges below contained about 8 rules and 4 metarules. These expand to a simple CFG of about 60 rules; this compiles to an RTN of over 200 states, and the final optimised RTN contains about 40 states.</Paragraph>
    <Paragraph position="6"> The parser is a standard RTN parser operating breadth first. The error detection routine is part of the main loop of the parser and works as follows: when no transition can be taken from a particular state in the network, a record is taken of the overall state of the machine. This contains information about how much of the sentence has been successfully parsed, the tree built, a list of states to POP to etc. If this record represents a more successful parse than any record so far it is preserved. This means that at the end of an unsuccessful parse the system has a record of the most successful path pursued, and this record is passed to the error reporting routine.</Paragraph>
    <Paragraph position="7"> If desired, all such records could be preserved during a parse and some procedure for choosing between then defined. This would mean that ambiguous parses can be treated independently, whereas at present only one record representing the most successful path through the input on any reading is retained.</Paragraph>
    <Paragraph position="8">  The error reporting routine is based around an RTN generator, which simply picks up from the point in the network indicated by the record handed to it, using the information in that record, as well as the RTN, and a special sub-lexicon described below. It is capable of generating error reports of several different types: (i) it can say what constituent(s) it was trying to complete (ii) it can say what type of item it was expecting to find at the point of failure either using the terminology of the grammar, or by example (iii) it can say what would be necessary to continue the sentence correctly, by generating example continuations.</Paragraph>
    <Paragraph position="9"> Here are some transcriptions of typical exchanges with the system using the small grammar  mentioned above: :go(); ** ready ? william put the book on the shelf ok ? did william put the book on the shelf yes ? was the book put on the shelf yes ? who put the book on the shelf william ? what did william put on the shelf the book ? what was put on the shelf the book ? who was the book put on the shelf by william ? what did william put sentence ok up to here:  what did william put ...</Paragraph>
    <Paragraph position="10"> expecting toofind one of the following preposition (in, on, etc) examples of grammatical continuations what did william put ...</Paragraph>
    <Paragraph position="11"> with something ? what did the read sentence ok up to here: what did the ...</Paragraph>
    <Paragraph position="12"> expecting to find one of the following adjective (nice, big etc.)/ noun (boy, girl etc.) examples ot grammatical continuations what did the ...</Paragraph>
    <Paragraph position="13"> whatdoyoucallit hit ? william hit jumble with a stick big sentence ok up to here: william hit jumble with a stick ...</Paragraph>
    <Paragraph position="14"> expecting to find end of sentence (NB this response is not as helpful as it could be, since the system does not look at the input after the point of failure).</Paragraph>
    <Paragraph position="15"> v who did was hit sentence ok up to here: who did ...</Paragraph>
    <Paragraph position="16"> expecting to find one of the following noun phrase examples of grammatical continuations who did ...</Paragraph>
    <Paragraph position="17"> something's thing hit ? who william did hit sentence ok up to here: who ...</Paragraph>
    <Paragraph position="18"> expecting to find one of the following verbl (did, was, etc.)/ verb2 (hit, read, etc.) examples of grammatical continuations who ...</Paragraph>
    <Paragraph position="19"> read something put something with something An attraction of this mechanism, apart from its simplicity, is that it is defined for the whole class of CFGs; this class of grammars is currently believed to be more or less adequate for English and for most of most other languages (Gazdar 1982). The two problems faced by the system of Weischedel and Black seem to have been overcome in a reasonably satisfying way: since after optimisation, the only non-determinism in the RTN is due to genuine ambiguity, we can be sure that the system will, given the way it operates, almost always locate accurately the point of failure in all non-ambiguous cases. And of course, when working with such limited domains we can control for ambiguity to a large extent, and deal with it by brute force if necessary. However, no such procedure can be wholly learner-proof, (as one of our referees has pointed out). A user might, for example, misspell his intended word and accidentally produce another legitimate word which could fit syntactically. Under these circumstances the parser would proceed unknowingly past the real point of error.</Paragraph>
    <Paragraph position="20"> The error reports delivered by the system can be as technical or informal as the grammar writer wants, or simply be prompts and examples of correct usage. In practice, simple one word prompts seem to be as useful as any more elaborated response. As will be clear from the examples, both for prompts and continuations, the system uses a restricted sub-lexicon to m~nimise the likelihood of generating grammatical nonsense. This sub-lexicon contains vague and general purpose words like 'thing' and 'whatsit'. This apart, no extra work has to be done once the grammar has been written: the system uses only its knowledge of what is grammatical to diagnose and report on what is ungrammatical.</Paragraph>
  </Section>
class="xml-element"></Paper>