File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/83/e83-1023_metho.xml
Size: 21,653 bytes
Last Modified: 2025-10-06 14:11:36
<?xml version="1.0" standalone="yes"?> <Paper uid="E83-1023"> <Title>LEARNING TRANSLATION SKILLS WITH A KNOWLEDGE-BASED TUTOR: FRENCH-ITALIAN CONJUNCTIONS IN CONTEXT</Title> <Section position="3" start_page="0" end_page="133" type="metho"> <SectionTitle> I. INTRODUCTION </SectionTitle> <Paragraph position="0"> In the framework of a large research and development project - called DART - concerned with the construction of an environment for the design of large scale Intelligent Teaching Systems (ITS~, a prototype ITS - called ELISA - was developed which teaches words (conjunctions~ of a foreign language in context (Cerri & Breuker, 1980, 1981; Breuker & Cerri, 1982~.</Paragraph> <Paragraph position="1"> The DART system is an authoring environment based on the formalism of ATNs for the representation of the procedural part of the teaching dialogue and on Semantic Networks for the representation of the conceptual and linguistic structures. The main achievement of DART was the integration of traditional Computer Assisted Learning (CAL~ facilities - such as the ones available in the PLATO system in an Artificial Intelligence framework, thus offer........ null The DART system on PLATO is the result of a joint effort of the University of Pisa (I~ and the University of Amsterdam (NL~ and its property rights are reserved. It can be distributed for experimentation and research.</Paragraph> <Paragraph position="2"> This work was ;artially financed by a grant of the GRIS group of the Italian National Research Council.</Paragraph> <Paragraph position="3"> ing authors a friendly environment for a smooth CAL - ITS transition when they design and develop teaching programs.</Paragraph> <Paragraph position="4"> ELISA was a testbed of the ideas underlying the DART project and at the same time a simple, but operational, &quot;intelligent&quot; foreign language teacher acting on a small subset of English, Dutch and Italian conjunctions. The sample dialogues of ELISA were chosen intentionally to exemplify, in the clearest way, issues such as the diagnostic of misconceptions in the use of foreign language conjunctions, which were addressed by the research. In particular, the assumption was made that a very simple representation of the correct knowledge needed for using f.l. conjunctions in context would have been sufficient to model the whole subject matter as well as the incorrect behaviour of the student.</Paragraph> <Paragraph position="5"> Owing to its prototypical and experimental character, ELISA was not ready for concrete, large scale experimentation on any pair of the languages mentioned.</Paragraph> <Paragraph position="6"> The research described in this report has been carried out with the concrete goal of making ELISA a realistic &quot;intelligent&quot; automatic foreign language teacher. In fact, we wanted to verify whether the simple representation of the knowledge in a semantic network was sufficient to represent a complete set of transformations from the first into the second language and vice versa.</Paragraph> <Paragraph position="7"> Italian and French were chosen. A complete contrastive representation of the use of conjunctions in meaningful contexts was produced.</Paragraph> <Paragraph position="8"> The set of these unambiguous, meaningful contexts - about 600 - defines the use of the conjunctions - about 40 for each language. Their correct use can be classified according to 60 distinguishing &quot;concepts&quot; which provide for all potential trans~la tions. null The classification was done on an empirical ground and is not based on any linguistic rule or theory. This was actually a contrastive bottom-up analysis of the use of conjunctions in Italian and French.</Paragraph> <Paragraph position="9"> The specific choice of the teaching material highlighted many (psyeho~linguistic and computational problems related to the compatibility between the design constraints of ELISA on the one hand and the subtleties of the full use of natural language fragments in translations on the other.</Paragraph> <Paragraph position="10"> In particular, the complexity of the full network of conjunctions, concepts and contexts in the two languages suggests a large set of possible misconceptions to be discovered from the (partially> incorrect behaviour of the students but only the subset of plausible ones should guide the diagnostic dialogue.</Paragraph> <Paragraph position="11"> In the following, we briefly present the teaching strategy of ELISA and some examples of dialogue in order to introduce the problems referred above and the solutions we propose.</Paragraph> <Paragraph position="12"> The full set of data is available in Merger & Cerri (19837 and a subset of it as well as a more extended description of this work can be found in Cerri & Merger (1982~. A detailed description of DART and ELISA is a work in preparation. null Notice that for the development of this knowledge base no other expertise was required than that of a professional teacher, once the principles are provided by AI experts. This is a proof of the potential power of AI representations in educational settings and in projects of natural language translation.</Paragraph> <Paragraph position="13"> Practically, our program is one of the few Intelligent Systems available in the field of Foreign Language Teaching and usable on a large scale for Computer Assisted Learning.</Paragraph> </Section> <Section position="4" start_page="133" end_page="135" type="metho"> <SectionTitle> II ELISA : A RATHER INTELLIGENT TUTOR OF FOREIGN LANGUAGE WORDS </SectionTitle> <Paragraph position="0"> A. The Purpose of ELISA ELISA teaches a student to disambiguate conjunctions in a foreign language by means of a dialogue. The purpose of ELISA's dialogue is to build a representation of the student's behaviour which coincides with the correct representation of the knowledge needed to translate words in a foreign language in context.</Paragraph> <Paragraph position="1"> ELISA has a student model, which is updated each time the student answers a question. According to the classification of the answer, and the phase of the dialogue, ELISA selects one or more new questions to be put to the student in order to achieve its purpose.</Paragraph> <Paragraph position="2"> The mother and the foreign language can be associated to the source and the target language (s.l. and t.l.~ respectively, or vice versa: the system is symmetric.</Paragraph> <Paragraph position="3"> The main phases of ELISA are Presentation and Assessment.</Paragraph> <Paragraph position="4"> B. The Presentation Phase The presentation phase is traditional. The teacher constructs an exhaustive set of Question Types from the subject matter represented in a knowledge network containing conjunctions and contexts in two languages as well as concepts adequately linked to conjunctions and contexts (see for instance Figs.l and 2~. These are pairs: conjunction in the source language/conceptual meaning. For each conjunction in the s.l. and each concept possibly associated to it a question type is generated. null For each question type, a classification of the conjunctions in the target language may be constructed. This classification is a partition of the t.l. conjunctions into three classes, namely expected right, expected wrong and unexpected wrong. The Expected Right conjunctions are all t. i. conjunctions which can be associated to the conceptual meaning of the question type. The Expected Wrong conjunctions are all t.l. conjunctions which can be a correct translation of the s.l. conjunction of the question type, but in a ~onceptual meaning different from that of the question type considered. The remaining conjunctions in the t.l. are classified as Unexpected Wrong: they do not have any relation in the knowledge base with the s.l. conjunction, nor with the concept in the question type considered.</Paragraph> <Paragraph position="5"> Notice that &quot;concepts&quot; are defined pragmatically i.e. in terms of the purpose of the representation which is to teach students to translate correctly conjunctions in context. This defintion of concepts is not based on any (psycho~linguistic theory or phenomenon. In fact, we looked for contexts which have a one-to-one correspondence with concepts, so that for each context all the conjunctions associated to its specific conceptual meaning can be valid completions of the sentence, in both languages.</Paragraph> <Paragraph position="6"> The question is generated from the question type by selecting (randomly~ a context linked to the concept of the question type, and inserting the conjunction of the question type. One of the (equivalent~ translations of the context into target language is also presented to the student. The student is required to insert the conjunction in the target language which correctly completes the sentence. null When the student makes an error, the correction consists simply in informing him/her of the correct answer(s~. This feedback strategy should have the effect of teaching the student the correct associations and is similar to that used in most CAL programs.</Paragraph> <Paragraph position="7"> In contrast to most CAL programs, in ELISA questions are generated at execution time from information stored in the knowledge network, The classification of answers is computed dynamically from the knowledge network, it is not a simple local pattern matching procedure.</Paragraph> <Paragraph position="8"> C. The Assessment Phase The purpose of the assessment phase is to verify the acquisition of knowledge and skills on the part of the student during the presentation phase. It includes the diagnosis and remedy of misconceptions. null Questions are generated as in the presentation phase, but in case of a consistent incorrect answer - a bug (see for instance Brown & van Lehn, 19801, - a complete dialogue with the student is performed in order to test the hypothesis that the bug arises from a whole set of errors grouped into one or more misconceptions.</Paragraph> <Paragraph position="9"> The procedure operates briefly as follows: each bug invokes a. one concept called Source Misconcept which represents the meaning of the context of the question put to the student (e.g., conditional, temporal, etc.1, and b. one or more concepts called Target Misconcepts which represent the possible meanings of the conjunction used by the student in the answer.</Paragraph> <Paragraph position="10"> The set of target misconcepts does not include the source misconcept by definition of the bug.</Paragraph> <Paragraph position="11"> For each pair of source/target misconcept, question types are generated and the questions are in turn put to the student. The selection of adequate question types is done on the basis of the Possible misconception(sl; a more skilled selection should include constraints ahout the Plausible (expectedl misconceptions, instead of considering exhaustively all the theoretical combinations. This is a maSn issue of further empirical research, as will be remarked later.</Paragraph> <Paragraph position="12"> During each of these diagnostic dialogues, it is possible that new bugs, i.e. bugs not related to the source and target misconcept, are discovered. When this is the case, these bugs are s=ored in a bug stack. Once the original misconception has been diagnosed and remedied, each bug in the bug stack triggers (recursivelyl the same diagnostic procedure. null Again, a more skilled stra=egy for the ordering of bugs to be diagnosed and remedied could be easily designed, on the basis of empirical evidence drawn by experiments on studentfs behaviour.</Paragraph> <Paragraph position="13"> Finally, let us discuss in more detail the evaluation of the student model as it was built according to a diagnostic dialogue. By &quot;student model&quot;, we mean the set of &quot;misconception matrixes&quot; each related to the source and a target misconcept, and related to two or more conjunctions.</Paragraph> <Paragraph position="14"> As these matrixes may, in principle, present a large variety of different patterns, and even allow for variations in their dimensions, it would be a rather complex task to design a minimal set of typical erroneous patterns unless some reduction procedure is applied.</Paragraph> <Paragraph position="15"> So, we first compress the misconception matrixes into &quot;confusion kernels&quot; which are (2x3~ matri xes, then we compare the kernels with standard patterns of stereotypical misconceptions. Once the match is found, the diagnostic phase is considered ended, and a remedy phase is begun.</Paragraph> <Paragraph position="16"> The remedy consists in informing the student of the &quot;nature of the misconception&quot;, i.e. the interpretation of the confusion kernel. This interpretation is possible by applying some (psychollinguistic criteria. In the following section, some of these Criteria will be outlined in order to explain the behaviour of ELISA in the examples of dialogue presented.</Paragraph> <Paragraph position="17"> In other words, the remedy is not a paraphrase of the history of the dialogue during the diagnosis, but an interpretation of the significant aspects of that dialogue. Although the ELISA project is to be considered completed, research is currently carried out in order to design a cognitively grounded theory of misconceptions occurring in this translation task. For some preliminary work, see Breuker & Cerri (1982~.</Paragraph> <Paragraph position="18"> It should be noticed that this is the most delicate aspect of this investigation. When ELISA was in a preliminary phase, and its dialogues were realistic but limited to a &quot;toy&quot; knowledge about the discriminative use of a few conjunctions, we did not expect that its extension to &quot;real&quot; knowledge would have implied such an explosion of possible right (and wrong~ links in the network, thus implying an explosion of possible models of student's behaviour. Now, the reduction of the number and complexity of these possible models requires undoubtedly empirical evidence. Currently, ELISA embodies enough intuitions to be considered a mature experimental tool, but not a complete theory of behaviour in translation, which will only be possible after many refinements of the simple theory ~ embodied by ELISA according to the experimental evidence in real educational settings.</Paragraph> <Paragraph position="19"> After a misconception has been remedied, the (newl bug stack is examined and each bug triggers a diagnostic-remedial procedure, possibly suggesting new bugs and so recursively.</Paragraph> <Paragraph position="20"> When a (new~ bug stack is empty, ELISA checks if all pairs of source/target misconcept have been examined, if it was not the case a diagnostic procedure is called, else the (original~ bug is considered remedied and ELISA formulates once more the question which received initially the wrong answer. We expect that now the student will not fail.</Paragraph> </Section> <Section position="5" start_page="135" end_page="135" type="metho"> <SectionTitle> III STEREOTYPICAL PROTOCOLS OF DIALOGUE </SectionTitle> <Paragraph position="0"> In this section we will present some examples of dialogue which may well represent atypical interaction occurring as diagnosis and remedy of a student's misconceptions.</Paragraph> <Paragraph position="1"> A. Conceptual Inversion The dialogue in Fig. 1 presents a prototype for a class of misconceptions which may be classified as &quot;conceptual inversion&quot;, i.e. the model of the student represents the fact the (s~he distinguishes between the source and target misconcept, but associates each of the two with a conjunction specific for the other of the two.</Paragraph> <Paragraph position="3"> El: Non vedo perchd non io farebbe.</Paragraph> <Paragraph position="4"> (I don't see why (s~he wouldn't do it.~ Je ne vois pas ... il ne le ferait pas.</Paragraph> <Paragraph position="5"> SI: Parce que E2: Non sei venuto? - No, perch@ non ne avevo voglia. null (You didn't come? - No, because I didn't feel like it.~ Tu n'es pas venu? - Non, ... je n'en avais pas envie.</Paragraph> </Section> <Section position="6" start_page="135" end_page="136" type="metho"> <SectionTitle> $2: Pourquoi </SectionTitle> <Paragraph position="0"> Fig, 1 Example of a dialogue concerning a &quot;Conceptual Inversion&quot; type of misconception. An excerpt of the knowledge network of ELISA concerning the (I12~ and (CR~ concepts is also presented.</Paragraph> <Paragraph position="1"> In this example, the first question of ELISA: E1 has the type (perch@, (ll2~2)and the expected right answer is &quot;pourquoi&quot;.</Paragraph> <Paragraph position="2"> 2 (I12~ means: 'Indirect Interrogation, 2nd type'. Usually, students know that &quot;pourquoi&quot; is correct in interrogative clauses, but sometimes they do not know that an interrogative clause might be indirect, as is our case. Therefore, the translation &quot;pourquoi&quot; is discarded, and the alternative &quot;parce que&quot; preferred. This conjunction is ind~ed a correct translation of &quot;perch,&quot;, but in (CR~ J contexts. This bug is classified as &quot;expected wrong&quot; and the diagnostic strategy is entered.</Paragraph> <Paragraph position="3"> The question E2 of ELISA checks if the student knows that the translation of &quot;perch,&quot; in (CR~ contexts is &quot;parce que&quot;. If this is the case, it could be guessed that the student does not know (the use of~ &quot;pourquoi&quot;, or alternatively knows (the use of~ pourquoi but believes &quot;pourquoi&quot; to be correct in a meaning different from (112) or (CR), and translates &quot;perch,&quot; with &quot;parce que&quot; irrespective of the context. This misconception will be described in more detail in the next subsection.</Paragraph> <Paragraph position="4"> Instead, the student answers: &quot;pourquoi&quot; which allows one to draw the following conclusions: a. the student distinguishes between (112) and (CR) contexts, but b. (s)he binds (112) with &quot;parce que&quot; and (CR) with &quot;pourquoi&quot;, which is the reverse of the correct knowledge about French conjunctions.</Paragraph> <Paragraph position="5"> We call this misconception Conceptual Inversion, the remedy of ELISA will explain to the student this result and give more examples of the use of these conjunctions as translations of &quot;perch,&quot; in each of the two conceptual meanings.</Paragraph> <Paragraph position="6"> B. Direct Translation The second example refers to the dialogue presented in F~g. 2. The question type of E1 is: (come, (SI) N and the expected right response of the student is either &quot;aussitSt que&quot; or d~s que&quot;. E2: Non appena so qualcosa, Le telefono.</Paragraph> <Paragraph position="7"> (As soon as I know something, I'ii phone you.) ... je sais quelque chose, je vous t~l~phone.</Paragraph> <Paragraph position="8"> An excerpt from the knowledge network related to the dialogue is also included.</Paragraph> <Paragraph position="9"> The French &quot;co~e&quot;,which is interfering with the Italian &quot;come&quot;, is not bound in any way to the concept (SI), but instead can be use d correctly as a translation of &quot;come&quot; in (CP) 5 contexts. This interference can be at the origin of the misconception consisting of the conviction that, although (SI) and (CP)contexts are clearly distinguishable in Italian, also because there is a specific Italian conjunction &quot;(non) appena&quot; for (SI), which was not true for the disambiguation of (112) and (CR) in the example of Fig. I, the Italian student consistently translates &quot;come&quot; with &quot;con~ne&quot; irrespective of the co~text.</Paragraph> <Paragraph position="10"> The answer to E1 of type (come, (Sl))is SI: &quot;comme&quot; which is expected wrong. ELISA puts a question E2 of type (non appena, (SI)) which is correctly answered by S2:&quot;d~s que&quot;. Finally, ELISA puts a question E3 of type (come, (CP)) and gets as answer &quot;comme&quot; which is again correct.</Paragraph> <Paragraph position="11"> It can be concluded that: a. it is possible, but not certain, that the student distinguishes between (SI) and (CP) contexts.</Paragraph> <Paragraph position="12"> Since &quot;non appena&quot; and &quot;d~s que&quot; are both unambiguously bound to (SI), the answer S2 does not show that the student recognizes the context (SI); (s)he might instead associate directly the conjunction &quot;non appena&quot; with &quot;d~s que&quot; without being aware of the conceptual meaning of the context; b. the last hypothesis has to be considered confirmed by the behaviour of the student shown by SI and $3: (s)he binds &quot;come&quot; to &quot;comme&quot; irrespective of the contexts~ probably because of the interference between the two conjunctions.</Paragraph> <Paragraph position="13"> We call this misconception Direct Translation.</Paragraph> </Section> <Section position="7" start_page="136" end_page="136" type="metho"> <SectionTitle> IV CONCLUSIONS </SectionTitle> <Paragraph position="0"> ELISA was a testbed for Intelligent Teaching</Paragraph> </Section> class="xml-element"></Paper>