<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1710">
  <Title>The Syntax Student's Companion: an eLearning Tool designed for (Computational) Linguistics Students</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Motivating reasons for the
</SectionTitle>
    <Paragraph position="0"> creation of the program Supervised time in university courses tends moretodiminishthantoaugment. Itishowever particularly crucial in introductory courses that students can get a good grasp of the concepts by regular supervised practice. Exercise sheets are often found useful only if su-cient time in the classroom can be devoted to go throught all the difierent subtleties encoded in the exercises. It can therefore be advantageous to ofier students a means to practise outside of the classroom, while still being able to ask their teachers for help. There are a number of criteria that should be taken into account when designing a computer program for supporting this kind of practice, including the following: + The program should be attractive to students. It is well known in computer engineering that good programs can end up not being used if the user was not taken into consideration from the very beginning of the engineering process. Students are a particular kind of users who may not be willing to use programs that are tedious or overcomplicated to use, and seen as not helpful as a result.</Paragraph>
    <Paragraph position="1"> + Teachers should have the feeling that they can control what the program does. Not only should it be simple for teachers to add new data conforming to predeflned exercise types, but it should also be possible to extend the program.1 + The program should provide useful feedback to students. While it is probably the case that an asynchronous mode of practice whereby a student would do exercises on a computer and then send the results electronically to a teaching supervisor would yield good results in some con1It is not expected that teachers would write computer code themselves, but the program could be extended by means of predeflned building bricks or by the addition of code by a computer engineer with access to a clear application programming interface (API). texts, students will expect the program to assess their answers and possibly provide feedback, and therefore support self-study to some extent.</Paragraph>
    <Paragraph position="2"> + The use of the program should be independent from place and time. It is our own experience as teacher at the university level that a signiflcant proportion of post-2000 students prefer to work from home when they have this possibility. Booking computer rooms for practice for speciflc courses may work for some students, but certainly not for all of them.</Paragraph>
    <Paragraph position="3"> This said, supervised sessions with computers may still be a fruitful option. This, of course, further implies that the program should not be too costly for both the university and the students, if not free.</Paragraph>
    <Paragraph position="4"> When we flrst worked on the development of a program that would support the practice of syntax exercises, back in 1999, there were already programs in this area. Trees 2 2, developed at the University of Pennsylvania, allowed students to visually build syntactic trees, but in such a way that they could only be valid relative to the grammar used. Moreover, at the time the program could only be run locally on Macintosh computers and required the purchase of a licence. Syntactica3, developed at Sunny Brooks University, allowed students to build grammars and then ask the program to build the syntactic tree for them, which they could subsequently modify. Again, at the time the program only existed for NeXT computers.</Paragraph>
    <Paragraph position="5"> The free Java applet from the University of Bangor, The Syntax Tutor4, permitted a student to enter a set of context-free rules and to ask the system to parse a sentence with it.</Paragraph>
    <Paragraph position="6"> Except for the case of the The Syntax Tutor, these programs had to be bought, and could only be run on speciflc computer families. Nevertheless, their existence shows that there was a very promising trend, supported by encouraging evaluation (see e.g. (Larson, 1996; Phillips,  When designing the program, we had two types of considerations in mind, pedagogical and technical. The basic idea was to let students build syntactic trees in a simple way, and to edit or consult the underlying grammars. What seemed very important was to let the students the possibility to make errors, considering that trial and error, providing appropriate feedback is given, can be part of a sound learning process. Therefore, students should be able to draw syntactic trees that are not valid relative to a given grammar, which was given to them or was build by them, and was accessible and modiflable or hidden. The syntactic theory used would initially be the X' theory5, and the types of exercises would include the drawing of ambiguous sentences based on some data, and the modiflcation of existing trees to illustrate syntactic transformations.</Paragraph>
    <Paragraph position="7"> Technical considerations included the fact that the program should be runnable anywhere and on any computer family. The Java programming language (Sun Microsystems, 1995) was the obvious choice, as it was already quite mature and could be run over the Internet on any platform that had a Java virtual machine.</Paragraph>
    <Paragraph position="8"> Furthermore, a Java program can exist in two avors, as an application that can be installed and run locally on a personal computer, and as an applet that can be downloaded at execution time over the Internet and run by the virtual machine of a web browser installed on computers of a university department without any installation nor maintenance.</Paragraph>
    <Paragraph position="9"> Furthermore, exercises and resources for the program had to be modiflable. For a local use with the application version, the user should be able to create new exercises using a simple description language. For a distributed use with the applet version, the administrator of the website where the applet is hosted should be able to add resources that would be immediately accessible to all the remote users. Modiflable resources include grammars, trees, exercise deflnitions, and language resource flles for running the program in the language of the user. XML (W3C, 2000) was chosen as the format for most of the resources, and a simple  schema was designed to allow the creation of new resources. It was initially believed that this provided a simple way of creating new resources and modifying existing ones.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Presentation of the program
</SectionTitle>
    <Paragraph position="0"> Our program is called the Syntax Student's Companion. Figure 1 shows its main interface running in English.6 The top panel contains the active grammar (Simple CFG for English in the example), a button to launch the grammar editor, the active mode (Free drawing mode) and a button to switch to the exercise mode.</Paragraph>
    <Paragraph position="1"> The panel on the left contains buttons for all the nonterminal and terminal categories of the active grammar, and a list for the words in the lexicon. The main panel is a scrollable zone called the workspace where trees can be drawn. Menus contain commands relative to the customization of the program, user modes, grammars, and trees.</Paragraph>
    <Paragraph position="2">  workspace subtrees corresponding to partial structures described in the grammar used. We plan to add this feature in the next version of the program, as it allows students to concentrate on more advanced notions.  will become a subtree of another tree should be selected with the mouse (1), and dragged onto the node that will become its mother (2). If that node has not any children yet, then the attachment is done. Otherwise, the user has to select the position of the new subtree among the daughters of its mother (3). When the position has been chosen, the attachment is done, and the new layout of the tree is produced (4), so as to ensure that the trees are always well-balanced.8 Alternatively, categories and words can be directly dropped onto the workspaces as children of existing nodes. Trees or subtrees can be copied and pasted onto the workspace, allowing faster construction. To detach a subtree, the root of the subtree should simply be dragged away from its parent tree.</Paragraph>
    <Paragraph position="3"> Trees and subtrees can also be removed from the workspace by using the rubber tool.</Paragraph>
    <Paragraph position="4"> All these adjunction operations can be done regardless of the rules deflned in the active grammar. Therefore, students may make errors and be aware of them only after they try to validate their trees with the active grammar.</Paragraph>
    <Paragraph position="5"> Indeed, contexts where students could use a tree drawing application with grammars designed in such a way that irrelevant errors were not possible revealed in some cases that the students had become too dependent on the helping hand of the program and were not able to perform as well without it (Phillips, 1998). The current version only supports simple context-free grammars. Grammars can either come from a remote or a local flle, or they can be created from scratch by the student.</Paragraph>
    <Paragraph position="6"> The grammar editor (see flgure 3) allows the 8We are aware that some textbooks use trees with upright lefthand branches and sloping righthand branches, so we will add this possibility as a new parameter. Likewise, we will allow trees to be built bottom up, with all the words of a sentence aligned horizontally.</Paragraph>
    <Paragraph position="7">  context-free grammars consultation and modiflcation of the current grammar. It shows all the derivation rules corresponding to a given nonterminal category9, and allows specifying of how they are presented on the window of the main interface.</Paragraph>
    <Paragraph position="8"> Once students have built trees, they can ask the program to check their validity according to the active grammar. If the active grammar is modiflable, they can modify it so as to ensure that the coverage of the grammar include their trees. If the active grammar is hidden (i.e. not accessible), the validation of their trees indicates whether they conform to an 9In the presented implementation, lexical categories appear as just any other nonterminal categories in the grammar editor dialog box, but that may be confusing for students. We therefore think that the lexicon should be distinguished from the grammar itself, as it is done on the left panel of the main interface (see flgure 1).  implicit grammar speciflcation (such as one that would have been described during lecture sessions). Tree nodes that violate the rules of the grammar are shown in red, and passing the mouse cursor over them displays a message indicating the nature of the error, as illustrated in flgure 4, subflgure (3).</Paragraph>
    <Paragraph position="9"> Three modes of exercises have been deflned and can be encoded in XML resource flles. The drawing of non-ambiguous trees requires the student to draw the tree for a given sentence using a given grammar, whereby the analysis of the sentence is unambiguous. An example of such an exercise encoded into XML format is given in flgure 5 for the Spanish phrase convocatoria de proyectos de innovaciPon educativa. Figure 6 illustrates the ambiguous tree drawing exercise type. The student is asked to draw the syntactic tree for a sentence (Time ies like an arrow in this case) given several data that permit to disambiguate the sentence and flnd the correct syntactic derivation. The last type of exercise asks students to modify trees (see flgure 7) to re ect syntactic transformations.</Paragraph>
    <Paragraph position="10"> Instead of asking the student to draw the syntactic tree for the sentence (in the example, Who will come tomorrow?), she is provided with a base tree (in the example, the tree for the sentence Bobby-Joe will come tomorrow), in order to better illustrate the transformations that take place.</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Initial evaluation
</SectionTitle>
    <Paragraph position="0"> As we are not ourselves involved in syntax teaching10, we have not been able to perform any formal evaluation of the presented version of the program. It is however crucial to be able to assess the efiectiveness of such a tool, both in terms of the type of help it gives to 10Our initial motivation was to ofier such a program to fellow students.</Paragraph>
    <Paragraph position="1">  the student and the support it provides to the teacher. The initial evaluation elements we have been able to gather from emails sent to us via the website of the project constitute the  basis for an updated speciflcation for the new version of the program that we will introduce in the next section.</Paragraph>
    <Paragraph position="2"> Several teachers have reported that they had used the program at some point in their teaching, but we suspect that in most cases the program was demonstrated to students (for example, using a data projector in the classroom), hoping that they would use it for self-study. The most important limitation user feedback told us was the di-culty to add new exercises for teachers. Only few people contributed exercises in XML format11, suggesting that this way of specifying resources was probably not adequate for linguistics teachers. Although the program can support any grammar theory based on context-free grammars, the default grammars made some users think that only the X' theory could be used, and some users had di-culty to see that the grammars could in fact be edited and totally new sets of categories deflned.</Paragraph>
    <Paragraph position="3"> Unsurprisingly, some teachers said they were interested in the support of feature structures. A not-so-expected use of the program was for producing graphical trees for inclusion into documents. This, corroborated with several user testimonies, seems to indicate that the program is considered easy to use. Its simplicity was in fact often mentionned as one of the preferred characteristics by students who used the program without any prior recommandation from a teacher. We also think that the availability of the program and its online user manual in several languages may have contributed to this.12 Some technical issues were also reported.</Paragraph>
    <Paragraph position="4"> Most users of the program, who are not supposed to be computer scientists, found it di-cult to set up the Java program and run it as an application. Moreover, some web browsers did not run the applet perfectly. The existing version of the program is based on the Java technology that existed in 1999, and the language is now more mature and better supported, so it is now simpler to set up a Java virtual machine on one's computer and to run Java programs, and support for Java in web browsers is much better than it used to be.</Paragraph>
    <Paragraph position="5"> As regards the evaluation we would like to be able to conduct, we believe that user questionnaires and logging of student activity would be good indicators of its efiectiveness. Also, it would be interesting to see if the use 11Some people may have written exercises of which we are not aware.</Paragraph>
    <Paragraph position="6"> 12Evaluation results for the Syntactica grammar workbench revealed that the use of this kind of computer-assisted instruction surprisingly increased the need for instructor support (Larson, 1996). We assume that this was partly due flrst to the number of functions of the program, as well as the fact that at the time linguistics students were for the most part new to the use of computers.</Paragraph>
    <Paragraph position="7"> of the program can make signiflcant difierences in the evaluation of the performance of student groups.</Paragraph>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
6 Current work
</SectionTitle>
    <Paragraph position="0"> We have specifled a new version of the program that will be partly developed by two Masters students during a computer engineering project. We present the main changes from the existing version in this section, and we conclude with some perspectives in the next section.</Paragraph>
    <Paragraph position="1"> First of all, the main lesson we can draw from user feedback is that no matter how much time is spent on speciflcation, not all features that would be useful to users could be imagined.</Paragraph>
    <Paragraph position="2"> Therefore, it seems a good idea that such a non-commercial program be extensible by other contributors who would like to add new features such as new exercise types, or support for other grammatical theories. The new version will have an OpenSource licence, which implies that we pay a particular attention to the genericity, modularity and documentation of the source code, and that the program will continue to be free to use, which seems essential to us.</Paragraph>
    <Paragraph position="3"> A bottleneck to a more widespread use of the program is certainly the di-culty to create new resources, mainly exercises. A particular mode for the deflnition of exercises will be integrated into the program. This mode will allow a teacher to describe an exercise and its solution in a way as similar as possible to the exercise mode itself. We also want to support the description of possible errors and their appropriate corrections and comments, in order to provide better feedback to students. Once the exercises are deflned, it would be possible to submit them to a repository on a web server, on a collaborative mode.13 A novel use of the applet version will allow using it inline in web pages, instead of as a separate application window. This will not only allow the dynamic drawing of tree descriptions specifled as parameters to the Java applet (and possibly tree animations), but also the insertion of exercises within online course material. We 13Collaborative projects, such as the Papillon project for multilingual lexical resources, show that this approach can work if submitters can also beneflt from the submissions of other contributors.</Paragraph>
    <Paragraph position="4"> plan to use this for the tutorial of the program.</Paragraph>
    <Paragraph position="5"> On the content side, several ideas have been submitted and will be implemented depending on time. Notably, it seems particularly interesting to provide actual linguistic data from corpora to students from which grammars can be inferred, as in (Borin and Dahllof, 1999). A new exercise type will ask students to write a grammar accounting for a given small corpus, which could already be morphologically annotated or not. Lexicons will be separated from grammars, in order to make them reusable when possible. Feature structures will also be supported, both for the edition of grammars and for the validation of syntactic derivations.</Paragraph>
    <Paragraph position="6"> A number of new features concern the graphical display of trees. Notably, it will be possible to collapse or expand subtrees (using the triangle notation), and to draw trees top-down with the terminal symbols immediately under the non-terminal that dominates them, or bottom-up with the terminal symbols aligned horizontally.14 It will also be possible to specify display properties (such as font and color) at the level of nodes and subtrees, and to export trees as bitmap flles for easy inclusion into documents like assignments and course notes.</Paragraph>
  </Section>
class="xml-element"></Paper>