<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-0208">
  <Title>Automatic Evaluation of Students' Answers using Syntactically Enhanced LSA</Title>
  <Section position="7" start_page="0" end_page="0" type="concl">
    <SectionTitle>
6 Conclusion
</SectionTitle>
    <Paragraph position="0"> Automatic evaluation of students' answers in an intelligent tutoring system can be performed using LSA. But LSA lacks syntactic information which can be also useful for meaning representation of a text document. So, we have developed and implemented a model called syntactically enhanced LSA which generalizes LSA by augmenting a word with the POS tag of the preceding word to derive a latent syntactic-semantic information. Experimental results on the AutoTutor task of evaluating students' answers to computer science questions show a range of performance comparison between SELSA and LSA. In terms of the correlation measure with human raters, LSA is slightly better than SELSA. But SELSA is at least as good as LSA in terms of the mean absolute difference measure. On the other end, SELSA is able to correctly evaluate a few more answers than LSA is. SELSA can do better if the training and testing corpora have a good syntactic structure.</Paragraph>
    <Paragraph position="1"> From the correlation performance analysis, it is observed that SELSA is more robust in discriminating the semantic information across a wider threshold width than LSA. It is also found that SELSA uses the syntactic information to expand the document similarity measure i.e., mere semantically similar documents are placed wider apart than syntactic-semantically similar documents in SELSA space.</Paragraph>
    <Paragraph position="2"> These initial results are part of an ongoing research towards an overall improvement of natural language understanding and modeling. Although the present version of SELSA has limited improvements over LSA, it leads to future experiments with robust characterization of syntactic neighbourhood in terms of headwords or phrase structure as well as applying smoothing across syntax to tackle the problem of sparse data estimation.</Paragraph>
  </Section>
class="xml-element"></Paper>