<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1708">
  <Title>Automatic Measuring of English Language Proficiency using MT Evaluation Technology</Title>
  <Section position="7" start_page="0" end_page="0" type="concl">
    <SectionTitle>
5 Conclusion
</SectionTitle>
    <Paragraph position="0"> We proposed an automatic measurement method for English language proficiency. The proposed method applies automatic MT evaluation to measure human English language proficiency. This method focuses on measuring the communicative skill of structuring sentences, which is indispensable in writing and speaking.</Paragraph>
    <Paragraph position="1"> However, it does not measure elementary capabilities such as vocabulary and grammar. The method also involves a new test sentence selection scheme to enable efficient testing.</Paragraph>
    <Paragraph position="2"> In the experiments, we used TOEIC as an objective measure of English language proficiency. We then applied some currently available automatic evaluation methods: BLEU, NIST and a DP-based method. We carried out experiments on two test sets: BTEC and SLTA1. According to the experimental results, the proposed method gave a good measurement result on a small-sized test set. The standard error of measurement is around 120 points on the TOEIC score with BTEC and less than 100 TOEIC points score with SLTA1. In both cases, the optimum size of the test set is 30 to 60 test sentences. null The proposed method still needs human labour to make the references. To obtain higher portability, we will apply an automatic paraphrase scheme (Finch et al., 2002; Shimohata and Sumita, 2002) to make the references automatically. null</Paragraph>
  </Section>
class="xml-element"></Paper>