<?xml version="1.0" standalone="yes"?>
<Paper uid="A94-1043">
  <Title>An Interactive Rewriting Tool for Machine Acceptable Sentences</Title>
  <Section position="6" start_page="207" end_page="207" type="evalu">
    <SectionTitle>
4. Experiment and Evaluation
</SectionTitle>
    <Paragraph position="0"> The rewriting tool has been evaluated from two aspects: the knowledge which forms the basis of our pre-editing rules and the tool as a whole, including the interface.</Paragraph>
    <Section position="1" start_page="207" end_page="207" type="sub_section">
      <SectionTitle>
4.1 Evaluation of Rewriting Rules
</SectionTitle>
      <Paragraph position="0"> We have carried out an experiment on the evaluation of our rewriting knowledge using a new test text(211 sentences). The experiment showed almost the same result(62% for the precision ratio and 90% for the recall ratio) as is in Section 3.1.</Paragraph>
      <Paragraph position="1"> To evaluate the generation of rewriting candidates in terms of rewriting positions and rewriting expressions, the precision ratio and the recall ratio of these two have been calculated for the target text, focusing on the sentences for which the tool produced rewriting candidates (i.e., type 1) in Section 3.2.</Paragraph>
      <Paragraph position="2"> The precision ratio and the recall ratio for identifying division points are 63% and 100% respectively; the ratios for the rewriting candidates are 10.5% and 93% respectively. The recall ratios for both division points and rewriting expressions exceed 90%, which means that the probability of obtaining the correct positions and candidates are sufficiently high. The precision ratio for substitution generation is low, but 5 to 6 candidates per division point would not be much of a burden on the user.</Paragraph>
    </Section>
    <Section position="2" start_page="207" end_page="207" type="sub_section">
      <SectionTitle>
4.2 Evaluation of the Operating Cost
</SectionTitle>
      <Paragraph position="0"> A preliminary experiment was made on five subjects to evaluate an overall appraisal, including the interface, the rate of reduction in operation time and improvement in tim quality of rewriting.</Paragraph>
      <Paragraph position="1"> The rate of reduction in rewriting time for the five subjects averages 23%. The recall ratio went up from 83% to 96%, the precision ratio from 96% to 99%. For those subjects whose recall ratio scored high when the tool was not used, the rate of reduction in time also tends to be high. For others whose recall ratio is low when the tool was not used, the rate of reduction in time does not change much, but the recall ratio improved by far. That is, users capable of rewriting without outside help can further shorten the total time using the tool. Moreover, those with low rewriting skills can benefit from the tool to improve rewriting quality.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>