<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3235">
  <Title>Error Measures and Bayes Decision Rules Revisited with Applications to POS Tagging</Title>
  <Section position="7" start_page="0" end_page="0" type="concl">
    <SectionTitle>
6 Conclusion
</SectionTitle>
    <Paragraph position="0"> So far, the experimental tests have shown no improvement when we use the Bayes decision rule for minimizing the number of symbol errors rather than the number of string errors. However, the important result is that the new approach results in comparable performance. More work is needed to contrast the two approaches.</Paragraph>
    <Paragraph position="1"> The main purpose of this paper has been to show that, in addition to the widely used decision rule for minimizing the string errors, it is possible to derive a decision rule for minimizing the number of symbol errors and to build up the associated mathematical framework.</Paragraph>
    <Paragraph position="2"> There are a number of open questions for future work: 1) The error rates for the two decision rules are comparable. Is that an experimental coincidence? Are there situations for which we must expect a significance difference between the two decision rules? We speculate that the two decision rules could always have similar performance if the error rates are small.</Paragraph>
    <Paragraph position="3"> 2) Ideally, the training criterion should be closely related to the error measure used in the decision rule. Right now, we have used the training criteria that had been developed in the past and that had been (more or less) designed for the string error rate as error measure. Can we come up with a training criterion tailored to the symbol error rate? 3) In speech recognition and machine translation, more complicated error measures such as the edit distance and the BLEU measure are used. Is it possible to derive closed-form Bayes decision rules (or suitable analytic approximations) for these error measures? What are the implications?</Paragraph>
  </Section>
class="xml-element"></Paper>