<?xml version="1.0" standalone="yes"?>
<Paper uid="H89-2041">
  <Title>TIED MIXTURES IN THE LINCOLN ROBUST CSR 1</Title>
  <Section position="10" start_page="297" end_page="297" type="ackno">
    <SectionTitle>
EVALUATION TESTS RESULTS
</SectionTitle>
    <Paragraph position="0"> Immediately after the February 89 meeting, a bug was found in the recognition network generation software for the WCD models. (This bug only affected our SD WCD system.) The fix was not a change in concept, only a correction of the implementation. The development test results are shown in Table 1 as &amp;quot;'Feb89 WCD with bug&amp;quot; and &amp;quot;Feb89 WCD&amp;quot;. The comparisons in this paper have been made with the &amp;quot;Feb89 WCD&amp;quot; system because it is (barring other bugs) the implementation of the system described in the talk and paper \[10\]. These tests were rerun and filed with NIST.</Paragraph>
    <Paragraph position="1"> A summary of the February 89 SD evaluation test results with and without the bug are shown in  There appears to be a bias in the SD test data relative to other test data--the SD tests show significantly fewer insertions than deletions. The effect is very strong for the with grammar test where 10 of the 12 speakers showed no insertion errors. In contrast, only one speaker showed no deletion errors. The February 89 tests of the same SD system show a balance between the two forms of errors (Table 3). (There is a similar, but weaker, bias in the no grammar SD case. This weaker bias would not be noteworthy if the with grammar case did not call attention to it.) The insertion penalty, which controls this trade-off, was set for minimum word error rate on the development test data. Usually, this minimum occurs when the insertion and deletion error rates are similar.</Paragraph>
    <Paragraph position="2"> The skew observed here is large enough that the SD word error rate would probably be reduced if the insertion penalty were adjusted to match this test data. (The excess of deletion errors in the current SI tests also occurred in the February 89 tests and is, therefore, not noteworthy.)</Paragraph>
  </Section>
class="xml-element"></Paper>