<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1205">
  <Title>Look-Back and Look-Ahead in the Conversion of Hidden Markov Models into Finite State Transducers</Title>
  <Section position="7" start_page="11" end_page="11" type="evalu">
    <SectionTitle>
4 Experiments and Results
</SectionTitle>
    <Paragraph position="0"> This section compares different FSTs with each other and with the original ttMM.</Paragraph>
    <Paragraph position="1"> As expected, the FSTs perform tagging faster than the HMM.</Paragraph>
    <Paragraph position="2"> Since all FSTs are approximations of HMMs, they show lower tagging accuracy than the ttMMs. In the case of FSTs with fl &gt; 1 and a = 1, this difference in accuracy is negligible. Improvement in accuracy can be expected since these FSTs can be composed with FSTs encoding correction rules for frequent errors (sec. 1).</Paragraph>
    <Paragraph position="3"> For all tests below an English corpus, lexicon and guesser were used, which were originally annotated with 74 different tags. We automatically recoded the tags in order to reduce their number, i.e. in some cases more than one of the original tags were recoded into one and the same new tag. We applied different recodings, thus obtaining English corpora, lexicons and guessers with reduced tag sets of 45, 36, 27, 18 and 9 tags respectively.</Paragraph>
    <Paragraph position="4"> FSTs with fl= 2 and ~ = 1 and with fl= 1 and a = 2 were equivalent, in all cases where they could be computed.</Paragraph>
    <Paragraph position="5"> Table 1 compares different FSTs for a tag set of 36 tags.</Paragraph>
    <Paragraph position="6"> The b-type FST with no look-back and no look-ahead which is equivalent to an n0-type FST (Kempe, 1997), shows the lowest tagging accuracy (b-FST ()3=0, a=0): 87.21%). It is also the smallest transducer (1 state and 181 arcs, as many as tag classes) and can be created faster than the other FSTs (6 sec.).</Paragraph>
    <Paragraph position="7"> The highest accuracy is obtained with a b-type FST with fl= 2 and a = 1 (b-FST (/3=2,~=1): 97.34 %) and with an s-type FST (Kempe, 1997) trained on 1 000 000 words (s+nl-FST (1M, F1): 97.33 %). In these two cases the difference in accuracy with respect to the underlying ttMM (97.35 %) is negligible. In this particular test, the s-type FST comes out ahead because it is considerably smaller than the b-type FST.</Paragraph>
    <Paragraph position="8"> The size of a b-type FST increases with the size of the tag set and with the length of look-back plus look-ahead, ~+c~. Accuracy improves with growing b-Type FSTs may produce ambiguous tagging resuits (sec. 2.4)'. In such instances only the first result was retained (see. 3).</Paragraph>
    <Paragraph position="9"> Kempe 34 Look-Back and Look-Ahead in the Conversion of HMMs</Paragraph>
    <Paragraph position="11"> Tagging accuracy and agreement with the tIMM for tag sets of different sizes 297 cls. 214 cls. 181 cls. 119 els. 97 ds. 67 C/ls.  Multiple, i.e. ambiguous tagging results: Only first result retained Tagging accuracy of 97.06 %, and agreement of FST with HMM tagging results of 99.98 % Transducer could not be computed, for reasons of size.  of the underlying HMM, for tag sets of different sizes Table 2 shows the tagging accuracy and the agreement of the tagging results with the results of the underlying HMM for different FSTs and tag sets of different sizes.</Paragraph>
    <Paragraph position="12"> To get results that are almost equivalent to those of an HMM, a b-type FST needs at least a look-back of/5 = 2 and a look-ahead of a = 1 or vice versa. For reasons of size, this kind of FST could only be computed for tag sets with 36 tags or less. A b-type FST with/5 = 3 and a = 1 could only be computed for the tag set with 9 tags. This FST gave exactly the same tagging results as the underlying HMM.</Paragraph>
    <Paragraph position="13"> Table 3 illustrates which of the b-type FSTs are sequential, i.e. always produce exactly one tagging result, and which of the FSTs are non-sequential. For all tag sets, the FSTs with no look-back (/5 = 0) and/or no look-ahead (a = 0) behaved sequentially. Here 100 % of the tagged sentences had only one result. Most of the other FSTs (/5. o~ &gt; 0) behaved non-sequentially. For example, in the case of 27 tags withl3=l anda=l, 90.08%of the tagged sentences had one result, 9.46 % had two results, 0.23 % had tree results, etc.</Paragraph>
    <Paragraph position="14"> Non-sequentiality decreases with growing look-back and look-ahead,/5+c~, and should completely disappear with sufficiently large/5+~. Such b-type FSTs can, however, only be computed for small tag sets. We could compute this kind of FST only for the case of 9 tags with/5=3 and a=l.</Paragraph>
    <Paragraph position="15"> The set of alternative tag sequences for a sentence, produced by a b-type FST with/5, a &gt; 0, always contains the tag sequence that corresponds with the result of the underlying HMM.</Paragraph>
  </Section>
  <Section position="8" start_page="11" end_page="11" type="evalu">
    <SectionTitle>
5 Conclusion and Future Research
</SectionTitle>
    <Paragraph position="0"> The algorithm presented in this paper describes the construction of a finite-state transducer (FST) that approximates the behaviour of a Hidden Markov Model (HMM) in part-of-speech tagging.</Paragraph>
    <Paragraph position="1"> The algorithm, called b-type approximation, uses look-back and look-ahead of freely selectable length. The size of the FSTs grows with both the size of the tag set and the length of the look-back plus lookahead. Therefore, to keep the FST at a computable size, an increase in the length of the look-back or look-ahead, requires a reduction of the number of tags. In the case of small tag sets (e.g. 36 tags), the look-back and look-ahead can be sufficiently large to obtain an FST that is almost equivalent to the original HMM.</Paragraph>
    <Paragraph position="2"> In some tests s-type FSTs (Kempe, 1997) and b-type FSTs reached equal tagging accuracy. In these cases s-type FSTs are smaller because they encode the most frequent ambiguity class sequences of a training corpus very accurately and all other sequences less accurately, b-Type FSTs encode all sequences with the same accuracy. Therefore, a b-type FST can reach equivalence with the original HMM, but an s-type FST cannot.</Paragraph>
    <Paragraph position="3"> The algorithms of both conversion and tagging are fully implemented.</Paragraph>
    <Paragraph position="4"> The main advantage of transforming an HMM is that the resulting FST can be handled by finite state calculus ~ and thus be directly composed with other FSTs.</Paragraph>
    <Paragraph position="5"> The tagging speed of the FSTs is up to six times higher than the speed of the original HMM.</Paragraph>
    <Paragraph position="6"> Future research will include the composition of HMM transducers with, among others: * FSTs that encode correction rules for the most frequent tagging errors in order to significantly improve tagging accuracy (above the accuracy of the underlying HMM). These rules can either be extracted automatically from a corpus (Brill, 1992) or written manually (Chanod and Tapanalnen, 1995).</Paragraph>
    <Paragraph position="7"> * FSTs for light parsing, phrase extraction and other text analysis (Ait-Mokhtar and Chanod, 1997).</Paragraph>
    <Paragraph position="8"> An HMM transducer can be composed with one or more of these FSTs in order to perform complex text analysis by a single FST.</Paragraph>
  </Section>
class="xml-element"></Paper>