<?xml version="1.0" standalone="yes"?>
<Paper uid="W97-0306">
  <Title>Mistake-Driven Learning in Text Categorization</Title>
  <Section position="7" start_page="61" end_page="62" type="concl">
    <SectionTitle>
6 Conclusions
</SectionTitle>
    <Paragraph position="0"> Theoretical analyses of the Winnow family of algorithms have predicted an exceptional ability to deal with large numbers of features and to adapt to new trends not seen during training. Until recently, these properties have remained largely undemonstrated.</Paragraph>
    <Paragraph position="1"> We have shown that while these algorithms have many advantages there is still a lot of room to explore when applying them to a real-world problem.</Paragraph>
    <Paragraph position="2"> In particular, we have demonstrated (1) how variation in document length can be tolerated through either normalization or negative weights, (2) the positive effect of applying a threshold range in training,  (3) alternatives in considering feature frequency, and (4) the benefits of discarding irrelevant features as  part of the training algorithm. The main contribution of this work, however, is that we have presented an algorithm, BalancedWinnow +, which performs significantly better than any other algorithm tested on these tasks using unigram features.</Paragraph>
    <Paragraph position="3"> We have exhibited that, as expected, multiplicative-update algorithms have exceptionally good behavior in high dimensional feature spaces, even in the presence of irrelevant features. One advantage this important property has is that is allows one to decompose the learning problem from the feature selection problem. Using this family of algorithms frees the designer from the need to choose the appropriate set of features ahead of time: A large set of features can be used and the algorithm will eventually discard those that do not contribute to the accuracy of the classifier. While we have chosen in this study to use a fairly simple set of features, it is straight forward to plug in instead a richer set of features. We expect that this will further improve the results of the algorithm, although further research is  needed on policies of discarding features and avoidance of over-fitting. In conclusion, we suggest that the demonstrated advantages of the Winnow-family of algorithms make it an appealing candidate for further use in this domain.</Paragraph>
  </Section>
class="xml-element"></Paper>