<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-1039">
  <Title>Exponential Priors for Maximum Entropy Models</Title>
  <Section position="7" start_page="2" end_page="2" type="concl">
    <SectionTitle>
6 Conclusion
</SectionTitle>
    <Paragraph position="0"> We have shown that an exponential prior for maxent models leads to a simple update formula that is easy to implement, and to models that are easy to understand: observations are discounted, subject to the constraint that l [?] 0. We have also shown that in at least one case, this prior better matches the underlying model, and that for two applications, it leads to improved accuracy. The prior also inspired an improved version of Good-Turing smoothing with lower perplexity. Finally, an exponential prior explains why models that discount by a constant can be Bayesian, giving an alternative to Dirichlet priors which add a constant. This helps justify Kneser-Ney smoothing, the best performing smoothing technique in language modeling. In the future, we would like to use our technique of examining the distribution of model parameters to see if other problems exhibit other priors besides Gaussian and Laplacian/exponential, and if performance on those problems can be improved through this observation.</Paragraph>
  </Section>
class="xml-element"></Paper>