<?xml version="1.0" standalone="yes"?>
<Paper uid="J96-1002">
  <Title>Renaissance Technologies</Title>
  <Section position="7" start_page="65" end_page="66" type="concl">
    <SectionTitle>
6. Conclusion
</SectionTitle>
    <Paragraph position="0"> We began by introducing the building blocks of maximum entropy modeling--realvalued features and constraints built from these features. We then discussed the maximum entropy principle. This principle instructs us to choose, among all the models  Berger, Della Pietra, and Della Pietra A Maximum Entropy Approach consistent with the constraints, the model with the greatest entropy. We observed that this model was a member of an exponential family with one adjustable parameter for each constraint. The optimal values of these parameters are obtained by maximizing the likelihood of the training data. Thus two different philosophical approaches-maximum entropy and maximum likelihood--yield the same result: the model with the greatest entropy consistent with the constraints is the same as the exponential model which best predicts the sample of data.</Paragraph>
    <Paragraph position="1"> We next discussed algorithms for constructing maximum entropy models, concentrating our attention on the two main problems facing would-be modelers: selecting a set of features to include in a model, and computing the parameters of a model containing these features. The general feature-selection process is too slow in practice, and we presented several techniques for making the algorithm feasible.</Paragraph>
    <Paragraph position="2"> In the second part of this paper we described several applications of our algorithms, concerning modeling tasks arising in Candide, an automatic machine translation system under development at IBM. These applications demonstrate the efficacy of maximum entropy techniques for performing context-sensitive modeling.</Paragraph>
  </Section>
class="xml-element"></Paper>