<?xml version="1.0" standalone="yes"?>
<Paper uid="P95-1030">
  <Title>New Techniques for Context Modeling</Title>
  <Section position="6" start_page="226" end_page="226" type="concl">
    <SectionTitle>
4 Conclusion
</SectionTitle>
    <Paragraph position="0"> In ongoing work, we are investigating extension mixture models as well as improved model selection algorithms. An extension mixture model is an extension model whose ~(~lw) parameters are estimated by linearly interpolating the empirical probability estimates for all extensions that dominate w with respect to c~, ie., all extensions whose symbol is and whose context is a suffix of w. Extension mixing allows us to remove the uniform flattening of zero frequency symbols in our parameter estimates (5). Preliminary results are promising. The idea of context mixing is due to Jelinek and Mercer (1980).</Paragraph>
    <Paragraph position="1"> Our results highlight the fundamental tension between model complexity and data complexity. If the model complexity does not match the data complexity, then both the total codelength of the past observations and the predictive error increase. In other words, simply increasing the number of parameters in the model does not necessarily increase predictive power of the model. Therefore, it is necessary to have a a fine-grained model along with a heuristic model selection algorithm to guide the expansion of the model in a principled manner.</Paragraph>
    <Paragraph position="2"> Acknowledgements. Thanks to Andrew Appel, Carl de Marken, and Dafna Scheinvald for their critique. The paper has benefited from discussions with the participants of DCC95. Both authors are partially supported by Young Investigator Award IRI0258517 to the first author from the National Science Foundation. The second author is additionally supported by a tuition award from the Princeton University Research Board. The research was partially supported by NSF SGER IRI-9217208.</Paragraph>
  </Section>
class="xml-element"></Paper>