<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0618">
  <Title>An Information-Theoretic Empirical Analysis of Dependency-Based Feature Types for Word Prediction Models</Title>
  <Section position="7" start_page="145" end_page="145" type="concl">
    <SectionTitle>
5. Conclusion
</SectionTitle>
    <Paragraph position="0"> We have described a series of corpus-based analyses that take a Chinese treebank and quantify the information gain and the information redundancy for various feature types combinations involving both dependency and bigram feature types. The analysis yields several interesting conclusions that explain linguistic observations from an information theoretic point of view, and in addition will find practical use in the design of language models. Although perhaps we have been aware of some of the observations to varying extents, here we introduce a methodology that uses concrete evidence drawn from real contexts in order to give more reliable and objective results.</Paragraph>
    <Paragraph position="1"> We have already begun conducting similar experiments on an English training corpus \[61, which so far yield the same types of behavior described in this paper. We aim to discover which, if any, claims about the information present in dependency based features are peculiar to Chinese language, which are peculiar to English, and which are common across multiple languages.</Paragraph>
    <Paragraph position="2"> Based on the analysis, we will design, construct, and incrementally refine new language models for written and spoken English and Chinese that incorporate varying levels of linguistic structure. These models will aim to capture regularities that arise from long-distance dependencies, which n-gram models cannot represent. At the same time, we will retain as many of the n-gram parameters as needed to capture important lexical dependencies.</Paragraph>
  </Section>
class="xml-element"></Paper>