<?xml version="1.0" standalone="yes"?>
<Paper uid="W95-0108">
  <Title>Beyond Word N-Grams</Title>
  <Section position="10" start_page="103" end_page="104" type="concl">
    <SectionTitle>
6 Conclusions and Further Work
</SectionTitle>
    <Paragraph position="0"> PSTs are able to capture longer correlations than traditional fixed order n-grams, supporting better generalization ability from limited training data. This is especially noticeable when phrases longer than a typical n-gram order appear repeatedly in the text. The PST learning algorithm allocates a proper node for the phrase whereas a bigram or trigram model captures only atruncated version of the statistical dependencies among words in the phrase.</Paragraph>
    <Paragraph position="1"> Our current learning algorithm is able to handle moderate size corpora, but we hope to adapt it to work with very large training corpora (100s of millions of words). The main obstacle to those applications is the space required for the PST. More extensive pruning may be useful for such large training sets, but the most promising approach may involve a batch training algorithm that builds a compressed representation of the PST final from an efficient representation, such as a suffix array, of the relevant subsequences of the training corpus.</Paragraph>
    <Paragraph position="2">  Negative Log. Likl. Posterior Probability from god and over wrath grace shall abound 74.125 0.642 from god but over wrath grace shall abound from god and over worth grace shall abound from god and over wrath grace will abound before god and over wrath grace shall abound from god and over wrath grace shall a bound from god and over wrath grape shall abound</Paragraph>
  </Section>
class="xml-element"></Paper>