<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-3032">
  <Title>Dyna: A Declarative Language for Implementing Dynamic Programs[?]</Title>
  <Section position="7" start_page="0" end_page="0" type="relat">
    <SectionTitle>
6 Related Work
</SectionTitle>
    <Paragraph position="0"> This project tries to synthesize much folk wisdom. For NLP algorithms, three excellent longer papers have atrithms (e.g., neural networks, constraint programming, clustering, and dynamic graph algorithms). However, it introduces several interesting design complications in the Dyna language and the implementation.</Paragraph>
    <Paragraph position="1"> 12Dyna spends most of its time manipulating hash tables and the priority queue. Inference is very fast because it is compiled.</Paragraph>
    <Paragraph position="2"> 13The code size comparisons are rough ones, because of mismatches between the programs being compared.</Paragraph>
    <Paragraph position="3">  1. need(s,0) = 1. % begin by looking for an s that starts at position 0 2. constit(Nonterm/Needed,I,I) += SIDE(need(Nonterm,I)) * rewrite(Nonterm,Needed). % traditional predict step 3. constit(Nonterm/Needed,I,K) += constit(Nonterm/cons(W,Needed),I,J) * word(W,J,K). % traditional scan step 4. constit(Nonterm/Needed,I,K) += constit(Nonterm,cons(X,Needed),I,J) * constit(X/nil,J,K). % traditional complete step 5. goal += constit(s/nil,0,N) * end(N). % we want a complete s constituent covering the sentence 6. need(Nonterm,J) += constit( /cons(Nonterm, ), ,J). % Note: underscore matches anything (anonymous wildcard)  np constituent that is still missing the list of subconstituents in Needed. In particular, np/nil is a complete np. (A list [n,pp] is encoded here as cons(n,cons(pp,nil)), although syntactic sugar for lists is also available.) need(np,3) is derived if some partial constituent seeks an np subconstituent starting at position 3. As usual, probabilistic, agenda-based lattice parsing comes for free, as does training.</Paragraph>
    <Paragraph position="4"> tempted similar syntheses (though without covering variant search and storage strategies, which Dyna handles).</Paragraph>
    <Paragraph position="5"> Shieber et al. (1995) (already noting that &amp;quot;many of the ideas we present are not new&amp;quot;) showed that several unweighted parsing algorithms can be specified in terms of inference rules, and used Prolog to implement an agenda-based interpreter for such rules. McAllester (1999) made a similar case for static analysis algorithms, with a more rigorous discussion of indexing the chart.</Paragraph>
    <Paragraph position="6"> Goodman (1999) generalized this line of work to weighted parsing, using rules of the form c += a1*a2*****ak (with side conditions allowed); he permitted values to fall in any semiring, and generalized the inside-outside algorithm. Our approach extends this to a wider variety of processing orders, and in particular shows how to use a prioritized agenda in the general case, using novel algorithms. We also extend to a wider class of formulas (e.g., neural networks).</Paragraph>
    <Paragraph position="7"> The closest implemented work we have found is PRISM (Zhou and Sato, 2003), a kind of probabilistic Prolog that claims to be efficient (thanks to tabling, compilation, and years of development) and can handle a subset of the cases described by Goodman. It is interesting because it inherits expressive power from Prolog. On the other hand, its rigid probabilistic framework does not permit side conditions (Fig. 2), general semirings (Goodman), or general formulas (Dyna). PRISM does not currently seem practical for statistical NLP research: in CKY parsing tests, it was only able to handle a small fraction of the Penn Treebank ruleset (2400 high-probability rules) and tended to crash on sentences over 50 words. Dyna, by contrast, is designed for real-world use: it consistently parses over 10x faster than PRISM, scales to full-sized problems, and attempts to cover real-world necessities such as prioritization, bottom-up inference, pruning, smoothing, underflow avoidance, maxent, non-EM optimization techniques, etc.</Paragraph>
  </Section>
class="xml-element"></Paper>