<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3224">
  <Title>A Distributional Analysis of a Lexicalized Statistical Parsing Model</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Motivation
</SectionTitle>
    <Paragraph position="0"> A parsing model coupled with a decoder (an algorithm to search the space of possible trees for a given terminal sequence) is largely an engineering e ort. In the end, the performance of the parser with respect to its evaluation criteria--typically accuracy, and perhaps also speed--are all that matter.</Paragraph>
    <Paragraph position="1"> Consequently, the engineer must understand what the model is doing only to the point that it helps make the model perform better. Given the somewhat crude method of determining a feature's benefit by testing a model with and without the feature, a researcher can argue for the e cacy of that feature without truly understanding its e ect on the model. For example, while adding a particular feature may improve parse accuracy, the reason may have little to do with the nature of the feature and everything to do with its canceling other features that were theretofore hurting performance. In any case, since this is engineering, the rationalization for a feature is far less important than the model's overall performance increase.</Paragraph>
    <Paragraph position="2"> On the other hand, science would demand that, at some point, we analyze the multitude of features in a state-of-the-art lexicalized statistical parsing model. Such analysis is warranted for two reasons: replicability and progress. The first is a basic tenet of most sciences: without proper understanding of what has been done, the relevant experiment(s) cannot be replicated and therefore verified. The second has to do with the idea that, when a discipline matures, it can be di cult to determine what new features can provide the most gain (or any gain, for that matter). A thorough analysis of the various distributions being estimated in a parsing model allows researchers to discover what is being learned most and least well. Understanding what is learned most well can shed light on the types of features or dependencies that are most e cacious, pointing the way to new features of that type. Understanding what is learned least well defines the space in which to look for those new features.</Paragraph>
  </Section>
class="xml-element"></Paper>