<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1619">
  <Title>Extremely Lexicalized Models for Accurate and Fast HPSG Parsing</Title>
  <Section position="7" start_page="160" end_page="161" type="evalu">
    <SectionTitle>
5 Discussion
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="160" end_page="161" type="sub_section">
      <SectionTitle>
5.1 Supertagging
</SectionTitle>
      <Paragraph position="0"> Our probabilistic model of lexical entry selection can be used as an independent classifier for selecting lexical entries, which is called the supertagger (Bangalore and Joshi, 1999; Clark and Curran, 2004b). The CCG supertagger uses a maximum entropy classifier and is similar to our model.</Paragraph>
      <Paragraph position="1"> We evaluated the performance of our probabilistic model as a supertagger. The accuracy of the resulting supertagger on our development set (Section 22) is given in Table 5 and Table 6. The test sentences were automatically POS-tagged. Results of other supertaggers for automatically ex- null numbers under &amp;quot;test data&amp;quot; are the PTB section numbers of the test data.</Paragraph>
      <Paragraph position="2"> g tags/word word acc. (%) sentence acc. (%)  tracted lexicalized grammars are listed in Table 5. Table 6 gives the average number of supertags assigned to a word, the per-word accuracy, and the sentence accuracy for several values of g, which is a parameter to determine how many lexical entries are assigned.</Paragraph>
      <Paragraph position="3"> When compared with other supertag sets of automatically extracted lexicalized grammars, the (effective) size of our supertag set, 1,361 lexical entries, is between the CCG supertag set (398 categories) used by Curran and Clark (2003) and the LTAG supertag set (2920 elementary trees) used by Shen and Joshi (2003). The relative order based on the sizes of the tag sets exactly matches the order based on the accuracies of corresponding supertaggers. null</Paragraph>
    </Section>
    <Section position="2" start_page="161" end_page="161" type="sub_section">
      <SectionTitle>
5.2 Efficacy of extremely lexicalized models
</SectionTitle>
      <Paragraph position="0"> The implemented parsers of models 1 and 2 were around four times faster than the previous model without a loss of accuracy. However, what surprised us is not the speed of the models, but the fact that they were as accurate as the previous model, though they do not use any phrase-structure-based probabilities. We think that the correct parse is more likely to be selected if the correct lexical entries are assigned high probabilities because lexical entries include specific information about subcategorization frames and syntactic alternation, such as wh-movement and passivization, that likely determines the dominant structures of parse trees. Another possible reason for the accuracy is the constraints placed by unification-based grammars. That is, incorrect parse trees were suppressed by the constraints.</Paragraph>
      <Paragraph position="1"> The best performer in terms of speed and accuracy was model 3. The increased speed was, of course, possible for the same reasons as the speeds of models 1 and 2. An unexpected but very impressive result was the significant improvement of accuracy by two points in precision and recall, which is hard to attain by tweaking parameters or hacking features. This may be because the phrase structure information and lexical information complementarily improved the model.</Paragraph>
      <Paragraph position="2"> The lexical information includes more specific information about the syntactic alternation, and the phrase structure information includes information about the syntactic structures, such as the distances of head words or the sizes of phrases.</Paragraph>
      <Paragraph position="3"> Nasr and Rambow (2004) showed that the accuracy of LTAG parsing reached about 97%, assuming that the correct supertags were given. We exemplified the dominance of lexical information in real syntactic parsing, i.e., syntactic parsing without gold-supertags, by showing that the probabilities of lexical entry selection dominantly contributed to syntactic parsing.</Paragraph>
      <Paragraph position="4"> The CCG supertagging demonstrated fast and accurate parsing for the probabilistic CCG (Clark and Curran, 2004a). They used the supertagger for eliminating candidates of lexical entries, and the probabilities of parse trees were calculated using the phrase-structure-based model without the probabilities of lexical entry selection. Our study is essentially different from theirs in that the probabilities of lexical entry selection have been demonstrated to dominantly contribute to the disambiguation of phrase structures.</Paragraph>
      <Paragraph position="5"> We have not yet investigated whether our results can be reproduced with other lexicalized grammars. Our results might hold only for HPSG because HPSG has strict feature constraints and has lexical entries with rich syntactic information such as wh-movement.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>