<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1055">
  <Title>Skeletons in the parser: Using a shallow parser to improve deep parsing</Title>
  <Section position="7" start_page="5" end_page="6" type="evalu">
    <SectionTitle>
4.2 Results
</SectionTitle>
    <Paragraph position="0"> Results show significant improvements in the speed of parsing. Table 3 shows the exact match sentence accuracy and timing results for parsing with and without skeletons with a maximum chart size of 10000. The first row shows how many utterances of 6 words or longer were parsed in each dialog.</Paragraph>
    <Paragraph position="1"> The next two rows show exact match sentence accuracy results for parses obtained with and without  maximum chart size 10000 for utterances of 6 or more words.</Paragraph>
    <Paragraph position="2"> skeletons. The next two rows show the total time (in seconds) to parse the dialogs with and without the skeletons. The last row shows the speed up factor (computed as time-without-skeletons/time-withskeletons). null  We see substantial speed-ups in the parser using this technique. The parser using skeletons completed the parses in less than half of the time of the original parser. Looking at individual utterances, 70% were parsed more quickly with the skeletons, while 25% were slower. Overall, our simple approach appears to provide a substantial payoff in speed along with a small improvement in accuracy. Note that we use a strict criterion for accuracy, so both the correct logical form as well as the correct syntactic structure must be computed by the parser for an analysis to be considered correct in our evaluation. A correct logical form requires correct word sense disambiguation, constituent dependencies, and semantic role assignment (see section 2.2). For example, in some cases the parser produces a structurally correct parse, but selects an inappropriate word sense, in which case the analysis is considered incorrect. One such case is the utterance You know where the little loop is, in which the where is assigned the sense TO-LOC (which should only be used for trajectories, as in Where did he go), when in this utterance the correct sense for where is SPATIAL-LOC.</Paragraph>
    <Paragraph position="3"> To explore the question of how much of the speed increase is the result of time spent on difficult sentences that cause the parser to reach the maximum chart size limit, we performed the same experiment with a smaller maximum chart size of 5000, shown in Table 4. As expected the speed-up gain declined to 1.8, still quite a respectable gain, and again there  These experiments were run with CMU Common LISP 18e and a Linux 2.4.20 kernel on a 2 GHz Xeon dual processor  maximum chart size 1500 for utterances of 6 more words.</Paragraph>
    <Paragraph position="4"> is no loss of accuracy.</Paragraph>
    <Paragraph position="5"> As we drop the chart size to 1500, the speed-up drops to just 1.4, as shown in Table 5. However, we have improvements in accuracy using skeletons when we parse with low upper limits. In certain cases the skeleton guides the parser to the correct parse more quickly, so it can be found even when the maximum chart size is reduced. For example, for the utterance And meanwhile we send two ambulances from the Strong Hospital to take the six wounded people from the airport (from dialog 1), a correct full sentence analysis is found with the larger maximum chart sizes (5000 or more), but with a maximum chart size of 1500 the correct analysis for this utterance is found only with the help of the skeleton.</Paragraph>
    <Paragraph position="6"> Our best results are similar to those reported in (Frank et al., 2002), who show a speed-up factor of 2.26, although they use a much larger maximum chart size (70,000). Because of the differences in grammars and parsers, it is not clear how to fairly compare the chart sizes.</Paragraph>
  </Section>
class="xml-element"></Paper>