<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2305">
  <Title>Robust Parsing: More with Less</Title>
  <Section position="5" start_page="30" end_page="31" type="concl">
    <SectionTitle>
6 Conclusions and future work
</SectionTitle>
    <Paragraph position="0"> We haveinvestigatedtheeffectofsystematicallyreducing the coverage of a general grammar of German. By removing support for 21 rare phenomena, the overall parsing accuracy could be improved.We confirmed the initial assumption about the effects that broad coverage has on the parser: while it allows some special sentences to be analysed more accurately, it also causes a slight decrease on the much more numerous normal sentences.</Paragraph>
    <Paragraph position="1"> This result shows that at least with respect to this particular grammar, more coverage can indeed lead to less parsing accuracy. In the first experiment we measured the overallloss throughadding coveragewhere it is not needed as about 0.4% of structural accuracy on newscast text, and 0.1% on NEGRA sentences. This figure can be interpreted as the result of overgenerating or 'leaking' of rare constructions into sentences where they are not wanted.</Paragraph>
    <Paragraph position="2"> Although we found that it makes little difference whether to remove support for very rare or for somewhat rare phenomena, judging constructions by how many leaks they actually cause leads to a greater improvement. On the NEGRA test set, removing the 'known troublemakers'leads to a greater increase of in accuracy of 0.4%, reducing the error rate for structural attachment by 4.2%.</Paragraph>
    <Paragraph position="3"> Of course, removing rare phenomena is not a viable technique to substantially improve parser accuracy, if only for the simple fact that it does not scale up. However, it confirms that as soon as a certain level of coverage has been reached, robustness, i.e. the ability to deal with unexpected data, is more crucial than coverage itself to achieve high quality results on unrestricted input.</Paragraph>
    <Paragraph position="4"> On the other hand, the improvementwe obtained is not  very large, compared to the already rather high over-all performance of the parser. This may be due to the consistent use of weighted constraints in the original grammar, which slightly disprefer many of the 21 phenomena even when they are allowed, and we assume that the original grammar is already reasonably effective at preventingleaks. This claim might be confirmed by reversing the experiment: if all phenomena were allowed and all dispreferences switched off, we would expect even more leaks to occur.</Paragraph>
    <Paragraph position="5"> To carry out comparable experiments on generative stochastic models presents us with the difficulty that it would first be necessary to determine which of its parameters are responsible for covering a specific phenomenon, and whether they can be modified as to remove the construction from the coverage without affecting others as well. Even in WCDG it is difficult to quantify how much of the observed improvement results from plugged leaks, and how much from focussing. This could only be done by observing all intermediatestepsinthesolutionalgorithm,andcounting null how many trees that were used as intermediate results or consideredas alternativesexhibiteach phenomenon.</Paragraph>
    <Paragraph position="6"> The most promising result from the last experiment is that it is possible to detect particularly detracting phenomena, which are prime candidates for exclusion, in one part of a corpus and use them on another. This suggests itself to be exploitedas a methodto automatically adapt a broad-coverage grammar more closely to the characteristics of a particular corpus.</Paragraph>
  </Section>
class="xml-element"></Paper>