<?xml version="1.0" standalone="yes"?>
<Paper uid="P89-1013">
  <Title>SOME CHART-BASED TECHNIQUES FOR PARSING ILL-FORMED INPUT</Title>
  <Section position="10" start_page="106" end_page="107" type="concl">
    <SectionTitle>
EVALUATION AND FUTURE WORK
</SectionTitle>
    <Paragraph position="0"> The preliminary results show that, for small sentences and only one error, enumerating all the possible minimum-penalty errors takes no worse than 10 times as long as parsing the correct sentences.</Paragraph>
    <Paragraph position="1"> Finding the first minimal-penalty error can also be quite fast. There is, however, a great variability between the types of error. Errors involving completely unknown words can be diagnosed reasonably  quickly because the presence of an unknown word allows the estimation of penalty scores to be quite accurate (the system still has to work out whether the word can be an addition and for what categories it can substitute for an instance of, however). We have not yet considered multiple errors in a sentence, and we can expect the behaviour to worsten dramatically as the number of errors increases. Although Table 1 does not show this, there is also a great deal of variability between sentences of the same length with the same kind of introduced error. It is noticeable that errors towards the end of a sentence are harder to diagnose than those at the start. This reflects the leRfight orientation of the parsing rules - an attempt to find phrases starting to the right of an error will have a PBG score at least one more than the estimated PB, whereas an attempt m find phrases in an open-ended portion of the chart starting before an error may have a PBG score the same as the PB (as the error may occur within the phrases to be found). Thus more parsing attempts will be relegated to the lower parts of the agenda in the first case than in the second.</Paragraph>
    <Paragraph position="2"> One disturbing fact about the statistics is that the number of minimal-penalty solutions may be quite large. For instance, the ill-formed sentence: who has John seen on that had was formed by adding the extra word &amp;quot;had&amp;quot; to the sentence &amp;quot;who has John seen on that&amp;quot;. Our parser found three other possible single errors to account for the sentence. The word &amp;quot;on&amp;quot; could have been an added word, the word &amp;quot;on&amp;quot; could have been a substitution for a complementiser and there could have been a missing NP after &amp;quot;on&amp;quot;. This large number of solutions could be an artefact of our particular gramram&amp;quot; and lexicon; certainly it is unclear how one should choose between possible solutions in a grammar-independent way. In a few cases, the introduction of a random error actually produced a grammatical sentence - this occurred, for instance, twice with sentences of length 5 given one random A__dded word.</Paragraph>
    <Paragraph position="3"> At this stage, we cannot claim that our experiments have done anything more than indicate a certain concreteness to the ideas and point to a number of unresolved problems. It remains to be seen how the performance will scale up for a realistic grammar and parser. There are a number of detailed issues to resolve before a really practical implementation of the above ideas can be produced. The indexing strategy of the chart needs to be altered to take into account the new parsing rules, and remaining problems of duplication of effort need to be addressed.</Paragraph>
    <Paragraph position="4"> For instance, the generalised version of the fundamental rule allows an active edge to combine with a set of inactive edges satisfying its needs in any order. The scoring of errors is another ar~ which should be better investigated. Where extra words are introduced accidentally into a text, in practice they are perhaps unlikely to be words that are already in the lexicon. Thus when we gave our system sentences with known words added, this may not have been a fair test. Perhaps the scoring system should prefer added words to be words outside the lexicon, substituted words to substitute for words in open categories, deleted words to be non-content words, and so on. Perhaps also the confidence of the system about possible substitutions could take into account whether a standard spelling corrector can rewrite the acnmi word to a known word of the hypothesised category. A more sophisticated error scoring strategy could improve the system's behaviour considerably for real examples (it might of course make less difference for random examples like the ones in our experiments).</Paragraph>
    <Paragraph position="5"> Finally the behaviour of the approach with realistic grammars written in more expressive notations needs to be established. At present, we are investigating whether any of the current ideas can be used in conjunction with Allport's (1988) &amp;quot;interesting corner&amp;quot; parser.</Paragraph>
  </Section>
class="xml-element"></Paper>