<?xml version="1.0" standalone="yes"?>
<Paper uid="N03-1022">
  <Title>COGEX: A Logic Prover for Question Answering</Title>
  <Section position="9" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
8 Results
</SectionTitle>
    <Paragraph position="0"> COGEX was implemented and integrated into a state-of-the-art Question Answering system that participated in TREC 2002. All questions are attempted by the prover, but if the proof fails the QA system resorts to other answer extraction methods that were part of the system before the prover. Thus, some questions are answered by the QA system without the prover, some only by the prover and some by both the non-prover system and the prover. The complete system answered 415 questions out of 500 TREC 2002 questions. Of these, 206 were answered by COGEX but some of these answers were also provided by QA system without COGEX. A careful analysis indicates that the QA system without logic prover answered 317 questions and the prover can answer only 98 additional questions for which the system without prover failed. Table 1 summarizes these results.</Paragraph>
    <Paragraph position="1"> Questions answered by the complete system 415 Questions answered by COGEX 206 Questions answered only by COGEX 98 Questions answered without COGEX 317  The added value of automated reasoning to the QA system is 30.9% (98/317). This represents a significant improvement in the performance of the logic prover for QA over the one reported in (Moldovan 2002). The failures of the prover are due primarily to the lack of linguistic axioms.</Paragraph>
  </Section>
class="xml-element"></Paper>