<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1209">
  <Title>Knowledge Extraction and Recurrent Neural Networks: An Analysis of an Elman Network trained on a Natural Language Learning Task</Title>
  <Section position="6" start_page="0" end_page="0" type="concl">
    <SectionTitle>
5. Conclusions
</SectionTitle>
    <Paragraph position="0"> This study has demonstrated one method for extracting the knowledge encoded in a trained neural network.</Paragraph>
    <Paragraph position="1"> Quite omen knowledge extracted from neural networks is in the form of propositional rules (Andrews et al., 1995) but these are not always the most appropriate format for explication of network learning. For example where the network has been required to induce a grammar, cluster analysis of hidden unit activations and preparation of an FSA is a powerful technique to explicate the learned grammar. However, for this particular task, there is a trade-off between comprehensibility of the FSA (fewer states means more comprehensible) and its predictive performance compared to the original neural network. In these experiments an FSA with 18-states performed almost as well as a trigram model. The trigram model had the advantage of compactness, but the FSA had the advantage of comprehensibility.</Paragraph>
  </Section>
class="xml-element"></Paper>