<?xml version="1.0" standalone="yes"?>
<Paper uid="N03-2015">
  <Title>Unsupervised Learning of Morphology for English and Inuktitut</Title>
  <Section position="5" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
4 Evaluation
</SectionTitle>
    <Paragraph position="0"> As noted above, Linguistica uses many techniques to learn morphology, including a fairly complex system for counting bits. We tested whether the two techniques presented in this paper, hub searching and simple node merging, achieve the same performance as Linguistica.</Paragraph>
    <Paragraph position="1"> If so, the simpler techniques might be preferred. Also, we would be justified using them for more complex morphologies.</Paragraph>
    <Paragraph position="2"> The input to Linguistica and HubMorph was the text of Tom Sawyer. The performance of both was compared against a gold standard division of the distinct words in that novel. The gold standard was based on dictionary entries and the judgment of two English speakers.</Paragraph>
    <Paragraph position="3"> In matching the gold standard words to divisions predicted by either system, we made the following assumptions. a) Words with hyphens are split at the hyphen to match Linguistica's assumption. b) If the gold standard has a break before and after a single character, to capture non-concatenative modification, either break matches. An example would be 'mud-d-y'. c) An apostrophe at a morpheme boundary is ignored for comparison matching to allow it to stick to the root or to the suffix. d) The suffix split proposed must result in a suffix of 5 or fewer characters, again to match Linguistica's assumption.</Paragraph>
    <Paragraph position="4"> Table 1 show the results of this comparison for Linguistica, hub-searching alone, and HubMorph (both hub searching and node merging). Hub-searching alone is sufficient to achieve the same precision as Linguistica and nearly the same recall. Both of the techniques together are sufficient to achieve the same precision and recall as Linguistica. The recall for all is low because the list of words in Tom Sawyer is not long enough to include most acceptable combinations of roots and suffixes. A longer input word list would improve this score.</Paragraph>
    <Paragraph position="5">  Hub-searching alone, and HubMorph. Recall is the proportion of distinct words from Tom Sawyer that are correctly divided into root and suffix. Precision is the proportion of predicted divisions that are correct. null</Paragraph>
  </Section>
class="xml-element"></Paper>