<?xml version="1.0" standalone="yes"?>
<Paper uid="C00-1022">
  <Title>Exogeneous and Endogeneous Approaches to Semantic Categorization of Unknown Technical Terms</Title>
  <Section position="7" start_page="149" end_page="149" type="evalu">
    <SectionTitle>
6 Experiments and Evaluation
</SectionTitle>
    <Paragraph position="0"> To estimate the accuracy of the exogeueous method, we used a domain-specific corpus of 541,964 words, coml)osed of documents I)ertaintug to various textual genres (software st)ecifications, maintenance procedures, manufacturing notices...). This corpus covered 63 of the 70 categories. Each run starts with the selection eta document among the corpus documents.</Paragraph>
    <Paragraph position="1"> The known terms identified in this document are considered as test: terms. We used relatively wide contexts. The cues were extracted in a window of :t=20 words around the term.</Paragraph>
    <Paragraph position="2"> Each run involved more than 70,000 contexts of term occurrences. To experiment the endogeneous apI)roach, test sets of col~xl)ound terms have l)een randomly extracted from the, terlninological database.</Paragraph>
    <Paragraph position="3"> We adopted an evaluation scheme similar to that detlned in (Tokunaga et al., 1997) for thesaux'us extension. The categorization is considered successful if' the right categox'y appears among the t~: first categories assigned by the classifier. Within a semi-automatic acquisition framework, this evaluation scheme is more suitable than strict evaluation where only the first category assigned by the classifier is considered as relevant (evaluation restricted to h=l.) 'l. Frolu oux' application 1)erspective, it is useful to provide to the terminologist a restricted set of less than 5 t)lausible categories instead of the complete set of 70 categories without prior filtering. null In the experiments des(:ribed in (Tokunaga et al., 1997), k takes the values 5, 10, 20 and 30 for average(t 1)ertbrmance ranging from 26.4:% to 55.9% (the choice is made among 544 categories). Some of their precise experiments yielded an accm'acy greater than 80% fox' t~: = 30. In ore&amp;quot; experinmnts, we measured accuracy tbr h=l to 5. Some results are given in tables 3 and 4. The scores are higher than those achieved in thesaurus extension, especially with the exogeneous approach (fi'om 52.75% to 97.61%). We should however keel) in mind that we deal with a diflbrent kind of data (see section 3).</Paragraph>
  </Section>
class="xml-element"></Paper>