<?xml version="1.0" standalone="yes"?>
<Paper uid="C94-1006">
  <Title>Two Methods for Learning ALT-J/E \]Y=anslation Rules from Examples and a Semantic Hierarchy llussein Ahnuallim ln\[o. and Coml)uter Science Dept. King Fahd University of l)etroleum and Minerals l)hahran 312(;1, Sated( lXrahia</Title>
  <Section position="6" start_page="61" end_page="61" type="evalu">
    <SectionTitle>
6 Experimental Work
</SectionTitle>
    <Paragraph position="0"> The goad of tile experiments reI}orted here is to evaluate the qmdity of the partiad translation rules learned by the two h.~m'ning methods we have just descril}ed.</Paragraph>
    <Paragraph position="1"> The comi}arison includes the folh}wing three settings:  1. Using llaussler's algorithm to learn fr{}ill training examl}les ~ffter removing the mnl)igulty.</Paragraph>
    <Paragraph position="2"> 2. Using ID3 to h;arn from training examl)les af null ter removing the ambiguity atnd performing the transformation given in the Subsection 5.3.</Paragraph>
    <Paragraph position="3"> 3. Using ID3 to learn from tnfining examI}les after performing the transfi)rmation given ill tile Sub-section 5.3, trot without removing the. ambiguity. In a sense, the first setting rellresents the lmst we can do in the absence of the ambiguity since llmlssler's algoritl}m does at good job in exi)loiting the baekgT{mnd knowledge fi-om the Selnanti{: Ilierarchy. Comparing Setting 2 with Setting 1 tells us how successfifl our transformati{m of the training examl}les is in letting 1D3 make use of the available I}ackground knowledge.</Paragraph>
    <Paragraph position="4"> Fimdly, comparing Setting 3 with Settir,g 2 tells ns how successful our transhn'mation is in letting 1133 learn directly froin amt)igalous training examl)les.</Paragraph>
    <Paragraph position="5"> The experiments were done tbr six ditl'erent .lapanese ver/}s. '.\['able 1 shows a list of these verbs, along with the II/lltl})er of training eKauli\])h!s llsed, and the a{:cura{:y levels obtained by each meth{}d. In the table, &amp;quot;tlausslcr&amp;quot;, &amp;quot;ID3 NA&amp;quot; and &amp;quot;11)3 A&amp;quot; de.note Setting 1, Setting 2 and Setting 3, resl}e{:tively. The a(:curacy was esthnated using the leaLvt&gt;olle-{}llt {:rosswflidation meth{}d ' |, m,d assuming that the rules {:{)m-I)osed rnamutlly by human experts are t}erfect (that is, we are measuring how close tim learned rules are to those {:Omllosed mmmally).</Paragraph>
    <Paragraph position="6"> The i)erti}rmanee levels of both lhmssler's alg()rithm and ID3 when learning from unambiguous examples are quite similar in Sl)ite of the fact that each algorithm implements a different bias and has a completely diftin'ent way {}f' exl}loiting the background knowledge. Coml}aring tim l}erformance of ID3 in the two cases of leil.rIlillg froI\[l itIIl\]}ig/l(}llS &amp;ll(\[ IllHlI\[l-I)iguous examl}les , ambiguity is not harntful t(} ll)3's l}erforman(:e in most cases. In fact, for some of the verbs, the t}erforlIl~tn{:e is evelk \])etter when aml)iguity is present. This suggests that the apl}roach we have chosen to de.al with ambiguity is effective for our task, and tl,at ext}licit retll{}vitl o\[ ambiguity is not an attractive strategy sim:e it is not easy to {t(}, and since it does not greatly improve the a(:{:m'aey anyway.</Paragraph>
    <Paragraph position="7"> The most important ll(}int here is that the ol}served a{:cura{:y of both the. 11)3 a.lgorithm aim llaussler's algorithm is satisfactorily high overa!.l in spite of the limited mmfl}er of the training examl}k's used. Such a high level of at(:curat(:y str{mgly indicates that the use of these algo,'ithms will provide significant aid in the c{}l,struction of AI/.I'-J/E's trmMati{}\]t rules.</Paragraph>
  </Section>
class="xml-element"></Paper>