<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2927">
  <Title>Multi-lingual Dependency Parsing at NAIST</Title>
  <Section position="6" start_page="193" end_page="194" type="evalu">
    <SectionTitle>
3 Experiment
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="193" end_page="193" type="sub_section">
      <SectionTitle>
3.1 Experimental setting
</SectionTitle>
      <Paragraph position="0"> Our system consists of three parts; first, the SVM-based tagger extracts the neighboring attachment relations of the input sentence. Second, the parser analyzes further dependency attachments. If a new dependency attachment is generated, the MaxEnt based tagger estimates the label of the relation. The three parts of our parser are trained on the available data of the languages.</Paragraph>
      <Paragraph position="1"> In our experiment, we used the full information of each token (FORM, LEMMA, CPOSTAG, POSTAG, FEATS) when we train and test the model. Fig. 2 describes the features of each token.</Paragraph>
      <Paragraph position="2"> Some languages do not include all columns; such that the Chinese data does not include LEMMA and FEATURES, these empty columns are shown by the symbol &amp;quot;-&amp;quot; in Fig. 2. The features for the neighboring dependency tagging are the information of the focused word, two preceding words and two succeeding words. Fig. 2 shows the window size of our features for estimating the word dependency in the main procedures. These features include the focused words (n, t), two preceding words and two succeeding words and their children.</Paragraph>
      <Paragraph position="3"> The features for estimating the relation label are the same as the features used for word dependency analysis. For example, if the machine learner estimates the operation of this situation as &amp;quot;Left&amp;quot; or &amp;quot;Right&amp;quot; by using the features in Fig. 2, the parser uses the same features in Fig. 2 and the dependency relation to estimate the label of this relation. For training the models efficiently, we divided the training instances of all languages at the CPOSTAG of the focused word n in Fig .2. In our preceding work, we found this procedure can get better performance than training with all the instances at once. However, only the instances in Czech are divided at the CPOSTAG of the focused word-pair t-n  . The performance of this procedure is worse than using the CPOSTAG of the focused word n, because the training instances of each CPOSTAG-pair will become scarce. However, the data size of Czech is much larger than other languages; we couldn't finish the training of Czech using the CPOSTAG of the focused word n, before the deadline for submitting. Therefore we used this procedure only for the experiment of Czech.</Paragraph>
      <Paragraph position="4">  For example, we have 15 SVM-models for Arabic according to the CPOSTAG of Arabic (A, C, D, F, G...etc.). However, we have 139 SVM-models for Czech according the CPOSTAG pair of focused words (A-A, A-C, A-D...etc.) All our experiments were run on a Linux machine with XEON 2.4GHz and 4.0GB memory.</Paragraph>
      <Paragraph position="5"> The program is implemented in JAVA.</Paragraph>
    </Section>
    <Section position="2" start_page="193" end_page="194" type="sub_section">
      <SectionTitle>
3.2 Results
</SectionTitle>
      <Paragraph position="0"> Table 1 shows the results of our parser. We do not take into consideration the problem of cross relation. Although these cross relations are few in training data, they would make our performance worse in some languages. We expect that this is one reason that the result of Dutch is not good. The average length of sentences and the size of training data may have affected the performance of our parser. Sentences of Arabic are longer and training data size of Arabic is smaller than other languages; therefore our parser is worse in Arabic. Similarly, our result in Turkish is also not good because the data size is small.</Paragraph>
      <Paragraph position="1"> We compare the result of Chinese with our preceding work. The score of this shared task is better than our preceding work. It is expected that we selected the FORM and CPOSTAG of each nodes as features in the preceding work. However, the POSTAG is also a useful feature for Chinese, and we grouped the original POS tags of Sinica Tree-bank from 303 to 54 in our preceding work. The number of CPOSTAG(54) in our preceding work is more than the number of CPOSTAG(22) in this shared task, the training data of each CPOSTAG in our preceding work is smaller than in this work.</Paragraph>
      <Paragraph position="2"> Therefore the performance of our preceding work in Sinica Treebank is worse than this task.</Paragraph>
      <Paragraph position="3"> The last column of the Table 1 shows the unlabeled scores of our parser without the preprocessing. Because our parser estimates the label after the dependency relation is generated. We only consider whether the preprocessing can improve the unlabeled scores. Although the preprocessing can not improve some languages (such as Chinese, Spanish and Swedish), the average score shows that using preprocessing is better than parsing without preprocessing.</Paragraph>
      <Paragraph position="4"> Comparing the gold standard data and the system output of Chinese, we find the CPOSTAG with lowest accuracy is &amp;quot;P (preposition)&amp;quot;, the accuracy that both dependency and head are correct is 71%. As we described in our preceding work and Section 2.3, we found that boundaries of prepositional phrases are ambiguous for Chinese. The bottom-up algorithm usually wrongly parses the prepositional phrase short. The parser does not capture the correct information of the children of the preposition. According to the results, this problem does not cause the accuracy of head of  CPOSTAG &amp;quot;P&amp;quot; decrease. Actually, the head accuracy of &amp;quot;P&amp;quot; is better than the CPOSTAG &amp;quot;C&amp;quot; or &amp;quot;V&amp;quot;. However, the dep. accuracy of &amp;quot;P&amp;quot; is worse. We should consider the properties of prepositions in Chinese to resolve this question. In Chinese, prepositions are derived from verbs; therefore some prepositions can be used as a verb. Naturally, the dependency relation of a preposition is different from that of a verb. Important information for distinguishing whether the preposition is a verb or a preposition is the information of the children of the preposition. The real POS tag of a preposition which includes few children is usually a verb; on the other hand, the real POS tag of a preposition is usually a preposition.</Paragraph>
      <Paragraph position="5"> If our parser considers the preposition which leads a short phrase, the parser will estimate the relation of the preposition as a verb. At the same time, if the boundary of prepositional phrase is analyzed incorrectly, other succeeding words will be wrongly analyzed, too.</Paragraph>
      <Paragraph position="6"> Error analysis of Japanese data (Kawata and Bartels, 2000) shows that CNJ (Conjunction) is a difficult POS tag. The parser does not have any module to detect coordinate structures. (Kurohashi, 1995) proposed a method in which coordinate structure with punctuation is detected by a coefficient of similarity. Similar framework is necessary for solving the problem.</Paragraph>
      <Paragraph position="7"> Another characteristic error in Japanese is seen at adnominal dependency attachment for a compound noun. In such dependency relations, adjectives and nouns with &amp;quot;no&amp;quot; (genitive marker) can be a dependent and compound nouns which consist of more than one consecutive nouns can be a head.</Paragraph>
      <Paragraph position="8"> The constituent of compound nouns have same POSTAG, CPOSTAG and FEATS. So, the machine learner has to disambiguate the dependency attachment with sparce feature LEMMA and FORM. Compound noun analysis by semantic feature is necessary for addressing the issue.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>