<?xml version="1.0" standalone="yes"?>
<Paper uid="P92-1023">
  <Title>GPSM: A GENERALIZED PROBABILISTIC SEMANTIC MODEL FOR AMBIGUITY RESOLUTION</Title>
  <Section position="8" start_page="180" end_page="183" type="concl">
    <SectionTitle>
5. Semantic Score
</SectionTitle>
    <Paragraph position="0"> Semantic score evaluation is similar to syntactic score evaluation. From Eqn. (2), we have the following semantic model for semantic score:  where 3~j am the semantic tags from the children of A1. For example, we have terms like e(VP(sta, anim) \[ a, VP ~- v NP,fl) and P(VP(sta, in) la, Ve~v NP PP,fl),respecfively, for the left and right trees in Figure 2. The annotations of the context am ignored in evaluating Eqn. (6) due to the assumption of semantics compositionality. The operation mode will be called LLRR+Alv, where N is the dimension of the N-tuple, and the subscript L (or R) refers to the size of the context window. With an appropriate N, the score will provide sufficient discrimination power for general disambiguation problem without resorting to full-blown semantic analysis. where At = At (ft,l,fln,...,fuv) is the annotated version of At, whose semantic N-tuple is (fl,1, fl,2,-&amp;quot;, ft,N), and 57, fit are the annotated context symbols. Only Ft.1 is assumed to be significant for the transition to Ft in the last equation, because all required information is assumed to have been percolated to Ft-j through semantics composition.</Paragraph>
    <Paragraph position="1"> Each term in Eqn. (5) can be interpreted as the probability thatAt is annotated with the particular set of head features (fs,1, ft,2,..., fI,N), given that X1 ... XM are reduced to At in the context of a7 and fit. So it can be interpreted informally as P(At (fl,1, ft,2, . . . , fz ~v) I Ai ~ X1. . . XM , in the context of ~-7, fit ). It corresponds to the semantic preference assigned to the annotated node A t&amp;quot; Since (11,1, fl,~,&amp;quot;&amp;quot; ft,N) are the head features from various heads of the substructures of A, each term reflects the feature co-occurrence preference among these heads. Furthermore, the heads could be very far apart. This is different from most simple Markov models, which can deal with local constraints only. Hence, such a formulation well characterizes long distance dependency among the heads, and provides a simple mechanism to incorporate the feature co-occurrence preference among them. For the semantic N-tuple model, the semantic score can thus be expressed as follows:</Paragraph>
    <Paragraph position="3"> 6. Major Categories and</Paragraph>
    <Section position="1" start_page="181" end_page="183" type="sub_section">
      <SectionTitle>
Semantic Features
</SectionTitle>
      <Paragraph position="0"> As mentioned before, not all constituents are equally important for disambiguation. For instance, head words are usually more important than modifiers in determining the compositional semantic features of their mother node. There is also lots of redundancy in a sentence. For instance, &amp;quot;saw boy in park&amp;quot; is equally recognizable as &amp;quot;saw the boy in the park.&amp;quot; Therefore, only a few categories, including verbs, nouns, adjectives, prepositions and adverbs and their projections (NP, VP, AP, PP, ADVP), are used to carry semantic features for disambiguation. These categories are roughly equivalent to the major categories in linguistic theory \[Sells 85\] with the inclusion of adverbs as the only difference.</Paragraph>
      <Paragraph position="1"> The semantic feature of each major category is encoded with a set of semantic tags that well describes each category. A few rules of thumb are used to select the semantic tags. In particular, semantic features that can discriminate different linguistic behavior from different possible semantic N-tuples are preferred as the semantic tags.</Paragraph>
      <Paragraph position="2"> With these heuristics in mind, the verbs, nouns, adjectives, adverbs and prepositions are divided into 22, 30, 14, 10 and 28 classes, respectively.</Paragraph>
      <Paragraph position="3"> For example, the nouns are divided into &amp;quot;human,&amp;quot; &amp;quot;plant,&amp;quot; &amp;quot;time,&amp;quot; &amp;quot;space,&amp;quot; and so on. These semantic classes come from a number of sources and the semantic attribute hierarchy of the ArchTran  7. Test and Analysis  The semantic N-tuple model is used to test the improvement of the semantic score over syntactic score in structure disambiguation. Eqn. (3) is adopted to evaluate the syntactic score in L2RI mode of operation. The semantic score is derived from Eqn. (6) in L2R~ +AN mode, for N = 1, 2, 3, 4, where N is the dimension of the semantic S-tuple.</Paragraph>
      <Paragraph position="4"> A total of 1000 sentences (including 3 un-ambiguous ones) are randomly selected from 14 computer manuals for training or testing. They are divided into 10 parts; each part contains 100 sentences. In close tests, 9 parts are used both as the training set and the testing set. In open tests, the rotation estimation approach \[Devijver 82\] is adopted to estimate the open test performance. This means to iteratively test one part of the sentences while using the remaining parts as the training set. The overall performance is then estimated as the average performance of the 10 iterations.</Paragraph>
      <Paragraph position="5"> The performance is evaluated in terms of Top-N recognition rate (TNRR), which is defined as the fraction of the test sentences whose preferred interpretation is successfully ranked in the first N candidates. Table 1 shows the simulation resuits of close tests. Table 2 shows partial results for open tests (up to rank 5.) The recognition rates achieved by considering syntactic score only and semantic score only are shown in the tables.</Paragraph>
      <Paragraph position="6"> (L2RI+A3 and L2RI+A4 performance are the same as L2R~+A2 in the present test environment. So they are not shown in the tables.) Since each sentence has about 70-75 ambiguous constructs on the average, the task perplexity of the current disambiguation task is high.</Paragraph>
      <Paragraph position="7">  The close test Top-1 performance (Table 1) for syntactic score (87%) is quite satisfactory.</Paragraph>
      <Paragraph position="8"> When semantic score is taken into account, substantial improvement in recognition rate can be observed further (97%). This shows that the semantic model does provide an effective mechanism for disambiguation. The recognition rates in open tests, however, are less satisfactory under the present test environment. The open test performance can be attributed to the small database size and the estimation error of the parameters thus introduced. Because the training database is small with respect to the complexity of the model, a significant fraction of the probability entries in the testing set can not be found in the training set. As a result, the parameters are somewhat &amp;quot;overtuned&amp;quot; to the training database, and their values are less favorable for open tests. Nevertheless, in both close tests and open tests, the semantic score model shows substantial improvement over syntactic score (and hence stochastic context-free grammar). The improvement is about 10% for close tests and 14% for open tests.</Paragraph>
      <Paragraph position="9"> In general, by using a larger database and better robust estimation techniques \[Su 91a, Chiang 92\], the baseline model can be improved further.</Paragraph>
      <Paragraph position="10"> As we had observed from other experiments for spoken language processing \[Su 91a\], lexical tagging, and structure disambiguation \[chiang 92\], the performance under sparse data condition can be improved significantly if robust adaptive leaming techniques are used to adjust the initial parameters. Interested readers are referred to \[Su 91a, Chiang 92\] for more details.</Paragraph>
      <Paragraph position="11"> 8. Concluding Remarks In this paper, a generalized probabilistic semantic model (GPSM) is proposed to assign semantic preference to ambiguous interpretations. The semantic model for measuring preference is based on a score function, which takes lexical, syntactic and semantic information into consideration and optimizes the joint preference. A simple yet effective encoding scheme and semantic tagging procedure is proposed to characterize various interpreta- null tions in an N dimensional feature space. With this encoding scheme, one can encode the interpretations with discriminative features, and take the feature co-occurrence preference among various constituents into account. Unlike simple Markov models, long distance dependency can be managed easily in the proposed model. Preliminary tests show substantial improvement of the semantic score measure over syntactic score measure. Hence, it shows the possibility to overcome the ambiguity resolution problem without resorting to full-blown semantic analysis.</Paragraph>
      <Paragraph position="12"> With such a simple, objective and trainable formulation, it is possible to take high level semantic knowledge into consideration in statistic sense. It also provides a systematic way to construct a disambiguation module for large practical machine translation systems without much human intervention; the heavy burden for the linguists to write fine-grained &amp;quot;rules&amp;quot; can thus be relieved.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>