<?xml version="1.0" standalone="yes"?>
<Paper uid="E06-1026">
  <Title>Latent Variable Models for Semantic Orientations of Phrases</Title>
  <Section position="6" start_page="204" end_page="206" type="evalu">
    <SectionTitle>
4.2 Results
</SectionTitle>
    <Paragraph position="0"> The classification accuracies of the four methods with b and M predicted by the held-out method are shown in Table 1. Please note that the naive bayes method is irrelevant of b and M. The table shows that the triangle model and the U-shaped 1The complete list of the 17 Japanese adjectives with their English counterparts are : takai (high), hikui (low), ookii (large), chiisai (small), omoi (heavy), karui (light), tsuyoi (strong), yowai (weak), ooi (many), sukunai (few/little), nai (no), sugoi (terrific), hageshii (terrific), hukai (deep), asai (shallow), nagai (long), mizikai (short).</Paragraph>
    <Paragraph position="1">  model achieved high accuracies and outperformed the naive bayes method. This result suggests that we succeeded in capturing the internal structure of semantically oriented phrases by way of latent variables. The more complex structure of the triangle model resulted in the accuracy that is higher than that of the U-shaped model.</Paragraph>
    <Paragraph position="2"> The performance of the 3-PLSI method is even worse than the baseline method. This result shows thatweshoulduseamodelinwhichadjectivescan directly influence the rating category.</Paragraph>
    <Paragraph position="3"> Figures 2, 3, 4 show cross-validated accuracy valuesforvariousvaluesofb, respectivelyyielded by the 3-PLSI model, the triangle model and the U-shapedmodelwithdifferentnumbersM ofpossible states for the latent variable. As the figures show, the classification performance is sensitive to thevalueofb. M = 100 andM = 300 aremostly better than M = 10. However, this is a tradeoff between classification performance and training time, since large values of M demand heavy computation. In that sense, the U-shaped model is useful in many practical situations, since it achieved a good accuracy even with a relatively small M.</Paragraph>
    <Paragraph position="4"> To observe the overall tendency of errors, we show the contingency table of classification by the U-shaped model with the predicted values of hyperparameters, in Table 2. As this table shows, most of the errors are caused by the difficulty of classifying neutral examples. Only 2.26% of the errors are mix-ups of the positive orientation and the negative orientation.</Paragraph>
    <Paragraph position="5"> We next investigate the causes of errors by observing those mix-ups of the positive orientation and the negative orientation.</Paragraph>
    <Paragraph position="6"> One type of frequent errors is illustrated by the pair &amp;quot;food ('s price) is high&amp;quot;, in which the word &amp;quot;price&amp;quot; is omitted in the actual example 2. As in this expression, the attribute (price, in this case) of an example is sometimes omitted or not correctly  identified. To tackle these examples, we will need methods for correctly identifying attributes and objects. Some researchers are starting to work on this problem (e.g., Popescu and Etzioni (2005)).</Paragraph>
    <Paragraph position="7"> We succeeded in addressing the data-sparseness problem by introducing a latent variable. However, this problem still causes some errors. Precise statistics cannot be obtained for infrequent words. This problem will be solved by incorporating other resources such as thesaurus or a dictionary,orcombiningourmethodwithothermethods null using external wider contexts (Suzuki et al., 2006; Turney, 2002; Baron and Hirst, 2004).</Paragraph>
    <Section position="1" start_page="206" end_page="206" type="sub_section">
      <SectionTitle>
4.3 Examples of Obtained Clusters
</SectionTitle>
      <Paragraph position="0"> Next, wequalitativelyevaluatetheproposedmethods. For several clusters z, we extract the words that occur more than twice in the whole dataset and are in top 50 according to P(z|n). The model used here as an example is the U-shaped model.</Paragraph>
      <Paragraph position="1"> The experimental settings are b = 0.6 and M = 60. Although some elements of clusters are composed of multiple words in English, the original Japanese counterparts are single words.</Paragraph>
      <Paragraph position="2"> Cluster 1 trouble, objection, disease, complaint, anxiety, anamnesis, relapse Cluster 2 risk, mortality, infection rate, onset rate Cluster 3 bond, opinion, love, meaning, longing, will Cluster 4 vote, application, topic, supporter Cluster 5 abuse, deterioration, shock, impact, burden Cluster 6 deterioration, discrimination, load, abuse Cluster 7 relative importance, degree of influence, number, weight, sense of belonging, wave, reputation These obtained clusters match our intuition. For example, in cluster 2 are the nouns that are negative when combined with &amp;quot;high&amp;quot;, and positive when combined with &amp;quot;low&amp;quot;. In fact, the posterior probabilities of semantic orientations for cluster 2 are as follows :</Paragraph>
      <Paragraph position="4"> With conventional clustering methods based on the cooccurrence of two words, cluster 2 would include the words resulting in the opposite orientation, such as &amp;quot;success rate&amp;quot;. We succeeded in obtaining the clusters that are suitable for our task, by incorporating the new variable c for semantic orientation in the EM computation.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>