<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-1133">
  <Title>Are These Documents Written from Different Perspectives? A Test of Different Perspectives Based On Statistical Distribution Divergence</Title>
  <Section position="7" start_page="1059" end_page="1061" type="evalu">
    <SectionTitle>
5 Experiments
</SectionTitle>
    <Paragraph position="0"> A test of different perspectives is acute when it can draw distinctions between document collection pairs of different perspectives and document collection pairs of the same perspective and others.</Paragraph>
    <Paragraph position="1"> We thus evaluate the proposed test of different perspectives in the following four types of document collection pairs (A,B): Different Perspectives (DP) A and B are written from different perspectives. For example, A is written from the Palestinian perspective and B is written from the Israeli perspective in the bitterlemons corpus.</Paragraph>
    <Paragraph position="2"> Same Perspective (SP) A and B are written from the same perspective. For example, A and B consist of the words spoken by Kerry.</Paragraph>
    <Paragraph position="3"> Different Topics (DT) A and B are written on different topics. For example, A is about  acquisition (ACQ) and B is about crude oil (CRUDE).</Paragraph>
    <Paragraph position="4"> Same Topic (ST) A and B are written on the same topic. For example, A and B are both about earnings (EARN).</Paragraph>
    <Paragraph position="5"> The effectiveness of the proposed test of different perspectives can thus be measured by how the distribution divergence of DP document collection pairs is separated from the distribution divergence of SP, DT, and ST document collection pairs. The little the overlap of the range of distribution divergence, the sharper the test of different perspectives. null To account for large variation in the number of words and vocabulary size across corpora, we normalize the total number of words in a document collection to be the same K, and consider only the top C% frequent words in the document collection pair. We vary the values of K and C, and find that K changes the absolute scale of KL divergence but does not change the rankings of four conditions. Rankings among four conditions is consistent when C is small. We only report results of</Paragraph>
    <Paragraph position="7"> There are two kinds of variances in the estimation of divergence between two posterior distribution and should be carefully checked. The first kind of variance is due to Monte Carlo methods.</Paragraph>
    <Paragraph position="8"> We assess the Monte Carlo variance by calculating a 100a percent confidence interval as follows, [ ^D[?]Ph[?]1(a2) ^s[?]M, ^D + Ph[?]1(1[?] a2) ^s[?]M] where ^s2 is the sample variance of th1,th2,...,thM, and Ph(*)[?]1 is the inverse of the standard normal cumulative density function. The second kind of variance is due to the intrinsic uncertainties of data generating processes. We assess the second kind of variance by collecting 1000 bootstrapped samples, that is, sampling with replacement, from each document collection pair.</Paragraph>
    <Section position="1" start_page="1059" end_page="1060" type="sub_section">
      <SectionTitle>
5.1 Quality of Monte Carlo Estimates
</SectionTitle>
      <Paragraph position="0"> The Monte Carlo estimates of the KL divergence from several document collection pair are listed in  ted due to the space limit. We can see that the 95% confidence interval captures well the Monte Carlo estimates of KL divergence. Note that KL divergence is not symmetric. The KL divergence  confidence interval (CI) of the Kullback-Leibler divergence of several document collection pairs (A,B) with the number of Monte Carlo samples</Paragraph>
      <Paragraph position="2"> of the pair (Israeli, Palestinian) is not necessarily the same as (Palestinian, Israeli). KL divergence is greater than zero (Cover and Thomas, 1991) and equal to zero only when document collections A and B are exactly the same. Here (ACQ, ACQ) is close to but not exactly zero because they are different samples of documents in the ACQ category.</Paragraph>
      <Paragraph position="3"> Since the CIs of Monte Carlo estimates are reasonably tight, we assume them to be exact and ignore the errors from Monte Carlo methods.</Paragraph>
    </Section>
    <Section position="2" start_page="1060" end_page="1060" type="sub_section">
      <SectionTitle>
5.2 Test of Different Perspectives
</SectionTitle>
      <Paragraph position="0"> We now present the main result of the paper.</Paragraph>
      <Paragraph position="1"> We calculate the KL divergence between posterior distributions of document collection pairs in four conditions using Monte Carlo methods, and plot the results in Figure 1. The test of different perspectives based on statistical distribution divergence is shown to be very acute. The KL divergence of the document collection pairs in the DP condition fall mostly in the middle range, and is well separated from the high KL divergence of the pairs in DT condition and from the low KL divergence of the pairs in SP and ST conditions. Therefore, by simply calculating the KL divergence of a document collection pair, we can reliably predict that they are written from different perspectives if the value of KL divergence falls in the middle range, from different topics if the value is very large, from the same topic or perspective if the value is very small.</Paragraph>
    </Section>
    <Section position="3" start_page="1060" end_page="1060" type="sub_section">
      <SectionTitle>
5.3 Personal Writing Styles or Perspectives?
</SectionTitle>
      <Paragraph position="0"> One may suspect that the mid-range distribution divergence is attributed to personal speaking or writing styles and has nothing to do with different perspectives. The doubt is expected because half of the bitterlemons corpus are written by one Palestinian editor and one Israeli editor (see Table 1), and the debate transcripts come from only two candidates.</Paragraph>
      <Paragraph position="1"> We test the hypothesis by computing the distribution divergence of the document collection pair (Israeli Guest, Palestinian Guest), that is, a Different Perspectives (DP) pair. There are more than 200 different authors in the Israeli Guest and Palestinian Guest collection. If the distribution divergence of the pair with diverse authors falls out of the middle range, it will support that mid-range divergence is due to writing styles. On the other hand, if the distribution divergence still fall in the middle range, we are more confident the effect is attributed to different perspectives. We compare the distribution divergence of the pair (Israeli Guest, Palestinian Guest) with others in Figure 2.</Paragraph>
      <Paragraph position="2">  collection pairs in the bitterlemons Guest subset (Israeli Guest vs. Palestinian Guest), ST ,SP, DP, DT conditions. The horizontal lines are the same as those in Figure 1.</Paragraph>
      <Paragraph position="3"> The results show that the distribution divergence of the (Israeli Guest, Palestinian Guest) pair, as other pairs in the DP condition, still falls in the middle range, and is well separated from SP and ST in the low range and DT in the high range. The decrease in KL divergence due to writing or speaking styles is noticeable, and the overall effect due to different perspectives is strong enough to make the test robust. We thus conclude that the test of different perspectives based on distribution divergence indeed captures different perspectives, not personal writing or speaking styles.</Paragraph>
    </Section>
    <Section position="4" start_page="1060" end_page="1061" type="sub_section">
      <SectionTitle>
5.4 Origins of Differences
</SectionTitle>
      <Paragraph position="0"> While the effectiveness of the test of different perspectives is demonstrated in Figure 1, one may  (DP), Same Perspective (SP), Different Topics (DT), and Same Topic (ST). Note that the x axis is in log scale. The Monte Carlo estimates ^D of the pairs in DP condition are plotted as rugs. ^D of the pairs in other conditions are omitted to avoid clutter and summarized in one-dimensional density using Kernel Density Estimation. The vertical lines are drawn at the points with equivalent densities. wonder why the distribution divergence of the document collection pair with different perspectives falls in the middle range and what causes the large and small divergence of the document collection pairs with different topics (DT) and the same topic (ST) or perspective (SP), respectively. In other words where do the differences result from? We answer the question by taking a closer look at the causes of the distribution divergence in our model. We compare the expected marginal difference of th between two posterior distributions p(th|A) and p(th|B). The marginal distribution of the i-th coordinate of th, that is, the i-th word in the vocabulary, is a Beta distribution, and thus the expected value can be easily calculated. We plot the [?]th = E[thi|A][?]E[thi|B] against E[thi|A] for each condition in Figure 3.</Paragraph>
      <Paragraph position="1"> How [?]th is deviated from zero partially explains different patterns of distribution divergence in Figure 1. In Figure 3d we see that the [?]th increases as th increases, and the deviance from zero is much greater than those in the Same Perspective (Figure 3b) and Same Topic (Figure 3a) conditions.</Paragraph>
      <Paragraph position="2"> The large [?]th not only accounts for large distribution divergence of the document pairs in DT conditions, but also shows that words in different topics that is frequent in one topic are less likely to be frequent in the other topic. At the other extreme, document collection pairs of the Same Perspective (SP) or Same Topic (ST) show very little difference in th, which matches our intuition that documents of the same perspective or the same topic use the same vocabulary in a very similar way.</Paragraph>
      <Paragraph position="3"> The manner in which [?]th is varied with the value of th in the Different Perspective (DP) condition is very unique. The [?]th in Figure 3c is not as small as those in the SP and ST conditions, but at the same time not as large as those in DT conditions, resulting in mid-range distribution divergence in Figure 1. Why do document collections of different perspectives distribute this way? Partly because articles from different perspectives focus on the closely related issues (the Palestinian-Israeli conflict in the bitterlemons corpus, or the political and economical issues in the debate corpus), the authors of different perspectives write or speak in a similar vocabulary, but with emphasis on different words.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>