File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/05/w05-0308_metho.xml
Size: 34,218 bytes
Last Modified: 2025-10-06 14:09:52
<?xml version="1.0" standalone="yes"?> <Paper uid="W05-0308"> <Title>Annotating Attributions and Private States</Title> <Section position="3" start_page="0" end_page="54" type="metho"> <SectionTitle> 2 The Core Scheme </SectionTitle> <Paragraph position="0"> This section overviews the core of the annotation scheme.</Paragraph> <Paragraph position="1"> Further details may be found in (Wilson and Wiebe, 2003; Wiebe et al., 2005).</Paragraph> <Section position="1" start_page="0" end_page="53" type="sub_section"> <SectionTitle> 2.1 Means of Expressing Private States </SectionTitle> <Paragraph position="0"> The goals of the annotation scheme are to represent internal mental and emotional states, and to distinguish subjective information from material presented as fact. As a result, the annotation scheme is centered on the notion of private state, a general term that covers opinions, beliefs, thoughts, feelings, emotions, goals, evaluations, and judgments. As Quirk et al. (1985) define it, a private state is a state that is not open to objective observation or verification: &quot;a person may be observed to assert that God exists, but not to believe that God exists. Belief is in this sense 'private'.&quot; (p. 1181) Following literary theorists such as Banfield (1982), we use the term subjectivity for linguistic expressions of private states in the contexts of texts and conversations.</Paragraph> <Paragraph position="1"> We can further view private states in terms of their functional components -- as states of experiencers holding attitudes, optionally toward targets. For example, for the private state in the sentence John hates Mary, the experiencer is &quot;John,&quot; the attitude is &quot;hate,&quot; and the target is &quot;Mary.&quot; We create private state frames for three main types of private state expressions in text: * explicit mentions of private states * speech events expressing private states * expressive subjective elements An example of an explicit mention of a private state is &quot;fears&quot; in (1): (1) &quot;The U.S. fears a spill-over,&quot; said Xirao- null Nima.</Paragraph> <Paragraph position="2"> An example of a speech event expressing a private state is &quot;said&quot; in (2): (2) &quot;The report is full of absurdities,&quot; Xirao-Nima said.</Paragraph> <Paragraph position="3"> Note that we use the term speech event to refer to both speaking and writing events.</Paragraph> <Paragraph position="4"> The phrase &quot;full of absurdities&quot; in (2) above is an expressive subjective element (Banfield, 1982). Other examples can be found in (3): (3) The time has come, gentlemen, for Sharon, the assassin, to realize that injustice cannot last long.</Paragraph> <Paragraph position="5"> The private states in this sentence are expressed entirely by the words and the style of language that is used. In (3), although the writer does not explicitly say that he hates Sharon, his choice of words clearly demonstrates a negative attitude toward him. As used in these sentences, the phrases &quot;The time has come,&quot; &quot;gentlemen,&quot; &quot;the assassin,&quot; and &quot;injustice cannot last long&quot; are all expressive subjective elements. Expressive subjective elements are used by people to express their frustration, anger, wonder, positive sentiment, etc., without explicitly stating that they are frustrated, angry, etc. Sarcasm and irony often involve expressive subjective elements.</Paragraph> </Section> <Section position="2" start_page="53" end_page="53" type="sub_section"> <SectionTitle> 2.2 Private State Frames </SectionTitle> <Paragraph position="0"> We propose two types of private state frames: expressive subjective element frames will be used to represent expressive subjective elements; and direct subjective frames will be used to represent both subjective speech events (i.e., speech events expressing private states) and explicitly mentioned private states. The frames have the following attributes: Direct subjective (subjective speech event or explicit private state) frame: * text anchor: a pointer to the span of text that represents the speech event or explicit mention of a private state.</Paragraph> <Paragraph position="1"> * source: the person or entity that expresses or experiences the private state, possibly the writer.</Paragraph> <Paragraph position="2"> * target: the target or topic of the private state, i.e., what the speech event or private state is about.</Paragraph> <Paragraph position="3"> * properties: - intensity: the intensity of the private state (low, medium, high, or extreme).</Paragraph> <Paragraph position="4"> - expression intensity: the contribution of the speech event or private state expression itself to the overall intensity of the private state. For example, &quot;say&quot; is often neutral, even if what is uttered is not neutral, while &quot;excoriate&quot; itself implies a very strong private state.</Paragraph> <Paragraph position="5"> - insubstantial: true, if the private state is not substantial in the discourse. For example, a private state in the context of a conditional often has the value true for attribute insubstantial.</Paragraph> <Paragraph position="6"> - attitude type: the type of attitude(s) composing the private state.</Paragraph> <Paragraph position="7"> Expressive subjective element frame: * text anchor: a pointer to the span of text that de- null notes the subjective or expressive phrase.</Paragraph> <Paragraph position="8"> * source: the person or entity that is expressing the private state, possibly the writer.</Paragraph> <Paragraph position="9"> * properties: - intensity: the intensity of the private state. - attitude type</Paragraph> </Section> <Section position="3" start_page="53" end_page="53" type="sub_section"> <SectionTitle> 2.3 Objective Speech Event Frames </SectionTitle> <Paragraph position="0"> To distinguish opinion-oriented material from material presented as factual, we also define objective speech event frames. These are used to represent material that is attributed to some source, but is presented as objective fact. They include a subset of the slots in private state frames: Objective speech event frame: * text anchor: a pointer to the span of text that denotes the speech event.</Paragraph> <Paragraph position="1"> * source: the speaker or writer.</Paragraph> <Paragraph position="2"> * target: the target or topic of the speech event, i.e., the content of what is said.</Paragraph> <Paragraph position="3"> For example, an objective speech event frame is created for &quot;said&quot; in the following sentence (assuming no undue influence from the context): (4) Sargeant O'Leary said the incident took place at 2:00pm.</Paragraph> <Paragraph position="4"> That the incident took place at 2:00pm is presented as a fact with Sargeant O'Leary as the source of information.</Paragraph> </Section> <Section position="4" start_page="53" end_page="53" type="sub_section"> <SectionTitle> 2.4 Agent Frames </SectionTitle> <Paragraph position="0"> The annotation scheme includes an agent frame for noun phrases that refer to sources of private states and speech events, i.e., for all noun phrases that act as the experiencer of a private state, or the speaker/writer of a speech event. Each agent frame generally has two slots. The text anchor slot includes a pointer to the span of text that denotes the noun phrase source. The source slot contains a unique alpha-numeric ID that is used to denote this source throughout the document. The agent frame associated with the first informative (e.g., non-pronominal) reference to this source in the document includes an id slot to set up the document-specific source-id mapping.</Paragraph> </Section> <Section position="5" start_page="53" end_page="54" type="sub_section"> <SectionTitle> 2.5 Nested Sources </SectionTitle> <Paragraph position="0"> The source of a speech event is the speaker or writer. The source of a private state is the experiencer of the private state, i.e., the person whose opinion or emotion is being expressed. The writer of an article is always a source, because he or she wrote the sentences of the article, but the writer may also write about other people's private states and speech events, leading to multiple sources in a single sentence. For example, each of the following sentences has two sources: the writer (because he or she wrote the sentences), and Sue (because she is the source of a speech event in (5) and of private states in (6) and (7)).</Paragraph> <Paragraph position="1"> (5) Sue said, &quot;The election was fair.&quot; (6) Sue thinks that the election was fair.</Paragraph> <Paragraph position="2"> (7) Sue is afraid to go outside.</Paragraph> <Paragraph position="3"> Note, however, that we don't really know what Sue says, thinks or feels. All we know is what the writer tells us. For example, Sentence (5) does not directly present Sue's speech event but rather Sue's speech event according to the writer. Thus, we have a natural nesting of sources in a sentence.</Paragraph> <Paragraph position="4"> In particular, private states are often filtered through the &quot;eyes&quot; of another source, and private states are often directed toward the private states of others. Consider sentence (1) above and (8) following: (8) China criticized the U.S. report's criticism of China's human rights record.</Paragraph> <Paragraph position="5"> In sentence (1), the U.S. does not directly state its fear. Rather, according to the writer, according to Xirao-Nima, the U.S. fears a spill-over. The source of the private state expressed by &quot;fears&quot; is thus the nested source <writer, Xirao-Nima, U.S.> . In sentence (8), the U.S. report's criticism is the target of China's criticism. Thus, the nested source for &quot;criticism&quot; is <writer, China, U.S. report> . Note that the shallowest (left-most) agent of all nested sources is the writer, since he or she wrote the sentence. In addition, nested source annotations are composed of the IDs associated with each source, as described in the previous subsection. Thus, for example, the nested source <writer, China, U.S. report> would be represented using the IDs associated with the writer, China, and the report being referred to, respectively.</Paragraph> </Section> <Section position="6" start_page="54" end_page="54" type="sub_section"> <SectionTitle> 2.6 Examples </SectionTitle> <Paragraph position="0"> We end this section with examples of direct subjective, expressive subjective element, and objective speech event frames (sans target and attitude type attributes, which are discussed in the next section).</Paragraph> <Paragraph position="1"> First, we show the frames that would be associated with sentence (9), assuming that the relevant source ID's have already been defined: The first objective speech event frame represents that, according to the writer, it is true that Xirao-Nima uttered the quote and is a professor at the university referred to. The implicit attribute is included because the writer's speech event is not explicitly mentioned in the sentence (i.e., there is no explicit phrase such as &quot;I write&quot;). The second objective speech event frame represents that, according to the writer, according to Xirao-Nima, it is true that the US fears a spillover. Finally, when we drill down to the subordinate clause we find a private state: the US fear of a spillover. Such detailed analyses, encoded as annotations on the input text, would enable a person or an automated system to pinpoint the subjectivity in a sentence, and attribute it appropriately.</Paragraph> <Paragraph position="2"> Now, consider sentence (10): (10) &quot;The report is full of absurdities,&quot; Xirao-Nima said.</Paragraph> <Paragraph position="3"> Objective speech event: Text anchor: the entire sentence The objective frame represents that, according to the writer, it is true that Xirao-Nima uttered the quoted string. The second frame is created for &quot;said&quot; because it is a subjective speech event: private states are conveyed in what is uttered. Note that intensity is high but expression intensity is neutral: the private state being expressed is strong, but the specific speech event phrase &quot;said&quot; does not itself contribute to the intensity of the private state. The third frame is for the expressive subjective element &quot;full of absurdities.&quot;</Paragraph> </Section> </Section> <Section position="4" start_page="54" end_page="55" type="metho"> <SectionTitle> 3 Annotation Process </SectionTitle> <Paragraph position="0"> To date, over 11,000 sentences in 550 documents have been annotated according to the annotation scheme described above. The documents are English-language versions of news documents from the world press. The documents are from 187 different news sources in a variety of countries. The original documents and their annotations are available at http://nrrc.mitre.org/NRRC/publications.htm.</Paragraph> <Paragraph position="1"> The annotation process and inter-annotator agreement studies are described in (Wiebe et al., 2005). Here, we want to highlight two themes of the annotation instructions: null 1. There are no fixed rules about how particular words should be annotated. The instructions describe the annotations of specific examples, but do not state that specific words should always be annotated a certain way.</Paragraph> <Paragraph position="2"> 2. Sentences should be interpreted with respect to the contexts in which they appear. The annotators should not take sentences out of context and think what they could mean, but rather should judge them as they are being used in that particular sentence and document.</Paragraph> <Paragraph position="3"> We believe that these general strategies for annotation support the creation of corpora that will be useful for studying expressions of subjectivity in context.</Paragraph> </Section> <Section position="5" start_page="55" end_page="59" type="metho"> <SectionTitle> 4 Extensions: Attitude and Target Annotations </SectionTitle> <Paragraph position="0"> Before we describe the new attitude and target annotations, consider the following sentence.</Paragraph> <Paragraph position="1"> (11) &quot;I think people are happy because Chavez has fallen.&quot; This sentence contains two private states, represented by direct subjective annotations anchored on &quot;think&quot; and &quot;happy,&quot; respectively.</Paragraph> <Paragraph position="2"> The word &quot;think&quot; is used to express an opinion about what is true according to its source (a positive arguing attitude type; see Section 4.1). The target of &quot;think&quot; is &quot;people are happy because Chavez has fallen.&quot; The word &quot;happy&quot; clearly expresses a positive attitude, with target &quot;Chavez has fallen.&quot; However, looking more closely at the private state for &quot;happy,&quot; we see that we can also infer a negative attitude toward Chavez, from the phrase &quot;happy because Chavez has fallen.&quot; Sentence (11) illustrates some of the things we need to consider when representing attitudes and targets. First, we see that more than one type of attitude may be involved when a private state is expressed. In (11), there are three (a positive attitude, a negative attitude, and a positive arguing attitude). Second, more than one target may be associated with a private state. Consider &quot;happy&quot; in (11). The target of the positive attitude is &quot;Chavez has fallen,&quot; while the target of the inferred negative attitude The representation also must support multiple targets for a single attitude, as illustrated by Sentence (12): (12) Tsvangirai said the election result was a clear case of highway robbery by Mugabe, his government and his party, Zanu-PF.</Paragraph> <Paragraph position="3"> In (12), the phrase &quot;a clear case of highway robbery&quot; expresses a negative attitude of Tsvangirai. This negative attitude has two targets: &quot;the election results&quot; and &quot;Mugabe, his government and his party, Zanu-PF.&quot; To capture the kind of detailed attitude and target information that we described above, we propose two new types of annotations: attitude frames and target frames.</Paragraph> <Paragraph position="4"> We describe these new annotations in Sections 4.2 and 4.3, but first we introduce the set of attitude types that we developed for the annotation scheme.</Paragraph> <Section position="1" start_page="55" end_page="56" type="sub_section"> <SectionTitle> 4.1 Types of Attitudes </SectionTitle> <Paragraph position="0"> One of our goals in extending the annotation scheme for private states was to develop a set of attitude types that would be useful for NLP applications. It it also important that the set of attitude types provide good coverage for the range of possible private states. Working with our annotators and looking at the private states already annotated, we developed the set of attitude types listed in Table 1.</Paragraph> <Paragraph position="1"> Below we give a brief description of each attitude type, followed by an example. In each example, the span of text that expresses the attitude type is in bold, and the span of text that refers to the target of the attitude type (if a target is given) is in angle brackets.</Paragraph> <Paragraph position="2"> Positive Attitudes: positive emotions, evaluations, judgments and stances.</Paragraph> <Paragraph position="3"> (13) The Namibians went as far as to say <Zimbabwe's election system> was &quot;water tight, without room for rigging&quot;.</Paragraph> <Paragraph position="4"> Negative Attitudes: negative emotions, evaluations, judgments and stances.</Paragraph> <Paragraph position="5"> (14) His disenfranchised supporters were seething.</Paragraph> <Paragraph position="6"> Positive Arguing: arguing for something, arguing that something is true or so, arguing that something did happen or will happen, etc.</Paragraph> <Paragraph position="7"> (15) Iran insists <its nuclear program is purely for peaceful purposes> .</Paragraph> <Paragraph position="8"> Negative Arguing: arguing against something, arguing that something is not true or not so, arguing that something did not happen or will not happen, etc.</Paragraph> <Paragraph position="9"> (16) Officials in Panama denied that <Mr.</Paragraph> <Paragraph position="10"> Chavez or any of his family members had asked for asylum> .</Paragraph> <Paragraph position="11"> Positive Intentions: aims, goals, plans, and other overtly expressed intentions.</Paragraph> <Paragraph position="12"> (17) The Republic of China government believes in the US committment <to separating its anti-terrorism campaign from the Taiwan Strait issue> , an official said Thursday.</Paragraph> <Paragraph position="13"> Negative Intentions: expressing that something is not an aim, not a goal, not an intention, etc.</Paragraph> <Paragraph position="14"> (18) The Bush administration has no plans <to ease sanctions against mainland China> .</Paragraph> <Paragraph position="15"> Speculation: speculation or uncertainty about what may or may not be true, what may or may not happen, etc. (19) <The president is likely to endorse the bill> .</Paragraph> <Paragraph position="16"> Other Attitudes: other types of attitudes that do not fall into one of the above categories.</Paragraph> <Paragraph position="17"> (20) To the surprise of many, <the dollar hit only 2.4 pesos and closed at 2.1> .</Paragraph> </Section> <Section position="2" start_page="56" end_page="57" type="sub_section"> <SectionTitle> 4.2 Attitude Frames </SectionTitle> <Paragraph position="0"> With the introduction of the attitude frames, two issues arise. First, which spans of text should the new attitudes be anchored to? Second, how do we tie the attitude frames back to the private states that they are part of? The following sentence illustrates the first issue.</Paragraph> <Paragraph position="1"> (21) The MDC leader said systematic cheating, spoiling tactics, rigid new laws, and shear obstruction - as well as political violence and intimidation - were just some of the irregularities practised by the authorities in the run-up to, and during the poll.</Paragraph> <Paragraph position="2"> In (21), there are 5 private state frames attributed to the MDC leader: a direct subjective frame anchored to &quot;said,&quot; and four expressive subjective element frames anchored respectively to &quot;systematic cheating . . . obstruction,&quot; &quot;as well as,&quot; &quot;violence and intimidation,&quot; and &quot;just some of the irregularities.&quot; We could create an attitude frame for each of these private state frames, but we believe the following is a better solution. For each direct subjective frame, the annotator is asked to consider the direct subjective annotation and everything within the scope of the annotation when deciding what attitude types are being expressed by the source of the direct subjective frame. Then, for each attitude type identified, the annotator creates an attitude frame and anchors the frame to whatever span of text completely captures the attitude type. In to sentence (21), this results in just one attitude frame being created to represent the negative attitude of the MDC leader. The anchor for this attitude frame begins with &quot;systematic cheating&quot; and ends with &quot;irregularities.&quot; Turning to the second issue, tying attitude frames to their private states, we do two things. First, we create a unique ID for the attitude frame. Then, we change the attitude type attribute on the direct subjective annotation into a new attribute called an attitude link. We place the attitude frame ID into the attitude link slot. The attitude link slot can hold more then one attitude frame ID, allowing us to represent a private state composed of more than one type of attitude.</Paragraph> <Paragraph position="3"> Because we expect the attitude annotations to overlap with most of the expressive subjective element annotations, we chose not to link attitude frames to expressive subjective element frames. However, this would be possible to do should it become necessary.</Paragraph> <Paragraph position="4"> The attitude frame has the following attributes: Attitude frame: * id: a unique alphanumeric ID for identifying the attitude annotation. The ID is used to link the attitude annotation to the private state it is part of.</Paragraph> <Paragraph position="5"> * text anchor: a pointer to the span of text that captures the attitude being expressed.</Paragraph> <Paragraph position="6"> * attitude type: one of the attitude types listed in Table 1.</Paragraph> <Paragraph position="7"> * target link: one or more target annotation IDs (see Section 4.3).</Paragraph> <Paragraph position="8"> * intensity: the intensity of the attitude.</Paragraph> <Paragraph position="9"> * properties: - inferred: true, if the attitude is inferred.</Paragraph> <Paragraph position="10"> - sarcastic: true, if the attitude is realized through sarcasm.</Paragraph> <Paragraph position="11"> - repetition: true, if the attitude is realized through the repetition of words, phrases, or syntax.</Paragraph> <Paragraph position="12"> - contrast: true, if the attitude is realized only through contrast with another attitude.</Paragraph> <Paragraph position="13"> Of the four attitude-frame properties, inferred was already discussed. The property sarcastic marks attitudes expressed using sarcasm. In general, we think this prop-erty will be of interest for NLP applications working with opinions. Detecting sarcasm may also help a system learn to distinguish between positive and negative attitudes. The sarcasm in Sentence (22), below, makes the word &quot;Great&quot; an expression of a negative rather than a positive attitude.</Paragraph> <Paragraph position="14"> (22) &quot;Great, keep on buying dollars so there'll be more and more poor people in the country,&quot; shouted one.</Paragraph> <Paragraph position="15"> The repetition and contrast properties are also for marking different ways in which an attitude might be realized. We feel these properties will be useful for developing an automatic system for recognizing different types of attitudes. null</Paragraph> </Section> <Section position="3" start_page="57" end_page="57" type="sub_section"> <SectionTitle> 4.3 Target Frames </SectionTitle> <Paragraph position="0"> The target frame is used to mark the target of each attitude. A target frame has two slots, the id slot and the text anchor slot. The id slot contains a unique alpha-numeric ID for identifying the target annotation. We use the target frame ID to link the target back to the attitude frame. The attitude frame has a target-link slot that can hold one or more target frame IDs. This allows us to represent when a single attitude is directed at more than one target.</Paragraph> <Paragraph position="1"> The text anchor slot has a pointer to the span of text that denotes the target. If there is more than one reference to the target in the sentence, the most syntactically relevant reference is chosen.</Paragraph> <Paragraph position="2"> To illustrate what we mean by syntactically relevant, consider the following sentence.</Paragraph> <Paragraph position="3"> (23) African observers generally approved of <his victory> while Western governments denounced <it> .</Paragraph> <Paragraph position="4"> The target of the two attitudes (in bold) in the above sentence is the same entity in the discourse. However, although we anchor the target for the first attitude to &quot;his victory,&quot; the anchor for the target of the second attitude is the pronoun &quot;it.&quot; As the direct object of the span that denotes the attitude &quot;denounced,&quot; &quot;it&quot; is more syntactically relevant than &quot;his victory.&quot;</Paragraph> </Section> <Section position="4" start_page="57" end_page="59" type="sub_section"> <SectionTitle> 4.4 Illustrative Examples </SectionTitle> <Paragraph position="0"> Figures 4.4 and 4.4 give graphical representations for the annotations in sentences (11) and (12). With attitude frame and target frame extensions, we are able to capture more detail about the private states being expressed in the text than the original core scheme presented in (Wiebe et al., 2005).</Paragraph> <Paragraph position="1"> 5 Pie in the Sky Annotation Among the items on the &quot;Pie in the Sky&quot; Check List of Desirable Semantic Information for Annotation, 1 the most closely related are epistemic values (&quot;attitude?&quot;), epistemic, deontic, and personal attitudes. These all fundamentally involve a self (Banfield, 1982), a subject of consciousness who is the source of knowledge assessments, judgments of certainty, judgments of obligation/permission, personal attitudes, and so on. Any explicit epistemic, deontic, or personal attitude expressions are represented by us as private state frames, either direct subjective frames (e.g., for verbs such as &quot;know&quot; referring to an epistemic state) or expressive subjective element frames (e.g., for modals such as &quot;must&quot; or &quot;ought to&quot;). Importantly, many deontic, epistemic, and personal attitude expressions do not directly express the speaker or writer's subjectivity, but are attributed by the speaker or writer to agents mentioned in the text (consider, e.g., &quot;John believes that Mary should quit her job&quot;). Our frame and nested-source representations were designed to support attributing subjectivity to appropriate sources. In future work, additional attributes could be added to private state frames to distinguish between, for example, deontic and epistemic usages of &quot;must&quot; and to represent different epistemic values.</Paragraph> <Paragraph position="2"> Other phenomena on the list overlap with subjectivity, such as modality and social style/register. As mentioned above, some modal expressions are subjective, such as those expressing deontic or epistemic judgments. However, hypotheticals and future expressions need not be subjective. For example, &quot;The company announced that if its profits decrease in the next quarter, it will lay off some employees&quot; may easily be interpreted as presenting objective fact. As for style, some are subjective by their nature. One is the literary style represented thought, used to present consciousness in fiction (Cohn, 1978; Banfield, 1982). Others are sarcastic or dismissive styles of speaking or writing. In our annotation scheme, sentences perceived to represent a character's consciousness are represented with private-state frames, as are expressions perceived to be sarcastic or dismissive. On the other hand, some style distinctions, such as degree of formality, are often realized in other ways than with explicit subjective expressions (e.g., &quot;can't&quot; versus &quot;cannot&quot;). Polarity, another item on the checklist, also overlaps with subjective positive and negative attitude types. Although many negative and positive polarity words are seldom used outside subjective expressions (such as &quot;hate&quot; and &quot;love&quot;), others often are. For example, words such as &quot;addicted&quot; and &quot;abandoned&quot; are included as negative polarity terms in the General Inquirer lexicon (General-Inquirer, 2000), but they can easily appear in objective d i r e c t s u b j e c t i v e f r a m e t e x t a n c h o r : t h i n k s o u r c e : < w r i t e r , I > i n t e n s i t y : m e d i u m e x p r e s s i o n i n t e n s i t y : m e d i u m a t t i t u d e l i n k : a 1 0 a t t i t u d e f r a m e</Paragraph> <Paragraph position="4"> a t t i t u d e t y p e : p o s i t i v e a r g u i n g i n t e n s i t y : m e d i u m t a r g e t l i n k : t 1 0 d i r e c t s u b j e c t i v e f r a m e t e x t a n c h o r : a r e h a p p y s o u r c e : < w r i t e r , I , p e o p l e > i n t e n s i t y : m e d i u m e x p r e s s i o n i n t e n s i t y : m e d i u m a t t i t u d e l i n k : a 2 0 , a 3 0</Paragraph> <Paragraph position="6"> C h a v e z h a s f a l l e n a t t i t u d e t y p e : n e g a t i v e a t t i t u d e</Paragraph> <Paragraph position="8"> d i r e c t s u b j e c t i v e f r a m e t e x t a n c h o r : s a i d s o u r c e : < w r i t e r , T s v a n g i r a i > i n t e n s i t y : h i g h e x p r e s s i o n i n t e n s i t y : n e u t r a l a t t i t u d e l i n k : a 4 0 a t t i t u d e f r a m e</Paragraph> <Paragraph position="10"> a n d h i s p a r t y , Z a n u - P F o b j e c t i v e s p e e c h e v e n t</Paragraph> <Paragraph position="12"> sentences (e.g., &quot;Thomas De Quincy was addicted to opium and lived in an abandoned shack&quot;).</Paragraph> <Paragraph position="13"> Integrating subjectivity with other layers of annotation proposed in the &quot;Pie in the Sky&quot; project would afford the opportunity to investigate how they interact. It would also enrich our subjectivity representations. While our scheme promises to be a good base, much remains to be added. For example, annotations of thematic roles and co-reference would add needed structure to the target annotations, which are now only spans of text. In addition, temporal and modal annotations would flesh out the insubstantial attribute, which is currently only a binary marker. Furthermore, individual private state expressions must be integrated with respect to the discourse context.</Paragraph> <Paragraph position="14"> For example, which expressions of opinions oppose versus support one another? Which sentences presented as objective fact are included to support a subjective opinion? A challenging dimension to add to the &quot;Pie in the Sky&quot; project would be the deictic center as conceived of in (Duchan et al., 1995), which consists of here, now, and I reference points updated as the text or conversation unfolds. Our annotation scheme was developed with this framework in mind.</Paragraph> </Section> </Section> <Section position="6" start_page="59" end_page="59" type="metho"> <SectionTitle> 6 Related Work </SectionTitle> <Paragraph position="0"> The work most similar to ours is Appraisal Theory (Martin, 2000; White, 2002) from systemic functional linguistics (see Halliday (19851994)). Both Appraisal Theory and our annotation scheme are concerned with identifying and characterizing expressions of opinions and emotions in context. The two schemes, however, make different distinctions. Appraisal Theory distinguishes different types of positive and negative attitudes and also various types of &quot;intersubjective positioning&quot; such as attribution and expectation. Appraisal Theory does not distinguish, as we do, the different ways that private states may be expressed (i.e., directly, or indirectly using expressive subjective elements). It also does not include a representation for nested levels of attribution.</Paragraph> <Paragraph position="1"> In addition to Appraisal Theory, subjectivity annotation of text in context has also been performed in Yu and Hatzivassiloglou (2003), Bruce and Wiebe (1999), and Wiebe et al. (2004). The annotations in Yu and Hatzivassiloglou (2003) are sentence-level subjective vs. objective and polarity judgments. The annotation schemes used in Bruce and Wiebe (1999) and Wiebe et al. (2004) are earlier, much less detailed versions of the annotation scheme presented in this paper.</Paragraph> </Section> class="xml-element"></Paper>