<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1008">
  <Title>Task-focused Summarization of Email</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Features
</SectionTitle>
    <Paragraph position="0"> Each sentence in the message body is described by a vector of approximately 53,000 features.</Paragraph>
    <Paragraph position="1"> The features are of three types: properties of the message (such as the number of addressees, the total size of the message, and the number of forwarded sections in the email thread), superficial features and linguistic features.</Paragraph>
    <Paragraph position="2"> The superficial features include word unigrams, bigrams and trigrams as well as counts of special punctuation symbols (e.g. @, /, #), whether the sentence contains words with so-called &amp;quot;camel caps&amp;quot; (e.g., SmartMail), whether the sentence appears to contain the sender's name or initials, and whether the sentence contains one of the addressees' names.</Paragraph>
    <Paragraph position="3"> The linguistic features were obtained by analyzing the given sentence using the NLPWin system (Heidorn 2000). The linguistic features include abstract lexical features, such as part-of-speech bigrams and trigrams, and structural features that characterize the constituent structure in the form of context-free phrase structure rewrites (e.g., DECL:NP-VERB-NP; i.e., a declarative sentence consisting of a noun phrase followed by a verb and another noun phrase). Deeper linguistic analysis yielded features that describe part-of-speech information coupled with grammatical relations (e.g., Verb-Subject-Noun indicating a nominal subject of a verb) and features of the logical form analysis such as transitivity, tense and mood.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Results
</SectionTitle>
    <Paragraph position="0"> We trained support vector machines (SVMs) (Vapnik, 1995) using an implementation of the sequential minimal optimization algorithm (Platt, 1999). We trained linear SVMs, which have proven effective in text categorization with large feature vectors (Joachims, 1998; Dumais et al., 1998).</Paragraph>
    <Paragraph position="1"> Figure 1 illustrates the precision-recall curve for the SVM classifier trained to distinguish tasks vs. non-tasks measured on the blind test set.</Paragraph>
    <Paragraph position="2"> We conducted feature ablation experiments on the development test set to assess the contribution of categories of features to overall classification performance. In particular we were interested in the role of linguistic analysis features compared to using only surface features. Within the linguistic features, we distinguished deep linguistic features (phrase structure features and semantic features) from POS n-gram features. We conducted experiments with three feature sets:  1. all features (message level features + word unigram, bigram and trigram 2. features + POS bigram and trigram features + linguistic analysis features) 3. no deep linguistic features (no phrase  structure or semantic features) 4. no linguistic features at all (no deep linguistic features and no POS n-gram features) Based on these experiments on the development test set, we chose the feature set used for our run-time applications.</Paragraph>
    <Paragraph position="3"> Figure 1 shows final results for these feature sets on the blind test set: for recall between approximately 0.2 and 0.4 and between approximately 0.5 and 0.6 the use of all features produces the best results. The distinction between the &amp;quot;no linguistic features&amp;quot; and &amp;quot;no deep linguistic features&amp;quot; scenarios is negligible; word n-grams appear to be highly predictive.</Paragraph>
    <Paragraph position="4"> Based on these results, we expect that for languages where we do not have an NLPWin parser, we can safely exclude the deeper linguistic features and still expect good classifier performance.</Paragraph>
    <Paragraph position="5"> Figure 2 illustrates the accuracy of distinguishing messages that contain tasks from those that do not, using all features. A message was marked as containing a task if it contained at least one sentence classified as a task. Since only one task has to be found in order for the entire message to be classified as containing a task, accuracy is substantially higher than on a per-sentence basis. In section 6, we discuss the scenarios motivating the distinction between sentence classification and message</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Reformulation of Tasks
</SectionTitle>
    <Paragraph position="0"> SmartMail performs post-processing of sentences identified as containing a task to reformulate them as task-like imperatives. The process of reformulation involves four distinct knowledge- null engineered steps: 1. Produce a logical form (LF) for the  extracted sentence (Campbell and Suzuki, 2001). The nodes of the LF correspond to syntactic constituents. Edges in the LF represent semantic and deep syntactic relations among nodes. Nodes bear semantic features such as tense, number and mood.</Paragraph>
    <Paragraph position="1"> 2. Identify the clause in the logical form that contains the task; this may be the entire sentence or a subpart. We consider such linguistic properties as whether the clause is imperative, whether its subject is second person, and whether modality words such as please or a modal verb are used. All parts of the logical form not subsumed by the task clause are pruned.</Paragraph>
    <Paragraph position="2"> 3. Transform the task portion of the LF to exclude extraneous words (e.g. please, must, could), extraneous subordinate clauses, adverbial modifiers, and vocative phrases. We replace certain deictic elements (i.e., words or phrases whose denotation varies according to the writer or the time and place of utterance) with nondeictic expressions. For example, first person pronouns are replaced by either the name of the sender of the email or by a third person pronoun, if such a pronoun would unambiguously refer to the sender.</Paragraph>
    <Paragraph position="3"> Similarly, a temporal expression such as Thursday, which may refer to a different date depending on the week in which it is written, is replaced by an absolute date (e.g., 4/1/2004).</Paragraph>
    <Paragraph position="4"> 4. Pass the transformed LF to a sentence realization module to yield a string (Aikawa et al., 2001).</Paragraph>
    <Paragraph position="5"> Below we illustrate the reformulation of tasks with some examples from our corpus.</Paragraph>
    <Paragraph position="6">  On the H-1 visa issue, I am positive that you need to go to the Embassy in London to get your visa stamped into your passport.</Paragraph>
    <Paragraph position="7"> Reformulation: Go to the Embassy in London to get your visa stamped into your passport.</Paragraph>
    <Paragraph position="8"> In this example, the embedded sentential complement, that is, the part of the sentence following positive, is selected as the part of the sentence containing the task, because of the modal verb need and the second person subject; only that part of the sentence gets reformulated. The modal verb and the second person subject are deleted to form an imperative sentence.</Paragraph>
    <Paragraph position="9">  Can you please send me the follow up information for the demo(s) listed in this Email ASAP.</Paragraph>
    <Paragraph position="10"> Reformulation: Send Kendall the follow up information for the demo listed in this Email ASAP.</Paragraph>
    <Paragraph position="11"> In this example, the whole sentence is selected as containing the task (modal verb, second person subject); modal elements including please are deleted along with the second person subject to form an imperative. In addition, the first person pronoun me is replaced by a reference to the sender, Kendall in this instance.</Paragraph>
    <Paragraph position="12">  it be to use Amalgam for learning requirements or code corpus structures and rules (and eventually rephrase them in some way).' This example illustrates what happens when NLPWin is unable to produce a spanning parse and hence a coherent LF; in this case NLPWin misanalyzed the clause following wondering as a main clause, instead of correctly analyzing it as a complement clause. SmartMail's back-off strategy for non-spanning parses is to enclose the entire original sentence in quotes, prefixed with a matrix sentence indicating the date and the name of the sender.</Paragraph>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
6 Task-Focused Summarization
</SectionTitle>
    <Paragraph position="0"> We have considered several scenarios for presenting the tasks that SmartMail identifies.</Paragraph>
    <Paragraph position="1"> Under the most radical scenario, SmartMail would automatically add extracted tasks to the user's &amp;quot;to do&amp;quot; list. This scenario has received a fairly negative receptio n when we have suggested it to potential users of a prototype. From an application perspective, this scenario is &amp;quot;fail hard&amp;quot;; i.e., classification errors might result in garbage being added to the &amp;quot;to do&amp;quot; list, with the result that the user would have to manually remove items. Since our goal is to reduce the workload on the user, this outcome would seem to violate the maxim &amp;quot;First, do no harm&amp;quot;.</Paragraph>
    <Paragraph position="2"> Figure 3 and Figure 4 illustrate several ideas for presenting tasks to the user of Microsoft Outlook.</Paragraph>
    <Paragraph position="3"> Messages that contain tasks are flagged, using the existing flag icons in Outlook for proof of concept.</Paragraph>
    <Paragraph position="4"> Users can sort mail to see all messages containing tasks. This visualization amounts to summarizing the message down to one bit, i.e., +/- Task, and is conceptually equivalent to performing document classification.</Paragraph>
    <Paragraph position="5"> The right-hand pane in Figure 3 is magnified as Figure 4 and shows two more visualizations. At the top of the pane, the tasks that have been identified are presented in one place, with a check box beside them. Checking the box adds the task to the Tasks or &amp;quot;to do&amp;quot; list, with a link back to the original message. This presentation is &amp;quot;fail soft&amp;quot;: the user can ignore incorrectly classified tasks, or tasks that were correctly identified but which the user does not care to add to the &amp;quot;to do&amp;quot; list. This list of tasks amounts to a task-focused summary of the document. This summary is intended to be read as a series of disconnected sentences, thus sidestepping the issue of producing a coherent text from a series of extracted sentences. In the event that users prefer to view these extracted sentences as a coherent text, it may prove desirable to attempt to improve the textual cohesion by using anaphoric links, cue phrases and so on.</Paragraph>
    <Paragraph position="6"> Finally, Figure 3 also shows tasks highlighted in context in the message, allowing the user to skim the document and read the surrounding text.</Paragraph>
    <Paragraph position="7"> In the prototype we allow the user to vary the precision and recall of the classifier by adjusting a slider (not illustrated here) that sets the probability threshold on the probability of Task.</Paragraph>
    <Paragraph position="8"> Figure 3 and Figure 4 illustrate a convention that we observed in a handful of emails: proper names occur as section headings. These names have scope over the tasks enumerated beneath them, i.e. there is a list of tasks assigned to Matt, a list assigned to Eric or Mo, and a list assigned to Mo. SmartMail does not currently detect this explicit assignment of tasks to individuals.</Paragraph>
    <Paragraph position="9"> Important properties of tasks beyond the text of the message could also be automatically extracted.</Paragraph>
    <Paragraph position="10"> For example, the schema for tasks in Outlook includes a field that specifies the due date of the task. This field could be filled with date and time information extracted from the sentence containing the task. Similarly the content of the sentence containing the task or inferences about social relationships of the email interlocutors could be used to mark the priority of tasks as High, Low, or Normal in the existing schema.</Paragraph>
  </Section>
class="xml-element"></Paper>