<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1121">
  <Title>Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis</Title>
  <Section position="4" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Data
</SectionTitle>
    <Paragraph position="0"> Our data consists of 11399 feedback items from a Global Support Services survey, and 29485 feedback items from a Knowledge Base survey for a total of 40884 items. We excluded pieces of feedback without any verbatim from the data.</Paragraph>
    <Paragraph position="1"> Along with the verbatim, customers provided a numeric satisfaction score on a scale from 1 (not satisfied) to 4 (very satisfied) for each of those pieces of feedback. The numeric score served as the target tag in our experiments, making it unnecessary to perform any costly human evaluation and tagging. The distribution of items across numerical scores is given in Table 1.</Paragraph>
    <Paragraph position="2">  satisfaction category The data is extremely noisy, and a human evaluation of a random set of 200 pieces of feedback could only assign a positive or negative sentiment to 117 (58.5%) items, the rest was either balanced (16 cases or 8%), expressed no sentiment (50 cases or 25%), or too incoherent or random to be classified (17 cases or 8.5%). Amongst the 117 classifiable cases, the human evaluator assigned the category &amp;quot;positive&amp;quot;: to 26 cases (or 22.2%) and the category &amp;quot;negative&amp;quot; to 91 cases (or 77.8%). After automatic sentence breaking into one sentence per line, the individual files contained an average of 2.56 lines. For our experiments we split the data 90/10 into training and held-out test data. We performed 10-fold cross validation for each of the experiments reported in this paper.</Paragraph>
    <Paragraph position="3"> For each of the various classification tasks, we trained a linear SVM using the standard settings of the SMO tool, and calculated accuracy, precision and recall numbers on the held-out test data, averaging them across the 10-fold cross validation.</Paragraph>
  </Section>
class="xml-element"></Paper>