<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1014">
  <Title>Evaluation Measures Considering Sentence Concatenation for Automatic Summarization by Sentence or Word Extraction</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
NEWS
</SectionTitle>
    <Paragraph position="0"> sentence strings (PREC3 to PREC5) didn't reflect the human judgments for all the conditions. These results show that meanings of the original article can maintain by the concatenations of only a few sentences in summarization through sentence extraction. null Table 2 lists the kappa statistics for the manual summaries and the human judgments for EDIT and NEWS. The manual results varied among humans</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
DATA SUMMARIES
</SectionTitle>
    <Paragraph position="0"> human judgments for sentence extraction.</Paragraph>
    <Paragraph position="1"> and the similarity among humans was low. The kappa statistics for NEWS is slightly higher than that for EDIT. The difference of similarities among manual summaries is due to the difference in structures of information in each article. Although the articles in EDIT had a discourse structure, NEWS had isolated and stereotyped information scattered throughout the articles.</Paragraph>
    <Paragraph position="2"> While the human judgments for NEWS were similar, those for EDIT varied.The difficulty in evaluating COH and SEM in EDIT is due to the variation in both manual summaries and human judgment.</Paragraph>
    <Paragraph position="3"> Figure 4 shows the correlation coefficients between the judgments of the subjects and the numerical evaluation results for summaries of broadcast news speech through word extraction. Table 3 lists  judgment and numerical evaluation results for summaries through word extraction the kappa statistics for the manual summaries and the human judgments for summaries through word extraction. In word extraction, the human judg-</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
DATA SUMMARIES
</SectionTitle>
    <Paragraph position="0"> human judgments for word extraction ments and the manual summaries were very similar among the subjects.</Paragraph>
    <Paragraph position="1"> As shown in figure 4, WSumACCY yielded the best correlation to the human judgments. This means that the correctness as a sentence and the weight (that is how many subjects support the extracted phrases in summarized sentences) are important in summarization through word extraction. In comparison with the results of sentence extraction in Figures 2 and 3, PREC1 effectively reflected the human judgments for word extraction. Since in the manual summarized sentences through word extraction under the low summarization ratio, the sentences were summarized based on significance word extraction rather than syntactic structure maintenance to generate grammatically correct sentences.</Paragraph>
  </Section>
class="xml-element"></Paper>