<?xml version="1.0" standalone="yes"?>
<Paper uid="P02-1059">
  <Title>Supervised Ranking in Open-Domain Text Summarization</Title>
  <Section position="8" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
7 Results and Discussion
</SectionTitle>
    <Paragraph position="0"> Tables 4-8 show performance of each ProbDT and its combination with the diversity (clustering) component. It also shows performance of Z model and DBS. In the tables, the slashed 'V' after the name of a classifier indicates that the relevant classifier is diversity-enabled, meaning that it is coupled with the diversity extension. Notice that each decision tree here is a ProbDT and should not be confused with its non-probabilistic counterpart. Also worth noting is that DBS is in fact Z/V, that is, diversity-enabled Z model.</Paragraph>
    <Paragraph position="1"> Returning to the tables, we find that for most of the times, the diversity component has clear effects on ProbDTs, significantly improving their performance. All the figures are in F-measure, i.e., F = 2/P/RP+R . In fact this happens regardless of a particular choice of ranking model, as performance of Z is also boosted with the diversity component. Not surprisingly, effects of supervised learning are also evident: diversity-enabled ProbDTs generally out-perform DBS (Z/V) by a large margin. What is surprising, moreover, is that diversity-enabled ProbDTs are superior in performance to their non-diversity counterparts (with a notable exception for SSDT at K , 1), which suggests that selecting marginal sentences is an important part of generating a summary.</Paragraph>
    <Paragraph position="2"> Another observation about the results is that as one goes along with a larger K, differences in performance among the systems become ever smaller: at K , 5, Z performs comparably to C4.5, MDL, and SSDT either with or without the diversity component. The decline of performance of the DTs may be caused by either the absence of recurring patterns in data with a higher K or simply the paucity of positive instances. At the moment, we do not know which is the case here.</Paragraph>
    <Paragraph position="3"> It is curious to note, moreover, that MDL-DT is not performing as well as C4.5 and SSDT at K , 1, K , 2, and K , 3. The reason may well have to do with the general properties of MDL-DT. Recall that MDL-DT is designed to produce as small a decision tree as possible. Therefore, the resulting tree would have a very small number of nodes covering the entire data space. Consider, for instance, a hypothetical data space in Figure 3. Assume that MDL-DT bisects the space into region A and B, producing a two-node decision tree. The problem with the tree is, of course, that point x and y in region B will be assigned to the same probability under the probabilistic tree model, despite the fact that point x is very close to region A and point y is far out. This problem could happen with C4.5, but in MDL-DT, which covers a large space with a few nodes, points in a region could be far apart, making the problem more acute. Thus the poor performance of MDL-DT may be attributable to its extensive use of pruning.</Paragraph>
  </Section>
class="xml-element"></Paper>