<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0509">
  <Title>Analysis of Semantic Classes in Medical Text for Question Answering</Title>
  <Section position="4" start_page="1" end_page="1" type="metho">
    <SectionTitle>
3 Analysis of Relations
</SectionTitle>
    <Paragraph position="0"> Recognition of individual semantic classes is not enough for text understanding; we also need to know how different entities in the same semantic class are connected, as well as what relations hold between different classes. Currently, all these relations are considered at the sentence level.</Paragraph>
    <Section position="1" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
3.1 Relations within the same semantic class
</SectionTitle>
      <Paragraph position="0"> Relations between different medications are the focus of this sub-section, as a sentence often mentioned more than one medication. Relations between diseases can be analyzed in a similar way, although they occur much less often than medications.</Paragraph>
      <Paragraph position="1"> Text from CE was analyzed manually to understand what relations are often involved and how they are represented. The text for the analysis is the same as in the class-identification task discussed above.</Paragraph>
      <Paragraph position="2"> As with classes themselves, it was found that these relations can be identified by a group of cue words or symbols. For example, the word plus refers to the COMBINATION of two or more medications, the word or, as well as a comma, often suggests the ALTERNATIVE relation, and the word versus (or v)usually implies a COMPARISON relation, as shown in the following examples:  (15) The combination of aspirin plus streptokinase significantly increased mortality at 3 months.</Paragraph>
      <Paragraph position="3"> (16) RCTs found no evidence that calcium channel antagonists, lubeluzole, aminobutyric acid (GABA) agonists, glycine antagonists, or N-methyl-D-aspartate (NMDA) antagonists improve clinical outcomes in people with acute ischaemic stroke.</Paragraph>
      <Paragraph position="4"> (17) One systematic review found no short or  long term improvement in acute ischaemic stroke with immediate systemic anticoagulants (unfractionated heparin, low molecular weight heparin, heparinoids,orspecific thrombin inhibitors) versus usual care without systemic anticoagulants.</Paragraph>
      <Paragraph position="5"> It is worth noting that in CE, the experimental conditions are often explained in the description of the outcomes, for example: (18) . . . receiving higher dose inhaled corticosteroids (3.6cm, 95% CI 3.0 to 4.2 with double dose beclometasone v 5.1cm, 95% CI 4.5 to 5.7 with salmeterol v 4.5cm, 95% CI 3.8 to 5.2 with placebo).</Paragraph>
      <Paragraph position="6">  (19) It found that . . . oral theophylline . . . versus placebo increased the mean number of symptom free days (63% with theophylline v 42% with placebo; P=0.02).</Paragraph>
      <Paragraph position="7"> (20) Studies of . . . inhaled steroid (see salme null terol v high dose inhaled corticosteroids under adult asthma).</Paragraph>
      <Paragraph position="8"> These descriptions are usually in parentheses. They are often phrases and even just fragments of strings that are not represented in a manner that is uniform with the other parts of the sentence. Their behavior is more difficult to capture and therefore the relations among the concepts in these descriptions are more difficult to identify. Because they usually are examples and data, omission of them will not affect the understanding of the whole sentence in most cases.</Paragraph>
      <Paragraph position="9"> Six common relations and their cue words were found in the text which are shown in Table 3. Cue words and symbols between medical concepts were first collected from the training text. Then the relations they signal were analyzed. Some cue words are ambiguous, for example, or, and,andwith. Or could also suggest a comparison relation although most of the time it means alternative, and could represent an alternative relation, and with could be a specification relation. It is interesting to find that and in the text when it connects two medications often suggests an alternative relation rather than a combination relation (e.g., the second and in example 5). Also, compared with versus, plus,etc.,and and with are weak cues as most of their appearances in the text do not suggest a relation between two medications.</Paragraph>
      <Paragraph position="10"> On the basis of this analysis, an automatic relation analysis process was applied to the test set, which was the same as in outcome identification. The test process was divided into two parts: one took parenthetical descriptions into account (case 1) and the other one did not (case 2). In the evaluation, for sentences that contain at least two medications, &amp;quot;correct&amp;quot; means that the relation that holds between the medications is correctly identified. We do not evaluate the relation between any two medications in a sentence; instead, we only considered two medications that are related to each other by a cue word or symbol (including those connected by cue words</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="1" end_page="1" type="metho">
    <SectionTitle>
Relation(s) Cue Words/Symbols
</SectionTitle>
    <Paragraph position="0"> comparison superior to, more than, versus, or, comparewith,between...and...</Paragraph>
    <Paragraph position="1"> alternative or, &amp;quot;,&amp;quot;, and combination plus, add to, addition of ...to..., combined use of, and, with, &amp;quot;(&amp;quot; specification with, &amp;quot;(&amp;quot; substitute substitute, substituted for preference rather than  other than the set collected from the training text). The results of the two cases are shown in Table 4. Most errors are because of the weak indicators with and and. As in the outcome identification task, both the training and test sets are rather small, as no standard annotated text is available.</Paragraph>
    <Paragraph position="2"> Some of the surface relationships in Table 3 reflect deeper relationships of the semantic classes. For example, COMPARISON, ALTERNATIVE,and PREFERENCE imply that the two (or more) medications have some common effects on the disease(s) they are applied to. The SPECIFICATION relation, on the other hand, suggests a hierarchical relation between the first medication and the following ones, in which the first medication is a higher-level concept and the following medications are at a lower level. For example, in example 17 above, systemic anticoagulants is a higher-level concept, unfractionated heparin, low molecular weight heparin,etc.,areexamples of it that lie at a lower level.</Paragraph>
    <Section position="1" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
3.2 Relations between different semantic
classes
</SectionTitle>
      <Paragraph position="0"> In a specific domain such as medicine, some default relations often hold between semantic classes. For example, a CAUSE-EFFECT relation is strongly embedded in the three semantic classes appearing in a sentence of the form: &amp;quot;medication ... disease ... outcome&amp;quot;, even if not in this exact order. This default relation helps the relation analysis because in most cases we do not need to depend on the text between the classes to understand the whole sentence. For instance, the CAUSE-EFFECT relation is very likely to express the idea that applying the intervention on the disease will have the outcome.</Paragraph>
      <Paragraph position="1"> This is another reason that semantic classes are important, especially in a specific domain.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="1" end_page="1" type="metho">
    <SectionTitle>
4 The polarity of outcomes
</SectionTitle>
    <Paragraph position="0"> Most clinical outcomes and the results of clinical trials are either positive or negative: (21) Positive: Thrombolysis reduced the risk of death or dependency at the end of the studies. null (22) Negative: In the systematic review, thrombolysis increased fatal intracranial haemorrhage compared with placebo.</Paragraph>
    <Paragraph position="1"> Polarity information is useful for several reasons. First of all, it can filter out positive outcomes if the question is about the negative aspects of a medication. Secondly, negative outcomes may be crucial even if the question does not explicitly ask about them. Finally, from the number of positive or negative descriptions of the outcome of a medication applying to a disease, clinicians can form a general idea about how &amp;quot;good&amp;quot; the medication is. As a first step in understanding opposing relations between scenarios in medical text, the polarity of outcomes was determined by an automatic classification process. null We use support vector machines (SVMs) to distinguish positive outcomes from negative ones.</Paragraph>
    <Paragraph position="2"> SVMs have been shown to be efficient in text classification tasks (Joachims, 1998). Given a training sample, the SVM finds a hyperplane with the maximal margin of separation between the two classes. The classification is then just to determine which side of the hyperplane the test sample lies in. We used the SVM light package (Joachims, 2002) in our experiment.</Paragraph>
    <Section position="1" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
4.1 Training and test examples
</SectionTitle>
      <Paragraph position="0"> The training and test sets were built by collecting sentences from different sections in CE; 772 sentences were used, 500 for training (300 positive, 200 negative), and 272 for testing (95 positive, 177 negative). All examples were labeled manually.</Paragraph>
    </Section>
    <Section position="2" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
4.2 Evaluation
</SectionTitle>
      <Paragraph position="0"> The classification used four different sets of features. The first feature set includes every unigram that appears at least three times in the whole training set. To improve the performance by attenuating the sparse data problem, in the second feature set, all names of diseases were replaced by the same tag disease. This was done by pre-processing the text using MetaMap to identify all diseases in both the training and the test examples. Then the identified diseases were replaced by the disease tag automatically. As medications often are not mentioned in outcomes, they were not generalized in this manner.</Paragraph>
      <Paragraph position="1"> The third feature set represents changes described in outcomes. Our observation is that outcomes often involve the change in a clinical value. For example, after a medication was applied to a disease, something was increased (enhanced, more, ...) or decreased (reduced, less, ...). Thus the polarity of an outcome is often determined by how change happens: if a bad thing (e.g., mortality) is reduced then it is a positive outcome; if the bad thing is increased, then the outcome is negative. We try to capture this observation by adding context features to the feature set. The way they were added is similar to incorporating the negation effect described by Pang et al. (2002). But instead of just finding a &amp;quot;negation word&amp;quot; (not, isn't, didn't, etc.), we need to find two groups of words: those indicating more and those indicating less. In the training text, we found 9 words in the first group and 7 words in the second group. When pre-processing text for classification, following the method of Pang et al., we attached the tag MORE to all words between the more-words and the following punctuation mark, and the tag LESS to the words after the less-words.</Paragraph>
      <Paragraph position="2"> The fourth feature set is the combination of the effects of feature set two and three. In representing each sentence by a feature vector, we tested both presence (feature appears or not) and frequency (count the number of occurrences of the feature in the sentence).</Paragraph>
      <Paragraph position="3"> The accuracy of the classification is shown in Table 5. The baseline is to assign a random class (here we use negative, as they are more frequent in the test set) to all test samples.</Paragraph>
      <Paragraph position="4"> The presence of features performs better than frequency of features in general. Using a more general category instead of specific diseases has a positive effect on the presence-based classification. We speculate that the effect of this generalization will be bigger if a larger test set were used. Pang et al.</Paragraph>
      <Paragraph position="5"> (2002) did not compare the result of using and not using the negation context effect, so it is not clear how much it improved their result. In our task, it is clear that the MORE/ LESS feature has a significant effect on the performance, especially for the frequency features.</Paragraph>
      <Paragraph position="6">  We have described our work in medical text analysis by identifying semantic classes and the relations between them. Our work suggests that semantic classes in medical scenarios play an important role in understanding medical text. The scenario view may be extended to a framework that acts as a guideline for further semantic analysis.</Paragraph>
      <Paragraph position="7"> Semantic classes and their relations have direct applications in medical question answering and query refinement in information retrieval. In question answering, the question and answer candidates will contain some semantic classes. After identifying them on both sides, the question can be compared with the answer to find whether there is a match. In information retrieval, relations between semantic classes can be added to the index. If the query posed by the user is too general, the system will ask the user to refine the query by adding more concepts and even relations so that it will be more pertinent according to the content of the source. For example, a user may search for a document describing the comparison of aspirin and placebo. Instead of just using aspirin and placebo as the query terms, the user can specify the comparison relation as well in the query.</Paragraph>
      <Paragraph position="8"> We will continue working on the second level of the semantic analysis, to explore the relations on the scenario level. A complete scenario contains all three semantic classes. One scenario may be the explanation or justification of the previous scenario(s), or contradictory to the previous scenario(s). Detecting these relationships will be of great help for understanding-based tasks, such as context-related question answering, topic-related summarization, etc. As different scenarios might not be adjacent to each other in the texts, classical rhetorical analysis cannot provide a complete solution for this problem.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>