<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-2176">
  <Title>Learning Correlations between Linguistic Indicators and Semantic Constraints: Reuse of Context-Dependent Descriptions of Entities</Title>
  <Section position="6" start_page="1075" end_page="1076" type="evalu">
    <SectionTitle>
5 Results and Evaluation
</SectionTitle>
    <Paragraph position="0"> We have performed a standard evaluation of the precision and recall that our system achieves in selecting a description. Table 4 shows our results under two sets of parameters.</Paragraph>
    <Paragraph position="1"> Precision and recall are based on how well the system predicts a set of semantic constraints.</Paragraph>
    <Paragraph position="2"> Precision (or P) is defined to be the number of matches divided by the number of elements in the predicted set. Recall (or R) is the number of matches divided by the number of elements in the correct set. If, for example, the system predicts \[A\] \[B\] \[C\], but the set of constraints on the actual description is \[B\] \[D\], we would compute that P = 33.3% and R --- 50.0%. Table 4 reports the average values of P and R for all training examples 4.</Paragraph>
    <Paragraph position="3"> Selecting appropriate descriptions based on our algorithm is feasible even though the values of precision and recall obtained may seem only moderately high. The reason for this is that the problem that we axe trying to solve is underspecified. That is, in the same context, more than one description can be potentially used. Mutually interchangeable descriptions include synonyms and near synonyms (&amp;quot;leader&amp;quot; vs. &amp;quot;chief) or pairs of descriptions of different generality (U.S. president vs. president). This 4We run Ripper in a so-called &amp;quot;noise-free mode&amp;quot;, which causes the condition parts of the rules it discovers to be mutually exclusive and therefore, the values of P and R on the training data are both 100~.</Paragraph>
    <Paragraph position="4">  type of evaluation requires the availability of human judges.</Paragraph>
    <Paragraph position="5"> There are two parts to the evaluation: how well does the system performs in selecting semantic features (WordNet nodes) and how well it works in constraining the choice of a description. To select a description, our system does a lookup in the profile for a possible description that satisfies most semantic constraints (e.g., we select a row in Table 1 based on constraints on the columns).</Paragraph>
    <Paragraph position="6"> Our system depends crucially on the multiple components that we use. For example, the shallow CREP grammar that is used in extracting entities and descriptions often fails to extract good descriptions, mostly due to incorrect PP attachment. We have also had problems from the part-of-speech tagger and, as a result, we occasionally incorrectly extract word sequences that do not represent descriptions.</Paragraph>
  </Section>
  <Section position="7" start_page="1076" end_page="1077" type="evalu">
    <SectionTitle>
6 Applications and Future Work
</SectionTitle>
    <Paragraph position="0"> We should note that PROFILE is part of a large system for information retrieval and summarization of news through information extraction and symbolic text generation (McKeown and Radev, 1995). We intend to use PROFILE to improve lexical choice in the summary generation component, especially when producing user-centered summaries or summary updates (Radev and McKeown, 1998 to appear). There are two particularly appealing cases - (1) when the extraction component has failed to extract a description and (2) when the user model (user's interests, knowledge of the entity and personal preferences for sources of information and for either conciseness or verbosity) dictates that a description should be used even when one doesn't appear in the texts being summarized.</Paragraph>
    <Paragraph position="1"> A second potentially interesting application involves using the data and rules extracted by PROFILE for language regeneration. In (Radev and McKeown, 1998 to appear) we show how the conversion of extracted descriptions into components of a generation grammar allows for flexible (re)generation of new descriptions that don't appear in the source text. For example, a description can be replaced by a more general one, two descriptions can be combined to form a single one, or one long description can be deconstructed into its components, some of which can be reused as new descriptions.</Paragraph>
    <Paragraph position="2"> We are also interested in investigating another idea - that of predicting the use of a description of an entity even when the corresponding profile doesn't contain any description at all, or when it contains only descriptions that contain words that are not directly related to the words predicted by the rules of PROFILE. In this case, if the system predicts a semantic cat- null egory that doesn't match any of the descriptions in a specific profile, two things can be done: (1) if there is a single description in the profile, to pick that one, and (2) if there is more than one description, pick the one whose semantic vector is closest to the predicted semantic vector.</Paragraph>
    <Paragraph position="3"> Finally, the profile extractor will be used as part of a large-scale, automatically generated Who's who site which will be accessible both by users through a Web interface and by NLP systems through a client-server API.</Paragraph>
  </Section>
class="xml-element"></Paper>