<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-1028">
  <Title>Improved Automatic Keyword Extraction Given More Linguistic Knowledge</Title>
  <Section position="8" start_page="0" end_page="0" type="concl">
    <SectionTitle>
6 Concluding Remarks and Future Work
</SectionTitle>
    <Paragraph position="0"> In this paper I have shown how keyword extraction from abstracts can be achieved by using simple statistical measures as well as syntactic information from the documents, as input to a machine learning algorithm. If first considering the term selec-Method Assign. tot. Assign. mean Corr. tot. Corr. mean Prec. Recall F-score  document; the number of correct (Corr.) terms in total and mean per document; precision; recall; and Fscore. The highest value is shown in bold. The total number of manually assigned terms present in the abstracts is 3 816, and the mean is 7.63 terms per document.</Paragraph>
    <Paragraph position="1"> tion approaches, extracting NP-chunks gives a better precision, while extracting all words or sequences of words matching any of a set of POS tag patterns gives a higher recall compared to extracting ngrams. The highest F-score is obtained by one of the n-gram runs. The largest amount of assigned terms present in the abstracts are assigned by the pattern approach without the tag feature. The pattern approach is also the approach which keeps the largest number of assigned terms after that the data have been pre-processed. Using phrases means that the length of the potential terms is not restricted to something arbitrary, rather the terms are treat as the units they are. However, of the patterns that were selected for the experiments discussed here none was longer than four tokens. If looking at all assigned keywords in the training set, 3.0% are then ruled out as potential terms. The longest chunks in the test set that were correctly assigned are five tokens long. As for when syntactic information is included as a feature (in the form of the POS tag(s) assigned to the term), it is evident from the results presented in this paper that this information is crucial for assigning an acceptable number of terms per document, independent of what term selection strategy is chosen.</Paragraph>
    <Paragraph position="2"> One shortcoming of the work is that there is currently no relation between the different POS tag feature values. For example, a singular noun has no closer relationship to a plural noun than to an adjective. In the future, the patterns should somehow be categorised reflecting their semantics, perhaps in a hierarchical manner, or morphological information could be removed.</Paragraph>
    <Paragraph position="3"> In this paper I have not touched upon the more intricate aspects of evaluation, but simply treated the manually assigned keywords as the gold standard.</Paragraph>
    <Paragraph position="4"> This is the most severe way to evaluate a keyword extractor, as many terms might be just as good, although for one reason or another not chosen by the human indexer. Future work will examine alternative approaches to evaluation. One possibility for a more liberal evaluation could be to use human evaluators with real information needs, as done by Turney (2000). Another possibility would be to let several persons index each document, thus getting a larger set of acceptable terms to choose from. This would hopefully lead to a better precision, while recall probably would be affected negatively; the importance of recall would then need to be reconsidered. null Future work should also go in the direction of generating (as opposed to extracting) keywords, by for example exploring potential knowledge provided by a thesaurus.</Paragraph>
  </Section>
class="xml-element"></Paper>