<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1052">
  <Title>Generating Referring Expressions in Open Domains</Title>
  <Section position="4" start_page="5" end_page="75" type="metho">
    <SectionTitle>
3 The Lexicalised Approach
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="5" end_page="5" type="sub_section">
      <SectionTitle>
3.1 Quantifying Discriminating Power
</SectionTitle>
      <Paragraph position="0"> We de ne the following three quotients.</Paragraph>
      <Paragraph position="1"> Similarity Quotient (SQ) We de ne similarity as transitive synonymy. The idea is that if X is a synonym of Y and Y is a synonym of Z, then X is likely to be similar to Z. The degree of similarity between two adjectives depends on how many steps must be made through WordNet synonymy lists to get from one to the other.</Paragraph>
      <Paragraph position="2"> Suppose we need to nd a referring expression for e0. For each adjective aj describing e0, we calculate a similarity quotient SQj by initialising it to 0, forming a set of WordNet synonyms S1 of aj, forming a synonymy set S2 containing all the Word-Net synonyms of all the adjectives in S1 and forming S3 from S2 similarly. Now for each adjective describing any distractor, we increment SQj by 4 if it is present in S1, by 2 if it is present in S2, and by 1 if it is present in S3. SQj now measures how similar aj is to other adjectives describing distractors.</Paragraph>
      <Paragraph position="3"> Contrastive Quotient (CQ) Similarly, we de ne contrastive in terms of antonymy relationships. We form the set C1 of strict WordNet antonyms of aj. The set C2 consists of strict WordNet antonyms of members of S1 and WordNet synonyms of members of C1. C3 is similarly constructed from S2 and C2. We now initialise CQj to zero and for each adjective describing each distractor, we add w =2 f4; 2; 1g to CQj, depending on whether it is a member of C1, C2 or C3.</Paragraph>
      <Paragraph position="4"> CQj now measures how contrasting aj is to other adjectives describing distractors.</Paragraph>
      <Paragraph position="5"> Discriminating Quotient (DQ) An attribute that has a high value of SQ has bad discriminating power. An attribute that has a high value of CQ has good discriminating power. We can now de ne the Discriminating Quotient (DQ) as DQ = CQ SQ. We now have an order (decreasing DQs) in which to incorporate attributes. This constitutes our *preferred* list. We illustrate the bene ts of our approach with two examples.</Paragraph>
      <Paragraph position="6"> Example 1: The Importance of Lexicalisation Previous referring expression generation algorithms ignore the issue of realising the logical description for the referent. The semantic labels are chosen such that they have a direct correspondence with their linguistic realisation and the realisation is thus considered trivial. Ambiguity and syntactically optional arguments are ignored. To illustrate one problem this causes, consider the two entities  If we followed the strict typing system used by previous algorithms, with *preferred*=fage, tenureg, to refer to e1 we would compare the age attributes and rule out e2 and generate the old president. This expression is ambiguous since old can also mean previous. Models that select attributes at the semantic level will run into trouble when their linguistic realisations are ambiguous.</Paragraph>
      <Paragraph position="7"> In contrast, our algorithm, given attened attribute lists: e1 e2 head president attrib old; current head president attrib young; past successfully picks the current president as current has a higher DQ (2) than old (0): attribute distractor CQ SQ DQ old e2fyoung, pastg 4 4 0 current e2fyoung, pastg 2 0 2 In this example, old is a WordNet antonym of young and a WordNet synonym of past. Current is a WordNet synonym of present, which is a WordNet antonym of past. Note that WordNet synonym and antonym links capture the implicit gradation in the lexicalised values of the age and tenure attributes.  To illustrate another problem with the original incremental algorithm, consider three dogs: e1(a big black dog), e2(a small black dog) and e3(a tiny white dog).</Paragraph>
      <Paragraph position="8"> Consider using the original incremental algorithm to refer to e1 with *preferred*=fcolour, sizeg. The colour attribute black rules out e3.</Paragraph>
      <Paragraph position="9"> We then we have to select the size attribute big as well to rule out e2, thus generating the sub-optimal expression the big black dog. Here, the use of a predetermined *preferred* list fails to capture what is obvious from the context: that e1 stands out not because it is black, but because it is big.</Paragraph>
      <Paragraph position="10"> In our approach, for each of e1's attributes, we calculate DQ with respect to e2 and e3: attribute distractor CQ SQ DQ big e2fsmall, blackg 4 0 4 big e3ftiny, whiteg 2 0 2 black e2fsmall, blackg 1 4 -3 black e3ftiny, whiteg 2 1 1 Overall, big has a higher discriminating power (6) than black (-2) and rules out both e2 and e3. We therefore generate the big dog. Our incremental approach thus manages to select the attribute that stands out in context. This is because we construct the *preferred* list after observing the context. We discuss this issue further in the next section. Note again that WordNet antonym and synonym links capture the gradation in the lexicalised size and colour attributes. However, this only works where the gradation is along one axis; in particular, this approach will not work for colours in general, and cannot be used to deduce the relative similarity between yellow and orange as compared to, say, yellow and blue.</Paragraph>
    </Section>
    <Section position="2" start_page="5" end_page="5" type="sub_section">
      <SectionTitle>
3.2 Justifying our Algorithm
</SectionTitle>
      <Paragraph position="0"> The psycholinguistic justi cation for the incremental algorithm (IA) hinges on two premises:  1. Humans build referring expressions incrementally. 2. There is a preferred order in which humans select attributes (e.g., colour&gt;shape&gt;size:::).</Paragraph>
      <Paragraph position="1">  Our algorithm is also incremental. However, it departs signi cantly from premise 2. We assume that speakers pick out attributes that are distinctive in context (cf., example 2, previous section). Averaged over contexts, some attributes have more discriminating power than others (largely because of the way we visualise entities) and premise 2 is an approximation to our approach.</Paragraph>
      <Paragraph position="2"> We now quantify the extra effort we are making to identify attributes that stand out in a given context. Let N be the maximum number of entities in the contrast set and n be the maximum number of attributes per entity. The table below compares the computational complexity of an optimal algorithm (such as Reiter (1990)), our algorithm and the IA. Incremental Algo Our Algorithm Optimal Algo</Paragraph>
      <Paragraph position="4"> Both the IA and our algorithm are linear in the number of entities N. This is because neither algorithm allows backtracking; an attribute, once selected, cannot be discarded. In contrast, an optimal search requires O(2N) comparisons. As our algorithm compares each attribute of the discourse referent to every attribute of every distractor, it is quadratic in n. The IA compares each attribute of the discourse referent to only one attribute per distractor and is linear in n. Note, however, that values for n of over 4 are rare.</Paragraph>
    </Section>
    <Section position="3" start_page="5" end_page="5" type="sub_section">
      <SectionTitle>
3.3 Relations
</SectionTitle>
      <Paragraph position="0"> Semantically, attributes describe an entity (e.g., the small grey dog) and relations relate an entity to other entities (e.g., the dog in the bin). Relations are troublesome because in relating an entity eo to e1, we need to recursively generate a referring expression for e1. The IA does not consider relations and the referring expression is constructed out of attributes alone. The Dale and Haddock (1991) algorithm allows for relational descriptions but involves exponential global search, or a greedy search approximation. To incorporate relational descriptions in the incremental framework would require a classi cation system which somehow takes into account the relations themselves and the secondary entities e1 etc. This again suggests that the existing algorithms force the incrementality at the wrong stage in the generation process. Our approach computes the order in which attributes are incorporated after observing the context, by quantifying their utility through the quotient DQ. This makes it easy for us to extend our algorithm to handle relations, because we can compute DQ for relations in much the same way as we did for attributes.We illustrate this for prepositions.</Paragraph>
    </Section>
    <Section position="4" start_page="5" end_page="5" type="sub_section">
      <SectionTitle>
3.4 Calculating DQ for Relations
</SectionTitle>
      <Paragraph position="0"> Suppose the referent entity eref contains a relation [prepo eo] that we need to calculate the three quotients for (cf., gure 1 for representation of relations in AVMs). We consider each entity ei in the contrast set for eref in turn. If ei does not have a prepo relation then the relation is useful and we increment CQ by 4. If ei has a prepo relation then two cases arise. If the object of ei's prepo relation is eo then we increment SQ by 4. If it is not eo, the relation is useful and we increment CQ by 4. This is an ef cient non-recursive way of computing the quotients CQ and SQ for relations. We now discuss how to calculate DQ. For attributes, we de ned DQ = CQ SQ. However, as the linguistic realisation of a relation is a phrase and not a word, we would like to normalise the discriminating power of a relation with the length of its linguistic realisation. Calculating the length involves recursively generating referring expressions for the object of the preposition, an expensive task that we want to avoid unless we are actually using that relation in the nal referring expression. We therefore initially approximate the length as follows. The realisation of a relation [prepo eo] consists of prepo, a determiner and the referring expression for eo. If none of eref's distractors have a prepo relation then we only require the head noun of eo in the referring expression and length = 3. In this case, the relation is suf cient to identify both entities; for example, even if there were multiple bins in gure 1, as long as only one dog is in a bin, the reference the dog in the bin succeeds in uniquely referencing both the dog and the bin. If n distractors of eref contain a prepo relation with a non-eo object that is distractor for eo, we set length = 3 + n. This is an estimate for the word length of the realised relation that assumes one extra attribute for distinguishing eo from each distractor. Normalisation by estimated length is vital; if eo requires a long description, the relations's DQ should be small so that shorter possibilities are considered rst in the incremental process. The formula for DQ for relations is therefore</Paragraph>
      <Paragraph position="2"> This approach can also be extended to allow for relations such as comparatives which have syntactically optional arguments (e.g., the earlier ight vs the ight earlier than UA941) which are not allowed for by approaches which ignore realisation.</Paragraph>
    </Section>
    <Section position="5" start_page="5" end_page="75" type="sub_section">
      <SectionTitle>
3.5 The Lexicalised Context-Sensitive IA
</SectionTitle>
      <Paragraph position="0"> Our lexicalised context-sensitive incremental algorithm (below) generates a referring expression for Entity. As it recurses, it keeps track of entities it has used up in order to avoid entering loops like the dog in the bin containing the dog in the bin.... To generate a referring expression for an entity, the algorithm calculates the DQs for all its attributes and approximates the DQs for all its relations (2). It then forms the *preferred* list (3) and constructs the referring expression by adding elements of *preferred* till the contrast set is empty (4). This is straightforward for attributes (5). For relations (6), it needs to recursively generate the prepositional phrase rst.</Paragraph>
      <Paragraph position="1"> It checks that it hasn't entered a loop (6a), generates a new contrast set for the object of the relation (6(a)i), recursively generates a referring expression for the object of the preposition (6(a)ii), recalculates DQ (6(a)iii) and either incorporates the relation in the referring expression or shifts the relation down the *preferred* list (6(a)iv). This step ensures that an initial mis-estimation in the word length of a relation doesn't force its inclusion at the expense of shorter possibilities. If after incorporating all attributes and relations, the contrast set is still nonempty, the algorithm returns the best expression it can nd (7).</Paragraph>
      <Paragraph position="2">  set generate-ref-exp(Entity, ContrastSet, UsedEntities) 1. IF ContrastSet = [] THENRETURNfEntity.headg 2. Calculate CQ, SQ and DQ for each attribute and relation of Entity (as in Sec 3.1 and 3.4) 3. Let *preferred* be the list of attributes/ relations sorted in decreasing order of DQs. FOR each element (Mod) of *preferred* DO steps 4, 5 and 6 4. IF ContrastSet = [] THEN RETURN RefExp [ fEntity.headg 5. IF Mod is an Attribute THEN (a) LET RefExp = fModg[ RefExp (b) Remove from ContrastSet, any entities Mod rules out 6. IF Mod is a Relation [prepi ei] THEN (a) IF ei 2 UsedEntitiesTHEN i. Set DQ = 1 ii. Move Mod to the end of *preferred* ELSE i. LET ContrastSet2 be the set of non-ei entities that are the objects of prepi relations in members of ContrastSet ii. LET RE = generate-referring-exp(ei,  We now trace the algorithm above as it generates a referring expression for d1 in gure 1.</Paragraph>
      <Paragraph position="4"/>
      <Paragraph position="6"> return f[in the fbing], dogg The algorithm presented above is designed to return the shortest referring expression that uniquely identi es an entity. If the scene in gure 1 were cluttered with bins, the algorithm would still refer to d1 as the dog in the bin as there is only one dog that is in a bin. The user gets no help in locating the bin.</Paragraph>
      <Paragraph position="7"> If helping the user locate entities is important to the discourse plan, we need to change step 6(a)(ELSE)i so that the contrast set includes all bins in context, not just bins that are objects of in relations of distractors of d1.</Paragraph>
    </Section>
    <Section position="6" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
3.6 Compound Nominals
</SectionTitle>
      <Paragraph position="0"> Our analysis so far has assumed that attributes are adjectives. However, many nominals introduced through relations can also be introduced in compound nominals, for example:  1. a church in Paris $ a Paris church 2. a novel by Archer $ an Archer novel 3. a company from London $ a London company  This is an important issue for regeneration applications, where the AVMs for entities are constructed from text rather than a semantic knowledge base (which could be constructed such that such cases are stored in relational form, though possibly with an underspeci ed relation). We need to augment our algorithm so that it can compare AVMs like: head church in head Paris and head church attrib [Paris] Formally, the algorithm for calculating SQ and CQ for a nominal attribute anom of entity eo is: FOR each distractor ei of eo DO  1. IF anom is similar to any nominal attribute of ei THEN SQ = SQ + 4 2. IF anom is similar to the head noun of the object of any relation of ei THEN (a) SQ = SQ + 4 (b) atten that relation for ei, i.e., add the at- null tributes of the object of the relation to the attribute list for ei In step 2, we compare a nominal attribute anom of eo to the head noun of the object of a relation of ei. If they are similar, it is likely that any attributes of that object might help distinguish eo from ei. We then add those attributes to the attribute list of ei. Now, if SQ is non-zero, the nominal attribute anom has bad discriminating power and we set DQ = SQ. If SQ = 0, then anom has good discriminating power and we set DQ = 4. We also extend the algorithm for calculating DQ for a relation [prepj ej] of eo as follows:  1. IF any distractor ei has a nominal attribute anom THEN (a) IF anom is similar to the head of ej THEN i. Add all attributes of eo to the attribute list and calculate their DQs 2. calculate DQ for the relation as in section 3.4  We can demonstrate how this approach works using entities extracted from the following sentence (from the Wall Street Journal): Also contributing to the rmness in copper, the analyst noted, was a report by Chicago purchasing agents, which precedes the full purchasing agents report that is due out today and gives an indication of what the full report might hold.</Paragraph>
      <Paragraph position="1"> Consider generating a referring expression for eo when the distractor is e1:  The distractor the full purchasing agents report contains the nominal attribute agents. To compare report by Chicago purchasing agents with full purchasing agents report, our algorithm attens the former to Chicago purchasing agents report. Our algorithm now gives: DQagents = 4, DQpurchasing = 4, DQChicago = 4, DQby Chicago purchasing agents = 4=4 We thus generate the referring expression the Chicago report. This approach takes advantage of the exibility of the relationships that can hold between nouns in a compound: although examples can be devised where removing a nominal causes ungrammaticality, it works well enough empirically. To generate a referring expression for e1 (full purchasing agents report) when the distractor is eo(report by Chicago purchasing agents), our algorithm again attens eo to obtain: DQagents = 4, DQpurchasing = 4 DQfull = 4 The generated referring expression is the full report. This is identical to the referring expression used in the original text.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="75" end_page="75" type="metho">
    <SectionTitle>
4 Evaluation
</SectionTitle>
    <Paragraph position="0"> As our algorithm works in open domains, we were able to perform a corpus-based evaluation using the Penn WSJ Treebank (Marcus et al., 1993). Our evaluation aimed to reproduce existing referring expressions (NPs with a de nite determiner) in the Penn  For each referring expression (NP with a de nite determiner) in the Penn Treebank, we automatically identi ed its rst mention and all its distractors in a four sentence window, as described in x4.1. We then used our program to generate a referring expression for the rst mention NP, giving it a contrastset containing the distractor NPs. Our evaluation compared this generated description with the original WSJ reference that we had started out with.</Paragraph>
    <Paragraph position="1"> Our algorithm was developed using toy examples and counter-examples constructed by hand, and the Penn Treebank was unseen data for this evaluation.</Paragraph>
    <Section position="1" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
4.1 Identifying Antecedents and Distractors
</SectionTitle>
      <Paragraph position="0"> For every de nite noun phrase NPo in the Penn Treebank, we shortlisted all the noun phrases NPi in a discourse window of four sentences (the two preceding sentences, current sentence and the following sentence) that had a head noun identical to or a WordNet synonym of the head noun of NPo.</Paragraph>
      <Paragraph position="1"> We compared the set of attributes and relations for each shortlisted NPi that preceded NPo in the discourse window with that of NPo. If the attributes and relations set of NPi was a superset of that of NPo, we assumed that NPo referred to NPi and added NPi to an antecedent set. We added all other NPi to the contrast set of distractors.</Paragraph>
      <Paragraph position="2"> Similarly, we excluded any noun phrase NPi that appeared in the discourse after NPo whose attributes and relations set was a subset of NPo's and added the remaining NPi to the contrast set. We then selected the longest noun phrase in the antecedent set to be the antecedent that we would try and generate a referring expression from.</Paragraph>
      <Paragraph position="3"> The table below gives some examples of distractors that our program found using WordNet synonyms to compare head nouns:</Paragraph>
    </Section>
    <Section position="2" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
Entity Distractors
</SectionTitle>
      <Paragraph position="0"> rst half-free Soviet vote fair elections in the GDR military construction bill scal measure steep fall in currency drop in market stock permanent insurance death bene t coverage</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="75" end_page="75" type="metho">
    <SectionTitle>
4.2 Results
</SectionTitle>
    <Paragraph position="0"> There were 146 instances of de nite descriptions in the WSJ where the following conditions (that ensure that the referring expression generation task is nontrivial) were satis ed:  1. The de nite NP (referring expression) contained at least one attribute or relation.</Paragraph>
    <Paragraph position="1"> 2. An antecedent was found for the de nite NP. 3. There was at least one distractor NP in the dis- null course window.</Paragraph>
    <Paragraph position="2"> In 81.5% of these cases, our program returned a referring expression that was identical to the one used in the WSJ. This is a surprisingly high accuracy, considering that there is a fair amount of variability in the way human writers use referring expressions. For comparison, the baseline of reproducing the antecedent NP performed at 48%2. Some errors were due to non-recognition of multiword expessions in the antecedent (for example, our program generated care product from personal care product). In many of the remaining error cases, it was dif cult to decide whether what our program generated was acceptable or wrong. For example, the WSJ contained the referring expression the one-day limit, where the automatically detected antecedent was the maximum one-day limit for the 2We are only evaluating content selection (the nouns and pre- and post-modi ers) and ignore determiner choice. S&amp;P 500 stock-index futures contract and the automatically detected contrast set was: fthe ve-point opening limit for the contract, the 12-point limit, the 30-point limit, the intermediate limit of 20 pointsg Our program generated the maximum limit, where the WSJ writer preferred the one-day limit.</Paragraph>
  </Section>
  <Section position="7" start_page="75" end_page="75" type="metho">
    <SectionTitle>
5 Further Issues
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
5.1 Reference Modifying Attributes
</SectionTitle>
      <Paragraph position="0"> The analysis thus far has assumed that all attributes modify the referent rather than the reference to the referent. However, for example, if e1 is an alleged murderer, the attribute alleged modi es the reference murderer rather than the referent e1 and referring to e1 as the murderer would be factually incorrect. Logically e1 could be represented as (alleged1(murderer1))(x), rather than alleged1(x) ^ murderer1(x). This is no longer rst-order, and presents new dif culties for the traditional formalisation of the reference generation problem. One (inelegant) solution would be to introduce a new predicate allegedMurderer1(x).</Paragraph>
      <Paragraph position="1"> A working approach in our framework would be to add a large positive weight to the DQs of reference modifying attributes, thus forcing them to be selected in the referring expression.</Paragraph>
    </Section>
    <Section position="2" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
5.2 Discourse Context and Salience
</SectionTitle>
      <Paragraph position="0"> The incremental algorithm assumes the availability of a contrast set and does not provide an algorithm for constructing and updating it. The contrast set, in general, needs to take context into account. Krahmer and Theune (2002) propose an extension to the IA which treats the context set as a combination of a discourse domain and a salience function. The black dog would then refer to the most salient entity in the discourse domain that is both black and a dog.</Paragraph>
      <Paragraph position="1"> Incorporating salience into our algorithm is straightforward. As described earlier, we compute the quotients SQ and CQ for each attribute or relation by adding an amount w 2 f4; 2; 1g to the relevant quotient based on a comparison with the attributes and relations of each distractor. We can incorporate salience by weighting w with the salience of the distractor whose attribute or relation we are considering. This will result in attributes and relations with high discriminating power with regard to more salient distractors getting selected rst in the incremental process.</Paragraph>
    </Section>
    <Section position="3" start_page="75" end_page="75" type="sub_section">
      <SectionTitle>
5.3 Discourse Plans
</SectionTitle>
      <Paragraph position="0"> In many situations, attributes and relations serve different discourse functions. For example, attributes might be used to help the hearer identify an entity while relations might serve to help locate the entity. This needs to be taken into account when generating a referring expression. If we were generating instructions for using a machine, we might want to include both attributes and relations; so to instruct the user to switch on the power, we might say switch on the red button on the top-left corner.</Paragraph>
      <Paragraph position="1"> This would help the user locate the switch (on the top-left corner) and identify it (red). If we were helping a chef nd the salt in a kitchen, we might want to use only relations because the chef knows what salt looks like. The salt behind the corn akes on the shelf above the fridge is in this context preferable to the white powder. If the discourse plan that controls generation requires our algorithm to preferentially select relations or attributes, it can add a positive amount to their DQs. Then, the resultant formula is DQ = (CQ SQ)=length + , where length = 1 for attributes and by default = 0 for both relations and attributes.</Paragraph>
    </Section>
  </Section>
  <Section position="8" start_page="75" end_page="75" type="metho">
    <SectionTitle>
6 Conclusions and Future Work
</SectionTitle>
    <Paragraph position="0"> We have described an algorithm for generating referring expressions that can be used in any domain.</Paragraph>
    <Paragraph position="1"> Our algorithm selects attributes and relations that are distinctive in context. It does not rely on the availability of an adjective classi cation scheme and uses WordNet antonym and synonym lists instead.</Paragraph>
    <Paragraph position="2"> It is also, as far as we know, the rst algorithm that allows for the incremental incorporation of relations and the rst that handles nominals. In a novel evaluation, our algorithm successfully generates identical referring expressions to those in the Penn WSJ Treebank in over 80% of cases.</Paragraph>
    <Paragraph position="3"> In future work, we plan to use this algorithm as part of a system for generation from a database of user opinions on products which has been automatically extracted from newsgroups and similar text. This is midway between regeneration and the classical task of generating from a knowledge base because, while the database itself provides structure, many of the eld values are strings corresponding to phrases used in the original text. Thus, our lexicalised approach is directly applicable to this task.</Paragraph>
  </Section>
class="xml-element"></Paper>