<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1315">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Empirical Verification of Adjacency Pairs Using Dialogue Segmentation</Title>
  <Section position="5" start_page="2" end_page="2" type="metho">
    <SectionTitle>
DELIBERATE
</SectionTitle>
    <Paragraph position="0"> B: I don't have anything scheduled that morning and we are leaving at one</Paragraph>
  </Section>
  <Section position="6" start_page="2" end_page="104" type="metho">
    <SectionTitle>
INFORM
</SectionTitle>
    <Paragraph position="0"> The response (INFORM) finally comes, but the forgetful 'previous tag' feature is now looking for what comes after DELIBERATE.</Paragraph>
    <Paragraph position="1"> What is needed is a way to not only determine what is likely to happen next, but to retain that expectation over longer distances when unfulfilled, until that expectation is no longer needed. Such information would conform more closely to this description of a conversational game (but which could be applied to any communicative subgoal):  For a full description of the Verbmobil speech acts, see Alexandersson 1997.</Paragraph>
    <Paragraph position="2">  A conversational game is a sequence of moves starting with an initiation and encompassing all moves up until that initiation's purpose is either fulfilled or abandoned. (Carletta 1997, italics mine.)</Paragraph>
  </Section>
  <Section position="7" start_page="104" end_page="104" type="metho">
    <SectionTitle>
2 Dialogue segmentation
</SectionTitle>
    <Paragraph position="0"> This work grew out of related research into finding expectations in dialogue, but we were also interested in dialogue segmentation.</Paragraph>
    <Paragraph position="1"> Dialogues taken as a whole are very different from each other, so segmentation is necessary to derive meaningful information about their parts.</Paragraph>
    <Paragraph position="2"> The question is, then, how best to segment dialogues so as to reveal dialogue information or to facilitate some language task, such as DA classification? Various schemes for dialogue segmentation have been tried, including segmentation based on fulfilment of expectation (Ludwig et al.</Paragraph>
    <Paragraph position="3"> 1998), and segmenting by propositionality (Midgley 2003).</Paragraph>
    <Paragraph position="4"> One answer to the question of how to segment dialogue came from the pioneering work of Sacks and Schegloff (1973) article.</Paragraph>
    <Paragraph position="5"> A basic rule of adjacency pair operation is: given the recognizable production of a first pair part, on its first possible completion its speaker should stop and a next speaker should start and produce a second pair part from the same pair type of which the first is recognizably a member. (p. 296, italics mine.) Thus, if a speaker stops speaking, it is likely that such a handover has just taken place. The last utterance of a speaker's turn, then, will be the point at which the first speaker has issued a first pair part, and is now expecting a second pair part from the other speaker. This suggests a natural boundary.</Paragraph>
    <Paragraph position="6"> This approach was also suggested by Wright (1998), who used a &amp;quot;most recent utterance by previous speaker&amp;quot; feature in her work on DA tagging. This feature alone has boosted classification accuracy by about 2% in our preliminary research, faring better than the traditional 'previous tag' feature used in much DA tagging work.</Paragraph>
    <Paragraph position="7"> We collected a training corpus of 40 English-speaking dialogues from the Verbmobil-2 corpus, totalling 5,170 utterances. We then segmented the dialogues into chunks, where a chunk included everything from the last utterance of one speaker's turn to the last-butone utterance of the next speaker.</Paragraph>
  </Section>
  <Section position="8" start_page="104" end_page="105" type="metho">
    <SectionTitle>
3 Results of segmentation
</SectionTitle>
    <Paragraph position="0"> This segmentation revealed some interesting patterns. When ranked by frequency, the most common chunks bear a striking resemblance to the adjacency pairs posited by Schegloff and Sacks.</Paragraph>
    <Paragraph position="1"> Here are the 25 most common chunks in our training corpus, with the number of times they appeared. The full list can be found at http:/</Paragraph>
    <Paragraph position="3"/>
    <Paragraph position="5"> The data suggest a wide variety of language behaviour, including traditional adjacency pairs (e.g. SUGGEST: ACCEPT), acknowledgement (INFORM: BACKCHANNEL), formalised exchanges (POLITENESS_FORMULA: FEEDBACK_POSITIVE) offers and counteroffers (SUGGEST: SUGGEST), and it even hints at negotiation subdialogues (SUGGEST: REQUEST_CLARIFY).</Paragraph>
    <Paragraph position="6"> However, there are some drawbacks to this list. Some of the items are not good examples of adjacency pairs because the presence of the first does not create an expectation for the second half (e.g. NOT_CLASSIFIABLE: INFORM). In  some cases they appear backwards (ACCEPT: SUGGEST). Legitimate pairs appear further down the list than more-common bogus ones. For example, SUGGEST: REJECT is a well-known adjacency pair, but it does not appear on the list until after several less-worthy-seeming pairs. Keeping the less-intuitive chunks may help us with classification, but it falls short of providing empirical verification for pairs.</Paragraph>
    <Paragraph position="7"> What we need, then, is some kind of noise reduction that will strain out spurious pairs and bring legitimate pairs closer to the top of the list. We use the well-known kh  test tells how the observed frequency of an event compares with the expected frequency. For our purposes, it tells whether the observed frequency of an event (in this case, one kind of speech act following a certain other act) can be attributed to random chance. The test has been used for such tasks as feature selection (Spitters 2000) and translation pair identification (Church and Gale 1991).</Paragraph>
    <Paragraph position="8"> The kh  value for any two speech acts A and B can be calculated by counting the times that an utterance marked as tag A (or not) is followed by an utterance marked as tag B (or not), as in</Paragraph>
    <Paragraph position="10"> .</Paragraph>
    <Paragraph position="11"> These counts (as well as N, the total number of utterances) are plugged into a variant of the kh  equation used for 2x2 tables, as in Schutze et al. (1995).</Paragraph>
    <Paragraph position="13"> We trained the kh  method on the aforementioned chunks. Rather than restrict our focus to only adjacent utterances, we allowed a match for pair A:B if B occurred anywhere within the chunk started by A. By doing so, we hoped to reduce any acts that may have been interfering with the adjacency pairs, especially hesitation noises (usually classed as DELIBERATE) and abandoned utterances (NOT_CLASSIFIABLE).</Paragraph>
  </Section>
class="xml-element"></Paper>