<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-4003">
  <Title>SmartNotes: Implicit Labeling of Meeting Data through User Note-Taking and Browsing</Title>
  <Section position="2" start_page="261" end_page="261" type="metho">
    <SectionTitle>
2 Automatic Meeting Understanding
</SectionTitle>
    <Paragraph position="0"> Topic detection and segmentation: We are attempting to automatically detect the topics being discussed at meetings. This task consists of two subtasks: discovering the points in a meeting when the topic changes, and then associating a descriptive labeltothesegmentbetweentwotopicshifts. Ourcurrent strategy for topic shift detection (Banerjee and Rudnicky, 2006a) is to perform an edge detection using such features as speech activity (who spoke when and for how long), the words that each per-son spoke, etc. For labeling, we are currently simply associating the agenda item names recorded in the notes with the segments they are most relevant to, as decided by a tf.idf matching technique. Topic detection is particularly useful during meeting information retrieval; (Banerjee et al., 2005) showed that when users wish to retrieve information from past meetings, they are typically interested in a specific discussion topic, as opposed to an entire meeting.</Paragraph>
    <Paragraph position="1"> Action item detection: An obvious application of meeting understanding is the automatic discovery and recording of action items as they are discussed during a meeting. Arguably one of the most important outcomes of a meeting are the action items decided upon, and automatically recording them could be a huge benefit especially to those participants that are likely to not note them down and consequently forget about them later on.</Paragraph>
    <Paragraph position="2"> Meeting participant role detection: Each meeting participant plays a variety of roles in an institution. These roles can be based on their function in the institution (managers, assistants, professors, students, etc), or based on their expertise (speech recognition experts, facilities experts, etc). Our current strategy for role detection (Banerjee and Rudnicky, 2006b) is to train detectors on hand labeled data. Our next step is to perform discovery of new roles through clustering techniques. Detecting such roles has several benefits. First, it allows us to build prior expectations of a meeting between a group of participants. For example, if we know person A is a speech recognition expert and person B a speech synthesis expert, a reasonable expectation is that when they meet they are likely to talk about technologies related speech processing. Consequently, we can use this expectation to aid the action item detection and the topic detection in that meeting.</Paragraph>
  </Section>
  <Section position="3" start_page="261" end_page="263" type="metho">
    <SectionTitle>
3 SmartNotes: System Description
</SectionTitle>
    <Paragraph position="0"> We have implemented SmartNotes to help users take multi-media notes during meetings, and retrieve them later on. SmartNotes consists of two major components: The note taking application which meeting participants use to take notes during the meeting, and the note retrieval application which users use to retrieve notes at a later point.</Paragraph>
    <Section position="1" start_page="261" end_page="262" type="sub_section">
      <SectionTitle>
3.1 SmartNotes Note Taking Application
</SectionTitle>
      <Paragraph position="0"> The note taking application is a stand-alone system, that runs on each meeting participant's laptop, and allows him to take notes during the meeting. In addition to recording the text notes, it also records the participant's speech, and video, if a video camera is connected to the laptop. This system is an extension of the Carnegie Mellon Meeting Recorder (Banerjee et al., 2004).</Paragraph>
      <Paragraph position="1"> Figure 1 shows a screen-shot of this application.</Paragraph>
      <Paragraph position="2"> It is a server-client application, and each participant logs into a central server at the beginning of each meeting. Thus, the system knows the precise identity of each note taker as well as each speaker in the meeting. This allows us to avoid the onerous problem of automatically detecting who is speaking at any time during the meeting. Further, after logging on, each client automatically synchronizes itself with a central NTP time server. Thus the time stamps that each client associates with its recordings are all synchronized, to facilitate merging and play back of audio/video during browsing (described in the next sub-section).</Paragraph>
      <Paragraph position="3"> Once logged in, each participant's note taking  area is split into two sections: a shared note taking area, and a private note taking area. Notes written in the shared area are viewable by all meeting participants. This allows meeting participants to share the task of taking notes during a meeting: As long as one participant has recorded an important point during a meeting, the other participants do not need to, thus making the note taking task easier for the group as a whole. Private notes that a participant does not wish to share with all participants can be taken in the private note taking area.</Paragraph>
      <Paragraph position="4"> The interface has a mechanism to allow meeting participants to insert an agenda into the shared area. Once inserted, the shared area is split into as many boxes as there are agenda items. Participants can then take notes during the discussion of an agenda item in the corresponding agenda item box. This is useful to the participants because it organizes the notes as they are being taken, and, additionally, the notes can later be retrieved agenda item by agenda item. Thus, the user can access all notes he has taken in different meetings regarding &amp;quot;buying a printer&amp;quot;, without having to see the notes taken for the other agenda items in each such meeting.</Paragraph>
      <Paragraph position="5"> In addition to being useful to the user, this act of inserting an agenda and then taking notes within the relevant agenda item box results in generating (unbeknownst to the participant) labeled data for the topic detection component. Specifically, if we define each agenda item as being a separate &amp;quot;topic&amp;quot;, and make the assumption that notes are taken approximately concurrent with the discussion of the contents of the notes, then we can conclude that there is a shift in the topic of discussion at some point between the time stamp on the last note in an agenda item box, and the time stamp on the first note of the next agenda item box. This information can then be used to improve the performance of the topic shift detector. The accuracy of the topic shift data thus acquired depends on the length of time between the two time points. Since this length is easy to calculate automatically, this information can be factored into the topic detector trainer.</Paragraph>
      <Paragraph position="6"> The interface also allows participants to enter action items through a dedicated action item form.</Paragraph>
      <Paragraph position="7"> Again the advantage of such a form to the participants is that the action items (and thus the notes) are better organized: After the meeting, they can perform retrieval on specific fields of the action items. For example, they can ask to retrieve all the action items assigned to a particular participant, or that are due a particular day, etc.</Paragraph>
      <Paragraph position="8"> In addition to being beneficial to the participant, the action item form filling action results in generating labeled data for the action item detector.</Paragraph>
      <Paragraph position="9"> Specifically, if we make the assumption that an action item form filling action is preceded by a discussion of the action item, then the system can couple the contents of the form with all the speech within a window of time before the form filling action, and use this pair as a data point to retrain its action item detector.</Paragraph>
    </Section>
    <Section position="2" start_page="262" end_page="263" type="sub_section">
      <SectionTitle>
3.2 SmartNotes Note Retrieval Website
</SectionTitle>
      <Paragraph position="0"> As notes and audio/video are recorded on each individual participant's laptop, they also get transferred over the internet to a central meeting server. This transfer occurs in the background without any intervention from the user, utilizes only the left-over bandwidth beyond the user's current bandwidth usage, and is robust to system shut-downs, crashes, etc. This process is described in more detail in (Banerjee et al., 2004).</Paragraph>
      <Paragraph position="1"> Once the meeting is over and all the data has been transferredtothecentralserver, meetingparticipants can use the SmartNotes multi-media notes retrieval system to view the notes and access the recorded audio/video. This is a web-based application that uses the same login process as the stand-along note  taking system. Users can view a list of meetings they have recorded using the SmartNotes application in the past, and then for each meeting, they can view the shared notes taken at the meeting. Figure 2 shows a screen shot of such a notes browsing session. Additionally, participants can view their own private notes taken during the meeting.</Paragraph>
      <Paragraph position="2"> In addition to viewing the notes, they can also access all recorded audio/video, indexed by the notes. That is, they can access the audio/video recorded around the time that the note was entered. Further they can specify how many minutes before and after the note they wish to access. Since the server has the audio from each meeting participant's audio channel, the viewer of the notes can choose to listen to any one person's channel, or a combination of the audio channels. The merging of channels is done in real time andis achievable because theirtime stamps have been synchronized during recording.</Paragraph>
      <Paragraph position="3"> In the immediate future we plan to implement a simple key-word based search on the notes recorded in all the recorded meetings (or in one specific meeting). This search will return notes that match the search using a standard tf.idf approach. The user will also be provided the option of rating the quality of the search retrieval on a one bit satisfied/notsatisfied scale. If the user chooses to provide this rating, it can be used as a feedback to improve the search. Additionally, which parts of the meeting the user chooses to access the audio/video from can be used to form a model of the parts of the meetings most relevant to the user. This information can help the system tailor its retrieval to individual preferences. null</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="263" end_page="263" type="metho">
    <SectionTitle>
4 The Demonstration
</SectionTitle>
    <Paragraph position="0"> We shall demonstrate both the SmartNotes note taking client as well as the SmartNotes note-retrieval website. Specifically we will perform 2 minute long mock meetings between 2 or 3 demonstrators. We will show how notes can be taken, how agendas can be created and action items noted. We will then show how the notes and the audio/video from the 2 minute meeting can be accessed through the SmartNotes note retrieval website. We shall also show the  automaticallylabeleddatathatgetscreatedbothduring the mock meeting, as well as during the browsing session. Finally, if time permits, we shall show results on how much we can improve the meeting understanding components' capabilities through labeled meeting data automatically acquired through participants' use of SmartNotes at CMU and other institutions that are currently using the system.</Paragraph>
  </Section>
class="xml-element"></Paper>