<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0206">
  <Title>An Experiment to Evaluate the Effectiveness of Cross-Media Cues in</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Experiment
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 Overview
</SectionTitle>
      <Paragraph position="0"> As a first step, we must address a basic question: is it ever worthwhile to generate cross-media cues in computer presentations? Thus we designed a between-groups experiment (Lewis &amp; Rieman, 1994) to test whether performance on tasks requiring a subject to skim for information presented in text and graphics via a web browser would benefit from the inclusion of cross-media cues in the text. Skimming, defined as &amp;quot;moving rapidly through text to locate specific information or gain the gist&amp;quot;, is a type of reading strategy often used by readers of web pages (Dyson and Haselgrove 2001).</Paragraph>
      <Paragraph position="1"> Each of the three groups of subjects receives a different version of a presentation consisting of four articles. Each article fills a 19 inch computer screen and consists of a short text followed by several figures with information graphics such as line graphs and bar charts. The graphics are arranged in a row near the bottom of the screen so that the cost to the user of looking up and down between text and graphics is the same for each figure.</Paragraph>
      <Paragraph position="2"> Multiple figures are provided so that the reader is required to determine which figure is relevant to the task.</Paragraph>
      <Paragraph position="3"> In version 1, the layout of each article consists of text containing no cross-media cues followed by the figures. A short caption is given under each graphic. In version 2, the caption text has been removed from the figures and integrated into the paragraph of text above the figures, i.e., it now functions as commentary text. Version 3 is identical to version 2 except that for each figure a cross-media cue of the form 'See Figure n.' has been inserted in the text; the CMC is inserted following the commentary created from the corresponding caption in Version 1.</Paragraph>
      <Paragraph position="4"> Version 1 represents the case where it is feasible to design the layout so that text commenting upon a figure can be placed in proximity to the figure (i.e. maximizing adherence to the Spatial Contiguity Principle). We assume that task performance will be best for version 1 and include it in the experiment to provide a baseline. The main point of the experiment, however, is to compare performance on version 2 with performance on version 3. Then, if performance on version 3 is better, we have shown that CMCs can be useful to readers performing a similar task.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Experimental Design
</SectionTitle>
      <Paragraph position="0"> The independent variable is the version of the article that is presented. The three versions are constructed by varying layout and presence of cross media cue phrases as described above.</Paragraph>
      <Paragraph position="1"> The dependent variables are the time to complete the tests (Time) and score on the tests (Score). Time and Score are compared between groups.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.3 Participants
</SectionTitle>
      <Paragraph position="0"> The participants (subjects) are undergraduate college students. The participants are randomly assigned to one of three groups. Each group is tested on a different version of the same articles. Information about college major and experience using computers is collected via a short questionnaire before the experiment.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.4 Materials
</SectionTitle>
      <Paragraph position="0"> Each article was constructed by the experimenter by selecting an excerpt from a published source; the sources of the four articles represent different genre, topics, layouts, and audiences. (We chose to use excerpts rather than authoring our own articles to avoid experimenter bias.) The excerpts are approximately the same word-length and, except for the first article, which is used for practice and only includes two figures, each excerpt includes three figures. The layout was modified by the experimenter to create versions 1 through 3. Other differences in presentation (e.g., line length, color scheme, font style, and font size) between different versions of the same article and between articles were minimized as much as possible.</Paragraph>
      <Paragraph position="1"> The multiple choice test for each article consists of one question asking the subject to identify one of the main points of the presentation, and three questions asking the subject to identify where in the presentation certain facts were given. For the identification questions the subject is asked to select one or more of the following choices: in the text, in the graph in Figure 1, in the graph in Figure 2, in the graph in Figure 3, or none of the above.</Paragraph>
    </Section>
    <Section position="5" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.5 Procedure
</SectionTitle>
      <Paragraph position="0"> Each participant is given a series of four tests displayed on a desktop PC with a 19 inch color monitor. The first test is used as a practice test and data collected from it will not be used. The test series is implemented by a computer program written in HTML and Javascript that is run by a web browser. Scrolling is disabled throughout the test series. The first screen of each test presents an article; the next screen contains the four test questions described above. The participant is free to move back and forth between the article and the test question screen for it by using Forward/Backward buttons, but cannot see the article and test question screens at the same time. The participant cannot go back to previous tests, and is not allowed to go on to the next test until he or she has answered all questions on the current test and has confirmed that he or she is ready to go on to the next test. The participant answers the test questions using the computer mouse. The program records the participant's answers and times automatically. Subjects are not told that their task time is being measured.</Paragraph>
    </Section>
    <Section position="6" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.6 Status of Work
</SectionTitle>
      <Paragraph position="0"> We have finished running the pilot version of the experiment and are currently running the main experiment. It is interesting that in the post-experiment questionnaire, some subjects who have received version 2 have commented that references to the figures (i.e. CMCs) would have been helpful.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>