<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1310">
  <Title>69 On a possible role for pronouns in the acquisition of verbs</Title>
  <Section position="4" start_page="69" end_page="72" type="metho">
    <SectionTitle>
2 Experiment 1
</SectionTitle>
    <Paragraph position="0"> The first experiment consisted of a corpus analysis to identify patterns of co-occurrence between pronouns and verbs in the childs input.</Paragraph>
    <Section position="1" start_page="69" end_page="70" type="sub_section">
      <SectionTitle>
2.1 Method
</SectionTitle>
      <Paragraph position="0"> Parental utterances from the CHILDES database (MacWhinney, 2000) were coded for syntactic categories, then subjected to cluster analysis. The mean age of target children represented in the transcripts that were coded for this experiment was 3;0 (SD1;2).</Paragraph>
      <Paragraph position="1">  The following corpora were used: Bates, Bliss,  Kleeck and Warren-Leubecker. Coding was performed using a custom web application that randomly selected transcripts, assigned them to coders as they became available, collected coding  input, and stored it in a MySQL database. The application occasionally assigned the same transcript to all coders, in order to measure reliability. Five undergraduate coders were trained on the coding task and the use of the system.  Each coder was presented, in sequence, with each main tier line of each transcript she was assigned, together with several lines of context; the entire transcript was also available by clicking a link on the coding page. For each line, she indicated (a) whether the speaker was a parent, target child, or other; (b) whether the addressee was a parent, target child, or other; (c) the syntactic frames of up to 3 clauses in the utterance; (d) for each clause, up to 3 subjects, auxiliaries, verbs, direct objects, indirect objects and obliques. Because many utterances were multi-clausal, the unit of analysis for assessing pronoun-verb co-occurrences was the clause rather than the utterance.</Paragraph>
      <Paragraph position="2"> The syntactic frames were: no verb, question, passive, copula, intransitive, transitive and ditransitive. These were considered to be mutually exclusive, i.e., each clause was tagged as belonging to one and only one frame, according to which of the following frames it matched first: (1) The no verb frame included clauses such as Yes or OK with no main verb. (2) The question frame included any clause using a question word such as Where did you go? or having inverted word order such as Did you go to the bank? but not merely a question mark such as You went to the bank? (3) The passive frame included clauses in the passive voice, such as John was hit by the ball. (4) The copula frame included clauses with the copula as the main verb, such as John is angry. (5) The intransitive frame included clauses with no direct object, such as John ran. The transitive frame included clauses with a direct object but no indirect object, such as John hit the ball. (6) The ditransitive frame included clauses with an indirect object, such as John gave Mary a kiss. All nouns were coded in their singular forms, whether they were singular or plural (e.g., boys was coded as boy), and all verbs were coded in their infinitive forms, whatever tense they were in (e.g., ran was coded as run).</Paragraph>
      <Paragraph position="3"> In total, 59,977 utterances were coded from 123 transcripts. All of the coders coded 7 of those transcripts for the purpose of measuring reliability. Average inter-coder reliability (measured for each coder as the percentage of items coded exactly the same way they were coded by each other coder) was 86.1%. Given the number of variables, the number of levels of each variable (3 speakers, 3 addressees, 7 frames, and 6 syntactic relations), and the number of coders (5), the probability of chance agreement is very low. Although there are some substantive errors (usually with complex embedded clauses or other unusual constructions), many of the discrepancies are simple spelling mistakes or failures to trim words to their roots.</Paragraph>
      <Paragraph position="4"> We only considered parental child-directed speech (PCDS), defined as utterances where the speaker was a parent and the addressee was a target child. A total of 24,286 PCDS utterances were coded, including a total of 28,733 clauses. More than a quarter (28.36%) of the PCDS clauses contained no verb at all; these were excluded from further analysis. Clauses that were questions (16.86%), passives (0.02%), and copulas (11.86%) were also excluded from further analysis. The analysis was conducted using only clauses that were intransitives (17.24% of total PCDS clauses), transitives (24.36%) or ditransitives (1.48%), a total of 12,377 clauses.</Paragraph>
    </Section>
    <Section position="2" start_page="70" end_page="72" type="sub_section">
      <SectionTitle>
2.2 Results
</SectionTitle>
      <Paragraph position="0"> The most frequent nouns in the corpusboth subjects and objectsare pronouns, as shown in Figures 1 and 2. The objects divided the most common verbs into three main classes: verbs that take the pronoun it and concrete nouns as objects, verbs that take complement clauses, and verbs that take specific concrete nouns as objects. The subjects divided the most common verbs into four main classes: verbs whose subject is almost always I, verbs whose subject is almost always you, verbs that take I or you almost equally as subject, and other verbs. The verbs divided the most common object nouns into a number of classes, including objects of telling and looking verbs, objects of having and wanting verbs, and objects of putting and getting verbs. The verbs also divided the most common subject nouns into a number of classes, including subjects of having and wanting verbs, and subjects of thinking and knowing verbs.</Paragraph>
      <Paragraph position="2"/>
      <Paragraph position="4"> by their number of occurrences.</Paragraph>
      <Paragraph position="5"> 2.2.1 Verbs that take it as an object The verbs that take it as their most common object include verbs of motion and transfer, as shown in Table 1.</Paragraph>
      <Paragraph position="6">  Most verbs that did not take it as their most common object instead took complement clauses. These are primarily psychological verbs, as shown in Table 2.</Paragraph>
      <Paragraph position="7">  Most remaining verbs in the corpus took unique sets of objects. For example, the most common object used with read was book, followed by it and story; the most common object used with play was game, followed by it, block, and house.  Verbs whose most common subject is I include bet (23 out of 23 uses with a subject, or 100%), guess (21/22, 95.4%), think (212/263, 80.6%), and see (95/207, 45.9%). Parents were not discussing their gambling habits with their children bet was being used to indicate the epistemic status of a subsequent clause, as were the other verbs.  Verbs whose most common subject is you include like (86 out of its 134 total uses with a subject, or 64.2%), want (192/270, 71.1%), and need (33/65, 50.8%). These verbs are being used to indicate the deontic status of a subsequent clause, including disposition or inclination, volition, and compulsion.</Paragraph>
      <Paragraph position="8">  Verbs that take I and you more or less equally as subject include mean (15 out of 32 uses, or 46.9%, with I and 12 of 32 uses, or 37.5%, with you), know (I: 159/360, 44.2%; you: 189/360, 52.5%), and remember (I: 9/23, 39.1%; you: 12/23, 52.2%).</Paragraph>
      <Paragraph position="9">  The objects me, us, Daddy and Mommy formed a cluster in verb space, appearing frequently with the verbs tell and look at.</Paragraph>
      <Paragraph position="10">  The objects one, stuff, box, and toy occurred most frequently with get, and frequently with put. The objects them, h i m, h e r , bed, and mouth occurred most frequently with put and, in some cases, also frequently with get.</Paragraph>
      <Paragraph position="11">  The objects cookie, some, money, coffee, milk, and juice formed a cluster in verb space, appearing frequently with verbs such as have and want, as well as, in some cases, give, take, pour, drink, and eat.</Paragraph>
      <Paragraph position="12"> 2.2.10 Subjects of think and know The subject I appeared most frequently with the verbs think and know.</Paragraph>
    </Section>
    <Section position="3" start_page="72" end_page="72" type="sub_section">
      <SectionTitle>
2.3 Discussion
</SectionTitle>
      <Paragraph position="0"> Although pronouns are semantically light, their particular referents determinable only from context, they may nonetheless be potent forces on early lexical learning by statistically pointing to some classes of verbs as being more likely than others. The results of Experiment 1 clearly show that there are statistical regularities in the co-occurrences of pronouns and verbs that the child could use to discriminate classes of verbs.</Paragraph>
      <Paragraph position="1"> Specifically, when followed by it, the verb is likely to describe physical motion, transfer, or possession. When followed a relatively complex complement clause, by contrast, the verb is likely to attribute a psychological state. Finer distinctions may also be made with other objects, including proper names and nouns. Verbs followed by me, us, Daddy, and Mommy are likely to have to do with telling or looking. Verbs followed by one, stuff, them, him, or her are likely to have to do with getting or putting. Verbs followed by certain concrete objects such as cookie, milk, or juice are likely to have to do with having or wanting. Fine distinctions may also be made according to subject. If the subject is I, the verb is likely to have to do with thinking or knowing, whereas if the subject is you, she, we, he, or they, the verb is likely to have to do with having or wanting. This regularity most likely reflects the ecology of parents and childrenparents know and children want but it could nonetheless be useful in distinguishing these two classes of verbs.</Paragraph>
      <Paragraph position="2"> The results thus far show that there are potentially usable regularities in the statistical relations between pronouns and verbs. However, they do not show that these regularities can be used to cue the associated words.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="72" end_page="73" type="metho">
    <SectionTitle>
3 Experiment 2
</SectionTitle>
    <Paragraph position="0"> To demonstrate that the regularities in pronoun-verb co-occurrences in parental speech to children can actually be exploited by a statistical learner, we trained an autoassociator on the corpus data, then tested it on incomplete utterances to see how well it would fill in the blanks when given only a pronoun, or only a verb. An autoassociator is a connectionist network that is trained to take each input pattern and reproduce it at the output. In the process, it compresses the pattern through a small set of hidden units in the middle, forcing the network to find the statistical regularities among the elements in the input data. The network is trained by backpropagation, which iteratively reduces the discrepancies between the networks actual outputs and the target outputs (the same as the inputs for an autoassociator).</Paragraph>
    <Paragraph position="1"> In our case, the inputs (and thus the outputs) are subject-verb-object sentences. Once the network has learned the regularities inherent in a corpus of complete SVO sentences, testing it on incomplete sentences (e.g., I ___ him) allows us to see what it has gleaned about the relationship between the given parts (subject I and object him in our example) and the missing parts (the verb in our example).</Paragraph>
    <Section position="1" start_page="72" end_page="73" type="sub_section">
      <SectionTitle>
3.1 Method
3.1.1 Data
</SectionTitle>
      <Paragraph position="0"> The network training data consisted of the subject, verb, and object of all coded utterances that contained the 50 most common subjects, verbs and objects. There were 5,835 such utterances. The inputs used a localist coding wherein there was one and only one input unit out of 50 activated for each subject, and likewise for each verb and each object. Absent and omitted arguments were counted among the 50, so, for example, the utterance John runs would have 3 units activated even though it only has 2 wordsthe third unit being the no object unit.</Paragraph>
      <Paragraph position="1"> With 50 units each for subject, verb and object, there were a total of 150 input units to the network. Active input units had a value of 1, and inactive input units had a value of 0.</Paragraph>
      <Paragraph position="2">  The network consisted of a two-layer 150-8-150 unit autoassociator with a logistic activation function at the hidden layer and a three separate softmax activation functions (one each for the subject, verb and object) at the output layersee  which ensures that all the outputs in the bank sum to 1, together with the cross-entropy error measure, allows us to interpret the network outputs as probabilities (Bishop, 1995). The network was trained by the resilient backpropagation algorithm (Riedmiller and Braun, 1993) to map its inputs back onto its outputs. We chose to use eight units in the hidden layer on the basis of some pilot experiments that varied the number of hidden units. Networks with fewer hidden units either did not learn the problem sufficiently well or took a long time to converge, whereas networks with more than about 8 hidden units learned quickly but tended to overfit the data.</Paragraph>
    </Section>
    <Section position="2" start_page="73" end_page="73" type="sub_section">
      <SectionTitle>
3.1.3 Training
</SectionTitle>
      <Paragraph position="0"> The data was randomly assigned to two groups: 90% of the data was used for training the network, while 10% was reserved for validating the networks performance. Starting from different random initial weights, five networks were trained until the cross-entropy on the validation set reached a minimum for each of them. Training stopped after approximately 150 epochs of training, on average. At that point, the networks were achieving about 81% accuracy on correctly identifying subjects, verbs and objects from the training set. Near perfect accuracy on the training set could have been achieved by further training, with some loss of generalization, but we wanted to avoid overfitting.</Paragraph>
    </Section>
    <Section position="3" start_page="73" end_page="73" type="sub_section">
      <SectionTitle>
3.1.4 Testing
</SectionTitle>
      <Paragraph position="0"> After training, the networks were tested with incomplete inputs corresponding to isolated verbs and pronouns. For example, to see what a network had learned about it as a subject, it was tested with a single input unit activatedthe one corresponding to it as subject. The other input units were set to 0. Activations at the output units were recorded. The results presented below report average activations over all five networks.</Paragraph>
    </Section>
    <Section position="4" start_page="73" end_page="73" type="sub_section">
      <SectionTitle>
3.2 Results
</SectionTitle>
      <Paragraph position="0"> The networks learn many of the co-occurrence regularities observed in the data. For example, when tested on the object it (see Figure 4 on page 7 below), the most activated verbs are get, hold, take and have, which are among the most common verbs associated with it in the input (see Table 1). Similarly, tell, make and say are the most activated verbs when networks are tested with the clause unit activated in the object position (figure not shown), and they are also among the verbs most commonly associated with a clause in the input (see Table 2).</Paragraph>
      <Paragraph position="1"> However, the network does not merely learn the relative frequencies of pronouns with verbs. For example, the verbs most activated by the subject you are have and get (see Figure 5 on page 8 below), neither of which appears in Table 3. The reason for this, we believe, is that the subject you is strongly associated with the object it (note the strong activation of it in the right column of Figure 5), and the object it, as mentioned in the previous paragraph, is strongly associated with the verbs h a v e and get. The difference may be observed most clearly when the network is prompted simultaneously with you as the subject and clause as the object (see Figure 6 on page 8 below). In that case, the verb want is strongly preferred and, though get still takes second place, t e l l and k n o w rank third and fourth, respectivelyconsistent with the results in Table 1. This demonstrates that the network model is sensitive to high-order correlations among words in the input, not merely the first-order correlations between pronoun and verb occurrences.</Paragraph>
      <Paragraph position="2"> These results do not depend on using an autoassociation network, and we do not claim that children in fact use an autoassociation architecture to learn language. Any statistical learner that is able to discover higher-order correlations will produce results similar to the ones shown here. An autoassociator was chosen only as a simple means of demonstrating in principle that a statistical learner can extract the statistical regularities from the data.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>