<?xml version="1.0" standalone="yes"?>
<Paper uid="C88-1038">
  <Title>Sequencing in a Connectionist Model of Language Processing I</Title>
  <Section position="4" start_page="186" end_page="186" type="metho">
    <SectionTitle>
4. Sequencing
</SectionTitle>
    <Paragraph position="0"> It is not a straightforward matter to implement sequential behavior within the confines of a system consisting of simple processing units that are activated in parallel. Alongside the basic problem of ereating emergent sequential behavior from a parallel process, there is the need for sequencing information of two types to be transmitted. When it is time for a constituent to be produced, it needs to signal its own daughter constituents to be produced in the appropriate sequence and, when these are completed, to signal sister constituents which follow it to be produced.</Paragraph>
    <Paragraph position="1"> The lhmst of this paper is that sequencing can be modelled like the rest of language processing, that is, as a series of selections made on the basis of interacting quantitative factors. Consider first how the parallel activation spread is turned into a sequential process during generation. Activation spreads initially from nodes representing the semantics and pragmatics of the utterance to nodes representing the lexical and grammatical patterns to be used, hut the thresholds of the constituent nodes of these patterns are such that the nodes cannot yet fire. They fire ~nly when they have received addition,&amp;quot;d activation along connections specifying sequencing relations between constituents.</Paragraph>
    <Paragraph position="2"> When more than one constituent may follow a given constituent, there arc connections to all of the alternatives. The weights on these connections represent degrees of syntactic expectation regarding which constituent will follow, and the constituent nodes inhibit each other tlu~ugh a WFA network wlfich permits only one at a time to fire. It is the combination of the activation ret)resenting syntactic information mid that from other sources which determines which constituent wins out over the others and fires. The firing of the winning constituent represents the selection of an item to fill the next output position.</Paragraph>
    <Paragraph position="3"> A second problem involves the two types of signals which constituent nodes must send. This problem is handled by having two nodes for each constituent or phrase, one representing the start and the other the end of the unit. The start node signals daughter constituents to be produced, and the end node signals following sister constituents to be produced.</Paragraph>
    <Paragraph position="4"> Figure 2 illustrates some of the sequencing infonnation in thc *SEND-MAIL entry. Sta,t-end node pai1~ arc denoted by pairs of small squares surrounded by rectangles with rounded corners. The upper square represents the start, the lower square the end of the word or phrase. Single directional connections are indicated by arrow heads, and pairs of inhibitory coimections are denoted by fuzzy lines. Tile figalre includes some sequence connections and the WTA network which represents the competition between the OBJEC%REFERENCE and RECIPIENT-REFERENCE constituents for the position following the VERB. Hem the WTA source is the VERB/end node, which sends activation to both the OBJECT-REFERENCE/start and RECIPIENT-REFERENCE\]start nodes. These two nodes inhibit each other.</Paragraph>
  </Section>
  <Section position="5" start_page="186" end_page="188" type="metho">
    <SectionTitle>
5. An Example
5.1. Generation
</SectionTitle>
    <Paragraph position="0"> Consider now the generation of sentence (la): Instead of calling John, Mary sent him a letter. Generation begins with the firing of a set of network nodes representing a goal of the speaker. In this ease the goal is that the hearer believe that a particular event (the sending of the letter) replaces one previously assumed to occur (the making of a telephone call). This type of goal leads the system to generate a STATEMENT referring to the event preceded by a phrase which denies the assumption (instead of calling John). We concentrate here on the generation of the clause beginning with Mary and in  particular on the Sequencing of the last two constituents.</Paragraph>
    <Paragraph position="1"> The event to be referred to is represented as an instance of the general ABSTRACT-TRANSFER predicate (Schank &amp; Abelson 1977) with MARY as the ACTOR, an instance of the concept LETI'ER as the OBJECT, JOHN as the RECIPIENT, and MAIL as the MEANS of the transfer. We ignore time and tense in order to simplify the discussion. The utterance of the initial instead of phrase results in processing of the concepts of MARY and JOHN, so there is residual activation on these nodes and the nodes immediately connected to them. A portion of the network at this point is shown in Figure 3. Nodes with hatched patterns are those with activation below the firing threshold level.</Paragraph>
    <Paragraph position="3"> ..... Figure 3: Portion of Input to Generation of (1 a) ...........</Paragraph>
    <Paragraph position="4"> Activation spreading from ABSTRACT-TRANSFER8 (i.e., the specific transfer instance) converges on a set of verb lexical entries that may be used to describe the input notion. Competition among the CONTENT roles of these entries eventually forces one to win out. For this example, we assume that the *SEND-MAIL entry would predominate because of the fact that it matches the input MEANS feature, though the entry for the verb mail would also be a strong candidate. A simplified view of this lexical entry selection process is shown in Figure 4. The path of activation spread is indicated by arrows in the figure, blackened nodes are those that fire initially, and nodes with wide borders are those that fire in response to the spread of activation. The fuzzy lines emanating from *SEND-MAIL:CONTENT are inhibitory connections to other verb CONTENT roles.</Paragraph>
    <Paragraph position="5">  Once the *SEND-MAIL entry has been selected, activation spreads through it, resulting in the priming of the nodes representing the constituents of the clause. At the same time activation has also spread to the constituent nodes of the higher-level CLAUSE schema.</Paragraph>
    <Paragraph position="6"> The connections within this schema determine the order of the SUBJECT and VERB in the sentence. The fact that the event referred to * occurred before the time of speaking also leads to the selection of the PAST-CLAUSE schema, and this in combination with the *SEND-MAIL schema results in the firing of the node representing the word sent. For the purposes of this paper, we ignore the details o(these processes.</Paragraph>
    <Paragraph position="7"> When the verb has been produced, the VERB/end node in the *SEND-MAIL entry fires. From here activation spreads to the nodes representing the beginnings of the two possible following constituents: RECIPIENT-REFERENCE/start and OBJECT-REFERENCE/starL These nodes compete with one another via a WTA network. In this case the priming on the RECIPIENT-REFERENCE/start node leads this constituent to win out over OBJECT-REFERENCE/start. The situation at this point is shown in Figure 6.</Paragraph>
    <Paragraph position="8">  Next the NP schema takes over. At this point there is competition between the schema for pronouns and that for full NPs.</Paragraph>
    <Paragraph position="9"> The pronoun schema wins out when there is evidence that the hearer is currently conscious of the referent. In this case such evidence is available in the form of residual activation resulting from the reference to John in the phrase instead of calling John. For details on how spreading activation and competition implement the selection of pronouns over full NPs, see Gasser (1988).</Paragraph>
    <Paragraph position="10"> When the NP is complete, activation is sent back to the RECIPIENT-REFERENCE/end node, which then activates the nodes representing the two possibilities for what follows. One is that the clause is complete. This option would be the appropriate one if the RECIPIENT-REFERENCE had followed the OBJECT-REFERENCE (as in Mary sent a letter to John). The other option, the one that is appropriate for tiffs example, is that the OBJECT-REFERENCE follows. The reason that both possibilities need to be represented is that the system has no explicit memory for what has or has not already been generated. The weights on the two connections are such that the second alternative is the default and will be preferred in this ease. That  is, OBJECT-REFERENCE wins out, and the OBJECT-REFERENCE/start Imde fires. As shown in Figure 7, tire selection of file OBJECT.REFERENCE role leads eventually to the firing of the OBJECT iole ill ATRANS8 and the LE'lqT.,R node, \[~_%&amp;quot; ...... ~,.. ~</Paragraph>
    <Paragraph position="12"> .Agaill contlol is passed to file NP schema, l/ere two lilrdlcr selections tat e place. The fact that them is no evidcnce that the hearer knows the referent leads to selection of the INDEFINITE-NP schema over the DEFINITE-NP schema by default. INDENNITE-NP six:cities the indclinik~ mlicle a. Finally, the lexical entry *\[,ETIEI( is selected as as a result of activation spreading from the I,ETTER i:lode. This schema provides fl~e l~tun letter lor the OBJECf NP.</Paragraph>
    <Paragraph position="13"> Once the final constituent is complete, actiwttion is sent back to tile OBJECT-REFERF, NCE/end node. Again there are two possibilities lot what may ~ollow, the end of rite clau~z or file to case marke~' and the following RECIPIENT-RFJ3ERENCE cottstitoent, Noun, however, rant there is an inhibitory connection from RECIPIENT-REFERENCE/end to RECIPIENT-MARKER/star.. That is, tl,,e completion of the RECIPIENT-tlEFERENCE effectively prevents the later generali~m of the case marker, and as a consequence the repetition of the RECIPIENT-REFFIIENCE itself. Tile state of the network at this point is shown ill Figure 8. The fuzzy filled pattern on RECIP1ENT-MARKFa//sla,:t indicates lhat tie node is inhibited.</Paragraph>
    <Paragraph position="15"> ......... l:~; Co~letion of Genelation of (In) bl this example we have iw.-,~de use of sequencing intbrmation found in the l~ieal entry *SEND-MA~L. 'Ills sort of irttb~anation also appear~ in irate general lexical entries such ~ *SEND and in nonlexieal GUs such ~ ATRANS-CLAUSE, file schema for elanses ~eferfing to sa ABSTRAC~I'-TRANSFER. If a specilic entry lacks the required itffol mutton, a mote germral schema is used automatically.</Paragraph>
    <Paragraph position="16"> ~.2. Parshtg Now cow,sider how i/~z same information wotdd Ix; used in file pulsing of tl~, sentence Mary sent hbn a letter. Recognition of the word Mary leads to rite selection of the *MARY entry and the consequent firing of the MARY node. Recognition of the wold sent results in the selection of the *SEND entry, which is similar to the more specific *SEND-MAlt, entry shown in Figures 1, 2, 4, 5, 6, 7, ,and 8. Activation is scant immediately to the SUBJEC:r cons'tituent of the enlry, resulting eventually in file firing of tile ACTOR node. li. is file close pmximation of the firing of MARY and ACTOR which represents the role binding aspect of parsing. Recall from 3.3 aixwe that thet~e is currently no way to record this binding pemlanently in file system's memory.</Paragraph>
    <Paragraph position="17"> The firing of tile VERB\]end node in file *SEND schelna leads, as in the generation of the same sentence, to file activation of ntxlcs for both of the constituents which may follow. At this point neither of these constituents has enough activation to fire. The activation that is present l~prescnts tile expectation that there will now be a reference ~o either file RECIPIENT or the OP.JE(71'.</Paragraph>
    <Paragraph position="18"> Next the word him is recognized, leading to file activation of all male humm~s lhat the system is cmrently &amp;quot;thinking abeut&amp;quot;. There is only one such entity, John, and the JOHN node then fires. Activation spreads to nodes for featnrcs of John including tile HUMAN node.</Paragraph>
    <Paragraph position="19"> Since humanness is a default property of the RECIPIF, NT of an ABS'IRACI'-TRANSFER, Ibis last node is connected to the RECIPIENT node, which can now fin., sending activation ill turn eventually to tilt RECIPIENT-REFERENCE/start node in the *SEND schema. 't'hc additional activalion now causes this node to lire, rcpresentir~g the system's recognition that tile current constituent refers Io tile RECIPIENT rather titan the OBJECF of the ABSTRA(YF-Ti~ANSI: ElL.</Paragraph>
    <Paragraph position="20"> Frorn this point on, the process, at least with respcc~ ,x) sequencing, is sinfilar to what goes on during generation. After lh,? apt~aranee of the word him, activation spreads from the RECIPIENT.</Paragraph>
    <Paragraph position="21"> REFERENCE/end node to the nodes representing the two possible alternatives, the end of the clause or the appearance of the OBJLCT-REFERENCE. &amp;quot;File latter will predominate in this ez, a~nple once the bcgimfing of tile NP the letter is lecognizxxl. POllowing the completion of this NP, there, will again be two alternatives, lu this case the CLAUSE/end option will win out, as in the generation case, because of inlfibition on the node for the al|em~ttive.</Paragraph>
    <Paragraph position="22"> 6. implementation and Coverage Tim, model described in this paper is implemented ill a plogrmn called CIilE. The program has two components, a hand-coded memory network representing both world knowledge and linguistic knowledge and a set of procedures implementing spreading activation and inhibition through WTA networks. CHIE generates sentences in English and Japanese given input in the form of activated network nodes representing speaker gnats. The model has been tested for a small li'agment of the grammars of these languages: simple declarative and interrogative clauses and imun phrases with adjective modifiers.</Paragraph>
    <Paragraph position="23"> In addition to obligatory constituents like those in the example above, tile program handles optional and optionally iterating constituents. The program also &amp;quot;parses&amp;quot; tilt structm'es that it ganerates using the same memory and the stone basic procedures, but, as noted in 3.3, it does not save a semantic interpretation; that is, it does not know how to create schema instantiations with role bindings. (See Dolan &amp; Dyer 1987 for an approach to this problem within the connectionist fi'eanework.)</Paragraph>
  </Section>
class="xml-element"></Paper>