<?xml version="1.0" standalone="yes"?>
<Paper uid="C96-1085">
  <Title>Restricted Parallelism in Object-Oriented Lexical Parsing</Title>
  <Section position="3" start_page="0" end_page="502" type="metho">
    <SectionTitle>
2 Design Requirements for the Parser
</SectionTitle>
    <Paragraph position="0"> The application framework for the parsing device under consideration is set up by the analysis of real-world expository texts (viz. information technology product reviews and medical findings reports). The parser operates as the NLP component of a text knowledge acquisition and knowledge base synthesis system. null The analysis of texts (as opposed to sentences in isolation) requires the consideration of discourse phenomena. This includes the interaction of discourse entities (organized in focus spaces and center lists) with structural descriptions from the parser and conceptual information from the domain knowledge base. Thus, different knowledge sources have to be integrated in the course of an incremental text understanding process. null Within realistic NLP scenarios the parsing device will encounter ungrammatical and extragrammatical input. In any of these cases, the parser should guarantee a robust, graceful degradation performance, i.e., produce fragmentary parses and interpretations corresponding to the degree of violation or lack of grammar constraints. Depending on the severity of fragmentation, changes in the parsing strategies which drive the text analysis might also be reasonable.</Paragraph>
    <Paragraph position="1"> These requirements obviously put a massive burden on the control mechanisms of a text understanding system. In particular, entirely serial control schemata seem inappropriate, since they would introduce artificial serialization constraints into basically parallel processes (Waltz &amp; Pollack, 1985).</Paragraph>
  </Section>
  <Section position="4" start_page="502" end_page="505" type="metho">
    <SectionTitle>
3 Object-oriented Lexical Parsing
</SectionTitle>
    <Paragraph position="0"> In this section, we introduce the PARSETALK system, whose specification and implementation is based on an object-oriented, inherently concurrent approach to natural language analysis. We consider constraints which introduce increasing restrictions on the parallel execution of the parsing task. This leads us to a parsing algorithm with restricted parallelism, whose experimental evaluation is briefly summarized.</Paragraph>
    <Section position="1" start_page="502" end_page="502" type="sub_section">
      <SectionTitle>
3.1 The PARSETALK Model
</SectionTitle>
      <Paragraph position="0"> The PARSETALK grammar we use (for a survey, cf.</Paragraph>
      <Paragraph position="1"> BrOker et al. (1994)) is based on binary relations between words, e.g., dependency relations between a head and its modifier, or textual relations between an anaphor and its antecedent. Restrictions on possible relations are attached to the words, e.g., expressed as valencies in the case of dependency relations, yielding a strictly lexicalized grammar in the sense of Schabes et al. (1988). The individual behavior of words is generalized in terms of word classes which are primarily motivated by governability or phrasal distribution; additional criteria include inflection, anaphoric behavior, and possible modifiers. A word class specifies morphosyntactic features, valencies, and allowed orderings for its instances. Further abstraction is achieved by organizing word classes at different levels of speciticity in terms of inheritance hierarchies. The specification of binary constraints already provides inherent means for robust analysis, as grammatical functions describe relations between words rather than welltormed constituents. Thus, ill-formed input does often still have an (incomplete) analysis.</Paragraph>
      <Paragraph position="2"> Tile PARSETALK parser (for a survey, cf. Neuhaus &amp; Hahn (1996)) generates dependency structures for sentences and coherence relations at the text level of analysis. In order to establish, e.g., a dependency relation the syntactic and semantic constraints relating to the head and its prospective modifier are checked in tandem. Due to this close coupling of grammatical and conceptual constraints syntactically possible though otherwise disallowed structures are filtered out as early as possible. Also, the provision of conceptual entities which are incrementally generated by the semantic interpretation process supplies the necessary anchoring points for the continuous resolution of textual anaphora and ellipses (Strube &amp; Hahn, 1995; Hahn et al., 1996).</Paragraph>
      <Paragraph position="3"> The lexical distribution of grammatical knowledge one finds in many lexiealized grammar formalisms (e.g., LTAGS (Schabes et al., 1988) or HPSG (Pollard &amp; Sag, 1994)) is still constrained to declarative notions. Given that the control flow of text understanding is globally unpredictable and, also, needs to be purposefully adapted to critical states of the analysis (e.g., cases of severe extragrammaticality), we drive lexicalization to its limits in that we also incorporate procedural control knowledge at the lexical gr,'unmar level. The specification of lexiealized communication primitives &amp;quot;allows heterogeneous and local lorms of interaction among (groups of) lexical items. We, nevertheless, take care not to mix up both levels and provide a formally clean specification platform in terms of the actor model of computation (Agha &amp; Hewitt, 1987).</Paragraph>
      <Paragraph position="4"> In this model each object (actor) constitutes a process on its own. Actors communicate by sending messages, usually, in an asynchronous mode. Upon reception of a message, the receiving actor processes the associated method, a program composed of several grammatical predicates, e.g., SYNTAXCtIECK, which accounts for morphosyntactic or word order constraints, or CONCEPTCItECK, which refers to the terminological knowledge representatiou layer and accounts for type and further conceptual admissibility constraints (number restrictions, etc.).</Paragraph>
      <Paragraph position="5"> The grammatical description of single words is organized in a hierarchy of so-called word actors which not only inherit the declarative portions of grammatieal knowledge, but are also supplied with lexicalized procedural knowledge that specifies their parsing beo havior in terms of a message protocol. A specialized actor type, called phrase actor, comprises word actors which are connected by dependency relations and encapsulates inh)rmation about that phrase.</Paragraph>
    </Section>
    <Section position="2" start_page="502" end_page="503" type="sub_section">
      <SectionTitle>
3.2 Parallelism in Parsing
</SectionTitle>
      <Paragraph position="0"> In the following, we discuss three stages of increasing restrictions of parallelism at the word level, all of which were considered for the design of the algorithm provided in Section 3.3.</Paragraph>
      <Paragraph position="1"> Unbounded Parallelism. Brute-force parsing models such as the primordial soup algorithm (Janssen et al., 1992), at first sight, exhibit a vast potential for parallel execution, since the central operation of building a structure from two independent parts (in our application, e.g., the combination of a head and a single modifier) apparently does not require any centr,'dized control. In such an entirely unconstrained parallel model, a word actor is instantiated from the input string and sends search messages to all other word actors in order to establish a dependency relation, eventually generating a complete parse of the input.</Paragraph>
      <Paragraph position="2"> Consider, however, the case in which a Noun is preceded by a Determiner and an Adjective. In order to form the noun phrase \[Det Aclj \[N\]\] two computation sequences will occur in a primordial soup parser: attaching Det to the N first, then adding Adj, or vice versa. Hence, the major drawback of unrestricted parallel algorithms is their non-confluency and, subsequently, either the large (exponential, in the worst  case) number of spuriously ambiguous analyses, or the global operation of subsequent duplicate elimination. This led us to restrain from unbounded parallelism and, rather, guarantee contluent behavior by the design of the parsing algorithm.</Paragraph>
      <Paragraph position="3"> Conflueney. In the first prototype, we enforced confluency by an incremental structure-building condition on the basis of a synchronization schema. Messages were forwarded strictly from right to left wandering through the preceding context rather than being broadcasted. Partial structures were organized such that a message which could be successfully processed at a larger structure was not forwarded to any of its constituent parts. But still, the number of ambiguities remained prohibitively large, often due to unnecessary partial structures with large discontinuities. For instance, any determiner preceding a noun forms a new structure, with the Det modifying the N. Usually, a contiguity restriction would filter out those structures given perfectly well-formed input. But such a restriction is detrimental to requirements set up in a realistic text parsing environment, in which the analysis of (possibly large) fractions of un- as well as extragrammatical input must be skipped. Furthermore, ordering restrictions on dependency analyses lor German can be formulated more transparently, if discontinuous structures are allowed.</Paragraph>
      <Paragraph position="4"> Depth-First Approach. These experiences led to a redesign of the first prototype. The parser's forwarding mechanism for search messages was further restricted to circumvent the above mentioned problem of erroneous discontinuous (over)analyses. In this approach, we let phrases that constituted alternative analyses for the same part of the input text be encapsulated in a container actor. Container actors play a central role in controlling the parsing process, because they encapsulate information about the preceding containers that hold the left context and the chronologically previous containers, i.e., a part of the parse history. Container actors comprising single-word phrases are called lexical containers. All phrases in the active container send a search message to the current context container that forwards them to its encapsulated phrases. The search messages are then asynchronously distributed to words within each phrase. If at least one attachment to one of these phrases is possible, no further forwarding to containers which cover text positions preceding the current position will occur. Thus, the new container composed at this stage will contain only those phrases that were encapsulated in the context container and that could be enlarged by attaching a phrase from the active container.</Paragraph>
      <Paragraph position="5"> This procedure enforces a depth-first style of progression, leaving unconsidered many of the theoretieally possible combinations of partial analyses. Still, some information has to be retained in order to backtrack after failures or to employ alternative parsing strategies. We encounter a trade-off between robustness, efficiency, and completeness in parsing. If we were to allow for unrestricted backtracking, we would just trade in run-time complexity for space complexity (for a more detailed discussion, cf. Neuhaus &amp; Hahn (1996)). Hence, we rather restrict backtracking to those containers in the parse history which hold governing phrases, while the containers with modifying phrases are immediately deleted after attachment 1 .</Paragraph>
    </Section>
    <Section position="3" start_page="503" end_page="505" type="sub_section">
      <SectionTitle>
3.3 Restricted Parallel Parsing Algorithm
</SectionTitle>
      <Paragraph position="0"> The parsing algorithm of the PARSETALK system is centered around the head search process of the currently active word actor. If it fails, a modifier search process is triggered; if it succeeds, a new dependency structure is constructed combining the partial analyses. In case both of these protocols are not successful, containers may be skipped so that discontinuous analyses may occur. If the skipping process encounters a linguistically valid boundary (in the most trivial case, the punctation mark of the previous sentence) it stops and switches to a backtracking mode leading to a kind of roll-back of the parser invalidating the currently pursued analysis. In a companion paper (Neuhaus &amp; Hahn, 1996), we give an integrated description of the various subprotocols needed for head/modifier search, ambiguity handling, skipping, backtracking, preferential and predictive parsing.</Paragraph>
      <Paragraph position="1"> In this paper, we concentrate instead on the basic message passing patterns for the establishment of dependency relations, viz. the searchHead protocol, and its concurrency aspects. For illustration purposes we here introduce the protocol in a diagrammatic form (Figs. 1 to 3). The figures depict the main steps of word actor initialization, head search, and phrasal attachment. This format eases communication, while formal specifications based on a temporal logic framework are used when dealing with formal properties of the parser (cf., e.g., Schacht (1995) for a partial termination proof of the receipt handler introduced below). The parser is started by an analyze: message directed to a ParserActor, which is responsible for the global administration of the parsing process (cf. Fig. 1). It instantiates a LexkcalConizaine~:-Actor that encapsulates the (potentially)ambiguous readings of the first word of the text, as accessed from the lexicon and corresponding word classes.</Paragraph>
      <Paragraph position="2"> Upon receiving the analyzeWithContext: message from the ParserActor (of. Fig. 2), X Hence, the incompleteness property of our parser stems firom the selective storage of analyses (i.e., an &amp;quot;incomplete chart&amp;quot; in chart terms), partially compensated by reanalysis.  a ReceiptHandler is instantiated by a synchronous message, intended to detect the partial termination of the subsequently started search protocol. The performSearchHead message triggers (via performSearchHead'2o : messages) asynchronous searchHeadFor: messages to be forwarded by each receiving p h r a s e Ac t o r to its right-most we rdAct o r. From this origin, the search message can be distributed to all word actors at the right &amp;quot;rim&amp;quot; of the dependency Irec by simply forwarding it to Ihe respective heads. After forwarding a, each searehHead message evokes the check of syntactic and semantic restrictions by the corresponding methods. These restrictions nmst be \[net in order to establish a dependency relation between the receiving word actor and the sender of the n|essage. Provided that these constraints are fnltilled, an attach : message is sent to the encapsulating PhraseActor. Before the new composite phrase can be built, the address of the next container actor must be determined. Accordingly, the getNextContainer message either returns this address directly eL if it is not available yet, it will create the next container actor first (the actual creation protocol is not shown). The newIn : message subsequently creates a new PhraseActor that will encapsulate the word actors of the new phrase. Norice, that several attach : messages can be received by a phrase, because the searchttead messages are evaluated in parallel by its word actors.</Paragraph>
      <Paragraph position="3"> In order to actually build the new phrase a eopyAndAttach : message is sent 3. Fig. 3 depicts the copying of the governing and modifying phrases ZSince forwarded messages are sent asynchronously the processing of the searchHoadFor: message takes place concurrently at the forwarding sender m~d the respective receivers. 3As an alternative to the immediate establishment of a dependency relation, a hoadFound message can be returned to enable the subsequent selection of preferred attachments (cf. Neuhaus &amp; Hahn (1996) for such a protocol extension).  into the new PhraseActor by copyHeadFor: and copyModFor : messages, respectively -- cot)ying enables alternative attachments in the concurrent system, i.e., no destructive operations are carried out. Note that in contrast to the searchHead message the PhraseActor lbrwards the copy message to its root actor from where it is distributed in the tree. The dependency relation connecting the copied phrases (indicated by the bold edge in the newly built PhraseAetor) is created by the establish: message. Since word actors hold information (such as features or coverage) locally, updates ,are necessary to propagate the effects of the cre~ttion of the relation. Alier updating inlormation at relevant word actors in the resultant tree, successful termination of the scratch message is signalled to the ReceiptHandler. If none of the receipts signals success to the handler, the search head protocol will be followed by moditier search or backtracking 4 protocols not shown here (cf.</Paragraph>
      <Paragraph position="4"> 4Thus synchronization of protocols enables word-wise scanning, backtracking, etc. This avoids severe problems usually encountered in parsers with unrestricted parallelism.  Neuhaus &amp; Hahn (1996)). In the scenario we have discussed, the Rece ipt Handle r eventually will detect the success and the termination of the search head protocol. Next, the new ContainerActor will be sent an analyzeWithContext : message to continue the parsing process.</Paragraph>
    </Section>
    <Section position="4" start_page="505" end_page="505" type="sub_section">
      <SectionTitle>
3.4 Preliminary Experimental Evaluation
</SectionTitle>
      <Paragraph position="0"> The efficiency gain that results from the parser design introduced in the previous sections can empirically be demonstrated by a comparison of the PARSETALK system (abbreviated as &amp;quot;PT&amp;quot; below) with a standard active chart parser 5 (abbreviated as &amp;quot;CP&amp;quot;). Since the chart parser is implemented in Smalltalk, while the PARSETALK system is implemented in Actalk (Briot, 1989) -- a language which simulates the parallel execution of actors --, a direct comparison of run times is not reasonable (though even at that level of consideration the PARSETALK system outperforms the chart parser). We therefore compare, at a more abstract computation level, the number of method executions given exactly the same dependency grammar 6. The computationally most expensive methods we consider are SYNTAXCHECK and CONCEPTCHECK (cf. Section 3.1). Especially the latter consumes large computational resources, since for each interpretation variant a knowledge base context has to be built and conceptual consistency must be checked. Therefore, it is only considered when the syntactic criteria are fulfilled. The number of calls to these methods for a sample of 13 randomly chosen, increasingly complex sentences from the information technology domain test library is given by Fig. 4 (&amp;quot;CP.syn&amp;quot; and &amp;quot;PT.syn&amp;quot;) and Fig. 5 (&amp;quot;CP.con&amp;quot; and &amp;quot;PT.con&amp;quot;). A reduction by a factor of four to five in the (unweighted) average case can be observed applying the PARSETALK strategy.</Paragraph>
      <Paragraph position="1"> Furthermore, the PARSETALK parser, by design, is able to cope with discontinuities stemming from unor extragrammatical input. The performance of a revised version of the chart parser which also handles these cases is given as &amp;quot;CP.disc.syn/con&amp;quot; in the figures. The missing value for sentence 10 results from the chart parser crashing on this input because of space restrictions of the run-time system (the experiments were conducted on a SPARCstation 10 with 64 MB of main memory). The average reduction in compar5Winograd' s (1983) chartparser was adapted to parsing a dependency grammar. No packing or structure-sharing techniques could be used, since semantic interpretation occurs online, thus requiring continuous referential instantiation of structural linguistic items (cf. also Section 4).</Paragraph>
      <Paragraph position="2"> 6This can only serve as a rough estimate, since it does not take into account the exploitation of PARSETALK'S concurrency. Furthermore, the chart parser performs an extremely resource-intensive subsumption checking method unnecessary in the PARSETALK system.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>