<?xml version="1.0" standalone="yes"?>
<Paper uid="C96-1008">
  <Title>Communication in large distributed AI Systems for Natural Language Processing</Title>
  <Section position="3" start_page="35" end_page="35" type="metho">
    <SectionTitle>
2 Application acrchitecture
</SectionTitle>
    <Paragraph position="0"> Verbmobil, the primary application for which ICE was built, aims at developing an automatic interpreting device tbr a special type of negotiation between business people. The dialogue situation is as follows: Two business persons, speaking difl'ereat languages, are involved in a face-to-face dialogue trying to schedule an appointment. 'l.'hey both have at least; some knowledge of English and use English as a common hmguage. In case one of the dialogue partners runs into problems, he or she activates the interpretation system by pressing a button and switches back to his or her mothertongue. The system interprets the respective utterances into English. Therefore, it interprets the dialogue on demand in certain situations.</Paragraph>
    <Paragraph position="1"> The Verbmobil system consists of a large number of components, each of them designed to cope with specific aspects of the interpretation process. Among them are a recorder for speech signMs, a HMM-based word recognizer, modules for prosodic, syntactic and semantic analysis, dialogue processing, semantic evaluation as well as components for both german and english synthesis. There are several interfaces between the individual parts of the application which are used to forward results or to realize question-answering behavior.</Paragraph>
    <Paragraph position="2"> The interchanged data between components (a component normMly corresponds to a unique software module) is very heterogeneous with regard to both type and quantity: Speech information as it is sent from the recorder to the speech recognizer consists of a stream of short integer values which may amount to several megabytes. The objects exchanged between semantics construction and transfer are relatively small, but highly structured: Semantic representations with several embedded layers.</Paragraph>
  </Section>
  <Section position="4" start_page="35" end_page="37" type="metho">
    <SectionTitle>
3 ICE: Design and structrue
</SectionTitle>
    <Paragraph position="0"> As briefly noted above, we are using a channel abstraction to model communication between components. The model is largely oriented at the approach of CSP (Communicating Sequential Processes, iIoare (1978)), mainly for two reasons: * We decided to use a message-passing approach to communication. The two other kinds of process communication largely available, namely shared memory and remote procedure calls are disadvantegous for our purposes: The employment of shared memory may lead to memory or bus contention when several processors are sinmltaneously attached to the same physical memory segment. l?urthermore, multiple concurrent write attempts have to be synchronized. Remote procedure cMls did not seem to be the right choice either since their use implies a rendez-vous-synchronization which slows down a system due to network latencies 1.</Paragraph>
    <Paragraph position="1"> * Making the objects involved in communication explicit, offers several ways to manipulate them. Without too much effort, we were able to introduce split channels in order to incorporate visualization tools or introdnce different modes of communication depending on the type of data to be exchanged.</Paragraph>
    <Paragraph position="2"> The low level basis of ICE is realized by PVM (Geist et al., 1994), which is a message passing system for multiple hardware architectures. It has been developed and extended for almost seven years now and is very reliable. It allows a net of Unix workstations to behave like a single large parallel computer. PVM supplies each message with a tag which simplified the introduction of channels to a large extent (roughly, a message is tagged uniquely to identify the channel it is sent on. This enables a receiving component to select messages Oll individual channels).</Paragraph>
    <Section position="1" start_page="35" end_page="36" type="sub_section">
      <SectionTitle>
3.1 System structure
</SectionTitle>
      <Paragraph position="0"> The architecture of a system using ICE as communication framework is depicted in Pig. 1. Before describing in detail the structure of a component, we will point out the overall layout of an application. null We assume that an application consists of a number of components. We could have adopted the notion of agents cooperating to a certain degree while carrying out a certain task cooperatively, but this would have meant to mix up different conceptual levels of a system: The communication facilities we are describing here establish the means by which pieces of sottware may communicate with each other. They do not prescribe the engineering approaches used to implement the individual software components themselves. We do not state that agent architectures 1 rph e channels of CSP and Occam both use rendezvous-synchronization. In this respect we deviated fi'om the original model.</Paragraph>
      <Paragraph position="1">  (e.g. Col-,en et al. (:t9!)4)) can not be realized with our :mechanis:nl 2, but the range of cases where ICI,', can 1)e applied is broader than this.</Paragraph>
      <Paragraph position="2"> All communication is clone by the trieatls of channels, as set out above. We (listinguish two types of channels: * Base channels ~re the primary \['acilities of communication. They are configured in a way guaranteeing that each comlxment is able to interm:t with each other component it wishes to, regardless of programming hmguages, hardware architectures, or system software being used. This is achiewxl 1)y using the standard communication mode of PVM, which supports XI)I{ a. Message passing is done asynchronously.</Paragraph>
      <Paragraph position="3"> * Additional channels were added in order to satis\[~y some needs that frequently arise during the design ~md implementation of b~rge A l-systems with heavy use of communication.</Paragraph>
      <Paragraph position="4"> They can 1)e used to separate data st,'cants Dora control messages or may 1)c configured it, various ways, e.g. by switching off {;he X1)t{ encoding to speed up message passing.</Paragraph>
    </Section>
    <Section position="2" start_page="36" end_page="37" type="sub_section">
      <SectionTitle>
3.2 Split ('hannels
</SectionTitle>
      <Paragraph position="0"> Both types of channels can be configured in an ~dditional way. Beyond being bidirectionM communication devices between two components, other 2In(Iced, distributed 1)lackboards as used in (Johen et al. (\]994) can easily be modelled using a (:hanncl-bascd al)proach.</Paragraph>
      <Paragraph position="1"> aeXternal 1)ata Representation, see Corbin (1990), an encoding schema for data objects independent of the current programming environment.</Paragraph>
      <Paragraph position="2"> modules can be attached to listen to data transported on a channel or to inject messages. These split channels are achieved by dividing ~L channel into two endpoints, one at each side of the channel. null Iloth ends are described using a conlignr~tion lile that is read by the ILS (see below) upon startup. In l;his file, \[br each endpoint a list of real chaimels is defined, e~mh of which points to a compolmnt and is equipped with a name, (:onfiguration flags and its purpose (whieh can be sending or receiving). Any uumber of' real channels may be marked sendiug or receiving. The behavior of l;he components allotted by split chammls does not have to be changed, since splitting occurs trans-I)arently for them.</Paragraph>
      <Paragraph position="3"> Consider Fig. 2 as an exa.mi)\[e for what purpose split channels were used.</Paragraph>
      <Paragraph position="4">  ing a channel which is depicted by a dashed line. The channel endpoints are split up to allow visualization of message data sent by either component. The visualization is performed by two additional components la.belled UI_A and UI_B. lqn:thermore, the data sent by component A must undergo some moditication while being transported to cOlnl)onellt l~. Thus, another component C is contigurcd capable of transforming the data. It is spliced into the (h~ta path between A and B.</Paragraph>
      <Paragraph position="5"> Note that data sent by component B arrives at A unaffeeted from modification by component C.</Paragraph>
      <Paragraph position="6"> a.a ILS: Ilfforlnation Service Channels can be established by any component.</Paragraph>
      <Paragraph position="7"> There is no need for synchronization between conlponents during the configuration of the con&gt; munication system. To support this schema, a dedicated component named ILS (Intarc License ,%rver) was introduced, lilt stores information about the actuM structure of the applieation system. This information includes names and locations of all components participating in the sys- null tern as well as an overview about all channels currently established between components. The actions performed by the ILS include: * Attachment and Detachment of components.</Paragraph>
      <Paragraph position="8"> A component desiring to take part in the communication activities of the application has to identify and register itself at the ILS.</Paragraph>
      <Paragraph position="9"> This is done by sending a message containing the name of the component to the ILS. Analogously, a component should detach itself from the ILS by sending an appropriate message before leaving the application. In case of a program failure resulting in the inability of a component to detach the 1LS is capable of handling the detachment autonomously.</Paragraph>
      <Paragraph position="10"> * Configuration of channels. Each creation and destruction of a channel is done by interacting with the ILS in order to notify the ILS of the request and to get back information about the necessary data structures. The creation of a channel is done in two phases: First, any of the endpoint components sends a channel creation request to the ILS. The ILS updates its internal configuration map taking care that split channel definitions are taken into account; it then answers to the requesting component the individual tag used for this channel and the process identity of the target component 4. If the target component has not, yet registered within the application, this fact is acknowledged to the source component. The only point at which this matters is the time of the first message sending attempt which will be blocked until the target component registers at the ILS. In that case, the ILS notifies the source component of the event and communication c~n take place norreally. null The second phase handles the notification of the target component. As just described, this component need not be present by the time of the channel creation request. In this case the notification is simply delayed. The notification consists of the necessary data to create the intended channel within the component. The implementor need not track those configuration messages, the communication layer handles this transparently. Fur4pVM addresses components which are identical to processes for it - by a task id that is assigned by the pwn daemon. The ILS maintains a mapping fl'om compolmnt names to those task ids. This mapping need not be bijective, since we allow multiple componen6s within one process (see below).</Paragraph>
      <Paragraph position="11"> thermore, concurring channel requests do not inteffer.</Paragraph>
    </Section>
    <Section position="3" start_page="37" end_page="37" type="sub_section">
      <SectionTitle>
3.4 Component structure
</SectionTitle>
      <Paragraph position="0"> The interior structure of a component (see Fig. 1) is layered as far as the communication parts of the software are concerned. The low level communication routines are provided by PVM (see above).</Paragraph>
      <Paragraph position="1"> Next, a software layer defines the functions of ICE.</Paragraph>
      <Paragraph position="2"> This is comprised of the basic functionality of ICE itself and a set of interface functions for different programming languages. We currently support C, C++, Lisp (Allegro Common Lisp, Lucid Common Lisp and CMSP), Prolog (Quintus Prolog and Sicstus Prolog) and Tcl/Tk.</Paragraph>
      <Paragraph position="3"> These software layers suffice to communicate basic data types like nmnbers and strings. Additionally, a separate layer (IDL) is present to allow the exchange of more compex data types. One may specify routines to encode and decode user-defined data types which can then be transmitted just as the predefined scalar types. At the lnoment, this schema is used for a few dedicated data structures, e.g. for speech data or arbitrary prolog terms, which may be even cyclic.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="37" end_page="37" type="metho">
    <SectionTitle>
4 Experiences with the application
</SectionTitle>
    <Paragraph position="0"> Verbmobil is built up by two sorts of components.</Paragraph>
    <Paragraph position="1"> The &amp;quot;(:ore&amp;quot; components are used to transform the input data into the output data (e.g. recording, speech recognizer etc.). These Nl,P-&lt;'omponents are embedded in the so called &amp;quot;testbed&amp;quot; that serves as an application Damework. The testbed is designed as an experimental enviromnent that provides all the features required to test the core components and to study the operation of the whole application. '\]?he testbed consists mainly of the following parts: * The graphical user interface (GUI) provides a comlbrtable Dontcnd to the application. Using the GUI the user can watch the operation of the whole system, control the behavior of the components and monitor the datafiow between the components.</Paragraph>
    <Paragraph position="2"> * The testbed manager (TBM) is used to start up the whole application and to distribute the processes of the application to the hosts of the network. Further, the testbed manager collects data about the operation of the components and visnalizes this intbrmation using the GUI.</Paragraph>
    <Paragraph position="3"> (r) The visualisation manager (VIM) collects all the data transferred between any of the components using IC'E channels.</Paragraph>
    <Paragraph position="4">  If one wants to study only some parts of the system, it is t)ossib\]e to start the al)l)li(;ation containing only a subset of the existing components (e.g. only the speech recording module aim some speech recognizers). The testbed provides the fa.cility to choose in an oflline process the components that are desired to I)e executed. This configuration is done by simply editing a coMiguration file and selecting the keywords &amp;quot;yes&amp;quot; or &amp;quot;no&amp;quot; for each cornl)onent. All the comf)onents not selected are automatically replaced by &amp;quot;stut)-modules,&amp;quot; so there is no need to change source code and recompile the components, ew:n if data is sent 1,o a nou-existent component. On the other side it; is possible to configure the usage of alternative components (e.g. two gerlnan speech recognizers). In this case l)oth eoml)onents are started and we are at)It to select fl'om the GUI which of both (:onq)o-Ilents we a(:tllally wallt to \]lSe.</Paragraph>
    <Paragraph position="5"> (htrrent\]y there are 32 existing eon~l)otmnts that contribute to roughly 650 Mill disk space (the executables, libraries and data liles required at rllntime use up 380 MB). Some of the components ~u:e structured using sul)eomi)onents that are iml)le mented in different programnfing languages and are executed in own l)rocesses. The ',{2 main con&gt; ponents are implemented using the following l)ro tramming languages: C (10 components), l,isl) (r (:o.u)ouents), l'r(,log (S ,'onU,onents), (:++ ,:on u)onents), t,'ortra,, C/:o, ,pouents), 'r,:F'rk (J (:o111 l)Onellt).</Paragraph>
    <Paragraph position="6"> Starting a heavy weight system containing all the currently existing eoml)onents, we get al)out 95 UNIX l)roeesses requiring 520 M\]l memory. In this configuration we are using 52 I)ase channels and 24 additional channels (76 ICI'; channels in total). Six of these 24 additional (:hannels are configured not to use the XI)R coding, 1)eeause they are used to transfer high volume data (e.g. audk) data).</Paragraph>
    <Paragraph position="7"> Because the communication is built u 1) I)y strictly using the featm:es of ICE and the underh~ying PVM, the apl)licatiott cnn run on ~ single host ;~s well as distributed to the hosts of a.a local area network. The decision which cOlnl)onent will rtm on which host of the network is conligurable.</Paragraph>
    <Paragraph position="8"> Each coml)onent can I)e assigned to a sl)ecilie host, or wc can leave the assignment of an adequate host to PVM.</Paragraph>
  </Section>
  <Section position="6" start_page="37" end_page="39" type="metho">
    <SectionTitle>
5 Experiences with an
</SectionTitle>
    <Paragraph position="0"> architectural experiment la addition to the employment wil;hin the Verl)mo-I)il l)rototype, we used l(Jl,', as con,,,mm(:ation &amp;'vice ('or some eXl)erjt~mnts i l, the \['ra.ntewor\]C/ o1&amp;quot; the  architC/'ctm'M branch of the project. The apl)roac\]l here is to develop a. speech translation system obeying design principles that have their orighl in the goal of constructing a system retlecting some of the assumed properties of human speech proeessing, namely working incrementally fi'om left to right and exploring the ell'ects of il~teraetion between dilDrent levels of speech recognition and understanding. These two principles have serious implications for the design of individual tempo uents and the (:on-,ph;te system. To give a concrete exmnple, consider the interface between a speech recognizer and a synt~mtic parser. The re&lt;:ognizer In'educes a eonllected graph where each edge denotes a word hypothesis. Due to the inability to remove paths in adwmee that can not be pursued fln:ther at a late\]: stage of operation, the input to the syntactic parser grows enormously.</Paragraph>
    <Paragraph position="1"> We noticed that wordgri~phs produced inerenmn tMly laity \])e tell tiIlles larger than conw'.ntionally constructed gr~q)hs (resulting in over 2000 word hypol, heses for an utterance of 4.7 seconds).</Paragraph>
    <Paragraph position="2"> The exlmrinmntM system architecture is shown in Fig. 3. It, consists off several modules interconnected by ,t lnain da.tlt path that delivers resuits according to the &amp;quot;standard&amp;quot; linguistic hie&gt; archy, viz. from word recognition to syntax, seuumtics and fitmtly transfer !;. Besides t, his inain stream data path we set t\] l) several interaction facilities that ~u'e used to propagate int'ornmtiou r&gt;J'ltc (;r;ttlsf(w (:ompontm{, i,';t not, shown in \]&amp;quot;it,+ 3.  backwards, which may consist of binary judgemeats about the applicability of a hypothesis, a ranking among different possible analyses or even predictions about what might be expected in the future.</Paragraph>
    <Paragraph position="3"> These methods were for example examined at the crucial interface between a HMM-based speech recognition device and a syntactic parser (ttauenstein and Weber, 1994). A tight interaction between these two components was created which was used to model a synchronization point at every frame in the speech input (i.e. every 10 ms).</Paragraph>
    <Paragraph position="4"> At each of these points a set of word hypotheses is sent to the parser. The parser then tries to integrate the new hypotheses into existing partial analyses constructed so far. The feedback loop to the speech recognizer consists of information about the syntactic ranking of the parse each word is integrated into. If a word can not be used in any way, it is simply rejected. In the case of integration of a word into a parse a ranking is produced which incorporates values from a statistical n-gram language model and a stochastic unifica|;ion grammar which models the probability of a syntactic derivation.</Paragraph>
    <Paragraph position="5"> To realize a prediction mode in this interaction, a different schema was used: At each frame the parser computes a set of possible continuations for each word, i.e. it restricts the language model to pairs of words (in case of a bigram model) which are syntacticallly plausible and could be integrated into a currently existing syntactic derivation. By doing so, the search space of the speech recognizer is restricted.</Paragraph>
  </Section>
class="xml-element"></Paper>