<?xml version="1.0" standalone="yes"?>
<Paper uid="P97-1001">
  <Title>Interleaving Universal Principles And Relational Constraints Over Typed Feature Logic Thilo Giitz and Detmar Meurers</Title>
  <Section position="1" start_page="0" end_page="5" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> We introduce a typed feature logic system providing both universal implicational principles as well as definite clauses over feature terms. We show that such an architecture supports a modular encoding of linguistic theories and allows for a compact representation using underspecification. The system is fully implemented and has been used as a workbench to develop and test large HPSG grammars. The techniques described in this paper are not restricted to a specific implementation, but could be added to many current feature-based grammar development systems.</Paragraph>
    <Paragraph position="1"> Introduction A significant part of the development of formalisms for computational linguistics has been concerned with finding the appropriate data structures to model the linguistic entities. The first order terms of Prolog and DCGs were replaced by feature structures in PATR style systems, 1 and in recent years systems using typed feature structures have been developed.</Paragraph>
    <Paragraph position="2"> Following the tradition of DCG and PATR, these typed feature systems are generally definite clause based, e.g., CUF (D5rre and Dorna 1993), or phrase structure based, e.g., ALE (Carpenter and Penn 1994). Instead of permitting the grammar writer to express universal well-formedness constraints directly, the systems require the grammar writer to express relational constraints and attach them locally at the appropriate places in the grammar. 2 We believe there are several reasons why the advances in the linguistic data structures should entail the development of systems offering more expressive means for designing grammars. Using universal implicative constraints, or universal principles as they are usually called in the linguistic literature, grammatical generali1Cf. Shieber (1986) for a discussion of these formalisms. 2ALE has a restricted form of universal constraints, see the comparison section.</Paragraph>
    <Paragraph position="3"> sations can be expressed in a more compact and modular way. Another advantage of an architecture including principles is that it computationally realizes the architecture assumed in Pollard and Sag (1994) for HPSG.</Paragraph>
    <Paragraph position="4"> It thus becomes possible to develop and test HPSG grammars in a computational system without having to recode them as phrase structure or definite clause grammars. The architecture can also serve as extended architecture for principle based parsing (e.g., Stabler and Johnson 1993) since it facilitates the implementation of GB-style universal principles. Offering both more perspicuous grammar code and closeness to linguistic theory, it seems well motivated to explore an architecture which allows both relational constraints and universal restrictions to be expressed.</Paragraph>
    <Paragraph position="5"> Our implementation is based on the idea to compile implicational constraints into a relational representation (GStz and Meurers 1995) where calls to the constraint solver are made explicit. This allows for an integration of implicational and relational constraints and a uniform evaluation strategy. Efficient processing is achieved through user-supplied delay patterns that work on both relations and implicational constraints, as well as preferred execution of deterministic goals at run-time.</Paragraph>
    <Paragraph position="6"> The paper is organised as follows. We will start out by illustrating our architecture with an example. We then go on to describe the key aspects of the implementation. We compare our work to other approaches before presenting some conclusions and open issues in the last section.</Paragraph>
    <Paragraph position="7"> Motivating the architecture Consider the Head Feature Principle (HFP) of Pollard and Sag (1994) as an introductory example for a grammatical principle. The HFP requires that in a headed construction the head features of the mother are identified with the head features of the head daughter. In a typed feature logic 3 this may be expressed by the prin- null ciple shown in Fig. 1.</Paragraph>
    <Paragraph position="8"> phrase A dtrs : headed-struc -+ synsem : loc : cat : head : X A dtrs : head-dtr : synsem : loc : cat : head : X  In CUF, we can encode the HFP as a clause defining a unary relation hfp as shown in Fig. 2. 4 h/p := synsem : loc : cat : head: X A dtrs : head-dtr : synsem : loc : cat : head : X  For the relation hfp to take effect, calls to it need to be attached at the appropriate places. Expressing grammatical constraints in such a way is both time consuming and error prone.</Paragraph>
    <Paragraph position="9"> Suppose we want to define the unary relation wf-phrase to hold of all grammatical phrases. In case all grammatical phrases are constrained by a term C/ and some relation P, we can define the relation wf-phrase shown in Fig. 3.</Paragraph>
    <Paragraph position="11"> To specify that C/ A P holds for all phrases while the HFP only holds for headed phrases, we now have to manually split the definition into two clauses, the subcase we want to attach the HFP to and the other one.</Paragraph>
    <Paragraph position="12"> This is both inelegant and, barring a clever indexing scheme, inefficient. Using universal principles on the other hand, the global grammar organisation does not need to account for every possible distinction. The organisation of the data structure as typed feature structures already provides the necessary structure and the grammatical constraints only need to enforce additional constraints on the relevant subsets. Thus, the implica3Following King (1989) and Carpenter (1992), we use a typed feature logic with appropriateness restrictions for the domains and ranges of features. For space reasons we cannot provide a formal definition of the logic here, but the interested reader is referred to Carpenter (1992) for an exposition.</Paragraph>
    <Paragraph position="13"> 4Throughout the paper and as syntax for the system discussed here we use the functional style notation of CUE (DSrre and Dorna 1993) for our relations, where a designated result argument is made explicit. The denotation of a relation thus is a set of objects just like the denotation of any other feature term.</Paragraph>
    <Paragraph position="15"> modate the HFP call tional constraint encoding the HFP shown in Fig. 1 constrains only the headed phrases, and the non-headed ones do not need to be considered.</Paragraph>
    <Paragraph position="16"> Finally, a system providing both universal principles and relational constraints at the same level offers a large degree of flexibility. While in ttPSG theories the principles usually form the main part of the grammar and relations such as append are used as auxiliary constraints called by the principles, a more traditional kind of grammar for which one prefers a relational organisation can also be expressed more compactly by adding some universal principles constraining the arguments of the relations to the relational core. Both kinds of interaction are possible in the non-layered architecture we propose.</Paragraph>
    <Paragraph position="17"> With respect to our example, the first kind of interaction would be obtained by also expressing the general restriction on phrase as a universal constraint as shown in Fig. 5, while the more traditional kind of grammar  would keep the relation defining well-formed phrases shown in Fig. 3 and combine it with the universal constraint of Fig. 1 in order to avoid splitting up the relation as was shown in Fig. 4. The proper interaction of relational and universal constraints then needs to be taken care of by the system.</Paragraph>
    <Paragraph position="18"> An example grammar To further motivate our approach, we now show how to code a simple principle based grammar in our framework. Figure 6 shows the inheritance hierarchy of types which we will use for our example with the appropriateness conditions attached.</Paragraph>
    <Paragraph position="19"> Our example grammar consists of some universal principles, phrase structure rules and a lexicon. The lexicon and phrase structure rules are encoded in the wfs (well-formed sign) relation shown in Fig. 7 and the implicational principles in Fig. 8. The wfs predicate takes three arguments: a difference list pair threading the string through the tree in the style of DCGS, and level  the syntactic category (sign) as functional style result argument, s The analysis tree is encoded in two daughters features as part of the syntactic categories in the style of HPSG. Clause 1 of wfs combines a verbal projection with its subject, and clause 2 with its complements. The lexical entries for the verbs &amp;quot;loves&amp;quot; and &amp;quot;sleeps&amp;quot; are specified in clauses 3 and 4, respectively. Finally, clause 5 defines lexical entries for the proper names &amp;quot;Arthur&amp;quot; and &amp;quot;Tintagel&amp;quot;.</Paragraph>
    <Paragraph position="20"> Now consider the principles defined in Fig. 8.</Paragraph>
    <Paragraph position="21"> Constraints 1-4 encode a simple version of X-bar theory, constraint 5 ensures the propagation of categorial information along the head path, and constraint 6 ensures that complements obey the subcategorization requirements of the heads.</Paragraph>
    <Paragraph position="22"> We may now ask the system for a solution to queries like wfs(\[arthur, sleeps\], D). The solution in this case is the AVM in Fig. 9.</Paragraph>
    <Paragraph position="23"> We can also query for a term like word A subcat : he-list and check it against the implications alone, as it contains no relational goals. The result in Fig. 10 shows that our X-bar principles have applied: bar level two requires that the subcat list must be empty, and bar level one can only appear on phrases. The system thus correctly infers that only bar level zero is possible.  phrase A head : verb A subeat : \[\] A bar : two A comp-dtr : wfs(PO, P1) A head-rift : wfs( P1, P) 2. wfs(PO, P) := phrase A head : verb A subcat : ne-list A head-dtr : wfs(PO, P1) A eomp-dtr : wfs(P1, P) 3. wfs(\[XlY\],Y) := word A head : verb A bar : zero A subcat : \[head : noun, head : noun\] A phon : (loves A X) 4. wfs(\[X\[Y\],Y) := word A head : verb A bar : zero A subcat : \[head : noun\] h phon : (sleeps A X) 5. wfs(\[XIY\],Y) := word A head : noun A bar : two A subcat : \[\] A phon : ( ( arthur V tintageO A X)  1. bar: zero -+ word 2. bar: one ~ head-dtr : bar: (-~two) 3. bar:two -~ subcat : \[ \] 4. phrase -~ comp-dtr : bar : two 5. phrase --~ head : X A head-dtr : head : X 6. phrase --+ comp-dtr : X A subeat : Y A head-dtr : subcat : \[XIY \]  The advantages of such a modular encoding of grammatical principles are obvious. The intuition behind the constraints is clear, and new rules are easily added, since the principles apply to any rule. On the other hand, one can experiment with individual principles without having to change the other principles or rules. Finally, the option of encoding grammatical constraints as either implicational constraints or relations opens the possibility to chose the encoding most naturally suited to the specific problem. We feel that this improves on earlier, purely definite-clause-based approaches.  Compilation Building on the compilation method described in GStz and Meurers (1995), our compiler  collects the types for which principles are formulated, defines a relational encoding of the principles, and attaches calls to the relations at the places in the grammar where a constrained type can occur. We assume that the grammar writer guarantees that each type in the grammar is consistent (for a grammar G and every type t there is a model of G that satisfies t). One therefore does not need to attach calls to each possible occurrence of a constrained type, but only to those occurrences where the grammar contains additional specifications which might lead to an inconsistency (GStz and Meurers 1996). The interpretation of the resulting program is lazy in the sense that we do not enumerate fully specific solutions but compute more general answers for which a grammatical instantiation is guaranteed to exist. A good example for this behaviour was shown in Fig. 10: the system does not instantiate the PHON and the HEAD values of the solution, since the existence of grammatical values for these attributes is independent of the query.</Paragraph>
    <Paragraph position="24"> A way in which we deviate from the compilation method of GStz and Meurers (1995) is that our system performs all constraint inheritance at compile-time. While inheriting all principles to the most specific types and transforming the resulting constraints to a disjunctive normal form can significantly slow down compile times, the advantage is that no inheritance needs to be done on-line. To influence this trade-off, the user can instruct the system to hide a disjunctive principle in an auxiliary relation in order to keep it from being multiplied out with the other constraints. Such auxiliary relations, which will be discussed further in connection with the delay mechanism, have turned out to be especially useful in conjunction with principles with complex antecedents. The reason is that our compiler transforms an implication with complex antecedents to an implication with a type antecedent. The negation of the complex antecedent is added to the consequent, which can result in highly disjunctive specifications. Interpretation As a guiding principle, the interpreter follows the ideas of the Andorra Model 6 in that it always executes deterministic goals before non-deterministic ones. We consider determinacy only with respect to head unification: a goal is recognised to be determinate if there is at most one clause head that unifies with it. This evaluation strategy has two advantages: it reduces the number of choice points to a minimum, and it leads to early failure detection. In our implementation, the overhead of determining which goals are determinate has turned out to be by far outweighed by the reduction in search space for our linguistic applications. An additional speed-up can be expected from applying known pre-processing techniques (Santos Costa, Warren, and Yang 1991) to automatically extract so-called determinacy code.</Paragraph>
    <Paragraph position="25"> The execution order of non-determinate goals can be influenced by the user with wait declarations (Naish 1985). The execution of some goal is postponed until the call is more specific than a user-specified term. Speculative computation may thus be reduced to a necessary minimum. For our previous example, we might define the delay statements in Fig. 11. The first state-</Paragraph>
    <Paragraph position="27"> ment says that calls to wfs must be delayed until the first argument is instantiated to some list value. Similarly, the second statement delays the principles on phrase until the subcat information is known. , The third statement is of a slightly different form, based on the preferred treatment of determinate goals described above. Instead of specifying the instantiation state required for execution, the delay_deterministic statement 6Cf. Haxidi and Janson (1990) and references cited therein.</Paragraph>
    <Paragraph position="28">  specifies that the universal principles about signs can only be executed in case they are determinate.</Paragraph>
    <Paragraph position="29"> The delay mechanism for relational goals is very close to the one used in CUF. We extended this mechanism to the universal principles: the constraints on a certain type were only checked, once certain attributes were sufficiently instantiated (w.r.t. the delay statement). Our experience has shown, however, that delaying universal principles in such a way turns out to be too weak. Instead of delaying all constraints on a type until some condition is met, one wants to be able to postpone the application of some particular universal principle. A subcategorization principle applying to phrases, for example, should be delayed until the valence requirements of the mother or the daughters are known. We therefore allow the user to name a principle and supply it with a specific delay. Internally, this corresponds to introducing an auxiliary relation under the name supplied by the user and delaying it accordingly so that the choice points introduced by the principle are hidden.</Paragraph>
    <Paragraph position="30"> Let us illustrate the problem and its solution with a schematic example. Suppose the grammar writer writes a principle C/ --4 C/. Our compiler will generate from this a constraint t --~ (-~C/) V (C/ A C/), for some appropriate type t. If C/ is a complex conjunctive description, then the result of normaiising -~C/ might be highly disjunctive. This has two undesirable consequences. Firstly, if there is another constraint t --4 ~ with disjunctive ~, then the compiler will need to normalise the expression ((-~C/)V(C/AC/))A~. This is the appropriate thing to do in those cases where many of the generated disjuncts are inconsistent and the resulting disjunction thus turns out to be small. If, however, these constraints talk about different parts of t's structure, then the resulting disjunction will be big and the expansion at compile-time should be avoided.</Paragraph>
    <Paragraph position="31"> The other problem is that we can only specify delays on all constraints on t at once, and cannot delay individual principles. In other words, the control for the execution of principles is not fine-grained enough.</Paragraph>
    <Paragraph position="32"> We solved these problems by offering the user the possibility to name constraints, e.g., principle1 : C/ --4 C/. This prohibits the compile-time cross-multiplication described above, and it allows the user to specify delays for such a principle, e.g. delay(principlel .... ) or even delay_deterministic (principlel), if that is appropriate.</Paragraph>
    <Paragraph position="33"> Debugging Having addressed the key issues behind compilation and interpretation, we now turn to a practical problem which quickly arises once one tries to implement larger grammars. On the one hand, the complex data structures of such grammars contain an overwhelming number of specifications which are difficult to present to the user. On the other hand, the interaction of universal principles and relations tends to get very complex for realistic linguistic theories. While a powerful graphical user interface 7 solves the presentation problem, a sophisticated tracing and debugging tool was developed to allow stepwise inspection of the complex constraint resolution process. The debugger displays the feature structure(s) to be checked for grammaticality and marks the nodes on which constraints still have to be checked. As a result of the determinacy check, each such node can also be marked as failed, delayed or deterministic. Similar to standard Prolog debuggers, the user can step, skip, or fail a constraint on a node, or request all deterministic processing to be undertaken. An interesting additional possibility for non-deterministic goals is that the user can inspect the matching defining clauses and chose which one the system should try. Figure 12 below shows a screen shot of the debugger.</Paragraph>
    <Paragraph position="34"> The debugger has turned out to be an indispensable tool for grammar development. As grammar size increases, it becomes very difficult to track down bugs or termination problems without it, since these problems are often the result of some global interaction and thus cannot be reduced to a manageable sub-part of the grammar.</Paragraph>
    <Paragraph position="35"> The reader interested in further practical aspects of our system is referred to (GStz and Meurers 1997) Comparison with previous work There are quite a number of typed feature systems available today, among them ALE (Carpenter and Penn 1994), CUF (DSrre and Dorna 1993) and TFS (Emele and Zajac 1990; Emele 1994).</Paragraph>
    <Paragraph position="36"> TFS also offered type constraints and relations and to our knowledge was the first working typed feature systems. However, it had some serious drawbacks. TFS did not allow universal principles with complex antecedents, but only type constraints. And the system did not include a delay mechanism, so that it was often impossible to ensure termination or efficient processing. The addition of a delay mechanism as described in this paper would certainly increase the efficiency of TFS.</Paragraph>
    <Paragraph position="37"> ALE provides relations and type constraints (i.e., only types as antecedents), but their unfolding is neither lazy, nor can it be controlled by the user in any way.</Paragraph>
    <Paragraph position="38"> 7To view grammars and computations our system uses a GUI which allows the user to interactively view (parts of) AVMS, compare and search AVM8, etc. The ouI comes with a clean backend interface and has already been used as front-end for other natural language applications, e.g., in VERB-MOBIL. The GUI was developed by Carsten Hess.</Paragraph>
    <Paragraph position="39">  This can lead to severe termination problems with recursive constraints. The ALE type constraints were designed to enhance the typing system, and not for recursive computation. This should be done in the phrase structure or procedural attachment part. However, we believe that the addition of delaying and an interpretation strategy as described in this paper would add to the attractiveness of ALE as a constraint-based grammar development platform.</Paragraph>
    <Paragraph position="40"> The definite clause part of our system is very similar to the one of CUF: both use delay statements and preferred execution of deterministic goals. Although CUF does not offer universal principles, their addition should be relatively simple. Given that CUF already offers the control strategies required by our scheme, the changes to the run-time system would be minimal.</Paragraph>
    <Paragraph position="41"> Conclusion and future research We have presented an architecture that integrates relational and implicational constraints over typed feature logic. We showed how such an architecture facilitates the modular and compact encoding of principle based grammars.</Paragraph>
    <Paragraph position="42"> Our implementation has been tested with several smaller and one large (&gt; 5000 lines) grammar, a linearisation-based grammar of a sizeable fragment of German (Hinrichs et al. 1997). As the grammar constraints combine sub-strings in a non-concatenative fashion, we use a preprocessor that &amp;quot;chunks&amp;quot; the input string into linearisation domains, which are then fed to the constraint solver. With our Prolog based interpreter, parse times axe around 1-5 sec. for 5 word sentences and 10-60 sec. for 12 word sentences. It should be pointed out that parsing with such a grammar would be difficult with any system, as it does neither have nor allow the addition of a context-free backbone.</Paragraph>
    <Paragraph position="43"> We are currently experimenting with a C based compiler (Zahnert 1997) using an abstract machine with a specialised set of instructions based on the WAM (Warren 1983; A~-Kaci 1991). This compiler is still under development, but it is reasonable to expect speed improvements of at least an order of magnitude. Abstractmachine-based compilation of typed feature logic languages has recently received much attention (Carpenter and Qu 1995, Wintner 1997, Penn in prep.). True compilation is the logical development in a maturing field that has hitherto relied on interpreters in high-level programming languages such as Prolog and Lisp.</Paragraph>
    <Paragraph position="44"> We also plan to investigate a specialised constraint language for linearisation grammars, to be able to optiraise the processing of freer word order languages such as German.</Paragraph>
  </Section>
class="xml-element"></Paper>