<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0609">
  <Title>Towards Ontology-Based Natural Language Processing</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Ontological Reasoning for FOCAL
</SectionTitle>
    <Paragraph position="0"> The main task of the KR work package within the FOCAL project is to provide the FOCAL users with automated knowledge management and with automated reasoning capabilities about a complex domain. Ontologies have been chosen as the type of representation most suited for this task, and the provision of ontological reasoning capabilities has been one of the main thrusts. An ontology for FOCAL has been built and a number of reasoning activities are now ontology-based.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 Conceptualisation
</SectionTitle>
      <Paragraph position="0"> Lambert (2001) advocated Dennett's Intentional Stance framework (Dennett, 1987). Dennett  identified three means by which people predict and explain outcomes.</Paragraph>
      <Paragraph position="1"> 1. The first is the Physical Stance, where one engages principles of Physics to predict outcomes. People employ this when playing snooker or assessing the trajectories of projectile weapons.</Paragraph>
      <Paragraph position="2"> 2. The second is the Design Stance, where one  engages principles of design to predict and explain outcomes. People employ this when troubleshooting an automobile fault or coding and maintaining computer programs.</Paragraph>
      <Paragraph position="3"> 3. The third is the Intentional Stance, where one engages principles of rationality to predict outcomes. People employ this when forecasting the actions of a fighter pilot or when competing with an advanced computer game.</Paragraph>
      <Paragraph position="4"> The Design Stance is used whenever the physics of the situation is too difficult or laborious. The Intentional Stance is used whenever the design underpinning the situation is too difficult or laborious.</Paragraph>
      <Paragraph position="5"> Lambert (2001, 2003) adopts Dennett's framework for representing knowledge about the world, but adds two other layers: a metaphysical layer below the physical layer, and a social layer above the intentional layer. Therefore, formal theories that allow one to represent and reason about the world, would be assigned to the following levels:  individuals.</Paragraph>
      <Paragraph position="6"> This five level framework proposed by Lambert suggests a way to conceptualise the domain in terms of processes, namely metaphysical, physical, functional, intentional and social processes (M, P, F, I, S processes). The resulting conceptualisation is referred to as a Mephisto conceptualisation (Nowak, 2003) and is the basis for the ontologies we are constructing for FOCAL.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Ontological languages
</SectionTitle>
      <Paragraph position="0"> Ontologies are concerned with the meaning of terms. It is therefore appropriate when selecting an ontological language to choose a language which is equipped with a formal semantics. This requirement excludes XML from the list of possible candidates, as XML does not offer semantics, but only syntax. RDF provides some semantics, but proper, formal semantics requires languages based on logics. Description logics (DL) provide some frameworks, and several languages used for building and processing ontologies are DL-based, e.g. DAML and DAML+OIL languages, including such languages as SHF and SHIQ, and the OWL language (Horrocks et al., 2003).</Paragraph>
      <Paragraph position="1"> A commonly used view of an architecture for the Semantic Web is a layered architecture, with XML as the bottom layer, RDF as the middle layer, and logic (e.g. DL) as the top layer (sometimes the top layer distinguishes ontological vocabulary, logic, proof; on top of the logic layer a trust layer is sometimes placed). The logic layer is a necessary component if the Semantic Web is to be equipped with a formal semantics; this logic layer can be based on a description logic (such as SHIQ or OWL), on first-order logics, KIF or CycL, and whichever logic is used determines the expressibility and tractability of the framework, but in every case a formal semantics is added.</Paragraph>
      <Paragraph position="2"> Frameworks based on DL (description logics) are most successful, because they provide expressive languages with practical tractability. SHIQ is one such language, another is the closely related language OWL The ontological language chosen for FOCAL is SHIQ, a DL language of the DAML+OIL project (http://www.daml.org/), a successor of the OIL project (http://www.ontoknowledge,org/oil/).</Paragraph>
      <Paragraph position="4"> is a reasoner for the SHIQ logic employed in the OilEd ontology editor (http://oiled.man.ac.uk/).</Paragraph>
      <Paragraph position="5"> The logic SHIQ has also been implemented in the (www.cs.concordia.ca/~faculty/haarslev/racer/) RACER project.</Paragraph>
      <Paragraph position="6"> SHIQ is closely related to OWL (Horrocks et al., 2003). In fact, there are a few variants of OWL, namely OWL Lite, OWL DL and OWL Full. OWL Lite is similar to a description logic SHIF(D), while OWL DL is similar to a description logic SHOIN(D). The language implemented in the RACER framework is a version of SHIQ, which provides some functionalities for dealing with individuals, and dealing with concrete domains; this makes the RACER's version of SHIQ very close to OWL Lite. A proper discussion on these languages is beyond the scope of the paper, but clearly the RACER language is an implemented language and reasoner for a logic very close to OWL DL.</Paragraph>
      <Paragraph position="7"> References related to OWL, SHIQ and OIL include (Horrocks et al., 2003), (Bechhofer and Horrocks, 2003) and (Horrocks, Sattler and Tobies, 2000).</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.3 Ontological frameworks
</SectionTitle>
      <Paragraph position="0"> Ontology frameworks provide formalisms for building ontologies, but do not provide the contents. Therefore, they should do at least two things: * provide a formal language in which the ontologies can be expressed or specified, and * provide some reasoning capabilities, so that an ontology can be demonstrated to be consistent (i.e. free of contradictions, assuming that contradictions indicate modelling mistakes or errors).</Paragraph>
      <Paragraph position="1"> Given this standpoint, frameworks that do not provide reasoning capabilities are unsatisfactory. Note also that a formal language is usually a logical language, with clearly specified syntax and semantics, and the logic should be sound, complete, decidable, and hopefully tractable (or tractable in practice). These properties of the logical framework are necessary to obtain reasoning facilities. The most attractive ontology frameworks seem to be the following (see Table 1 for a more detailed comparison of the different frameworks):  1. the OIL framework based on description logics, 2. the OntoEdit/OntoBroker framework (F-logic), 3. the Ontolingua framework based on the KIF  logic.</Paragraph>
      <Paragraph position="2"> For FOCAL, we have chosen to employ the OIL and RACER frameworks. Ontologies are built using the OilEd ontology editor and verified using FaCT. At run-time, a RACER agent is initialised with the ontology (see section 3.4).</Paragraph>
      <Paragraph position="3"> Higher order relations and Description Logic Although description logics on which OIL and RACER are based allow only binary relations, we use OIL and Racer in a way that also allows us to employ arbitrary n-ary relations and higher-order relations. Given that a ternary relation can be represented as a binary relation that takes another binary relation as one of its argument, any n-ary relations can be represented via higher-order relations, i.e. relations which take other relations as arguments. Suppose that we want to implement a second-order relation that takes as its first argument a binary relation- more precisely, the second order relation takes as its first argument instances of that binary relation- rather than instances of a concept. The instances of the binary relation can be mapped to instances of a newly created concept, i.e. the concept of individuals which are single entities but correspond to (and are linked to) the instances of the binary relation. There is an exact correspondence between the second-order relation taking a binary relation instance as its first argument and its implementation in terms of a binary relation that takes as its first argument an instance of the concept which has instances of the other binary relation as its individuals. The approach we described here has now been used to implement in the FOCAL ontology information which extends beyond the binary relation based language.</Paragraph>
      <Paragraph position="4"> Multiple facts involving n-ary relation and higher-order relation are present in the current version of the FOCAL ontology. ATTITUDE agents are currently being built to allow automated reasoning with this extended language.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
Implemented Ontology
</SectionTitle>
      <Paragraph position="0"> As mentioned in section 1 the FOCAL scenario, which is based on real material for training exercise, provides background information in a number of domains, including geography, political situation, logistics, weather. For now, the scenario also specifies what kinds of questions can be asked by FOCAL users, to be answered by the FOCAL agents. The ontology serves as a formal, clearly specified knowledge base containing the background information and allowing the agents to query that knowledge base and to get replies helping them to answer the queries.</Paragraph>
      <Paragraph position="1"> An initial version of the FOCAL ontology has been created manually using OilEd and verified using FaCT.1 There are in fact several ontologies, for the different domains covered in the scenario, and an important research issue is that of the combining (or merging) of the ontologies in the larger FOCAL one. Another issue is that the manual creation of the ontologies is a time consuming and tedious process, but the existence of tools such as FaCT ensures that the result is consistent and free of mistakes due to user input errors.</Paragraph>
    </Section>
    <Section position="5" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.4 Ontological reasoning
</SectionTitle>
      <Paragraph position="0"/>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
1 The FOCAL ontology currently contains over 300
</SectionTitle>
    <Paragraph position="0"> concepts, about 80 relations and over 100 individuals (plus a large number of facts connecting all of these).</Paragraph>
    <Paragraph position="1"> Both the FaCT and RACER reasoning agents provide reasoning facilities, FaCT during the building of the ontologies to ensure coherence and consistency, and RACER at run-time. When integrated within the FOCAL system, the RACER server can be initialised with a given ontology and there is a RACER client wrapped as a CoABS agent on the grid, which can connect to the server.</Paragraph>
    <Paragraph position="2"> Other FOCAL agents, e.g. the Dialogue Manager (see section 4.1), can then communicate with the RACER server (via the RACER client agent) and receive answers using the ontology.</Paragraph>
    <Paragraph position="3"> The ontology can be also be accessed and queried outside of the FOCAL system, still using a client-server connection.</Paragraph>
    <Paragraph position="4"> * Using OilEd, the ontology &amp;quot;focal.daml&amp;quot; can be saved in the DIG format as a file named  can be used to connect to the RACER server.</Paragraph>
    <Paragraph position="5"> At the &amp;quot;&gt;&amp;quot; prompt, queries can be entered. The queries are received and replied to by the server. For instance, we show in (1) an example of a query as to whether (the individual) AUSTRALIA is an instance of (the concept) nation, and give the server's answer to that query, i.e. T (for true).</Paragraph>
    <Paragraph position="7"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.5 Hierarchies of concepts and relations
</SectionTitle>
      <Paragraph position="0"> A DL-based ontology, such as our OilEd &amp;quot;Focal&amp;quot; ontology, is a knowledge base (KB) expressed in a DL language. Every DL language provides facilities for defining concepts, with the relation of subsumption between the concepts being the core relation and the basis for building the definitions.</Paragraph>
      <Paragraph position="1"> The set of concepts can be seen as an ordered set, the subsumption relation being the ordering relation; hence, we have a hierarchy of concepts.</Paragraph>
      <Paragraph position="2"> There is also a hierarchy of relations ordered by the subsumption relation. These two hierarchies, together with the concepts' definitions, can be taken to form a lexicon, i.e. a list of words (for 2 OilEd can export to SHIQ, OWL and other formats.</Paragraph>
      <Paragraph position="3"> concepts and relations) with well-defined meanings for those words.</Paragraph>
      <Paragraph position="4"> These two hierarchies of concepts and relations thus provide a basis for a domain specific lexicon and one of the advantages which ontologies can offer NLP systems is that a properly built knowledge base (as on ontology) will allow the semi-automatic creation of a lexicon.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 NLP in FOCAL
</SectionTitle>
    <Paragraph position="0"> The underlying architecture for dialogue management has been developed using ATTITUDE agents (Estival et al., 2003). Input from FOCAL users can be either spoken or typed and is processed by the same NLP component. We use Nuance for speaker-independent speech recognition (Nuance, 2002) and the open source Regulus NLP package (Rayner et al., 2001) for grammar development.3 We are in the process of integrating language input with input from other devices, e.g. pointing devices such as mouse or wand, gesture tracking device and, in the future, gaze tracking.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Dialogue Agents
</SectionTitle>
      <Paragraph position="0"> wrapper around a Nuance Client implementation.</Paragraph>
      <Paragraph position="1"> It forwards on to the Input Fuser the interpretations of speech recognition results (in the form of lists of Attribute-Value pairs), notifications of failed recognition events and the interpretations of typed input. It also passes on instructions to activate and de-activate the recogniser.</Paragraph>
    </Section>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
* Input Fuser
</SectionTitle>
    <Paragraph position="0"> The Input Fuser (IF) is responsible for receiving and combining user input. This input can be via speech (Nuance), keyboard (typed input), gesture, gaze etc. The IF turns streams of input events into a Bayesian network of discrete communicative acts</Paragraph>
  </Section>
  <Section position="8" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 The existing grammar was developed using Regulus 1,
</SectionTitle>
    <Paragraph position="0"> but we are currently developing a larger, more flexible grammar with Regulus 2 (Rayner et al., 2002) which will provide a broader coverage, allowing the more naive users to be recognised more easily.</Paragraph>
    <Paragraph position="1"> which are then interpreted by the Dialogue Manager.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
The Internal Reasoning Agents comprise:
* Reference Resolver
</SectionTitle>
      <Paragraph position="0"> This is currently a stub, but the Reference Resolver is meant to assist other agents (particularly the Input Fuser and the Dialogue Manager) resolve anaphoric references found in user communicative acts by maintaining context and linking dialogue variables to referents.</Paragraph>
    </Section>
  </Section>
  <Section position="9" start_page="0" end_page="0" type="metho">
    <SectionTitle>
* Dialogue Manager
</SectionTitle>
    <Paragraph position="0"> The Dialogue Manager (DM) is activated by a message that includes an activation context symbol. The DM receives the Bayesian network of interpretations of user(s) communicative acts from the IF and it finds the interpretation with the highest probability that unifies with the current dialogue context. The DM then informs the IF of which interpretation of the communicative act was chosen, so the IF can forward the full information on to the Transcriber. At the same time, the DM requests that the Presentation Planner present the response to this communicative act; this request is termed a communicative goal.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
* Presentation Planner
</SectionTitle>
      <Paragraph position="0"> The Presentation Planner (PP) receives requests from the DM to achieve communicative goals. For now a communicative goal will succeed if there is a presentation clip which is marked-up with the conjunction of the DM's activation context and the meaning representation for the query, but current work is extending the PP agent along the lines given in (Colineau and Paris, 2003).</Paragraph>
      <Paragraph position="1"> The Output Agents comprise:</Paragraph>
    </Section>
  </Section>
  <Section position="10" start_page="0" end_page="0" type="metho">
    <SectionTitle>
* Transcriber
</SectionTitle>
    <Paragraph position="0"> The Transcriber agent receives notification of user's communicative acts from IF and of the system's communicative acts from DM. It produces an HTML listing of these communicative acts, which includes speech recognition results and a link pointing to the audio recording.</Paragraph>
    <Paragraph position="1"> * Text-to-Speech If the output is to be presented verbally by the Virtual Advisers, it is sent to the Text-to-Speech (TTS) component. We use the rVoice TTS system, which gives us a choice of voices for the different VAs (rVoice, 2002).</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Lexicon for NLP
</SectionTitle>
      <Paragraph position="0"> As described above, language processing is performed by the Nuance/Regulus grammar.</Paragraph>
      <Paragraph position="1"> Regulus is an Open Source environment which compiles typed unification grammars into context-free grammar language models compatible with the Nuance Toolkit.4 The lexicon for Regulus 2 is of the form shown in (2) and (3), where the macro in (2) defines the properties of a noun class, and the instances in (3) specify the lexical items belonging to that class, in this case result, results, outcome, outcomes.</Paragraph>
      <Paragraph position="2"> (2) macro defining noun class macro(noun_like_result(Words,Sem), @noun(Words, [sem= @noun_sem(abstract, Sem), sem_n_type=abstract, takes_det_type=def\/null, n_of_mod_type=_])).</Paragraph>
      <Paragraph position="3"> (3) examples of nouns for that class: @noun_like_result([result, results], result).</Paragraph>
      <Paragraph position="4"> @noun_like_result([outcome, outcomes], result).</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.3 Meaning representation
</SectionTitle>
      <Paragraph position="0"> The Meaning Representation produced by the NLP component, and passed on by the Speech Input agent, is translated into an ATTITUDE expression.</Paragraph>
      <Paragraph position="1"> For example, if a user can ask the question given in (4.a), it will first be translated into the  (simplified) list of attribute value pairs given in (4.b) and sent to the Speech Input agent. Speech Input then translates these attribute value pairs into the (simplified) ATTITUDE expression given in (4.c) and forwards it on to the Input Fuser agent. (4) a. What is our relationship with PNG? b. (question whquestion concept relationship obj1</Paragraph>
    </Section>
  </Section>
  <Section position="11" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Natural Language &amp; Ontological Processing
</SectionTitle>
    <Paragraph position="0"> for FOCAL There are at least two ways that ontologies can facilitate language processing. Firstly, an ontology can be used directly when building the lexicon, defining the terms (concepts and relations) for content words. Secondly, an ontology is a knowledge base (KB), expressed in a formal language, and therefore it provides (formally 4 Regulus is described in detail in (Rayner et al., 2001). expressed) knowledge for more complex language processing.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.1 Ontology and the lexicon
</SectionTitle>
      <Paragraph position="0"> We view an ontology as a knowledge base, consisting of a structured list of concepts, relations and individuals. The ontology provides partial definitions for these, through the taxonomy relation between the terms and the properties specified for them. An example of how a fragment of a lexicon, for the content words in the domain, can be obtained from an ontology is presented below.</Paragraph>
      <Paragraph position="1"> We give in (6) an ontology fragment, where every concept is listed in the format shown in (5).  Simplified lexical entries for the words aircraft, airplane, airplanes, plane, planes, ship, ships, frigate, frigates and FFG are shown in (7) and (8). (7) macro for noun class &amp;quot;platform&amp;quot;: macro(noun_like_platform(Words,Sem), @noun(Words, [sem= @noun_sem(platform, Sem), sem_n_type=platform, takes_det_type=def\/null, n_of_mod_type=_])).</Paragraph>
      <Paragraph position="2"> (8) examples of nouns for class &amp;quot;platform&amp;quot;: @noun_like_platform([frigate, frigates], ship). @noun_like_platform([ffg], ship).</Paragraph>
      <Paragraph position="3"> @noun_like_platform([ship, ships], ship).</Paragraph>
      <Paragraph position="4"> @noun_like_platform([airplane,airplanes,plane,planes], aircraft).</Paragraph>
      <Paragraph position="5"> @noun_like_platform([aircraft], aircraft).</Paragraph>
      <Paragraph position="6"> This example shows how synonyms are handled in our system, with the same semantic interpretation, and the same parent class, given to a number of lexical items.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
5.2 Ontology as knowledge
</SectionTitle>
      <Paragraph position="0"> Since an ontology is a knowledge base expressed in a formal language, it provides formally expressed knowledge for language processing.</Paragraph>
      <Paragraph position="1"> Although at this point not all this knowledge can be used directly by the speech recognition system which processes the speech input, nor by the grammar which builds the meaning representations, some of this knowledge can already be used by the other Dialogue agents, in particular the Dialogue Manager, and later by the Reference Resolver.</Paragraph>
      <Paragraph position="2"> The best example is the resolution of ambiguity, such as the polysemy of some terms. For instance the name Adelaide can refer to a city (Adelaide in South Australia), a ship (&amp;quot;HMAS Adelaide&amp;quot;), a river (the Adelaide River in the Northern Territory of Australia), or even a person, (e.g. &amp;quot;Queen Adelaide&amp;quot;). While, as shown in Section 5.1, synonymy is handled by the lexicon, polysemy is resolved by drawing on a variety of sources, including the ontology.</Paragraph>
      <Paragraph position="3"> When the Dialogue Manager receives from the Input Fuser a set of communicative acts, if one of these communicative acts correspond to distinct plausible interpretation results, e.g.</Paragraph>
      <Paragraph position="4"> &amp;quot;Adelaide:{city, ship}&amp;quot;, it can try to resolve the ambiguity by using the context information and by sending a request to the KR agent.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>