<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-0739">
  <Title>Learning from a Substructural Perspective</Title>
  <Section position="4" start_page="176" end_page="179" type="metho">
    <SectionTitle>
2 Substructural logic
</SectionTitle>
    <Paragraph position="0"> In Gentzen style sequential formalisms a sub-structural logic shows itself by the absence of (some of) the so-called structural rules. Examples of such logics are relevance logic (Dunn, 1986), linear logic (Girard, 1987) and BCK logic (Grishin, 1974). Notable is the substructural behavior of categorial logic, which in its prototype form is the Lambek calculus. Categorial logics are motivated by its use as grammar for natural languages. The absence of the structural rules degrades the abstraction of sets in the semantic domain to strings, where elements in a string have position and arity, while they do not have that in a set. As we will see further on in this paper the elimination of the structural rules in the learning context of the boolean concepts will transform the learning framework from sets of valuated variables to strings of valuated variables.</Paragraph>
    <Paragraph position="1"> Example 2.1 In a domain of sets the following 'expressions' are equivalent, while they are not in the domain of strings: a, a, b, a ~ a, b, b In a calculus with all the structural rules the features 'position' and 'arity' are irrelevant in the semantic domain, because aggregates that differ in these features can be proved equivalent with the structural rules. To see this observe that the left side of the above equation can be transformed to the right side by performing the following operation: a, a, b, a a, b, a a, a, b a, b a, b, b contract a, a in .first two positions to a exchange b, a in last to positions to a,b contract again a, a in first two positions to a weaken expression b in last position to b, b In figure 2 we list the axiomatics of the first order propositional sequent calculus 1, with the axioms , the cut rule, rules for the connectives and the structural rules for exchange, weakening and contraction.</Paragraph>
    <Paragraph position="2">  Consider the sample space for boolean concepts. An example is a vector denoting the truth (presence,l) or falsehood (absence,0) of propositional variables. Such an example vector can be described by a formula consisting of the conjunction of all propositional variables or negations of propositional variables, depending on the fact whether there is a 1 or a 0 in the position of the propositional variable name in the vector. A collection of vectors, i.e. a concept, in its turn can be denoted by a formula too, being the disjunction of all the formula's of the vectors.</Paragraph>
    <Paragraph position="3">  A little more extensive: Let universe \[.j, = {a,b,c} and let concept</Paragraph>
    <Paragraph position="5"> Then the following formula exactly describes fl (with a clear translation): (~AbAa) V (~ AbA c) V (~A bA c) V (aAbAc) Note that these formulas are in Disjunctive normal form (DNF).</Paragraph>
    <Paragraph position="6"> An interesting observation now is that the learning algorithm of Valiant that learns k-CNF formulas actually is trying to prove the equivalence between a DNF formula and a k-CNF formula.</Paragraph>
    <Paragraph position="7"> Example 3.2 Let universe U = {a,b} and let concept f = {(0, 1)}, then the following sequent should be 'learned' by a 2-CNF learning algorithm 2: ~ A b ,C/:,. (aVb) A (~Vb) A (~Vb) A little more extensive: Let U' = {a, b, c} and let concept f' = {(0, 0, 0), (0, 0, 1), (0, 1, 1), (1, 1, 1)} Then the following sequent should be 'learned' by a</Paragraph>
    <Paragraph position="9"> The above observation says in logical terms that the learning algorithm needs to implement an inductive procedure to find this desired proof and the concluding concept description (2-CNF formula) from examples. In the search space for this proof the learning algorithm can use the axioms and rules from the representational theory.</Paragraph>
    <Paragraph position="10"> In the framework of boolean concept learning this means that the learning algorithm may use all the rules and axioms from the representational theory of classical propositional logic. Example 3.3 Let IJ = {a, b} and let concept f = {(0, 1)} and assume f can be represented by a 2-CNF formula, to learn the 2-CNF description of concept f the learning algorithm needs to find the proof for a sequent starting 2i.e. an algorithm that can learn 2-CNF boolean concepts. null from the DNF formula ~ A b to a 2-CNF formula and vice versa (C/~.) and to do so it may use all the rules and axioms from the first order propositional calculus including the structural rules. The proof for one side of such a sequent is spelled out in figure 3.</Paragraph>
    <Paragraph position="11"> In general an inductive logic programming algorithm for the underlying representational theory can do the job of learning the concept; i.e. from the examples (DNF formulas) one can induce possible sequents, targeting on a 2-CNF sequent on the righthand side. The learning algorithm we present here is more specific and simply shows that an efficient algorithm for the proof search exists.</Paragraph>
    <Paragraph position="12"> The steps:  1. Form the collection G of all 2-CNF clauses (p V q) 2. do l times (a) (b)  pick an example al A.-. Aam form the collection of all 2-CNF clauses deducible from al A ... A am and intersect this collection with G resulting in a new C Correctness proof (outline): By (Ax), (RV), (Weak), (LA) and (Ex) we can proof that for any conjunction (i.e. example vector) al A ... A am we have for all 1 _&lt; i &lt; m and any b a clause of a 2-CNF in which ai occurs with b, hence having all clauses deducible from the vector proven individually enabling one to form the collection of all clauses deducible from  a vector; i.e.</Paragraph>
    <Paragraph position="13"> al A ... Aam ~ ai Vb al A ... A am :::*&amp;quot; b V ai By (RA) and (Contr) we can proof the conjunc null tion of an arbitrary subset of all the clauses deducible from the vector, in particular all those clauses that happen to be common to all the vectors for each individual vector we have seen so far, hence proving the 2-CNF for every individual vector; i.e.</Paragraph>
    <Paragraph position="14"> al A .. * A am ~ clause1 A .. * A clausep  Now by (LV) we can prove the complete DNF to 2-CNF sequent; i.e.</Paragraph>
    <Paragraph position="15"> vector1 V * * * V vector/ ~ clause1 A * * * A clausep It is easy to see that for the above algorithm the same complexity analysis holds as for the Valiant algorithm, because we have the same progression in l steps, an the individual steps have constant overhead.</Paragraph>
  </Section>
  <Section position="5" start_page="179" end_page="179" type="metho">
    <SectionTitle>
4 PAC learning substructural logic
</SectionTitle>
    <Paragraph position="0"> When we transform the representational theory of the boolean concept learning framework to a substructural logic, we do the following: * eliminate the structural rules from the calculus of first order propositional logic When we want to translate the learnability result of k-CNF expressible boolean concepts we need to do the same with the formal learning framework and the strategy (algorithm). In other words: * the learning framework will contain concepts that are sensitive to the features which were before abstracted by the structural rules ('position' and 'arity' ) * the learning algorithm from above is no longer allowed to use the structural rules in its inductive steps.</Paragraph>
    <Paragraph position="1"> Below we present a learning algorithm for the substructural logic representational theory. Suppose again the universe U = {al,... ,an}, and the concept f is a CNF expressible concept for vectors of length m.</Paragraph>
    <Paragraph position="2">  1. start with m empty clauses (i.e. disjunction of zero literals) clause1,..., clausem 2. do l times (a) pick an example al A... A am (b) for all 1 &lt; i &lt; m add ai to clause/ if  ai does not occur in clause/.</Paragraph>
    <Paragraph position="3"> Correctness proof (outline): By (Ax) and (RV) we can proof for any ai that the sequent ai =-~ clause/for any clause/containing ai as one of its disjuncts, especially for a clause/containing next to ai all the a~ from the former examples. Then by (RA) and (LA) we can position all the vectors and clauses in the right-hand position; i.e.</Paragraph>
    <Paragraph position="4"> al A ... A am ~ clause1 A -.. A clausem Hence justifying the adding of the literal ai of a vector in clausei. Now (LV) completes the sequent for all the example vectors; i.e.</Paragraph>
    <Paragraph position="5"> (al A... A am) V (a i A ... A aim ) V . . . clause1 A .-. A clausem For the algorithmic complexity in terms of PAC learning, suppose we want present examples of concept f and that the algorithm learned concept ff in l steps. Concept ff then describes a subset of concept f because on every position in the CNF formula contains a sub-set of the allowed variables; i.e. those variables that have encountered in the examples 3. anote that the CNF formula's can only describe particular sets of n-strings; namely those sets that are complete for varying symbols locally on the different positions in the string.</Paragraph>
    <Paragraph position="6">  Now let e = P(fAf ~) be the error then again 5 = (1 - e) TM is the confidence parameter as we have m positions in the string. By the same argument as for the Valiant algorithm we may conclude that e and 5 decrease exponentially in the number of examples l, meaning that we have an efficient polynomial time learning algorithm for arbitrary e and 5.</Paragraph>
  </Section>
class="xml-element"></Paper>