Content uploaded by Osslan Vergara
Author content
All content in this area was uploaded by Osslan Vergara on Sep 13, 2016
Content may be subject to copyright.
Series Editors
Jaime G. Carbonell, Carnegie Mellon University, Pittsburgh, PA, USA
Jörg Siekmann, University of Saarland, Saarbrücken, Germany
Volume Editors
Alexander Gelbukh
National Polytechnic Institute (IPN)
Center for Computing Research (CIC)
Av. Juan Dios Bátiz, Col. Zacatenco
07738 DF, Mexico
E-mail: gelbukh@gelbukh.com
Álvaro de Albornoz
Technológico de Monterrey (ITESM)
Campus Ciudad de México (CCM)
Calle del Puente 222, Col. Ejudos de Huipulco
Tlalpan, 14360 DF, Mexico
E-mail: albornoz@itesm.mx
Hugo Terashima-Marín
Technológico de Monterrey (ITESM)
Campus Monterrey (MTY)
Eugenio Garza Sada 2501, Col. Technológico
64849, Monterrey, NL, Mexico
E-mail: terashima@itesm.mx
Library of Congress Control Number: 2005935947
CR Subject Classification (1998): I.2, F.1, I.4, F.4.1
ISSN 0302-9743
ISBN-10 3-540-29896-7 Springer Berlin Heidelberg NewYork
ISBN-13 978-3-540-29896-0 Springer Berlin Heidelberg NewYork
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting,
reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication
or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965,
in its current version, and permission for use must always be obtained from Springer.Violations are liable
to prosecution under the German Copyright Law.
Springer is a part of Springer Science+Business Media
springeronline.com
© Springer-Verlag Berlin Heidelberg 2005
Printed in Germany
Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India
Printed on acid-free paper SPIN: 11579427 06/3142 543210
Table of Contents
Knowledge Representation and Management
Modelling Human Intelligence: A Learning Mechanism
Enrique Carlos Segura, Robin Whitty ............................ 1
Compilation of Symbolic Knowledge and Integration with Numeric
Knowledge Using Hybrid Systems
Vianey Guadalupe Cruz S´anchez, Gerardo Reyes Salgado,
Osslan Osiris Vergara Villegas, Joaqu´ın Perez Ortega,
Azucena Montes Rend´on ....................................... 11
The Topological Effect of Improving Knowledge Acquisition
Bernhard Heinemann .......................................... 21
Belief Revision Revisited
Ewa Madali´nska-Bugaj, Witold Lukaszewicz ...................... 31
Knowledge and Reasoning Supported by Cognitive Maps
Alejandro Pe˜na, Humberto Sossa, Agustin Guti´errez ............... 41
Temporal Reasoning on Chronological Annotation
Tiphaine Accary-Barbier, Sylvie Calabretto ....................... 51
EventNet: Inferring Temporal Relations Between Commonsense Events
Jose Espinosa, Henry Lieberman ............................... 61
Multi Agent Ontology Mapping Framework in the AQUA Question
Answering System
Miklos Nagy, Maria Vargas-Vera, Enrico Motta ................... 70
A Three-Level Approach to Ontology Merging
Agustina Buccella, Alejandra Cechich, Nieves Brisaboa ............ 80
Domain and Competences Ontologies and Their Maintenance for an
Intelligent Dissemination of Documents
Yassine Gargouri, Bernard Lefebvre, Jean-Guy Meunier ........... 90
Modelling Power and Trust for Knowledge Distribution: An
Argumentative Approach
Carlos Iv´an Ches˜nevar, Ram´on F. Brena, Jos´eLuisAguirre........ 98
Compilation of Symbolic Knowledge and Integration
with Numeric Knowledge Using Hybrid Systems
Vianey Guadalupe Cruz Sánchez, Gerardo Reyes Salgado,
Osslan Osiris Vergara Villegas, Joaquín Perez Ortega, Azucena Montes Rendón
Centro Nacional de Investigación y Desarrollo Tecnológico (cenidet),
Computer Science Department, Av. Palmira S/n,
Col. Palmira. C. P. 62490. Cuernavaca Morelos México
{vianey, greyes, osslan, jperez, amr}@cenidet.edu.mx
Abstract. The development of Artificial Intelligence (AI) research has fol-
lowed mainly two directions: the use of symbolic and connectionist (artificial
neural networks) methods. These two approaches have been applied separately
in the solution of problems that require tasks of knowledge acquisition and
learning. We present the results of implementing a Neuro-Symbolic Hybrid
System (NSHS) that allows unifying these two types of knowledge representa-
tion. For this, we have developed a compiler or translator of symbolic rules
which takes as an input a group of rules of the type IF ... THEN..., converting
them into a connectionist representation. Obtained the compiled artificial neural
network this is used as an initial neural network in a learning process that will
allow the “refinement” of the knowledge. To prove the refinement of the hybrid
approach, we carried out a group of tests that show that it is possible to improve
in a connectionist way the symbolic knowledge.
1 Introduction
During the last years a series of works have been carried out which tend to diminish
the distance between the symbolic paradigms and connectionist: the neuro–symbolic
hybrid systems (NSHS). Wertmer [1] proposes a definition: “the NSHS are systems
based mainly on artificial neural network that allows a symbolic interpretation or an
interaction with symbolic components”. These systems make the transfer of the
knowledge represented by a group of symbolic rules toward a module connectionist.
This way, the obtained neural network allows a supervised learning starting from a
group of examples.
For their study, the symbolic knowledge (obtained from rules) has been treated as
the “theory” that we have on a problem and the numeric knowledge (obtained from
examples) as the “practice” around this problem. This way, the objective of imple-
menting a neuro-symbolic hybrid system is to combine " theory " as well as the "
practice " in the solution of a problem, because many of the times neither one source
of knowledge nor the other are enough to solve it.
From the works developed by Towell [2] and Osorio [3] we have implemented a
symbolic compiler that allows transforming a set of symbolic rules into an ANN. This
ANN would be “refined” later on thanks to a training process in a neural simulator
A. Gelbukh, A. de Albornoz, and H Terashima (Eds.): MICAI 2005, LNAI 3789, pp.11 – 20, 2005.
© Springer-Verlag Berlin Heidelberg 2005
12 V. G Cruz Sánchez et al.
using a base of examples. To prove the NSHS we use the benchmark Monk's Problem
[4], which has a base of examples as well as a base of rules.
2 Symbolic and Numeric Knowledge
The symbolic knowledge is the set of theoretical knowledge that we have in a particu-
lar domain. For example, we can recognize an object among others by means of the
set of characteristics of that object. This description can be considered a symbolic
representation. A disadvantage of this kind of representation is that the theory in
occasions cannot describe all the characteristics of the object. The above-mentioned
is due to the fact that it cannot make an exhaustive description of the object in all its
modalities or contexts.
For example, the description of a Golden apple says that “the fruit is big and of
golden colour, longer than wide, with white and yellowish meat, fixed, juicy, per-
fumed and very tasty. The peduncle is long or very long and the skin is thin” [5]. If a
person uses only the theory to be able to recognize this fruit in a supermarket, it is
possible that this person may have difficulty to recognize an apple that is blended or
next to another kind of fruits (for example, pears) or another kind of apples. These
fruits have a very similar symbolic description. Here, we realize that the symbolic
knowledge can be insufficient for a recognition task.
On the other hand, so that this knowledge can be used in a computer system, a for-
mal representation should be used. For this, we have different knowledge representa-
tions: propositional logic, predicates logic, semantic networks, etc. The representation
mostly used is the symbolic rules.
Another source of knowledge, is that called "practical", integrated by a group of
examples about an object or problem in different environments or contexts.
For the case of the Golden apple, we need to describe it presenting an image base
of the fruit in different environments, contexts, positions and with different degrees of
external quality. We will use for it a numeric description (colour in RGB, high, wide,
etc.). As it happened to the symbolic representation, it is impossible to create a base
of images sufficiently big so as to cover all the situations previously mentioned. For
the above-mentioned, a base of examples is also sometimes insufficient to describe all
and each one of the situations of an object. We believe that a hybrid approach can be
the solution to problems of objects recognition.
3 Characteristic of Neuro-Symbolic Hybrid Systems
A hybrid system is formed by the integration of several intelligent subsystems, where
each one maintains its own representation language and a different mechanism of
inferring solutions. The goal of the implementation of the hybrid systems is to im-
prove the efficiency and the reasoning power as well as the expression power of the
intelligent systems.
The hybrid systems have potential to solve some problems that are very difficult to
confront using only one reasoning method. Particularly, the neuro-symbolic hybrid
systems can treat numeric and symbolic information with more effectiveness than
Compilation Of Symbolic Knowledge and Integration with Numeric Knowledge 13
systems that act separately. Some of the advantages of the hybrid systems are: they
exploit the whole available knowledge of a problem; they mix different kinds of in-
formation (symbolic, numeric, inexact and imprecise); they improve the total execu-
tion and they eliminate the weaknesses of the methods applied individually creating
robust reasoning systems.
4 Compilation of Symbolic Knowledge into an ANN
The neuro-symbolic hybrid system that we proposed has a compiler in charge of
mapping the symbolic rules into a neural network. This compiler generates the topol-
ogy and the weights of an ANN (we will call this ANN "compiled"). The topology of
the neural network represents the set of rules that describes the problem and the
weights represent the dependences that exist between the antecedents and the conse-
quents of these rules. The compilation is carried out by means of a process that maps
the components of the rules towards the architecture of an ANN (see Table 1).
Table 1. Correspondence between the symbolic knowledge and the ANN
Symbolic knowledge ANN
Final conclusions Output units
Input data Input units
Intermediate conclusions Hide units
Dependences Weights and connections
4.1 Compilation Algorithm
For the compilation module we implemented the algorithm proposed by Towell [2]
which consists of the following steps:
1. Rewrite the rules.
2. Map the rules into an ANN.
3. Determine the level between the hidden and output units and the input units.
4. Add links between the units.
5. Map the attributes not used in the rules.
1. Rewrite the rules. The first step of the algorithm consists of translating the set of
rules to a format that clarifies their hierarchical structure and that makes it possible to
translate the rules directly to an artificial neural network.
In this step, we verify if a consequent is the same for one or more antecedents. If it
exists more than an antecedent to a consequent, then each consequent with more than
one antecedent it will be rewritten. For example, in Figure 1 the rewriting of two
rules, with the same consequent is observed.
14 V. G Cruz Sánchez et al.
Fig. 1. Rewrite two rules with the same consequent.
Where :
:- it means “IF . . . THEN”;
, it means the conjunction of antecedents.
In Figure 1, the rule B: - C, D has as antecedent C and D and as consequent B. It is
interpreted as: If C and D are true, Then B is true. The rule B: - E, F, G is read in the
following way: If E and F and D are true, Then B is true. Because these two rules
have the same consequent, these should be rewrite as to be able to be translated to the
ANN. The rules at the end of this step can be observed in the Figure 1.
2. Mapping the rules to an ANN. The following step of the algorithm consists of map-
ping a transformed group of rules to an artificial neural network.
If we map the rules of the Figure 1, these would be like it is shown in the Figure
2a. In the Figure 2.b the weight and the assigned bias are shown. The assigned weight
will be –w or w if the antecedent is denied or not, respectively. Towell proposes a
value of the weight of w = 4, [2]. On the other hand, it will be assigned to B' and B'' a
bias of (-2P-1/2)*w, since they are conjunctive rules, while we will assign a bias of –
w/2 to B because it is a disjunction.
Fig. 2. a) Mapping of rules to an ANN b) Weights and bias assigned.
3. Determine the level between the hidden and output units and the input units. The
level can be defined by means of some of the following ways: Minimum or maximum
length that exists among a hidden or output unit to an input unit (see Figure 3).
4. Add links among the units. In this step the links are added between the units that
represent the non existent attributes in the rules and the output units.
5. Mapping the attributes not used in the rules. In this stage an input is added to the
neural network by each one of the attributes of the antecedents not present in the
initial rules. These inputs will be necessary for the later learning stage.
Compilation Of Symbolic Knowledge and Integration with Numeric Knowledge 15
Fig. 3. a) Determination of the level using the minimum distance toward an input unit. b) De-
termination of the level using the maximum distance toward an input unit.
5 Implementation of the Symbolic Compiler
In our research we implement a symbolic compiler (we call this SymbComp). This
compiler uses the method proposed by Towell (see section 4.1). For the implementa-
tion of SymbComp we use the software AntLr Terence [6] that allows carrying out
the lexical, syntactic and semantic analysis of a file type text. This file will be the
input to SymbComp. The system is made of four sections (see program bellow):
• Head. It contains the variables and constant definition, as well as the name of the
file and comments.
• Attributes. It contains the definition of each one of the attributes that appear in the
antecedents and consequent of the rules. The attributes are made of Labels; Type;
Value. CompSymb implements four types of attributes:
a) Nominal: it has discrete values such as small, medium, big.
b) Continuous: it has values in a continuous space.
c) Range: it has continuous values with a maximum and minimum value.
d) Binary: it has two possible values, true / false.
• Rules. It contains the symbol
• ic rules that will be compiled and which are written in language Prolog.
• Attributes added by the user. It contains attributes that are not used in the antece-
dents of the previous rules but that are present in the base of examples.
SymbComp carries out the lexical, syntactic and semantic analysis and the compi-
lation (together with Builder C++ 5.0 Charte [7]) of the previously defined file. The
result of this process is two files: one of topology and another one of weight of an
artificial neural network that we will call "compiled network".
//section one
//Simbolic file example.symb
#BEGIN
//section two
16 V. G Cruz Sánchez et al.
#ATTRIBUTES
stalk: BINARY:T/F;
size: CONTINUOS:small[3,5], medium[6,8], big[9,11];
shape: NOMINAL: heart, circle;
#END_ ATTRIBUTES
//section three
#RULES
Red_Delicius:-shape(heart),
IN_RANGE(Colour_Red,108,184),
IN_RANGE(Colour_Green,100,150),
IN_RANGE(Colour_Blue,50,65);
Quality:- stalk(T),size(big),Red_Delicius.
#END_RULES
//section four
#ATTRIB_ADDED_USER
Colour_Red: RANGE: [0,255];
Colour_Green: RANGE: [0,255];
Colour_Blue: RANGE: [0,255];
#END
6 Integration of Symbolic and Numeric Knowledge
The integration of numeric and symbolic knowledge will be carried out once we have
compiled the rules. For it, we will use the compiled neural network and the base of
examples of the problem. For the integration the Neusim system Osorio[3] was used,
which uses the CasCor Fahlman [8] paradigm. Neusim was selected because it allows
loading a compiled neural network and from this to begin the learning. Also, with
Neusim it is possible to carry out the learning directly from the base of examples or to
use the neural network as classifier. On the other hand, the advantage of using
CasCor as a learning paradigm is that we can see the number of hidden units added
during the learning and thus, follow the process of incremental learning. We have
also planned for a near future to carry out a detailed analysis of the added units with
the purpose of explaining knowledge.
By means of this integration stage we seek that the numeric and symbolic knowl-
edge are integrated in a single system and at the end we have a much more complete
knowledge set.
7 Test using the NSHS for the Integration and the Refinement of
Knowledge
To prove the integration and refinement capacity of knowledge of the NSHS we made
use of examples and rules base known as Monk's Problem Thrun [4]. This benchmark
is made of a numeric base of examples and of a set of symbolic rules. Monk's prob-
lem is presented with three variants: Monk's 1, Monk's 2 and Monk's 3. For our ex-
perimentation we use the first case. The characteristics of Monk's 1 are as follows:
Compilation Of Symbolic Knowledge and Integration with Numeric Knowledge 17
• Numeric base of examples.
Input attributes: 6 (a1, a2, ... a6).
Output: 1 (class).
Total of examples of the base: 432.
Description of attributes:
a1: 1, 2, 3
a2: 1, 2, 3
a3: 1, 2
a4: 1, 2, 3
a5: 1, 2, 3, 4
a6: 1, 2.
class: 0, 1
• Symbolic Rules.
IF (a1 = a2) OR (a5 = 1), THEN class = 1.
This rule can be translated into two disjunctive rules:
R1 : IF (a1 = a2),THEN class = 1;
R2 : IF (a5 = 1), THEN class = 1;
As it can be observed in these two rules, the used attributes are a1, a2 and a5,
while the attributes a3, a4 and a6 are not considered in these rules. This observation is
important because in the base of examples the six attributes are present and they will
be used during the learning step. To compile these rules it was necessary to create a
text file that will serve as input to SymbComp. Once compiled the rules carry out
three types of tests using NeuSim:
Case a) Connectionist approach: Learning and generalization from the base of ex-
amples.
Case b) Symbolic approach: Generalization from the compiled ANN.
Case c) Hybrid approach: Learning and generalization using the compiled ANN
and the base of examples.
For the three cases we vary the number of used rules (two rules, one rule) and the
percentage of the examples used during the learning (100%, 75%, 50%, 25%). The
results are shown in the Table 2. In the Table the percentage of obtained learning and
the time when it was reached, the generalization percentage on the total of the base of
examples and the hidden units added are indicated.
From the results obtained we can observe that in the some cases where the numeric
and the symbolic knowledge are integrated, the percentage in the generalization is, in
most of cases, higher than those tests where only the numeric knowledge is used.
For example, for the connectionist approach (case a) using 50% of the base of ex-
amples we achieve a generalization of 59.95%, while if we use the hybrid approach
(case c) with the rule 1 and 50% of the base of examples we obtain a generalization of
94.21%. Another interesting result is when we use the rule 2 and 50% of the base of
examples, obtaining a generalization of 63.42% (also better than the connectionist
approach). For the same hybrid approach, if we use the rule 1 and 25% of the base of
18 V. G Cruz Sánchez et al.
examples we arrive at a generalization percentage of 77.03%, much better than
55.09% of the connectionist approach. For all these cases we reduce the number of
learning periods and the number of hidden units added.
Table 2. Results obtained for the study of Monk's problem
On the other hand, if we compare the symbolic approach (case b) where we use
only the rule 2 against the hybrid approach using the same rule and 100% of the base
of examples, we realize that the generalization is better in the second case (81.94%
against 95.83%, respectively). The same thing would happen if we use the rule 1
instead of the rule 2.
A result at the side of the objectives of this work is the discovery that rule 1 is
“more representative” that rule 2. This can be observed on table 2: with the compila-
tion of the rule 1 a generalization of 93.05% is reached against 81.94% for the rule 2.
8 Conclusions
In this paper we present a method of knowledge refinement using a neuro-symbolic
hybrid system. For this, a compiler that converts a set of symbolic rules into an artifi-
cial neural network was implemented. Using this last one, we could “complete” the
knowledge represented in the rules by means of a process of numeric incremental
learning. Thanks to the tests carried out on Monk's Problem we could demonstrate
that when we use the numeric or symbolic knowledge separately it is unfavourable
with regards to the use of a hybrid system.
Case
(approach) Compiled
rules
Examples
used
%
Knowledge
% Generalization Hidden
units
added
Learning
epochs
a)
Connectionist
-
-
-
-
100
75
50
25
100
100
100
55.09
99.07
87.03
59.95
55.09
2
7
5
2
484
1614
983
320
b)
Symbolic
R1 ,R2
R1
R2
-
-
-
-
-
-
100
93.05
81.94
0
0
0
0
0
0
c)
Hybrid
R1 ,R2
R1 ,R2
R1 ,R2
R1, R2
R1
R1
R1
R1
R2
R2
R2
R2
100
75
50
25
100
75
50
25
100
75
50
25
100
100
100
100
100
100
100
100
100
100
100
100
98.61
98.61
98.61
95.13
99.53
99.30
94.21
77.03
95.83
73.14
63.42
51.85
1
1
1
1
1
1
1
1
3
6
4
1
84
95
92
82
82
78
69
67
628
1226
872
58
Compilation Of Symbolic Knowledge and Integration with Numeric Knowledge 19
The results obtained from tests as: the increase in the generalization percentage,
the reduction of the number of hidden neurons added and the reduction of the number
of learning epochs, demonstrate that the hybrid systems theory is applicable when a
lack of knowledge exists and this can be covered with the integration of two types of
knowledge. At the moment, we are working to apply this principle to problems of
visual classification of objects in which descriptive rules and numeric examples (vis-
ual characteristic) are known. Our objective in this case is to create tools that help the
supervision and control processes of the visual quality. We are also intending to
enlarge SymbComp to be able to compile other kind of rules (for example fuzzy
rules), as well as to study the problem of the knowledge explicitation from the refined
neural networks.
We consider that this theory (compilation mechanism) it is applicable to simple
rules of the type IF…THEN... and it cannot be generalized to all type of rules. For the
group of rules that was proven, the accepted attributes were of type: nominal, binary
and in range. For the compilation of another rule types, for example rules that accept
diffuse values, the application of this theory would imply modifications to the algo-
rithm. Some works that actually are developed addressed this problem.
9 Acknowledgement
Authors would like to thank: Centro Nacional de Investigación y Desarrollo Tec-
nológico (cenidet) the facilities offered for the execution of this research.
References
1. Wermter, Stefan and Sun, Ron. (Eds.): Hybrid neural systems. Springer Heidelberg. (2000).
2. Towell, G.: Symbolic Knowledge and Neural Networks: Insertion, Refinement and Extrac-
tion. Ph.D. Thesis. Univ. of Wisconsin - Madison. USA. (1991).
3. Osório, F. S.: INSS - Un Système Hybride Neuro-Symbolique pour l'Apprentissage Auto-
matique Constructif. PhD Thesis, LEIBNIZ-IMAG, Grenoble – France (February 1998).
4. Thrun Sebastian.: The Monk’s Problems. School of Computer Science, Carnegie Mellon
University. Pittsburgh, PA 15213, USA. (1992).
5. The apple. Copyright infoagro.com (2003).
6. Terence Parr.: An Introduction To ANTLR. Another tool for language recognition. (2003).
7. Charte, Francisco: Programation in C++Builder 5. Anaya Multimedia Editions. (2000)
8. Fahlman, S.E., Lebiere, C.: The Cascade-Correlation Learning Architecture. Carnegie
Mellon University. Technical Report. CMU-CS-90-100. (1990).
9. Arevian G., Wermter S., Panchev C.: Symbolic state transducers and recurrent neural pref-
erence machines for text mining. International Journal on Approximate Reasoning, Vol. 32,
No. 2/3, pp. 237-258. (2003).
10. McGarry K., MacIntyre J.: Knowledge transfer between neural networks, proceedings of the
sixteenth european meeting on cybernetics and systems research. Vienna, Austria, pp. 555-
560. (April 2002).
20 V. G Cruz Sánchez et al.
11. Osorio, F. S., AMY, Bernard.: Aprendizado de máquinas: métodos para inserção de regras
simbólicas em redes neurais artificiais aplicados ao controle em robótiva autônoma. Revista
SCIENTIA, Vol. 12, Nro. 1, p.1-20. Editora da Unisinos, Out. (2001).
12. Rashad U., Arullendran P., Hawthorne M., Kendal S.: A hybrid medical information system
for the diagnosis of dizziness. Proceedings 4th International Conference Neural Networks
and Expert Systems in Medicine and Healthcare. Greece. (June 2001).
13. Wermter S., Panchev C.: Hybrid preference machines based on inspiration from neurosci-
ence. Cognitive Systems Research. Vol. 3, No. 2, pp. 255-270. (2002).
