Conference PaperPDF Available

BET : An Inductive Logic Programming Workbench

Authors:

Abstract and Figures

Existing ILP (Inductive Logic Programming) systems are implemented in different languages namely C, Progol, etc. Also, each system has its customized format for the input data. This makes it very tedious and time consuming on the part of a user to utilize such a system for experimental purposes as it demands a thorough understanding of that system and its input specification. In the spirit of Weka [1], we present a relational learning workbench called BET(Background + Examples = Theories), implemented in Java. The objective of BET is to shorten the learning curve of users (including novices) and to facilitate speedy development of new relational learning systems as well as quick integration of existing ILP systems. The standardized input format makes it easier to experiment with different relational learning algorithms on a common dataset.
Content may be subject to copyright.
BET : An Inductive Logic Programming
Workbench
Srihari Kalgi, Chirag Gosar, Prasad Gawde
Ganesh Ramakrishnan, Kekin Gada,
Chander Iyer, Kiran T.V.S, and Ashwin Srinivasan
Department of Computer Science and Engineering
IIT Bombay, India
{cgosar,ganesh}@cse.iitb.ac.in
{chanderjayaraman,kekin.gada,prasg41,srihari.kalgi,t.kiran05}@gmail.com
ashwin.srinivasan@in.ibm.com
http://www.cse.iitb.ac.in/~bet
Abstract. Existing ILP (Inductive Logic Programming) systems are
implemented in different languages namely C, Progol, etc. Also, each
system has its customized format for the input data. This makes it very
tedious and time consuming on the part of a user to utilize such a sys-
tem for experimental purposes as it demands a thorough understand-
ing of that system and its input specification. In the spirit of Weka [1],
we present a relational learning workbench called BET(Background +
Examples = Theories), implemented in Java. The objective of BET is to
shorten the learning curve of users (including novices) and to facilitate
speedy development of new relational learning systems as well as quick in-
tegration of existing ILP systems. The standardized input format makes
it easier to experiment with different relational learning algorithms on a
common dataset.
Key words: BET, ILP systems, Golem, Progol, FOIL, PRISM, TILDE
1 Introduction
There have been several Inductive Logic Programming (ILP) system implemen-
tations. Each system has its own specifications for input data. Different sys-
tems do not necessarily agree on their inputs. This often makes comparisons
across different implementations tricky, owing to either a difference in names
or semantics of parameters and/or a difference in the choice of the program-
ming language. Very often, the primitives and core components employed, such
as theorem provers, SAT solvers, inference engines, etc., are also different across
different implementations, rendering the computation and accuracy comparisons
less reliable. This paper discusses a Workbench for ILP called BET. BET is de-
veloped in Java and it standardizes the specification of input using XML (eX-
tensible Markup Language). It provides a common framework for “building”
as well as “integrating” different ILP systems. The provision for implementing
2 Srihari Kalgi et. al.
algorithms in a common language (Java) improves the feasibility of comparing
algorithms on their computational speeds. Whereas, the input standardization
enables sound comparison of accuracies (or related measures) and also allows
for experimenting with multiple ILP systems on the same dataset without any
input conversion required to the system specific format.
BET includes several evaluation functions and operator primitives such as
Least General Generalization (LGG) [9], Upward cover, Downward cover, etc.
These features facilitate the rapid development of new relational learning systems
and also ease out the integration of existing relational systems into BET. BET
also allows a user to choose from multiple theorem provers as plug and play
components. Presently, YAP (Yet Another Prolog) [12] and SWI-Prolog [13] are
included in the basic package. The initial version of BET has three relational
systems implemented namely FOIL [4], Golem [3] and TILDE [2][10] and four
relational systems integrated namely Progol [11], FOIL, Golem and PRISM [5][6]
(Though PRISM is not a standard ILP system, it has been integrated with the
specific intension of learning probabilities in order to associate uncertainity with
theories learnt using ILP systems).
We proceed with an overview of BET and the motivation to develop such a
system in Section 2. Section 3 focuses on the design of the BET system. Section
4 explains how new relational learning systems can be integrated/implemented
in BET. Section 5 compares BET with existing workbenches and highlights the
advantages of BET. We summarize BET in Section 6.
2 Overview
A principal motivation for developing a system like BET is the reduction of the
learning curve for both expert-users and novices as well as programmers in the
area of relational learning, particularly ILP. For example, with BET, the end-
user will need to understand only the standardized input parameters for BET.
This reduces the time overheads involved in comparing different algorithmic
implementations as well as in experimenting with individual implementations.
More specifically, a user need not convert datasets from one format to another
while switching between ILP systems. Further, the standardized APIs in BET
make development of algorithms within BET much easier. Figure 1 shows the
block diagram of BET.
Any ILP system in BET takes four files as its input namely positive examples,
negative examples, background knowledge and language restriction files, and it
outputs in the form of theories. There are primarily two important components
of BET namely the BET GUI and the BET Engine. The BET Engine is the
back-end engine at the core of BET and the BET GUI communicates with the
BET engine through the medium of a properties file. BET users can experiment
with the system using the GUI whereas programmers can use the APIs provided
and extend BET with new ILP or relational learning systems. Section 5 explains
how BET could be extended.
BET : An ILP Workbench 3
Fig. 1. Block diagram of BET
3 Design of BET
Almost all ILP algorithms as well as systems require following five inputs: posi-
tive examples, negative examples, background knowledge, mode declarations and
hyper-parameters for tuning the system. The clauses learnt may be customized
by means of the mode declarations, though some algorithms/systems may not
allow for the specification of mode declarations (example is Confidence-based
Concept Discovery system [7]). In this Section, we discuss the need for having
a standard for the input, and eventually specify the BET standard input file
formats.
3.1 Need for Standard Format
There is little or no consensus between the input specifications of ILP systems.
For example, FOIL has its standard input specification which is completely differ-
ent from that of Golem or Progol. FOIL accepts the positive examples, negative
examples and background knowledge in a single file. On the other hand, Golem
takes three files as input viz. positive example file (.f ), negative example file (.n)
and the background knowledge file (.b) which includes the mode declarations.
Another ILP system Progol has an input format that is slightly similar to Golem,
but has many more setting parameters.
Clearly any end user, desiring to experiment with a new ILP system, is
required to understand its input format thoroughly, convert the data in hand to
that format and only then can he/she use it. BET assists the end user with a
standardized input specification that is generic enough and can be converted to
the specification of any existing ILP system. Section 3.2 will discuss about BET
standard formats.
3.2 BET Standard Format
As already stated, BET’s inputs are comprised of four XML files namely posi-
tive examples, negative examples, background knowledge and language restric-
tion file. Sample positive example, negative example, background knowledge and
language restriction files for learning the ancestor relation are shown in Table 1,
4 Srihari Kalgi et. al.
Table 2, and Table 3 respectively in the document available at [15]. The hyper-
parameters can be specified in Language Restriction file. The format for each
file is as follows:
Positive Example File: The file consists of clauses which represents positive
examples for the relation (predicate) for which the theory needs to be learned.
Negative Example File: This file consists of clauses which represents negative
examples of the predicate (relation) for which the theory needs to be learned.
There may be some ILP systems which do not consider any negative examples
during construction of theory, but may utilize negative examples for pruning the
theories that have been learned.
Background knowledge File: The background knowledge (BGK) file contains
all the domain specific information which is required by the ILP system in order
to learn the target predicate. All this information should be in the form of
clauses. It can also contain rules instead of just facts.
Language Restriction File: The structured search space (lattice or simply a
partial order) of clauses which needs to be searched in order to arrive at a rea-
sonably good hypothesis is almost infinite. Language Restrictions endow an ILP
system with the flexibility of reducing the search space. There are three generic
parts to the Language Restriction viz. Mode declaration, Type and Determi-
nation. Also we can specify the system-specific hyper-parameters like stopping
threshold, minimum accuracy, etc. in this file only. Each of them, is explained
below.
Type: Type defines the type of constants. For example, the constant john is of
type person, constant 1is of type integer. Type specification can be done in
the language restriction file.
Mode Declarations: Mode declarations declare the mode of the predicates
that can be present in any of clausal theories. The mode declaration takes the
form mode(RecallNumber,PredicateMode), where
RecallNumber: bounds the non-determinacy of a form of predicate call.
PredicateMode: has the following syntax
predicate(ModeType,ModeType,...)
predicate(<+/-/#>Type,<+/-/#>Type,...)
+Type : the term must be an input variable of type Type.
-Type : the term must be an output variable of type Type.
#Type : the term must be a ground constant of type Type.
BET : An ILP Workbench 5
Determination: This component of mode declarations provides information
regarding which predicates need to be learned in terms of which other predicates.
Hyper-parameters: Hyper-parameters are very much system specific and we
can find the different parameters for a system by doing system-name –help or
through the graphical interface for that particular system.
4 Class Diagram and Components of BET
The class diagram of BET is shown in Figure 1 in the document available at
[15]. The major components of BET are explained in subsequent sections.
ClauseSet: ClauseSet refers to a set of clauses, which is often referred to in the
description of ILP algorithms. The ClauseSet interface defines the signature of
the methods used to access the ClauseSet object.
LanguageRestriction: LanguageRestriction is an interface for mode declara-
tions, determinations and types.
InputEncoder: InputEncoder is mainly used for integrating legacy systems
(i.e. ILP systems written in native code) into BET. Any wrapper around an
integrated legacy system will require conversion of the BET standard data format
to the format accepted by the legacy system. Any legacy system which requires
to be integrated into BET has to implement InputEncoder interface and provide
a method of converting the standard BET file format to the format accepted by
the legacy system.
TheoryLearner: TheoryLearner is an important component of BET. It is re-
sponsible for learning the theory (hypothesis) from the input. TheoryLearner
has objects of type InputEncoder and EvalMetric (explained next) as its data
members. In case the ILP system is completely implemented in Java (i.e., if it is
not a legacy system), it can use a dummy implementation of the InputEncoder
interface and pass the object to the TheoryLearner class. The dummy imple-
mentation of InputEncoder interface is mainly used to access the BET input
files.
EvalMetric (Evaluation Metric): Any ILP algorithm will try to explore the
structured space (subsumption lattice or partial order) of clauses, by traversing
the graph using refinement operators. At every point in the space, the algorithm
has to decide whether a clause in the space is good enough to be present in the
hypothesis. For this purpose, the covers relation is employed.
If the covers relation is defined by Entailment (|=), then the ILP system
is said to learn from entailment [9]. Some other ILP systems learn from proof
traces [9], while few others learn from interpretations [9], etc.
6 Srihari Kalgi et. al.
Any implementation of EvalMetric will specify the covers relation, i.e., defi-
nition of an example being covered by background knowledge (with candidate
hypothesis).
TheoremProver: The theorem prover is the backbone of most ILP systems.
BET has two different theorem provers namely YAP and SWI-Prolog. YAP is
perhaps the fastest theorem prover as of today. YAP is built in C and doesn’t
have any JNI (Java Native Interface), so the YAP process is run by spawning
a process through a BET program. SWI-Prolog is also built in C, but it does
support JNI (Java Native Interface). SWI-Prolog has a standard interface called
JPL which comes bundled with SWI-Prolog itself.
5 Support for Building ILP Systems in BET
It is very easy to integrate1an existing system into BET. Also the implementa-
tion2of a new system within BET is faster than implementing it from scratch.
The following section briefs on how to integrate and implement a system in BET.
5.1 Integrating a Legacy System in BET
Legacy systems can be integrated into BET as follows:
Implement the interface InputEncoder, which should provide a functionality
to convert files from BET standard format to the format accepted by the
legacy system.
Extend the TheoryLearner class and override the method learnTheory.
5.2 Implementing a New ILP System in BET
A dummy implementation of the interface InputEncoder is required for this
purpose, since the TheoryLearner class expects an object of InputEncoder
while creating its object.
Extend the TheoryLearner class and override the method learnTheory.
Different types of Evaluation Metric, Theorem provers (YAP, SWI-Prolog)
can be used in implementing the ILP system.
6 Related Work
Aleph and GILPS (Generic Inductive Logic Programming System) are two earlier
efforts put in to develop a workbench like BET for Inductive Logic Programming.
Aleph [8] is developed by Ashwin Srinivasan, whereas the GILPS [14] workbench
is developed by Jose Santos.
1The authors of this paper took two weeks to integrate PRISM in BET and one week
individually for integrating Golem and FOIL
2The authors of this paper implemented TILDE, FOIL and Golem in BET within a
week for each algorithm
BET : An ILP Workbench 7
6.1 Aleph
Aleph is written in Prolog principally for use with Prolog compilers like Yap
and SWI-Prolog compiler. Aleph offers a number of parameters to experiment
with its basic algorithm. These parameters can be used to choose from different
search strategies, various evaluation functions, user-defined constraints and cost
specifications. Aleph also allows user-defined refinement operators.
6.2 GILPS
GILPS implements three relational systems namely TopLog, FuncLog and Pro-
Golem. GILPS requires at least YAP 6.0 for its execution. Many of its user
predicates and settings are shared with Aleph.
6.3 Advantages of BET over Existing Systems
BET offers following advantages over existing systems (Aleph, GILPS ,etc.):
BET is more likely to find use in the larger applied machine learning and
Mining communities who are less familiar with prolog and where it is im-
portant to interface learning tools in Java (such as BET) with applications
largely written in procedural languages (very often Java). This can also make
diagnosis and debugging easy. As against this, GILPS and Aleph assume fa-
miliarity with Prolog and cannot be easily invoked from applications written
in Java.
In the spirit of Weka, BET provides a nice framework for the integration of
legacy systems as well as for the implementation of new systems. In fact,
while the basic BET framework was provided by the first two co-authors,
the remaining co-authors used that framework to implement several ILP
algorithms. This is something that Aleph and GILPS do not cater to as
well.
BET allows saving the learned models as serialized files which can later be
used for finding models for new problems.
BET provides a choice of theorem prover where as GILPS can work only
with YAP.
BET comes with a YAP installer and all the “integrated” systems.
BET has classes to convert the background knowledge, positive examples
and negative examples files into XML format. Language restriction file needs
little human intervention for this purpose.
BET provides a standard input/output format which enables end-user to
experiment with multiple relational systems with the same datasets. Aleph
uses three different files namely (.f), (.n) and (.b) whereas GILPS takes input
as prolog files (.pl).
8 Srihari Kalgi et. al.
7 Summing up BET
The standardized input format of BET makes it more convenient for user to
have only one input format for all relational systems. The ability of BET to
write learned classification/regression models as serialized files allows for saving
the model and reusing the model at a later stage for classification or regression
on new problems. The BET graphical user interface makes experimentation with
implemented algorithms and integrated systems much more easier. Use of JAVA
as an implementation language makes BET extensible and platform indepen-
dent. The only visible disadvantage of BET is that the JAVA implementation
of relational systems are slower as compared to their C/C++/Progol counter-
parts, but we chose Java over C++ owing to ease of extending and platform
independence of the former over the latter.
References
1. E. Frank and M. A. Hall and G. Holmes and R. Kirkby and B. Pfahringer and
I. H. Witten: “Weka- A machine learning workbench for data mining.” In : Data
Mining and Knowledge Discovery Handbook- A Complete Guide for Practitioners
and Researchers, pp. 1305–1314. Springer, Berlin (2005)
2. Blockeel, Hendrik and De Raedt, Luc: “Top-down induction of first-order logical de-
cision trees.” In: Artificial Intelligence Journal (1998), pp. 285–297. Elsevier Science
Publishers Ltd.
3. S.H. Muggleton and C. Feng: “Efficient induction of logic programs.”: Proceedings
of the First Conference on Algorithmic Learning Theory, pp. 368–381. Ohmsha,
Tokyo (1990)
4. J. Ross Quinlan and R. Mike Cameron-Jones: “FOIL: A Midterm Report.” : Machine
Learning: ECML-93, European Conference on Machine Learning, Proceedings, vol.
667 pp. 3–20. Springer-Verlag (1993)
5. Taisuke Sato, Yoshitaka Kameya, Neng-Fa Zhou: “Generative Modeling with Failure
in PRISM.”: IJCAI, pp. 847–852 (2005)
6. Taisuke Sato and Yoshitaka Kameya: “PRISM: A Language for Symbolic-Statistical
Modeling.” : IJCAI, pp. 1330–1339 (1997)
7. Y. Kavurucu, P. Senkul and I.H. Toroslu: “ILP-based concept discovery in multi-
relational data mining.” : Expert Systems with Applications, vol. 36 pp. 11418 -
11428 (2009)
8. “Aleph Manual”, http://www.comlab.ox.ac.uk/activities/machinelearning/
Aleph/aleph.html
9. “Statistical Relational Learning Notes”, http://www.cse.iitb.ac.in/~cs717/
notes/
10. “TILDE: Top-down Induction of Logical Decision Trees”, http://www-ai.ijs.si/
~ilpnet2/systems/tilde.html
11. “Progol”, http://www.doc.ic.ac.uk/~shm/progol.html
12. “YAP Manual”, http://www.dcc.fc.up.pt/~vsc/Yap/
13. “SWI-Prolog”, http://www.swi- prolog.org/
14. “General Inductive Logic Programming System”, http://www.doc.ic.ac.uk/
~jcs06/GILPS/
15. “ILP Systems Integrated and Implemented in BET and Sample Examples”,http:
//www.cse.iitb.ac.in/~bet/appendix.pdf
... Another problem is a theoretical term generation problem [4,6,8]. To dene a target predicate, Prolog programs use several auxiliary predicates in general. ...
... In recent years, a number of inference systems are proposed in the area of inductive logic programming concerning to theoretical term generation problem [4,8]. From our main result, we know the existence of an eective inference algorithm that infers a linearly covering program dening an unknown target relation by receiving positive examples of the target relation. ...
Article
Full-text available
To appear in Proc. 3rd Eur-Jap Seminar on Information Modeling and Knowledge Bases In this paper, we study inductive inference of a subclass of Prolog programs from positive examples. The subclass, called linearly covering programs, allows shared variables occurring only in the bodies of a clause, which are excluded in the class of linear Prologs already known to be inferable from positive data. In a linearly covering programs, any data passing between subgoals preserves the total size of data contents from a data-dependency constraint. Using Shinohara's work on the inferability of concept defining framework, we prove that for every fixed $ k,m > 0 $, if the length of subgoals in a clause is bounded by a k, the class of linearly covering programs containing at most m definite clauses is inferable from positive data, without any oracle for auxiliary predicates. The result gives a partial answer to the theoretical term problem in model inference. Furthermore, we show that the restriction on the length of bodies is necessary for the inferability.
... The question of whether our first-order logic definition is a correct and complete classifier, thus a solution to an inductive prediction learning problem covering all and only the positive examples [208], turns out to be problematic from a methodological point of view. First, because the concept of a junction and other higher order network features is not unambiguously established in existing network data models, it is unclear where to get an unbiased sample from, and who applies the 'right' concept. ...
... The question of whether our first-order logic definition is a correct and complete classifier, thus a solution to an inductive prediction learning problem covering all and only the positive examples [208], turns out to be problematic from a methodological point of view. First, because the concept of a junction and other higher order network features is not unambiguously established in existing network data models, it is unclear where to get an unbiased sample from, and who applies the 'right' concept. ...
Book
Full-text available
In this thesis, I propose a method of constructing semantic reference systems for geographic information based on reproducible observations. Similar to spatial reference systems, semantic reference systems consist of formal theories (reference theories) with conventionally established interpretations into perceptual and constructive operations (such as pointing to a physical monument, and describing locations relative to it). They can be used to annotate data and describe and compare their semantics. The thesis addresses the grounding part of reference systems. I argue that persisting problems of information ontologies, namely the grounding problem and the problem of reference, could be solved using certain principles of construction and imitation in terms of perceptual operations. These operations focus joint human attention on pre-conceptual cognitive mechanisms, i.e. Gestalts, in the perceived space around the body. They also allow an observer to relate foci of attention based on the Gestalt. The memorized relations are expressed by observation predicates, to be established by convention in terms of speech acts of an observation language. These speech acts can be shared, and thus represent lowest level information items. I propose a kind of practical constructivism guided by a formal language. The idea is to describe data categories in terms of observation predicates, i.e. bottom-up, in order to reconstruct the underlying observation procedure, instead of presuming abstract concepts. For example, bodies, surfaces, and different kinds of media in the human environment are grounded in terms of perceived affordances. Object properties such as waterdepth are grounded in experiential geometry, visual surfaces and media of diving. I propose a corresponding reference theory and put it to a practical test in this thesis: I define an essential road network category, namely a junction, and test the definition in Open Street Map.
... The limited representation formalism of propositional learning algorithms and their di culties in using substantial background knowledge has led to an increasing interest in investigating learning methods in a rst order framework. Inductive Logic Programming (ILP) Mug90,Mug93] is an approach that has received a lot of attention recently. The task of ILP is to learn logic programs by induction from examples in the presence of background knowledge. ...
Article
. Restrictions on the number and depth of existential variables as defined in the language series of Clint [Rae92] are widely used in ILP and expected to produce a considerable reduction in the size of the hypothesis space. In this paper we show that this is generally not the case. The lower bounds we present lead to intractable hypothesis spaces except for toy domains. We argue that the parameters chosen in Clint are unsuitable for sensible bias shift operations, and propose alternative approaches resulting in the desired reduction of the hypothesis space and allowing for a natural integration of the shift of bias. 1 Introduction The limited representation formalism of propositional learning algorithms and their difficulties in using substantial background knowledge has led to an increasing interest in investigating learning methods in a first order framework. Inductive Logic Programming (ILP) [Mug90, Mug93] is an approach that has received a lot of attention recently. The task of ...
Conference Paper
Existing propositionalisation approaches mainly deal with categorical attributes. Few approaches deal with continuous attributes. A first solution is then to discretise numeric attributes to transform them into categorical ones. Alternative approaches dealing with numeric attributes consist in aggregating them with simple functions such as average, minimum, maximum, etc. We propose an approach dual to discretisation that reverses the processing of objects and thresholds, and whose discretisation corresponds to quantiles. Our approach is evaluated thoroughly on artificial data to characterize its behaviour with respect to two attribute-value learners, and on real datasets.
Chapter
Full-text available
The Weka workbench is an organized collection of state-of-the-art machine learning algorithms and data preprocessing tools. The basic way of interacting with these methods is by invoking them from the command line. However, convenient interactive graphical user interfaces are provided for data exploration, for setting up large-scale experiments on distributed computing platforms, and for designing configurations for streamed data processing. These interfaces constitute an advanced environment for experimental data mining. The system is written in Java and distributed under the terms of the GNU General Public License.
Conference Paper
Full-text available
PRISM is a logic-based Turing-complete symbolic- statistical modeling language with a built-in pa- rameter learning routine. In this paper,we enhance the modeling power of PRISM by allowing gen- eral PRISM programs to fail in the generation pro- cess of observable events. Introducing failure ex- tends the class of definable distributions but needs a generalization of the semantics of PRISM pro- grams. We propose a three valued probabilistic se- mantics and show how failure enables us to pur- sue constraint-based modeling of complex statisti- cal phenomena.
Conference Paper
Full-text available
We present an overview of symbolic-statistical modeling language PRISM whose programs are not only a probabilistic extension of logic pro-grams but also able to learn from examples with the help of the EM learning algorithm. As a knowledge representation language appropriate for probabilistic reasoning, it can describe v ar-ious types of symbolic-statistical modeling for-malism known but unrelated so far in a sin-gle framework. We show by examples, to-gether with learning results, that most popular probabilistic modeling formalisms, the hidden Markov model and Bayesian networks, are de-scribed by PRISM programs.
Chapter
FOIL is a learning system that constructs Horn clause programs from examples. This paper summarises the development of FOIL from 1989 up to early 1993 and evaluates its effectiveness on a non-trivial sequence of learning tasks taken from a Prolog programming text. Although many of these tasks are handled reasonably well, the experiment highlights some weaknesses of the current implementation. Areas for further research are identified.
Article
A first order framework for top-down induction of logical decision trees is introduced. Logical decision trees are more expressive than the flat logic programs typically induced by empirical inductive logic programming systems because logical decision trees introduce invented predicates and mix existential and universal quantification of variables. An implementation of the framework, the Tilde system, is presented and empirically evaluated. 1 Introduction Top-down induction of decision trees (TDIDT) [Qui86] is the best known and most succesful machine learning technique. It has been used to solve numerous practical problems. It employs a divide-and-conquer strategy, and in this it differs from its rulebased competitors (e.g. AQ [MMHL86]), which are based on covering strategies (cf. [Bos95]). Within attribute-value learning (or propositional concept-learning) TDIDT is more popular than the covering approach. Yet, within first order approaches to concept-learning, only a few learning sy...
Article
Multi-relational data mining has become popular due to the limitations of propositional problem definition in structured domains and the tendency of storing data in relational databases. Several relational knowledge discovery systems have been developed employing various search strategies, heuristics, language pattern limitations and hypothesis evaluation criteria, in order to cope with intractably large search space and to be able to generate high-quality patterns. In this work, an ILP-based concept discovery method, namely Confidence-based Concept Discovery (C2D), is described in which strong declarative biases and user-defined specifications are relaxed. Moreover, this new method directly works on relational databases. In addition to this, a new confidence-based pruning is used in this technique. We also describe how to define and use aggregate predicates as background knowledge in the proposed method. In order to use aggregate predicates, we show how to handle numerical attributes by using comparison operators on them. Finally, we analyze the effect of incorporating unrelated facts for generating transitive rules on the proposed method. A set of experiments are conducted on real-world problems to test the performance of the proposed method.
Article
Although top&dash;down induction of decision trees is a very popular induction method, up till now it has mainly been used for propositional learning&semi; relational decision tree learners are scarce. This dissertation discusses the application domain of decision tree learning and extends it towards the first order logic context of Inductive Logic Programming.
Article
Recently there has been increasing interest in systems which induce first order logic programs from examples. However, many difficulties need to be overcome. Well-known algorithms fail to discover correct logical descriptions for large classes of interesting predicates, due either to the intractability of search or overly strong limitations applied to the hypothesis space. In contrast, search is avoided within Plotkin's framework of relative least general generalisation (rlgg). It is replaced by the process of constructing a unique clause which covers a set of examples relative to given background knowledge. However, such a clause can in the worst case contain infinitely many literals, or at best grow exponentially with the number of examples involved. In this paper we introduce the concept of h-easy rlgg clauses and show that they have finite length. We also prove that the length of a certain class of "determinate" rlgg is bounded by a polynomial function of certain features of the ba...
Article
: FOIL is a learning system that constructs Horn clause programs from examples. This paper summarises the development of FOIL from 1989 up to early 1993 and evaluates its effectiveness on a non-trivial sequence of learning tasks taken from a Prolog programming text. Although many of these tasks are handled reasonably well, the experiment highlights some weaknesses of the current implementation. Areas for further research are identified. 1. Introduction The principal differences between zeroth-order and first-order supervised learning systems are the form of the training data and the way that a learned theory is expressed. Data for zeroth-order learning programs such as ASSISTANT [Cestnik, Kononenko and Bratko, 1986], CART [Breiman, Friedman, Olshen and Stone, 1984], CN2 [Clark and Niblett, 1987] and C4.5 [Quinlan, 1992] comprise preclassified cases, each described by its values for a fixed collection of attributes. These systems develop theories, in the form of decision trees o...