Deep Transfer as Structure Learning in Markov Logic Networks


ABSTRACT Learning the relational structure of a domain is a funda-mental problem in statistical relational learning. The deep transfer algorithm of Davis and Domingos at-tempts to improve structure learning in Markov logic networks by harnessing the power of transfer learn-ing, using the second-order structural regularities of a source domain to bias the structure search process in a target domain. We propose that the clique-scoring process which discovers these second-order regularities constitutes a novel standalone method for learning the structure of Markov logic networks, and that this fact, rather than the transfer of structural knowledge across domains, accounts for much of the performance bene-fit observed via the deep transfer process. This claim is supported by experiments in which we find that clique scoring within a single domain often produces results equaling or surpassing the performance of deep transfer incorporating external knowledge, and also by explicit algorithmic similarities between deep transfer and other structure learning techniques.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We present a new approach to learning hypertext classifiers that combines a statistical text-learning method with a relational rule learner. This approach is well suited to learning in hypertext domains because its statistical component allows it to characterize text in terms of word frequencies, whereas its relational component is able to describe how neighboring documents are related to each other by hyperlinks that connect them. We evaluate our approach by applying it to tasks that involve learning definitions for (i) classes of pages, (ii) particular relations that exist between pairs of pages, and (iii) locating a particular class of information in the internal structure of pages. Our experiments demonstrate that this new approach is able to learn more accurate classifiers than either of its constituent methods alone.
    Machine Learning 01/2001; 43:97-119. DOI:10.1023/A:1007676901476 · 1.69 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Many real-world applications of AI require both probability and first-order logic to deal with uncertainty and structural complexity. Logical AI has focused mainly on handling complexity, and statistical AI on handling uncertainty. Markov Logic Networks (MLNs) are a powerful representation that combine Markov Networks (MNs) and first-order logic by attaching weights to first-order formulas and viewing these as templates for features of MNs. State-of-the-art structure learning algorithms of MLNs maximize the likelihood of a relational database by performing a greedy search in the space of candidates. This can lead to suboptimal results because of the incapability of these approaches to escape local optima. Moreover, due to the combinatorially explosive space of potential candidates these methods are computationally prohibitive. We propose a novel algorithm for learning MLNs structure, based on the Iterated Local Search (ILS) metaheuristic that explores the space of structures through a biased sampling of the set of local optima. The algorithm focuses the search not on the full space of solutions but on a smaller subspace defined by the solutions that are locally optimal for the optimization engine. We show through experiments in two real-world domains that the proposed approach improves accuracy and learning time over the existing state-of-the-art algorithms.
    ECAI 2008 - 18th European Conference on Artificial Intelligence, Patras, Greece, July 21-25, 2008, Proceedings; 01/2008
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Transfer learning addresses the problem of how to leverage knowledge acquired in a source domain to improve the ac- curacy and speed of learning in a related target domain. This paper considers transfer learning with Markov logic networks (MLNs), a powerful formalism for learning in relational do- mains. We present a complete MLN transfer system that first autonomously maps the predicates in the source MLN to the target domain and then revises the mapped structure to fur- ther improve its accuracy. Our results in several real-world domains demonstrate that our approach successfully reduces the amount of time and training data needed to learn an accu- rate model of a target domain over learning from scratch.
    Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, July 22-26, 2007, Vancouver, British Columbia, Canada; 01/2007


Available from