Inference of large-scale topology of gene regulation networks by neural nets

Abstract

This paper addresses the problem of inferring topological features of gene regulation networks from data that are likely to be available from current experimental methods, such as DNA microarrays. The proposed method uses neural networks to predict the topology class from histograms of perturbation propagation data. The preliminary results with simulated data are encouraging. The trained neural network is able to classify the network topology as random (exponential) or scale-free with 90% accuracy. Compare to the previous network connectivity inference methods that are often problematic with current noisy data, this method is expected to be more robust because it uses global characteristics of dynamic networks.

Full text

Page 1

Inference of large-scale topology of gene regulation
networks by neural nets*

Sohyoung Kim
School of Computational Sciences
George Mason University
Manassas, VA, U.S.A
skime@gmu.edu
Bethesda, MD, U.S.A
weinstein@dtpax2.ncifcrf.gov

John N. Weinstein
Center for Cancer Research
National Cancer Institute
National Institutes of Health
John J. Grefenstette
School of Computational Sciences
George Mason University
Manassas, VA, U.S.A
jgrefens@gmu.edu

* 0-7803-7952-7/03/$17.00     2003 IEEE.
Abstract - This paper addresses the problem of inferring
topological features of gene regulation networks from
data that are likely to be available from current
experimental methods, such as DNA microarrays. The
proposed method uses neural networks to predict the
topology class from histograms of perturbation
propagation data. The preliminary results with simulated
data are encouraging. The trained neural network is able
to classify the network topology as random (exponential)
or scale-free with 90% accuracy. Compare to the
previous network connectivity inference methods that are
often problematic with current noisy data, this method is
expected to be more robust because it uses global
characteristics of dynamic networks.
Keywords: Gene regulation networks, perturbation
analysis, topology, inference.
1 Introduction

regulation networks provides insights of universal
biological structural design fundamentals and mediates a
better understanding of dynamical processes of gene
regulation in situation such as cancer. Recently, there has
been growing interest in network structure models that
have specific design principles and their inherent
statistical properties [1]. Three main classes of network
models, random (exponential) networks [8], scale-free
networks [4], and small-world networks [19], have been
used to describe topological features of various naturally
occurring large networks including the Internet, social
networks, and some biological networks [2]. Recent
analysis of network properties of the metabolic maps [12],
molecular interaction networks [9], and protein–protein
interaction networks [11] have shown that these biological
networks display scale free features. The neural network
map of C. elegance shows features of small world network
model [19].
Knowledge of large-scale structures of gene

networks are encouraging because the model provides
important features that might be especially beneficial to
biological systems [2, 11]. Particularly, high tolerance of
scale free networks for the random node failures in terms
of structural integrity might suggest that network topology
partially accounts for the robustness of biological systems.

Although recent analyses of network properties of
known biological pathways or networks have shown that
the most examples of biological networks display scale
free features [5, 11, 12, 17], it is not yet clear whether the
scale free features are common to most of biological
networks. The problem is that only limited information on
actual biological pathways and their connectivity is
available. To overcome these limitations, there have been
attempts to understand topology by inferring the
underlying connectivity of gene networks from microarray
time series data [5]. However, inferences of local
connection or interaction from microarrays are often
problematic given noisy experimental data.
The observations of scale free features in biological

the large-scale topology of gene regulation networks from
features that can be derived from realistic microarray data
sets. The long-term goals of this project are to:

1. Determine characteristic features derivable from time
series microarray data that reflect topological structure of
the underlying gene regulation networks;

2. Develop an inference method that uses these features to
infer the underlying topology class of the gene regulation
network; and

3. Apply these inference methods to determine possible
differences in gene regulation networks in biologically
significant situations, such as cancer.

This paper presents results on the first two steps of
this project.
Thus, this paper addresses the problem of identifying
3969

Page 2

measurable
propagation in the system. That is, one can perturb the
initial conditions of a cell, for example, through gene
knockouts or gene silencing by siRNA [18], and compare
the changes in expression level that results in. The
remainder of this paper suggests that statistics from such
perturbation studies can be used by a neural net to classify
the topology of the underlying gene expression network,
as shown in Figure 1.
Given current
feature
microarray
is patterns
technology,
of perturbation
one

Simulation system:
Random and Scale Free
mRNA
expression
Simulate
mRNA
expression
Feature 3
Feature 2
Feature 1
…
…
Feature 3
Feature 2
Feature 1
…
…
Selected Feature
Predict
Network
Types of topology:
Random or SF?Random or SF?
Types of topology:
Random
Network
Scale Free
Network
Train Neural
Random or Scale Free?
Predict
…
…

Figure 1. Schematic view of development of NN-TIM
(Neural Net based Topology Inference Method.)

describes the approach taken, including the theoretical
foundations for selecting perturbation data as the basis for
classification, the simulator used to generate training data
for the neural net classifier, the neural net used, and the
validation method. Section 3 presents the results on the
artificial data sets, and Section 4 discusses directions for
further work.
The paper is organized as follows: Section 2
2 Methods
2.1 Perturbation Data

gene network modeling [13]. In this class of models,
nodes represent genes, and directed edges represent
regulatory interactions between genes. A gene exhibits
two possible states, 0 (inactivated) or 1 (activated). The
transition function for each gene with in-degree k is
selected from the set of all possible Boolean function of k
variables. It is usually assumed that all genes perform a
simultaneous transition, yielding a well-defined next state
for the entire network. Even this highly simplified model
supports some useful modeling of real biological gene
regulation networks [3]. Some notable advantages of the
Boolean model are the intuitive analogy of Boolean rules
to combinatorial gene regulation [6, 10, 21, 23] and the
tractability of numerical simulation of large network [13].
Kauffman first introduced the Boolean network in

assign an initial state (perhaps at random) to all genes in
the network at time t1 and then observe the resulting
network state at time t2. Now suppose one perturbs the
initial state by making random changes to a given
percentage of the genes at time t1’ and then observes the
resulting network state at time t2’. The perturbation
propagation is defined as the change in the gene-state
vectors between t2 and t2’. For a network of N genes, the
perturbation propagation can be described as a binary
vector of length N such that the element i is 1 if the gene i
has a different state at times t2 and t2’, or otherwise 0.
Now imagine performing many repetitions of the
perturbation experiments, each time selecting a random
initial state, and a random perturbation of that state, and
measuring the perturbation propagation between the two
resulting states. Adding the resulting perturbation vectors
yields a count of how often each gene was perturbed, and
this can be summarized by a histogram showing the
number of genes that display a given frequency of
perturbation. The perturbation frequency is used as an
estimate of the perturbation propagation probability of
gene i, or PPi. The entire process is illustrated in Figure 2.
Given a Boolean gene regulation network, one can

100100
0011
state transition
01
101011
001111
t1
t2
t1’
t2’
State overlap (t2 , t2’) ? 0
Otherwise ? 1
101010
001000
001010
…
000011
Perturbation
propagation
vectors
(PP vectors)
Summary of PP vectors for each
gene as propagation frequency
g1 g2 g3 g4 g5 g6
g1 g2g3g4g5g6
518282
Histogram of perturbation
propagation frequency
010111
g1 g2 g3 g4 g5 g6
E1
E2
E3
E4
En
(t2 , t2’)

Figure 2. Illustrated process of measuring perturbation
propagations, using a simple network that includes only
six genes. The shaded genes at t1’ represent perturbed
genes, and shaded genes at t2’ represent propagated
perturbation. When the introduced perturbation at t1’ is
propagated to some of downstream genes at the next
states, t2’, the transferred perturbation causes the
difference in the gene state between t2 and t2’. In the E1
vector, ‘1’ represents state difference between t2 and t2’.
3970

Page 3

using the resulting histograms to classify to infer global
features of the topology of the underlying gene network.
The next section provides the theoretical foundations for
this approach.
The goal of this paper is to evaluate the possibility of
2.2 Theoretical Background

infer topological class (e.g. random vs. scale-free) from
connectivity distributions. Analytic results below indicate
that perturbation probability is monotonically related to a
gene's input connectivity for the Boolean network model.
Therefore, it may be reasonable to assume that we can
infer connectivity distribution, and therefore topological
class, from perturbation data.
Previous results [4] show that we should be able to

inputs to each gene, Derida showed that the overall
propagation of perturbation during a limited number of
state transitions depends on the degree k of the system
[13]. We extend this analysis to network with mean in-
degree <k>. Given system perturbation at t1’ (when no
gene interaction information is available), the perturbation
propagation probability of gene i, PPi (p), of large k (k ≥2)
is obtained according to
For Boolean networks with a fixed number of k

{}()
01
2
1

1
2
1
=−×=






−×≈
XP
bationene perturno input g
ofyprobabilit
i
PP
. (1)
The (1-P{X=0}) is the probability that any of its input
genes were perturbed at t1’. The 1/2 in equation 1 is an
approximate probability of state update of the ith gene
given input gene perturbation for random Boolean
functions (since the next state associated with the
perturbed and unperturbed states are uncorrelated).

perturbed at t1’, the probability that no input is perturbed,
P{X=0}, follows a hypergeometric distribution (Equation
2). For simplicity, we are assuming that the numbers of
input genes and output genes are independent. When a
gene has no input gene, k=0, the state update of the gene is
possible only if the gene was one of the perturbed genes.
When a gene has one input gene, k=1, the perturbation of
the input gene state will change the state of the gene
(assuming that all input genes are necessary). When a
gene has more than two inputs, the probability that binary
state changes given the input gene perturbation is
considered as approximately 1/2. Thus, perturbation
propagation probability of ith gene at given p percent
perturbation, PPi (p) follows
Since a certain number of genes are randomly

( )
p
()
,
0
1
2
1
}0{1
2
1

, 2 if
,
0
1}0{1
, 1 if
, , 0 if
























−
n






−×==−×
≥
























−
n






−==−
=
=
≈
n
N
kNk
XP
k
n
N
kNk
XP
k
N
n
k
PPi
(2)
where

( )
%100
×=N
n
p
. (3)
p is percent perturbation, N is a total number of genes in
the networks, and n is a number of perturbed genes at the
provided experiment.

derived from a simulator (described below).
These analytic results were compared with data

Figure 3. Monotonic increase of PP (Perturbation
Propagation Probability) as a function of k, number of in-
degree, in any system topology. Cross symbols represent
observed perturbation propagation from simulated system.
Circle symbols represent result from analytical approach in
Equation 2. Scale free network, Net (500, 3), is simulated
with 11% introduced perturbations to generate this plot.
3971

Page 4

include 500 genes and mean connectivity <k>, 3. Figure 3
shows that perturbation propagation probability increases
monotonically as a function of in-degree for k ≥ 2, both for
the simulated time-series data and the analytic prediction.
For k ≤ 20, the analytic approach slightly overestimates
(data are not shown), but two results are in good
agreement.
Figure 3 is a result from scale free networks that

by this analysis: does the observed distribution of PP allow
us infer the underlying topology of the given system?
The next section turns to the main question suggested
2.3 Gene Regulation Network Simulation

gene regulation network topology from perturbation data,
artificial data was generated using a gene regulation
network simulator. The simulator is written in C++ under
Unix and allows the simulation of Boolean networks with
any number of genes. The user specifies the mean input
degree <k> and whether a random or scale-free topology
should be generated.
In order to investigate the possibility of inferring the

applying design principles or methods [4, 8]. Random
pairing of existing nodes generates a random network
where the distribution of in-degree follows Poisson
distribution [8]. The relation between genes is directional,
and the probability to be an input gene at the given one
interaction is 1/2. Preferential attachment of newly
introduced nodes toward highly connected nodes generates
scale free networks [4]. For simplicity, we assumed that
newly introduced nodes are incapable of serving as an
input to already existing nodes in the system.
Networks of each topology are generated by

transition function that uses all k input variables in a non-
trivial way. (The simulator actually allows the generation
of other network topologies and classes of transfer
functions, but only the above features were used in this
study.)
Each gene is assigned a random Boolean state

generated
topology/connectivity/perturbation rates. The topology T
had values
The training data for the inference methods was
by considering 120 combinations of
T ∈ {random (exponential), scale-free}.
The in-degree mean connectivity had five possible values,
<k> ∈ {1, 2, 3, 4, 5}.
The perturbation rate took on the values
p ∈ {1%, 3%, ..., 21%, 40%}.
For each triple <T, <k>, p>, 20 networks were generated
with 500 genes each, perturbation were applied and the
resulting perturbation histograms were collected, as shown
in Figure 1. The histograms were discretized to 20 bins
(Figure 4). Although the size makes it difficult to
appreciate in these proceedings, Figure 4 shows that each
histogram is visually distinctive for each combination of
topology, mean input connectivity <k>, and perturbation
rate, especially when the perturbation rate is greater than
3%.

Figure 4. Perturbation propagation (PP) distributions from
two main classes of network topology: random network
and scale free network. Each histogram is a mean profile
from 20 times repeated simulations of one type of
topology network, Net (500, <k>). Each histogram is
characteristic to each triple <T, <k>, p>, topology, mean
input connectivity <k>, and percent perturbation. Rows
represent different mean connectivity, <k>, at given
topology. Columns represent different amount of
introduced perturbation.

this data, we investigated the sensitivity of the patterns of
histograms to N, the number of genes in the network, since
we desire an inference method that is independent of N.
So, with various N, the PP was compared for the given
various perturbations and k. As shown in Figure 5, the
dependency of perturbation propagation probability on N
is reasonably small when N is large (N≥100 is shown).
This implies that we may assume any large N for the
underlying system.
Before applying the neural net learning method to
3972

Page 5

Figure 5. Dependency of PP (perturbation propagation
probability) on system size (N). Dependencies are
examined under introduced 1%, 10%, and 20%
perturbation.

topology T from these reference histograms of PP
distribution.
We now proceed to investigate the inference of
2.4 Neural Net Classifiers

algorithms inspired by neurons and synapses in the brain.
In ANNs, processing elements (PEs) that are analogous to
neurons and weighted connections that are analogous to
synapses are fundamental building blocks. ANNs can be
trained to recognize and classify complex patterns [16]
and are widely used in various areas of the biomedical
sciences given the increase of biomedical data. The
biomedical data are being applied to pattern recognition or
decision-making in diverse fields including sequence
analysis [22], predicting mechanisms of drug action [20],
and clinical cancer diagnosis [14].
Artificial neural networks (ANNs) are computational

gene regulation networks, we developed feedforward
backpropagation ANNs such as that shown in Figure 6.
The networks has 20 inputs PEs, one for each bin in the
20-binned histogram of PP distribution, and one output
PE for the classification of type of topology either as a
random network topology or a scale-free network
topology. Between the inputs and outputs of the ANN,
there is one hidden layer that includes ten hidden PEs. As
shown in Figure 6, each PE in the hidden layer is
connected to all inputs and to one output.
To predict the features of underlying topology of

fold crossover validation method. The 2400 20-binned
histograms from the simulated data set were partitioned
The performance of ANN was evaluated by three-
into three equally sized groups. The first 1600 histograms
were used to train the neural network model over 800
backpropagation epochs, and then the other 800
histograms were used as a test set to calculate the
generalization capability of the classifier. These processes
were repeated using each of the three groups as a test set
in turn, and the prediction results obtained over the three
runs. To get average sensitivity and specificity, the 2400
histograms were then randomized and partitioned again to
repeat entire three-fold cross validation process 201 times,
resulting in 603 (3×201) trained models. Matlab Neural
Net toolbox (version 6) [7] on SGI Origin 3400
supercomputer was used for the neural net training and
simulation.

…
…
Bias
Bias
23 44
21
19
7
0
1
0
0
0.95
0.12
Random network
Scale free network 61
Random network
Scale free network
1
0
Random network
Scale free network
20 Input PEs (20-binned histogram of PP distribution)
Ten hidden layer
One output PEs
(Class of topology)
0
0

Figure 6. Architecture of ANN used to predict underlying
topology of gene regulation networks.
3 Results

topology are simulated to generate perturbed dynamic
data. Neural networks are used to predict the topology
class from histograms of perturbation propagation data.
This procedure is illustrated in Figure 1.
Gene regulation networks of various complexity and

iterations are shown in Figure 7. In nearly every case, the
mean squared error falls below 0.5 within 100 epochs.
The results of training and testing during training

network topology are summarized in Table 1. Since only
two classes are discriminated, sensitivity and specificity
display symmetric information.
classification, classification as a random network shows
higher sensitivity than classification as a scale free
network. Overall, around 90% accuracy is achieved.
The results of classification as a random or scale free
In this binary
3973