An overview of Boltzmann Machine and its special class

Abstract

Boltzmann machines are stochastic and gererative
neural networks that are capable of learning internal representations to solve difficult combinatoric problems and are the
basis of the early optimization techniques used in artificial neural
networks. Boltzmann Machine was invented by Geoffrey Hinton
and Terry Sejnowski in 1985. They comprises of only two types
of nodes which are hidden and visible nodes.They do not have
output nodes because they are non-deterministic. In this paper
we are going to discuss about Boltzmann machine and it’s special
class called Restricted Boltzmann Machine (two-layer two layered
artificial neural network) with an illustrated example.

Full text

An overview of Boltzmann Machine and its special
class
*
1st Anjali Patel
High Integrity Systems - Computer Science
Frankfurt University of Applied Sciences
Frankfurt am Main , Germany
anjal.patel@stud.fra-uas.de or Matriculation - 1322635
2nd Ranjith Kumar Rama
High Integrity Systems - Computer Science
Frankfurt University of Applied Sciences
Frankfurt am Main , Germany
ranjith.rama@stud.fra-uas.de, Matriculation - 1322172
Abstract—Boltzmann machines are stochastic and gererative
neural networks that are capable of learning internal repre-
sentations to solve difficult combinatoric problems and are the
basis of the early optimization techniques used in artificial neural
networks. Boltzmann Machine was invented by Geoffrey Hinton
and Terry Sejnowski in 1985. They comprises of only two types
of nodes which are hidden and visible nodes.They do not have
output nodes because they are non-deterministic. In this paper
we are going to discuss about Boltzmann machine and it’s special
class called Restricted Boltzmann Machine (two-layer two layered
artificial neural network) with an illustrated example.
Index Terms—Boltzmann machine, stochastic, neural networks
, combinatoric problems, optimization techniques, Restricted
Boltzmann Machine
I. INTRODUCTION
This paper presents “Boltzmann Machine” which is also
referred as a constraint satisfaction network. It has capability
to learn the underlying constraints that characterize a domain.
The network modifies the strengths of its connections to
construct an internal generative model [6].
A Graphical representation of Boltzmann machine is shown
in Fig. 1 where each and every node are connected as like
mesh topology in network. Nodes are distributed in 2 category.
Firstly, visible nodes which will work as our input nodes and
rest nodes are our hidden nodes which will represent features
of our input.
To learn structure of network is often begin with assump-
tions that networks are randomly wired. The view that all
knowledge is innate seems to us to be just as wrong. If there
are connectivity structures that are good for particular tasks
that the network will have to perform, building these in at the
start is much more efficient. But not all tasks can be foreseen,
and fine-tuning may still be helpful even for those that can
[6].
The Boltzmann Machine is a parallel computational organi-
zation that is well suited to restrict tasks of satisfaction involv-
ing many ”weak” constraints. Constraint-satisfaction searches
Fig. 1. A graphical representation of an example Boltzmann machine. Each
undirected edges represents dependency. In this example there are 3 hidden
units and 4 visible units [5].
(e.g., Waltz, 1975; Winston, 1984) normally employ ”strong”
constraints that any solution must satisfy. For example, in
problematic domains like games and puzzles, the goal criteria
often have this character, so the rule is strong constraints.
In some problem areas, such as finding the most plausible
interpretation of an image, many of the criteria are not all-
or-none and often even the best solution breaches certain
constraints. A variation which is more suitable for such do-
mains uses weak constraints which incur costs when violated.
After that, the quality of a solution is determined by the
total cost of all the constraints it violates. For example, in
a perceptual interpretation task, that total cost should reflect
the interpretation’s implausibility [6].
The machine consists of primitive computing elements
called units which are connected by bidirectional connections.

A unit is always, on or off, in one of two states and adopts
these states as a probabilistic function of the states of its
neighboring units and the weights on their connections to
them. Weights of either sign can take on real values. A unit on
or off is taken to mean that some elementary hypothesis about
the domain is currently accepted or rejected by the system. Be-
tween two hypotheses, the weight on a link represents a weak
pair-limit. A positive weight indicates that the two hypotheses
tend to support one another; if at present one is accepted,
it should be more likely to accept the other. In contrast , a
negative weight suggests that the two hypotheses should not
both be accepted, other things being equal. Link weights are
symmetrical, have the same force in both directions. [6]
Mainly, two quite different computational problems are
solved with Boltzmann machines.
The weights on the connections are fixed for a search
problem, and are used to represent the cost function of an op-
timization issue. A Boltzmann machine’s stochastic dynamics
then allow it to sample binary state vectors that represent good
solutions to the problem of optimization [7].
A set of binary data vectors is shown to the Boltzmann
machine for a learning problem, and it must find weights on
the connections so that the data vectors are good solutions to
the optimization problem defined by those weights. Boltzmann
machines make many small updates to their weights to solve
a learning problem and every update requires them to solve a
lot of different search issues [7].
A. Energy of a global configuration
The resulting structure is related to a system described by
Hopfield (1982), and as in its system , a single number, called
that state’s ”energy,” can be assigned to each global state of
the network. The individual units can be made to act with the
right assumptions so as to minimize the global energy. If some
of the units are forced outwardly or ”clamped” into specific
states to represent a specific input, the system will then find
the minimum energy configuration compatible with that input
[6].
The energy of a configuration can be interpreted as the
extent to which that combination of hypotheses violates the
constraints implied in the problem domain, thus the sys-
tem evolves towards ”interpretations” of that input which
increasingly satisfy the constraints of the problem domain in
minimizing energy [6].
The energy of a global configuration is defined as
E=−(X
i<i
wij sisj+X
i
θisi)(1)
In Eq. 1 Where wij is the strength of connection between
units iand j,siis 1 if unit iis on otherwise it is 0, θiis
the bias of the unit iin the global energy function(−θiis the
activation threshold for the unit) [6].
B. Minimizing Energy
A simple algorithm for finding a combination of truth values
that is a local minimum is to switch each hypothesis into
whichever of its two states yields the lower total energy given
the current states of the other hypotheses. If hardware units
make their decisions asynchronously, and if transmission times
are negligible, then the system always settles into a local
energy minimum [6].
The connections are symmetrical, the difference between the
energy of the whole system with the rejected kth hypothesis
and its energy with the accepted kth hypothesis can be
determined locally by the kth unit, and this ”energy gap” is
shown in Eq. 2,
∆Ek=X
i
wkisi−θk(2)
Therefore, the rule for minimizing the energy contributed
by a unit is to adopt the on state if its total input from the
other units and from outside the system exceeds its threshold.
This is the familiar rule for binary threshold units.
Terms of threshold may be removed from Eq. 1 and 2
commenting as follows: the effect of θion the global energy
or on the energy gap of an individual unit is identical to the
effect of a link with strength — θibetween unit iand a special
unit that is by definition always held in the on state. This
”true unit” need have no physical reality, but it simplifies the
computations by allowing the threshold of a unit to be treated
in the same manner as the links. The value — θiis called the
bias of unit i[6].
If a permanently active ”true unit” is assumed to be part of
every network, then Eq. 1 and 2 can be written as, respectively:
E=−(X
i<i
wij sisj)(3)
∆Ek=X
i
wkisi(4)
C. Using Noise to Escape from Local Minima
Gradient descent methods is a simple and deterministic
algorithm which has one con that sysyem gets trapped in local
minima that are not global optimal solutions; as shown in Fig.
2. For constraint satisfaction tasks, is to find the configuration
that is the global minimum given the current input and get out
of local minima [6].
Make jump to configurations of higher energy is the easy
option to escape from local minima. If the energy gap between
the on and off states of the kth unit is ∆Ekthen regardless
of the previous state set sk=1 with probability
Pk=1
1 + e−∆Ek/T (5)
where, T is parameter that act like a Temperature [6].

Fig. 2. Concept of Local minima and Global Minima [8]
D. Learning in Boltzmann Machines
If the units are updated sequentially in any order that does
not depend on their total inputs, the network will eventually
reach a Boltzmann distribution (also called its equilibrium
or stationary distribution) in which the probability of a state
vector, v, is determined solely by the “energy” of that state
vector relative to the energies of all possible binary state
vectors: [7]
P(v) = e−E(v))
Pue−E(v)(6)
The energy of state vector v is defined as,
E(v) = −(X
i<i
wij sv
isv
j+X
i
θisv
i)(7)
Where, sv
iis the binary state assigned to unit iby state
vector v. Because of a training set of state vectors (the data),
learning is about finding weights and biases (the parameters)
that make those state vectors good. More specifically, the
goal is to find weights and biases that fine a distribution of
Boltzmann in which the training vectors are highly likely [7].
By differentiating Eq. 6 and using the fact Eq. 8
∂E(v)
∂wi j
=−sv
isv
j(8)
it can be shown that
X
v∈data
∂log P(v)
∂wi j
=< si, sj>d at a−< si, sj>M o de l (9)
where < si, sj>d at a is the expected value of si, sjin the
data distribution and < si, sj>M od el is the expected value
when the Boltzmann machine is sampling state vectors from
its equilibrium distribution at a temperature 1 [7].
To perform gradient ascent in the log probability that the
Boltzmann machine would generate the observed data when
sampling from its equilibrium distribution, wij is incremented
by a small learning rate times the RHS of Eq. 9. The learning
rule for the bias, bi, is the same as a above equation, but with
sjis omitted [7].
If the observed data specifies a binary state for every unit in
the Boltzmann machine, the learning problem is convex: There
are no non-global optima in the parameter space. Learning
becomes much more interesting if the Boltzmann machine
consists of some “visible” units, whose states can be observed,
and some “hidden” units whose states are not specified by the
observed data [7].
The hidden units act as latent variables (features) allowing
Machine Boltzmann to model the distributions over visible
state vectors , Direct pair-time interactions between the visible
units can not be modelled. Even with hidden units, a surprising
property of Boltzmann machines is the rule of Learning
remains the same. This enables the learning of binary features
capturing higher-order structure in the data. With hidden
units, the expectation < si, sj>d at a is the average of the
expected value si, sjover all data vectors when a data vector
is clamped onto the visible units and the hidden units are
updated repeatedly until they reach an equilibrium with the
clamped data vector [7].
It is surprising that the rule of learning is so simple, because
∂log P(V)
∂wi j depends on all the other weights in the network.
Fortunately, the difference in the two correlations in Eq. 9
informs wij everything it needs to know about the other
weights. This makes it unnecessary to explicitly propagate
error derivatives, as in the backpropagation algorithm [7].
E. Problem
The Boltzmann machine is, theoretically, a rather general
media of computation. For example, if trained on photographs,
the machine would theoretically model the photograph dis-
tribution, and could use that model to complete a partial
photograph, for example [5].
Practically it’s a problem, when the machine is scaled
up to anything larger than a trivial size it seems to stop
learning correctly. This is because of the significant effects,
in particular: The time order required to collect equilibrium
statistics grows exponentially with the size of the machine
and the magnitude of the strengths of the connections [5].
Connection strengths are more plastic when the connected
units have intermediate probabilities of activation between zero
and one which results in a so-called variance trap. The net
effect is that noise causes the strengths of the connection to
follow a random walk until the activities become saturated [5].
II. RESTRICTED BOLTZMANN MACHINE
”The learning algorithm in Boltzmann machine is very slow
with many layers of feature detectors, but it is fast in re-
stricted Boltzmann machines that have a single layer of feature
detectors. Many hidden layers can be learned efficiently by
composing restricted Boltzmann machines, using the feature
activations of one as the training data for the next” [1].

A. Architecture
”Restricted Boltzmann machines(RBM’s) are a special class
of Boltzmann machines, comprises of a two layered artificial
neural network which have capabilities to learn probability
distribution over a set of input and they have restricted the
connections between the visible and hidden nodes. RBMs
were invented by Geoffrey Hinton and they can be useful for
dimensionality reduction, classification, regression, collabra-
tive filtering, feature learning, and topic modeling. Compared
to general class of Boltzmann machines RBM’s are easy to
implement as they don’t have intra-layer connections between
the same kind of nodes. As shown in Fig. 3, every node in
visible layer is connected to every node in the hidden layer.
There are no connections from visible to visible or hidden
nodes. This kind of structure enables RBM’s to be more
efficient than the general class of Boltzmann machines” [2].
Fig. 3. Structure of Restricted Boltzmann machine [10]
RBM’s are represented as symmetric bipartite graph because
the nodes in intra-layer are not connected together as shown in
Fig. 3. Stacking multiple RBM’s and fine-tuning them through
the process of gradient descent and back-propagation, a Deep
Belief Network is formed [2].
B. Working of RBM’s
Each neuron in RBM’s have random behavior when acti-
vated. There is a hidden and visible bias units in RBM. Every
visible node in RBM’s takes a low-level feature from a single
data in a dataset. Consider a dataset containing grayscale
images, then each visible node in RBM would receive one
pixel value for each pixel in one image. For example, MNIST
images have 784 pixels, RBM should have 784 input nodes
present on visible layer to process [4].
When a single pixel value x, of the grayscale image is
passed as an input to a single visible node as shown in Fig. 4,
x is multiplied by a weight and added to a bias at the node 1
of the hidden layer. The result of this operation is fed into the
activation function shown below produces the node’s output
or the strength of the signal passing through it. The activation
function is activation f((weight w * input x) + bias b ) =
output a [4].
When several inputs at visible nodes are passed and com-
bined through one hidden node, then the input of each visible
Fig. 4. One input path of RBM [4]
Fig. 5. Weighted inputs combine at hidden node [4]
node x is multiplied by a separate weight as shown in Fig.5,
the products are summed and added to a bias. The results are
then passed through an activation function to produce node’s
output [4].
In Fig. 6 each node at hidden layer, the each input x is then
multiplied by their respective weight w. These weights form a
matrix where rows are represented as input nodes and columns
are represented as output nodes. Then the multiplied product
is added with the bias and passed through activation function
to produce output [2] [4].
”In the reconstruction phase, the activations of hidden layer
no. 1 become the input in a backward pass. They are multiplied
by the same weights, one per internode edge, just as x
was weight-adjusted on the forward pass. The sum of those
products is added to a visible-layer bias at each visible node,
and the output of those operations is a reconstruction; i.e. an
approximation of the original input ” [4]. Reconstruction is
shown in the Fig.7

Fig. 6. Multiple inputs at visible nodes [4]
Fig. 7. Reconstruction of RBM’s to minimize the error [4]
In Reconstruction phase, the weights are randomly initial-
ized and the difference between reconstruction ras shown in
Fig. 7 and the original data is large and this difference error
is then backpropagated against the weights until the error gets
minimum [4].
Reconstruction is different from regression that estimates a
continuous value based on inputs which makes the decision
about which discrete label to apply for the given set of input.
Reconstruction is used to determine the probability distribution
of the original input [4].
RBM’s use Kullback Leibler(KL) Divergence to measure
the distance between estimated probability distribution and
distribution of the input. ”KL-Divergence measures the non-
overlapping, or diverging, areas under the two curves, and an
RBM’s optimization algorithm attempts to minimize those ar-
eas so that the shared weights, when multiplied by activations
of hidden layer one, produce a close approximation of the
original input” [4]. Fig 8 left: shows probability distribution,
Right: shows differences using KL-Divergence.
RBM learns to approximate the original data by iteratively
adusting the weights to reduce the error. Fig. 9 shows that
Fig. 8. left: shows probability distribution of a set of original input pwith
the reconstructed distribution q, Right: shows integration of their differences
using KL-Divergence [4]
adjusting the weights that the distributions q(x)slowly reflect
to the structure of original input that is, p(x)[4].
Fig. 9. shows how RBM’s adjust the weights to reduce the difference between
the original input and reconstructed distribution [4]
C. An energy based model
A joint configuration, (v, h) of the visible and hidden units
has an energy (Hopfield, 1982) given by [9]:
E(v, h) = X
i∈visible
aivi−X
j∈hidden
bjhj−X
i,j
vihjwij (10)
where vi,hjare the binary states of visible unit iand hidden
unit j,ai,bjare their biases and wij is the weight between
them. The energy function depends on the configurations of
input states, hidden states, weights and biases [9].
D. A probabilistic model
”An RBM represents a probability distribution where low-
energy configurations have higher probability” [3]. RBM only
assigns probabilities rather than discrete values. The RBM
assigns a probability to every possible pair of a visible and
a hidden vector via this energy function [9]:
p(v, h) = 1
Ze−E(v, h)(11)

Here Z is called the ‘partition function’ which gives the
summation of over all possible pairs of visible and hidden
vectors [9].
Z=X
v,h
e−E(v, h)(12)
Boltzmann Distribution is also known as joint distribution
in physics which gives the probability with which a particle
can be observed in the state with the energy E. It is understood
that calculating joint probability to large combination of v and
h is very difficult. Contrast to joint probabilities, conditional
probabilities of states v and h can be calculated easily. It is
give by [10]:
p(h|v) = Y
i
p(hi|v)
P(v|h) = Y
i
p(vi|h)(13)
Each node in RBM’s can have only binary state that is, 0
or 1. If a visible or hidden layer is said to be active then it is
in state 1. Given an input vector v the probability for a single
hidden neuron jbeing activated is [10]:
p(hj= 1|v) = 1
1 + e(−(bj+Wjvi))
=σ(bj+X
i
viwij )(14)
σis called the sigmoid function. Eq. 14 is obtained by
applying Bayes rule to Eq. 13. Analogous to Eq. 14, given
an hidden vector h the probability for a single visible neuron
ibeing activated is [10]:
p(vi= 1|h) = 1
1 + e(−(ai+Wihj))
=σ(ai+X
j
hjwij )(15)
E. Training of RBM’s
The training of RBM’s differs from training of regular
neural networks. In this paper we are going to discuss about
two main training steps that is, Gibbs sampling and Contrastive
Divergence.
1) Gibbs Sampling: In this method, Eq. 14 and Eq. 15 are
used to predict respective hidden and new input values are
predicted.
Fig 10 shows when a input vector v0is known then the
corresponding hidden values is calculated using the Eq. 14.
Then the next input value v1is calculated by using the
previously obtained h value in the Eq. 15. This process is
repeated for k times until we obtain value of vkvector, which
is obtained by repeated steps by passing the original vector v0
value [10].
Fig. 10. Predicting the new input and hidden values [10]
2) Contrastive Divergence: Fig. 6 shows how the weight
matrix are formed. The vectors vkand hkare calculated using
the Eq.14 and Eq. 15 respectively. The new weight matrix is
updated during Contrastive Divergence. The updated matrix is
the difference between the outer products of those probabilities
with input vectors v0and vk, given by [10].
∆W=v0⊗p(h0|v0)−Vk⊗p(hk|vk)(16)
With the updated matrix, new respective weights can be
calculated with gradient ascent, given by [10]:
Wne w =Wol d + ∆W(17)
The K-step Contrastive Divergence algorithm is shown
below .
Fig. 11. K-step Contrastive Divergence [11]
”The idea of k-step Contrastive Divergence Learning(CD-
k) is: Instead of approximating the second term in the log-
likelihood gradient by a sample from the RBM-distribution
(which would require to run a Markov chain until the station-
ary distribution is reached), a Gibbs chain is run for only k
steps (and usually k = 1)” [11].
”The Gibbs chain is initialized with a training example v(0)
of the Training set and yields the sample v(k) after k steps.
Each step t consists of sampling h(t) from p(h—v(t)) and
sampling v(t+1) from p(v—h(t)) subsequently. The gradient
w.r.t. θof the log-likelihood for one training pattern v(0) is
then approximated by” [11]:

CDk(θ , v(0) ) = −X
h
p(h|v(0 ))∂E(v( 0) , h)
∂θ
+X
h
p(h|v(k))∂E(v(k), h)
∂θ
(18)
F. Illustration : Collaborative Filtering with RBM
There are two following steps involved:
1) Recognizing latent factors in the data: ”Lets assume
some people were asked to rate a set of movies on a scale
of 1–5 stars. In classical factor analysis each movie could be
explained in terms of a set of latent factors.
Fig. 12. Identification of latent factors [10]
For example, movies like Harry Potter and Fast and the
Furious might have strong associations with a latent factors
of fantasy and action. On the other hand users who like Toy
Story and Wall-E might have strong associations with latent
Pixar factor. RBMs are used to analyse and find out these
underlying factors. After some epochs of the training phase
the neural network has seen all ratings in the training date set
of each user multiply times. At this time the model should
have learned the underlying hidden factors based on users
preferences and corresponding collaborative movie tastes of
all users. The analysis of hidden factors is performed in a
binary way. Instead of giving the model user ratings that are
continues (e.g. 1–5 stars), the user simply tell if they liked
(rating 1) a specific movie or not (rating 0). The binary rating
values represent the inputs for the input/visible layer. Given
the inputs the RMB then tries to discover latent factors in the
data that can explain the movie choices. Each hidden neuron
represents one of the latent factors. Given a large dataset
consisting out of thousands of movies it is quite certain that
a user watched and rated only a small amount of those. It is
necessary to give yet unrated movies also a value, e.g. -1.0
so that the network can identify the unrated movies during
training time and ignore the weights associated with them.
Lets consider the following example where a user likes Lord
of the Rings and Harry Potter but does not like The Matrix,
Fight Club and Titanic. The Hobbit has not been seen yet so
it gets a -1 rating. Given these inputs the Boltzmann Machine
may identify three hidden factors Drama, Fantasy and Science
Fiction which correspond to the movie genres” [10].
”Given the movies the RMB assigns a probability p(h|v)
for each hidden neuron. The final binary values of the
neurons are obtained by sampling from Bernoulli distribution
using the probability p. In this example only the hidden
neuron that represents the genre Fantasy becomes activate.
Given the movie ratings the Restricted Boltzmann Machine
recognized correctly that the user likes Fantasy the most” [10].
2) Using latent factors for prediction: ”After the training
phase the goal is to predict a binary rating for the movies that
had not been seen yet. Given the training data of a specific
user the network is able to identify the latent factors based on
this users preference. Since the latent factors are represented
by the hidden neurons we can use p(v|h)and sample from
Bernoulli distribution to find out which of the visible neurons
now become active” [10].
Fig. 13. Using hidden neurons for the inference to find the active neurons
[10]
Fig. 13 ”shows the new ratings after using the hidden neuron
values for the inference. The network did identified Fantasy
as the preferred movie genre and rated The Hobbit as a movie
the user would like” [10].
The summary of the above illustrated prediction phase is as
follows :
•”Train the network on the data of all users
•During inference time take the training data of a specific
user
•Use this data to obtain the activation of hidden neurons
•Use the hidden neuron values to get the activation of input
neurons
•The new values of input neurons show the rating the user
would give yet unseen movies” [10].
G. Advantages of RBM’s
•RBM’s are used in applications such as dimensionality
reduction, classification, collaborative filtering, feature
learning, topic modelling [12].
•Restricted Boltzmann machine is used for neuroimaging,
Sparse image reconstruction in mine planning and also
in Radar target recognition [13].

•RBM able to solve imbalanced data problem by SMOTE
procedure [13].
•RBM find mis sing values by Gibb’s sampling which is
applied to cover the unknown values [13].
•RBM overcomes the problem of noisy labels by uncor-
rected label data and its reconstruction errors [13].
•The problem of unstructured data is rectified by feature
extractor which transforms the raw data into hidden units
[13].
III. CONCLUSION
Symmetric weight systems form an interesting computa-
tional device class because their dynamics are governed by
an energy function. This is what makes it possible to analyze
their behavior and use them for satisfaction with iterative
constraints. With intention to mitigates cons of Boltzmann
machine, with some limitations RBM is introduced.
In recent spectacular developments in neural network mod-
els and deep learning, RBM has played an important role.
RBM is a generative model which extracts the patterns from
the input data by modifying weights to minimize the error
while generating [3]. Salakhutdinov, R. R., Mnih, A., and
Hinton showed that RBM’s can be successfully applied to
large dataset containing over 100 million user/movie ratings
[14]. They have been used in many applications such as
dimensionality reduction, classification, collabrative filtering,
feature learning, topic modelling [12], neuroimaging, Sparse
image reconstruction [13], etc.
In the RBM section of this paper, we discussed about
architecture of RBM’s where they are far efficient and easy
to implement comapred to Boltzmann machines. We dis-
cussed the working of RBM’s by using activation function,
which takes parameters such as weights, input data and bias
to produce output, in reconstruction phase, they use KL-
Divergence to measure the distance between estimated proba-
bility distribution and distribution of the input . We discussed
about energy model, probabilistic model where RBM’s only
assigns probabilities rather than discrete value. They can be
trained by Gibbs sampling, Contrastive Divergence (K-step
CD algorithm). We also discussed about the use of RBM’s in
collaborative filtering to find the latent factors in data and to
predict the active neurons.
REFERENCES
[1] Hinton, Geoffrey E. (2007-05-24). ”Boltzmann machine”.
Scholarpedia. 2 (5): 1668. Bibcode:2007SchpJ...2.1668H.
doi:10.4249/scholarpedia.1668. ISSN 1941-6016.
[2] Restricted Boltzmann Machines — Simplified,
https://towardsdatascience.com/restricted-boltzmann-machines-
simplified-eab1e5878976, 2018, Accessed August 16, 2020.
[3] Upadhya, V., Sastry, P.S. An Overview of Restricted
Boltzmann Machines. J Indian Inst Sci 99, 225–236 (2019).
https://doi.org/10.1007/s41745-019-0102-z
[4] A Beginner’s Guide to Restricted Boltzmann Ma-
chines, https://web.archive.org/web/20170211042953/
https://deeplearning4j.org/restrictedboltzmannmachine.html, Archived
from the original on February 11, 2017. Accessed August 16, 2020.
[5] Boltzmann machine, https://en.wikipedia.org/wiki/Boltzmann machine,
Accessed September 1, 2020.
[6] David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski, “A
Learning Algorithm for Boltzmann Machines”, Cognitive Science 9 147-
169(1985).
[7] Georey E. Hinton,” Boltzmann Machines”, March 25, 2007
[8] What are Local Minima and Global Minima in Gradient, Descent
https://www.i2tutorials.com/what-are-local-minima-and-global-minima-
in-gradient-descent/, Accessed September 2, 2020.
[9] G. Hinton. A practical guide to training restricted Boltzmann machines.
Technical Report UTML TR 2010003, Department of Computer Sci-
ence, University of Toronto, 2010.
[10] Deep Learning meets Physics: Restricted Boltzmann Machines Part I,
https://towardsdatascience.com/deep-learning-meets-physics-restricted-
boltzmann-machines-part-i-6df5c4918c15, Accessed September 2,
2020.
[11] Boltzmann Machines — Transformation of Unsupervised Deep Learning
— Part 1, https://medium.com/@neuralnets/boltzmann-machines-
transformation-of-unsupervised-deep-learning-part-1-42659a74f530,
Accessed September 3, 2020.
[12] Restricted Boltzmann machine ,
https://en.wikipedia.org/wiki/Restricted Boltzmann machine, Accessed
on September 4, 2020.
[13] Restricted Boltzmann Machine, https://www.educba.com/restricted-
boltzmann-machine/, Accessed on September 5, 2020.
[14] Salakhutdinov, R. R., Mnih, A., and Hinton, G. E. (2007). Restricted
Boltzmann machines for collaborative filtering. In Ghahramani, Z., edi-
tor, Proceedings of the International Conference on Machine Learning,
volume 24, pages 791–798. ACM.