Question

# Can neural networks be used to derive formulas? Are genetic algorithms efficient in doing so?

If we have a data set of different values and we would like to derive a formula that is almost working with all of them. For example f(x) = x^2 or similar. Can this be done using neural networks. Through what I know, neural networks will produce output after performing computation by the hidden layers, but is it possible to use ANN to get an explicit candidate formula? What about genetic programming and/or genetic algorithms? Are there other heuristics or models used to reason and/or derive mathematical (or logical) expressions from data sets?

@luqman Abidoye, I'm interested to read the paper as mentioned earlier, kindly email me the paper at ashashwat.sharad.mec16@iitbhu.ac.in

27th Apr, 2014
Stephane Lallee
Institute for Infocomm Research
Basically you already know the "equation" encoded by your ANN after training. Considering that Y is the output of your network, X the input then ANN are following :
Yj = f( SUM(XiWij) ) with Wij being the connection weight between those two (eventually distributed over several layers) and f() being the activation function of your neuron. Through learning the ANN is just finding the coefficient of your equation in the form of weights.
If you want to go for the litteral approach you have to tune your learning algorithm so that it can deal with the symbols you are using. If what you want to achieve is to give numerical samples to an ANN and then derive the litteral equation of the function that the network learnt, either you do that analytically through development of you network computation (the big equation with weights as coefficients) or you face a very difficult problem. One way to achieve this automatic "literralization" of an arbitrary neural network would be to teach a separate network how to label know functions, ie:
- train a network A to learn a function f(x) that you now in advance, let say Y=X²
- train a second network B using as input the weights of A and as output the litteral expression of f(x)
- repeat the process until B has learnt several functions
Although that would probably imply that you need to find a way to make B as a recursive network that can output a function as a composition of other functions...
Genetic algorithm seem an easier approach, but you are dealing with a difficult issue ;)
10 Recommendations

25th Apr, 2014
Sudhakar Mishra
Indian Institute of Information Technology Allahabad
I think it is not possible because neural network perform two kind of tasks first regression and second classification , but deriving a formula to satisfy some input - output relation is a matter of arranging operators and operands not weight adjustment. Theoretically like our brain does derive formula for specific patterns, like situation i never encountered practically. So my suggestion is no.
1 Recommendation
It is possible to obtain an equation after developing an ANN for prediction. If fact, you will end up with a long equation including the inputs, weights, biases, etc.
9 Recommendations
25th Apr, 2014
Benyamin Khoshnevisan
University of Southern Denmark
After the network is well trained you have weights and biases by which you can write the formula.
4 Recommendations
As was mentioned before by Dr. Ahmadi-Nedushan you will be end up with a very long equation (useless!). Accordingly we leave the results as matrices. If you want a compact equation I suggest you try GP instead.
3 Recommendations
Every formula can be written as a binary tree, with (binary) operators
at the inner nodes and constants at the leaves. Usually genetic algorithms
are taught with genomes as linear lists, but it's also possible to have
genomes as trees.The most important operation is the crossover mutation.
It can be defined by exchanging sub-trees.
Not only should curve fitting performance serve as fitness criterion
but also size of the tree (length of the formula). It would be a combination
of two optimization goals.
Regards,
Joachim
2 Recommendations
25th Apr, 2014
National Council for Scientific Research, Lebanon
Hi everyone. I believe one way of creating equations from ANN is first to take the input data with target data and train ANN to create a model which create a relationship between the input and target data. Based on the output predicted by ANN one can create an equation based on the original input and predicted output. The equation is a curve fitting process which can be very complicated equation sometime.
Thank you for the great answers! The problem is predicting logical connectives in fuzzy logics, so from your answers, I think genetic programming would be more appropriate due to producing compact results.
Best regards,
Islam Deleted profile
You can create a NN with a genetic algorithm whose "DNA" codes for the NN architecture (neural activation functions, connection weights and biases). Then, through mutations and cross-overs you evolve the genome to minimize a cost function (e.g. mean squared error).
I used this approach to learn the functional relationship between temperature, ionic abundance, and line emission ratio indicators in galactic planetary nebulae/HII regions:
There is a more explanatory description of the process also in a PPT talk I gave here:
Hope this helps get you started.
Cheers,
Mike
1 Recommendation Deleted profile
After training the Neural Network (NN), you will end up with weights and biases. You can develop a model using the values of weights and biases. I agree with what Behrouz Ahmadi-Nedushan already said, "you will end up with a long equation including the inputs, weights, biases, etc".
However, Genetic Programming (GP) is a better option compare to NN. Genetic programming gives you an explicit arithmetic equation, that describes the relationship between the given input-output data-set. Moreover, by looking at the evolved equation, you can also understand the importance of each of the input variables. Thus, GP also automatically determines important input variables during evolutionary process and only those will be used in the final equation.
2 Recommendations
25th Apr, 2014
Marek Otahal
Czech Technical University in Prague
@down-votes: it's a voting mechanism, doesn't say some answer is incorrect, just some are better than others. (of course this must not be done systematically based on the poster, but based on quality of the answer)
1 Recommendation
26th Apr, 2014
Vivek Gupta
Indian Institute of Technology Mandi
ANN is basically nonlinear regression model. so after training the data set you will be having some weights and bias. by using theses weights you can develop regression formula
27th Apr, 2014
Stephane Lallee
Institute for Infocomm Research
Basically you already know the "equation" encoded by your ANN after training. Considering that Y is the output of your network, X the input then ANN are following :
Yj = f( SUM(XiWij) ) with Wij being the connection weight between those two (eventually distributed over several layers) and f() being the activation function of your neuron. Through learning the ANN is just finding the coefficient of your equation in the form of weights.
If you want to go for the litteral approach you have to tune your learning algorithm so that it can deal with the symbols you are using. If what you want to achieve is to give numerical samples to an ANN and then derive the litteral equation of the function that the network learnt, either you do that analytically through development of you network computation (the big equation with weights as coefficients) or you face a very difficult problem. One way to achieve this automatic "literralization" of an arbitrary neural network would be to teach a separate network how to label know functions, ie:
- train a network A to learn a function f(x) that you now in advance, let say Y=X²
- train a second network B using as input the weights of A and as output the litteral expression of f(x)
- repeat the process until B has learnt several functions
Although that would probably imply that you need to find a way to make B as a recursive network that can output a function as a composition of other functions...
Genetic algorithm seem an easier approach, but you are dealing with a difficult issue ;)
10 Recommendations
.
you might be interested in reading the attached overview describing how Megaputer's Polyanalist does it
.
Thank you all for your contributions!
1 Recommendation
Are you looking for something like this http://www.nutonian.com/products/eureqa/ ?
You can use gene expression programming for deriving formulas between input and output data.
1 Recommendation
Yes. You can derive a physical equation from ANN simulation based on weights and biases. In order to do this, you can check our journal: Abidoye and Mahdi (2014). Or you can mail me for a more explicit  paper that teaches the procedure. (kluqman2002@yahoo.co.uk)
1 Recommendation
22nd Jan, 2017
National Institute of Technology, Kurukshetra
@Abidoye hello I am also working for forecasting and want to develop equation by train In network. Kindly give a procedure with a suitable example. Thanku
@Luqman Abidoye Kindly mail me the same :) (shrutisneha2011@gmail.com)
18th Jul, 2018
Fahime Naghavi
Babol Noshirvani University of Technology
i also have respnses of the displacement in top node in structure obtained from opensees finite element that varies by using two random variable fy and E. I have properties of mean and stdv of random variable. how to predict relation of responses and random variable in form of explicit function. Can i use ann or ga?
18th Jul, 2018
Nor S Sani
Universiti Kebangsaan Malaysia
@luqman Abidoye, I'm interested to read the paper as mentioned earlier, kindly email me the paper at norsamsiasani@gmail.com
31st Oct, 2018
Ravish .H
National Institute of Mental Health and Neuro Sciences
@luqman abidoye , please mail me at neuroravish@gmail.com
31st Oct, 2018
Nor S Sani
Universiti Kebangsaan Malaysia
@luqman Abidoye, I'm interested to read the paper as mentioned earlier, kindly email me the paper at norsamsiahsani@gmail.com
@luqman Abidoye, Kindly also email me yours journal on this id iffy1487@gmail.com.....Thanking in anticipation
When the problem is formulated in a symbolic way, it is better Genetic programming.
1 Recommendation
28th May, 2019
Fahime Naghavi
Babol Noshirvani University of Technology
@luqman abidoye , please mail me at fahime_naghavi@yahoo.com
There are also some approaches based on three graphs
1 Recommendation

## Related Publications

Article
Evolutionary algorithms incorporate principles from biological population genetics to perform search, optimization, and learning. This article discusses issues arising in the application of evolutionary algorithms to problems in robotics. INTRODUCTION The field of robotics offers an endless supply of difficult problems, requiring an equally impress...
Conference Paper
Many games require opponent modeling for optimal performance. The implicit learning and adaptive nature of evolutionary computation techniques offer a natural way to develop and explore models of an opponent's strategy without significant overhead. In this paper, we compare two learning techniques for strategy development in the game of Spoof, a si...
Conference Paper
Full-text available
Neuroevolution, the application of evolutionary algorithms to artificial neural networks (ANNs), is well-established in machine learning. Cartesian Genetic Programming (CGP) is a graph-based form of Genetic Programming which can easily represent ANNs. Cartesian Genetic Programming encoded ANNs (CGPANNs) can evolve every aspect of an ANN: weights, t...
Got a technical question?