Copyright 2001, Society of Petroleum Engineers Inc.
This paper was prepared for presentation at the 2001 SPE Hydrocarbon Economics and
Evaluation Symposium, Dallas, TX, 2-3 April.
This paper was selected for presentation by an SPE Program Committee following review of
information contained in an abstract submitted by the author(s). Contents of the paper, as
presented, have not been reviewed by the Society of Petroleum Engineers and are subject to
correction by the author(s). The material, as presented, does not necessarily reflect any
position of the Society of Petroleum Engineers, its officers, or members. Papers presented at
SPE meetings are subject to publication review by Editorial Committees of the Society of
Petroleum Engineers. Electronic reproduction, distribution, or storage of any part of this paper
for commercial purposes without the written consent of the Society of Petroleum Engineers is
prohibited. Permission to reproduce in print is restricted to an abstract of not more than 300
words; illustrations may not be copied. The abstract must contain conspicuous
acknowledgment of where and by whom the paper was presented. Write Librarian, SPE, P.O.
Box 833836, Richardson, TX 75083-3836, U.S.A., fax 01-972-952-9435.
The industrial and residential market for natural gas produced
in the United States has become increasingly significant.
Within the past ten years the wellhead value of produced
natural gas has rivaled and sometimes exceeded the value of
crude oil. Forecasting natural gas supply is an economically
important and challenging endeavor. This paper presents a
new approach to predict natural gas production for the United
States using an artificial neural network.
We developed a neural network model to forecast U.S.
natural gas supply to the Year 2020. Our results indicate that
the U.S. will maintain its 1999 production of natural gas to
2001 after which production starts increasing. The network
model indicates that natural gas production will increase
during the period 2002 to 2012 on average rate of 0.5%/yr.
This increase rate will more than double for the period 2013 to
The neural network was developed with an initial large
pool of input parameters. The input pool included exploratory,
drilling, production, and econometric data. Preprocessing the
input data involved normalization and functional
transformation. Dimension reduction techniques and
sensitivity analysis of input variables were used to reduce
redundant and unimportant input parameters, and to simplify
the neural network. The remaining input parameters of the
reduced neural network included data of gas exploratory wells,
oil/gas exploratory wells, oil exploratory wells, gas depletion
rate, proved reserves, gas wellhead prices, and growth rate of
gross domestic product. The three-layer neural network was
successfully trained with yearly data starting from 1950 to
1989 using the quick-propagation learning algorithm. The
target output of the neural network is the production rate of
natural gas. The agreement between predicted and actual
production rates was excellent. A test set, not used to train the
network and containing data from 1990 to 1998, was used to
verify and validate the network performance for prediction.
Analysis of the test results shows that the neural network
approach provides an excellent match of actual gas production
data. An econometric approach, called stochastic modeling or
time series analysis, was used to develop forecasting models
for the neural network input parameters. A comparison of
forecasts between this study and other forecast is presented.
The neural network model has use as a short-term as well
as a long-term predictive tool of natural gas supply. The model
can also be used to examine quantitatively the effects of the
various physical and economic factors on future gas
In recent years, there has been a growing interest in applying
artificial neural networks
(NN) to various areas of science,
engineering, and finance. Among other applications to
petroleum engineering, NN’s have been used for pattern
recognition in well test interpretation
and for prediction in
Artificial neural networks are an information processing
technology inspired by the studies of the brain and nervous
system. In other words, they are computational models of
biological neural structures. Each NN generally consists of a
number of interconnected processing elements (PE) or neurons
grouped in layers. Fig. 1 shows the basic structure of a three-
layer network: one input layer, one hidden layer, and one
output layer. The neuron consists of multiple inputs and a
single output. Input is the values of the independent variables
and output is the dependent variables. Each input is modified
by a weight, which multiplies with the input value. The input
can be raw data or output of other PE’s or neurons. With
reference to a threshold value and activation function, the
neuron will combine these weighted inputs and use them to
determine its output. The output can be either the final product
or an input to another neuron.
This paper describes the methodology of developing an
artificial neural network model to predict U.S. natural gas
production. It presents the results of neural network modeling
approach and compares it to other modeling approaches.
Predicting Natural Gas Production Using Artificial Neural Network
S.M. Al-Fattah, SPE, Saudi Aramco, and R.A. Startzman, SPE, Texas A&M University
2 S.M. AL-FATTAH, R.A. STARTZMAN SPE 68593
The data used to develop the ANN model for U.S. gas
production were collected mostly from the Energy Information
U.S. marketed-gas production for the period
1918-97 was obtained from the Twentieth Century Petroleum
; the 1998 production data was from the EIA. Gas
discovery data from 1900 to 1998 were from Refs. 7 and 10.
Proved gas reserves for the period 1949 to 1999 was from the
Oil and Gas Journal (OGJ) database.
EIA provides various
statistics of historical data for U.S. energy including gas
production, exploration, drilling and econometrics. These data
are available for public and can be downloaded from the
internet with ease. The following data (1949 to 1998) were
downloaded from the EIA website.
• Gas discovery rate
• Gas wellhead price
• Oil wellhead price
• Gross domestic product (GDP) using purchasing power
parity (PPP), based on 1992 U.S. dollars
• Gas exploratory wells:
o Footage drilled
o Wells drilled
• Oil exploratory wells:
o Footage drilled
o Wells drilled
o Percentage of successful wells drilled
• Oil and gas exploratory wells:
o Footage drilled
o Wells drilled
• Proved gas reserves
Other input parameters were also derived from the data
parameters above. These derived input parameters include:
• GDP growth rate: measures the growth rate of the gross
domestic product. This input parameter was calculated using
the following formula
+ ii tt
, .. .. .…………... (1)
where GDP = gross domestic product, GDPGR = growth rate
of gross domestic product, t = time, and i = observation
• Average depth drilled per well: calculated by dividing the
footage drilled by the number of exploratory wells drilled each
year. This is done for the gas exploratory wells, oil
exploratory wells, and oil-and-gas exploratory wells, resulting
in additional three new input variables.
• Depletion rate: measures how fast the reserves are being
depleted each year at that year’s production rate. It is
calculated as the annual production divided by the proved
reserves expressed in percentage.
Data preparation is a critical procedure in the development of
an artificial neural network system. The preprocessing
procedures used in the construction process of the NN model
of this study are input/output normalization and
Normalization is a process of standardizing the possible
numerical range that the input data can take. It enhances the
fairness of training by preventing an input with large values
from swamping out another input that is equally important but
with smaller values. Normalization is also recommended
because the network training parameters can be tuned for a
given range of input data; thus, the training process can be
carried over to similar tasks.
We used the mean/standard deviation normalization
method to normalize all the input and output variables of the
neural network. The mean standard deviation preprocessing is
the most commonly used method and generally works well
with almost every case. It has the advantages that it processes
the input variable without any loss of information and its
transform is mathematically reversible. Each input variable as
well as the output were normalized using the following
, ……………………………… (2)
= normalized input/output vector, X = original
= mean of the original input/output,
standard deviation of the input/output vector, and i = number
of input/output vector. Each input/output variable was
normalized using the equation above with its mean and
standard deviation values. This normalization process was
applied to the whole data including the training and testing
sets. The single set of normalization parameters of each
variable (i.e. the standard deviation and the mean) were then
preserved to be applied to new data during forecasting.
Our experience found that NN performs better with normally
distributed data and unseasonal data. Having input data
exhibiting trend or periodic variations renders data
transformation necessary. There are different ways to
transform the input variables into forms, making the neural
network interpret the input data easier and perform faster in
the training process. Examples for such transformation forms
include the variable first derivative, relative variable
difference, natural logarithm of relative variable, square root
of variable, and trigonometric functions. In this study, all input
variables including the output were transformed using the first
SPE 68593 PREDICTING NATURAL GAS PRODUCTION USING ARTIFICIAL NEURAL NETWORK 3
derivative of each variable. The choice of this transform
removed the trend in each input variable, thus helping
reducing the multicollinearity among the input variables. The
first-derivative transformation, however, resulted in a loss of
one data point due to its mathematical formulation.
Selection of Neural Network Inputs and Output
Gas production was selected as the output of the neural
network since it is the target for prediction. Diagnostic
techniques such as scatter plots and correlation matrix were
performed on the data to check their validity and to study the
relationships between the target and each of the predictor
variables. As an example, a scatter plot for average footage
drilled per oil and gas exploratory well versus gas production
is shown in Fig. 2. The correlation coefficients for all inputs
versus the target, gas production, are given in Table 1. The
highest correlation coefficient value is 0.924 for the input I-9,
average footage drilled per oil and gas exploratory well. This
is also shown in Fig. 2 by the high linear correlation of this
variable with gas production. The correlation matrix helps to
reduce the number of input variables by excluding those with
high correlation coefficients. Some input variables with high
correlation coefficients, however, are important and needed to
be included in the network model because of their physical
relations with the target. This problem can be alleviated by
applying transformation techniques to remove the trend and
reduce the high correlation coefficient. Fig. 3 shows a scatter
plot of the input I-9 versus gas production after performing the
normalization and the first derivative transformation. The
figure shows the data points are more scattered and fairly
distributed around the zero horizontal line. The preprocessing
procedure resulted in a 45% reduction of the correlation
coefficient of this input from 0.924 to 0.512.
Neural Network Model Design
There are a number of design factors that must be considered
in constructing a neural network model. These considerations
include the selection of neural network architecture, the
learning rule, the number of processing elements in each layer,
the number of hidden layers, and the type of transfer function.
Fig. 4 depicts an illustration of the neural network model
designed in this study.
The neural network architecture determines the method that
the weights are interconnected in the network and specifies the
type of learning rules that may be used. Selection of network
architecture is one of the first things done in setting up a
neural network. The multilayer normal feed forward
most commonly used architecture and is generally
recommended for most applications; hence it is selected to be
used for this study.
Selection of a learning rule is also an important step because it
affects the determination of input functions, transfer functions,
and associated parameters. The network used is based on a
back-propagation (BP) design,
the most widely recognized
and most commonly used supervised-learning algorithm. In
this study, quick propagation (QP)
learning algorithm, which
is an enhanced version of the back-propagation algorithm, is
used for its performance and speed. The advantage of QP is
that it runs faster than BP by minimizing the time required for
finding a good set of weights using heuristic rules. These rules
automatically regulate the step size and detect conditions that
accelerate learning. The optimum step size is then determined
by evaluating the trend of the weight updates over time.
The fundamental design of a backpropagation neural
network consists of an input layer, a hidden layer, and an
output layer. Fig. 4 shows the architecture of a BP neural
network. A layer consists of a number of processing elements
or neurons. The layers are fully connected, indicating that each
neuron of the input layer is connected to each hidden layer
node. Similarly, each hidden layer node is connected to each
output layer node. The number of nodes needed for the input
and output layers depends on the number of inputs and outputs
designed for the neural network.
A transfer function acts on the value returned by the input
function. An input function combines the input vector with the
weight vector to obtain the net input to the processing element
given a particular input vector. Each of the transfer functions
introduces a nonlinearity into the neural network, enriching its
representational capacity. In fact, it is the nonlinearity of the
transfer function that gives a neural network its advantage
over conventional or traditional regression techniques. There
are a number of transfer functions. Among those are sigmoid,
arctan, sin, linear, Gaussian, and Cauchy. The most commonly
used transfer function is the sigmoid function. It squashes and
compresses the input function when it takes on large positive
or large negative values. Large positive values asymptotically
approach 1, while large negative values are squashed to 0. The
sigmoid is given by
)( . ………………………………….. (3)
Fig. 5 is a typical plot of the sigmoid function. In essence,
the activation function acts as a nonlinear gain for the
processing element. The gain is actually the slope of the
sigmoid at a specific point. It varies from a low value at large
negative inputs, to a high value at zero input, and then drops
back toward zero as the input becomes large and positive.
In the first step of the development process, the available data
were divided into training and test sets. The training set was
selected to cover the data from 1949 to 1989 (40 year-data
points) while the testing set covers the data from 1990 to 1998
(nine year-data points). We chose to split the data based on an
80/20 rule. We first normalized all input variables and the
4 S.M. AL-FATTAH, R.A. STARTZMAN SPE 68593
output using the average/standard deviation method, then, took
the first derivative of all input variables including the output.
In the initial training and testing phases, We developed the
network model using most of the default parameters in the
neural network software. Generally, these default settings
provided satisfactory results to start with. We examined
different architectures, different learning rules, different input
and transfer functions, with increasing numbers of hidden-
layer neurons, on the training set to find the optimal learning
parameters, then the optimal architecture. We used primarily
the black-box testing approach, comparing network results to
actual historical results, to verify that the inputs produce the
desired outputs. During training, We used several diagnostic
tools to facilitate understanding how the network is training.
• the mean square error of the entire output,
• a plot of the mean square error versus the number of
• the percentage of training or testing set samples that are
correct based on a chosen tolerance value,
• a plot of the actual output and the network output, and
• a histogram of all the weights in the network.
The three-layer network with initially all 15 input variables
was trained over the training samples. We chose the number of
neurons in the hidden layer on the basis of existing rules of
and on experimentation. One rule of thumb states that
the number of hidden-layer neurons should be about 75% of
the input variables. Another rule suggests that the number of
hidden-layer neurons be approximately 50% of the total
number of input and output variables. One of the advantages
of the neural software used in this study is that it allows the
user to specify a range for the minimum and maximum
number of hidden neurons. Putting all this knowledge
together, along with my experimentation experience, we
specified the range of 5 to 12 hidden neurons for the single
We used the input sensitivity analysis to study the
significance of each input parameter and how it is affecting
the performance of the network. This procedure helps to
reduce the redundant input parameters and to determine the
optimum number of input parameters of the neural network. In
each of the training runs, the results of the input sensitivity
analysis are examined and the least-significant input parameter
is deleted at a time, then the weights are reset and the network-
training process is restarted with the remaining input
parameters. This process is repeated until all the input
parameters are found to have significant contribution to the
network performance. The input is considered significant
when its effect normalized value equal to or greater than 0.7 in
the training set and 0.5 in the test set. We varied the number of
iterations used to train the network from 500 to 7,000 to find
the optimal number of iterations to train the network. The
number of 3,000 iterations was used for most of training runs
performed on the network. In the training process, the training
is automatically terminated when the maximum iterations are
reached or the mean square error of the network falls below
the limit set, specified as 1.0x10
. While training the network,
the test set is also evaluated. This step enables a test pass
through the test set for each pass through the training set.
However, this step does not intervene with the training
statistics rather than evaluating the test set while training for
fine-tuning and generalizing the network parameters.
After the training has been performed, the performance of
the network was then tested. During the testing process, the
test set was used to determine how well the network
performed on data that it has not been previously seen during
To evaluate the performance of the network, the
classification option of the network output as being correct
based on a specified tolerance was used. This method
evaluates the percentage of training and testing samples that
faithfully generalizes the patterns and values of the network
outputs. We used a tolerance of 0.05 (the default value is 0.5)
in this study, meaning that all outputs for a sample must be
within this tolerance for a sample to be considered correct.
Another measure that we took to examine the network
performance is the plot of the mean square error versus the
number of iterations. A well-trained network is characterized
by decreasing errors for both the training and the test sets as
the number of iterations increases.
Results of Training and Testing
We used the input sensitivity analysis technique
gauging the sensitivity of the gas production rate (output) for
any particular input. The method makes use of the weight
values of a successfully trained network to extract the
information relevant to any particular input node. The
outcome of the method is the effect values as well as
normalized effect values for each input variable on the output
of gas production rate. These effect values represent an
assessment of the influence of any particular input node on the
The results of the input-identification process and training
procedure indicated that the network has excellent
performance with 11 input parameters. We found these input
parameters, which are described in Table 2, have significant
contributions on the network performance.
Tables 3 and 4 present the results of the input sensitivity
analysis for the training and test sets, respectively. The
normalized effect values indicate that all 11 inputs
significantly contribute to the improvement of the network
performance and to the prediction of the U.S. natural gas-
production rate for both the training and test sets. The training
set input-sensitivity analysis, Table 3, shows that the gas
annual depletion rate (I15) is the most significant input
parameter that contributes to the network performance, hence
in predicting the U.S. natural gas production. Although we
found it important to the improvement of the network
performance and kept it in the network model, the input of gas
wellhead prices (I3) has the least effect normalized value of
0.7 among all other inputs in the training set. Table 4 shows
SPE 68593 PREDICTING NATURAL GAS PRODUCTION USING ARTIFICIAL NEURAL NETWORK 5
that all inputs in the test set exceed the arbitrary specified
threshold value of 0.5, indicating that all inputs contribute
significantly to the network model.
The network was trained with 5,000 iterations using the
QP learning algorithm. We found that the optimum number of
hidden-layer nodes is 5. Fig. 6 shows the prediction of the
neural network model, after the training and validation
processes, superimposed on the normalized actual U.S. gas
production. The neural network prediction results show
excellent agreement with the actual production data in both the
training and testing stages. These results indicate that the
network is trained and validated very well, and the network is
ready to be used for forecasting. In addition, statistical and
graphical error analyses were used to examine the
performance of the network.
Optimization of Network Parameters
We attempted different configurations of the network to
optimize the number of hidden nodes and number of
iterations, and thus fine-tune the network performance,
running numerous simulations in the optimization process.
Table 5 presents only potential cases for illustration purposes.
The table shows that increasing the number of iterations to
more than 5,000 improves the training-set performance but
worsens the test-set performance. In addition, decreasing the
number of iterations to 3,000 yields higher errors for both the
training and test sets. The number of hidden-layer nodes was
also varied in the range of 4 to 22 nodes. Increasing the
number of hidden nodes more than 5 shows good results for
the training set but gives unsatisfactory results for the test set,
which is the most important. From these analyses, the optimal
network configuration for this specific U.S. gas production
model is a three-layer QP network with 11 input nodes, 5
hidden nodes, and 1 output node. The network is optimally
trained with 5,000 iterations.
Statistical accuracy of this network performance is given in
Table 5 (Case 11a). The mean squared error (MSE) of the
training set is 0.0034 and for the test set is 0.0252. Fig. 7
shows the MSE versus the iterations for both the training and
test sets. The errors of training-set samples decrease
consistently throughout the training process. In addition, the
errors of the test-set samples decrease fairly consistently along
with the training-set samples, indicating that the network is
generalizing rather than memorizing. All the training- and test-
set samples yield results of 100% correct based on 0.05
tolerance, as shown in Fig. 8.
Fig. 9 shows the residual plot of the neural network model
for both the training and test samples. The plot shows not only
that the errors of the training set are minimal but also that they
are evenly distributed around zero, as shown by Fig. 10. As is
usually the case, the errors of the test samples are slightly
higher than the training samples. The crossplots of predicted
vs. actual values for natural gas production are presented in
Figs. 11 and 12. Almost all the plotted points of this study’s
neural network model fall very close to the perfect 45
line, indicating its high degree of accuracy.
After the successful development of the neural network model
for the U.S. natural gas production, future gas production rates
must be forecast. To implement the network model for
prediction, forecast models should be developed for all 11
network inputs or be obtained from independent studies. We
developed forecasting models for all the independent network
inputs, except for the input of gas wellhead prices, using the
time-series analysis approach. The forecasts for the gas
wellhead prices came from the Annual Energy Outlook 2000
We adjusted the EIA forecasts of gas prices, based
on 1998 U.S. dollars/Mcf, to 1992 U.S. dollars/Mcf so that the
forecasts would be compatible with the gas prices historical
data used in the network development. We developed the
forecasting models for the input variables of the neural
network using the Box-Jenkins
methodology of time-series
analysis. Details of forecasts development for other network
inputs are described in Ref. 17.
Before implementing the network model for forecasting,
We took one additional step, taking the test set back and
adding it to the original training set. Then the network could
be trained only one time, keeping the same configuration and
parameters of the original trained network intact. The purpose
of this step is to have the network take into accounts the
effects of all the available data, since the number of data is
limited, and to ensure the generalization of the network
performance yields better forecasting.
Next, we saved data for the forecasted network inputs for
the period 1999 to 2020 as a test-set file, whereas the training
set-file contained data from 1950 to 1998. Then we ran the
network with one pass through all the training and test sets.
We retained the obtained data results to their original form by
adding the output value at a given time to its previous one.
After decoding the first difference output values, we
denormalized the obtained values for the training and test
samples using the same normalization parameters as in the
Fig. 13 shows this study’s neural network forecasting
model for the U.S. gas production to the year 2020. The figure
also shows the excellent match between the neural network
model results and the actual data of natural gas production.
The neural network-forecasting model indicates that the U.S.
gas production in 1999 is in decline at 1.8% of the 1998
production. The production will stay at the 1999 level of
production with slight decline until the year 2001, after which
gas production starts to increase. From 2002 to 2012 gas
production will increase steadily, with an average growth rate
of approximately 0.5%/yr. The neural network model indicates
that this growth will more than double for the period 2013 to
2020, with a 1.3%/yr average growth rate. By 2019, gas
production is predicted at 22.6 Tcf/yr, approximately the same
as the 1973 level of production.
6 S.M. AL-FATTAH, R.A. STARTZMAN SPE 68593
The neural network-forecasting model developed in this
study is not only dependent on the performance of the trained
data set, but also on the future performance of forecasted input
parameters. Therefore, the network model should be updated
periodically when new data become available. While it is
desirable to update the network model with new data, the
network’s architecture and its parameters need not be
necessarily changed. However, a one-time run to train the
network with the updated data is necessary.
Comparison of Forecasts
This section compares the forecasts of U.S. natural gas
production from the Energy Information Admin. (EIA)
the neural network approach, and stochastic modeling
approach developed by Al-Fattah
. The EIA 2000 forecast of
U.S. gas supply is based on USGS estimates of U.S. natural
gas resources, including conventional and unconventional gas.
The main assumptions of the EIA forecast are:
• Drilling, operating, and lease equipment costs are
expected to decline ranging from 0.3 to 2%.
• Exploratory success rates are expected to increase by
• Finding rates will improve by 1 to 6%/yr.
Fig. 14 shows the forecast of EIA compared with the
forecasts of this study using the neural network and time series
analysis (or stochastic modeling). The EIA forecast of U.S.
gas supply with approximately 20 Tcf/yr for year 2000 is
higher than the neural network forecast with approximately
19.5 Tcf/yr. However, the EIA forecast matches the neural
network forecast from 2001 to 2003, after which the EIA
forecast increases considerably with annual average increases
of 2.4% from 2004 to 2014 and 1.3% thereafter.
The stochastic-derived model gives a production forecast
much higher than the forecasts of the EIA and neural network.
The forecast of U.S. gas supply by the stochastic-derived
model shows an exponential trend with an average growth rate
The neural network forecast is based on the following
assumptions of its independent input forecasts:
• Gas prices are expected to increase by 1.5%/yr.
• The gas depletion rate is expected to increase by
• Drilling of gas exploratory wells will improve by 3.5%/yr.
• Drilling of oil/gas exploratory wells will increase on
average of 2.5%/yr.
• GDP will have an average increase of 2.1%/yr.
The forecast of the neural network takes into account the
effects of the physical and economics factors on U.S. gas
production. These render forecasts of natural gas supply
reliable. The neural network model indicates that production
of U.S. gas will increase from 2002 to 2012 with a 0.5%/yr
average increase. Thereafter, gas production will have a higher
increase, averaging 1.3%/yr through 2020.
In this study, we developed a new approach to forecast the
future production of U.S. natural gas using a neural network.
The three-layer network was trained and tested successfully,
and comparison with actual production data showed excellent
agreement. Forecasts of the network input parameters were
developed using stochastic modeling approach to time-series
analysis. The network model includes various physical and
economic input parameters, rendering the model a useful
short-term as well as long-term forecasting tool for future gas
The forecasting results of the neural network model
showed that the 1998 U.S. gas production would decline at a
rate of 1.8%/yr in 1999 extending to year 2001 at the same
1999 production level. After 2001 gas production starts to
increase steadily to year 2012 with approximately 0.5%/yr
average growth rate. This growth will more than double for
the period 2013 to 2020, with 1.3%/yr average growth rate. By
2020, gas production is predicted at 23 Tcf/yr, slightly higher
than the 1973 level of production.
The neural network model is a useful as a short-term as
wells as long-term predictive tool for future gas production. It
can also be used to quantitatively examine the effects of
various physical and economical factors on future gas
production. With the neural network model developed in this
study, we recommend further analysis to evaluate
quantitatively the effects of the various physical and economic
factors on future gas production.
GDP = gross domestic product, US $
GDPGR = growth rate of gross domestic product
t = time, 1/t, 1/yr
X = input/output vector
= mean or arithmetic average
= standard deviation
S.M. Al-Fattah would like to thank Saudi Aramco for
supporting his PhD study at Texas A&M University.
1. Haykin, S.: Neural Networks: A Comprehensive Foundation,
Macmillan College Publishing Co., New York City (1994).
2. Azoff, E.M.: Neural Network Time Series Forecasting of
Financial Markets, John Wiley & Sons Ltd. Inc., Chichester,
3. Pham, D.T., and Liu, X.: Neural Networks for Identification,
Prediction and Control, Springer-Verlag London Ltd., London
4. Neural Networks in Finance and Investing: Using Artificial
Intelligence to Improve Real-World Performance, revised edition,
R.R. Trippi and E. Turban (eds.), Irwin Professional Publishing,
SPE 68593 PREDICTING NATURAL GAS PRODUCTION USING ARTIFICIAL NEURAL NETWORK 7
5. Al-Kaabi, A.U. and Lee, W.J.: “Using Artificial Neural Nets to
Identify the Well-Test Interpretation Model,” SPEFE (September
6. Habiballah, W.A., Startzman, R.A., and Barrufet, M.A.: “Use of
Neural Networks for Prediction of Vapor/Liquid Equilibrium K
Values for Light-Hydrocarbon Mixtures,” SPERE (May 1996)
7. EIA, Internet Home Page: http://www.eia.doe.gov/.
8. Twentieth Century Petroleum Statistics, 52
ed., DeGolyer and
MacNaughton, Dallas, TX (1996).
9. Twentieth Century Petroleum Statistics, 54
ed., DeGolyer and
MacNaughton, Dallas, TX (1998).
10. Attanasi, E.D. and Root, D.H.: “The Enigma of Oil and Gas Field
Growth,” AAPG Bulletin (March 1994) 78, 321.
11. Energy Statistics Sourcebook, 13
ed., OGJ Energy Database,
PennWell Pub. Co., Tulsa, OK (1998).
12. “World Energy Projection System,” DOE/EIA-M050, Office of
Integrated Analysis and Forecasting, U.S. Dept. of Energy, EIA,
Washington, DC (September 1997).
13. Kutner, M.H., Nachtschiem, C.J., Wasserman W., and Neter, J.:
Applied Linear Statistical Models, fourth edition, Irwin, Chicago
14. ThinksPro: Neural Networks Software for Windows User’s Guide,
Logical Designs Consulting Inc., La Jolla, CA (1995).
15. “Annual Energy Outlook 2000,” DOE/EIA-0383, Office of
Integrated Analysis and Forecasting, U.S. Dept. of Energy, EIA,
Washington, DC (December 1999).
16. Box, G.E., Jenkins, G.M., and Reinsel, G.C.: Time Series Analysis
Forecasting and Control, third edition, Prentice-Hall Inc.,
Englewood Cliffs, NJ (1994).
17. Al-Fattah, S.M.: “New Approaches for Analyzing and Predicting
Global Natural Gas Production,” PhD dissertation, Texas A&M
U., College Station, TX (2000).
SI Metric Conversion Factors
x 2.831 685 E-02 = m
8 S.M. AL-FATTAH, R.A. STARTZMAN SPE 68593
Fig. 1-Basic structure of a three-layer back-propagation neural
0 5000 10000 15000 20000 25000
Gas Production, Bcf/yr
Average Footage Drilled per Oil & Gas Exploratory Well,
Fig. 2-Scatter plot of gas production and average footage drilled
per oil and gas exploratory well.
TABLE 1-CORRELATION COEFFICIENTS OF NETWORK INPUTS
WITH GAS PRODUCTION.
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5
Gas Production, Bcf/yr
Average Footage Drilled per Oil & Gas Exploratory Well
Fig. 3-Scatter plot of gas production and average footage drilled
per oil and gas exploratory well, after data preprocessing.
Input layer Output layerHidden layer
SPE 68593 PREDICTING NATURAL GAS PRODUCTION USING ARTIFICIAL NEURAL NETWORK 9
GDP growth rate
Fig. 4-Neural network design of this study.
-20 -15 -10 -5 0 5 10 15 20
Fig. 5-Sigmoid function.
TABLE 2- INPUT VARIABLES OF TRAINED NEURAL NETWORK
No. Input Description
1 I3 Gas wellhead prices
Average depth drilled per well in gas
3 I7 Footage drilled in gas exploratory wells
Number of wells drilled in gas exploratory
Average depth drilled per well in oil/gas
6 I10 Footage drilled in oil/gas exploratory wells
Number of wells completed in oil/gas
Average depth drilled per well in oil
9 I13 Growth rate of gross domestic product
10 I14 Gas proved reserves
11 I15 Annual gas depletion rate
TABLE 3-RESULTS OF INPUT SENSITIVITY ANALYSIS FOR
Training Set Input Sensitivity Analysis
Input # Effect Effect Normalized
I3 0.145 0.699
I6 0.187 0.904
I7 0.202 0.973
I8 0.168 0.810
I9 0.212 1.025
I10 0.155 0.748
I11 0.193 0.932
I12 0.158 0.761
I13 0.264 1.276
I14 0.285 1.376
I15 0.310 1.496
10 S.M. AL-FATTAH, R.A. STARTZMAN SPE 68593
TABLE 4-RESULTS OF INPUT SENSITIVITY ANALYSIS FOR TEST
Test Set Input Sensitivity Analysis
Input # Effect Effect Normalized
I3 0.142 0.856
I6 0.186 1.124
I7 0.136 0.821
I8 0.167 1.011
I9 0.192 1.163
I10 0.084 0.506
I11 0.094 0.567
I12 0.229 1.385
I13 0.222 1.340
I14 0.169 1.020
I15 0.200 1.207
1950 1960 1970 1980 1990 2000
Normalized Gas Production
Fig. 6-Performance of the neural network model with actual U.S.
TABLE 5-NEURAL NETWORK PERFORMANCE WITH DIFFERENT
HIDDEN NODES AND ITERATIONS*
Case Hidden Iterations
number nodes number MSE Correct, % MSE Correct, %
11-9 5 3000 0.0065 100 0.0321 78
11a 5 5000 0.0034 100 0.0252 100
11b1 5 6000 0.0025 100 0.0286 89
11b2 5 7000 0.0020 100 0.0323 89
11c 6 5000 0.0012 100 0.0619 44
* All cases were run with the same 11 inputs.
0 1000 2000 3000 4000 5000
Mean Squared Error
Fig. 7-Convergence behavior of the Quick Propagation three-layer
network (11,5,1) that learned from the U.S. natural gas production
0 1000 2000 3000 4000 5000
Fig. 8-Behavior of training and test samples classified correct.
SPE 68593 PREDICTING NATURAL GAS PRODUCTION USING ARTIFICIAL NEURAL NETWORK 11
1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000
Fig. 9-Residual plot of the neural network model.
-0.023 -0.014 -0.004 0.005 0.014 0.023 0.033 0.042
Fig. 10-Frequency of residuals of the neural network model.
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5
Actual Gas Production (1st difference), Tcf/yr
Predicted Gas Production (1st difference), Tcf/yr
Fig. 11-Crossplot of neural network prediction model and actual
gas production (first difference).
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5
Actual Gas Production (Normalized), Tcf/yr
Predicted Gas Production (Normalized), Tcf/yr
Fig. 12-Crossplot of neural network prediction model and actual
gas production (normalized).
1950 1960 1970 1980 1990 2000 2010 2020
Gas Production, Bcf/yr
Fig. 13-Neural network forecasting model of U.S. gas production.
1950 1960 1970 1980 1990 2000 2010 2020
Gas Production, Bcf/yr
Fig. 14-Comparison of U.S. gas production forecasts.