ArticlePDF Available

Formulating large-scale quantity-quality bilinear data reconciliation problems

Authors:
  • Industrial Algorithms Limited

Abstract

This short note describes the relevant details of formulating and implementing general bilinear quantity–quality balances found in industrial processes when data reconciliation is applied. The modeling also allows for the straightforward generation of analytical first-order derivatives. Quantity–quality balance problems are those that involve both extensive and intensive stream variables such as flows and compositions, respectively and are related through laws of conservation of material, energy and momentum. The balance equations involve both linear and bilinear terms (multi-linear) of quantity and quality where quantity times quantity and quality times quality are not germane although they can be included easily. Two numerical examples are provided to demonstrate the new formulation technique.
Formulating Large-Scale Quantity-Quality Bilinear Data Reconciliation Problems
Jeffrey Dean Kelly1
Keywords: Data reconciliation, multi-linear, equality constrained optimization, analytical derivatives and quantity-quality
problem.
1 Industrial Algorithms, 15 St. Andrews Road, Toronto, Ontario, Canada, M1P 4C3
E-mail: jdkelly@industrialgorithms.ca
Abstract
This short note describes the relevant details of formulating and implementing general bilinear quantity-quality balances found
in industrial processes when data reconciliation is applied. The modeling also allows for the straightforward generation of
analytical first-order derivatives. Quantity-quality balance problems are those that involve both extensive and intensive stream
variables such as flows and compositions respectively and are related through laws of conservation of material, energy and
momentum. The balance equations involve both linear and bilinear terms (multi-linear) of quantity and quality where quantity
times quantity and quality times quality are not germane although they can be included easily. Two numerical examples are
provided to demonstrate the new formulation technique.
Introduction
The process industries that typically require some form of data reconciliation of their aggregated and real-time process data
include mineral processing, oil and gas recovery, petroleum refining and petrochemical manufacturing. These industries use
data reconciliation to verify custody transfer and field flowmeters, field analyzers and laboratory instruments including automatic
tank gauging systems. The resulting data from data reconciliation are then used to monitor the effectiveness and efficiency of the
manufacturing process and to benchmark its performance against other plants consuming and producing similar feedstocks and
productstocks.
Obviously in order to have data reconciliation we need some level of redundancy in the system or network. This is achieved
through the existence of independent measuring devices and laws of conservation of material, energy and momentum which
allow us to relate one measurement to another through the topology of the integrated process. Unfortunately, these relationships
are nonlinear in the sense that there may exist a product of one variable times another, as we shall see further, thus requiring
iterative or successive substitution solving techniques. Fortunately, for the types of data reconciliation problems found in the
process industries, these problems are relatively well studied with some recent and relevant literature available in Crowe (1986),
Stephenson and Shewchuk (1986), MacDonald and Howat (1988), Pai and Fisher (1988), Swartz (1989), Tjoa and Biegler
(1991), Crowe (1996), Sanchez and Romagnoli (1996), Weiss et. al. (1996), Chen et. al. (1998), Kelly (1998), Schraa and Crowe
(1998) and Puming and Gang (2002). There is also an important structure that can be exploited in terms of presenting the
problem to the solver and this is what we term the quantity-quality problem which is also found at the core of the models that
plan and schedule these processes.
In the quantity-quality problem, we hierarchically class the variables into two sub-classes called quantity variables and quality
variables. Quantity variables are flows and inventories of mass, moles, volume, energy and momentum. Quality variables are
compositions, properties and conditions such as concentrations, densities, temperatures, pressures and velocity. Quantities are
known as extensive variables and qualities are intensive variables. Extensive variables scale with the system whereas intensive
variables do not. If the size of a reactor doubles then the throughput will double but the temperature and pressure at which the
reactor is operated will still be according to the process design and operating procedure. These variables are then related through
2
the quantity-quality balances of the system, where for the purposes of data reconciliation, there is no distinction being made
between differential (continuous or rate-based) and integral (batch or amount-based) balances. Moreover, it is implied that these
are steady-state balances with no accumulation. These variables are further classed into sub-categories specific to the problem of
data reconciliation defined shortly. It should also be noted that a possible advantage of this formulation is that it allows and
supports the reconciliation and calculation of both the measured and unmeasured variables explicitly using one optimization
solution. In contrast, the nonlinear matrix projection method of Crowe (1986) for example aggregates measured flows with
measured concentrations (i.e., category 1 in Crowe’s terminology). After a solution has been obtained by matrix projection a post
one-stream-at-a-time optimization must be performed to disaggregate a stream’s measured flow and concentration. The
formulation presented here does not require aggregating a stream’s extensive and intensive variables together and hence can be
used to provide consistent covariance information necessary for gross error detection and identification on the variables
explicitly.
Using the well-known definition of a process, which is an operation (or series of operations) that causes a physical or chemical
change in a substance, it is possible to explain a little further the relevance of the quantity-quality balances. We know that matter
can neither be created nor destroyed and consequently all processes must conserve mass and therefore a quantity-quality balance
can be used to perform a mass (or weight) balance involving flows in volume units times densities. Moles always conserve
except when a chemical reaction occurs (e.g., combusting natural gas with air) and hence a quantity-quality balance can be
formulated for a mole balance involving flows in mass units times the reciprocal molecular weights. Volume always conserves
except when either a chemical or physical reaction occurs (e.g. mixing alcohol and water) and thus a quantity-quality balance can
be modeled for a volume balance involving flows in mass units times specific volumes (or the reciprocal of density). Other
notable quantity-quality balances that would be found in the process industries mentioned are component balances such as
hydrogen, sulfur and propylene, property balances such as flash point, octane and vapor pressure and process condition balances
such as heat, work and entropy.
The next two sections that follow describe in more detail how to formulate both the functions and the first-order derivatives of the
quantity-quality balances but first it is important to highlight the mathematical description of the data reconciliation problem to
show their relevance. Adopting the nomenclature found in Kelly (1998), it is possible to pose the data reconciliation as a local,
equality constrained optimization problem with a quadratic objective function and general nonlinear constraints as
0zy,x,f
x)(xQ)x(x
mm
x
=
=
)(
min
1T
J
(1)
where
J
is the objective function, to be compared against a theoretical lower bound to detect the presence of gross errors (Crowe
(1996)),
m
x
is the vector of measurements for quantity and quality measured variables,
x
is the vector of reconciled quantity
and quality variables,
y
is the vector of calculated quantity and quality variables (unmeasured variables),
z
is the vector of
3
fixed or constant quantity and quality variables (more appropriately parameters),
Q
is the matrix of weights for quantity and
quality measured variables (i.e., proportional to the uncertainty in the measurements) and
f
is the vector of quantity-quality
balances or functions. The notation used in equation (1) for the functions describes them as general nonlinear equality
constraints with ad hoc structure, however for our purposes they are shown to be described in a specific quantity-quality bilinear
framework. The linear data reconciliation sub-problem solved at each cycle or iteration
of the solution method is
0f+yB+x A
)x(xQ)x(x
sssss
smsm
x
s
=
=
min
1T
s
J
(2)
where
s
x
and
s
y
are the vectors of reconciled and calculated variables
s
A
and
s
B
represent the matrix of first-order
derivatives (Jacobian) re-computed at each iteration. There is also the matrix
s
C
described in the section to follow which is
required to compute
s
f
.
Formulating the Functions (Vector of Equations)
In this section we show how the quantity-quality bilinear problem can be formulated generically. The nomenclature may be
somewhat daunting at first however it can be easily programmed directly into commercial matrix programming languages such
as MATLAB and MATHEMATICA for example. Our objective is to replace the general function of equation (1) with a more
structured version as follows.
0zIzE,,yI,yE,xI,xEf
sssss
=
)(
(3)
where the reconciled (
ss
xI,xE
), calculated (
ss
yI,yE
) and fixed (
zIzE,
) variables are classed into the extensive and
intensive categories. Equation (3) shows the sub-division of the variables but we also require a decomposition of the functions or
equality constraints as well
C
s
B
s
A
s
L
ss
fffff
+++=
(4)
where
L
s
f
is the vector of functions for the extensive and intensive linear constraints (see equation (5)),
A
s
f
is the vector of
functions evaluated for the constraints involving the bilinear terms of extensive reconciled variables times intensive reconciled,
calculated and fixed variables (see equation (6)),
B
s
f
is the vector of functions involving extensive calculated variables with
intensive variables (see equation (7)) and
C
s
f
is the vector of functions involving extensive fixed variables with intensive
variables (see equation (8)).
zICIzECEyIBIyEBExIAIxEAEf
ssss
L
s
+++++=
(5)
sss
A
s
xEzIAECIxEyIAEBIxExIAEAIf ++=
(6)
4
sss
B
s
yEzIBECIyEyIBEBIyExIBEAIf
++=
(7)
zEzICECIzEyICEBIzExICEAIf
ss
C
s
++=
(8)
The six incidence matrices found in equation (5) correspond to the linear relationships for all of the variables where all of the
matrices shown in equations (5) to (8) have the same number of rows (i.e., the number of functions
NF
). As an example,
AE
would contain the 1’s and –1’s in the appropriate elements for a mass balance involving mass reconciled extensive variables for a
reactor and
AI
would contain the 1’s and –1’s for a splitter involving density reconciled intensive variables requiring the
density in to be equal to the densities out. The bilinear variable terms found in for example the vector
s
xExI
represent all of
the bilinear products in the quantity-quality system balance for extensive reconciled variables times intensive reconciled
variables. In essence, we simply create extra variables for the bilinear terms which in planning and scheduling terminology of
the quantity-quality problem is known as cascading. The associated incidence matrix
AEAI
will also contain 1’s and –1’s
indicating where the bilinear term is found in the balances; note by convention, a +1 means a fluxion or movement of substance
into a piece of equipment or balance node and –1 means a fluxion of substance out of the node. All of these incidence matrices
do not have the subscript
attached because they do not change during the path to the reconciliation solution. It should also be
mentioned that we have neglected extensive variables times other extensive variables and intensive variables times other
intensive variables given that these are not necessarily germane to the quantity-quality problem however, they could be easily
added in a similar way.
In order to update the bilinear vectors such as
s
xExI
at each iteration with the new estimates of the variables, there must be
mechanism to do this. This can be easily accomplished with the use of two index sets
XE
XEXI
and
XI
XEXI
(there will be
two index sets for every bilinear term vector). These contain the extensive and intensive variable indices respectively and
s
xExI
can be computed as follows.
)(,)(,..1,..1,..1
)()()(
iXEXIkiXEXIjNXIkNEXjNXEXIi
kji
XIXE
======
sss
xIxExExI
(9)
where
NXEXI
specifies the number of reconciled extensive times intensive variable bilinear terms,
NXE
is the number of
reconciled extensive variables and
NXI
is the number of reconciled intensive variables. From the perspective of generating the
linear sub-problem information for equation (2), the vectors of reconciled and calculated variables (
s
x
,
s
y
) are formed as
=
s
s
s
xI
xE
x
and
=
s
s
s
yI
yE
y
where the parameters of the optimization are formed before the first iteration as
5
=
m
m
m
xI
xE
x
,
=
zI
zE
z
and
=
QI0
0QE
Q
Before we show how to form analytically the Jacobian, there are couple of modeling details that need to be discussed specific to
the quantity-quality problem structure. The first detail is that a variable, either extensive or intensive, cannot exist in the same
function more than once in either a linear or bilinear term. This was originally violated in the heat balance example of Swartz
(1989) found in Kelly (1998) and discussed in our numerical example 2. This means that every stream or fluxion of material
must have a unique set of variables created. For example, the original formulation of example 2 had the two terms
)2(*)1()2(*)1( xIxEyIxE
in one of the heat balance function definitions which violates our bilinear quantity-quality
requirement. The correct way to model this is to use the known and generated stream flow variable
)2(xE
in place of
)1(xE
in the second bilinear term. The second detail is that the number of non-zeros in the bilinear incidence matrices (
AEAI
, etc.) must be greater than or equal to the number of bilinear terms found in the index sets (
XE
XEXI
and
XI
XEXI
,
etc.) and less than or equal to two times the number of bilinear terms (
NXEXI
, etc.). This means that a bilinear term must
appear in at least one function and cannot appear more than twice in keeping with the first detail.
Formulating the First-Order Derivatives (Jacobian Matrix)
A major driving force to model the process industries’ data reconciliation problem in a structured quantity-quality framework is
the ability to easily generate analytical Jacobian or the matrix of first-order derivatives. Knowing that the underlying data
reconciliation is a bilinear problem, the Jacobian can be easily evaluated at each iteration simply by inserting one of the extensive
or intensive variable iterates into the appropriate location. For our purposes, the matrix definitions of the derivatives for the
reconciled variables are
[ ]
XE
s
XE
s
XE
s
s
s
E
s
AECIAEBIAEAIAE
xE
f
A
+++=
=
(10a)
[ ]
XI
s
XI
s
XI
s
s
s
I
s
CEAIBEAIAEAIAI
xI
f
A
+++=
=
(10b)
[ ]
I
s
E
s
ss
s
s
AA
xIxE
f
A
=
=
] [
TT
(10c)
where is
s
A
is the same matrix found in the equation (2) and is formed by the matrix augmentation in equation (10c). Similarly
the matrix of derivatives
s
B
and
s
C
are found in equations (11) and (12).
[ ]
YE
s
YE
s
YE
s
E
s
BECIBEBIBEAIBEB
+++=
(11a)
[ ]
YI
s
YI
s
YI
s
I
s
CEBIBEBIAEBIBIB
+++=
(11b)
6
[ ]
I
s
E
ss
BBB
=
(11c)
and
[ ]
ZEZE
s
ZE
s
E
s
CECICEBICEAICEC
+++=
(12a)
[ ]
ZIZI
s
ZI
s
I
s
CECIBECIAECICIC
+++=
(12b)
[ ]
I
s
E
ss
CCC
=
(12c)
The new matrices,
XE
s
AEAI
for example, symbolize derivative of the bilinear terms with respect to the extensive and intensive
variables. These are easily generated at iteration
for example as
)(),(,1,1..1
)()),((),(
kXEXImkXEXIj..NXEXIk..NXENF,ji
mkisignji
XIXE
======
s
XE
s
xIAEAIAEAI
(13)
where
)(
sign
is the sign of the 1 or –1 for the matrix element in question and
XE
XEXI
and
XI
XEXI
are the same index
sets as defined previously to generate the bilinear terms at each iteration used in the function evaluations. The last section
delineates the bilinear quantity-quality formulation and implementation for two numerical examples solved previously in the
literature.
Numerical Examples
The two numerical examples described below have been hitherto solved in Kelly (1998) and can also be found in other studies.
The examples are formulated and solved using MATLAB Release 13 (The Mathworks Inc. (2002)). The first example is a small
mining floatation circuit problem with copper (Cu) and zinc (Zn) concentrations in each of the eight streams. The dimension of
the problem is provided in Table 1 along with the statistics for example 2.
Table 1. Numbers of functions, variables and bilinear terms for examples 1 and 2 respectively.
N
F
NXE NXI NYE NYI NZE NZI NXEXI NXEYI NXEZI NYEXI NYEYI NYEZI NZEXI NZEYI NZEZ
I
12 0 14 7 2 1 0 0 0 0 12 2 0 2 0 0
17 6 10 9 5 0 0 5 1 0 5 4 0 0 0 0
Of the nine sets of possible bilinear term combinations, only three have any elements which is consistent with the observation
that the number of extensive measured variables and the number of intensive fixed variables is zero. Table 2 shows the division
of the variables into the designation of reconciled, calculated and fixed. Table 3 shows the index sets
YE
YEXI
,
XI
YEXI
,
YE
YEYI
,
YI
YEYI
,
ZE
ZEXI
and
XI
ZEXI
. The index set pairs
YE
YEXI
and
XI
YEXI
, for example, indicate the
xIyE
products found in the system hence bilinear term number seven specifies the product of
)1(yE
and
)9(xI
.
Table 2. Classification of variables for example 1.
7
xI yE yI zE
1 Cu in Stream 1 Flow of Stream 2 Cu in Stream 8 Flow of Stream 1
2 Cu in Stream 2 Flow of Stream 3 Zinc in Stream 8
3 Cu in Stream 3 Flow of Stream 4
4 Cu in Stream 4 Flow of Stream 5
5 Cu in Stream 5 Flow of Stream 6
6 Cu in Stream 6 Flow of Stream 7
7 Cu in Stream 7 Flow of Stream 8
8 Zn in Stream 1
9 Zn in Stream 2
10 Zn in Stream 3
11 Zn in Stream 4
12 Zn in Stream 5
13 Zn in Stream 6
14 Zn in Stream 7
Table 3. Index sets for bilinear terms in example 1.
123456789101112
YEXIYE 123456123456
YEXIXI 2 3 4 5 6 7 9 10 11 12 13 14
YEYIYE 7 7
YEYIYI 1 2
ZEXIZE 1 1
ZEXIXI 1 8
Due to space limitations we do not show the fifteen possible incidence matrices (i.e.,
AEAI,AI,AE,
etc.) however after
converging the model in 6 iterations using the solution method of Kelly (1998) a final objective function value of 17.9615269
was found compared to 17.9616400 found after 22 iterations previously. The reason for the discrepancy is due to the fact that the
Kelly (1998) problem instance was missing an element in the calculated variables Jacobian matrix
s
B
for function ten.
The second example’s problem size can be found in Table 1. This problem is a heat balance with four heat exchangers, one
splitter and one mixer. Table 4 provides the variable classification and Table 5 specifies the appropriate index sets.
8
Table 4. Classification of variables for example 2 (F = Flow and T = Temperature).
xE xI yE yI
1 F of Stream A1 T of Stream A1 F of Stream A2 T of Stream A2
2 F of Stream A3 T of Stream A3 F of Stream A4 T of Stream A6
3 F of Stream A6 T of Stream A4 F of Stream A5 T of Stream B2
4 F of Stream B1 T of Stream A5 F of Stream A7 T of Stream B3
5 F of Stream C1 T of Stream A7 F of Stream A8 T of Stream C2
6 F of Stream D2 T of Stream A8 F of Stream B2
7 T of Stream B1 F of Stream B3
8 T of Stream C1 F of Stream C2
9 T of Stream D1 F of Stream D1
10 T of Stream D2
Table 5. Index sets for bilinear terms in example 2.
12345
XEXIXE 12456
XEXIXI 1 2 7 8 10
XEYIXE 3
XEYIYI 2
YEXIYE 23459
YEXIXI 34569
YEYIYE 1678
YEYIYI 1345
Notice appropriately from Table 5 that there are as many bilinear terms as there are products of stream flow times stream
temperature which is identical to the number of streams in the problem (A1…A8, B1...B3, C1, C2, D1, D2). There can never be
more bilinear terms than streams or fluxions times the number of qualities for a stream however there can be fewer bilinear terms
if a quantity-quality balance involving a particular stream is not required. This is also a consistent observation for example 1
given that there are eight streams with two metal concentrations requiring sixteen bilinear terms which is exactly the number of
bilinear terms found in Table 3. Finally, when example 2 was re-solved using the new quantity-quality bilinear formulation an
objective function of 14.5861664 was found in 4 iterations. Previously it took 5 iterations with an objective function value of
14.5861656 using the same settings. Unfortunately, there is no obvious explanation for the slight difference in the iteration
count.
Conclusions
Presented in this short note are the fine points of formulating and implementing quantity-quality bilinear data reconciliation
problems found in the process industries. The contribution of this work is that the formulation is completely extensible to both
small and large bilinear quantity-quality problems given that either dense or sparse matrix implementations can be employed and
that first-order derivative information can be easily calculated in the form of the Jacobian matrix explicitly. The formulation can
also be straightforwardly modified to handle other multi-linear terms (i.e., trilinear and quadlinear) either by creating extra
variables and constraints or by extending the formulation with new incidence matrices and index sets. For example, adding a
trilinear product term for a volume flow times a density times a sulfur concentration can be achieved by creating a calculated
9
extensive variable to represent the bilinear product of volume flow times the density and an extra function to enforce that the
calculated extensive variable equals the volume flow times the density.
References
Chen, X., Pike, R.W., Hertwig, T.A. and Hopper, J.R. “Optimal Implementation of On-Line Optimization”, Computers chem.
Engng., 22, S435-S442, (1998).
Crowe, C.M., “Reconciliation of Process Flow Rates by Matrix Projection, Part II: The Nonlinear Case”, AIChE Journal, 32,
616-623, (1986).
Crowe, C.M., “Data Reconciliation - Progress and Challenges”, J. Proc. Cont., 6, 89-98, (1996).
Kelly, J.D., “A Regularized Solution to the Reconciliation of Constrained Data Sets”, Computers chem. Engng., 22, 1771-1788,
(1998).
MacDonald, R.J. and Howat, C.S., “Data Reconcilation and Parameter Estimation in Plant Performance Analysis”, AIChE
Journal, 34, 1-8, (1988).
The Mathworks Inc., MATLAB Release 13, June, (2002).
Pai, C.C.D. and Fisher, G.D., “Application of Broyden’s Method to Reconciliation of Nonlinearly Constrained Data”, AIChE
Journal, 34, 873-876, (1988).
Puming, Z. and Gang, R., “Steady-State Bilinear Data Reconciliation Dealing with Scheduling”, IFAC 15th Triennial World
Congress, Barcelona, Spain, (2002).
Sanchez, M. and Romagnoli, J., “Use of Orthongonal Transformations in Data Classification-Reconciliation”, Computers chem.
Engng., 20, 483-493, (1996).
Schraa, O.J. and Crowe, C.M., “The Numerical Solution of Bilinear Data Reconciliation Problems Using Constrained
Optimization Methods”, Computers chem. Engng., 22, 1215, (1998).
Stephenson, G.R., and Shewchuk, C.F., “Reconciliation of Process Data with Process Simulation”, AIChE Journal, 32, 247-254,
(1986).
Swartz, C.L.E., “Data Reconciliation for Generalized Flowsheet Applications”, Amer. Chem. Society National Meeting, Dallas,
Texas, (1989).
Tjoa, I.B., and Biegler, L.T., “Simultaneous Strategies for Data Reconciliation and Gross Error Detection of Nonlinear Systems”,
Computers chem. Engng, 15, 679-690, (1991).
Weiss, G.H., Romagnoli, J.A., and Islam, K.A., “Data Reconciliation – An Industrial Case Study”, Computers, chem. Engng.,
20, 1441-1449, (1996).
10
Preprint
Context: Over the last decade, the forestry sector has undergone substantial changes, evolving from a post-2008 financial crisis landscape to incorporating policies favoring sustainable and green alternatives, especially after the 2015 Paris agreement. This evolution was drastically disrupted with the advent of the COVID-19 pandemic in 2020, causing unprecedented interruptions in supply chains, product markets, and data collection. Grasping the aftermath of the COVID-19, regional instances of the forest supply chain sector need synthetic pictures of their present state and future opportunities for emerging wood products and better regional-scale carbon balance. But given the impact of COVID-19 lock-down on data collection, the production of such synthetic pictures has become more complex, yet essential. This was the case for the regional supply chain of the Grand-Est region in France that we studied. Aims: For this study, our aim was to demonstrate that an integrated methodology could provide such synthetic picture even though we sued heterogenous sources of data and different analytical objectives: i.e. (1) retrospectively evaluate the aftermath of COVID-19 pandemic on the supply chain outcomes within the forestry sector; and then (2) retrospectively explore possible options of structural change of regional supply chain that would be required to simultaneously recover from COVID-19 and transit to new objectives in line with the extraction of new bio-molecules from wood biomass, and with the reduction of the regional scale carbon footprint (in line with the IPCC Paris Agreement) Methods: To achieve this, our methodological approach was decomposed into three steps. We first used a Material Flow Analysis (MFA) recently conducted on the forestry sector in the Grand Est region to establish a Sankey diagram (i.e. a schematic representation of industrial sectors and biomass flows along the supply chain) for the pre-Covid-19 period (2014-2018). Then we compared pre-Covid-19 Sankey diagram to the only source of data we could access from the post-Covid-19 period (2020-2021) in order to estimate the impact of Covid-19. Finally, we used as input the reconciled supply chain model into a consequential Wood Product Model (WPMs), called CAT (carbon accounting tool) in order to compare three prospective scenarios: (1) a scenario that projected 2020-2021 Covid-19 conditions and assumed pre-Covid-19 business as usual practices, (2) a scenario illustrating the consequence or rerouting some of the biomass to satisfy the expected increase in pulp and paper production to satisfy the needs of the industry after Covid-19, and (3) a scenario that explored new opportunities in term of extraction of novel bio-molecules by the emerging biochemical wood industry. For every scenario we also evaluated the regional carbon gains and losses that these changes implied. Results: Our study conducted a detailed analysis of the impacts of the COVID-19 pandemic on the forestry sector's supply chain in the Grand Est region, using a dynamic and integrated Wood Product Model. We found significant disruptions during the pandemic period, with notable declines in industrial wood chips and timber hardwood production by 41.8% and 40%, respectively. Conversely, there were substantial increases in fuelwood, timber sawdust, and timber softwood, rising by 14.15%, 44.23%, and 15.29% respectively. These fluctuations underscore the resilience and vulnerabilities within the regional wood supply chain. Our findings also emphasize the potential for strategic rerouting of biomass flows to meet changing industry demands, which could play a crucial role in supporting the sector's recovery and adaptation to post-pandemic conditions. Discussion and conclusion: In addition, our study recognizes the limitations of the current approach combining MFA and WPM and suggests potential areas of enhancement. Ultimately, our findings shed light on the need to develop more integrated analytical methods to provide useful synthetic pictures of regional scale supply chains, when there is a need to adapt it to evolving situations and complex data landscapes.
Article
The sodium aluminate solution of evaporation process is provided for the digestion process to recycle useful resources, decrease alkali concentration in waste solution, and reduce the environmental pollution. However, the concentration of recycled sodium aluminate solution as an indispensable indicator for manipulating the evaporation process of the industrial alumina production, is acquired off-line, leading to delayed feedback information. To ensure stable production of subsequent process, reduce energy and resource consumption, this paper focuses on developing a hybrid prediction model for recycled sodium aluminate solution concentration in evaporation process. First, data reconciliation is utilized to improve the quality of material flow information, and the process mechanism model is established through mechanism research and equilibrium principle to obtain the concentration prediction result. Moreover, an industrial production condition analysis as well as fuzzy expert rules, are introduced for modifying prediction results from mechanism model. Furthermore, the errors are predicted by the kernel extreme learning machine, and a concentration prediction model integrated error compensation results and modified predictions is established. The experimental simulations and industrial application show that the accuracy of the prediction error obtained by the proposed model reaches to 90% within ±2%, and the advantages of the hybrid model are particularly prominent under different conditions, which is beneficial for efficient and clean production.
Article
In modern industrial processes, data reconciliation plays a significant role in reducing measurement errors and adjusting process measurements to meet conservation laws and constraints of process models. Most traditional data reconciliation methods are based on the weighted least squares technique with diagonal weight matrix, in which the variable errors are assumed to be independent of each other. However, the measurement errors are often nonlinearly correlated due to reasons like kinetic relationships, external disturbances and feedback control, etc. To deal with this problem, a new data reconciliation strategy based on mutual information is proposed in this paper. In the new method, mutual information is first utilized to measure the nonlinear variable correlations. Then, a new objective function is designed for data reconciliation with a full weight matrix, whose elements are the mutual information coefficients between different variables. Finally, the effectiveness of the proposed data reconciliation method is demonstrated through a nonlinear numerical example and an industrial application.
Article
In real industrial production, many mass and heat transfer processes are influenced by high temperature, high pressure, and even strong acid or alkali conditions. In addition, some important variables cannot be measured and chemical compositions are analyzed offline with a long time delay, which leads to inaccurate measurements of the process data. In this paper, layered data reconciliation (LDR) method based on time registration is proposed to improve the measurement accuracy and estimate unmeasured vari8ables. Considering that the material cannot be tagged and tracked in process manufacturing, a temporal and spatial matching strategy for the process data is designed based on a time-correlation analysis matrix which is determined to describe the correlation of each time sequence in the data matrix. Then, a layered data reconciliation model with time registration is developed by reconciling the mass balance layer and the heat balance layer separately and stepwise, and the model is solved by the state transition algorithm. Meanwhile, regular terms and engineer's knowledge are introduced into the data reconciliation model to solve the problem of insufficient redundancy. The industrial verification results from the actual industrial evaporation process indicate that the accuracy of measured values is improved by using the proposed reconciliation strategy. This article is protected by copyright. All rights reserved
Article
Full-text available
The focus of this paper is to detail the quantity and quality modeling aspects of production flowsheets found in all process industries. Production flowsheets are typically at a higher-level than process flowsheets given that in many cases more direct business or economic related decisions are being made such as maximizing profit and performance for the overall plant and/or for several integrated plants together with shared resources. These decisions are usually planning and scheduling related, often referred to as production control, which require a larger spatial and temporal scope compared to more myopic process flowsheets which detail the steady or unsteady-state material, energy and momentum balances of a particular process unit-operation over a relatively short time horizon. This implies that simpler but still representative mathematical models of the individual processes are necessary in order to solve the multi time-period nonlinear system using nonlinear optimizers such as successive linear programming and sequential quadratic programming. In this paper we describe six types of unit-operation models which can be used as fundamental building blocks or objects to formulate large production flowsheets. In addition, we articulate the differences between continuous and batch processes while also discussing several other important implementation issues regarding the use of these unit-operation models within a decision-making system. It is useful to also note that the quantity and quality modeling system described in this paper complements the quantity and logic modeling used to describe production and inventory systems outlined in Zyngier and Kelly (Optimization and logistics challenges in the enterprise, Springer, New York 61–95, 2009).
Article
Full-text available
The focus of this article is to describe a relatively straightforward technique to improve the accuracy of predicting or tracing the internal qualities inside a production network when all of the stream qualities cannot be explicitly measured. The qualities usually represent intensive variables such as compositions, properties and conditions. Compositions can be the volume fractions of certain crude-oil species in a crude-oil mixture and usually the sum of the compositions equals unity or some other known constant. Properties can be the density, molecular weight or fat-content of a stream. Conditions are the physical state attributes of a stream such as its temperature, pressure or velocity. Properties can also be functions of both its stream’s compositions and conditions such as the density of a stream can alternatively be calculated by the sum of its compositions times the individual species density. The stream quantities, which also include the inventory or hold-up amounts for equipment that can support accumulation, are the extensive variables of the system and most often have more measurements available due to their lower-cost (i.e., an orifice plate with a differential pressure transmitter is considerably less expensive than a gas chromatograph with integrator for example). Currently in industry the qualities of a production system are inferred over some tracing time-horizon using discretized time by using known time-periods or time-slices. By assuming that the quantity values are error or defect-free and then using well-known quantity and quality material balances at each time-period, the dependent qualities starting from known initial or starting values can be calculated. Unfortunately the proliferation or propagation of quantity gross-errors can seriously distort the accuracy of these quality estimates even if accurate quality measurements exist. One remedy is to perform quantity and quality reconciliation at each time-period in the tracing horizon starting from the first time-period. This enables the analyst performing the task of tracing over some horizon of say a day to detect and isolate defects in both time and space where space implies some location in the production network. If an error is observed which is statistically significant then that measurement can be removed from the information set (i.e., deleted from the balance equations) and calculated provided enough other information exists. That is, if the measurement is redundant then the calculated value is observable (Crowe (1996)). The two sections to follow highlight the approach and its value however we briefly elucidate the business significance of tracing qualities in a production network. If all of the feedstocks, intermediates (work-in-progress) and products were known pure components or species, then in essence, only its material name and quantity are all that would be required to manage the production in terms of meeting the necessary supply and demand orders of the plant. This is typically the case in discrete-parts manufacturing but it is more the exception than the rule in the process industries especially in petroleum refining where a crude-oil by its very nature is a collection of almost unknown proportions and properties. As well, due to side-reactions, non-ideal separations, degrading performance and transition cycles, the quality of the materials flowing through any process industry plant can vary dramatically from one time to the next. Thus, careful attention must be paid to the tracing of qualities throughout the production timeline and network in order that accurate and timely information is available when making business decisions. These business decisions may include at what time to sell a particular material when its quality meets, but does not exceed, the specification limits of the customer.
Article
For data reconciliation of the process concerning the multi-component mass or energy balances, it will be a particular nonlinear problem, i.e. bilinear data reconciliation. To overcome the defects of the traditional methods for bilinear data reconciliation, a new method was proposed. Firstly, a new unobservable variable elimination approach was developed to decompose and regularize the data reconciliation problem. Then, the problem was solved by particle swarm optimization algorithm (PSO) after elimination of the constraint equations. Data from simulation of a bilinear steady case were reconciled and the performance of the proposed strategy was compared with that of traditional methods. The comparison results show that the new method may give more accurate estimates, have good computing efficiency and can work well even if some of the process variables are unobservable.
Article
Full-text available
Due to quantity times quality nonlinear terms inherent in the oil-refining industry, performing industrial-sized capital investment planning (CIP) in this field is traditionally done using linear (LP) or nonlinear (NLP) models whereby a gamut of scenarios are generated and manually searched to make expand and/or install decisions. Though mixed-integer nonlinear (MINLP) solvers have made significant advancements, they are often slow for large industrial applications in optimization; hence, we propose a more tractable approach to solve the CIP problem using a mixed-integer linear programming (MILP) model and input-output (Leontief) models, where the nonlinearities are approximated to linearized operations, activities, or modes in large-scaled flowsheet problems. To model the different types of CIP's known as revamping, retrofitting, and repairing, we unify the modeling by combining planning balances with scheduling concepts of sequence-dependent changeovers to represent the construction, commission, and correction stages explicitly in similar applications such as process design synthesis, asset allocation and utilization, and turnaround and inspection scheduling. Two motivating examples illustrate the modeling, and a retrofit example and an oil-refinery investment planning problem are also highlighted.
Article
Full-text available
A new iterative solution to the statistical adjustment of constrained data sets is derived in this paper. The method is general and may be applied to any weighted least squares problem containing nonlinear equality constraints. Other methods are available to solve this class of problem, but are complicated when unmeasured variables and model parameters are not all observable and the model constraints are not all independent. Of notable exception, however, are the methods of Crowe (1986) and Pai and Fisher (1988), although these implementations require the determination of a matrix projection at each iteration which may be computationally expensive. An alternative solution which makes the pragmatic assumption that the unmeasured variables and model parameters are known with a finite but equal uncertainty is proposed. We then re-formulate the well known data reconciliation solution in the absence of these unknowns to arrive at our new solution; hence the regularization approach. Another procedure for the classification of observable and redundant variables which does not require the explicit computation of the matrix projection is also given. The new algorithm is demonstrated using three illustrative examples previously used in other studies.
Article
Full-text available
Flow rate and concentration measurements in a steady state process are reconciled by weighted least squares so that the conservation laws and other constraints are obeyed. Two projection matrices are constructed in turn, in order to decompose the problem into three subproblems to be solved in sequence. The first matrix eliminates all unmeasured component flow rates and concentrations from the equations; the second then removes the unmeasured total flow rates. The adjustments to component flow rates are iteratively determined, starting with guessed values of unmeasured total flow rates. Chi-square and normal test statistics are derived by linearizing the equations, to allow detection of gross errors in imbalances and adjustments of measurements.
Article
Full-text available
Measurements in a chemical process are subject to errors, both random and systematic, so that the laws of conservation of mass and energy are not obeyed. In order to record the performance of the process, these measurements are adjusted in order that they conform to the conservation laws and any other constraints imposed upon them. This procedure is known as data reconciliation. Advances in the theory and application of data reconciliation are reviewed and current problems are highlighted. In addition to defining the basic problem, we discuss the detection of gross errors in data and of pre-adjustment of data, finding departures from steady state, estimation of the variance structure of the data, observability of unmeasured quantities and redundancy of measurements.
Conference Paper
Mass balance model for steady-state linear data reconciliation is insufficient, especially when short-term scheduling is performed on hybrid systems. Scheduling-equations are established and used as addition to the model. A new formulation of data reconciliation is thus proposed as steady-state bilinear data reconciliation. In this way, the redundancy degree is increased and the solvability of data reconciliation is improved. Simulation results demonstrate the efficiency and consistency of the proposed approach.
Article
Results from a theoretical and numerical evaluation of on-line optimization algorithms were used to recommend the best way to conduct on-line optimization. This optimal procedure conducts combined gross error detection and data reconciliation to detect and rectify gross errors in plant data sampled from the distributed control system. The Tjoa-Biegler method (the contaminated Gaussian distribution) was used for gross errors in the range of 3σ – 30σ- or the robust method (Lorentzian distribution) for larger gross errors. This step generates a set of measurements containing only random errors which is used for simultaneous data reconciliation and parameter estimation using the least squares method. Updated parameters are used in the plant model for economic optimization that generates optimal set points for the distributed control system. Applying this procedure to a Monsanto sulfuric acid contact plant, a 3% increase in profit and a 10% reduction in SO2 emissions were projected over current operating condition which is consistent with other reported applications.
Article
In this paper, the use of orthogonal factorizations, more precisely the Q-R decomposition, to analyze, decompose and solve the linear and bilinear data reconciliation problem is further investigated. It is shown that the decomposition provides additional insight in identifying structural singularities in the system topology, allowing the problem to decompose into lower dimension subproblems. Energy balances are explicitly considered. Two examples of application are presented.
Article
An iterative procedure is developed that makes use of Crowe's matrix projection and for the first time combines a quasi-Newton update with the Gauss-Newton scheme for solving nonlinear data reconciliation problems. Experimenting with the iterative scheme on a range of numerical problems leads to the following conclusions: 1. In most cases, one full evaluation of left bracket B vertical P right bracket initially is sufficient for the algorithm to reach a good solution. 2. The convergence rate is faster than when the constant-direction approach is adopted.
Article
Results of a computer study are reported applying data reconciliation and interpretation techniques to a single-stage flash operation. Measured errors in the process variables are significantly reduced. The techniques are successfully extended to estimate flash efficiency. The flash efficiency is estimated using two methods: (1) a statistically rigorous simultaneous data reconciliation and efficiency estimation procedure, and (2), a faster, nonrigorous sequential procedure. It is found that the simultaneous procedure provides better measured variable and flash efficiency estimates.
Article
The SACDA group at the University of Western Ontario has developed an equations-oriented modular process simulation program called MASSBAL. A model of a process is constructed by selecting and flowsheeting standard building blocks describing fundamental operations such as the mixing and separating of streams. Building blocks are also provided by which equipment operating or design equations may be incorporated into the model. Reconciliation of the inconsistency and redundancy of process measurements is an optional feature of the program. The measurements are reconciled by minimizing a weighted least-squares criterion while satisfying the model equations and any modeling constraints. The minimum is obtained directly, avoiding the need for laborious searching techniques.
Article
Process measurements are taken in chemical plants for the purpose of evaluating process control or process performance. As a result of random and possibly gross errors, these measurements do not generally satisfy the process constraints. Thus they need to be reconciled. In the data reconciliation process, the data are adjusted to satisfy the process constraints while minimizing the error in the least squares sense, and the unmeasured variables are estimated whenever possible. In this study, we instead minimize an objective function that is constructed using maximum likelihood principle to construct a new distribution function, which takes into account both contributions from random and gross errors. The advantages of minimizing this objective function are that it gives unbiased estimates in the presence of gross errors and that simultaneously a gross-error detection test can be constructed based on their distribution functions without the assumption on the linearity of the constraints. Furthermore, the structure of this objective function can be exploited under certain conditions. Thus, efficient nonlinear programming strategies, similar to the hybrid SQP method introduced by Tjoa and Biegler in 1991 for least squares objective functions, can also be developed. The effectiveness of this strategy is demonstrated on nonlinear example problems.