No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
System Identification, Reduced-Order Filtering and Modeling via Canonical Variate Analysis
Abstract
Very general reduced order filtering and modeling problems are phased in terms of choosing a state based upon past information to optimally predict the future as measured by a quadratic prediction error criterion. The canonical variate method is extended to approximately solve this problem and give a near optimal reduced-order state space model. The approach is related to the Hankel norm approximation method. The central step in the computation involves a singular value decomposition which is numerically very accurate and stable. An application to reduced-order modeling of transfer functions for stream flow dynamics is given.
... The CCA is a well-known method in statistics that finds a (lower dimensional) space that maximizes the correlation between two random variables (Rao, 1979). In our case, this translates to the objective of finding the unknowns O 0 f ,Ř p , and the subspace of x t by maximizing the correlation betweeny t+f,(c) t andM t,pž t−p t , e.g., see Chiuso (2007), Gičans (2009), Katayama (2005, Larimore (1983) and Larimore (2005) to mention a few. In Chiuso (2007) and Katayama (2005) statistical optimality of CCA in the LTI setting has been shown by formulating the optimal one-step-ahead predictor of the state based on either the past or future data. ...
... the log-likelihood function associated with the least-squares (LS) estimation problem of the unknowns O 0 fŘ p based on the signalš y t+f,(c) t andM t,pž t−p t of the model (23). Larimore (1983Larimore ( , 2005 claim maximum log-likelihood of the state estimation using CCA, however, the mathematical derivations are scattered within the literature and appear to be incomplete, as pointed out in Gičans (2009). In Theorem 1, we prove the maximum log-likelihood property for the LPV case. ...
... In conclusion, the SVD (26) maximizes the marginal-likelihood function of the linear estimation problem (23) w.r.t. the unknowns O 0 f ,Ř p and the covariance Ξ 2 with state-sequence (25) and log-likelihood function (26). Note that in early literature on CVA SID (Larimore, 1983(Larimore, , 1990, the constrained SVD (A.6) was performed with arbitrary positive-definite weight Λ ∈ S such that I =Ṽ ⊤ ΛṼ , which is called the CVA method. The CCA and CVA methods coincide with the weighting choice in (A.6). ...
In this paper, we establish a unified framework for subspace identification (SID) of linear parameter-varying (LPV) systems to estimate LPV state–space (SS) models in innovation form. This framework enables us to derive novel LPV SID schemes that are extensions of existing linear time-invariant (LTI) methods. More specifically, we derive the open-loop, closed-loop, and predictor-based data-equations (input–output surrogate forms of the SS representation) by systematically establishing an LPV subspace identification theory. We also show the additional challenges of the LPV setting compared to the LTI case. Based on the data-equations, several methods are proposed to estimate LPV-SS models based on a maximum-likelihood or a realization based argument. Furthermore, the established theoretical framework for the LPV subspace identification problem allows us to lower the number of to-be-estimated parameters and to overcome dimensionality problems of the involved matrices, leading to a decrease in the computational complexity of LPV SIDs in general. To the authors’ knowledge, this paper is the first in-depth examination of the LPV subspace identification problem. The effectiveness of the proposed subspace identification methods are demonstrated and compared with existing methods in a Monte Carlo study of identifying a benchmark MIMO LPV system.
... The CCA is a well-known method in statistics that finds a (lower dimensional) space that maximizes the correlation between two random variables [35]. In our case, this translates to the objective of finding the unknowns O 0 f ,Ř p , and the subspace of x t by maximizing the correlation betweeny t+f,(c) t andM t,pž t−p t , e.g., see [10,27,[36][37][38] to mention a few. In [10,27] statistical optimality of CCA in the LTI setting has been shown by formulating the optimal one-step-ahead predictor of the state based on either the past or future data. ...
... We will take an alternative viewpoint by formulating an estimate of the statesequence by maximizing the log-likelihood function associated with the least-squares (LS) estimation problem of the unknowns O 0 fŘ p based on the signalsy t+f,(c) t andM t,pž t−p t of the model (23). [37,38] claim maximum log-likelihood of the state estimation using CCA, however, the mathematical derivations are scattered within the literature and appear to be incomplete, as pointed out in [36]. In Theorem 24, we prove the maximum log-likelihood property for the LPV case. ...
... An important future direction of research is to imporve nummerical efficency and reduce computational load of the developped methods. There might be other solutions to the minimization problem of (A.12), however, we take the solution equal to the CVA solution of [37]. Hence, the latter choice of Q maximizes the marginal likelihood function (A. 5). ...
In this paper, we establish a unified framework for subspace identification (SID) of linear parameter-varying (LPV) systems to estimate LPV state-space (SS) models in innovation form. This framework enables us to derive novel LPV SID schemes that are extensions of existing linear time-invariant (LTI) methods. More specifically, we derive the open-loop, closed-loop, and predictor-based data-equations, an input-output surrogate form of the SS representation, by systematically establishing an LPV subspace identification theory. We show the additional challenges of the LPV setting compared to the LTI case. Based on the data-equations, several methods are proposed to estimate LPV-SS models based on a maximum-likelihood or a realization based argument. Furthermore, the established theoretical framework for the LPV subspace identification problem allows us to lower the number of to-be-estimated parameters and to overcome dimensionality problems of the involved matrices, leading to a decrease in the computational complexity of LPV SIDs in general. To the authors' knowledge, this paper is the first in-depth examination of the LPV subspace identification problem. The effectiveness of the proposed subspace identification methods are demonstrated and compared with existing methods in a Monte Carlo study of identifying a benchmark MIMO LPV system.
... Subspace algorithms such as the Canonical Variate Analysis (CVA) (Larimore, 1983) are used for the estimation of linear dynamical state space systems for time series. CVA is popular since it is numerically cheap (consisting of a series of regressions), asymptotically equivalent to quasi maximum likelihood estimation (using the Gaussian likelihood) for stationary processes and rate is provided. ...
... The CVA method of estimation proposed by Larimore (1983) is performed in three steps and uses two integers f, p ('future' and 'past') and information of the system order n (compare Bauer, 2005): ...
Subspace methods like canonical variate analysis (CVA) are regression based methods for the estimation of linear dynamic state space models. They have been shown to deliver accurate (consistent and asymptotically equivalent to quasi maximum likelihood estimation using the Gaussian likelihood) estimators for invertible stationary autoregressive moving average (ARMA) processes. These results use the assumption that the spectral density of the stationary process does not have zeros on the unit circle. This assumption is violated, for example, for over-differenced series that may arise in the setting of co-integrated processes made stationary by differencing. A second source of spectral zeros is inappropriate seasonal differencing to obtain seasonally adjusted data. This occurs, for example, by investigating yearly differences of processes that do not contain unit roots at all seasonal frequencies. In this paper we show consistency for the CVA estimators for vector processes containing spectral zeros. The derived rates of convergence demonstrate that over-differencing can severely harm the asymptotic properties of the estimators making a case for working with unadjusted data.
... Here too, prior information about the system to be modeled is assumed to be present so as to select n and m. With the model structure determined as in (12) and the performance metric as in (15), the next step is to determine the parameters of interest and the control input u. ...
... Other milestones in SIMs after the deterministic realization of [9] include Stochastic Realization [14], Canonical Variate Analysis (CVA) [15], Multivariable Output-Error State Space (MOESP) [13], and Numerical algorithms for Subspace State Space System Identification (N4SID) [11]. Efficient algorithms to compute the SVD matrices [16] inspired methods such as the Eigensystem Realization Algorithm (ERA) [10] and the Observer Kalman Identification (OKID) technique [17]. ...
The ushering in of the big-data era, ably supported by exponential advances in computation, has provided new impetus to data-driven control in several engineering sectors. The rapid and deep expansion of this topic has precipitated the need for a showcase of the highlights of data-driven approaches. There has been a rich history of contributions from the control systems community in the area of data-driven control. At the same time, there have been several new concepts and research directions that have also been introduced in recent years. Many of these contributions and concepts have started to transition from theory to practical applications. This paper will provide an overview of the historical contributions and highlight recent concepts and research directions.
... Those algorithms have been indicated to be applied so effectively and in a quantitatively stable manner by utilization of the modern numerical linear algebra techniques such as the singular value decomposition (SVD). Then, a new effort in digital signal processing and system identification based on the QR decomposition and the SVD arose in the mid-eighties and many papers have been published in the literature [28][29][30][31], etc. Those theory-based methods have guided to a development of diversified so-called subspace identification techniques, containing in [32][33][34][35][36][37], etc. Besides, Van Overschee and De Moor [16] have issued an initial exhaustive book on subspace identification of linear systems. ...
... In addition, Aoki [63] has reproduced stochastic realization algorithm based on the CCA and deterministic realization theory. Subspace methods for assigning state space models have been progressed by De Moor et al. [64], Larimore [27,28] and Van Overschee and De Moor [31]. Lindquist and Picci [65] have investigated state space identification algorithms with the help of geometric theory for stochastic realization. ...
Subspace method-based system identification and six different transfer functions as a product of this method have been developed and embedded in the motion control unit of the X-Y pedestal satellite antenna. Fit estimation data of these six transfer functions is changing from 95.46% to 99.99% with an RMSE (root mean square error) degree from 0.00502 to 0.00984, whereas the target RMSE degree value is 0.05.
... Alternatively, one may treat the latent variables as additional quantities to be estimated together with the model parameters. Such a strategy is termed data augmentation, and examples include subspace methods [5,6], the Gibbs sampler [7], and the Expectation Maximization (EM) algorithm [8,9,10,11]. ...
In the application of the Expectation Maximization algorithm to identification of dynamical systems, internal states are typically chosen as latent variables, for simplicity. In this work, we propose a different choice of latent variables, namely, system disturbances. Such a formulation elegantly handles the problematic case of singular state space models, and is shown, under certain circumstances, to improve the fidelity of bounds on the likelihood, leading to convergence in fewer iterations. To access these benefits we develop a Lagrangian relaxation of the nonconvex optimization problems that arise in the latent disturbances formulation, and proceed via semidefinite programming.
... The S 4 methods use projections to estimate the unobserved states [32], which are later used for system identification. In this section, we briefly review the canonical variate analysis (CVA) subspace method [13]. ...
State space subspace algorithms for input-output systems have been widely applied but also have a reasonably well-developedasymptotic theory dealing with consistency. However, guaranteeing the stability of the estimated system matrix is a major issue. Existing stability-guaranteed algorithms are computationally expensive, require several tuning parameters, and scale badly to high state dimensions. Here, we develop a new algorithm that is closed-form and requires no tuning parameters. It is thus computationally cheap and scales easily to high state dimensions. We also prove its consistency under reasonable conditions.
... for large integer s. For our purpose, the standard system identification method CVA is applied, and the reader is referred to, for example, references [53,54] for further details. The justin-time identification of the PES transfer function deals with possible variations due to nonlinearity and timevarying features. ...
The risk of leakages in process industry is environmentally critical and potentially hazardous. Many technologies and schemes for process monitoring are theoretically developed and applied in an industrial context. Nevertheless, most approaches still focus on individual monitoring of a process and its environment. The major challenge is the lack of a priori knowledge about the leakage. This paper introduces a new approach combining monitoring of the environment and its embedded process. The application on an industrial use‐case in a real plant environment illustrates the success of this combined monitoring approach as well as a decision support to localize an incipient leakage.
... Another important class of model selection and identification methods are Subspace identification regression-based methods, which are specially designed for ARMAX modelsꞌ state-space representations. The main protagonists of these methods are the CCA 3 method and the CVA 4 method, proposed by [25], as well as the N4SID 5 method [26], and the MOESP 6 method [27]. ...
As important as the classical approaches such as Akaikeꞌs AIC information and Bayesian BIC criterion in model-selection mechanism are, they have limitations. As an alternative, a novel modeling design encompasses a two-stage approach that integrates Fuzzy logic and Monte Carlo simulations (MCSs). In the first stage, an entire family of ARIMA model candidates with the corresponding information-based, residual-based, and statistical criteria is identified. In the second stage, the Mamdani fuzzy model (MFM) is used to uncover interrelationships hidden among previously obtained models criteria. To access the best forecasting model, the MCSs are also used for different settings of weights loaded on the fuzzy rules. The obtained model is developed to predict the road freight transport in Slovenia in the context of choosing the most appropriate electronic toll system. Results show that the mechanism works well when searching for the best model that provides a well-fit to the real data.
... Therefore, we describe a moment estimator based on subspace estimation of state space models which endows us with consistent initial estimates. The following discussion is based on Larimore (1983) and Bauer et al. (1999) 23 The procedure based on Larimore's CCA subspace algorithm has the following steps and is implemented in the RLDM package: ...
We propose a new parametrization for the estimation and identification of the impulse-response functions (IRFs) of dynamic factor models (DFMs). The theoretical contribution of this paper concerns the problem of observational equivalence between different IRFs, which implies non-identification of the IRF parameters without further restrictions. We show how the minimal identification conditions proposed by Bai and Wang (2015) are nested in the proposed framework and can be further augmented with overidentifying restrictions leading to efficiency gains. The current standard practice for the IRF estimation of DFMs is based on principal components, compared to which the new parametrization is less restrictive and allows for modelling richer dynamics. As the empirical contribution of the paper, we develop an estimation method based on the EM algorithm, which incorporates the proposed identification restrictions. In the empirical application, we use a standard high-dimensional macroeconomic dataset to estimate the effects of a monetary policy shock. We estimate a strong reaction of the macroeconomic variables, while the benchmark models appear to give qualitatively counterintuitive results. The estimation methods are implemented in the accompanying R package.
... The proposed method also has connections with state space models. There have been many ways to fit a state space model to vector time series, such as expectationmaximization (EM) (Shumway and Stoffer 1982), and N4SID (Moonen et al. 1989;Van Overschee and De Moor 1993) and CVA approach (Larimore 1983). In state space models, there have been methods developed to encourage sparse or low rank structures on the state transition matrix, such as She et al. (2018), Chen et al. (2017). ...
Large scale multi-dimensional time series can be found in many disciplines, including finance, econometrics, biomedical engineering, and industrial engineering systems. It has long been recognized that the time dependent components of the vector time series often reside in a subspace, leaving its complement independent over time. In this paper we develop a method for projecting the time series onto a low-dimensional time-series that is predictable, in the sense that an auto-regressive model achieves low prediction error. Our formulation and method follow ideas from principal component analysis, so we refer to the extracted low-dimensional time series as principal time series. In one special case we can compute the optimal projection exactly; in others, we give a heuristic method that seems to work well in practice. The effectiveness of the method is demonstrated on synthesized and real time series.
... As an attractive alternative in this respect the class of subspace algorithms is investigated in this paper. One particular member of this class, the so called canonical variate analysis (CVA) introduced by [16] (in the literature the algorithm is often called canonical correlation analysis; CCA), has been shown to provide system estimators which (under the assumption of known system order) are asymptotically equivalent to quasi maximum likelihood estimation (using the Gaussian likelihood) in the stationary case [17]. CVA shares a number of robustness properties in the stationary case with VAR estimators: [18] shows that CVA produces consistent estimators of the underlying transfer function in situations where the innovations are conditionally heteroskedastic processes of considerable generality. ...
This paper investigates the asymptotic properties of estimators obtained from the so called CVA (canonical variate analysis) subspace algorithm proposed by Larimore (1983) in the case when the data is generated using a minimal state space system containing unit roots at the seasonal frequencies such that the yearly difference is a stationary vector autoregressive moving average (VARMA) process. The empirically most important special cases of such data generating processes are the I(1) case as well as the case of seasonally integrated quarterly or monthly data. However, increasingly also datasets with a higher sampling rate such as hourly, daily or weekly observations are available, for example for electricity consumption. In these cases the vector error correction representation (VECM) of the vector autoregressive (VAR) model is not very helpful as it demands the parameterization of one matrix per seasonal unit root. Even for weekly series this amounts to 52 matrices using yearly periodicity, for hourly data this is prohibitive. For such processes estimation using quasi-maximum likelihood maximization is extremely hard since the Gaussian likelihood typically has many local maxima while the parameter space often is high-dimensional. Additionally estimating a large number of models to test hypotheses on the cointegrating rank at the various unit roots becomes practically impossible for weekly data, for example. This paper shows that in this setting CVA provides consistent estimators of the transfer function generating the data, making it a valuable initial estimator for subsequent quasi-likelihood maximization. Furthermore, the paper proposes new tests for the cointegrating rank at the seasonal frequencies, which are easy to compute and numerically robust, making the method suitable for automatic modeling. A simulation study demonstrates by example that for processes of moderate to large dimension the new tests may outperform traditional tests based on long VAR approximations in sample sizes typically found in quarterly macroeconomic data. Further simulations show that the unit root tests are robust with respect to different distributions for the innovations as well as with respect to GARCH-type conditional heteroskedasticity. Moreover, an application to Kaggle data on hourly electricity consumption by different American providers demonstrates the usefulness of the method for applications. Therefore the CVA algorithm provides a very useful initial guess for subsequent quasi maximum likelihood estimation and also delivers relevant information on the cointegrating ranks at the different unit root frequencies. It is thus a useful tool for example in (but not limited to) automatic modeling applications where a large number of time series involving a substantial number of variables need to be modelled in parallel.
... One approach is to construct a Hankel matrix of impulse responses, which was first proposed by Ho and Kalman [62], and later expanded upon by Tether [63], Risannen [64], Woodside [65], Silverman [66], Akaike [67], Chow [68,69], and Aoki [70,71]. These early papers spawned the field of subspace identification [72,73], which has resulted in techniques like N4SID [74,75,76,12], which take the SVD of a particular block Hankel matrix, and related techniques like MOESP [77,78,79] and CVA [80,81] (see, e.g., [82, §10.6] for a summary). We also point the reader to the paper by Jansson [83], which poses subspace identification methods as a regression, where the forecasting matrix is low rank. ...
We consider the problem of forecasting multiple values of the future of a vector time series, using some past values. This problem, and related ones such as one-step-ahead prediction, have a very long history, and there are a number of well-known methods for it, including vector auto-regressive models, state-space methods, multi-task regression, and others. Our focus is on low rank forecasters, which break forecasting up into two steps: estimating a vector that can be interpreted as a latent state, given the past, and then estimating the future values of the time series, given the latent state estimate. We introduce the concept of forecast consistency, which means that the estimates of the same value made at different times are consistent. We formulate the forecasting problem in general form, and focus on linear forecasters, for which we propose a formulation that can be solved via convex optimization. We describe a number of extensions and variations, including nonlinear forecasters, data weighting, the inclusion of auxiliary data, and additional objective terms. We illustrate our methods with several examples.
... The measure of correlation between the past and conditioned future inputs is computed using the canonical variate analysis (CVA) [10]. The linear time invariant MIMO process can be identified using Multivariable Output-Error State Space (MOESP) in the presence of process noise and measurement noise and the algorithm is based on the class of instrumental variables [17]. ...
The process model is very essential for the model based control design. The model of the process can be identified using system identification algorithm. The system identification is done through the open loop and closed loop approaches. In this work, the lab scale conical tank setup configured as a non square MIMO system. The conical tank system is identified through the both appraoches, the effectiveness and need of the both approaces are discussed. Based on the open loop identified model the controller designed and the controller implemented in the real time the to record the process data. From this data the closed loop identification are conducted uisng N4SID algorithm. The controller seetings are obtained using the smith predcitor based IMC based PI controller for the obtained model. The proposed identification algorithm and controller tuning show the better reults over the conventional method. Moreover, this method is applicable for all the non square MIMO system.
... К передовым методам параметрической идентификации, исполь-зующимся при построении стохастических моделей линейных стацио-нарных дискретных систем и реализованным в рамках пакета System Identification Toolbox [158,159] программной системы MATLAB, сле-дует отнести семейство так называемых подпространственных методов (в английском варианте Subspace methods), к которым относятся мето-ды N4SID (Numerical algorithms for subspace state space system identification), MOESP (Multivariable output error state space) и CVA (Сanonical variate analysis), описанные в [27,139,[160][161][162][163], [27,139,[162][163][164][165] и [162,166,167] соответственно. Указанные методы не предполагают численного решения задачи нелинейного программирования с ограни-чениями (1.2), что устраняет присущую МНК, ММП и МАВ проблему нахождения глобального экстремума, и работают наиболее эффектив-но при использовании канонических форм представления многомер-ных модельных структур [24,163,168]. ...
Излагаются теоретические и прикладные аспекты активной пара-
метрической идентификации стохастических линейных нестационар-
ных и нелинейных дискретных и непрерывно-дискретных систем с
предварительно выбранной модельной структурой. Рассматривается
случай, когда неизвестные параметры входят в уравнения состояния и
измерения, в начальные условия и в ковариационные матрицы шумов
системы и измерений. Приводится описание разработанного алгорит-
мического и программного обеспечения, позволяющего решать задачи
оптимального оценивания параметров с привлечением прямых и
двойственных градиентных процедур планирования эксперимента.
Монография будет интересна специалистам, научные и професси-
ональные интересы которых связаны с моделированием динамиче-
ских объектов стохастической природы.
... Further studies in this field have led to the development of another important branch in stochastic realization-based models, known as Subspace Identification methods (Akaike, 1974a;Akaike, 1974b;Ho and Kálmán, 1966). An advantage of subspace methods is that they are free from nonlinear optimization techniques and the need to impose, or investigate, canonical parametrization to the system (Larimore, 1983;Larimore, 1990;Qin and Ljung, 2003). Hence, modeling techniques following this framework, such as subspace methods, do not suffer from the inconveniences encountered in several well-established statistical techniques commonly used for a variety of applications, especially in hydrology. ...
Extended streamflow forecasting is nowadays important for early flood warning and risk mitigation under a changing climate. However, the absence of reliable estimates of the usual streamflow descriptors, at the considered time horizons, renders common input–output modeling approaches inapt for extended forecasting. For such a problem, system recurrence information is vital to produce relatively improved forecasts. Like any time series generated by complex systems, river flow can be represented by a time-varying parameter (TVP) model. TVP modeling frameworks often assume that the system evolution exhibits superstatistical random walks, and a number of statistical assumptions is followed upon, to infer better forecasts from the system. Also, the TVPs should hone a model's ability to capture system recurrence, which in this case translates to a number of state-based model structures. In this work, we develop an ensemble-based computationally efficient method that extracts useful cyclic-type TVPs which are strongly coupled over extended, but more continuous and less aggregated, time horizons. A multi-level modeling framework is proposed to facilitate the creation and direct use of updated parameter states to produce more reliable forecasts on the considered temporal resolutions. A dummy-variable optimization technique is also proposed within the state-updating procedure to allow smooth state transition. We show that the modeling framework produces stable results of parameter variations at different time horizons when applied to estimate the streamflow of Clearwater River. The proposed framework can incorporate exogenous descriptors at different levels without loss of generality and can be used for various forecasting applications.
... The initial condition of the system is defined as:π = 0 and c i1 = ΩI. The last step is to calculate the output for the next time step, which can be determined by the online prediction as in (12), (13), and (14). ...
We introduce a new system identification technique, leveraging on the benefits of the Evolutionary Takagi-Sugeno fuzzy system and the Type-2 interval fuzzy systems. Our technique has the ability to learn-from-scratch while having a better ability to accommodate the footprint-of-uncertainties (FoU). To support its mission in meeting the challenge in uncertain non-linear dynamic systems, we also introduce a new type reduction method, which is highly scalable to convert the Type-2 fuzzy systems into its Type-1 counterparts. As a part of its pruning strategy, the proposed system incorporates the concept of information entropy to avoid overfitting, which is a highly undesirable issue in modelling. We demonstrate the effectiveness of our technique in achieving a delicate balance between minimising the complexity of the structure of the acquired fuzzy model while maximising the prediction accuracy. We employ a set of challenging pH neutralisation data, known for its substantial non-linearity and large variation, in addition to a non-linear mechanical system. We conclude our research by conducting a rigorous comparative study to quantify the relative merits of our proposed technique with respect to the previous ETS algorithm (as its predecessor), the well-known KM-type reduction algorithm, and the higher order discrete transfer functions, widely implemented in conventional mathematical modelling techniques.
... Subspace identification methods (SIMs) have been known as very useful tools for state-space systems, which have been attracted considerable attention [1][2][3] in the past decades. The main proponents of these methods are MOESP (Multivariate Output Error State-space) proposed in [4], N4SID (Numerical algorithms for State-space Sub-Space IDentification) in [5], and CVA (Canonical Variate Analysis) in [6]. The statistical properties of these methods are fairly well established in a series of papers [7][8][9][10]. ...
In this paper, an alternative consistent subspace identification method using parity space is proposed. The future/past input data and the past output data are used to construct the instrument variable to eliminate the noise effect on consistent estimation. The extended observability matrix and the triangular block-Toeplitz matrix are then retrieved from a parity space of the noise-free matrix using a singular value decomposition based method. The system matrices are finally estimated from the above estimated matrices. The consistency of the proposed method for estimation of the extended observability matrix and the triangular block-Toeplitz matrix is established. Compared with the classical SIMs using parity space like SIMPCA and SIMPCA-Wc, the proposed method generally enhances the estimated model efficiency/accuracy thanks to the use of future input data. Two examples are presented to illustrate the effectiveness and merit of the proposed method.
... The SMI algorithms are attractive in that they provide convenient state-space models for multivariable linear systems directly from input-output data (Viberg, 1995). Among these SMI algorithms are the canonical variate analysis (Larimore, 1983(Larimore, , 1990, N4SID (Van Overschee and De Moor, 1994), MOESP (Verhaegen and Dewilde, 1992) and the deterministic-stochastic realization (Di Ruscio, 1996). We refer the readers to Qin (2006) and the references therein for a comprehensive overview of the available SMI algorithms. ...
Developing physically interpretable reduced-order models (ROMs) is critical as they provide an understanding of the underlying phenomena apart from computational tractability for many chemical processes. In this work, we re-envision the model reduction of nonlinear dynamical systems from the perspective of regression. In particular, we solve a sparse regression problem over a large set of candidate functional forms to determine the structure of the ROM. The method balances model complexity and accuracy by selecting a sparse model that avoids overfitting to accurately represent the system dynamics when subjected to a different input profile. By applying to a hydraulic fracturing process, we demonstrate the ability of the developed models to reveal important physical phenomena such as proppant transport and fracture propagation inside a fracture. It also highlights how a priori knowledge can be incorporated easily into the algorithm and results in accurate ROMs that are used for controller synthesis.
In this paper an identification method for state-space LPV models is presented. The method is based on a particular parameterization that can be written in linear regression form and enables model estimation to be handled using Least-Squares Support Vector Machine (LS-SVM). The regression form has a set of design variables that act as filter poles to the underlying basis functions. In order to preserve the meaning of the Kernel functions (crucial in the LS-SVM context), these are filtered by a 2D-system with the predictor dynamics. A data-driven, direct optimization based approach for tuning this filter is proposed. The method is assessed using a simulated example and the results obtained are twofold. First, in spite of the difficult nonlinearities involved, the nonparametric algorithm was able to learn the underlying dependencies on the scheduling signal. Second, a significant improvement in the performance of the proposed method is registered, if compared with the one achieved by placing the predictor poles at the origin of the complex plane, which is equivalent to considering an estimator based on an LPV auto-regressive (LPV-ARX) structure.
This article presents a novel approach for reconstructing an equivalent underlying model and deriving a precise equivalent expression through the use of direct causality topology. Central to this methodology is the transfer entropy method, which is instrumental in revealing the causality topology. The polynomial fitting method is then applied to determine the coefficients and intrinsic order of the causality structure, leveraging the foundational elements extracted from the direct causality topology. Notably, this approach efficiently discovers the core topology from the data, reducing redundancy without requiring prior domain-specific knowledge. Furthermore, it yields a precise equivalent model expression, offering a robust foundation for further analysis and exploration in various fields. Additionally, the proposed model for reconstructing an equivalent underlying framework demonstrates strong forecasting capabilities in multivariate time series scenarios.
In this paper, the reduced-order H∞ filtering problem for discrete-time singular systems under fading channels is studied. Our aim is to design a reduced-order filter to ensure the filtering error system with fading measurements is stochastically admissible and meets the given performance index. And the designed method is less conservative. However, the standard linear matrix inequality (LMI) cannot be obtained due to the nonlinear terms in decoupling. For this purpose, a new LMI is obtained by using new decoupling method. Finally, numerical example and electrical circuit example are given to clearly illustrate that the proposed method is less conservative than that of the existing method.
The subspace state-space system identification method has drawn extensive attention in structural modal identification, which is generally involved with the stabilization diagram for estimating structural modal parameters. However, the conventional stabilization diagram has an inherent problem, i.e., some spurious modes may be identified as stable results, leading to the adverse effect on structural modal identification. To address this issue, this paper proposes an improved subspace algorithm, in which a Monte Carlo-based stabilization diagram is involved. The performance of the Monte Carlo-based stabilization diagram for discriminating the poles denoting the physical modes from those representing spurious modes is demonstrated through a numerical study. The simulation results further prove that the proposed method can accurately estimate the time-varying structural modal parameters. Moreover, the proposed method is applied to field measurements on a 218-m-tall building during the 1994 Northridge earthquake event, and the identified results verify the applicability and effectiveness of the proposed method in field measurements. This paper aims to provide an effective tool for modal identification of high-rise buildings under earthquake excitations.
Computational fluid dynamics (CFD) models have been widely used in the chemical process industry to analyze various aspects of high-consequence rare events. However, CFD models are computationally intensive in nature, and therefore, several developments have been made in building computationally efficient models. The existing computationally efficient models in the field of high-consequence rare events are temporally static in nature and do not represent system evolution with time, which is crucial for consequence modeling of rare events. Further, the consequences depend on various parameters in addition to inputs. Since it is not affordable to construct a new model for every parameter value, incorporation of inputs and parameters is an additional challenge in developing a consequence model. Hence, to address these challenges, this work proposes a k-nearest neighbor (kNN)-based parametric reduced-order model (PROM) for consequence estimation of rare events to enhance numerical robustness with respect to parameter change. Specifically, local (with respect to parameters) ROMs are constructed based on multivariable output-error state space (MOESP) algorithm for a range of parameters, and they are linearly interpolated to estimate consequences at a new parameter value. The effectiveness of the proposed model is demonstrated through a case study of supercritical carbon dioxide release rare event.
A novel closed-loop numerical algorithm for subspace state space system identification (CN4SID) is proposed in this paper. Different from standard schemes, CN4SID algorithm can extend the standard LQ decomposition to the closed-loop cases by incorporating the controller information. Moreover, the proposed method can deliver unbiased pole estimation in the closed-loop framework. To this end, CN4SID algorithm shows superior pole estimation performance compared with a class of subspace identification method via principal component analysis algorithms. The effectiveness of the proposed CN4SID algorithm is finally verified through a practical dc motor system.
This paper is concerned with a novel data-driven bias-eliminated subspace identification approach for closed-loop systems. Compared with the existing methods, the proposed method firstly proposes to utilize the coprime factorization of the controller to construct an instrumental variable uncorrelated with noise under closed-loop conditions. Furthermore, it can reliably eliminate the pole estimation bias due to the correlation between inputs and noise under feedback control. More importantly, the proposed method establishes a general framework for both open-loop and closed-loop system identification. Performance comparisons with two other closed-loop methods are made from many different aspects. Finally, the performance of the identified system is again demonstrated in the vehicle lateral dynamic system.
This book addresses the needs of researchers and practitioners in the field of high-speed trains, especially those whose work involves safety and reliability issues in traction systems. It will appeal to researchers and graduate students at institutions of higher learning, research labs, and in the industrial R&D sector, catering to a readership from a broad range of disciplines including intelligent transportation, electrical engineering, mechanical engineering, chemical engineering, the biological sciences and engineering, economics, ecology, and the mathematical sciences.
Compared with signal analysis-based and model-based methods, data-driven FDD schemes can be implemented directly by sufficient use of information hidden in the abundant recorded data. Nowadays, a variety of advanced sensors can ensure implementation and promote development of the data-driven FDD methods. To our best knowledge, multivariate statistical analysis (MSA) and subspace identification method (SIM) are two parallel methods providing basic tools and techniques to deal with FDD problems in both stationary and dynamic operating conditions. Therefore, this chapter firstly describes the basics including MSA, SIM, together with the used test statistic, which serves as the fundamentals of this work; based on these basics, challenging topics of FDD applications to high-speed trains is then summarized.
The Covariance Based Realization Algorithm (CoBRA), one branch of subspace methods, enables the estimation of multivariable models using a large number of data points due to the use of finite size covariance matrices. In addition, the covariance pre-processing allows CoBRA to ignore any (high-order) noise dynamics and focus on the estimation of the low-order deterministic model. However, subspace methods and CoBRA in particular are not maximum-likelihood methods. In this paper, an in-depth study on the statistical behavior of the noise effects is conducted. An approach is provided to reduce the variance of an estimate obtained by CoBRA via the choice of optimal row and column weighting matrices. For closed-loop implementation of CoBRA, a two stage procedure is proposed with the estimate on an intermediate instrument. At the first stage, a least-length perturbation of a scalar accuracy function is used to obtain an analytic solution for the optimal weighting matrices. The resulting instruments produced by the first stage are used for the identification at the second stage to extract the plant model. Simulation examples are given to illustrate the efficiency of the proposed two stage CoBRA method on the basis of closed-loop data and compared with other subspace methods.
Subspace methods have been shown to be remarkably robust procedures providing consistent estimates of linear dynamical state-space systems for (multivariate) time series in different situations including stationary and integrated processes without the need for specifying the degree of persistence. Fractionally integrated processes bridge the gap between short-memory processes corresponding to stable rational transfer functions and integrated processes such as unit root processes. Therefore, it is of interest to investigate the robustness of subspace procedures for this class of processes. In this paper, it is shown that a particular subspace method called canonical variate analysis (CVA) that is closely related to long vector autoregressions (VAR) provides consistent estimators of the transfer function corresponding to the data generating process also for fractionally integrated processes of the VARFIMA or FIVARMA type, if integer parameters such as the system order tend to infinity as a suitable function of the sample size. The results are based on analogous statements for the consistency of long VAR modelling. In a simulation study, it is demonstrated that the model reduction implicit in CVA leads to accuracy gains for the subspace methods in comparison to long VAR modelling.
In identification of dynamical systems, the prediction error method (PEM) with a quadratic cost function provides asymptotically efficient estimates under Gaussian noise, but in general it requires solving a non-convex optimization problem, which may imply convergence to non-global minima. An alternative class of methods uses a non-parametric model as intermediate step to obtain the model of interest. Weighted null-space fitting (WNSF) belongs to this class, starting with the estimate of a non-parametric ARX model with least squares. Then, the reduction to a parametric model is a multi-step procedure where each step consists of the solution of a quadratic optimization problem, which can be obtained with weighted least squares. The method is suitable for both open- and closed-loop data, and can be applied to many common parametric model structures, including output-error, ARMAX, and Box-Jenkins. The price to pay is the increase of dimensionality in the non-parametric model, which needs to tend to infinity as function of the sample size for certain asymptotic statistical properties to hold. In this paper, we conduct a rigorous analysis of these properties: namely, consistency and asymptotic efficiency. Also, we perform a simulation study illustrating the performance of WNSF and identify scenarios where it can be particularly advantageous compared with state-of-the-art methods
In practice, sensor precision degradation is ubiquitous and early detection of such a degradation is important for monitoring task. In this paper, a cumulative canonical correlation analysis (CCA)-based sensor precision degradation detection method is presented in the Gaussian and non-Gaussian cases. At first, the fault sensitivity of the original CCA method to sensor precision degradation is theoretically analyzed. Then, the cumulative CCA-based method is proposed and delivers better fault detectability than the corresponding standard CCA-based method with respect to fault detection rate. For non-Gaussian case, an efficient and practical applicable approach, referred as threshold learning approach, is proposed to set appropriate threshold based on available historical measurements. Finally, with the application to a real laboratorial continuous stirred tank heater plant, the feasibility and superiority of the proposed method are demonstrated by a comparison with the standard CCA-based and principal component analysis-based methods.
It has been proven that combining open-loop subspace identification with prior information can promote the accuracy of obtaining state-space models. In this study, prior information is exploited to improve the accuracy of closed-loop subspace identification. The proposed approach initially removes the correlation between future input and past innovation, a significant obstacle in closed-loop subspace identification method. Then, each row of the extended subspace matrix equation is considered an optimal multi-step ahead predictor and prior information is expressed in the form of equality constraints. The constrained least squares method is used to obtain improved results, so that the accuracy of the closed-loop subspace can be enhanced. Simulation examples are provided to demonstrate the effectiveness of the proposed algorithm.
The applications of the canonical variate, Kalman filtering and maximum likelihood parameter identification techniques to the requirements of the National Weather Service in river flow forecasting are investigated. State space reduced-order models for unit hydrographs are obtained with the use of canonical variate methods. A complete state-space model for a catchment consisting of the Sacramento model as the soil moisture system and the basin's unit hydrograph as the channel routing system is constructed. This model is used in the design of extended Kalman filters for the prediction of the channel discharge and the state of the system, and also in the design of an algorithm for the identification of catchment model parameters through the use of maximum likelihood techniques. The performance of the algorithms is demonstrated with synthetic data generated with the models for the Bird Creek and White River basins.
This article is a study of infinite Hankel matrices and approximation problems connected with them.
Bibliography: 22 items.
Let and be two random vectors. Given any random vector , let be the best linear predictor of based on . Let p be any natural number smaller than m. We consider the problem of finding the p-dimensional random vector where each component is a linear function of , which minimizes the determinant of . We show that coincide with the first p canonical variables (except for a nonsingular linear transformation). We also show that the square of the th canonical correlation coefficient measures the relative improvement in the prediction of when 's are used instead of p.
This chapter discusses several well-known matrix decompositions and their relevance to statistical calculation. It discusses some of the properties of numerical algorithms. The chapter discusses Cholesky decomposition. Given the Cholesky decomposition, it is a simple matter to solve a system of linear equations or to compute the inverse. The chapter reviews the conditioning of matrices, iterative refinement, partial correlation, and least squares. In many statistical calculations, it is necessary to compute certain auxiliary information associated with ATA. These can readily be obtained from the orthogonal decomposition. The chapter reviews the classical Gram–Schmidt algorithm (CGSA) and the modified Gram–Schmidt algorithm (MGSA). One should never use the CGSA without reorthogonalization, which greatly increases the amount of computation. Reorthogonalization is never needed when using the MGSA. The MGSA has the advantages that it is relatively easy to program and experimentally, it seems to be slightly more accurate than the Householder procedure.
The structure of the information interface between the future and the past of a discrete-time stochastic process is analyzed by using the concepts of canonical correlation analysis. Two extreme Markovian representations are obtained with states defined by the sets of canonical variables which represent the past information projected on the future and the future information projected on the past, respectively. The result completely clarifies the probabilistic structure of the Faurre algorithm of realization of stochastic systems. By an extension of the basic result the Ho - Kalman algorithm of realization of general systems is also given a stochastic interpretation.
This chapter starts with a brief introductory review of some of the recent developments of time-series analysis. One of the most established procedures of time-series analysis is the method of estimation of power spectrum through windowed sample covariance sequence. Except for the special situations where the orders are specified in advance, any method of the autoregressive model fitting is not well-defined as a method of estimation of the covariance sequence in the absence of the description of the rules for the determination of the order. Because the covariance sequence determines the power spectrum, once an appropriate rule for the order determination is given, the autoregressive-model fitting procedure automatically provides an estimate of the power spectrum. A solution to the problem of order determination of an auto-regressive model was obtained in 1969 by using the concept of final prediction error (FPE), which is defined as the one-step ahead prediction error variance when the least squares estimates of the autoregressive coefficients are used for prediction. The concept of the one-step ahead of prediction error variance had difficulty in extending the minimum FPE procedure to the multivariate situation because of the non-uniqueness of the measure of variance in the multivariate situation. A solution was found with the aid of the Gaussian model and the concept of maximum likelihood, which suggested the use of the generalized variance of the one-step ahead of prediction error.
Summary The problem of identifiability of a multivariate autoregressive moving average process is considered and a complete solution
is obtained by using the Markovian representation of the process. The maximum likelihood procedure for the fitting of the
Markovian representation is discussed. A practical procedure for finding an initial guess of the representation is introduced
and its feasibility is demonstrated with numerical examples.
This paper is concerned with (1) the problem of the construction of lower-order models and (2) the Telated problem of the order determination of a real system based upon an estimated model with an overestimated order. Methods of the construction of stable lower-order models and the system-order determination are proposed. The approach adopted is to obtain a minimal realization of the original system by taking the principal components of the predictors of the outputs as the state and then to construct reduced models based upon a measure of reducibility defined in connection with the minimal realization algorithm. The measure of reducibility is useful to get a priori information about how small the order of the reduced model can be without much deterioration. Simulation studies are also carried out to demonstrate the effectiveness of the measure of reducibility and the proposed methods.
The existence and the explicit form of the minimal Markov process which contains as a component a given stationary process are established. It is shown in particular that the future/past splitting subspace of the multivariate stationary process is finite-dimensional if and only if the process has a rational spectral densities matrix.
The property of being stochastically continuous is obtained as the condition for continuation of the sigma-algebras associated with a process with independent increments which usually represents a stochastic disturbance of the system considered. This property gives us the left-side continuation of the sigma-algebras in the case of the arbitrary process in a metric space.
f
∗(γ) () denotes the regular extension of so that , ().
A technique is described for developing state-space models from vector time series. The technique is based on canonical variates analysis: a form of least-squares multi-step linear prediction. Unlike Gaussian maximum likelihood and one-step linear prediction techniques for state-space modeling, state-space models are generated by solving a finite number of linear equations. The approach is suited to off-line modeling and fragmented data sets. The technique has been used for spectrum estimation, reduced-order modeling, and Kalman filtering.
In a series of papers Lindquist and Picci [13- 19] have developed an abstract theory of stochastic realization for continuous-time stationary (and stationary increment) Gaussian vector processes, consisting of a basic geometric theory in Hilbert space and a characterization of realizations in terms of Hardy functions. In this paper we present a discrete-time version of this theory and illustrate it by a number of concrete examples. Although some of the results turn out to be straight-forward modifications of the continuous-time ones, obtained by translation from the imaginary axis to the unit circle, there are a number of decidedly nontrivial problems which are unique to the discrete-time setting and which yield new results. Among these are such questions as the definition of past and future, different choices creating different models, and, in particular, certain degeneracies which do not occur in the continuous-time case. One type of degeneracy is related to the singularity of the transition function, another to the concept of invariant directions.
For a finite measure d[Delta] on R1 the closed linear span in Lp(d[Delta]) of the exponentials {eiix: [short parallel] t [short parallel] <= T} is discussed. These results are applied in the characterization of the spectral densities of stationary Gaussian processes which exhibit a Markovian character.
Developments in the theory of linear least-squares estimation in the last thirty years or so are outlined. Particular attention is paid to early mathematica[ work in the field and to more modern developments showing some of the many connections between least-squares filtering and other fields.
The realization of Markovian models for nonstationary processes generated by time invariant linear systems is considered. A model is obtained by constructing an orthogonal finite-step predictor for the process. Optimal model approximation by order reduction is naturally defined in this framework. The construction and reduction of Markovian models from multiple data records and the numerical considerations involved are illustrated by examples. Copyright © 1981 by The Institute of Electrical and Electronics Engineers, Inc.
This paper represents a first attempt to derive a closed-form (Hankel-norm) optimal solution for multivariable system reduction problems. The basic idea is to extend the scalar ease approach in [5] to deal with the multivariable systems. The major contribution lies in the development of a minimal degree approximation (MDA) theorem and a computation algorithm. The main theorem describes a closed-form formulation for the optimal approximants, with the optimality verified by a complete error analysis. In deriving the main theorem, some useful singular value/vector properties associated with block-Hankel matrices are explored and a key extension theorem is also developed. Imbedded in the polynomial-theoretic derivation of the extension theorem is an efficient approximation algorithm. This algorithm consists of three steps: i) compute the minimal basis solution of a polynomial matrix equation; ii) solve an algebraic Riccati equation; and iii) find the partial fraction expansion of a rational matrix.
In this paper it is shown that a natural representation of a state space is given by the predictor space, the linear space spanned by the predictors when the system is driven by a Gaussian white noise input with unit covariance matrix. A minimal realization corresponds to a selection of a basis of this predictor space. Based on this interpretation, a unifying view of hitherto proposed algorithmically defined minimal realizations is developed. A natural minimal partial realization is also obtained with the aid of this interpretation.
A Gaussian stochastic process (y t ) with known covariance kernel is given: we investigate the generation of (y t ) by means of Markovian schemes of the type dx t = F(t)x t dt + dw t y t = H(t)x t . Such a generation of (y t ) as the "output of a linear dynamical system driven by white noise" is possible under certain finiteness conditions. In fact, this was shown by Kalman in 1965. We emphasize the probabilistic aspects and obtain an intrinsic characterization of the state of the process as the state of an externally described stochastic I/O map. Realizations of (y t ) can be constructed with respect to any increasing family of ω-fields; in particular, when the family of ω-fields is induced by the process itself, the driving white noise reduces to the innovation process of (y t ). The corresponding realization has been referred to as the "innovation representation" of (y t ).
Approximate Realization of Discrete Time Stochastic Processes
- E Camuto
- G Menga
Statistical Modeling of Gravity Disturbances Using Measured Anomaly Data
- W E Larimore
- S W Thomas
- S I Baumgartner
Monitor Guidance Study
- D W Porter
- W E Larimore