Fig 7 - uploaded by Péter Zoltán Csurcsia
Content may be subject to copyright.
a) Block structure of the linear FIR model considered in the first case. b) Unregularized (left) and regularized (right) FIR estimation. c) True (black) and modelled (red) system output.
Source publication
This paper presents an efficient nonparametric time domain nonlinear system identification method. It is shown how truncated Volterra series models can be efficiently estimated without the need of long, transient-free measurements. The method is a novel extension of the regularization methods that have been developed for impulse response estimates...
Contexts in source publication
Context 1
... response estimation has been implemented with the Marginal Likelihood and the residual analysis method, respectively. For both cases the Volterra models have also been obtained with the inversion-free method (see Section 4), and all the results on the performance of the models as well as of the different algorithms are summarized in Table.1. In Fig. 7-9 block structures equivalent to the identified models are provided for illustration purposes ( (), () and () represent linear FIR ...
Context 2
... system is first modelled with a simple FIR model depicted in Fig. 7a. Even though the prior information of stability and smoothness is imposed on the modelled response as suggested by Fig. 7b, it is clear from Fig. 7c that the linear dynamics are not sufficient to describe the input -output behaviour of the water tanks ...
Context 3
... system is first modelled with a simple FIR model depicted in Fig. 7a. Even though the prior information of stability and smoothness is imposed on the modelled response as suggested by Fig. 7b, it is clear from Fig. 7c that the linear dynamics are not sufficient to describe the input -output behaviour of the water tanks ...
Context 4
... system is first modelled with a simple FIR model depicted in Fig. 7a. Even though the prior information of stability and smoothness is imposed on the modelled response as suggested by Fig. 7b, it is clear from Fig. 7c that the linear dynamics are not sufficient to describe the input -output behaviour of the water tanks ...
Context 5
... model outputs shown in Fig. 7-9 are calculated on the transient-free estimation dataset. Figure 9c shows the second order model fit on the transient-free dataset, and Fig. 10 shows the model fit on the entire validation dataset. It can be observed that the level of transients is very high. The algorithm explained in Section 5 is used to remove the undesired transient ...
Similar publications
System identification is a fundamental problem in reinforcement learning, control theory and signal processing, and the non-asymptotic analysis of the corresponding sample complexity is challenging and elusive, even for linear time-varying (LTV) systems. To tackle this challenge, we develop an episodic block model for the LTV system where the model...
Citations
... Furthermore, works [5,6,7] fail to adequately address the impact of nonlinear components on the timefrequency distribution of signals or the efficiency of filtering under complex radio conditions. Key issues include managing computational complexity [8][9][10], optimizing convergence in adaptive models [11], and extending methods for real-time applications [12]. Despite advancements, critical gaps remain, including limited attention to high-dimensional Volterra kernels, inadequate real-time applicability, and insufficient adaptability to dynamic environments or higher-order nonlinearities. ...
... Presentation of the main material and substantiation of the obtained research results. In a linear system, the relationship between the input signal ( ) and the output signal ( ) is described by the impulse response ℎ( ), which represents the first-order Volterra kernel and is mathematically expressed as [9]: ...
... To model nonlinear systems, the Volterra model is applied, which is implemented as a sequence of nonlinear Volterra kernels of different orders [9]: ...
The article presents a study on the application of Volterra series for modeling nonlinear signal components in the frequency domain. The proposed spectral reconstruction algorithm accounts for the impact of signal nonlinearity on its frequency-time distribution. Volterra series enable the extraction of nonlinear components in the frequency spectrum, improve the accuracy of signal reconstruction, and optimize filtering in complex radio environments. Experimental calculations demonstrated the algorithm's effectiveness in reducing mean-square error (MSE) and mean-square deviation (MSD) by up to 20,55% compared to lower-order models. The algorithm showed the ability to preserve the accuracy of signal amplitude characteristics by 10,3% better than first-order models and to ensure more precise phase reproduction with a 5,2% improvement. In dynamic radio environments, the algorithm significantly reduced the impact of inter-channel and inter-symbol interference, enhancing signal robustness. Specifically, at key time points, the second-order model reduced MSE by an average of 43,6–57,8% compared to the first-order model. The prospects for further research include the development of the algorithm for multichannel communication systems, integration of machine learning methods for dynamic parameter tuning during reconstruction, and the expansion of its application in cognitive radio networks with highly variable environments
... The neural loop training achieved an RMSE of 0.2912 indicating a satisfactory level of accuracy in the model's predictions. Although our proposed approach has resulted in better RMSE results compared to the state-of-the-art blackbox nonlinear identification methods used for this benchmark (Birpoutsoukis, Csurcsia, & Schoukens, 2018;Relan, Tiels, Marconato, & Schoukens, 2017;Svensson & Schön, 2017), it is important to note that the main goal of this approach is not to obtain a highly accurate model of the system. Rather, the primary objective is to develop a model that can effectively function within the RL-MPC framework. ...
This paper presents an end-to-end learning approach to developing a Nonlinear Model Predictive Control (NMPC) policy, which does not require an explicit first-principles model and assumes that the system dynamics are either unknown or partially known. The paper proposes the use of available measurements to identify a nominal Recurrent Neural Network (RNN) model to capture the nonlinear dynamics , which includes constraints on the state variables and input. To address the issue of suboptimal control policies resulting from simply fitting the model to the data, the paper uses Reinforcement learning (RL) to tune the NMPC scheme and generate an optimal policy for the real system. The approach's novelty lies in the use of RL to overcome the limitations of the nominal RNN model and generate a more accurate control policy. The paper discusses the implementation aspects of initial state estimation for RNN models and integration of neural models in MPC. The presented method is demonstrated on a classic benchmark control problem: cascaded two tank system (CTS).
... Since time-varying systems are often misinterpreted as nonlinear systems, it is important to mention that when G varies over the measurement time, but at each time instant the principle of superposition is satisfied, then the system is called linear timevarying (LTV) [15] [16] [17]. In this article, we assume that the underlying systems are damped, bounded-input, bounded-output stable, time-invariant, nonlinear systems that can be adequately described with a (smooth) low degree Volterra-series [18] and the linear response of the system is still present and identifiable. In addition, we assume that the output of the underlying system has the same period as the excitation signal (i.e. the system has PISPO (period in, same period out) behavior [19]. ...
A known challenge when building nonlinear models from data is to limit the size of the model in terms of the number of parameters. Especially for complex nonlinear systems, which require a substantial number of state variables, the classical formulation of the nonlinear part (e.g. through a basis expansion) tends to lead to a rapid increase in the model size. In this work, we propose two strategies to counter this effect: 1) The introduction of a novel nonlinear-state selection algorithm. The method relies on the non-parametric nonlinear distortion analysis of the Best Linear Approximation framework to identify the state variables which are the most impacted by nonlinearities. Pre-selecting only the most appropriate states when constructing the nonlinear terms results in a considerable reduction of the model size. 2) The use of so-called 'decoupled' functions directly in the model estimation procedure. While it is known that function decoupling can reduce the model size in a secondary step, we show how a decoupled formulation can be imposed to advantage from the start. The results of this approach are benchmarked with the state-of-the-art a posteriori decoupling technique. Our strategies are demonstrated on real-life data of a multiple-input, multiple-output (MIMO) ground vibration test of an F-16 aircraft, a prime complex and nonlinear dynamic system. 1 INTRODUCTION Engineers and scientists want mathematical models of the observed system for understanding, design and control. Modeling nonlinear systems is essential because many systems are inherently nonlinear. The challenge lies in the fact that there are several differently behaving nonlinear structures and therefore modeling is very involved. As it becomes increasingly important to cope with nonlinear analysis and modeling, various approaches have been proposed; for a detailed overview we refer to [1] [2] and [3]. In this work, we propose a data-driven nonlinear modeling procedure where we build upon a number of well-known, matured, system identification techniques, and add two novel tools in order to overcome some of the drawbacks of the classical approach. In doing so, we provide a complete modeling strategy which allows retrieving compact nonlinear state-space models from data. The procedure combines both nonparametric and parametric nonlinear modeling techniques and is particularly useful when dealing with complex nonlinear systems, such as dynamic structures with many resonances. An important domain of application is found in the modeling of multiple-input, multiple-output (MIMO) real-life vibro-acoustic measurements. We illustrate the methodologies on a ground vibration test of an F-16 aircraft. The recommended nonlinear modeling procedure is listed below and illustrated in Figure 1. ▪ In the experiment design step, systems are excited by broadband (multisine) signals at multiple excitation levels. The recommended multisine (also known as pseudo-random noise) excitation signal consists of a series of periodic multisines that are mutually independent over the experiments. The main advantage of the recommended signals is that there is no problem with spectral leakage or transients. They deliver excellent linear models while providing useful information about the level and type of nonlinearities. ▪ In the second step, the measured signals are (nonparametrically) analyzed by applying the (multisine-driven) Best Linear Approximation (BLA) framework of MIMO systems as a generalization of the conceptual work [4]. Even though the technique works best with the recommended multisines, (with some loss of accuracy) any (orthogonal) signal can be applied. This (multisine-driven) BLA analysis differs from the classical H 1 Frequency Response Function (FRF) estimation process [5]. The key idea is to make use of the statistical features of the excitation signal. The outcome of the BLA analysis results in a series of nonparametric FRFs together with noise and nonlinear distortion estimates.
... This together with = 2 ∕(̂) yields ≤ . Meanwhile, as there are ( − 1) equations and ( − 1) unknowns in Eq. (26), to avoid an under-determined case requires ≥ . Therefore, and must be identical and̂= 2 ∕ . ...
... Therefore, and must be identical and̂= 2 ∕ . Now, in Eq. (26) is a full-rank square matrix, and numerically solving can be done by the Gaussian elimination method or an iterative solver with regularization techniques when dealing with noisy data [26]. Solving amounts to solving . ...
This study develops an efficient method using input-output data on estimating non-parametric and then parametric frequency response functions (FRFs) associated with a dynamic system. Contrary to most existing FRF estimation methods that have been theoretically based on the steady-state or stationary responses, the proposed method uses the transient responses that are induced by simple, such as sinusoidal or two-exponential, excitations instead. Its numerical procedure for obtaining a non-parametric FRF is nothing more than solving a set of linear equations in which the unknown variables are individual components of the sought FRF. Furthermore, from the non-parametric FRF, its corresponding poles and residues are extracted through a procedure that involves a usage of the inverse discrete Fourier transform and the Prony-SS method; afterwards, a partial fraction FRF in terms of poles and residues can be achieved. Three numerical examples, including computer simulations and lab experiment, are provided to demonstrate the superior performance of the developed method.
... RMS BLA [24] 0.75 Volterra model [5] 0.54 State-space with GP-inspired prior [31] 0.45 SCI [13] 0.40 NL-SS + NLSS2 [24] 0.34 TSEM [13] 0.33 Tensor B-splines [17] 0.30 neural ODE [9] with normalization (∆t/τ = 0.03) 0.18 (0.33) Grey-Box with physical overflow model [26] 0.18 Table 1 also contains neural ODE with normalization. Only a small fraction of the estimated neural ODE with normalization models was able to get comparable results to the state-of-the-art. ...
Continuous-time (CT) models have shown an improved sample efficiency during learning and enable ODE analysis methods for enhanced interpretability compared to discrete-time (DT) models. Even with numerous recent developments, the multifaceted CT state-space model identification problem remains to be solved in full, considering common experimental aspects such as the presence of external inputs, measurement noise, and latent states. This paper presents a novel estimation method that includes these aspects and that is able to obtain state-of-the-art results on multiple benchmarks where a small fully connected neural network describes the CT dynamics. The novel estimation method called the subspace encoder approach ascertains these results by altering the well-known simulation loss to include short subsections instead, by using an encoder function and a state-derivative normalization term to obtain a computationally feasible and stable optimization problem. This encoder function estimates the initial states of each considered subsection. We prove that the existence of the encoder function has the necessary condition of a Lipschitz continuous state-derivative utilizing established properties of ODEs.
... An alternative approach to this issue is the data-driven modeling method known as system identification, which can be used to establish the numerical model of a nonlinear system by the input and output data sets. The Volterra model [3,4], some modern Volterra series approaches [5,6], and Hammerstein-Wiener model [7][8][9] have been widely used for representing the numerical representation of the nonlinear system. In the past years, the Nonlinear Auto-Regressive with exogenous inputs (NARX) model, which was introduced in 1985 [10,11], was widely used for establishing the nonlinear system models for prediction and analysis due to the convenience (see [12][13][14][15]). ...
... θ 4 (ϕ) = 0.0461 + 0.0032ϕ + 0.0096ϕ 2 + 1.42 × 10 −4 ϕ 3 θ 5 (ϕ) = 0.0037 + 0.0049ϕ + 0.0084ϕ 2 + 7.7501 × 10 −4 ϕ 3 (19) Note that the coefficients of these polynomial functions are randomly generated. Three data sets of 5121 sample points were generated as the input signals, and then the output signals corresponding to the design parameter ϕ = [2,4,6] are obtained by Equations (18) and (19), which are used for modeling. The data sets are shown in Figure 1. ...
... Note that the coefficients of these polynomial functions are randomly generated. Three data sets of 5121 sample points were generated as the input signals, and then the output signals corresponding to the design parameter φ = [2,4,6] are obtained by Equations (18) and (19), which are used for modeling. The data sets are shown in Figure 1. ...
In this paper, the Nonlinear Auto-Regressive with exogenous inputs (NARX) model with parameters of interest for design (NARX-M-for-D), where the design parameter of the system is connected to the coefficients of the NARX model by a predefined polynomial function is studied. For the NARX-M-for-D of nonlinear systems, in practice, to predict the output by design parameter values are often difficult due to the uncertain relationship between the design parameter and the coefficients of the NARX model. To solve this issue and conduct the analysis and design, an improved algorithm, defined as the Weighted Extended Forward Orthogonal Regression (WEFOR), is proposed. Firstly, the initial NARX-M-for-D is obtained through the traditional Extended Forward Orthogonal Regression (EFOR) algorithm. Then a weight matrix is introduced to modify the polynomial functions with respect to the design parameter, and then an improved model, which is referred to as the final NARX-M-for-D is established. The genetic algorithm (GA) is used for deriving the weight matrix by minimizing the normalized mean square error (NMSE) over the data sets corresponding to the design parameter values used for modeling and first prediction. Finally, both the numerical and experimental studies are conducted to demonstrate the application of the WEFOR algorithm. The results indicate that the final NARX-M-for-D can accurately predict the system output of a nonlinear system. The new algorithm is expected to provide a reliable model for dynamic analysis and design of the nonlinear system.
... Unfortunately, this number grows rapidly with the system memory and the order of the Volterra series. Despite the valuable implementation adjustments proposed in Birpoutsoukis et al. (2018), these requirements already make it hard to introduce monomials of degree three. ...
... The performance of SED-MPK is compared with the results previously obtained on this benchmark relying on Volterra series. In particular, we considered the results reported in Birpoutsoukis, Csurcsia et al. (2017), Birpoutsoukis et al. (2018), where authors also described the evolution of the water level with a third order regularized Volterra series (RVS), applying the strategy described in Section 2 that relies on (6), (7). Fig. 5 plots the test output together with the estimateŷ returned by SED-MPK. ...
... In fact, while RVS requires 6.5 h to identify the model, our approach needs only 2 minutes. As explained in Section 2, this is due to the fact that the computational time of the solution proposed in Birpoutsoukis, Csurcsia et al. (2017), Birpoutsoukis et al. (2018) depends on the number of coefficients of the Volterra maps while the computational time of SED-MPK depends only on the number of training samples. ...
Volterra series approximate a broad range of nonlinear systems. Their identification is challenging due to the curse of dimensionality: the number of model parameters grows exponentially with the complexity of the input–output response. This fact limits the applicability of such models and has stimulated recently much research on regularized solutions. Along this line, we propose two new strategies that use kernel-based methods. First, we introduce the multiplicative polynomial kernel (MPK). Compared to the standard polynomial kernel, the MPK is equipped with a richer set of hyperparameters, increasing flexibility in selecting the monomials that really influence the system output. Second, we introduce the smooth exponentially decaying multiplicative polynomial kernel (SED-MPK), that is a regularized version of MPK which requires less hyperparameters, allowing to handle also high-order Volterra series. Numerical results show the effectiveness of the two approaches.
... The performance measurement of Volterra-based nonlinear system using both the methods against the various identification performance metrics is shown in Figs. 3,4,5,6,7,8,9,10,11,12. From these figures, it is noticed that the GGS-KF results in the most accurate identification compared to the basic KF and other reported approaches. ...
... The output y(n) is the water level of the bottom tank to be measured. The state-space models of cascaded tanks, shown in Fig. 12a to measure the water level of the lower tank, are given in (37)-(39), These have been modelled based on the Bernoulli's principle and conservation of mass under no saturation condition [8,43]. ...
... Moreover, authors have provided a web link to download these data sets in [43]. In [8], Birpoutsoukis et al., have used those data sets for estimation of cascaded water tanks using the Volterra model. In this paper, the same data sets have also been used to estimate the Volterra kernels of cascaded water tanks using KF and GGS-KF methods. ...
This paper proposes an efficient global gravitational search (GGS) algorithm-assisted Kalman filter (KF) design, called a GGS-KF technique, for accurate estimation of the Volterra-type nonlinear systems. KF is a well-known estimation technique for the dynamic states of the system. The best estimate is achieved if the system dynamics and noise statistical model parameters are available at the beginning. However, to estimate the real-time problems, these parameters are unstipulated or partly known. Due to this limitation, the performance of the KF degrades or sometimes diverges. In this work, two steps have been proposed for unknown system identification while overcoming the difficulty encountered in KF. The first step is to optimise the parameters of the KF using the GGS algorithm by considering a properly balanced fitness function. The second step is to estimate the unknown coefficients of the system by using the basic KF method with the optimally tuned KF parameters obtained from the first step. The proposed GGS-KF technique is tested on five different Volterra systems with various levels of noisy (10 dB, 15 dB and 20 dB) and noise-free input conditions. The simulation results confirm that the GGS-KF-based identification approach results in the most accurate estimations compared to the conventional KF and other reported techniques in terms of parameter estimation error, mean-squared error (MSE), fitness percentage (FIT%), mean-squared deviation (MSD), and cumulative density function (CDF). To validate the practical applicability of the proposed technique, two benchmark systems have also been identified based on the original data sets.
... The RMSE index is 0.08 V and 0.33 V on the identification and on the test dataset, respectively. These results compare favorably with stateof-the-art black-box nonlinear identification methods applied to this benchmark [46, 47,48]. To the best of our knowledge, the best previously published result was obtained in [46] using non-linear basis expansion state-space models and 580 exploiting a state initialization procedure (called NLSS2) based on a preliminary (linear) subspace identification for optimization. ...
This paper presents tailor-made neural model structures and two custom fitting criteria for learning dynamical systems. The proposed framework is based on a representation of the system behavior in terms of continuous-time state-space models. The sequence of hidden states is optimized along with the neural network parameters in order to minimize the difference between measured and estimated outputs, and at the same time to guarantee that the optimized state sequence is consistent with the estimated system dynamics. The effectiveness of the approach is demonstrated through three case studies, including two public system identification benchmarks based on experimental data.
... For example, Volterra kernels are generalizations of finite impulse responses to higher orders and are therefore expected to be smoothly decaying. Recent research enforces these constraints explicitly through regularization Birpoutsoukis et al. (2017Birpoutsoukis et al. ( , 2018, but these methods are unfortunately limited to third order kernels. Another way of adding structure to the kernels is by expanding them in terms of orthonormal basis functions Campello et al. (2004); Diouf et al. (2012). ...
The estimation of an exponential number of model parameters in a truncated Volterra model can be circumvented by using a low-rank tensor decomposition approach. This low-rank property of the tensor decomposition can be interpreted as the assumption that all Volterra parameters are structured. In this article, we investigate whether it is possible to explicitly enforce symmetry of the Volterra kernels to the low-rank tensor decomposition. We show that low-rank symmetric Volterra identification is an ill-conditioned problem as the low-rank property of the exact symmetric kernels cannot be upheld in the presence of measurement noise. Furthermore, an algorithm is derived to compute the symmetric Volterra kernels directly in tensor network form.
