No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Fast time-series prediction using high-dimensional data: Evaluating confidence interval credibility
Abstract
I propose an index for evaluating the credibility of confidence intervals for future observables predicted from high-dimensional time-series data. The index evaluates the distance from the current state to the data manifold. I demonstrate the index with artificial datasets generated from the Lorenz'96 II model [Lorenz, in Proceedings of the Seminar on Predictability, Vol. 1 (ECMWF, Reading, UK, 1996), p. 1], the Lorenz'96 I model [Hansen and Smith, 2859:TROOCI>2.0.CO;2">J. Atmos. Sci. 57, 2859 (2000).
... In addition, whether our modeling becomes the first order or not because there were few similar events in the past can be evaluated simultaneously without additional computational costs. In Ref. 19, the first author proposed an index to identify when the prediction goes wrong because there were few similar events in the past. ...
... The proposed method is different from the method of Ref. 19 because the proposed method constructs barycentric coordinates for high-dimensional dynamics, while the method of Ref. 19 proposes when the prediction may go wrong because few similar events happened in the past, while Eq. (5) may be used as an alternative for the index of Ref. 19. ...
... The proposed method is different from the method of Ref. 19 because the proposed method constructs barycentric coordinates for high-dimensional dynamics, while the method of Ref. 19 proposes when the prediction may go wrong because few similar events happened in the past, while Eq. (5) may be used as an alternative for the index of Ref. 19. ...
The increasing development of novel methods and techniques facilitates the measurement of high-dimensional time series but challenges our ability for accurate modeling and predictions. The use of a general mathematical model requires the inclusion of many parameters, which are difficult to be fitted for relatively short high-dimensional time series observed. Here, we propose a novel method to accurately model a high-dimensional time series. Our method extends the barycentric coordinates to high-dimensional phase space by employing linear programming, and allowing the approximation errors explicitly. The extension helps to produce free-running time-series predictions that preserve typical topological, dynamical, and/or geometric characteristics of the underlying attractors more accurately than the radial basis function model that is widely used. The method can be broadly applied, from helping to improve weather forecasting, to creating electronic instruments that sound more natural, and to comprehensively understanding complex biological data.
... This part is a topic for future research. Although our examples here are limited to the ones in foreign exchange markets, the proposed methods themselves are general, and thus can be applied to wider contexts including the prediction of abrupt changes for renewable energy outputs [39][40][41]. These applications are for the example of the Lorenz'96 model with 1% observational noise using equation (4). ...
We apply the idea of dynamical network markers (Chen et al 2012 Sci. Rep. 2 342) to foreign exchange markets so that early warning signals can be provided for any abrupt changes. The dynamical network marker constructed achieves a high odds ratio for forecasting these sudden changes. In addition, we also extend the notion of the dynamical network marker by using recurrence plots so that the notion can be applied to delay coordinates and point processes. Thus, the dynamical network marker is useful in a variety of contexts in science, technology, and society.
Many problems in the geophysical sciences demand the ability to calibrate the parameters and predict the time evolution of complex dynamical models using sequentially-collected data. Here we introduce a general methodology for the joint estimation of the static parameters and the forecasting of the state variables of nonlinear, and possibly chaotic, dynamical models. The proposed scheme is essentially probabilistic. It aims at recursively computing the sequence of joint posterior probability distributions of the unknown model parameters and its (time varying) state variables conditional on the available observations. The latter are possibly partial and contaminated by noise. The new framework combines a Monte Carlo scheme to approximate the posterior distribution of the fixed parameters with filtering (or {\em data assimilation}) techniques to track and predict the distribution of the state variables. For this reason, we refer to the proposed methodology as {\em nested filtering}. In this paper we specifically explore the use of Gaussian filtering methods, but other approaches fit naturally within the new framework. As an illustrative example, we apply three different implementations of the methodology to the tracking of the state, and the estimation of the fixed parameters, of a stochastic two-scale Lorenz 96 system. This model is commonly used to assess data assimilation procedures in meteorology. For this example, we compare different nested filters and show estimation and forecasting results for a 4,000-dimensional system.
Although predicting sudden rapid changes of renewable energy outputs is useful for maintaining the stability of power grids with many renewable energy resources, the prediction is difficult so far. Here we list causes for the uncertainty for our prediction, quantify them, and forecast whether such sudden rapid changes are likely to happen or not by integrating their quantifications with a method of machine learning. We test the proposed forecast using a toy model and real datasets of solar irradiance and wind speed.
We summarize our recent developments of time series prediction for renewable energy. Given the past parts of high-dimensional time series for renewable energy outputs, we can predict their multistep future in real time with confidence intervals. We also proposed a way to evaluate the closeness in the high-dimensional space for improving the prediction, and an index showing when the prediction is more likely to fail. In addition, it is straightforward to apply the proposed framework to predict the electricity demands. Therefore, we can generate information necessary to consider efficient unit commitments for a case where more renewable energy resources are installed.
Critical transitions in multistable systems have been discussed as models for
a variety of phenomena ranging from the extinctions of species to
socio-economic changes and climate transitions between ice-ages and warm-ages.
From bifurcation theory we can expect certain critical transitions to be
preceded by a decreased recovery from external perturbations. The consequences
of this critical slowing down have been observed as an increase in variance and
autocorrelation prior to the transition. However especially in the presence of
noise it is not clear, whether these changes in observation variables are
statistically relevant such that they could be used as indicators for critical
transitions. In this contribution we investigate the predictability of critical
transitions in conceptual models. We study the the quadratic integrate-and-fire
model and the van der Pol model, under the influence of external noise. We
focus especially on the statistical analysis of the success of predictions and
the overall predictability of the system. The performance of different
indicator variables turns out to be dependent on the specific model under study
and the conditions of accessing it. Furthermore, we study the influence of the
magnitude of transitions on the predictive performance.
We propose a general method for predicting multiple steps ahead of our target system and estimating simultaneously the prediction errors in a real time. The requirement of the proposed method is that we have a time series of the target system. We demonstrate the method by artificial data, real wind speed data, and real solar irradiation data.
Qualitative features of a one-dimensional lattice of coupled-logistic maps are investigated. First, kink-antikink patterns of 2(n) -periodic cycles with their period-doubling bifurcations are found. Secondly, antiferro-like structures with some kinks are observed, which show the transition from torus to chaos. Lastly, spatial intermittent structures are investigated, with the emphasis on the propagation of bursts.
This paper investigates the nature of model error in complex deterministic nonlinear systems such as weather forecasting models. Forecasting systems incorporate two components, a forecast model and a data assimilation method. The latter projects a collection of observations of reality into a model state. Key features of model error can be understood in terms of geometric properties of the data projection and a model attracting manifold. Model error can be resolved into two components: a projection error, which can be understood as the model's attractor being in the wrong location given the data projection, and direction error, which can be understood as the trajectories of the model moving in the wrong direction compared to the projection of reality into model space. This investigation introduces some new tools and concepts, including the shadowing filter, causal and noncausal shadow analyses, and various geometric diagnostics. Various properties of forecast errors and model errors are described with reference to low-dimensional systems, like Lorenz's equations; then, an operational weather forecasting system is shown to have the same predicted behavior. The concepts and tools introduced show promise for the diagnosis of model error and the improvement of ensemble forecasting systems.
Solar power, wind power, and co-generation (com-bined heat and power) systems are possible candidate for household power generation. These systems have their advan-tages and disadvantages. To propose the optimal combination of the power generation systems, the extraction of basic patterns of energy consumption of the house is required. In this study, energy consumption patterns are modeled by mixtures of Gaussian distributions. Then, using the symmetrized Kullback-Leibler divergence as a distance measure of the distributions, the basic pattern of energy consumption is extracted by means of hierarchical clustering. By an experiment using the Annex 42 dataset, it is shown that the proposed method is able to extract typical energy consumption patterns.
The problem of parameter estimation in model maps from noisy time series is addressed. We suggest a new technique for a special case of one-dimensional maps and chaotic signals. It is based on the maximum likelihood (ML) principle and evaluation of the cost function via backward iterations of a model map. We demonstrate in numerical experiments and, in part, justify theoretically that this “backward ML technique” gives more accurate estimates than previously known techniques for low and moderate noise levels. In particular, global optimisation of the cost function becomes much easier; biases in the estimates vanish as the time series length N increases; variances of the estimates decrease as fast as N−α where α depends on the original system, typical values being about α=2.0 under mild conditions on the original systems.
PACS number(s): 02.70.Rr, 05.45.Tp, 05.45.Pq Artificial neural networks (ANN) are typically composed of a large number of nonlinear functions (neurons) each with several linear and nonlinear parameters that are fitted to data through a computationally intensive training process. Longer training results in a closer fit to the data, but excessive training will lead to overfitting. We propose an alternative scheme that has previously been described for radial basis functions (RBF). We show that fundamental differences between ANN and RBF make application of this scheme to ANN nontrivial. Under this scheme, the training process is replaced by an optimal fitting routine, and overfitting is avoided by controlling the number of neurons in the network. We show that for time series modeling and prediction, this procedure leads to small models (few neurons) that mimic the underlying dynamics of the system well and do not overfit the data. We apply this algorithm to several computational and real systems including chaotic differential equations, the annual sunspot count, and experimental data obtained from a chaotic laser. Our experiments indicate that the structural differences between ANN and RBF make ANN particularly well suited to modeling chaotic time series data. Author name used in this publication: C. K. Tse
We perform a global reconstruction of differential and difference equations, which model an object in a wide domain of a phase space, from a time series. The efficiency of using time realizations of transient processes for this purpose is demonstrated. Time series of transients are shown to have some advantages for the realization of a procedure of model structure optimization.
The success of modeling from an experimental time series is determined to a significant extent by the choice of dynamical variables. We propose a method for preliminary investigation of a time series whose purpose is to find out whether a global dynamical model with smooth functions can be constructed for the chosen variables. The method consists in the estimation of single valuedness and continuity of relations between dynamical variables and variables to enter left-hand sides of model equations. The method is explained with numerical examples. Its efficiency is demonstrated by modeling a real nonlinear electric circuit.
The setting of a continuing source of data such that retaining and processing of all observations become impractical was studied. In local linear predictions the refined learning set was illustrated, using the Ikeda map. The reduction of noise provided efficient advantage to network approaches. The real time construction was complementary to the methods which incresed the complexity in poor forecast regions. Analysis shows that the ease with which the local models updated themselves by merely changing the learning set was another advantage of local models.
. Adaptive observation strategies in numerical weather prediction aim to improve forecasts by exploiting additional observations, at locations which are themselves determined by the current state of the atmosphere. The role played by an inexact estimate of the current state of the atmosphere (i.e. error in the "analysis") in restricting adaptive observation strategies is investigated; necessary conditions valid across a broad class of modeling strategies are identified for strategies based on linearised model dynamics to be productive. It is demonstrated that the assimilation scheme, or more precisely, the magnitude of the analysis error, is crucial in limiting the scope of application of dynamically based strategies. In short, strategies based on linearised dynamics require that analysis error is sufficiently small that the model linearisation about the analysis is relevant to linearised dynamics of the full system about the true system state. Inasmuch as the analysis error depends o...
An online multi-step prediction method is constructed based on data streams, for which using all observed data points for prediction is impractical. The proposed method is superior in prediction accuracy and computational time, at various prediction steps, compared to two simple extensions of Kwasniok and Smith [F. Kwasniok, L.A. Smith, Phys. Rev. Let. 92 (2004) 164101]. We apply our proposed prediction method to artificial data sets generated from the Lorenz-63 model and to measured wind speed data streams.
We introduce the idea of local Lyapunov exponents which govern the way small perturbations to the orbit of a dynamical system grow or contract after afinite number of steps,L, along the orbit. The distributions of these exponents over the attractor is an invariant of the dynamical system; namely, they are independent of the orbit or initial conditions. They tell us the variation of predictability over the attractor. They allow the estimation of extreme excursions of perturbations to an orbit once we know the mean and moments about the mean of these distributions. We show that the variations about the mean of the Lyapunov exponents approach zero asL and argue from our numerical work on several chaotic systems that this approach is asL
–v. In our examplesv 0.5–1.0. The exponents themselves approach the familiar Lyapunov spectrum in this same fashion.
We develop methods for determining local Lyapunov exponents from observations of a scalar data set. Using average mutual information and the method of false neighbors, we reconstruct a multivariate time series, and then use local polynomial neighborhood-to-neighborhood maps to determine the phase space partial derivatives required to compute Lyapunov exponents. In several examples we demonstrate that the methods allow one to accurately reproduce results determined when the dynamics is known beforehand. We present a new recursive QR decomposition method for finding the eigenvalues of products of matrices when that product is severely ill conditioned, and we give an argument to show that local Lyapunov exponents are ambiguous up to order 1/L in the number of steps due to the choice of coordinate system. Local Lyapunov exponents are the critical element in determining the practical predictability of a chaotic system, so the results here will be of some general use.
Preface; Acknowledgements; Part I. Basic Topics: 1. Introduction: why
nonlinear methods?; 2. Linear tools and general considerations; 3. Phase
space methods; 4. Determinism and predictability; 5. Instability:
Lyapunov exponents; 6. Self-similarity: dimensions; 7. Using nonlinear
methods when determinism is weak; 8. Selected nonlinear phenomena; Part
II. Advanced Topics: 9. Advanced embedding methods; 10. Chaotic data and
noise; 11. More about invariant quantities; 12. Modelling and
forecasting; 13. Non-stationary signals; 14. Coupling and
synchronisation of nonlinear systems; 15. Chaos control; Appendix A:
using the TISEAN programs; Appendix B: description of the experimental
data sets; References; Index.
This paper describes a simple method of obtaining longer-term predictions from a nonlinear time-series, assuming one already has a reasonably good short-term predictor. The usefulness of the technique is that it eliminates, to some extent, the systematic errors of the iterated short-term predictor. The technique we describe also provides an indication of the prediction horizon. We consider systems with both observational and dynamic noise and analyse a number of artificial and experimental systems obtaining consistent results. We also compare this method of longer-term prediction with ensemble prediction.
A very simple method to compute the maximal Lyapunov exponent of a time series is introduced. The algorithm makes use of the statistical properties of the local divergence rates of nearby trajectories. It does not depend explicitly on the knowledge of the correct embedding dimension or on other parameters.