Some Peculiarities of Causal Analysis of Coupled Chaotic Systems

Abstract

On a test example of uni-directionally coupled Rössler systems we demonstrate some of the pitfalls of causal analysis of chaotic data. The method based on evaluating predictability in reconstructed state spaces is used here to detect causality. The results show that the predictability of the driven Rössler system is improved by incorporating information about the present state of the driver to the prediction process. The predictability improvement correctly reveals the presence and the direction of the coupling. However, causal analysis of the time-reversed test signals does not allow to uncover that the cause precedes the effect. In addition, causal analysis of complex systems may also encounter other complications such as transient chaos, or irreversibility of dissipative chaos sometimes masked by the dominance of limit cycles. 1. Introduction Detecting causal relations between systems, based on observations in the form of time series, often referred to the time series predictions. Namely, if the information on the recent past of the time series x helps improve the prediction of y, then we say that the x causes y. The concept was introduced by Clive Granger in 1969 [1] but the same idea is essential for more recent methods such as conditional mutual information or transfer entropy [2], [3]. However, to consider Granger causality, separability is required. It means that information about the driver is expected to be available as an explicit variable. Separability does not apply to cases of dynamically linked variables with bidirectional links and cyclic information flow. In such systems, the causes and effects are entangled. This is reflected in Takens' theorem, which claims that under some conditions the underlying dynamics is reconstructable from any measured observable [4]. As we have demonstrated in [5], the Granger causality test produces false positives for time series coming from coupled dynamical systems. Some other techniques, including the method of predictability improvement (PI) used in this paper, may be more successful [6]. The PI determines whether a prediction of an observable from Y , made in a reconstructed state space, improves when observable from system X is included in the reconstruction. If the predictability improves, then, analogously to the idea of Granger's causality of autoregressive processes, we hypothesize that X causes Y. However, Cummins et al. have introduced a comprehensive theory showing that using delay reconstruction techniques on deterministic systems, we have very limited options. The best we can hope for is finding the strongly connected components of the graph (sets of mutually reachable vertices) which represent distinct subsystems coupled through one-way driving links [7]. We cannot recover self-loops, and, we cannot distinguish the direct driving, indirect driving, correlate of a direct or indirect driving. In the next section, the method of causality detection and the example of uni-directionally coupled Rössler systems are presented. Then the results of the causal analysis are given and the complications arising from the presence of chaos are discussed.

Full text

Some Peculiarities of Causal Analysis of Coupled Chaotic Systems
Anna Krakovská
Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, Slovakia
Email: krakovska@savba.sk
Abstract. On a test example of uni-directionally coupled Rössler systems we demonstrate some
of the pitfalls of causal analysis of chaotic data. The method based on evaluating predictability
in reconstructed state spaces is used here to detect causality. The results show that the pre-
dictability of the driven Rössler system is improved by incorporating information about the
present state of the driver to the prediction process. The predictability improvement correctly
reveals the presence and the direction of the coupling. However, causal analysis of the time-
reversed test signals does not allow to uncover that the cause precedes the effect. In addition,
causal analysis of complex systems may also encounter other complications such as transient
chaos, or irreversibility of dissipative chaos sometimes masked by the dominance of limit cycles.
Keywords: Causality, Rössler System, Chaos, Irreversibility, Arrow of Time
1. Introduction
Detecting causal relations between systems, based on observations in the form of time series,
often referred to the time series predictions. Namely, if the information on the recent past of the
time series xhelps improve the prediction of y, then we say that the xcauses y. The concept was
introduced by Clive Granger in 1969 [1] but the same idea is essential for more recent methods
such as conditional mutual information or transfer entropy [2], [3].
However, to consider Granger causality, separability is required. It means that information
about the driver is expected to be available as an explicit variable. Separability does not apply
to cases of dynamically linked variables with bidirectional links and cyclic information flow. In
such systems, the causes and effects are entangled. This is reflected in Takens’ theorem, which
claims that under some conditions the underlying dynamics is reconstructable from any mea-
sured observable [4]. As we have demonstrated in [5], the Granger causality test produces false
positives for time series coming from coupled dynamical systems. Some other techniques, in-
cluding the method of predictability improvement (PI) used in this paper, may be more success-
ful [6]. The PI determines whether a prediction of an observable from Y, made in a reconstructed
state space, improves when observable from system Xis included in the reconstruction. If the
predictability improves, then, analogously to the idea of Granger’s causality of autoregressive
processes, we hypothesize that Xcauses Y. However, Cummins et al. have introduced a com-
prehensive theory showing that using delay reconstruction techniques on deterministic systems,
we have very limited options. The best we can hope for is finding the strongly connected com-
ponents of the graph (sets of mutually reachable vertices) which represent distinct subsystems
coupled through one-way driving links [7]. We cannot recover self-loops, and, we cannot dis-
tinguish the direct driving, indirect driving, correlate of a direct or indirect driving.
In the next section, the method of causality detection and the example of uni-directionally
coupled Rössler systems are presented. Then the results of the causal analysis are given and the
complications arising from the presence of chaos are discussed.
2. Subject and Methods
In this study, to detect the causality from Xto Ywe used a simplified version of The Method of
Predictability Improvement introduced in [6]. The method proceeds as follows:
MEASUREMENT 2019, Proceedings of the 12th International Conference, Smolenice, Slovakia
ISBN 978-80-972629-3-8
102

1. Based on suitable reconstruction parameters dYand τYthe manifold MYis performed
so that the state corresponding to time twould be [y(t),y(t−τY),...,y(t−(dY−1)τY)]. This
provides space to get prediction ˆ
Yof Ywithout using information from X. The one-point pre-
dictions of a large-enough statistical sample of points over the reconstructed trajectory are com-
puted, using the method of analogues [8]. The resulting errors are eY(t)=Y(t)−ˆ
Y(t).
2. To get the predictions of Yusing information from both Xand Y, we reconstruct the
manifold MX+Ycontaining some of the coordinates from MYand some from MX. If we used all
the coordinates, the state in time twould be [y(t),y(t−τY),...,y(t−(dY−1)τY),w.x(t),w.x(t−
τX),...,w.x(t−(dX−1)τX)], where the weight wrepresents the impact of X. Analogously as
in step 1, the predictions and the corresponding errors eX+Y(t)=Y(t)−ˆ
YX+Y(t)are computed.
3. To decide whether the addition of information from Ximproves significantly the predic-
tion of Y, i.e. eX+Y<eY, we use the Welch test.
After exchanging the roles of Xand Yabove we get instructions to test causality in the
opposite direction, i.e., from Yto X.
Rössler 1.015 →Rössler 0.985
Our test data comes from the next uni-directionally connected chaotic Rössler oscillators:
˙x1=−ω1x2−x3
˙x2=ω1x1+0.15x2
˙x3=0.2+x3(x1−10)(1)
˙y1=−ω2y2−y3+C(x1−y1)
˙y2=ω2y1+0.15y2
˙y3=0.2+y3(y1−10)
where ω1=1.015,ω2=0.985.
The interconnections of the variables are represented by the interaction graph on Fig. 1.
The variables form a set of nodes connected by directed edges wherever one variable appears
nontrivially on the right-hand side of the equation for the derivative of the second variable. The
two Rössler systems can be seen as distinct dynamical subsystems Xand Ycoupled through
one-way driving of variable y1by x1.
Fig. 1: Interaction graph for the coupling of two Rössler systems (Eq. (1)).
The coupling strength is controlled by the parameter C, with C=0 for uncoupled systems.
Plots of conditional Lyapunov exponents in [2], similarly as correlation dimension estimates in
[9], show that, synchronization takes place between C=0.11 and C=0.13. The direction of
the coupling can only be uncovered before the emergence of synchronization. In the following,
coupling C=0.07 will be used, which is well below the synchronization value. Starting from the
point [0,0,0.4,0,0,0.4], the solutions were generated by Matlab solver ode45 at an integration
step size of 0.1. This gives about 60 samples per one average orbit around the attractor.
MEASUREMENT 2019, Proceedings of the 12th International Conference, Smolenice, Slovakia
ISBN 978-80-972629-3-8
103

3. Results and Discussion
We performed the causal analysis between Xand Ywith the assumption that we only know time
series x1and y1.
We started with the reconstruction of the trajectories on manifolds MXand MY. Then, 240
one-point predictions of y1were computed, with help of 1200 historical points. Fig. 2 shows
variances of prediction errors for increasing sampling values.
020 40 60
0
1
2
3
4
5
X → Y
sampling
020 40 60
0
1
2
3
4
5
Y → X
sampling
variance of errors
Fig. 2: Causality analysis of the coupled Rössler systems. On the left: Variances of errors of prediction
(blue) and retrodiction (red) of time series x1. Involvement of y1did not significantly lower the errors
(dotted lines). On the right: Variances of errors of prediction (blue) and retrodiction (red) of time series
y1. Involvement of x1significantly reduced the errors (dotted lines).
For a wide range of sampling, information from x1helped significantly with the one-point
prediction of y1(p<0.01). Testing the opposite direction (see the red lines on Fig. 2) showed
no predictability improvement and correctly rejected the causal link of Y→X.
Also, the fractal dimension based method to investigate causal links, introduced in [11],
confirmed that only the relation X→Yis possible here. Correlation dimension of the state por-
trait reconstructed from y1was markedly higher than that of the portrait from x1[9]. Similarly,
models needed for prediction of y1had a higher degree of freedom than the models to predict
x1.
Although the PI method seems to have done its job, causal analysis of chaotic systems can
bring some unexpected pitfalls. Let us begin with the notion of reversibility, meaning that the
process has the same statistical properties as its time reversal. White noise, or linearly correlated
Gaussian processes and conservative (volume preserving) chaotic systems are reversible, while
dissipative chaotic dynamics belongs to irreversible processes. Looking at it from the perspec-
tive of predictions, the predictability of a reversible signal is the same as the predictability of its
time-reversed version, while in case of irreversible signals, differences between the prediction
and the retrodiction suggest that the arrow of time is playing a role here. One should be able to
make predictions of dissipative chaotic trajectories easier if moving forward, if only because of
the divergence of initially nearby trajectories of chaotic systems trajectories and the tendency
to converge from a bigger space to a smaller attracting set (lower uncertainty).
However, although this was not the case with the system being tested, let us mention that,
despite an underlying irreversible evolution, some systems exhibit the properties of time re-
versible (time symmetric) dynamics, because their long term behavior becomes dominated by
limit cycles. And, on the contrary, in some other systems, the attractor itself is simple (a fixed
point or a limit cycle) but its coexistence with an unstable chaotic repeller or chaotic saddle
can cause that the trajectory first shows transient chaos, and only after a relatively long time it
settles down on the simple, stable attractor.
Regarding our test example, Fig. 2 clearly demonstrate the irreversibility related to the ex-
ponential divergence of trajectories. Retrodiction errors are significantly higher than prediction
MEASUREMENT 2019, Proceedings of the 12th International Conference, Smolenice, Slovakia
ISBN 978-80-972629-3-8
104

errors. Now, imagine a simple turn of x1and y1over time and repeating the causal analysis.
Considering the first principle of Granger causality that the cause precedes the effect one ex-
pects change of the direction of causality from X→Yto Y→X. Indeed, this happens when
causally linked AR models and their time-reversals are analyzed by the Granger test. It might
even look like a good idea to use this "turning test" routinely to confirm the conclusion about
the direction of the causal link. However, Paluš et al. pointed out that the expected change of
the direction of causality did not happen after time reversing of the tested chaotic signals [10].
The authors suggested that the observed paradox is probably related to the dynamic memory of
the systems. If this paradox appears in the numerical analysis of real data, and Paluš et al. gives
such an example in [10], it might indicate dealing with interactions of dynamical systems.
Many scientists in the history including Henri Poincare and in particular the Nobel Prize
winner Ilya Prigogine were thinking about how to make irreversible processes and the arrow of
time a fundamental part of physics. It is becoming increasingly clear that the problem goes to
the core of complex dynamical systems and with the development of this area, problems around
time-reversed processes are becoming more relevant than ever.
Acknowledgements
Supported by the Slovak Grant Agency for Science (Grant 2/0081/19) and by the Slovak Re-
search and Development Agency (Grant APVV-15-0295).
References
[1] Granger, C.W. (1969). Investigating causal relations by econometric models and cross-spectral
methods. Econometrica: Journal of the Econometric Society, 424–438.
[2] Paluš, M., Vejmelka, M. (2007). Directionality of coupling from bivariate time series: How to avoid
false causalities and missed connections. Physical Review E, 75(5), 056211.
[3] Schreiber, T. (2000). Measuring information transfer. Physical Review Letters, 85(2), 461–464.
[4] Takens, F. (2002). Detecting strange attractors in turbulence. In Dynamical Systems and Turbulence.
Rand, D.A. and Young, L.S., Springer-Verlag, Berlin, 366–381.
[5] Krakovská, A., Jakubík, J., Chvosteková, M., Coufal, D., Jajcay, N., Paluš, M. (2018). Comparison
of six methods for the detection of causality in a bivariate time series. Physical Review E, 97(4),
042207.
[6] Krakovská, A., Hanzely, F. (2016). Testing for causality in reconstructed state spaces by an opti-
mized mixed prediction method. Physical Review E, 94(5), 052203.
[7] Cummins, B., Gedeon, T., Spendlove, K. (2015). On the efficacy of state space reconstruction
methods in determining causality. SIAM Journal on Applied Dynamical Systems, 14(1), 335–381.
[8] Lorenz, E.N. (1969). Atmospheric predictability as revealed by naturally occurring analogues. Jour-
nal of the Atmospheric sciences, 26(4), 636–646.
[9] Krakovská, A., Jakubík, J., Budáˇ
cová, H., Holecyová, M. (2015). Causality studied in recon-
structed state space. Examples of uni-directionally connected chaotic systems. arXiv preprint,
arXiv:1511.00505.
[10] Paluš, M., Krakovská, A., Jakubík, J., Chvosteková, M. (2018). Causality, dynamical systems and
the arrow of time. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28(7), 075307.
[11] Krakovská, A., Budáˇ
cová, H. (2013). Interdependence measure based on correlation dimension. In
Proceedings of the 9th International Conference on Measurement. 31–34.
MEASUREMENT 2019, Proceedings of the 12th International Conference, Smolenice, Slovakia
ISBN 978-80-972629-3-8
105