Predictability improvement as a tool to detect causality

Abstract

Abstract. A causality method based on predictions in reconstructed state spaces is presented and tested in relation to the ability to detect coupling and synchronization of interconnected dynamical systems. The method is demonstrated on a test example of two uni-directionally coupled chaotic systems of Rössler type. The results show that the predictability of the driven system can be improved by incorporating information about past values of the driving system
to the prediction process. This predictability improvement reveals the presence and the direction of the coupling and may help to detect the onset of full synchronization and distinguish causality from mere correlation.

Full text

MEASUREMENT 2017, Proceedings of the 11th International Conference, Smolenice, Slovakia
39
Predictability Improvement As a Tool to Detect Causality
A. Krakovská
Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, Slovakia
Email: krakovska@savba.sk
Abstract. A causality method based on predictions in reconstructed state spaces is presented
and tested in relation to the ability to detect coupling and synchronization of interconnected
dynamical systems. The method is demonstrated on a test example of two uni-directionally
coupled chaotic systems of Rössler type. The results show that the predictability of the driven
system can be improved by incorporating information about past values of the driving system
to the prediction process. This predictability improvement reveals the presence and the
direction of the coupling and may help to detect the onset of full synchronization and distinguish
causality from mere correlation.
Keywords: Causality, Reconstructed State Space, Predictability Improvement, Rössler System
1. Introduction
The study of drive-response relationships between dynamical systems is a topic of active
interest. In this paper, the next type of uni-directional coupling in a bivariate dynamical system
is studied: ܩ(t) = F(X(t)),
ܭ(t) = G(Y(t), X(t)),
where X and Y are the state vectors of the driving and the driven systems. If Y(t) =
<
(X(t)) for
some smooth and invertible function
<
 then there is said to be a generalized synchronization
between X and Y. If
<
is an identity, the synchronization is called identical. The direction of
coupling can only be uncovered before the emergence of synchronization. Once the systems
are synchronized, there is a one-to-one relation between their states and the future values of the
driver X can be predicted from the response Y equally well as vice versa.
A first mathematical approach to detect causality has been proposed in 1969 by Clive Granger
[1]. In Granger sense, a time series X is said to cause Y if it can be shown, through statistical
hypothesis tests, that past X values provide significant information about future values of Y. To
be able to consider Granger causality, separability is required. Namely, information about the
causative factor X is expected to be available as an explicit variable. This requirement can be
problematic in a case of dynamically linked variables. Moreover, the initially linear concept
also requires generalizations to enable investigation of complex nonlinear processes. Therefore,
new approaches have been proposed, including extended Granger causality, conditional mutual
information (transfer entropy), cross predictability, measures evaluating distances of
conditioned neighbours in reconstructed state spaces, etc. (see [2] for short review). The
method, used in this paper, reminds the original Granger approach in that it also addresses the
usefulness of past values of the first system for characterizing the second one. However, our
approach belongs to the state-space methods. It determines whether a prediction made in a
reconstructed state space of a time series improves when data from another system are included
in the state space reconstruction. The potential of this idea has been identified in [3], [4].
The paper is organized as follows. In Section 2, the example of uni-directionally coupled
Rössler systems, and the method of causality detection are presented. Then the results of the
causal analysis are given and discussed in Section 3.

MEASUREMENT 2017, Proceedings of the 11th International Conference, Smolenice, Slovakia
40
2. Subject and Methods
Data
As a testing example, we use the next uni-directionally coupled non-identical Rössler systems
at different coupling strengths C:
ܪ1 íȦ1x2 í[3
ܪ2 Ȧ1x1 + 0.15x2
ܪ3 = 0.2 + x3(x1 í (1)
ܮ1 íȦ2y2 í\3 + C(x1 í\1)
ܮ2 Ȧ2y1 + 0.72y2
ܮ3 = 0.2 + y3(y1 í
where Ȧ1 = 0.5 and Ȧ2 = 2.515.
The two Rössler systems represent distinct dynamical subsystems coupled through the one-way
driving relationship between variables x1 and y1. Mutual interconnections of the variables may
be represented by the next interaction graph:
20000 data points of x1 and y1 were generated by Matlab solver of ordinary differential
equations ode45 at an integration step 0.3. The starting point was [0, 0, 0.4, 0, 0, 0.4]. First 1000
data points were thrown away. The coupling strength C was chosen from 0 to 1.1 with a step
size of 0.1. The synchronization takes place at the coupling of about 1 [6].
The frequency ratio of about 1:5 used in this example reminds cardio-respiratory interactions,
where the influence of the (slower) respiratory dynamics on the (faster) cardiac dynamics is
larger than in the opposite direction. The system was first used in [5] to show that in this case,
the problem of detecting directionality is much more challenging than in cases of two systems
with more closely related oscillations.
a) b) c)
Fig. 1. a) piece of trajectory corresponding to 100 points of the slow system X (x1, x2, x3 of Eq. 1, C=0)
b) piece of trajectory corresponding to 100 points of the fast system Y (y1, y2, y3 of Eq. 1, C=0)
c) X apparently causes a slowdown of Y: piece of trajectory corresponding to 100 points of “enslaved”
Y (y1, y2, y3 of Eq. 1, C>1)
The Method - Predictability Improvement Using Mixed Prediction
Let the systems X and Y be represented by a time series x and y, respectively, and the task be to
detect causal link from X to Y. Similarly, as in the case of Granger meaning, we say that variable
x causes variable y, if the better prediction of y can be obtained using information from both x
and y rather than using only information from y. However, unlike the Granger's method, ours
is operating in reconstructed multidimensional state spaces. Therefore, as a first step, the dY-

MEASUREMENT 2017, Proceedings of the 11th International Conference, Smolenice, Slovakia
41
dimensional manifold MY is reconstructed from lags of observable y so that the state of the
system in time t is [y(t), y(t – IJY\WíIJY), . . . , y(t – (dY-IJY)]. From Taken’s theorem [7],
we know that given certain conditions, the reconstructed manifold is diffeomorphic to the
original one. The reconstruction is determined by two parameters, namely the size d of the
reconstructed space and the delay IJ. The optimal choice of d and IJ requires a task-specific
approach [8]. If the task is related to predicting, as in our case, then it is appropriate to evaluate
the prediction error for various combinations of possible embedding parameters. Consequently,
the lowest errors lead us to proper choices of d and IJ.
With regard to the actual way of prediction, the method of analogues was used [9]. This is based
on the finding historical data similar to the current situation and assuming the system will
continue the same way as it did in the past. There are many ways to estimate the follower of the
point y(t), the simplest version being finding the time index i of its nearest neighbour from past
states on the reconstructed trajectory and declaring ǔW= y(i+1). A modification, which was
used in this study, improves the simplest version by averaging followers of several neighbours
while considering exponential weighting based on distances from y(t) for the neighbours.
The predictions in the reconstructed spaces MX and MY were compared to predictions computed
in MX+Y, i.e., in a mixed state space built from delayed observables of both systems. If, for
example, the predictability of Y in MX+Y was better than its predictability in MY, we concluded
that X causally affects Y. A method built on this idea, with special attention attributed to the
optimization of the reconstructed space based on the weighting of each of its coordinates, has
been proposed in [2]. Because of its computational demands, we decided to work with a
simplified variant of the former method in this study.
The specific steps of the causality method used here are as follows:
1. The manifold MY is performed. This provides space to get predictions of y without
using additional information from X. The one point predictions ǔ of large enough
statistical sample of N points over the reconstructed trajectory are computed. The
resulting errors are given by eY(t) = y(t) – ǔW.
2. To get predictions of y using information both from X and Y, the manifold MX+Y
(mixed state space) is reconstructed. The reconstruction contains some of coordinates
from MY and some from MX. If we used the full number of coordinates, the state
corresponding to time t would be [y(t), y(t – IJY\WíIJY), . . . , y(t – (dY-IJY), wx(t),
wx(t – IJXZ[W íIJX), . . . , wx(t – (dX-IJX)], where the weight w represents the
impact of the system X. Analogously as in step 1, one-point predictions of y are
computed. Let us denote the prediction in time t as ǔX+Y(t) and the corresponding error
as eX+Y(t) = y(t) – ǔX+Y(t).
3. To decide whether the addition of information from system X improves the prediction
of Y (X drives Y), the null hypothesis H0: Var(eX+Y9DUHY) is tested. If H0 is rejected
on a 5% significance level, then we accept that Var(eX+Y) < Var(eY), or equivalently,
that inclusion of knowledge of X significantly improves the prediction of Y .
To get instructions to test causality in the opposite direction, i.e., from Y to X, just replace X by
Y and vice versa in the above three steps.
3. Results and Discussion
Knowing 20000 data-points of variable x1 of the driving Rössler system X and variable y1 of
the responsive Rössler system Y (Eq. 1) we were looking for the causal relationship between
the two systems. First of all, we made reconstructions of the state portraits. The embedding
parameters and the impact w of the system X were individually selected for each coupling
strength. For this, we calculated the prediction errors for several combinations of parameters
and selected the ones that led to the lowest error. As a result, the space MX was reconstructed

MEASUREMENT 2017, Proceedings of the 11th International Conference, Smolenice, Slovakia
42
with delay IJX = 1 and embedding dimension dX = 3, while for MY delay IJY = 1 or 2 and
embedding dimensions dY = 4, 5, 6, or 7 were optimal (the stronger the coupling, the higher the
so called embedding window IJYdY [8]). Then the corresponding mixed state spaces MX+Y had
dimensions between 7 and 10. The values of the weights w ranged from 0 to 0.35. In total, 1000
one point predictions were computed, with help of 19000 historical points. Figure 1 shows the
prediction errors for increasing couplings. Obviously, for zero coupling, information from X
does not help in the prediction of Y. It is similar for the couplings above a synchronization level.
The predictability improvement was not significant for the two weakest couplings, but
otherwise the observable from X helps to predict Y (p<0.001). Testing the opposite direction
showed no predictability improvement and rejected the causal link of YĺX.
Fig. 2. Average errors of predictions of y1 (Eq.1)
made in MY (black) and in MX+Y (gray) for
increasing coupling. The results show that
the observable from X helps to predict
observable from Y, referring to causality
;ĺY.
Finally, we conclude that monitoring the predictability improvement in reconstructed mixed
state spaces resulted in a successful detection of the presence and the direction of causal links
even for the complicated case of connection of chaotic systems with slow and fast dynamics.
Acknowledgements
Supported by the Slovak Grant Agency for Science (Grant 2/0011/16) and by the Slovak Research and
Development Agency (Grant APVV–15–0295).
References
[1] Granger CWJ. Investigating Causal Relations by Econometric Models and Cross-spectral
Methods. Econometrica, 37 (3): 424-438, 1969.
[2] Krakovská A, Hanzely F. Testing for causality in reconstructed state spaces by an optimized
mixed prediction method. Physical Review E, 94 (5): 052203, 2016.
[3] Wiesenfeldt M, Parlitz U, Lauterborn W. Mixed state analysis of multivariate time series.
International Journal of Bifurcation and Chaos, 11 (08): 2217-2226, 2001.
[4] Feldmann U, Bhattacharya J. Predictability improvement as an asymmetrical measure of
interdependence in bivariate time series. International Journal of Bifurcation and Chaos, 14 (02):
505-514, 2004.
[5] Paluš M, Vejmelka M. Directionality of coupling from bivariate time series: How to avoid false
causalities and missed connections. Physical Review E, 75: 056211, 2007.
[6] Krakovská A, et al. Causality studied in reconstructed state space. Examples of uni-directionally
connected chaotic systems. arXiv preprint arXiv:1511.00505, 2015.
[7] Takens F. Detecting strange attractors in turbulence. In Dynamical systems and turbulence.
Warwick, Springer Berlin Heidelberg, 366-381, 1981.
[8] .UDNRYVNi$0H]HLRYi.%XGiþRYi+8VHRIIDOVHQHDUHVWQHLJKERXUVIRUVHOHFWLQJYDULDEOHV
and embedding parameters for state space reconstruction. Journal of Complex Systems, ID
932750, 12 pages, 2015.
[9] Lorenz EN. Atmospheric predictability as revealed by naturally occurring analogues. Journal of
the Atmospheric Sciences, 26 (4): 636-646, 1969.