Content uploaded by Andreas Hense
Author content
All content in this area was uploaded by Andreas Hense on May 22, 2018
Content may be subject to copyright.
Content uploaded by Andreas Hense
Author content
All content in this area was uploaded by Andreas Hense on May 22, 2018
Content may be subject to copyright.
A Mat´ern based multivariate Gaussian random process for a
consistent model of the horizontal wind components and related
variables
R¨udiger Hewer∗, Petra Friederichs, Andreas Hense
Meteorological Institute, University Bonn, Auf dem H¨ugel 20, Bonn, Germany1
Martin Schlather2
School of Business Informatics and Mathematics, University Mannheim, B3, Mannheim,
Germany
3
4
∗Corresponding author address: Meteorological Institute, University Bonn, Auf dem H¨ugel
20, Bonn, Germany
5
6
E-mail: rhewer@uni-bonn.de7
Generated using v4.3.2 of the AMS L
A
T
E
X template 1
arXiv:1707.01287v1 [stat.AP] 5 Jul 2017
1. Introduction
An appropriate representation of the covariance structure in spatial models of meteo-
rological variables is essential when analyzing (Gandin 1963; Kalnay 2003) meteorological
data using data assimilation (Hollingsworth and L¨onnberg 1986; Evensen 1994; Bonavita
et al. 2012; Pu et al. 2016). This generally requires an appropriate representation of the
background error covariance matrix. Further, spatial stochastic models for meteorological
variables should respect physical relationships.
One of the first approaches to include physical consistency via differential relations be-
tween variables can be found in Kolmogorov (1941). Thi´ebaux (1977) introduced a covari-
ance model for wind fields assuming geostrophic balance, thereby incorporating anisotropy
in the geopotential height. Daley (1985) derived a covariance model for the horizontal
wind components assuming a Gaussian covariance model for the velocity potential and the
streamfunction, where he derived the differential relations between the potentials and the
wind field. The covariance model proposed by Daley (1985) is rather flexible as it allows
for geostrophic coupling, non-zero correlation of streamfunction and velocity potential, and
differing scales for the two potentials. Daley (1985) also considered geopotential height
as an additional model variable. However, the resulting covariance function for the wind
fields is not positive definite for many parameter combinations. Hollingsworth and L¨onnberg
(1986) adapted Daley’s method and formulated a covariance function for the potentials us-
ing cylindrical harmonics. They show that on the synoptic scale the correlation between
the potentials is small, such that Daley (1991) reformulated his model for zero correlations.
These approaches (Thi´ebaux 1977; Hollingsworth and L¨onnberg 1986; Daley 1985) as well as
our model differ from current data assimilation methods, as they provide an explicit, para-
metric and analytic covariance model for the background error. So-called control variable
transform methods (Bannister 2008) describe the background error matrix in an implicit
non-parametric way via its square root 1using latent variables which model the physical
variables. Sample based methods like the ensemble Kalman filter (Evensen 1994) describe
the error statistics based on estimates obtained from an ensemble.
The data assimilation literature (e.g. Thi´ebaux 1977; Hollingsworth and L¨onnberg 1986;
Daley 1985) typically uses the stochastic models in order to describe the covariance matrix
of the background error, which is the difference of the a forecast and the true field. Similar
methods have also been used in order to describe the full turbulent field (Frehlich et al.
2001). There has also been considerable interest in describing the statistics of the velocity
field directly or via its spectrum (B¨uhler et al. 2014; Lindborg 2015; Bierdel et al. 2016).
1e.g. Cholesky decomposition
2
While Thi´ebaux (1977), Hollingsworth and L¨onnberg (1986), and Daley (1985) include
physical relations via differentiation of the covariance function, finite difference operators
are used in Bayesian hierarchical models. For example, Royle et al. (1999) modeled the
geostrophic relation of pressure and wind field.
In this paper, we propose a multivariate Gaussian random field (GRF) formulation for
six atmospheric variables in a horizontal two-dimensional Cartesian space. Assuming a bi-
variate Mat´ern covariance for streamfunction ψand velocity potential χ, we derive the
covariance structure of the horizontal wind components ~
U= (u,v)Tas well as vortic-
ity ∇ × ~
U:=−∂
∂e2u+∂
∂e1vand divergence ∇ · ~
U. All of these quantities are connected
via the Helmholtz decomposition, which states that for any given wind field ~
Uthere
exists a streamfunction ψand velocity potential χ, such that ~
U=∇ × ψ+∇χ, where
∇× ψ:=−∂
∂e2ψ, ∂
∂e1ψT. In dimension two and with appropriate boundary conditions this
decomposition is unique. Curl and divergence of the wind field are given as ∇ × ~
U= ∆ψ
and ∇ · ~
U= ∆χ, respectively, where ∆ is the 2-dimensional Laplace operator.
Our multivariate GRF formulation is novel for several reasons. While e.g. Daley (1985)
only used the potentials to derive the covariance function of the wind fields, our model
is formulated for all related variables, including a formulation for the potential functions
and the wind field, as well as vorticity and divergence. Secondly, our model provides a
formulation for anisotropy in the wind field and the related potentials. Further, we allow for
non-zero correlations between the rotational and divergent wind component, which might be
particularly relevant for atmospheric fields on sub-geostrophic scales. We show that the scale
parameters considered by Daley (1985) are inconsistent with non-zero correlations between
streamfunction and velocity potential, as they do not lead to a positive definite model. An
exact derivation of the condition under which the covariance function of Daley’s model is
positive definite is given in the appendix. Further our model is a counter example to a
theorem of Obukhov (1954), which claims that there is no isotropic wind field with non-zero
correlation of the rotational and non-rotational component of the wind field. More details
to Obukhovs claim are given in the appendix.
The covariance function of our multivariate GRF will be incorporated into an upcoming
version of the spatial statistics R package RandomFields (Schlather et al. 2016). This opens
the possibility for a wealth of applications in spatial statistics, including the conditional
simulation of streamfunction and vector potential given an observed wind field, a consis-
tent formulation of the covariance structure for both the potential and the horizontal wind
components to be used in data assimilation, or stochastic interpolation (kriging) of each of
3
the involved variables given the others. Kriging is the process of computing the conditional
expectation of a certain variable given others. It is typically used to interpolate fields.
To exemplify the multivariate GRF we estimated its parameters for atmospheric fields
of the numerical ensemble weather prediction system, COSMO-DE-EPS (Gebhardt et al.
2011), provided by the German Meteorological Service (DWD). COSMO-DE is a high-
resolution forecast system, that provides forecasts on the atmospheric mesoscale (Baldauf
et al. 2011). Estimation is realized using the maximum likelihood method, while uncertainty
in the parameter estimation is assessed by parametric bootstrap (Efron, B., & Tibshirani
1994). We also discuss the meteorological relevance of the parameters.
The remainder of the paper is organized as follows. In Section 2 we introduce the multi-
variate GRF, and demonstrate how the physical relations and anisotropy are included in the
model formulation. Section 3 introduces the COSMO-DE-EPS data. Section 4 is devoted to
the parameter estimation and the assessment of the uncertainties, while Section 5 presents
and interprets the results of the estimation. We conclude in Section 6 and discuss potential
applications, limitsand extensions of our multivariate GRF.
2. Theory
An important aspect of our multivariate GRF is the inclusion of the differential relations
between the atmospheric variables. Under weak regularity assumptions the derivative of a
Gaussian process is again a Gaussian process (Adler and Taylor 2007). Hence, the assump-
tion of Gaussianity of the streamfunction and the velocity potential implies Gaussianity of
all the considered variables. A zero-mean Gaussian process is uniquely characterized by the
covariance function, we only need to study the joint covariance of a random field and its
derivatives. A Gaussian process Xs,s ∈Rdis a continuously indexed stochastic process.
For each finite number of locations (si,i = 1,...,n) the variables (Xsi,i = 1,...,n) have a
multivariate Gaussian distribution.
Let Xs,s ∈R, be a stochastic process with finite second moments, and assume that the
covariance function C(s, t) = Cov(Xs,Xt) is twice continuously differentiable, then the co-
variance model of the process and its mean-square derivative is given by
Cov
Xs
dsXs
,
Xt
dtXt
=
Cov(Xs,Xt)dtCov (Xs,Xt)
dsCov(Xs,Xt)dsdtCov (Xs,Xt)
,(1)
4
where s,t ∈R(Ritter 2000). Using the linearity in the arguments the validity of this equation
can be roughly seen by
Cov(Xs,dtXt) = lim
∆→0CovXs,Xt−Xt+∆
∆
= lim
∆→0
Cov(Xs,Xt)−Cov (Xs,Xt+∆)
∆
=dtCov(Xs,Xt).
One key advantage of this approach is that the bivariate covariance in (1) allows us to
model the dependence between the process and its derivative. In order to provide a better
theoretical basis for this idea, we consider the following definiton.
Definition. A stochastic process Xt, t ∈Rd, is mean square differentiable at t∈Rdin di-
rection ei,i = 1,...,d, if there exists a random variable X(i)
twith EX(i)
t2
<∞such that,
E Xt−Xt+∆ei
∆!−X(i)
t!2
→0 as ∆ →0,
where eidenotes the unit vector in the i−th coordinate direction. In this case, we use the
following notation ∂
∂eiXt=X(i)
t.
A stochastic process is mean square differentiable if its covariance function is twice contin-
uously differentiable (Ritter 2000). However, this condition is neither sufficient nor necessary
for the differentiability of the sample paths. For Gaussian processes the following conditions
on the derivatives of the process guarantees continuity of the sample paths. The paths of a
Gaussian process are continuous, if there exist 0 < C < ∞and α, η > 0 such that
E
∂
∂s Xs−∂
∂t Xt
2
≤C
|log|s−t||1+α,
for all |s−t|< η, see Theorem 1.4.1. in Adler and Taylor (2007).
In our case, the covariance function describes the dependence of the horizontal wind com-
ponents usand vs, streamfunction ψ, velocity potential χ, and the Laplacian of the potentials
(i.e. vorticity ζ= ∆ψand divergence D= ∆χ) at locations s, t ∈R2,
C(s,t) = Covψs, χs, us, vs,∆ψs,∆χsT,ψt, χt, ut, vt,∆ψt,∆χtT.(2)
The covariance function C(s, t) is well-defined, if
Cψ,χ (s, t) = CovψsχsT,ψtχtT
5
is four times continuously differentiable. Four times differentiability of the covariance func-
tion is equivalent to the process being twice mean square differentiable, see Lemma 14 in
Ritter (2000).
In the remainder of the paper we will consider stationary processes, which means that
C(s,t) depends only on the lag vector h=t−s. We will adopt a commonly used notation
for stationary processes, C(h):=C(0, h). Our next step is to review two notions of isotropy
that exist for multivariate processes. Following Schlather et al. (2015) a vector of scalar
quantities is called isotropic if the covariance function Cfulfills
C(Qh) = C(h)h∈Rd,(3)
for all rotation matrices Qand h=t−s. A matrix Q is a rotation matrix if QQTequals
the d-dimensional identity matrix and det(Q) = 1. Under the assumption of stationarity
(3) is equivalent to the more typically used notion of isotropy C(h) = C(khk). Bi- (multi-)
variate variables consisting of scalar quantities such as streamfunction, velocity potential or
the Laplacian thereof fulfill (3). A multivariate process is vector isotropic if its covariance
functions fulfills
C(h) = QTC(Qh)Qfor all h∈Rd.(4)
This relation shows that EX0XT
h=EQTX0QTXQhT,which means that the covari-
ance is preserved if the lag vector hand the random vector are rotated simultaneously.
In the remainder of the paper we consider isotropic processes, hence Cψ,χ (Qh) = Cψ,χ (h)
for all rotation matrices Q. Using the notation,
A=
r1cosθ r1sin θ
−r2sinθ r2cos θ
,(5)
we set Cψ,χ,A (h) = Cψ,χ (Ah).
The effect of the anisotropy matrix Aon the covariance function of the vector components,
namely the rotational part ∇ × ψand the divergent part ∇χ, is non-trivial. The divergent
part satisfies
Cov(∇χ(As),∇χ(At)) = ATCov((∇χ) (As),(∇χ)(At)) A. (6)
The rotational part fulfills a more complex formula
Cov(∇ × ψ(As),∇ × ψ(At)) (7)
=RATRTCov((∇ × ψ) (As),(∇ × ψ) (At)) RART,
6
where
R=
0−1
1 0
.
If Ais simply a rotation matrix (i.e. r1=r2= 1), then RART=A, which implies that
both the divergent and the rotational part are vector-isotropic. For the Laplacians we obtain
the following transformation
Cov(∆χ(As),∆χ(At)) = r4
1Cov∂2
e1χAs , ∂2
e1χAt+r4
2Cov∂2
e2χAs , ∂2
e2χAt
+ 2r2
1r2
2Cov∂2
e1χAs , ∂2
e2χAt.(8)
In the appendix we provide the formulae for all entries of the covariance matrix (2) in the
isotropic case. Equations (6)−(8) are useful since they are the easiest way to compute the
covariance in the anisotropic case from the covariance in the isotropic case. They have been
derived using the chain rule and the linearity of the covariance function in both arguments.
Our GRF is a counter example to a theorem of Obukhov (1954), which claims that the
rotational and divergent component of isotropic vector fields are necessarily uncorrelated,
which is equivalent to streamfunction and velocity potential being uncorrelated. Obukhov
considers an invalid expression for the covariance of a rotational field and deduces from this
expression that it is necessarily uncorrelated to a gradient field. We present the detailed
argument in the Appendix.
In the remainder of the paper we will exemplify the full process in the case that the
potential functions have the following bivariate structure.
Cψ,χ (s, t) =
σ2
ψρσψσχ
ρσψσχσ2
χ
M(kA(t−s)k2,ν),(9)
where M(·,ν) denotes the Mat´ern correlation function with smoothness parameter ν, and
kt−sk2the L2norm. Goulard and Voltz (1992) consider a more general model and prove
its positive definiteness, implying the positive definiteness of our model (9).
Fig. 1 represents a realization of the full stochastic process, with parameters chosen in
order illustrate the flexibility of the model. The rotational wind component is larger than
the divergent wind component with a ratio of σχ/σψ= 0.3. The two potential functions are
strongly correlated with a correlation coefficient of ρ= 0.7. The coherence of the variables
can be very well spotted, although the simulation of the process is inherently stochastic. The
smoothness is set to (ν= 5), which implies that not only the potentials but also vorticity
and divergence are continuously differentiable. We will see later in Section 4, that realistic
mesoscale wind fields have a smoothness parameter close to 1.25. This suggests that the
vorticity and divergence fields are dis-continuous.
7
10 20 30 40 50
10 20 30 40 50
−2
−1
0
1
2
3
4
[m2s]
a)
−2
−1
0
1
2
3
4
10 20 30 40 50
10 20 30 40 50
−2
−1
0
1
2
3
4
[m2s]
b)
−2
−1
0
1
2
3
4
10 20 30 40 50
10 20 30 40 50
−20
−10
0
10
20
[1 s]
c)
−20
−10
0
10
20
10 20 30 40 50
10 20 30 40 50
−20
−10
0
10
20
[1 s]
d)
−20
−10
0
10
20
Figure 1: Isotropic realization of the multivariate GRF with parameters ν= 5, σχ/σψ=
0.3, ρ = 0.7, r1=r2= 0.25. In color are shown a) streamfunction, b) velocity potential, c)
vorticity, and d) divergence. The arrows represent the associated wind fields in m/s. The
arrow in the right upper corner is a standard arrow of 0.5 m/s. The x/y-axis indicate
distance measured in grid points.
3. Data
The horizontal wind fields are taken from the numerical weather prediction (NWP) model
COSMO-DE, namely the wind fields at model level 20 (i.e. at approximately 7 km height).
COSMO-DE is the operational version of the non-hydrostatic limited-area NWP model
COSMO (Consortium of Small-scale Modeling) operated by DWD (Baldauf et al. 2011).
It provides forecasts over Germany and surrounding countries on a 2.8 km horizontal grid
and 50 vertical levels. At this grid size deep convection is permitted by the dynamics,
and COSMO-DE is able to generate deep convection without an explicit parameteriza-
tion thereof. Thus COSMO-DE particularly aims at the prediction of mesoscale convective
precipitation with a forecast horizon of up to one day. The ensemble prediction system
8
Figure 2: Zonal wind component at 12 UTC on 5 June 2011. a) Shows the inner-LBC
anomalies, b) the transformed inner-LBC anomalies. The colors represent wind speed in
m/s. The x/y- axis are in longitude and latitude.
(COSMO-DE-EPS) uses COSMO-DE with different lateral boundary conditions (LBC),
perturbed initial conditions, and slightly modified parameterizations. The four LBC are
generated by the Global Forecast Systems of NCEP, the Global Model of DWD, the Inte-
grated Forecast System of ECMWF and the Global Spectral Model of the Meteorological
Agency of Japan. For details on the setup of COSMO-DE-EPS the reader is referred to
Gebhardt et al. (2011), Peralta et al. (2012), and references therein.
In our application we concentrate on a COSMO-DE forecast for 12 UTC on 5 June 2011
initialized on 00 UTC. COSMO-DE-EPS provides 20 forecasts of horizontal wind fields on
a grid with 461 ×421 grid points. Five ensemble members are forced with identical LBC,
respectively. They only differ due to perturbed initial conditions and four different param-
eterizations. Thus differences between the members with identical LBC are mainly due to
small-scale internal dynamics. These differences are the differences obtained from subtract-
ing two fields which have been generated using the same lateral boundary conditions. All
combinations of fields with different model physics and identical lateral boundary condi-
tions generate a set of 40 different fields of differences. The differences are referred to as
inner-LBC anomalies.
To illustrate the data, Fig. 2 displays a field of inner-LBC anomalies of the zonal wind
component. The fields exhibits small scale anomalies with amplitudes that vary over the
model region while the spatial structure seems relatively homogeneous. Thus, the data vio-
9
late the assumption of stationarity. In order to model the instationarity of the variance we
estimate the spatial kinetic energy b
gby applying a kernel smoother to the kinetic energy
field. In analogy to the field of electric susceptibility (1 + χe) which models the spatial vary-
ing potential polarization of the dielectric medium (Jackson 1962), we apply the following
transformation to the data
e
Us=Us
c+b
gs
,
where c∈R+. Such a transformation, if applied to the full field e
χ, e
ψ, e
U, f
D, e
ζ=
(χ,ψ,U, D, ζ)/(c+b
g), violates the differential relations that hold between the variables,
though they are still valid approximately. For example for a non-rotational field we have
∇ χ
c+b
g!=∇χ
c+b
g+ε. (10)
The smoother the transformation the smaller the approximation error
ε=−χ∇(c+b
g)
(c+b
g)2.
Due to the constant c > 0 the transformation (10) does not resolve the full instationarity of
the data. Still we find that this transformation is superior to the more natural transformation
e
U=U/b
g, as the approximation error for the potential functions is strongly reduced by the
introduction of c > 0. We observe a trade-off between the differential relations being hardly
violated and on the other side Gaussian marginal distribution and constant variance in space
by a rougher function b
gand values of cclose to zero. We chose c= 1/3 and a kernel such that
the transformation kurtosis of the data is reduced from 24 to 16, while we have to accept
an error of the potential fields close to 15 percent. The error is measured by comparing
the potential that satisfies ∇e
χ=U/(c+b
g) and the potential that satisfies ∇χ=Uand is
normalized by c+b
g(the same is done for the rotational part). Figure 2 shows that the
instationarity of the original fields is mitigated by the transformation. Figure 3 shows
the marginal distribution of the transformed inner-LBC anomalies for the zonal and the
meridional wind component. Both distributions deviate from the assumption of Gaussian
marginals, although Gaussianity is a common assumption for wind fields in the meteorolog-
ical literature (Frehlich et al. 2001). The kurtosis amounts to about 16 instead of 3, which
results in heavier extreme values than expected under the assumption of Gaussianity.
4. Parameter estimation
We start by parameter estimation of the bivariate GRF model for the transformed inner-
LBC anomalies of the horizontal wind fields described in Section 3. Since the computation of
10
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●
●●
●
●●
●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●
●●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●
●●
●
●●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●●
●
●●
●●
●
●●
●
●●
●●
●
●●
●
●●
●
●●
●
●●
●●
●
●●
●●
●
●●
●
●●
●●
●
●●
●●
●
●●
●
●●
●●
●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●
●●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●●
●●
●
●●
●●
●●
●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●
●●●
●
●●●
●●
●●
●●
●●
●●
●
●●●
●●
●●
●●
●●
●
●●
●●
●●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●●
●●
●●●
●●
●●
●●●
●●
●●
●●●
●●
●●
●●●
●●
●●●
●●
●●●
●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●
●●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●●
●●
●●●
●●
●●●
●●●
●●
●●●
●●
●●●
●●
●●●
●●●
●●
●●●
●●
●●●
●●●
●●
●●●
●●●
●●
●●●
●●●
●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●●
●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●
●●●
●●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●●
●●●
●●●
●●●●
●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●
●●●●
●●●
●●●●
●●●
●●●●
●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●
●●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●
●●●●●●●