Anatoly Zhigljavsky

Cardiff University | CU · School of Mathematics

We consider the problem of predicting values of a random process or field satisfying a linear model y(x)=θ⊤f(x)+ε(x)\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y(x)...

We consider global optimization problems, where the feasible region X\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\mathcal {X}}$$\end{document} is a compact subset...

The main purpose of the paper is to uncover the connections between kriging, energy minimization and properties of the ordinary least squares and best linear unbiased estimators in the location model with correlated observations. We emphasize the special role of the constant function and illustrate our results by several examples.

Let $(\mathcal{X},\rho)$ be a metric space and $\lambda$ be a Borel measure on this space defined on the $\sigma$-algebra generated by open subsets of $\mathcal{X}$; this measure $\lambda$ defines volumes of Borel subsets of $\mathcal{X}$. The principal case is where $\mathcal{X} = \mathbb{R}^d$, $\rho $ is the Euclidean metric, and $\lambda$ is th...

Real-time bidding has transformed the digital advertising landscape, allowing companies to buy website advertising space in a matter of milliseconds in the time it takes a webpage to load. Joint research between Cardiff University and Crimtan has employed statistical modelling in conjunction with machine-learning techniques on big data to develop c...

The inverse square root of a covariance matrix is often desirable for performing data whitening in the process of applying many common multivariate data analysis methods. Direct calculation of the inverse square root is not available when the covariance matrix is either singular or nearly singular, as often occurs in high dimensions. We develop new...

Let $${\mathbb {Z}}_n = \{Z_1, \ldots , Z_n\}$$ Z n = { Z 1 , … , Z n } be a design; that is, a collection of n points $$Z_j \in [-1,1]^d$$ Z j ∈ [ - 1 , 1 ] d . We study the quality of quantisation of $$[-1,1]^d$$ [ - 1 , 1 ] d by the points of $${\mathbb {Z}}_n$$ Z n and the problem of quality of coverage of $$[-1,1]^d$$ [ - 1 , 1 ] d by $${{{\ma...

We consider global optimization problems, where the feasible region $\X$ is a compact subset of $\mathbb{R}^d$ with $d \geq 10$. For these problems, we demonstrate the following. First: the actual convergence of global random search algorithms is much slower than that given by the classical estimates, based on the asymptotic properties of random po...

In this paper, we study the behaviour of the so-called k -simplicial distances and k -minimal-variance distances between a point and a sample. The family of k -simplicial distances includes the Euclidean distance, the Mahalanobis distance, Oja’s simplex distance and many others. We give recommendations about the choice of parameters used to calcula...

For large classes of group testing problems, we derive lower bounds for the probability that all significant items are uniquely identified using specially constructed random designs. These bounds allow us to optimize parameters of the randomization schemes. We also suggest and numerically justify a procedure of constructing designs with better sepa...

A design is a collection of distinct points in a given set $X$, which is assumed to be a compact subset of $R^d$, and the mesh-ratio of a design is the ratio of its fill distance to its separation radius. The uniformity constant of a sequence of nested designs is the smallest upper bound for the mesh-ratios of the designs. We derive a lower bound o...

In this paper we study approximations for the boundary crossing probabilities of moving sums of i.i.d. normal random variables. We approximate a discrete time problem with a continuous time problem allowing us to apply established theory for stationary Gaussian processes. By then subsequently correcting approximations for discrete time, we show tha...

In this paper, we derive explicit formulas for the first-passage probabilities of the process S ( t ) = W ( t ) − W ( t + 1), where W ( t ) is the Brownian motion, for linear and piece-wise linear barriers on arbitrary intervals [0, T ]. Previously, explicit formulas for the first-passage probabilities of this process were known only for the cases...

We present in this chapter our recent work in Bayesian approach to continuous non-convex optimization. A brief review precedes the main results to have our work presented in the context of challenges of the approach.

This chapter starts by considering, in Sect. 1.1, properties of high-dimensional cubes and balls; we will use many of these properties in other sections of this chapter and in Chap. 3. In Sect. 1.2, we discuss various aspects of uniformity and space-filling and demonstrate that good uniformity of a set of points is by no means implying its good spa...

It is not the aim of this chapter to cover the whole subject of the global random search (GRS). It only contains some potentially important notes on algorithms of GRS in continuous problems, mostly keeping in mind the use of such algorithms in reasonably large dimensions. These notes are based on the 40-year experience of the author of this chapter...

Multistart is a celebrated global optimization technique frequently applied in practice. In its pure form, multistart has low efficiency. However, the simplicity of multistart and multitude of possibilities of its generalization make it very attractive especially in high-dimensional problems where e.g. Lipschitzian and Bayesian algorithms are not a...

We show that polynomials do not belong to the reproducing kernel Hilbert space of infinitely differentiable translation-invariant kernels whose spectral measures have moments corresponding to a determinate moment problem. Our proof is based on relating this question to the problem of best linear estimation in continuous time one-parameter regressio...

For large classes of group testing problems, we derive lower bounds for the probability that all significant factors are uniquely identified using specially constructed random designs. These bounds allow us to optimize parameters of the randomization schemes. We also suggest and numerically justify a procedure of construction of designs with better...

The main problem considered in this paper is construction and theoretical study of efficient n-point coverings of a d-dimensional cube [−1, 1]d. Targeted values of d are between 5 and 50; n can be in hundreds or thousands and the designs (collections of points) are nested. This paper is a continuation of our paper (Noonan and Zhigljavsky, SN Oper R...

Accessible to a variety of readers, this book is of interest to specialists, graduate students and researchers in mathematics, optimization, computer science, operations research, management science, engineering and other applied areas interested in solving optimization problems. Basic principles, potential and boundaries of applicability of stocha...

We study properties of two probability distributions defined on the infinite set \(\{0,1,2, \ldots \}\) and generalizing the ordinary discrete uniform and binomial distributions. Both extensions use the grossone-model of infinity. The first of the two distributions we study is uniform and assigns masses \(1/\textcircled {1}\) to all points in the s...

The applications of SSA dealt with in Chap. 3 require the use of models and hence SSA of Chap. 3 is mostly model-based. As the main model, the assumption that the components of the original time series, which are extracted by SSA, satisfy (at least, locally) certain linear recurrence relations. The main emphasis in Chap. 3 is on time series forecas...

In Chap. 2, SSA is normally considered as a model-free technique. The main body of Chap. 2 is devoted to careful description of Basic SSA, its main capabilities, choice of parameters and various indicators which help in recognizing good separability between different components and hence successful SSA decompositions. Several approaches for improvi...

As the main problem, we consider covering of a d-dimensional cube by n balls with reasonably large d (10 or more) and reasonably small n, like n = 100 or n = 1000. We do not require the full coverage but only 90% or 95% coverage. We establish that efficient covering schemes have several important properties which are not seen in small dimensions an...

We develop algorithms for energy minimization for kernels with singularities. This problem arises in different fields, most notably in the construction of space-filling sequences of points where singularity of kernels guarantees a strong repelling property between these points. Numerical algorithms are based on approximating singular kernels by non...

The main problem considered in this paper is construction and theoretical study of efficient $n$-point coverings of a $d$-dimensional cube $[-1,1]^d$. Targeted values of $d$ are between 5 and 50; $n$ can be in hundreds or thousands and the designs (collections of points) are nested. This paper is a continuation of our paper \cite{us}, where we have...

Let $\mathbb{Z}_n = \{Z_1, \ldots, Z_n\}$ be a design; that is, a collection of $n$ points $Z_j \in [-1,1]^d$. We study the quality of quantization of $[-1,1]^d$ by the points of $\mathbb{Z}_n$ and the problem of quality of covering of $[-1,1]^d$ by ${\cal B}_d(\mathbb{Z}_n,r)$, the union of balls centred at $Z_j \in \mathbb{Z}_n$. We concentrate o...

The primary objective of this work is to model and compare different exit scenarios from the lock-down for the COVID-19 UK epidemic. In doing so we provide an additional modelling basis for laying out the strategy options for the decision-makers. The main results are illustrated and discussed in Part I. In Part II, we describe the stochastic model...

We model further development of the COVID-19 epidemic in the UK given the current data and assuming different scenarios of handling the epidemic. In this research, we further extend the stochastic model suggested in [1] and incorporate in it all available to us knowledge about parameters characterising the behaviour of the virus and the illness ind...

We model further development of the COVID-19 epidemic in the UK given the current data and assuming different scenarios of handling the epidemic. In this research, we further extend the stochastic model suggested in \cite{us} and incorporate in it all available to us knowledge about parameters characterising the behaviour of the virus and the illne...

Coronavirus COVID-19 spreads through the population mostly based on social contact. To gauge the potential for widespread contagion, to cope with associated uncertainty and to inform its mitigation, more accurate and robust modelling is centrally important for policy making. We provide a flexible modelling approach that increases the accuracy with...

Coronavirus COVID-19 spreads through the population mostly based on social contact. To gauge the potential for widespread contagion, to cope with associated uncertainty and to inform its mitigation, more accurate and robust modelling is centrally important for policy making. We provide a flexible modelling approach that increases the accuracy with...

In this article we discuss an online moving sum (MOSUM) test for detection of a transient change in the mean of a sequence of independent and identically distributed (i.i.d.) normal random variables. By using a well-developed theory for continuous time Gaussian processes and subsequently correcting the results for discrete time, we provide accurate...

As the main problem, we consider covering of a $d$-dimensional cube by $n$ balls with reasonably large $d$ (10 or more) and reasonably small $n$, like $n=100$ or $n=1000$. We do not require the full coverage but only 90\% or 95\% coverage. We establish that efficient covering schemes have several important properties which are not seen in small dim...

Matrix C can be blindly deconvoluted if there exist matrices A and B such that C=A⁎B, where ⁎ denotes the operation of matrix convolution. We study the problem of matrix deconvolution in the case where matrix C is proportional to the inverse of the autocovariance matrix of an autoregressive process. We show that the deconvolution of such matrices i...

This book gives an overview of singular spectrum analysis (SSA). SSA is a technique of time series analysis and forecasting combining elements of classical time series analysis, multivariate statistics, multivariate geometry, dynamical systems and signal processing. SSA is multi-purpose and naturally combines both model-free and parametric techniqu...

Matrix $\mathbf{C}$ can be blindly deconvoluted if there exist matrices $\mathbf{A}$ and $\mathbf{B}$ such that $\mathbf{C}= \mathbf{A} \ast \mathbf{B}$, where $\ast$ denotes the operation of matrix convolution. We study the problem of matrix deconvolution in the case where matrix $\mathbf{C}$ is proportional to the inverse of the autocovariance ma...

We consider the problem of predicting values of a random process or field satisfying a linear model $y(x)=\theta^\top f(x) + \varepsilon(x)$, where errors $\varepsilon(x)$ are correlated. This is a common problem in kriging, where the case of discrete observations is standard. By focussing on the case of continuous observations, we derive expressio...

In this paper we study approximations for the boundary crossing probabilities of moving sums of i.i.d. normal r.v. We approximate a discrete time problem with a continuous time problem allowing us to apply established theory for stationary Gaussian processes. By then subsequently correcting approximations for discrete time, we show that the develop...

In this article we study approximations for boundary crossing probabilities for the moving sums of i.i.d. normal random variables. We propose approximating a discrete time problem with a continuous time problem allowing us to apply developed theory for stationary Gaussian processes and to consider a number of approximations (some well known and som...

Slepian process S(t) is a stationary Gaussian process with zero mean and covariance ES(t)S(t′)=max{0,1−|t−t′|}. For any T≥0 and real h, define FT(h)=Prmaxt∈[0,T]S(t)<h and the constants Λ(h)=−limT→∞1TlogFT(h) and λ(h)=exp{−Λ(h)}; we will call them ‘Shepp’s constants’. The aim of the paper is construction of accurate approximations for FT(h) and hen...

In this paper we extend results of L.A. Shepp by finding explicit formulas for the first passage probability $F_{a,b}(T\, |\, x)={\rm Pr}(S(t)<a+bt \text{ for all } t\in[0,T]\,\, | \,\,S(0)=x)$, for all $T>0$, where $S(t)$ is a Gaussian process with mean 0 and covariance $\mathbb{E} S(t)S(t')=\max\{0,1-|t-t'|\}\,.$ We then extend the results to the...

In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is embedded in the wider theory of divergences and distances between distributions which includes Kullback–Leibler,...

We consider a continuous extension of a regularized version of the minimax, or dispersion, criterion widely used in space-filling design for computer experiments and quasi-Monte Carlo methods. We show that the criterion is convex for a certain range of the regularization parameter (depending on space dimension) and give a necessary and sufficient c...

Bayesian approach is actively used to develop global optimization algorithms aimed at expensive black box functions. One of the challenges in this approach is the selection of an appropriate model for the objective function. Normally, a Gaussian random field is chosen as a theoretical model. However, the problem of estimation of parameters, using o...

Bayesian approach is actively used to develop global optimization algorithms aimed at expensive black box functions. One of the challenges in this approach is the selection of an appropriate model for the objective function. Normally, a Gaussian random field is chosen as a theoretical model. However, the problem of estimation of parameters, using o...

Slepian process $S(t)$ is a stationary Gaussian process with zero mean and covariance $ E S(t)S(t')=\max\{0,1-|t-t'|\}\, . $ For any $T>0$ and $h>0$, define $F_T(h ) = {\rm Pr}\left\{\max_{t \in [0,T]} S(t) < h \right\} $ and the constants $\Lambda(h) = -\lim_{T \to \infty} \frac1T \log F_T(h)$ and $\lambda(h)=\exp\{-\Lambda(h) \}$; we will call th...

We study approximations of boundary crossing probabilities for the maximum of moving weighted sums of i.i.d. random variables. We consider a particular case of weights obtained from a trapezoidal weight function which, under certain parameter choices, can also result in an unweighted sum. We demonstrate that the approximations based on classical re...

In this paper we study approximations for boundary crossing probabilities for the maximum of moving sums of i.i.d. normal random variables. We propose approximating a discrete time problem with a continuous time problem allowing us to apply developed theory for stationary Gaussian processes and to consider a number of approximations (some well know...

In previous work the authors defined the k-th order simplicial distance between probability distributions which arises naturally from a measure of dispersion based on the squared volume of random simplices of dimension k. This theory is embedded in the wider theory of divergences and distances between distributions which includes Kullback-Leibler,...

A standard objective in computer experiments is to approximate the behaviour of an unknown function on a compact domain from a few evaluations inside the domain. When little is known about the function, space-filling design is advisable: typically, points of evaluation spread out across the available space are obtained by minimizing a geometrical (...

For a general linear regression model we construct a directional statistic which maximizes the probability that the scalar product between the vector of unknown parameters and any linear estimator is positive. Special emphasis is given to comparison of this directional statistic with the BLUE and explaining why the BLUE could be relatively poor. We...

Chapter 1 is introductory; it outlines the main principles and ideas of SSA, presents a unified view on SSA, reviews its computer implementation in the form of the Rssa package, compares SSA with other methods of time series analysis, gives a short literature review, and provides references to all data sources used. In this chapter, the main concep...

In Chap. 2, the use of SSA for analyzing one-dimensional data is thoroughly examined. In this chapter, the use of models is minimal so that the main techniques can be considered as non-parametric and descriptory. Relations with algorithms of space rotation and many other methods aiming at achieving better separability of signal from noise are outli...

In Chap. 4 the problem of simultaneous decomposition, reconstruction, and forecasting for a collection of time series is considered from the viewpoint of SSA; note that individual time series can have different length. The main method of this chapter is usually called either Multichannel SSA or Multivariate SSA, shortly MSSA. The principal idea of...

Chapter 3 is devoted to applications of SSA for one-dimensional series for forecasting, gap filling, low-rank approximation, parameter estimation, and change-point detection. The SSA analysis of time series of Chap. 2 is model-free. Methods of Chap. 3, on the contrary, are model-based. The model is constructed on the base of the approximating subsp...

Chapter 5 is devoted to extensions of SSA methods developed in previous chapters for the analysis of objects of dimension 2 and larger. The 2D case corresponds to the digital image processing. The objects with larger dimensions are also widely used. For example, a color image can be considered as a system of 2D images and its analysis can be perfor...

This comprehensive and richly illustrated volume provides up-to-date material on Singular Spectrum Analysis (SSA). SSA is a well-known methodology for the analysis and forecasting of time series. Since quite recently, SSA is also being used to analyze digital images and other objects that are not necessarily of planar or rectangular form and may co...

In this paper, we investigate the complexity of the numerical construction of the so-called Hankel structured low-rank approximation (HSLRA). Briefly, HSLRA is the problem of finding a rank r approximation of a given Hankel matrix, which is also of Hankel structure.

We study asymptotic properties of optimal statistical estimators in global random search algorithms when the dimension of the feasible domain is large. The results obtained can be helpful in deciding what sample size is required for achieving a given accuracy of estimation.

We consider the problem of estimating the optimal direction in regression by maximizing the probability that the scalar product between the vector of unknown parameters and the chosen direction is positive. The estimator maximizing this probability is simple in form, and is especially useful for situations where the number of parameters is much lar...

We consider functionals measuring the dispersion of a d-dimensional distribution which are based on the volumes of simplices of dimension k ≤ d formed by k + 1 independent copies and raised to some power δ. We study properties of extremal measures that maximize these functionals. In particular, for positive δ we characterize their support and for n...

In this paper the problem of best linear unbiased estimation is investigated for continuous-time regression models. We prove several general statements concerning the explicit form of the best linear unbiased estimator (BLUE), in particular when the error process is a smooth process with one or several derivatives of the response process available...

The technique of calibration in survey sampling is a widely used technique in the field of official statistics. The main element of the calibration process is an optimization procedure, for which a variety of penalty functions can be used. In this chapter, we consider three of the classical penalty functions that are implemented in many of the stan...

In this paper, we investigate the complexity of the numerical construction of the so-called Hankel structured low-rank approximation (HSLRA). Briefly, HSLRA is the problem of finding the closest (in some pre-defined norm) rank r approximation of a given Hankel matrix, which is also of Hankel structure.

We consider the problem of adaptive targeting for real-time bidding for internet advertisement. This problem involves making fast decisions on whether to show a given ad to a particular user. For demand partners, these decisions are based on information extracted from big data sets containing records of previous impressions, clicks and subsequent p...

In this paper, we investigate the complexity of the numerical construction of the Hankel structured low-rank approximation (HSLRA) problem, and develop a family of algorithms to solve this problem. Briefly, HSLRA is the problem of finding the closest (in some pre-defined norm) rank r approximation of a given Hankel matrix, which is also of Hankel s...

Let be a discrete random variable (r.v.) with uniform distribution on the support set . We study the problem of construction of non-degenerate independent r.v.’s and such that , if these r.v.’s exist. We describe a general form for the solutions to this problem, offer some analytic constructions and develop algorithms for computing the distribution...

In the one-parameter regression model with AR(1) and AR(2) errors we find explicit expressions and a continuous approximation of the optimal discrete design for the signed least square estimator. The results are used to derive the optimal variance of the best linear estimator in the continuous time model and to construct efficient estimators and co...

We consider a measure ψ k of dispersion which extends the notion of Wilk's generalised variance for a d-dimensional distribution, and is based on the mean squared volume of simplices of dimension k ≤ d formed by k + 1 independent copies. We show how ψ k can be expressed in terms of the eigenvalues of the covariance matrix of the distribution, also...

We investigate the problem of adaptive targeting for real-time bidding in online advertisement using independent advertisement exchanges. This is a problem of making decisions based on information extracted from large data sets related to previous experience. We describe an adaptive strategy for optimizing the click through rate which is a key crit...

We formulate the problem of deconvolution of a given vector as an optimal design problem and suggest numerical algorithms for solving this problem. We then discuss an important application of the proposed methods for problems of time series analysis and signal processing and also to the low-rank approximation of structured matrices.

This is a survey of the main achievements in the methodology and theory of stochastic global optimization. It comprises two complimentary directions: global random search and the methodology based on the use of stochastic models about the objective function. The main attention is paid to theoretically substantiated methods and mathematical results...

In the one-parameter regression model with AR(1) and AR(2) errors we find explicit expressions and a continuous approximation of the optimal discrete design for the signed least square estimator. The results are used to derive the optimal variance of the best linear estimator in the continuous time model and to construct efficient estimators and co...

We consider the problem of adaptive targeting for real-time bidding for internet advertisement. This problem involves making fast decisions on whether to show a given ad to a particular user. For intelligent platforms, these decisions are based on information extracted from big data sets containing records of previous impressions, clicks and subseq...

This paper discusses the problem of determining optimal designs for regression models, when the observations are dependent and taken on an interval. A complete solution of this challenging optimal design problem is given for a broad class of regression models and covariance kernels. We propose a class of estimators which are only slightly more comp...

In this paper we consider an important statistical problem of imputing missing values into a time series data. We formulate this problem as a problem of structured low-rank approximation (SLRA), which is a problem of matrix analysis. One of the main difficulties in this SLRA problem is related to the fact that the norm which defines the quality of...

Calibration is a technique of adjusting sample weights routinely used in sample surveys. In this chapter, we consider calibration as an optimization problem and show that the choice of optimization function has an effect on the calibrated weights. We propose a class of functions that have several desirable properties, which includes satisfying nece...

This paper presents a new and effcient method for the construction of optimal designs for regression models with dependent error processes. In contrast to most of the work in this field, which starts with a model for a finite number of observations and considers the asymptotic properties of estimators and designs as the sample size converges to inf...

Partition of the feasible domain is a key ingredient in a multitude of global optimization algorithms including all branch and bound algorithms, both deterministic and stochastic. Routinely, the rectangular partitions are used in applied in global optimization. These partitions may work well if the feasible domain is a box (hyper-rectangle). Howeve...

We consider a measure $\psi$ k of dispersion which extends the notion of Wilk's generalised variance, or entropy, for a d-dimensional distribution, and is based on the mean squared volume of simplices of dimension k $\le$ d formed by k + 1 independent copies. We show how $\psi$ k can be expressed in terms of the eigenvalues of the covariance matrix...

We consider the problem of construction of optimal experimental designs (approximate theory) on a compact subset XX of RdRd with nonempty interior, for a concave and Lipschitz differentiable design criterion ϕ(·)ϕ(·) based on the information matrix. The proposed algorithm combines (a) convex optimization for the determination of optimal weights on...

For a wide class of discrete distributions, we derive a representation of the inverse (negative) moments through the Stirling numbers of the first kind and inverse factorial moments. We specialize the results for the Poisson, binomial, hypergeometric and negative binomial distributions.

A general optimization problem can be schematically formulated as \(f(x) \rightarrow \min _{x \in X}\), where \(f(x)\) is the objective function and \(X\) is the feasible domain, which is usually defined by certain constraints. In problems of global optimization, the objective function \(f(x)\) is multiextremal, typically possessing many local mini...

Screening is a term applied to the general approach of detecting active or significant factors, elements, units, and so on when there are a large number of inactive factors. The meaning of ‘active’ depends on the context: effective treatments, presence of a disease, fault, etc. The methods are of considerable importance in medical and industrial ap...

This special issue of the Communications in Nonlinear Science and Numerical Simulation contains a collection of research papers dealing with various problems of nonlinearity in physics, pure and applied mathematics, and computer science. It also considers numerical techniques and problems arising when natural phenomena are modelled on computers. Th...

The problem of determining optimal designs for least squares estimation is considered in the common linear regression model with correlated observations. The approach is based on the determination of 'nearly' universally optimal designs, even in the case where the universally optimal design does not exist. For this purpose, a new optimality criteri...

In this paper we illustrate some optimization challenges in the structured low rank approximation (SLRA) problem. SLRA can be described as the problem of finding a low rank approximation of an observed matrix which has the same structure as this matrix (such as Hankel). We demonstrate that the optimization problem arising is typically very difficul...

We consider the solution of linear systems of equations Ax=b, with A a symmetric positive-definite matrix in ℝn×n , through Richardson-type iterations or, equivalently, the minimization of convex quadratic functions (1/2)(Ax,x)−(b,x) with a gradient algorithm. The use of step-sizes asymptotically distributed with the arcsine distribution on the spe...