Juan-Pablo Ortega

Nanyang Technological University | ntu · Division of Mathematical Sciences (DMS)

This work aims to prove that the classical Gaussian kernel, when defined on a non-Euclidean symmetric space, is never positive-definite for any choice of parameter. To achieve this goal, the paper develops new geometric and analytical arguments. These provide a rigorous characterization of the positive-definiteness of the Gaussian kernel, which is...

Kernel methods are powerful tools in machine learning. Classical kernel methods are based on positive-definite kernels, which map data spaces into reproducing kernel Hilbert spaces (RKHS). For non-Euclidean data spaces, positive-definite kernels are difficult to come by. In this case, we propose the use of reproducing kernel Krein space (RKKS) base...

Devising deep latent variable models for multi-modal data has been a long-standing theme in machine learning research. Multi-modal Variational Autoencoders (VAEs) have been a popular generative model class that learns latent representations which jointly explain multiple modalities. Various objective functions for such models have been suggested, o...

A well-specified parametrization for single-input/single-output (SISO) linear port-Hamiltonian systems amenable to structure-preserving supervised learning is provided. The construction is based on controllable and observable normal form Hamiltonian representations for those systems, which reveal fundamental relationships between classical notions...

On Euclidean spaces, the Gaussian kernel is one of the most widely used kernels in applications. It has also been used on non-Euclidean spaces, where it is known that there may be (and often are) scale parameters for which it is not positive definite. Hope remains that this kernel is positive definite for many choices of parameter. However, we show...

Reservoir computing systems are constructed using a driven dynamical system in which external inputs can alter the evolving states of a system. These paradigms are used in information processing, machine learning, and computation. A fundamental question that needs to be addressed in this framework is the statistical relationship between the input a...

Numerical evaluations of the memory capacity (MC) of recurrent neural networks reported in the literature often contradict well-established theoretical bounds. In this paper, we study the case of linear echo state networks, for which the total memory capacity has been proven to be equal to the rank of the corresponding Kalman controllability matrix...

Numerical evaluations of the memory capacity (MC) of recurrent neural networks reported in the literature often contradict well-established theoretical bounds. In this paper, we study the case of linear echo state networks, for which the total memory capacity has been proven to be equal to the rank of the corresponding Kalman controllability matrix...

Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems that extends the so-called generalized Barron functionals to a dynamic context. This new class is characterized by the readouts with a certain integral representation built on infinite-dimensional state-space systems. It is shown t...

Reservoir computing approximation and generalization bounds are proved for a new concept class of input/output systems that extends the so-called generalized Barron functionals to a dynamic context. This new class is characterized by the readouts with a certain integral representation built on infinite-dimensional state-space systems. It is shown t...

A complete structure-preserving learning scheme for single-input/single-output (SISO) linear port-Hamiltonian systems is proposed. The construction is based on the solution, when possible, of the unique identification problem for these systems, in ways that reveal fundamental relationships between classical notions in control theory and crucial pro...

A complete structure-preserving learning scheme for single-input/single-output (SISO) linear port-Hamiltonian systems is proposed. The construction is based on the solution, when possible, of the unique identification problem for these systems, in ways that reveal fundamental relationships between classical notions in control theory and crucial pro...

A well-specified parametrization for single-input/single-output (SISO) linear port-Hamiltonian systems amenable to structure-preserving supervised learning is provided. The construction is based on controllable and observable normal form Hamiltonian representations for those systems , which reveal fundamental relationships between classical notions...

A well-specified parametrization for single-input/single-output (SISO) linear port-Hamiltonian systems amenable to structure-preserving supervised learning is provided. The construction is based on controllable and observable normal form Hamiltonian representations for those systems, which reveal fundamental relationships between classical notions...

The first years of an infant's life represent a sensitive period for neurodevelopment where one can see the emergence of nascent forms of executive function (EF), which are required to support complex cognition. Few tests exist for measuring EF during infancy, and the available tests require painstaking manual coding of infant behaviour. In modern...

On Euclidean spaces, the Gaussian kernel is one of the most widely used kernels in applications. It has also been used on non-Euclidean spaces, where it is known that there may be (and often are) scale parameters for which it is not positive definite. Hope remains that this kernel is positive definite for many choices of parameter. However, we show...

On Euclidean spaces, the Gaussian kernel is one of the most widely used kernels in applications. It has also been used on non-Euclidean spaces, where it is known that there may be (and often are) scale parameters for which it is not positive definite. Hope remains that this kernel is positive definite for many choices of parameter. However, we show...

The empirical laws governing human-curvilinear movements have been studied using various relationships, including minimum jerk, the 2/3 power law, and the piecewise power law. These laws quantify the speed-curvature relationships of human movements during curve tracing using critical speed and curvature as regressors. In this work, we provide a res...

A universal kernel is constructed whose sections approximate any causal and time-invariant filter in the fading memory category with inputs and outputs in a finite-dimensional Euclidean space. This kernel is built using the reservoir functional associated with a state-space representation of the Volterra series expansion available for any analytic...

A universal kernel is constructed whose sections approximate any causal and time-invariant filter in the fading memory category with inputs and outputs in a finite-dimensional Euclidean space. This kernel is built using the reservoir functional associated with a state-space representation of the Volterra series expansion available for any analytic...

The first years of an infant’s life represent a sensitive period for neurodevelopment and see the emergence of nascent forms of executive function (EF), which are required to support complex cognition. Few tests exist for measuring EF during infancy, and the available tests require painstaking manual coding of infant behaviour. In modern clinical a...

Most existing results in the analysis of quantum reservoir computing (QRC) systems with classical inputs have been obtained using the density matrix formalism. This paper shows that alternative representations can provide better insights when dealing with design and assessment questions. More explicitly, system isomorphisms have been established th...

Most existing results in the analysis of quantum reservoir computing (QRC) systems with classical inputs have been obtained using the density matrix formalism. This paper shows that alternative representations can provide better insights when dealing with design and assessment questions. More explicitly, system isomorphisms have been established th...

Macroeconomic forecasting has recently started embracing techniques that can deal with large-scale datasets and series with unequal release periods. The aim is to exploit the information contained in heterogeneous data sampled at different frequencies to improve forecasting exercises. Currently , MIxed-DAta Sampling (MIDAS) and Dynamic Factor Model...

Macroeconomic forecasting has recently started embracing techniques that can deal with large-scale datasets and series with unequal release periods. The aim is to exploit the information contained in heterogeneous data sampled at different frequencies to improve forecasting exercises. Currently, MIxed-DAta Sampling (MIDAS) and Dynamic Factor Models...

Research in machine-learning classification of electroencephalography (EEG) data offers important perspectives for the diagnosis and prognosis of a wide variety of neurological and psychiatric conditions, but the clinical adoption of such systems remains low. We propose here that much of the difficulties translating EEG-machine learning research to...

Reservoir computing systems are constructed using a driven dynamical system in which external inputs can alter the evolving states of a system. These paradigms are used in information processing, machine learning, and computation. A fundamental question that needs to be addressed in this framework is the statistical relationship between the input a...

With the penetration of artificial intelligence (AI) technology into industrial applications, not only computational effectiveness but also computational efficiency in machine learning (ML) methods has been increasingly demanded. Reservoir computing (RC) is an ML framework leveraging a dynamic reservoir for a nonlinear transformation of sequentia...

This work studies approximation based on single-hidden-layer feedforward and recurrent neural networks with randomly generated internal weights. These methods, in which only the last layer of weights and a few hyperparameters are optimized, have been successfully applied in a wide range of static and dynamic learning problems. Despite the popularit...

We revisit the role played by sentiment extracted from news articles related to earnings announcements as a driver of firms' return, volatility, and trade volume dynamics. To this end we apply causal machine learning on the earnings announcements of a wide cross-section of US companies. This approach allows us to investigate firms' price and volume...

Research in machine-learning classification of electroencephalography (EEG) data offers important perspectives for the diagnosis and prognosis of a wide variety of neurological and psychiatric conditions, but the clinical adoption of such systems remains low. We propose here that much of the difficulties translating EEG-machine learning research to...

Many recurrent neural network machine learning paradigms can be formulated using state-space representations. The classical notion of canonical state-space realization is adapted in this paper to accommodate semi-infinite inputs so that it can be used as a dimension reduction tool in the recurrent networks setup. The so called input forgetting prop...

This paper shows that the celebrated Embedding Theorem of Takens is a particular case of a much more general statement according to which, randomly generated linear state-space representations of generic observations of an invertible dynamical system carry in their wake an embedding of the phase space dynamics into the chosen Euclidean state space....

This paper shows that the celebrated Embedding Theorem of Takens is a particular case of a much more general statement according to which, randomly generated linear state-space representations of generic observations of an invertible dynamical system carry in their wake an embedding of the phase space dynamics into the chosen Euclidean state space....

This paper shows that a large class of fading memory state-space systems driven by discrete-time observations of dynamical systems defined on compact manifolds always yields continuously differentiable synchronizations. This general result provides a powerful tool for the representation, reconstruction, and forecasting of chaotic attractors. It als...

This paper shows that a large class of fading memory state-space systems driven by discrete-time observations of dynamical systems defined on compact manifolds always yields continuously differentiable synchronizations. This general result provides a powerful tool for the representation, reconstruction, and forecasting of chaotic attractors. It als...

A new explanation of geometric nature of the reservoir computing phenomenon is presented. Reservoir computing is understood in the literature as the possibility of approximating input/output systems with randomly chosen recurrent neural systems and a trained linear readout layer. Light is shed on this phenomenon by constructing what is called stron...

We revisit the role played by sentiment extracted from news articles related to earnings announcements as a driver of firms' return, volatility, and trade volume dynamics. To this end we apply causal machine learning on the earnings announcements of a wide cross-section of US companies. This approach allows us to investigate firms' price and volume...

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bo...

Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various $L ^p$-type criteria. When $1\leq p< \infty$, only $p$-integrability hypotheses need to be imposed, while in the case $p=\infty$ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the l...

Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various $L ^p$-type criteria. When $1\leq p< \infty$, only $p$-integrability hypotheses need to be imposed, while in the case $p=\infty$ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the l...

This paper shows that a large class of fading memory state-space systems driven by discrete-time observations of dynamical systems defined on compact manifolds always yields continuously differentiable synchronizations. This general result provides a powerful tool for the representation, reconstruction, and forecasting of chaotic attractors. It als...

A new explanation of geometric nature of the reservoir computing phenomenon is presented. Reservoir computing is understood in the literature as the possibility of approximating input/output systems with randomly chosen recurrent neural systems and a trained linear readout layer. Light is shed on this phenomenon by constructing what is called stron...

Many recurrent neural network machine learning paradigms can be formulated using state-space representations. The classical notion of canonical state-space realization is adapted in this paper to accommodate semi-infinite inputs so that it can be used as a dimension reduction tool in the recurrent networks setup. The so called input forgetting prop...

Reservoir computing (RC) is a machine learning framework for temporal (sequential) pattern recognition, which originates from specific types of recurrent neural network models including echo state networks and liquid state machines. An RC system consists of a dynamical reservoir for mapping inputs into a high-dimensional temporal representation spa...

The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs makes natural the introduction of the network forecasting capacity, that measures the possibility of forecastin...

The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs makes natural the introduction of the network forecasting capacity, that measures the possibility of forecastin...

This work studies approximation based on single-hidden-layer feedforward and recurrent neural networks with randomly generated internal weights. These methods, in which only the last layer of weights and a few hyperparameters are optimized, have been successfully applied in a wide range of static and dynamic learning problems. Despite the popularit...

EXTENDED DEADLINE: 30 JANUARY 2020 *Aim and Scope. * Reservoir Computing (RC) defines a class of recurrent neural systems where the dynamical memory component is left untrained after initialization. Only a simple-typically linear-readout layer is adapted on a set of training examples, thereby allowing the use of simple learning strategies. The ove...

Fully explicit closed-form expressions are developed for the fair strike prices of discrete-time variance swaps under general affine GARCH type models that have been risk-neutralized with a family of variance dependent pricing kernels. The methodology relies on solving differential recursions for the coefficients of the joint cumulant generating fu...

We analyze the practices of reservoir computing in the framework of statistical learning theory. In particular, we derive finite sample upper bounds for the generalization error committed by specific families of reservoir computing systems when processing discrete-time inputs under various hypotheses on their dependence structure. Non-asymptotic bo...

Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so called echo state and fading memory properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this...

Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so called echo state and fading memory properties. These important features amount, in mathematical terms, to the existence and continuity of global reservoir system solutions. That research is complemented in this...

The universal approximation properties with respect to L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sup> -type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs are shown. First, it is proven that linear reservoir systems with either...

A new class of non-homogeneous state-affine systems is introduced. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters wi...

This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. This result guarantees that any fading memory input/output system in discrete time can be realized as a simple finite-dimensional neural network-type sta...

The universal approximation properties with respect to $L^p$-type criteria of three important families of reservoir computers with stochastic discrete-time semi-infinite inputs are shown. First, it is proved that linear reservoir systems with either polynomial or neural network readout maps are universal. More importantly, it is proved that the sam...

We propose a new approach based on a generalization of the logit model to improve prediction accuracy in US bank failures. Mixed-data sampling (MIDAS) is introduced in the context of a logistic regression. We also mitigate the class-imbalance problem in data and adjust the classification accuracy evaluation. In applying the suggested model to the p...

This paper shows that echo state networks are universal uniform approximants in the context of discrete-time fading memory filters with uniformly bounded inputs defined on negative infinite times. The proof uses newly introduced internal approximation results for filters associated to reservoir computing systems, as well as the external approximati...

The Hilbert metric is a widely used tool for analysing the convergence of Markov processes and the ergodic properties of deterministic dynamical systems. A useful representation formula for the Hilbert metric was given by Liverani. The goal of the present paper is to extend this formula to the non-compact and multidimensional setting with a differe...

A new class of non-homogeneous state-affine systems is introduced. Sufficient conditions are identified that guarantee first, that the associated reservoir computers with linear readouts are causal, time-invariant, and satisfy the fading memory property and second, that a subset of this class is universal in the category of fading memory filters wi...

In this paper, we derive fully explicit closed-form expressions for the fair strike prices of discrete-time variance swaps under general affine GARCH type models that have been risk-neutralized with a family of variance dependent pricing kernels. The methodology relies on solving differential recursions for the coefficients of the joint cumulant ge...

This paper investigates the pricing and weak convergence of an asymmetric nonaffine, non-Gaussian GARCH model when the risk neutralization is based on a variance-dependent exponential linear pricing kernel with stochastic risk aversion parameters. The risk-neutral dynamics are obtained for a general setting and its weak limit is derived. We show ho...

The aim of this presentation is to show how various ideas coming from the nonlinear stability theory of functional differential systems, stochastic modeling, and machine learning, can be put together in order to create an approximating model that explains the working mechanisms behind a certain type of reservoir computers. Reservoir computing is a...

This paper extends the notion of information processing capacity for non-independent input signals in the context of reservoir computing (RC). The presence of input autocorrelation makes worthwhile the treatment of forecasting and filtering problems for which we explicitly compute this generalized capacity as a function of the reservoir parameter v...

In this paper we study a conditional version of the Wang transform in the context of discrete GARCH models and their diffusion limits. Our first contribution shows that the conditional Wang transform and Duans generalized local risk-neutral valuation relationship based on equilibrium considerations, lead to the same GARCH option pricing model. We d...

This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those errors. Unlike other classical re...

This letter addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir architecture and its per...

Un des défis majeurs auxquels sont confrontés les services de réanimation est de pouvoir prédire précisément et précocement vers quel état évoluera la conscience des patients dans le coma pour anoxie cérébrale. Afin d’obtenir des indices directs sur l’état cérébral fonctionnel, l’emploi des potentiels évoqués est recommandé, et notamment l’enregist...

To measure the level of residual cognitive function in patients with disorders of consciousness, the use of electrophysiological and neuroimaging protocols of increasing complexity is recommended. This work presents an EEG-based method capable of assessing at an individual level the integrity of the auditory cortex at the bedside of patients and ca...

Numerous empirical proofs indicate the adequacy of the time discrete auto-regressive stochastic volatility models introduced by Taylor (Taylor S. J. [1982]; Taylor S. J. [1986]; Taylor S. J. [2005]) in the dynamical description of the log-returns of financial assets. The pricing and hedging of contingent products that use these models for their und...

This paper investigates the pricing and weak convergence of an asymmetric non-affine, non-Gaussian GARCH model when the risk-neutralization is based on a variance dependent exponential linear pricing kernel with stochastic risk aversion parameters. The risk-neutral dynamics are obtained for a general setting and its weak limit is derived. We show h...

In this paper we study a conditional version of the Wang transform in the context of discrete GARCH models and their diffusion limits. Our first contribution shows that the conditional Wang transform and Duan’s generalized local risk-neutral valuation relationship based on equilibrium considerations, lead to the same GARCH option pricing model. We...

This paper addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir parameters and architectu...

This paper extends the notion of information processing capacity for non-independent input signals in the context of reservoir computing (RC). The presence of input autocorrelation makes worthwhile the treatment of forecasting and filtering problems for which we explicitly compute this generalized capacity as a function of the reservoir parameter v...

This paper addresses the reservoir design problem in the context of delay-based reservoir computers for multidimensional input signals, parallel architectures, and real-time multitasking. First, an approximating reservoir model is presented in those frameworks that provides an explicit functional link between the reservoir parameters and architectu...

This paper extends the notion of information processing capacity for non-independent input signals in the context of reservoir computing (RC). The presence of input autocorrelation makes worthwhile the treatment of forecasting and filtering problems for which we explicitly compute this generalized capacity as a function of the reservoir parameter v...

This paper investigates the weak convergence of general non-Gaussian GARCH models together with an application to the pricing of European style options determined using an extended Girsanov principle and a conditional Esscher transform as the pricing kernel candidates. Applying these changes of measure to asymmetric GARCH models sampled at increasi...

This paper introduces a method based on the use of various linear and nonlinear state space models that uses non-synchronous data to extract global stochastic financial trends (GST). These models are specifically constructed to take advantage of the intraday arrival of closing information coming from different international markets in order to impr...

To improve the assessment of awareness in patients with disorders of consciousness, recent protocols using fMRI have been developed, and led some specialized coma centers to use this method on a routine basis. Recently, promising results have also been observed with EEG, a less expensive and widely available technique. However, since the spatiotemp...

This paper presents a method capable of estimating richly parametrized versions of the dynamic conditional correlation (DCC) model that go beyond the standard scalar case. The algorithm is based on the maximization of a Gaussian quasi-likelihood using a Bregman-proximal trust-region method to handle the various non-linear stationarity and positivit...

Recent studies have evidenced serious difficulties in detecting covert awareness with electroencephalography-based techniques both in unresponsive patients and in healthy control subjects. This work reproduces the protocol design in two recent mental imagery studies with a larger group comprising 20 healthy volunteers. The main goal is assessing if...