ArticlePublisher preview available

On the estimation of partially observed continuous-time Markov chains

Authors:
If you want to read the PDF, try requesting it from the authors.

Abstract and Figures

Motivated by the increasing use of discrete-state Markov processes across applied disciplines, a Metropolis–Hastings sampling algorithm is proposed for a partially observed process. Current approaches, both classical and Bayesian, have relied on imputing the missing parts of the process and working with a complete likelihood. However, from a Bayesian perspective, the use of latent variables is not necessary and exploiting the observed likelihood function, combined with a suitable Markov chain Monte Carlo method, results in an accurate and efficient approach. A comprehensive comparison with simulated and real data sets demonstrate our approach when compared with alternatives available in the literature.
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
Computational Statistics
https://doi.org/10.1007/s00180-022-01273-w
1 3
ORIGINAL PAPER
On theestimation ofpartially observed continuous‑time
Markov chains
AlanRiva‑Palacio1 · RamsésH.Mena1· StephenG.Walker2
Received: 17 December 2020 / Accepted: 5 August 2022
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2022
Abstract
Motivated by the increasing use of discrete-state Markov processes across applied
disciplines, a Metropolis–Hastings sampling algorithm is proposed for a partially
observed process. Current approaches, both classical and Bayesian, have relied on
imputing the missing parts of the process and working with a complete likelihood.
However, from a Bayesian perspective, the use of latent variables is not necessary
and exploiting the observed likelihood function, combined with a suitable Markov
chain Monte Carlo method, results in an accurate and efficient approach. A com-
prehensive comparison with simulated and real data sets demonstrate our approach
when compared with alternatives available in the literature.
Keywords Bayesian estimation· Transition matrix· Credit risk scoring
1 Introduction
We consider the inference problem of a partially observed continuous-time Markov
chain (CTMC), written as
X∶= {
X
(
t
);
t𝜏
}
, that take values on a finite state space,
𝕊∶= {1, ,m}
. Such continuous-time discrete-state systems find applications in
areas such as physics, Van Kampen (2007); ecology, Fukaya and Royle (2013);
neuroscience, Sauer (2016); and finance, Pardoux (2008). Hence, the need for
The authors gratefully acknowledge the support of project CONTEX 2018-9B and PAPIIT-UNAM
IG100221.
* Alan Riva-Palacio
alan@sigma.iimas.unam.mx
Ramsés H. Mena
ramses@sigma.iimas.unam.mx
Stephen G. Walker
s.g.walker@math.utexas.edu
1 IIMAS, UNAM, MexicoCity, Mexico
2 University ofTexas atAustin, Austin, USA
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
An amendment to this paper has been published and can be accessed via a link at the top of the paper.
Article
Full-text available
SciPy is an open-source scientific computing library for the Python programming language. Since its initial release in 2001, SciPy has become a de facto standard for leveraging scientific algorithms in Python, with over 600 unique code contributors, thousands of dependent packages, over 100,000 dependent repositories and millions of downloads per year. In this work, we provide an overview of the capabilities and development practices of SciPy 1.0 and highlight some recent technical developments. This Perspective describes the development and capabilities of SciPy 1.0, an open source scientific computing library for the Python programming language.
Article
Full-text available
Hepatocellular carcinoma (HCC) is the fourth most common cause of cancer deaths worldwide, and its early detection is a critical determinant of whether curative treatment is achievable. Early stage HCC is typically asymptomatic. Thus, screening programmes are used for cancer detection in patients at risk of tumour development. Radiological screening methods are limited by imperfect data, cost and associated risks, and additionally are unable to detect lesions until they have grown to a certain size. Therefore, some screening programmes use additional blood/serum biomarkers to help identify individuals in whom to target diagnostic cancer investigations. The GALAD score, combining the levels of several blood biomarkers, age and sex, has been developed to identify patients with early HCC. Here we propose a Bayesian hierarchical model for an individual’s longitudinal GALAD scores whilst in HCC surveillance to identify potentially significant changes in the trend of the GALAD score, indicating the development of HCC, aiming to improve early detection compared to standard methods. An absorbent two-state continuous-time hidden Markov model is developed for the individual level longitudinal data where the states correspond to the presence/absence of HCC. The model is additionally informed by the information on the diagnosis by standard clinical practice, taking into account that HCC can be present before the actual diagnosis so that there may be false negatives within the diagnosis data. We fit the model to a Japanese cohort of patients undergoing HCC surveillance and show that the detection capability of this proposal is greater than using a fixed cut-point.
Article
Full-text available
Bond rating Transition Probability Matrices (TPMs) are built over a one-year time-frame and for many practical purposes, like the assessment of risk in portfolios, one needs to compute the TPM for a smaller time interval. In the context of continuous time Markov chains (CTMC) several deterministic and statistical algorithms have been proposed to estimate the generator matrix. We focus on the Expectation-Maximization (EM) algorithm by \cite{BladtSorensen2005} for a CTMC with an absorbing state for such estimation. This work's contribution is fourfold. Firstly, we provide directly computable closed form expressions for quantities appearing in the EM algorithm. Previously, these quantities had to be estimated numerically and considerable computational speedups have been gained. Secondly, we prove convergence to a single set of parameters under reasonable conditions. Thirdly, we derive a closed-form expression for the error estimate in the EM algorithm allowing to approximate confidence intervals for the estimation. Finally, we provide a numerical benchmark of our results against other known algorithms, in particular, on several problems related to credit risk. The EM algorithm we propose, padded with the new formulas (and error criteria), is very competitive and outperforms other known algorithms in several metrics.
Article
Full-text available
We consider continuous time Markovian processes where populations of individual agents interact stochastically according to kinetic rules. Despite the increasing prominence of such models in fields ranging from biology to smart cities, Bayesian inference for such systems remains challenging, as these are continuous time, discrete state systems with potentially infinite state-space. Here we propose a novel efficient algorithm for joint state / parameter posterior sampling in population Markov Jump processes. We introduce a class of pseudo-marginal sampling algorithms based on a random truncation method which enables a principled treatment of infinite state spaces. Extensive evaluation on a number of benchmark models shows that this approach achieves considerable savings compared to state of the art methods, retaining accuracy and fast convergence. We also present results on a synthetic biology data set showing the potential for practical usefulness of our work.
Article
The estimation of the parameters of a continuous-time Markov chain from discrete-time observations, also known as the embedding problem for Markov chains, plays in particular an important role for the modeling of credit rating transitions. This missing data problem boils down to a latent variable setting and thus, maximum likelihood estimation is usually conducted using the expectation-maximization (EM) algorithm. We illustrate that the EM algorithm is likely to get stuck in local maxima of the likelihood function in this specific problem setting and adapt a stochastic approximation simulated annealing scheme (SASEM) as well as a genetic algorithm (GA) to combat this issue. Above that, our main contribution is to extend our method GA by a rejection sampling scheme, which allows one to derive stochastic monotone maximum likelihood estimates in order to obtain proper (non-crossing) multi-year probabilities of default. We advocate the use of this procedure as direct constrained optimization (of the likelihood function) will not be numerically stable due to the large number of side conditions. Furthermore, the monotonicity constraint enables one to combine structural knowledge of the ordinality of credit ratings with real-life data into a statistical estimator, which has a stabilizing effect on far off-diagonal generator matrix elements. We illustrate our methods by Standard and Poor’s credit rating data as well as a simulation study and benchmark our novel procedure against an already existing smoothing algorithm.
Article
This article introduces the R package ctmcd, which provides an implementation of methods for the estimation of the parameters of a continuous-time Markov chain given that data are only available on a discrete-time basis. This data consists of partial observations of the state of the chain, which are made without error at discrete times, an issue also known as the embedding problem for Markov chains. The functions provided comprise matrix logarithm based approximations as described in Israel et al. (2001), as well as Kreinin and Sidelnikova (2001), an expectation-maximization algorithm and a Gibbs sampling approach, both introduced by Bladt and Sørensen (2005). For the expectationmaximization algorithmWald confidence intervals based on the Fisher information estimation method of Oakes (1999) are provided. For the Gibbs sampling approach, equal-tailed credibility intervals can be obtained. In order to visualize the parameter estimates, a matrix plot function is provided. The methods described are illustrated by Standard and Poor's discrete-time corporate credit rating transition data.
Article
Scitation is the online home of leading journals and conference proceedings from AIP Publishing and AIP Member Societies