John Stachurski

• Australian National University

Recent approaches to the theory of dynamic programming view dynamic programs as families of policy operators acting on partially ordered sets. In this paper, we extend these ideas by shifting from arbitrary partially ordered sets to ordered vector space. The advantage of working in this setting is that ordered vector spaces have well integrated alg...

In the theory of dynamic programming, an optimal policy is a policy whose lifetime value dominates that of all other policies at every point in the state space. This raises a natural question: under what conditions does optimality at a single state imply optimality at every state? We show that, in a general setting, the irreducibility of the transi...

This paper integrates two strands of the literature on stability of general state Markov chains: conventional, total variation based results and more recent order-theoretic results. First we introduce a complete metric over Borel probability measures based on partial stochastic dominance. We then show that many conventional results framed in the se...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

It has become increasingly clear that economies can fruitfully be viewed as networks, consisting of millions of nodes (households, firms, banks, etc.) connected by business, social, and legal relationships. These relationships shape many outcomes that economists often measure. Over the past few years, research on production networks has flourished,...

Monetary conditions are frequently cited as a significant factor influencing fluctuations in commodity prices. However, the precise channels of transmission are less well identified. In this paper, we develop a unified theory to study the impact of interest rates on commodity prices and the underlying mechanisms. To that end, we extend the competit...

We introduce a framework that represents a dynamic program as a family of operators acting on a partially ordered set. We provide an optimality theory based only on order-theoretic assumptions and show how applications across almost all subfields of dynamic programming fit into this framework. These range from traditional dynamic programs to those...

Systems of the form $x = (A x^s)^{1/s} + b$ arise in a range of economic, financial and control problems, where $A$ is a linear operator acting on a space of real-valued functions (or vectors) and $s$ is a nonzero real value. In these applications, attention is focused on positive solutions. We provide a simple and complete characterization of exis...

This textbook is an introduction to economic networks, intended for students and researchers in the fields of economics and applied mathematics. The textbook emphasizes quantitative modeling, with the main underlying tools being graph theory, linear algebra, fixed point theory and programming. The text is suitable for a one-semester course, taught...

We propose a new approach to solving dynamic decision problems with unbounded rewards based on the transformations used in Q-learning. In our case, however, the objective of the transform is not learning. Rather, it is to convert an unbounded dynamic program into a bounded one. The approach is general enough to handle problems for which existing me...

We show that competitive equilibria in a range of models related to production networks can be recovered as solutions to dynamic programs. Although these programs fail to be contractive, we prove that they are tractable. As an illustration, we treat Coase's theory of the firm, equilibria in production chains with transaction costs, and equilibria i...

We obtain an exact necessary and sufficient condition for the existence and uniqueness of equilibrium asset prices in infinite horizon, discrete-time, arbitrage free environments. Using local spectral radius methods, we connect the condition, and hence the problem of existence and uniqueness of asset prices, with the recent literature on stochastic...

Transforming Dynamic Optimization Problems for Enhanced Computational Efficiency

This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results can be recovered. We also show that the condition cannot be significantly weakened. Our framework is general eno...

We propose a new approach to solving dynamic decision problems with unbounded rewards, based on an application of the Q-learning transform to the Bellman equation. Q-learning is a technique from the reinforcement learning literature with strong convergence properties. In our case, the objective of the transform is to convert an unbounded dynamic pr...

This note corrects the proof of Proposition 5 in Stachurski and Toda (2019), which shows that the consumption function has an explicit linear lower bound and is used to prove their main result that wealth inherits the tail behavior of income in Bewley–Huggett–Aiyagari models.

In recent years, a range of measures of “partial” stochastic dominance have been introduced. These measures attempt to determine the extent to which one distribution is dominated by another. We assess these measures from intuitive, axiomatic, computational and statistical perspectives. Our investigation leads us to recommend a measure related to op...

We analyze the household savings problem in a general setting where returns on assets, non-financial income and impatience are all state dependent and fluctuate over time. All three processes can be serially correlated and mutually dependent. Rewards can be bounded or unbounded, and wealth can be arbitrarily large. Extending classic results from an...

We propose a new approach to solving dynamic decision problems with rewards that are unbounded below. The approach involves transforming the Bellman equation in order to convert an unbounded problem into a bounded one. The major advantage is that, when the conditions stated below are satisfied, the transformed problem can be solved by iterating wit...

We show that competitive equilibria in a range of useful production chain models can be recovered as the solutions to a class of dynamic programming problems. Bringing dynamic programming to bear on the equilibrium structure of production chains adds analytical power and opens new avenues for computation. In addition, the dynamic programming proble...

This paper extends the core results of discrete time infinite horizon dynamic programming theory to the case of state-dependent discounting. The traditional constant-discount condition requires that the discount factor of the controller is strictly less than one. Here we replace the constant factor with a discount factor process and require, in ess...

It has been conjectured that canonical Bewley–Huggett–Aiyagari heterogeneous-agent models cannot explain the joint distribution of income and wealth. The results stated below verify this conjecture and clarify its implications under very general conditions. We show in particular that if (i) agents are infinitely-lived, (ii) saving is risk-free, and...

We analyze the household savings problem in a general setting where returns on assets, non-financial income and impatience are all state dependent and fluctuate over time. All three processes can be serially correlated and mutually dependent. Rewards can be bounded or unbounded and wealth can be arbitrarily large. Extending classic results from an...

In this paper we integrate two strands of the literature on stability of general state Markov chains: conventional, total-variation-based results and more recent order-theoretic results. First we introduce a complete metric over Borel probability measures based on ‘partial’ stochastic dominance. We then show that many conventional results framed in...

This paper studies the income fluctuation problem with capital income risk (i.e., dispersion in the rate of return to wealth). Wealth returns and labor earnings are allowed to be serially correlated and mutually dependent. Rewards can be bounded or unbounded. Under rather general conditions, we develop a set of new results on the existence and uniq...

This paper builds coordination costs, transaction costs, and other aspects of the theory of the firm into a production chain model with an infinite number of ex ante identical producers. The equilibrium determines prices, allocations of productive tasks across firms, firm sizes, and the number of active firms. These prices and allocations match sev...

We study a model in which income and capital flows between countries are jointly determined in a world economy with integrated financial markets. In a setting that combines risky entrepreneurial activity with moral hazard, we find that a shift from autarky to financial integration leads to boom-bust cycles in capital flows, output and consumption....

Building on insights of Jovanovic (1982) and subsequent authors, we develop a comprehensive theory of optimal timing of decisions based around continuation value functions and operators that act on them. Optimality results are provided under general settings, with bounded or unbounded reward functions. This approach has several intrinsic advantages...

Random dynamical systems encountered in economics have certain distinctive characteristics that make them particularly well suited to analysis using the tools for studying Markov processes developed by Rabi N. Bhattacharya and his coauthors over the last few decades. In this essay we discuss the significance of these tools for both mathematicians a...

In estimation and calibration studies, the convergence of time series sample averages plays a central role. At the same time, a significant number of economic models do not satisfy the classical ergodicity conditions. Motivated by existing work on asymptotics of stochastic economic models, we develop a new set of results on limits of sample moments...

In this paper we introduce a technique for perfect simulation from the stationary distribution of a standard model of industry dynamics. The method can be adapted to other, possibly non-monotone, regenerative processes found in industrial organization and other fields of economics. The algorithm we propose is a version of coupling from the past. It...

This paper studies the income fluctuation problem without imposing bounds on utility, assets, income or consumption. We prove that the Coleman operator is a contraction mapping over the natural class of candidate consumption policies when endowed with a metric that evaluates consumption differences in terms of marginal utility. We show that this me...

Consider a preordered metric space (X,d,⪯). Suppose that d(x,y)≤d(x ' ,y ' ) if x ' ⪯x⪯y⪯y ' . We say that a self-map T on X is asymptotically contractive if d(T i x,T i y)→0 as i↑∞ for all x,y∈X. We show that an order-preserving self-map T on X has a globally stable fixed point if and only if T is asymptotically contractive and there exist x,x * ∈...

This paper studies a value function iteration algorithm that can be applied to almost all stationary dynamic programming problems. Using nonexpansive function approximation and Monte Carlo integration, we develop a randomized fitted Bellman operator and a corresponding algorithm that is globally convergent with probability one. When additional rest...

Production takes time, and labor supply and profit maximization decisions that relate to current production are typically made before all shocks affecting that production have been realized. In this paper we re-examine the problem of stochastic optimal growth with aggregate risk where the timing of the model conforms to this information structure....

c1 Address correspondence to: Alain Venditti, GREQAM, 2 rue de la Charité, 13002 Marseille, France; e-mail: alain.venditti@univmed.fr.

This paper provides conditions for bounding tail probabilities in stochastic economic models in terms of their transition laws and shock distributions. Particular attention is given to conditions under which the tails of stationary equilibria have exponential decay. By way of illustration, the technique is applied to a threshold autoregression mode...

This paper formulates a model embedding the key ideas from Ronald Coase’s famous essay on the theory of the firm in a simple competitive equilibrium setting with anarbitrary number of firms. The model studies the structure of production when transaction costs and diminishing returns to management are treated as given. In addition to recovering Coas...

We introduce a goodness of fit test for ergodic Markov processes. Our test compares the data against the set of stationary densities implied by the class of models specified in the null hypothesis, and rejects if no model in the class yields a stationary density that matches with the data. No alternative needs to be specified in order to implement...

The look-ahead estimator is used to compute densities associated with Markov processes via simulation. We study a framework that extends the look-ahead estimator to a much broader range of applications. We provide a general asymptotic theory for the estimator, where both L1 consistency and L2 asymptotic normality are established.

This paper generalizes the sufficient conditions for stability of monotone economies and time series models due to Hopenhayn and Prescott (Econometrica, 60, p. 1387–1406, 1992). We introduce a new order-theoretic mixing condition and characterize stability for monotone economies satisfying this condition. We also provide a range of results that can...

In applied density estimation problems, one often has data not only on the target variable, but also on a collection of covariates. In this paper, we study a density estimator that incorporates this additional information by combining parametric estimation and conditional Monte Carlo. We prove an approximate functional asymptotic normality result t...

This paper strengthens the Hopenhayn and Prescott stability theorem for monotone economies. We extend the theorem to a larger class of applications, and develop new perspectives on the nature and causes of stability and instability. In addition, we show that models satisfying the Hopenhayn-Prescott theorem are ergodic, in the sense that sample aver...

We discuss the stability of discrete-time Markov chains satisfying monotonicity and an order-theoretic mixing condition that can be seen as an alternative to irreducibility. A chain satisfying these conditions has at most one stationary distribution. Moreover, if there is a stationary distribution, then the chain is stable in an order-theoretic sen...

Using a variation of the coupling from the past technique, this paper develops algorithms which generate independent observations from the stationary distributions of various dynamic economic models. These variates can be used for calibration, calculation of steady state phenomena, and simulation-based estimation. As an application, we demonstrate...

This note contains some technical results developed for Kamihigashi and Stachurski (2010). We first consider a stochastic kernel on an arbitrary measurable space and establish some general results. We then introduce a preorder and consider an increasing stochastic kernel. None of our results requires any topological assumption. To make this note se...

This paper presents new order-theoretic conditions for global stability of monotone Markov processes with possibly non-compact state spaces. Our main result shows that a Markov process induced by a continuous and increasing transition law is globally stable if it admits a Lyapunovlike function, and becomes larger than any given element of the state...

We provide a simple proof of geometric ergodicity for Samuelson's (1971) commodity pricing model. The proof yields a rate of convergence to the stationary distribution stated in terms of model primitives. We also provide a rate of convergence for prices to the stationary price process, and for the joint distribution of the state process to the stat...

We study a two-country version of Matsuyama's [K. Matsuyama, Financial market globalization, symmetry-breaking, and endogenous inequality of nations, Econometrica 72 (2004) 853–884] world economy model. As in Matsuyama's model, symmetry-breaking can be observed, and symmetry-breaking generates endogenously determined levels of inequality. In additi...

This paper presents a new mixing condition for dynamic economies with a Markov structure. The mixing condition is stated in terms of order, and generalizes a number of wellknown conditions used to establish stability of monotone dynamic models. By generalizing the key insights of the original conditions, we derive a set of results with applications...

This paper introduces a multisector model of commodity markets with storage, where equilibrium is defined by profit maximization, arbitrage and market clearing conditions. We then solve for the decentralized equilibrium via a corresponding dynamic program. We also describe the dynamics of the model, establishing geometric ergodicity, a Law of Large...

We propose a generalized conditional Monte Carlo technique for computing densities in economic models. Global consistency and functional asymptotic normality are established under ergodicity assumptions on the simulated process. The asymptotic normality result allows us to characterize the asymptotic distribution of the error in density space, and...

We study a Monte Carlo algorithm for computing marginal and stationary densities of stochastic models with the Markov property, establishing global asymptotic normality and O(n^(1/2)) convergence. Asymptotic normality is used to derive error bounds in terms of the distribution of the norm deviation. Copyright The Econometric Society 2008.

This paper studies fitted value iteration for continuous state numerical dynamic programming using nonexpansive function approximators. A number of approximation schemes are discussed. The main contribution is to provide error bounds for approximate optimal policies generated by the value iteration algorithm.

For Markovian economic models, long-run equilibria are typically identified with the stationary (invariant) distributions generated by the model. In this paper we provide new sufficient conditions for continuity in the map from parameters to these equilibria. Several existing results are shown to be special cases of our theorem.

This paper reviews the economic literature on the role of fees in patent systems. Two main research questions are usually addressed: the impact of patent fees on the behavior of applicants and the question of optimal fees. Studies in the former group confirm that a range of fees affect the behavior of applicants and suggest that a patent is an inel...

This paper studies a Monte Carlo algorithm for computing distributions of state variables when the underlying model is a Markov process. It is shown that the $L_1$ error of the estimator always converges to zero with probability one, and often at a parametric rate. A related technique for computing stationary distributions is also investigate

This note considers finite state Markov chains which overlap supports. While the overlapping supports condition is known to be necessary and sufficient for stability of these chains, the result is typically presented in a more general context. As such, one objective of the note is to provide an exposition, along with simple proofs corresponding to...

The stochastic optimal growth model (Brock and Mirman 1972) is a foundation stone of modern macroeconomic and econometric research. To accommodate the data, however, economists are often forced to go beyond the convex production tech- nology used in these original studies. Nonconvexities lead to technical difficulties which applied researchers woul...

This survey reviews models of self-reinforcing mechanisms that cause poverty to persist. Some of them examine market failure in environments where the neoclassical assumptions on markets and technology break down. Other mechanisms include institutional failure which can, by itself, perpetuate self-reinforcing poverty. A common thread in all these m...

The present paper studies existence, uniqueness and stability of stationary equilibrium distributions in a class of stochastic dynamic models common to economic analysis. We provide applications to a heterogeneous agent model and two nonlinear multisector time series models with unbounded state space.

The paper proposes an Euler equation technique for analyzing the stability of dier- entiable stochastic programs. The main innovation is to use marginal reward directly as a Foster-Lyapunov function. This allows us to extend known stability results for stochastic optimal growth models, both weakening hypotheses and strengthening conclusions. JEL cl...

The paper introduces a multiplicative drift condition for evaluating stochastic economic models. The drift condition is shown to permit computation of quantitative bounds for extreme event probabilities in terms of the model primitives. By way of illustration, the technique is applied to a simple threshold autoregression model of exchange rates.

This note considers finite state Markov chains which overlap supports. While the overlapping supports condition is known to be necessary and sufficient for stability of these chains, the result is typically presented in a more general context. As such, one objective of the note is to provide an exposition, along with simple proofs corresponding to...

We consider discrete time Markov chains on general state space. It is shown that a certain property referred to here as nondecomposability is equivalent to irreducibility and that a Markov chain with invariant distribution is irreducible if and only if the invariant distribution is unique and assigns positive probability to all absorbing sets.

This short note studies formally the common practice of log-linearizing stochastic economic models. We make precise the conditions under which stability of the original model can be inferred from that of the linearized model. A transformation to recover the stochastic equilibrium of the former from that of the latter is provided.

This paper proposes and implements a method to predict the evolution of the crosscountry income distribution using a stochastic parameterization of the Azariadis-- Drazen (1990) nonconvex growth model. We estimate the dynamic structure of that model from data in the Penn World Tables, and define inductively all future distributions as a sequence in...

The paper gives conditions under which stationary distributions of Markov models depend continuously on the parameters.

The standard one-sector stochastic optimal growth model is shown to be not just ergodic but geometrically ergodic. In addition, it is proved that the time series generated by the optimal path satisfy the Law of Large Numbers and the Central Limit Theorem.

It has been shown that long-run optimality of the limit of discounted optima when the discount rate vanishes is implied by a condition on the value function of the optimal program. We suggest a new method to verify this condition in the context of one-sector optimal growth. The idea should be more widely applicable.

no abstract given.

For many years the trend in macroeconomics has been towards models which are both explicitly stochastic and explicitly dynamic. With these models, researchers seek to replicate and explain observable properties of the major economic time series. One manifestation of this trend towards stochastic dynamic modeling has been increasing use of the inher...

The paper demonstrates global stability in a class of stochastic overlapping generations economies with increasing returns. These results are applied to the study of path dependent dynamics. In particular, for nonlinear stochastic models it is seen that persistence of the historical state and formal ergodicity may easily coincide. A new definition...

The paper considers random economic systems generating nonlinear time series on the positive half-ray . Using Lyapunov techniques, new conditions for existence, uniqueness and stability of stationary equilibria are obtained. The conditions generalize earlier results from the mathematical literature, and extend to models outside the scope of existin...

The paper demonstrates global stability in a class of stochastic overlapping generations economies with increasing returns. These results are applied to the study of path dependent dynamics. In particular, for nonlinear stochastic models it is seen that persistence of the historical state and formal ergodicity may easily coincide. A new definition...

The paper considers random economic systems generating nonlinear time series on the positive half-ray . Using Lyapunov techniques, new conditions for existence, uniqueness and stability of stationary equilibria are obtained. The conditions generalize earlier results from the mathematical literature, and extend to models outside the scope of existin...

This note studies conditions under which sequences of state variables generated by discrete-time stochastic optimal accumulation models have law of large numbers and central limit properties. Productivity shocks with unbounded support are considered. Instead of restrictions on the support of the shock, an “average contraction” property is required...

This short note studies formally the common practice of log-linearizing stochastic economic models. We make precise the conditions under which stability of the original model can be inferred from that of the linearized model. A transformation to recover the stochastic equilibrium of the former from that of the latter is provided.

Exxon Mobil and ConocoPhillips stock price has been predicted using the difference between core and headline CPI in the United States. Linear trends in the CPI difference allow accurate prediction of the prices at a five to ten-year horizon.

The paper considers random economic systems generating nonlinear time series on the positive half-ray R+. Using Liapunov techniques, new conditions for existence, uniqueness and stability of stationary equilibria are obtained. The conditions generalize earlier results from the mathematical literature, and extend to models outside the scope of exist...

This paper considers a neoclassical optimal growth problem where the shock that perturbs the economy in each time period is potentially unbounded on the state space. Sufficient conditions for existence, uniqueness, and stability of equilibria are derived in terms of the primitives of the model using recent techniques from the field of perturbed dyn...

This note studies conditions under which sequences of capital per head generated by stochastic optimal accumulation models have law of large numbers and central limit properties. The regularity condition used on the productivity shock is somewhat different to that of previous studies. In particular, no restrictions are placed on its support. Instea...

This paper establishes global stability for a class of stochastic increasing returns accumulation models. The nature of the unique stochastic steady state is investigated. It is found that the models generate highly path dependent time series over long horizons. The findings demonstrate that the standard stability concept used in stochastic growth...

We analyze the exploitation of an antibiotic in a market subject to open access on the part of antibiotic producers to the common pool of antibiotic efficacy. While the market equilibrium depends only on current levels of antibiotic efficacy and infection of the epidemiological system, the social optimum accounts for the dynamic externalities which...

This text provides an introduction to the modern theory of economic dynamics, with emphasis on mathematical and computational techniques for modeling dynamic systems. Written to be both rigorous and engaging, the book shows how sound understanding of the underlying theory leads to effective algorithms for solving real world problems. The material m...

are the computation of the density of the capital stock in the neoclassical growth model and the computation of the wealth density in an incomplete market economy.