Martin L. Puterman

• Bachelor of Arts;Masters of Science; PhD
• Professor Emeritus at University of British Columbia

For decades, National Football League (NFL) coaches' observed fourth down decisions have been largely inconsistent with prescriptions based on statistical models. In this paper, we develop a framework to explain this discrepancy using a novel inverse optimization approach. We model the fourth down decision and the subsequent sequence of plays in a...

A Theoretical Foundation for Value Functions in Sports

Life‐history theory postulates that physiological traits, such as energy metabolism, can be understood in terms of allocation trade‐offs between self‐maintenance and reproduction over an organism's life span, and data show that metabolic intensity and survival vary inversely with latitude, with tropical birds exhibiting a “slow” pace of life relati...

Supporting information for our publication in Functional Ecology: Survival is negatively related to basal metabolic rate in tropical Andean birds. Data include: Figures 1) Map of study area 2) Seasonal variation in BMR 3) Phylogeny of species used in the analysis 4) Comparison of survival rates with other studies Tables 1) Net hours 2) Results of...

As a consequence of a growing demand for chemotherapy, it has become increasingly important for cancer care providers to make informed process configuration, capacity planning and patient scheduling decisions in order to use medical resources efficiently and ensure that patients receive timely access to care. To support this decision-making process...

This paper describes a logistics optimisation case study for a Canadian beverage manufacturer and distributor. The goal was to determine production, distribution and inventory plans for a given product line to help the company with its challenges due to production shortages, stock-outs and high transportation costs in a new and highly competitive m...

Motivated by practices and issues at the British Columbia Cancer Agency (BCCA), we develop queuing network models to determine the appropriate number of patients to be managed by a single physician. This is often referred to as a physician’s panel size. The key features that distinguish our study of oncology practices from other panel size models a...

This speed talk examines (quickly) the relationships between apparent survival, energy metabolism, and the environment.

In amateur golf, lower handicap players “give strokes” to higher handicap players based on their handicap differential to make head-to-head matches fairer. In match play, the standard way to allocate handicap strokes uses the “course-defined hole ranking”. Using a bootstrapped simulation of over 70,000 matches based on 392 rounds of golf, we first...

We show that relative priorities can reduce queueing costs in systems that are multi-server and multi-class as long as customers choose their routing policy strategically. This is demonstrated in two models with multi-class Poisson arrivals and parallel memoryless servers with linear cost functions of class mean waiting times. For each model we inv...

This chapter describes the use of the linear programming approach to approximate dynamic programming as a means of solving advance patient appointment scheduling problems, which are problems typically intractable using standard solution techniques. Starting from the linear programming approach to discounted infinite-horizon Markov decision processe...

We present an approximate dynamic programming method based on simulation, policy iteration, a postdecision state formulation, and a logistic value function approximation. This method was developed as part of our efforts to determine whether nonlinear value function approximations could provide cost-effective policies for advance patient scheduling...

To ensure that patients receive timely access to care, it has become increasingly important to use existing care provider capacity as efficiently as possible and to make informed capacity planning decisions. To support this decision-making process at a regional cancer center in British Columbia (Canada), we undertook a simulation and optimization b...

To model and quantify the relationship between radiation therapy (RT) use and travel time to RT services. Population-based registries and databases were used to identify both incident cancer patient and patients receiving RT within 1 year of diagnosis (RT1y) in British Columbia, Canada, between 1992 and 2011. The effects of age, gender, diagnosis y...

We consider the search for a target whose precise location is uncertain. The search region is divided into grid cells, and the searcher decides which cell to visit next and whether to search it quickly or slowly. A quick search of a cell containing the target may damage it, resulting in a failed search, or it may locate the target safely. If the ta...

This paper describes the appointment scheduling game (ASG), an easy to use teaching tool that reveals the challenges in managing advance patient scheduling systems, and also provides an introduction to simulation and decision analysis. In addition to describing the game, the paper provides recommendations on how to play it, student questions and su...

We study a scheduling problem in which arriving patients require appointments at specific future days within a treatment specific time window. This research is motivated by a study of chemotherapy scheduling practices at the British Columbia Cancer Agency (Canada). We formulate this problem as a Markov Decision Process (MDP). Since the resulting MD...

L ean methods originated with the Toyota Production System developed shortly after World War II (Ohno 1988). Recently, they are finding widespread usage in healthcare. In industry, the primary objective of Lean is to enhance corporate earnings; but in healthcare, Lean seeks to improve patient-centred care and system efficiency (Institute of Medicin...

This paper describes a refined methodology for determining long-term care (LTC) capacity levels over a multi-year planning horizon based on a previous study. The problem is to find a capacity level in each year during the planning horizon to meet a wait time service level criterion. Instead of a static policy for capacity planning, we proposal an a...

This chapter discusses the use of analytical approaches for residential long-term care (LTC) capacity planning. The recommended method integrates demographic and survival analysis, discrete event simulation, and optimization. Through a case study based in British Columbia, Canada, it illustrates results of using this approach. Further, it discusses...

Clinical complexity, scheduling restrictions, and outdated manual booking processes resulted in frequent clerical rework, long waitlists for treatment, and late appointment notification for patients at a chemotherapy clinic in a large cancer center in British Columbia, Canada. A 17-month study was conducted to address booking, scheduling and worklo...

Seeking to reduce the potential impact of delays on radiation therapy cancer patients such as psychological distress, deterioration in quality of life and decreased cancer control and survival, and motivated by inefficiencies in the use of expensive resources, we undertook a study of scheduling practices at the British Columbia Cancer Agency (BCCA)...

This paper discusses a statistical model regarding intermediate price transitions of online auctions. The objective was to characterize the stochastic process by which prices of online auctions evolve and to estimate conditional intermediate price transition probabilities given current price, elapsed auction time, number of competing auctions, and...

In this article, we analyze the problem of selling through channels where the sales price and hence revenue is uncertain. Examples of such channels include real estate listings, traditional and online auctions, name-your-own-price channels and price guarantees. More specifically, we consider a seller, with a fixed inventory, who has to decide when...

This paper uses observations from two British Columbia studies to illustrate the shortcomings of widely used ratio-based approaches for residential long-term care capacity planning. It shows that capacity plans based on a fixed ratio of beds per population over age 75 may result in either excess capacity or long wait times for admission. It then in...

This paper describes a methodology for setting long-term care capacity levels over a multi-year planning horizon, to achieve target wait time service levels. Our approach integrates demographic and survival analysis, discrete event simulation, and optimization. Based on this methodology, a decision support system was developed for use in practice....

We present a novel approach to model individual disease progression of prostate cancer patients who receive hormone therapy before radiation therapy. Our model is used to decide when to initiate radiation therapy based on the patient's prostate specific antigen (PSA) dynamics. Each patient's PSA dynamics is modeled by a log quadratic curve. Prior d...

High surgical bed occupancy levels often result in heightened staff stress, frequent surgical cancellations, and long surgical wait times. This congestion is in part attributable to surgical scheduling practices, which often focus on the efficient use of operating rooms but ignore resulting downstream bed utilization. This paper describes a transpa...

This paper considers a mathematical programming model for the management of a salmon fishery and watershed. Management objectives are specified as bounds and desirable target levels for certain important variables. The mathematical program consists of minimizing the deviation from these targets subject to natural and policy constraints. A detailed...

This paper investigates how the structure of a relegation-promotion system impacts the competitiveness of a sports league. It proposes a rigorous mathematical model of a multi-division hierarchical sports league made up of teams with intrinsic skill levels (ISLs) that change from year to year. Since team skill changes over time, modification in div...

This article is the first of a 2-part series reporting the results of a 7-month study of porter operations at Vancouver General Hospital, Vancouver, British Columbia, Canada. Part 1 describes the importance of efficient porter services, the system's operation at the time of the study, the challenges faced in carrying out the study, the performance...

This article is the second of a 2-part series on a study of porter operations at Vancouver General Hospital, Vancouver, British Columbia, Canada. Part 1 describes the importance of efficient porter services, the system's operation at the time of the study, the challenges faced in carrying out the study, the performance measures developed, the recom...

We consider an ambulatory care unit (ACU) in a large cancer centre, where operational and resource utilization challenges led to overcrowding, excessive delays, and concerns regarding safety of critical patient care duties. We use simulation to analyze the simultaneous impact of operations, scheduling, and resource allocation on patient wait time,...

This paper describes a linear programming hierarchical planning model that determines the optimal number of nurses to train, promote to management and recruit over a 20 year planning horizon to achieve specified workforce levels. Age dynamics and attrition rates of the nursing workforce are key model components. The model was developed to help poli...

This paper uses K-means cluster analysis and multinomial mixture models to categorize professional golfer performance for the period 1980 to 2006. We collected, cleaned and analyzed final annual money list standings for PGA Tour players from www.pgatour.com. Correlation patterns between other measures suggested that career performance was well desc...

We present the case of an ambulatory clinic in which an operational review was conducted to identify opportunities for efficiency in appointment scheduling and capacity allocation. We required process data to compare that which was planned to that which actually happened and to develop advanced analytical models. Similar to other health care studie...

Operations research (OR) is playing an increasing role in the support of many health care initiatives. However one of the main challenges facing OR practitioners is the availability and the integrity of operations data. Hospital information systems (HIS) are often designed with a clinical or accounting focus and may lack the data necessary for oper...

This paper describes a periodic review, fixed lead time, single-product, single-facility model with random demand, lost sales and service constraints that was developed for potential application at a Western Canadian retailer. The objective of this study was to determine optimal (s, S) policies for a large number of products and locations. To this...

We present a method to dynamically schedule patients with different priorities to a diagnostic facility in a public health-care setting. Rather than maximizing revenue, the challenge facing the resource manager is to dynamically allocate available capacity to incoming demand to achieve wait-time targets in a cost-effective manner. We model the sche...

The authors explore the power and flexibility of using an operations research methodology known as linear programming to support health human resources (HHR) planning. The model takes as input estimates of the future need for healthcare providers and, in contrast to simulation, compares all feasible strategies to identify a long-term plan for achie...

Average Reward Optimality Equations: Multichain Models Policy Iteration for Multichain Models Linear Programming in Multichain Models Value Iteration Communicating Models

Model Classification and General Results Positive Bounded Models Negative Models Comparison of Positive and Negative Models

The Value of a Policy The Expected Total Reward Criterion The Expected Total Discounted Reward Criterion Optimality Criteria Markov Policies Vector Notation for Markov Decision Processes

Optimality Criteria Finite-Horizon Policy Evaluation Optimality Equations and the Principle of Optimality Optimality of Deterministic Markov Policies Backward Induction Examples Optimality of Monotone Policies

Existence of Optimal Policies Optimality Equations Policy Iteration The Expected Total-Reward Criterion Revisited

Policy Evaluation Optimality Equations Value Iteration and Its Variants Policy Iteration Modified Policy Iteration Spans, Bounds, Stopping Criteria, and Relative Value Iteration Action Elimination Procedures Convergence of Policies, Turnpikes and Planning Horizons Linear Programming Countable-State Models The Optimality of Structured Policies

Optimality Criteria Markov Reward Processes and Evaluation Equations Classification of Markov Decision Processes The Average Reward Optimality Equation-Unichain Models Value Iteration in Unichain Models Policy lteration in Unichain Models Modified Policy Iteration in Unichain Models Linear Programming in Unichain Models State Action Frequencies, Co...

Model Formulation Applications Discounted Models Average Reward Models Continuous-Time Markov Decision Processes

Problem Definition and Notation A One-Period Markov Decision Problem Technical Considerations

Two-State Markov Decision Process Single-Product Stochastic Inventory Control Deterministic Dynamic Programs Optimal Stopping Controlled Discrete-Time Dynamic Systems Bandit Models Discrete-Time Queueing Systems

Widespread public demand for improved access, political pressure for shorter wait times, a stretched workforce, an aging population and overutilized equipment and facilities challenge healthcare leaders to adopt new management approaches. This paper highlights the significant benefits that can be achieved by applying operations research (OR) method...

We outline the history, significance, and impact of four important contributions by Canadian researchers to stochastic modeling for operational research: the use of the uniformization method to compute transient probabilities for Markov chains, pioneered by Winfried K. Grassmann, contributions to Markov decision processes by Martin L. Puterman, con...

In this paper we analyze policies for optimally disposing inventory using online auctions. We assume a seller has a fixed number of items to sell using a sequence of, possibly overlapping, single-item auctions. The decision the seller must make is when to start each auction. The decision involves a trade-off between a holding cost for each period a...

We present a method to increase the utilization of and reduce the waiting times for an under-capacitated diagnostic resource in the presence of uncertain demand with several priority levels. We consider the case of a computed tomography (CT) scanning department that services both high-priority in-patients and lower priority outpatients. Current pra...

This paper considers the problem of setting prices dynamically to maximise expected revenues in a finite horizon model in which the demand parameters are unknown. At each decision epoch, the manager chooses a price and observes a binary response (buy or not) for each consumer visiting the website during that period. This paper focuses on comparing...

This paper considers the problem of changing prices over time to maximize expectedrevenues in the presence of unknown demand distribution parameters. It providesand compares several methods that use the sequence of past prices and observeddemands to set price in the current period. A Taylor series expansion of the futurereward function explicitly i...