Chaaben Kouki’s research while affiliated with ESSCA School of Management and other places

This study conducts a comprehensive analysis of deep reinforcement learning (DRL) algorithms applied to supply chain inventory management (SCIM), which consists of a sequential decision-making problem focused on determining the optimal quantity of products to produce and ship across multiple capacitated local warehouses over a specific time horizon. In detail, we formulate the problem as a Markov decision process for a divergent two-echelon inventory control system characterized by stochastic and seasonal demand, also presenting a balanced allocation rule designed to prevent backorders in the first echelon. Through numerical experiments, we evaluate the performance of state-of-the-art DRL algorithms and static inventory policies in terms of both cost minimization and training time while varying the number of local warehouses and product types and the length of replenishment lead times. Our results reveal that the Proximal Policy Optimization algorithm consistently outperforms other algorithms across all experiments, proving to be a robust method for tackling the SCIM problem. Furthermore, the (s, Q)-policy stands as a solid alternative, offering a compromise in performance and computational efficiency. Lastly, this study presents an open-source software library that provides a customizable simulation environment for addressing the SCIM problem, utilizing a wide range of DRL algorithms and static inventory policies.

Inventory systems are largely analyzed in the literature under the common assumption of backorders due to the complexity of lost sales. In this paper, we consider a two-echelon inventory system composed of a central warehouse and multiple local warehouses subject to lost sales. The demand faced by each local warehouse is a Poisson process and the stock in the warehouses is controlled according to a continuous review base-stock policy. This system has been analyzed in the literature under deterministic or exponential lead-times at the central warehouse, deterministic lead times at the local warehouses and approximate performance evaluations have been proposed for two cases: (1) the demand is lost if no items are available in the local warehouse, the central warehouse, or in the pipeline in between (i.e., a waiting time threshold for incoming demand equal to the local warehouse lead time), and (2) when there is a waiting time threshold less than the local warehouse lead time. Based on a queuing network representation of the system, we extend the performance analysis of the system in the first case by considering generally distributed lead times both at the central and local warehouses and by providing the exact closed-form expressions for the inventory performance measures. In the second case, we provide new approximate solutions under generally distributed lead times at the central warehouse. We numerically show that our exact and approximate solutions perform equally or better than those presented in the literature under deterministic lead times.

We study a repairable inventory system dedicated to a single component that is critical in operating a capital good. The system consists of a stock point containing spare components, and a dedicated repair shop responsible for repairing damaged components. Components are replaced using an age-replacement strategy, which sends components to the repair shop either preventively if it reaches the age-threshold, and correctively otherwise. Damaged components are replaced by new ones if there are spare components available, otherwise the capital good is inoperable. If there is free capacity in the repair shop, then the repair of the damaged component immediately starts, otherwise it is queued. The manager decides on the number of repairables in the system, the age-threshold, and the capacity of the repair shop. There is an inherent trade-off: A low (high) age-threshold reduces (increases) the probability of a corrective replacement but increases (decreases) the demand for repair capacity, and a high (low) number of repairables in the system leads to higher (lower) holding costs, but decreases (increases) the probability of downtime. We first show that the single capital good setting can be modelled as a closed queuing network with finite population, which we show is equivalent to a single queue with fixed capacity and state-dependent arrivals. For this queue, we derive closed-form expressions for the steady-state distribution. We subsequently use these results to approximate performance measures for the setting with multiple capital goods.

We considered two periodic review perishable inventory systems. The first system is controlled by a periodic review (T,S) policy, where orders are triggered at each review period T and when the stock is less than S. The second one is controlled by a periodic review (s,S) policy, where orders are triggered only if the stock level is below s. For both models, the lifetime of each product was assumed to follow a general distribution, whereas the lead time was constant. The demands followed a Poisson process, and all unsatisfied demand was lost. Using an approximate solution of the steady-state probabilities of the inventory level, we obtained a closed form of cost expression. Next, we conducted a simulation study to identify our solution accuracy. Finally, we performed a sensitivity analysis to evaluate the effect of the lifetime variation and cost parameters on the best policy and its cost. The obtained results were close to those of the optimal policy with an average accuracy of 3.7% for the periodic review (T,S) policy and 3.8% for the periodic review (s,S) policy. For the worst cases, where the accuracy is higher than 7%, we provided an analytical approach to improve the system accuracy.

We study a perishable inventory system controlled by a (T, S) periodic review order up to level policy. Items have a general lifetime distribution and the lead time is assumed to be constant. The demands arrive according to a Poisson process with rate λ. The customer is impatient. If he is not served, he will leave the queuing systems. Using the approximate solution of the steady-state probabilities, we are able to obtain the cost components’ analytical expression. Next, we compare the analytical results of our model with the simulation results. Finally, we perform a sensitivity analysis to evaluate the effect of the lifetime variation and cost parameters on the optimal cost and stock level S. Our model is an extension of an exponential distribution case. With the general lifetime distribution, we are able to have total flexibility for setting the lifetime variability. This approximation is closer to reality, especially in random environment conditions. Obtained results, from the analytical approximation model, are close to those of the optimal policy with an average accuracy of 7%.

We consider a spare parts inventory system controlled by a base-stock policy. We assume several demand classes, where the demands of each class follow a compound Poisson process, and a stochastic replenishment lead-time common to all classes. For this inventory system, we investigate two rationing policies that can be employed when the inventory level is critically low. The first policy is the reservation policy, where a part of the stock is reserved for high priority demand classes, whereas the second one is new, which we call a threshold policy, and accepts demands of a given class until the pipeline stock caused by the particular class (plus the amount of the incoming order) reaches a critical level. On average, for 8% (12%) of all instances, the threshold policy performs better than the reservation policy under the full (partial) lost sales case and when the objective is to minimise the average on-hand inventory subject to fill rate service requirements. Furthermore, in many cases where the reservation policy outperforms the threshold policy, the gap between the two policies is small under both service requirements and total cost minimisation criteria. This suggests that a trade-off between ‘fairness’ and ‘cost’ is required, and that when the performance of two policies are similar, the threshold policy is generally favoured.

A base-stock inventory system for perishables with Markovian demand and general lead-time and lifetime distributions is investigated. Using a queueing network model, we derive explicit expressions of the stationary distribution of the inventory state together with the total expected cost in a base-stock system with lost-sales. Next, we show some monotonicity properties of the cost function and propose a procedure to derive the optimal base-stock. Extensions to the backorders case and to a dual-sourcing system with multiple warehouses are also provided. Our results generalize existing ones from the literature where the lifetime and the lead-time follow either deterministic or exponential distributions. Finally, we investigate the effect of the lifetime and lead-time variability on the system cost and the optimal base-stock level. In particular, we show the substantial errors made when assuming deterministic or exponential distributions for the lifetime. This further shows the need to have results beyond exponential or deterministic assumptions.

We consider inventory systems for perishables with two-supply modes (a regular mode and an emergency one) characterized by different lead-times and costs. We investigate the value of dual-sourcing in the context of perishable items with a fixed or exponential lifetime. The inventory is controlled according to a base-stock policy where the emergency source is only used when the stock level is below a threshold level and by taking into account the age of pipeline orders. By using the information on the age of orders, which nowadays can be obtained in real time through technologies such as Radio Frequency Identification, we derive the steady-state behavior of the operating characteristics and thereby obtain the exact optimal expected total cost. Our results indicate that dual-sourcing can lead to substantial cost savings depending on the lifetime distribution. Under a fixed (exponential) lifetime, the shorter (the longer) the mean lifetime is, the higher the cost reduction, due to the reduced effect of the perishability when using dual sourcing. We also show that dual-sourcing exhibits higher savings when the excess demand is backordered as compared to the Case of lost sales. We show that the savings increase when the shortage cost and the outdating cost increase.

The continuous review base-stock policy under the lost sales assumption has been extensively studied in the inventory literature. However, most of the research that deals with the optimal policy parameter determination under compound Poisson demand focuses on the stuttering Poisson case. In this paper, we extend earlier research by providing a method that calculates the best cost-driven base-stock policy under more general compound Poisson demand processes and for both the complete and the partial rejection case. This is done by developing a simple recursive formula that computes the steady-state probabilities of the number of outstanding orders for general demand and lead-time distributions. Based on this, we are able to obtain the optimal base-stock level under the complete rejection case and an approximate base-stock level under the partial rejection policy. Through a numerical investigation, we show that our approach outperforms existing methods with regard to computational requirements.