Mark H.M. Winands

Mark H.M. Winands
Maastricht University | UM · Department of Data Science & Knowledge Engineering

PhD

About

142
Publications
66,416
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
2,162
Citations
Citations since 2017
44 Research Items
1080 Citations
2017201820192020202120222023050100150200
2017201820192020202120222023050100150200
2017201820192020202120222023050100150200
2017201820192020202120222023050100150200
Additional affiliations
October 2019 - present
Maastricht University
Position
  • Professor
Description
  • Professor in Machine Reasoning
January 2017 - September 2019
Maastricht University
Position
  • Professor (Associate)
January 2009 - December 2016
Maastricht University
Position
  • Professor (Assistant)

Publications

Publications (142)
Article
Full-text available
Monte Carlo Tree Search (MCTS) is a search technique that in the last decade emerged as a major breakthrough for Artificial Intelligence applications regarding board- and video-games. In 2016, AlphaGo, an MCTS-based software agent, outperformed the human world champion of the board game Go. This game was for long considered almost infeasible for ma...
Article
In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called splitting. So far, split moves have been applied only in forem...
Preprint
Proof-Number Search (PNS) and Monte-Carlo Tree Search (MCTS) have been successfully applied for decision making in a range of games. This paper proposes a new approach called PN-MCTS that combines these two tree-search methods by incorporating the concept of proof and disproof numbers into the UCT formula of MCTS. Experimental results demonstrate t...
Preprint
Full-text available
In many games, moves consist of several decisions made by the player. These decisions can be viewed as separate moves, which is already a common practice in multi-action games for efficiency reasons. Such division of a player move into a sequence of simpler / lower level moves is called \emph{splitting}. So far, split moves have been applied only i...
Article
Monte-Carlo Tree Search (MCTS) has been applied successfully in many domains, including games. However, its performance is not uniform on all domains, and it also depends on how parameters that control the search are set. Parameter values that are optimal for a task might be sub-optimal for another. In a domain that tackles many games with differen...
Chapter
Playout Policy Adaptation (PPA) is a state-of-the-art strategy that has been proposed to control the playouts in Monte-Carlo Tree Search (MCTS). PPA has been successfully applied to many two-player, sequential-move games. This paper further evaluates this strategy in General Game Playing (GGP) by first reformulating it for simultaneous-move games....
Book
This book constitutes the refereed proceedings of the First Workshop on Monte Carlo Search, MCS 2020, organized in conjunction with IJCAI 2020. The event was supposed to take place in Yokohama, Japan, in July 2020, but due to the Covid-19 pandemic was held virtually on January 7, 2021. The 9 full papers of the specialized project were carefully r...
Preprint
Full-text available
This article proposes a method for automated service selection to improve treatment efficacy and reduce re-hospitalization costs. A predictive model is developed using the National Home and Hospice Care Survey (NHHCS) dataset to quantify the effect of care services on the risk of re-hospitalization. By taking the patient's characteristics and other...
Conference Paper
Procedural Content Generation is an active area of research, with more interest being given recently to methods able to produce interesting content in a general context (without task-specific knowledge). To this extent, we focus on procedural level generators within the General Video Game AI framework (GVGAI). This paper proposes several topics of...
Conference Paper
Full-text available
Bien que les systèmes actuels de General Game Playing (GGP) facilitent la recherche en Intelligence Artificielle (IA) autour des jeux, ils sont souvent trop spécialisés et fournissent une capacité de calcul trop faible. Cet article décrit une première version du système ludémique de GGP dénommé LUDII qui apporte un outil efficace à la recherche en...
Preprint
Full-text available
Digital Archaeoludology (DAL) is a new field of study involving the analysis and reconstruction of ancient games from incomplete descriptions and archaeological evidence using modern computational techniques. The aim is to provide digital tools and methods to help game historians and other researchers better understand traditional games, their deve...
Preprint
Full-text available
While current General Game Playing (GGP) systems facilitate useful research in Artificial Intelligence (AI) for game-playing, they are often somewhat specialized and computationally inefficient. In this paper, we describe an initial version of a "ludemic" general game system called Ludii, which has the potential to provide an efficient tool for AI...
Article
Many enhancements for Monte-Carlo Tree Search (MCTS) have been applied successfully in General Game Playing (GGP). MCTS and its enhancements are controlled by multiple parameters that require extensive and time-consuming off-line optimization. Moreover, as the played games are unknown in advance, off-line optimization cannot tune parameters specifi...
Conference Paper
Monte-Carlo Tree Search (MCTS) has been found to show weaker play than minimax-based search in some tactical game domains. In order to combine the tactical strength of minimax and the strategic strength of MCTS, MCTS-minimax hybrids have been proposed in prior work. This article continues this line of research for the case where heuristic state eva...
Article
Monte-Carlo Tree Search (MCTS) has been found to show weaker play than minimax-based search in some tactical game domains. This is partly due to its highly selective search and averaging value backups, which make it susceptible to traps. In order to combine the strategic strength of MCTS and the tactical strength of minimax, MCTS-minimax hybrids ha...
Conference Paper
Credit card transactions predicted to be fraudulent by automated detection systems are typically handed over to human experts for verification. To limit costs, it is standard practice to select only the most suspicious transactions for investigation. We claim that a trade-off between exploration and exploitation is imperative to enable adaptation t...
Article
Credit card transactions predicted to be fraudulent by automated detection systems are typically handed over to human experts for verification. To limit costs, it is standard practice to select only the most suspicious transactions for investigation. We claim that a trade-off between exploration and exploitation is imperative to enable adaptation t...
Chapter
Full-text available
Monte-Carlo Tree Search (MCTS) has shown particular success in General Game Playing (GGP) and General Video Game Playing (GVGP) and many enhancements and variants have been developed. Recently, an on-line adaptive parameter tuning mechanism for MCTS agents has been proposed that almost achieves the same performance as off-line tuning in GGP.
Chapter
Many enhancements have been proposed for Monte-Carlo Tree Search (MCTS). Some of them have been applied successfully in the context of General Game Playing (GGP). MCTS and its enhancements are usually controlled by multiple parameters that require extensive and time-consuming computation to be tuned in advance. Moreover, in GGP optimal parameter va...
Conference Paper
Full-text available
Monte-Carlo Tree Search (MCTS) has shown particular success in General Game Playing (GGP) and General Video Game Playing (GVGP) and many enhancements and variants have been developed. Recently, an on-line adap-tive parameter tuning mechanism for MCTS agents has been proposed that almost achieves the same performance as off-line tuning in GGP. In th...
Book
This book constitutes revised selected papers from the 6th Workshop on Computer Games, CGW 2017, held in conjunction with the 26th International Conference on Artificial Intelligence, IJCAI 2017, in Melbourne, Australia, in August 2017. The 12 full papers presented in this volume were carefully reviewed and selected from 18 submissions. They cover...
Article
Full-text available
This paper showcases the setting and results of the first Two-Player General Video Game AI competition, which ran in 2016 at the IEEE World Congress on Computational Intelligence and the IEEE Conference on Computational Intelligence and Games. The challenges for the general game AI agents are expanded in this track from the single-player version, l...
Chapter
Monte-Carlo Tree Search (MCTS) is a best-first search method guided by the results of Monte-Carlo simulations. It is based on randomized exploration of the search space. Using the results of previous explorations, the method gradually builds up a game tree in memory and successively becomes better at accurately estimating the values of the most pro...
Conference Paper
General Game Playing (GGP) programs need a Game Description Language (GDL) reasoner to be able to interpret the game rules and search for the best actions to play in the game. One method for interpreting the game rules consists of translating the GDL game description into an alternative representation that the player can use to reason more efficien...
Book
This book constitutes the refereed proceedings of the 5th Computer Games Workshop, CGW 2016, and the 5th Workshop on General Intelligence in Game-Playing Agents, GIGA 2016, held in conjunction with the 25th International Conference on Artificial Intelligence, IJCAI 2016, in New York, USA, in July 2016. The 12 revised full papers presented were care...
Book
This book constitutes the refereed conference proceedings of the 15th International Conference, ACG 2017, held in Leiden, The Netherlands, in July 2017.The 19 revised full papers were selected from 23 submissions and cover a wide range of computer games. They are grouped in four classes according to the order of publication: games and puzzles, go a...
Article
Monte Carlo Tree Search (MCTS) is a popular approach for tree search in a variety of games. While MCTS allows for fine-grained time control, not much has been published on time management for MCTS programs under tournament conditions. This paper first investigates the effects of various time-management strategies on playing strength in the challeng...
Article
The nine papers in this special section focus on the development of physics-based simulation video games (PBSG). The focus is on artificial intelligence for specific PBSGs competitions such as Angry Birds and computational pool, as well as on further developments of physics simulators in order to launch the next generation of PBSGs.
Conference Paper
This paper investigates Sequential Halving as a selection policy in the following four partially observable games: Go Fish, Lost Cities, Phantom Domineering, and Phantom Go. Additionally, H-MCTS is studied, which uses Sequential Halving at the root of the search tree, and UCB elsewhere. Experimental results reveal that H-MCTS performs the best in G...
Conference Paper
The board game Surakarta has been played at the ICGA Computer Olympiad since 2007. In this paper the ideas behind the agent SIA, which won the competition five times, are revealed. The paper describes its \(\alpha \beta \)-based variable-depth search mechanism. Search enhancements such as multi-cut forward pruning and Realization Probability Search...
Article
Simultaneous move games model discrete, multistage interactions where at each stage players simultaneously choose their actions. At each stage, a player does not know what action the other player will take, but otherwise knows the full state of the game. This formalism has been used to express games in general game playing and can also model many d...
Book
This book constitutes the refereed proceedings of the Fourth Computer Games Workshop, CGW 2015, and the Fourth Workshop on General Intelligence in Game-Playing Agents, GIGA 2015, held in conjunction with the 24th International Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, in July 2015. The 12 revised full papers presen...
Conference Paper
Full-text available
This paper investigates Sequential Halving as a selection policy in the following four partially observable games: Go Fish, Lost Cities, Phantom Domineering, and Phantom Go. Additionally, H-MCTS is studied, which uses Sequential Halving at the root of the search tree, and UCB elsewhere. Experimental results reveal that H-MCTS performs the best in G...
Article
Monte Carlo tree search (MCTS) is a sampling-based search algorithm that is state of the art in a variety of games. In many domains, its Monte Carlo rollouts of entire games give it a strategic advantage over traditional depth-limited minimax search with $alphabeta$ pruning. These rollouts can often detect long-term consequences of moves, freeing t...
Article
Monte-Carlo Tree Search (MCTS) is a best-first search method guided by the results of Monte-Carlo simulations. It is based on randomized exploration of the search space. Using the results of previous explorations, the method gradually builds up a game tree in memory and successively becomes better at accurately estimating the values of the most pro...
Article
The aim of general game playing (GGP) is to create programs capable of playing a wide range of different games at an expert level, given only the rules of the game. The most successful GGP programs currently employ simulation-based Monte Carlo tree search (MCTS). The performance of MCTS depends heavily on the simulation strategy used. In this paper...
Conference Paper
Monte-Carlo Tree Search (MCTS) has been found to play suboptimally in some tactical domains due to its highly selective search, focusing only on the most promising moves. In order to combine the strategic strength of MCTS and the tactical strength of minimax, MCTS-minimax hybrids have been introduced, embedding shallow minimax searches into the MCT...
Article
Full-text available
In this article, Monte-Carlo Tree Search (MCTS) is introduced for controlling the Pac-Man character in the real-time game Ms Pac-Man. MCTS is used to find an optimal path for an agent at each turn, determining the move to make based on the results of numerous randomized simulations. Several enhancements are introduced in order to adapt MCTS to the...
Conference Paper
Full-text available
Monte Carlo Tree Search (MCTS) has improved the performance of game engines in domains such as Go, Hex, and general game playing. MCTS has been shown to outperform classic alpha-beta search in games where good heuristic evaluations are difficult to obtain. In recent years, combining ideas from traditional minimax search in MCTS has been shown to be...
Conference Paper
Monte Carlo Tree Search (MCTS) is a widely-used technique for game-tree search in sequential turn-based games. The extension to simultaneous move games, where all players choose moves simultaneously each turn, is non-trivial due to the complexity of this class of games. In this paper, we describe simultaneous move MCTS and analyze its application i...
Conference Paper
Full-text available
Monte-Carlo Tree Search is a best-first search technique based on simulations to sample the state space of a decision-making problem. In games, positions are evaluated based on estimates obtained from rewards of numerous randomized play-outs. Generally, rewards from play-outs are discrete values representing the outcome of the game (loss, draw, or...
Conference Paper
Full-text available
Regret minimization is important in both the Multi-Armed Bandit problem and Monte-Carlo Tree Search (MCTS). Recently, sim-ple regret, i.e., the regret of not recommending the best action, has been proposed as an alternative to cumulative regret in MCTS, i.e., regret accumulated over time. Each type of regret is appropriate in different contexts. Al...
Book
This book constitutes the refereed proceedings of the Computer Games Workshop, CGW 2013, held in Beijing, China, in August 2013, in conjunction with the Twenty-third International Conference on Artificial Intelligence, IJCAI 2013. The 9 revised full papers presented were carefully reviewed and selected from 15 submissions. The papers cover a wide r...
Conference Paper
Full-text available
MCTS has been successfully applied to many sequential games. This paper investigates Monte Carlo Tree Search (MCTS) for the simultaneous move game Tron. In this paper we describe two different ways to model the simultaneous move game, as a standard sequential game and as a stacked matrix game. Several variants are presented to adapt MCTS to simulta...
Conference Paper
Full-text available
Best-Reply Search (BRS) is a new search technique for game-tree search in multi-player games. In BRS, the exponentially many possibilities that can be considered by opponent players is flattened so that only a single move, the best one among all opponents, is chosen. BRS has been shown to outperform the classic search techniques in several domains....
Conference Paper
Full-text available
This paper introduces Monte Carlo *-Minimax Search (MCMS), a Monte Carlo search algorithm for turned-based, stochastic, two-player, zero-sum games of perfect information. The algorithm is de-signed for the class of densely stochastic games; that is, games where one would rarely expect to sample the same successor state multiple times at any particu...
Conference Paper
Monte Carlo Tree Search (MCTS) has become a widely popular sampled-based search algorithm for two-player games with perfect information. When actions are chosen simultaneously, players may need to mix between their strategies. In this paper, we discuss the extension of MCTS to simultaneous move games with and without chance events. We introduce a n...
Conference Paper
Monte-Carlo Tree Search is a sampling-based search algorithm that has been successfully applied to a variety of games. Monte-Carlo rollouts allow it to take distant consequences of moves into account, giving it a strategic advantage in many domains over traditional depth-limited minimax search with alpha-beta pruning. However, MCTS builds a highly...
Article
This paper introduces Monte Carlo *-Minimax Search (MCMS), a Monte Carlo search algorithm for turned-based, stochastic, two-player, zero-sum games of perfect information. The algorithm is designed for the class of of densely stochastic games; that is, games where one would rarely expect to sample the same successor state multiple times at any parti...
Article
Full-text available
In this article we investigate how three multi-player search policies, namely maxn, paranoid, and Best-Reply Search, can be embedded in the MCTS framework. The performance of these search policies is tested in four different deterministic multi-player games with perfect information by running self-play experiments. We show that MCTS with the maxn s...
Article
Full-text available
This article describes how Monte-Carlo Tree Search (MCTS) can be applied to play the hide-and-seek game Scotland Yard. This game is essentially a two-player game in which the players are moving on a graph-based map. First, we discuss how determinization is applied to handle the imperfect information in the game. We show how using determinization in...
Article
Classic methods such as A∗ and IDA∗ are a popular and successful choice for one-player games. However, without an accurate admissible evaluation function, they fail. In this article we investigate whether Monte-Carlo tree search (MCTS) is an interesting alternative for one-player games where A∗ and IDA∗ methods do not perform well. Therefore, we pr...
Article
Full-text available
Solving games is a challenging and attractive task in the domain of Artificial Intelligence. Despite enormous progress, solving increasingly difficult games or game positions continues to pose hard technical challenges. Over the last twenty years, algorithms based on the concept of proof and disproof numbers have become dominating techniques for ga...
Conference Paper
Full-text available
Monte-Carlo Tree Search (MCTS) is a state-of-the-art stochastic search algorithm that has successfully been applied to various multi- and one-player games (puzzles). Beam search is a search method that only expands a limited number of promising nodes per tree level, thus restricting the space complexity of the underlying search algorithm to linear...
Conference Paper
Full-text available
In this paper enhancements for the Monte-Carlo Tree Search (MCTS) framework are investigated to play Ms Pac-Man. MCTS is used to find an optimal path for an agent at each turn, determining the move to make based on randomised simulations. Ms Pac-Man is a real-time arcade game, in which the protagonist has several independent goals but no conclusive...
Article
kEver since humans achieved some degree of civilization, they have played games. The two most important reasons for games to be played are their intellectual challenge and their entertainment value. For the rst reason games are used as a testing ground for computational intelligence. Since the 1950s the AI community compares the computer performanc...
Conference Paper
Full-text available
Monte-Carlo Tree Search (MCTS) is state of the art for online planning in large MDPs. It is a best-first, sample-based search algorithm in which every state in the search tree is evaluated by the average outcome of Monte-Carlo rollouts from that state. These rollouts are typically random or directed by a simple, domain-dependent heuristic. We propo...
Article
Full-text available
The aim of general game playing (GGP) is to create programs capable of playing a wide range of different games at an expert level, given only the rules of the game. The most successful GGP programs currently employ simulation-based Monte Carlo tree search (MCTS). The performance of MCTS depends heavily on the simulation strategy used. In this paper...
Conference Paper
Full-text available
This article shows how the performance of a Monte-Carlo Tree Search (MCTS) player for Havannah can be improved, by guiding the search in the play-out and selection steps of MCTS. To improve the play-out step of the MCTS algorithm, we used two techniques to direct the simulations, Last-Good-Reply (LGR) and N-grams. Experiments re-veal that LGR gives...
Conference Paper
Full-text available
The dominant approach for programs playing the game of Go is nowadays Monte-Carlo Tree Search (MCTS). While MCTS allows for fine-grained time control, little has been published on time man-agement for MCTS programs under tournament conditions. This paper investigates the effects that various time-management strategies have on the playing strength i...
Conference Paper
Over the past few years, Monte-Carlo Tree Search (MCTS) has become a popular search technique for playing multi-player games. In this paper we propose a technique called Playout Search. This enhancement allows the use of small searches in the playout phase of MCTS in order to improve the reliability of the playouts. We investigate maxn, Paranoid, a...