Sean Luke's research while affiliated with George Mason University and other places

Publications (129)

Chapter
Disruption to supply chains can significantly influence the operation of the world economy and this has been shown to permeate and affect a large majority of countries and their citizens. We present initial results from a model that explores the disruptions to supply chains by a criminal agent and possible mitigation strategies. We construct a mode...
Chapter
Robot swarms hold great potential for accomplishing missions in a robust, scalable, and flexible manner. However, determining what low-level agent behavior to implement in order to meet high-level objectives is an unsolved inverse problem. Building on previous work on partially-centralized planner-guided robot swarms, we present an approach that ac...
Chapter
Robot swarms hold great potential for accomplishing missions in a robust, scalable, and flexible manner. However, determining what low-level agent behavior to implement in order to meet high-level objectives is an unsolved inverse problem. Building on previous work on partially-centralized planner-guided robot swarms, we present an approach that ac...
Chapter
Robot swarms have many virtues for large-scale task execution: this includes redundancy, a high degree of parallel task implementation, and the potential to jointly complete jobs that a single agent could not do. But because of their distributed nature, robot swarms face challenges in large-scale coordination, task serialization or ordering, and sy...
Chapter
Agent-based modeling (ABM) has many applications in the social sciences, biology, computer science, and robotics. One of the most important and challenging phases in agent-based model development is the calibration of model parameters and agent behaviors. Unfortunately, for many models this step is done by hand in an ad-hoc manner or is ignored ent...
Chapter
We present a method of supervised learning from demonstration for real-time, online training of complex heterogenous multiagent behaviors which scale to large numbers of agents in operation. Our learning method is applicable in domains where coordinated behaviors must be created quickly in unexplored environments. Examples of such problem domains i...
Conference Paper
ECJ is now 20 years old. Begun as a genetic programming and evolutionary computation library in Java, it has since established itself as historically one of the most popular EC toolkits worldwide. In 2016 we received a National Science Foundation grant to improve ECJ in many ways with an eye toward making it a useful toolkit not just for EC but for...
Chapter
Full-text available
MASON is a widely-used open-source agent-based simulation toolkit that has been in constant development since 2002. MASON’s architecture was cutting-edge for its time, but advances in computer technology now offer new opportunities for the ABM community to scale models and apply new modeling techniques. We are extending MASON to provide these oppor...
Chapter
Edisyn is a music synthesizer program (or “patch”) editor library which enables musicians to easily edit and manipulate a variety of difficult-to-program synthesizers. Edisyn sports a first-in-class set of tools designed to help explore the parameterized space of synthesizer patches without needing to directly edit the parameters. This paper discus...
Preprint
We explore Deep Reinforcement Learning in a parameterized action space. Specifically, we investigate how to achieve sample-efficient end-to-end training in these tasks. We propose a new compact architecture for the tasks where the parameter policy is conditioned on the output of the discrete action policy. We also propose two new methods based on t...
Conference Paper
Full-text available
MASON is a widely-used open-source agent-based simulation toolkit that has been in constant development since 2002. MASON's architecture was cutting-edge for its time, but advances in computer technology now offer new opportunities for the ABM community to scale models and apply new modeling techniques. We are extending MASON to provide these oppor...
Preprint
Full-text available
Policy gradient methods are often applied to reinforcement learning in continuous multiagent games. These methods perform local search in the joint-action space, and as we show, they are susceptable to a game-theoretic pathology known as relative overgeneralization. To resolve this issue, we propose Multiagent Soft Q-learning, which can be seen as...
Conference Paper
ECJ is a mature and widely used evolutionary computation library with particular strengths in genetic programming, massive distributed computation, and coevolution. In Fall of 2016 we received a three-year NSF grant to expand ECJ into a toolkit with wide-ranging facilities designed to serve the broader metaheuristics community. This report discusse...
Conference Paper
Full-text available
Current climate change causes significant biophysical effects in the Northern Hemisphere, especially in the Boreal and Arctic regions. Rising and more variable temperatures, permafrost thawing, and snow loading are major hazards affecting human societies on multiple spatial, temporal, and risk-related scales. The MASON NorthLands computational simu...
Conference Paper
Full-text available
Current climate change causes significant biophysical e↵ects in the Northern Hemisphere, especially in the Boreal and Arctic regions. Rising and more variable temperatures, permafrost thawing, and snow loading are major hazards a↵ecting human societies on multiple spatial, temporal, and risk-related scales. The MASON NorthLands computational simula...
Conference Paper
We describe our previous and current efforts towards achieving an unusual personal RoboCup goal: to train a full team of robots directly through demonstration, on the field of play at the RoboCup venue, how to collaboratively play soccer, and then use this trained team in the competition itself. Using our method, HiTAB, we can train teams of collab...
Conference Paper
When doing learning from demonstration, it is often the case that the demonstrator provides corrective examples to fix errant behavior by the agent or robot. We present a set of algorithms which use this corrective data to identify and remove noisy examples in datasets which caused errant classifications, and ultimately errant behavior. The objecti...
Conference Paper
Meta-evolutionary algorithms have long been proposed as an approach to automatically discover good parameter settings to use in later optimization runs. In this paper we instead ask whether a meta-evolutionary algorithm makes sense as an optimizer in its own right. That is, we're not interested in the resulting parameter settings, but only in the f...
Technical Report
Full-text available
MASON is an open source multiagent simulation library geared towards simulating very large numbers of relatively lightweight interacting agents. MASON has been used for a wide variety of simulation tasks in robotics, the social sciences, biology, and animation. On June 15 and 16, 2013, approximately two dozen invitees convened at George Mason Unive...
Data
Full-text available
MASON is an open source multiagent simulation library geared towards simulating very large numbers of relatively lightweight interacting agents. MASON has been used for a wide variety of simulation tasks in robotics, the social sciences, biology, and animation. On June 15 and 16, 2013, approximately two dozen invitees convened at George Mason Univ...
Article
Full-text available
We present the results of a community survey regarding genetic programming (GP) benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a blacklist o...
Article
Full-text available
Training robot or agent behaviors by example is an attractive alternative to directly coding them. However training complex behaviors can be challenging, particularly when it involves interactive behaviors involving multiple agents. We present a novel hierarchical learning from demonstration system which can be used to train both single-agent and s...
Conference Paper
Full-text available
Co-clustering is a machine learning task where the goal is to simultaneously develop clusters of the data and of their respective features. We address the use of co-clustering ensembles to establish a consensus co-clustering over the data. In this paper we develop a new preference-based multiobjective optimization algorithm to compete with a previo...
Conference Paper
Full-text available
Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternati...
Article
Full-text available
We present a supervised learning from demonstra-tion system capable of training stateful and recurrent behaviors, both in the single agent and multiagent case. Furthermore, behavior complexity due to state-fulness and multiple agents can result in a high di-mensional learning space, which can require many samples to learn properly. Our approach, wh...
Conference Paper
The synergy between climate change and social dynamics is a major challenge at the intersection of Earth science
Article
Full-text available
We present a supervised learning from demonstration system capable of training stateful and recurrent collective behaviors for multiple agents or robots. A model space of this kind is often high-dimensional and consequently may require a large number of samples to learn. Furthermore, the inverse prob-lem posed by emergent macrophenomena among multi...
Conference Paper
Full-text available
Cooperative Coevolutionary Algorithms (CCEAs) and Univariate Estimation of Distribution Algorithms (Univariate ED As) are closely related algorithms in that both update marginal distributions/populations, and test samples of those distributions/populations by grouping them with collaborators drawn from elsewhere to form a complete solution. Thus th...
Conference Paper
Full-text available
We present a study of cooperative coevolution applied to moderately complex optimization problems in large-population environments. The study asks three questions. First: what collaboration methods perform best, and when? Second: how many subpopulations are desirable? Third: is it worthwhile to do more than one trial per fitness evaluation? We disc...
Article
Full-text available
Developing robot behaviors is a tedious task requiring multiple coding, trial, and debugging cycles. This makes attractive the notion of learning from demonstration, whereby a robot learns behaviors in real time from the examples of a demonstrator. Learning from demonstra-tion can be problematic, however, because of the number of trials necessary t...
Article
Full-text available
How does one repeatedly choose actions so as to be fairest to the multiple beneficiaries of those actions? We examine approaches to discovering sequences of actions for which the worst-off beneficiaries are treated maximally well, then secondarily the second-worst-off, and so on. We formulate the problem for the situation where the sequence of acti...
Conference Paper
Full-text available
A classic example of multiagent coordination in a shared environment involves the use of pheromone deposits as a communication mechanism. Due to physical limitations in deploying actual pheromones, we propose a sparse representation of the pheromones using movable beacons. There is no communication between the beacons to propagate pheromones; inste...
Article
Full-text available
Programming robot or virtual agent behaviors can be a chal-lenging task, and makes attractive the prospect of automat-ically learning the behaviors from the actions of a human demonstrator. However, learning complex behaviors rapidly from a demonstrator may be difficult if they demand a large number of training samples. We describe an architecture...
Article
Full-text available
Multiagent systems often require coordination among the agents to maximize system utility. Using the notion of favors, we propose a technique, flexible reciprocal altruism, which determines when one agent should grant a favor to another agent based on past interactions. The desired rate of altruism is controllable, and as a result the loss associat...
Technical Report
Full-text available
MASON is a free, open-source Java-based discrete event multi-agent simulation toolkit that has been used to model network intrusions, unmanned aerial vehicles, nomadic migrations, and farmer/herder conflicts, among others. Many multi-agent models use georeferenced data which represent such things as road networks, rivers, vegetation coverage, popul...
Article
Full-text available
Developing behaviors for humanoid robots is dif-ficult due to the high complexity of programming these robots, which includes repeated trial and error cycles. We have recently developed a learning from demonstration system capable of training agent behaviors from a small number of training examples. Our system represents a complex behavior as a hie...
Conference Paper
In this paper, we discuss a curious relationship between Cooperative Coevolutionary Algorithms (CCEAs) and univariate Estimation of Distribution Algorithms (EDAs). Specifically, the distribution model for univariate EDAs is equivalent to the infinite population EGT model common in the analysis of CCEAs. This relationship may permit cross-pollinatio...
Article
Full-text available
The RoboPatriots are a team of three humanoid robots designed by the Computer Science Department at George Mason University. Each robot is based on the Kondo KHR-3HV, a customized Surveyor SVS camera, and a Gumstix
Article
We applied tools based on quantitative genetic theory in order to improve Evolutionary Algorithms for use with team learning tasks. We reviewed the quantitative genetics literature more widely, and developed a theoretical analysis applying genetic theory to the team learning problem. We then constructed and analyzed a neural network structure and n...
Article
Full-text available
This paper presents the dynamics of multiple learning agents from an evolutionary game theoretic perspective. We provide replicator dynamics models for cooperative coevolutionary algorithms and for traditional multiagent Q-learning, and we extend these differential equations to account for lenient learners: agents that forgive possible mismatched t...
Conference Paper
Full-text available
We examine opportunistic evolution, a variation of master- slave distributed evaluation designed for deployment of evo- lutionary computation to very large grid computing ar- chitectures with limited communications, severe evaluation overhead, and wide variance in evaluation node speed. In opportunistic evolution, slaves receive some N individuals...
Article
The following document describes the preferred formatting of technical reports and submission gu idelines. This document itself is formatted according to the preferred formatting rules.
Article
Full-text available
Solutions to non-cooperative multiagent systems often require achieving a joint policy which is as fair to all parties as possible. There are a variety of methods for determining the fairest such joint policy. One approach, it min fairness, finds the policy which maximizes the minimum average reward given to any agent. We focus on an extension, it...
Article
Cooperative coevolutionary algorithms (CEAs) offer great potential for concurrent multiagent learning domains and are of special utility to domains involving teams of multiple agents. Unfortunately, they also exhibit pathologies resulting from their game-theoretic nature, and these pathologies interfere with finding solutions that correspond to opt...
Conference Paper
Model- and simulation-designers are often interested not in the optimum output of their system, but in understanding how the output is sensitive to different parameters. This can require an inefficient sweep of a multidimensional parameter space, with many samples tested in regions of the space where the output is essentially all the same, or a spa...
Chapter
Full-text available
We present a new international project to develop temporally and spatially calibrated agent-based models of the rise and fall of polities in Inner Asia (Central Eurasia) in the past 5,000 years. Gaps in theory, data, and computational models for explaining long-term sociopolitical change-both growth and decay-motivate this project. We expect three...
Chapter
Full-text available
The purpose of this research was to replicate the Sugarscape model (Eptstein and Axtell 1996) and simulation outcomes as described in Growing Artificial Societies (GAS). Sugarscape is a classic agent-based model and contemporary simulation toolkits usually only have a very simple replication of a few core rules. There is scant evidence of significa...
Conference Paper
Full-text available
While support vector machines (SVMs) have shown great promise in supervised classification problems, researchers have had to rely on expert domain knowledge when choos- ing the SVM's kernel function. This project seeks to re- place this expert with a genetic programming (GP) system. Using strongly typed genetic programming and principled kernel clo...
Chapter
Multi-agent systems (MASs) is an area of distributed artificial intelligence that emphasizes the joint behaviors of agents with some degree of autonomy and the complexities arising from their interactions. The research on MASs is intensifying, as supported by a growing number of conferences, workshops, and journal papers. In this survey we give an...
Article
Genetic programming has highlighted the problem of bloat, the uncontrolled growth of the average size of an individual in the population. The most common approach to dealing with bloat in tree-based genetic programming individuals is to limit their maximal allowed depth. An alternative to depth limiting is to punish individuals in some way based on...
Conference Paper
Full-text available
The goal of this research is to explore the effects of social interactions between individual autonomous vehicles (AVs) in various problem scenarios. We take a look at one way to construct the social relationships and generate data from computer simulations to compare the behaviors of each. A difference can be noticed when synthetic social structur...
Article
Full-text available
Concurrent learning is a form of cooperative multiagent learning in which each agent has an independent learning process and little or no control over its teammates' ac-tions. In such learning algorithms, an agent's perception of the joint search space depends on the reward received by both agents, which in turn depends on the actions currently cho...
Conference Paper
What if traffic lights gave you a break after you've spent a long time waiting in traffic elsewhere? In this paper we examine a vari- ety of multi-agent traffic light controllers which consider vehicles' past stopped-at-red histories. For example, a controller might dis- tribute credits to cars as they wait and award the green light to lanes with t...
Conference Paper
Full-text available
Archive-based cooperative coevolutionary algorithms attempt to retain a set of individuals which act as good collaborators for other coevolved individuals in the evolutionary system. We introduce a new archive-based algorithm, called iCCEA, which compares favorably with other cooperative coevolutionary algorithms. We explain the current problems wi...
Conference Paper
In concurrent cooperative multiagent learning, each agent simul- taneously learns to improve the overall performance of the team, with no direct control over the actions chosen by its teammates. An agent's action selection directly influences the rewards received by all the agents, resulting in a co-adaptation among the concur- rent learning proces...
Conference Paper
Full-text available
Can a good learner compensate for a poor learner when paired in a coordination game? Previous work has given an example where a special learning algorithm (FMQ) is capable of doing just that when paired with a specific less capable algorithm even in games which stump the poorer algorithm when paired with itself. In this paper, we argue that this re...
Conference Paper
Full-text available
In concurrent learning algorithms, an agent's perception of the joint search space depends on the actions currently chosen by the other agents. These perceptions change as each agent's action selection is influenced by its learning. We observe that agents that show lenience to their teammates achieve more accurate perceptions of the overall learnin...
Article
Full-text available
Designing a robot control system that is able to intelligently decide how to handle prioritizing and combining the actions from multiple, conflicting goals is necessary for effective au-tonomous behavior. The method proposed in this paper uses a Genetic Algorithm (GA) to evolve the relative goal weights that enable agents to combine outputs from co...
Article
Cooperative multi-agent systems (MAS) are ones in which several agents attempt, through their interaction, to jointly solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task of p...
Article
MASON is a fast, easily extensible, discrete-event multi-agent simulation toolkit in Java, designed to serve as the basis for a wide range of multi-agent simulation tasks ranging from swarm robotics to machine learning to social complexity environments. MASON carefully delineates between model and visualization, allowing models to be dynamically de...
Article
Cooperative coevolutionary algorithms represent a pop-ular approach to learning via problem decomposition. Since they were proposed more than a decade ago, their properties have been studied both formally and empiri-cally. One important aspect of cooperative coevolution-ary algorithms concerns how to select collaborators for computing the fitness o...
Article
Full-text available
Abstract We introduce MASON, a fast, easily extensible, discrete-event multi-agent simulation toolkit in Java. MASON was designed,to serve as the basis for a wide,range of multi- agent simulation,tasks ranging,from,swarm,robotics to machine,learning,to social complexity,environments.,MASON carefully delineates between,model,and visual- ization, all...
Conference Paper
The C-value Paradox is the name given in biology to the wide variance in and often very large amount of DNA in eukaryotic genomes and the poor correlation between DNA length and perceived organism complexity. Several hypotheses exist which purport to explain the Paradox. Surprisingly there is a related phenomenon in evolutionary computation, known...
Conference Paper
Full-text available
Multi-agent problem domains may require distributed algorithms for a variety of reasons: local sensors, limitations of communication, and availability of distributed computational resources. In the absence of these constraints, centralized algorithms are often more efficient, simply because they are able to take advantage of more information. We in...
Article
Full-text available
Relocatable targets are mobile targets that will stay in a discrete location for an unknown, random length of time before moving to another location. Such targets include mobile missile launchers, air defense units, fuel trucks and other high value targets (e.g. maneuver forces). Using a combination of multiagent simulation and a multiobjective evo...
Conference Paper
We introduce a model for cooperative coevolutionary algo- rithms (CCEAs) using partial mixing, which allows us to compute the expected long-run convergence of such algorithms when individuals' fit- ness is based on the maximum payoff of some N evaluations with partners chosen at random from the other population. Using this model, we de- vise novel...
Article
Multi-agent research often borrows from biology, where remarkable examples of collective intelligence may be found. One interesting example is ant colonies' use of pheromones as a joint communication mechanism. In this paper we propose two pheromone-based algorithms for artificial agent foraging, trail-creation, and other tasks. Whereas practically...
Conference Paper
Recent theoretical work helped explain certain optimization-related pathologies in cooperative coevolutionary algorithms (CCEAs). Such explanations have led to adopting specific and constructive strategies for improving CCEA optimization performance by biasing the algorithm toward ideal collaboration. This paper investigates how sensitivity to the...
Conference Paper
Bloat control is an important aspect of evolutionary com- putation methods, such as genetic programming, which must deal with genomes of arbitrary size. We introduce three new methods for bloat con- trol: Biased Multi-Objective Parsimony Pressure (BMOPP), the Wait- ing Room, and Death by Size. These methods are unusual approaches to bloat control,...
Conference Paper
Genetic programming may be seen as a recent incarnation of a long-held goal in evolutionary computation: to develop actual computational devices through evolutionary search. Genetic programming is particularly attractive because of the generality of its application, but it has rarely been used in environments requiring iteration, recursion, or inte...
Conference Paper
Multi-agent research often borrows from biology, where remarkable examples of collective intelligence may be found. One interesting example is ant colonies' use of pheromones as a joint communication mechanism. In this paper we propose two pheromone-based algorithms for artificial agent foraging, trail-creation, and other tasks. Whereas practically...
Article
Full-text available
How does group memory affect sociality? Most computational multi-agent social simulation models are designed with agents lacking explicit internal information-processing structure in terms of basic cognitive elements. In particular, memory is usually not explicitly modeled. We present initial results from a new prototype called "Wetlands", designed...
Article
Most previous artificial ant foraging algorithms have to date relied to some degree on a priori knowledge of the environ-ment, in the form of explicit gradients generated by the nest, by hard-coding the nest location in an easily-discoverable place, or by imbuing the artificial ants with the knowledge of the nest direction. In contrast, the work pr...
Article
Full-text available
We introduce MASON, a fast, easily extendable, discrete- event multi-agent simulation toolkit in Java. MASON was designed to serve as the basis for a wide range of multi- agent simulation tasks ranging from swarm robotics to ma- chine learning to social complexity environments. MASON carefully delineates between model and visualization, al- lowing...
Article
The task of understanding the dynamics of coevolutionary algorithms or comparing performance between such algorithms is complicated by the fact the internal fitness measures are subjective. Though several techniques have been proposed to use external or objective measures to help in analysis, there are clearly properties of fitness payo#, like intr...
Article
The task of understanding the dynamics of coevolutionary algorithms or comparing performance between such algorithms is complicated by the fact the internal fitness measures are subjective. Though a variety of techniques for employing various kinds of external or objective measures exist to help with analysis, there are clearly properties of fitnes...
Conference Paper
Many evolutionary computation search spaces require fitness assessment through the sampling of and generalization over a large set of possible cases as input. Such spaces seem particularly apropos to Ge- netic Programming, which notionally searches for computer algorithms and functions. Most existing research in this area uses ad-hoc approaches to...