Conference PaperPDF Available

Artificial Swarm Intelligence vs human experts

Authors:
  • Unanimous AI

Abstract and Figures

"Artificial Swarm Intelligence" (ASI) strives to amplify the combined intelligence of networked human groups by enabling populations of participants to form real-time closed-loop systems modeled after biological swarms. Prior studies [Rosenberg 2015] have shown that "human swarms" can converge on more accurate decisions and predictions than traditional methods for tapping the wisdom of groups such as votes and polls. To further explore the predictive ability of ASI systems, 75 randomly selected sports fans were assembled into real-time human swarms using the UNU software platform and were tasked with predicting College Bowl football games against the spread. Results show intelligence amplification.
Content may be subject to copyright.
Rosenberg, Louis. "Artificial Swarm Intelligence vs Human Experts",
Neural Networks (IJCNN), 2016 International Joint Conference on. IEEE.
Artificial Swarm Intelligence vs Human Experts
Louis Rosenberg
Unanimous A.I.
San Francisco, CA 94115
unanimous.ai
Abstract Artificial Swarm Intelligence (ASI) strives to
facilitate the emergence of a super-human intellect by connecting
groups of human users in closed-loop systems modeled after bio-
logical swarms. Prior studies have shown that “human swarms”
can make more accurate predictions than traditional methods for
tapping the wisdom of groups, such as votes and polls. To further
test the predictive ability of swarms, 75 random sports fans were
assembled in the UNU platform for human swarming and tasked
with predicting College Bowl football games against the spread.
Expert predictions from ESPN were compared. The results are
as follows: (i) Individuals when working alone, test subjects
achieved on average, 5 correct predictions out of 10 games (50%
accuracy); (ii) Group Poll aggregating data across all 75
subjects, the group achieved 6 correct predictions out of 10
games (60% accuracy); (iii) Experts - as published by ESPN, the
college football experts averaged 5 correct predictions out of 10
games (50% accuracy); and (iv) Swarm when the 75 subjects
worked together as a real-time swarm, they achieved 7 correct
predictions out of 10 games (70% accuracy). Thus by forming a
real-time swarm intelligence, the group of random sports fans
boosted their collective performance and out-performed experts.
Keywords Swarm Intelligence, Artificial Intelligence, Human
Swarming, Wisdom of Crowds, Collective Intelligence
I. INTRODUCTION
In the field of A.I. research, practitioners have regularly
turned to Mother Nature for inspiration and guidance. Not
surprisingly, the first path explored was the most familiar our
own brains. Beginning with the Perceptrons of the 1950’s and
continuing to this day, Neural Networks have emerged as the
dominant biologically inspired model for A.I. research. Nature,
however, rarely reveals only a single pathway. Billions of years
of evolution have produced at least one alternate method for
generating high-level intelligence from smaller building blocks
and it’s not neural it’s collective.
Referred to as Swarm Intelligence (SI), countless species
are known to amplify a local group’s intellectual ability by
forming closed-loop systems among large numbers of
independent organisms. These dynamic systems demonstrate
that under the right conditions, a collective intelligence can
emerge that exceeds the capacity of the individual members in
the group. Artificial intelligence researchers have explored
swarm-based models for use among groups of networked
robots and simulated agents [1], but only recently has
swarming been applied to human networks [2, 3, 4, 5].
Known as Artificial Swarm Intelligence (ASI), these
computational methods enable human groups to work together
in real-time by forming a unified dynamic system that can
answer questions, make predictions, reach decisions, or take
actions. As a unified system, human swarms collectively
explore a decision-space and quickly converge upon preferred
solutions. Prior studies have shown that by working in swarms,
human groups can outperform their individual members as well
as outperform groups taking traditional votes or polls.
In a prior study, a randomly selected human group was
tasked with predicting the top awards of the 2015 Oscars, both
by taking a poll and by forming a swarm [5]. Across 48
participants, the average poll result achieved 6 of 15 correct
predictions (40% success). When taking most popular
prediction in the poll, the group achieved 7 of 15 correct
predictions (47% success). When working together as a real-
time swarm, the group achieved 11 of 15 correct predictions
(73% success). This suggests that ASI may be a superior
method for tapping the wisdom of crowds than traditional
votes, polls, and surveys. The present study aims to explore
this further, fielding a larger swarm of users and tasking them
with predicting 10 college football games against the spread. In
addition, the current study aims to compare the performance of
the human swarm, comprised of randomly selected novices,
with the performance of individual subject-matter experts.
II. SWARMS AS INTELLIGENT SYSTEMS
Among A.I. researchers, the word “swarm” often refers to
groups of robots or simulated agents governed by simple
localized rules [1]. These systems are generally inspired by
flocks of birds and schools of fish, which navigate complex
environments using similar processes. While such systems
have many applications, for example enabling robotic drones to
navigate in unison, the human swarms discussed herein are
modeled less after the motions of flocks and schools, and more
after the decision-making processes used by honeybee swarms.
This is because the decision-making abilities of honeybees
provide a powerful natural proof of the potential for an
emergent decentralized parallelized intelligence.
As studied by Seeley et al., the processes that govern
decision-making in honeybee swarms and neurological brains
are remarkably similar [6]-[9]. Both employ large populations
of simple excitable units (i.e., bees and neurons) that work in
parallel to integrate noisy evidence, weigh competing
alternatives, and converge on decisions in synchrony. In both,
decisions are arrived at through a real-time competition among
sub-populations of excitable units, each sub-population vying
for a different alternative solution. When one sub-population
exceeds a threshold level of support, the corresponding
alternative is chosen. The threshold in both brains and swarms
is not the unanimous support, or even a simple majority, but a
sufficient quorum of excitation. This helps to avoid deadlocks
and leads swarms to optimal decisions [10].
For example, every spring honeybees face a life-or-death
decision to select a new home location for the colony. From
hollow trees to abandoned sheds, the colony considers dozens
of candidate sites over a 30 square mile area, evaluating each
with respect to dozens of competing criteria. Does it have
sufficient ventilation? Is it safe from predators? Is it large
enough to store honey for winter? It’s a complex problem with
many tradeoffs and a misstep can mean death to the colony.
Using body vibrations known as “waggle dances”, hundreds of
bees express preferences for competing sites based on
numerous quality factors. Through a real-time negotiation, a
decision is reached when a sufficient quorum emerges.
Remarkably, the bees arrive at optimal decisions 80% of
the time [11]. Thus, although individual bees lack the mental
capacity to make a decision this complex and nuanced, when
hundreds of scout bees pool their knowledge and experience,
they evoke a Collective Intelligence that is not only able to
reach a decision, it finds an optimal solution. Thus by working
together as a unified dynamic system, the colony amplifies its
intelligence beyond the capacity of individual members. It is
this emergent amplification of intelligence that human
swarming aims to enable among groups of networked people.
III. ENABLING HUMAN SWARMS
Unlike many social species, human have not evolved the
natural ability to form a Swarm Intelligence, for we lack the
subtle connections that other organisms use to establish tight-
knit feedback-loops among members. Schooling fish detect
vibrations in the water around them. Flocking birds detect
motions propagating through the group. Swarming bees use
complex body vibrations. This suggests that to evoke a real-
time Artificial Swarm Intelligence (ASI) among groups of
networked humans, technology is required to close the loop
among members. To address this need, an online platform
called UNU was developed to allow distributed groups of users
to login from anywhere around the world and participate in a
closed loop swarming process.
Modeled after the decision-making of natural swarms,
UNU allows groups of independent actors to work in parallel
to (a) integrate noisy evidence, (b) weigh competing
alternatives, and (c) converge on final decisions in synchrony.
Because humans can’t waggle dance like honeybees, a novel
interface had to be developed to allow participants to convey
their individual intent with respect to a set of alternatives. In
addition, the interface had to be crafted to allow users to
perceive and react to the changing system in real-time, thereby
closing a feedback loop around the full population.
As shown in Figure 1, users of UNU answer questions by
collectively moving a graphical puck to select among a set of
alternatives. The puck is modeled as a physical system with a
defined mass, damping and friction. Users provide input by
manipulating a graphical magnet with a mouse or touchscreen.
By positioning their magnet, users impart their personal intent
as a force vector on the puck. The input from each user is not a
discrete vote, but a stream of vectors that varies freely over
time. Because the full set of users can adjust their intent at
every time-step, the puck moves, not based on the input of any
individual, but based on the dynamics of the full system. This
results in a real-time physical negotiation among the members
of the swarm, the group collectively exploring the decision-
space and converging on the most agreeable answer.
Fig 1. A human swarm comprised of user-controlled magnets.
We must note that users can only see their own magnet
during the decision, not the magnets of others users. Thus,
although they can view the puck’s motion in real time, which
represents the emerging will of the swarm, they are not
influenced by the specific breakdown of support across the
available options. This limits social biasing. For example, if the
puck slows due to an emerging deadlock, the participants must
evaluate their own willingness to shift support to alternate
options without knowing the distribution of support that caused
the deadlock. After each decision is over, users can view a
replay of all the magnets, allowing them to reflect on how their
contribution combined with others to produce the final answer.
Fig 2. A snapshot of a swarm answering a question.
In Figure 2 above, an example question is shown as it
appears simultaneously on the screens of all participants. In
this trial, a swarm of 90 users was asked a politically charged
question: “What should be Congress’s top priority?” Users are
then given a 3,2,1 countdown to coordinate the start of the
session. The swarm then springs into action, working in
synchrony to guide the puck to a preferred answer.
The decision process is generally a complex negotiation,
with individuals shifting their support numerous times to break
deadlocks or defend against options they disfavor. When a user
pulls towards one option in the answer set, a component of
their force also acts to impede the motion of the puck towards
competing options. In this way, users don’t only add support a
preferred solution when pulling towards it, but also suppress
solutions they don’t prefer. This enables the dual process seen
in natural swarms and neurological brains wherein individual
agents are enabled to both excite and inhibit [8], thereby
reducing the chances of a deadlock.
If a group happens to be in substantial agreement at the
start of the question, the puck moves smoothly to the preferred
answer. But, if two or more competing options have significant
support, the swarm negotiates as a unified system. Most users
begin by pulling towards the option they prefer most, then shift
to alternate choices if the puck starts moving towards an option
they dislike. With all users making these changes in parallel,
the swarm explores the decision space and converges on an
answer that optimizes group satisfaction.
It’s important to note that users don’t just vary the direction
of their input, but also the magnitude by adjusting the distance
between the magnet and the puck. Because the puck is in
motion, to apply full force users need to continually move their
magnet so that it stays close to the puck’s rim. This is
significant, for it requires all users to be engaged during the
decision process. If they stop adjusting their magnet to the
changing position of puck, the distance grows and their applied
force wanes. Thus, like bees executing a waggle dance or
neurons firing activation signals, the users in an artificial
swarm must continuously express their changing preferences
during the decision process or lose their influence over the
outcome.
Post testing interviews with participants suggest that users
with high levels of conviction in favor of a particular outcome
are more vigilant in maintaining maximum force on the puck.
Conversely, users who have lower conviction are less vigilant.
In this way, the swarming interface allows the population to
convey varying levels of conviction in real-time synchrony.
We believe this helps the swarms converge on solutions that
optimize the overall satisfaction of the group.
Observations and post-testing interviews also reveal that
human swarming yields consistent outcomes across varying
spatial placement of answer options. For example, if two
highly favored options are placed on opposite sides of the
puck’s starting position, the swarm will fall into an early
deadlock as it grapples between them. Conversely, if the two
highly favored options are placed on the same side of the
puck’s starting position, the swarm will not fall into an early
deadlock, but instead move the puck towards those two highly
favored options. Still, a deadlock will emerge as the puck
approaches midpoint between the two favored options. In this
way, the decision space can have alternate layouts, but the
swarm arrives at the same outcome. A similar robustness has
been observed in honeybee swarms, which are known to decide
upon optimal nesting locations regardless of the order in which
sites are discovered and reported by scout bees [11].
Referring again to Figure 2 the default layout of answers is
a set of six options in a hexagon pattern. The hexagonal
configuration was chosen because according to social-science
research, people are efficient decision-makers when presented
with up to six options, but suffer from increasing “choice-
overload” inefficiencies when confronted with larger sets [12].
To enable swarms to consider larger sets of answers, the
system employs an iterative approach, presenting users with a
series of six-option subsets of the full answer pool, then pitting
the winner of each subset against each other. The system also
allows swarms to select values on a continuous scale. This
enables swarms to collectively decide upon quantities, prices,
percentages, odds and other numerical values.
As shown in Figure 3 below, a swarm of users was asked to
decide upon the fair price of a movie ticket on a scale from $0
to $25. When using scale-based layout, the puck starts at the
center of the range and can move smoothly in either direction.
The swarm generally overshoots the final answer, then reverses
direction, oscillating in narrower and narrower bands. An
answer is chosen when the puck settles upon a value for more
than a threshold amount of time (e.g., 3 seconds).
Fig 3. A sample scale-based layout for human swarming
For the current set of tests, range-style questions we asked,
allowing users to predict both the winner and the point spread
of each bowl game.
IV. PERFORMANCE TESTING
To assess the predictive ability of human swarms, a formal
study was conducted with 75 randomly selected subjects. Each
participated in the experiment via online access. The only
requirement for participation was that each subject was self-
identified as a college football fan. Each subject was paid
$2.00 for their participation, which required them to make
predictions for the outcome of 10 college bowl football games,
first by on a blind poll using Survey Monkey, then as part of a
real-time human swarm using the UNU platform. In addition,
the researchers documented the predictions made by ESPN
experts for the same games [13]. Finally, the researchers
documented the Las Vegas point-spreads for each of the ten
games, which are designed by bookmakers to make each
prediction as close to a 50/50 proposition as possible.
When responding on the Survey Monkey poll, each
individual gave their own prediction about (a) which team
would win each of the 10 games, and (b) by how many points
would they win the game (point spread). This allowed for
predictions to be made against the spread, without explicitly
informing the subjects with what the spread was.
When working as a swarm, the participants were instructed
to move the graphical puck along a linear axis labeled with the
names of each team competing in a game. As the puck moved
along the axis, the closer to a particular team name, the higher
the chosen point spread victory for this team. Figure 4 shows a
screenshot for the Rose Bowl game, wherein the swarm
predicted Stanford University would win by 8 points. It’s
important to note that although Figure 4 shows all magnets
displayed, the subjects were only able to see their own magnet,
and thus could not see the pull directions of others.
Fig 4. Screenshot of Swarm predicting Rose Bowl
It should be noted that when predicting the 10th game (both
by poll and by swarm), a point spread was not used because the
10th game depended upon the outcome of prior games. There
were 4 possible winners of the final College Playoff game, the
subjects asked to predict which of the four teams would
prevail. The ESPN experts did the same.
V. RESULTS
Looking first at the poll results, we find that on average,
across 75 participants, the individuals made 5 correct picks for
the 10 games. This equates to 50% accuracy against the spread,
which is not impressive but confirms that the Las Vegas odds-
makers are skilled at what they do, picking spreads that for
average individuals, made each bet an approximate toss-up
among the competing teams.
Next we computed the most popular predictions in the
survey across all 75 subjects. This can be viewed as a
collective pick that taps the wisdom of crowds using traditional
polling. The group, as a collective, got 6 correct picks out the
10 games in question, yielding 60% accuracy. This supports
prior research into collective intelligence which suggest that
groups amplify their intelligence when averaging predictions.
Next we compared the swarm results wherein the group of
75 participants worked together as a real-time unified system.
The swarm produced 7 correct picks, yielding 70% accuracy.
This supports prior studies that show human swarming to be a
more effective means of tapping the collective intelligence of
groups than votes, polls, and surveys.
Comparing the swarm’s 7 correct predictions against the
individual picks made by the 75 participants, it was found that
the swarm outperformed 95% of participants. Thus, by
working together as a real-time swarm, 95% of the subjects
would have been better off going with the predictions made by
the Artificial Swarm Intelligence than their own picks. This
suggests that the ASI achieved a level of intelligence (with
respect to this defined task) that was superior to the intelligence
of the individual participants who comprised the swarm.
Finally we compared the swarm’s predictions to those of
the experts at ESPN. Based on publically published picks,
ESPN experts made 5 correct predictions against the spread for
the 10 games, yielding 50% accuracy. Thus, although their
predictions were made with professional expertise, they were
unable to beat the Las Vegas odds. The swarm, however, did
beat the odds by a good margin. In this way, a human swarm
of 75 sports fans, working as a unified system, produced more
accurate results than the topic-specific experts at ESPN.
As a final comparison, we computed the payouts that would
result if bets were by each of the parties. Because the swarm
picked two longshots, and only lost games that were toss-ups
(i.e. had nearly even odds), it did very well. The ESPN experts
on the other hand, lost both longshot. Had the ESPN experts
placed $10 on each of their picks, would have lost $24 of their
$100 bet (-24% ROI). The swarm, on the other hand, would
have won $34 across the ten games (+34% ROI). This further
supports the possibility that swarming can amplify intelligence,
allowing groups to behave as topic-specific experts.
VI. DISCUSSION AND CONCLUSIONS
Can swarms of average people rival the predictive abilities
of topic-specific professionals? The results of this study, along
with prior studies, suggest this might be the case. Furthermore,
swarming appears to be a more effective method of tapping the
wisdom of groups than traditional methods, like votes and
polls. This may be because unlike polls, which collect data
from individuals in isolation, swarms enable groups to
negotiate in real-time synchrony, adjusting and adapting as
decisions emerge before their eyes. The members of a swarm
don’t express static views, but continually assess and reassess
their own convictions with respect to each of the possible
outcomes, weighing their personal confidence and preferences.
With all participants doing this in parallel, the swarm
converges on solutions that reflect the collective will of the
group, tuned by each individual’s unique level of confidence.
Because of the potential of human swarming to enable
groups to combine their knowledge and intuition in real-time,
swarming likely offers the greatest benefit when groups make
complex decisions on topics that can be assessed from many
unique perspectives. This parallels the benefits of swarming
among honeybees, where the decision to pick a new home-site
must be evaluated across numerous competing factors. In fact,
when honeybee swarms choose a new colony site, they
consider dozen of locations, each evaluated with respect to at
least six independent attributes. Despite the complexity of the
decisions involved, honeybee swarms have been documented
as making nearly optimal decisions most of the time [11].
Looking forward, this experiment supports the possibility
that artificial swarms of networked humans have the potential
to produce an emergent intelligence that exceeds the
intellectual abilities of the individual participants for certain
tasks. This could lead to the development of a networked
super-intelligence that keeps humans in the loop. The fact that
human participants are central to the emergent intelligence is
promising, for it suggests that our human interests, values, and
morals would be integrated into to the process, achieving a
safer path to super-intelligence than a purely digital A.I.
Further research is needed, exploring how increasing the size
of swarms impacts the emergent intelligence produced.
ACKNOWLEDGMENT
Thanks to David Baltaxe and Joe Rosenbaum, both of
Unanimous A.I. for their help in making this study possible.
Also, this study was made possible by the use of UNU, an
online software platform that enables real-time human swarms.
REFERENCES
[1] Beni, G., Wang, J. Swarm Intelligence in Cellular Robotic Systems,
Proceed. NATO Advanced Workshop on Robots and Biological
Systems, Tuscany, Italy (1989).
[2] Rosenberg, L.B, “Human Swarms, a real-time paradigm for collective
intelligence.” Collective Intelligence 2015, Santa Clara CA.
[3] Palmer, Daniel W., et al. "Emergent Diagnoses from a Collective of
Radiologists: Algorithmic versus Social Consensus Strategies." Swarm
Intelligence. Springer International Publishing, 2014. 222-229.
[4] Eberhart, Russell, Daniel Palmer, and Marc Kirschenbaum. "Beyond
computational intelligence: blended intelligence." Swarm/Human
Blended Intelligence Workshop (SHBI), 2015. IEEE, 2015.
[5] Rosenberg, L.B., “Human Swarms, a real-time method for collective
intelligence.” Proceedings of the European Conference on Artificial Life
2015, pp. 658-659
[6] Seeley, Thomas D., et al. "Stop signals provide cross inhibition in
collective decision-making by honeybee swarms." Science 335.6064
(2012): 108-111.
[7] K.m. Passino, T.F. Seeley, P.K. Visscher, Swarm Cognition in
honeybees, Behav. Ecol. Sociobiol. 62, 401 (2008).
[8] J.A.R. Marchall, R. Bogacz, A. Dornhaus, R. Planque, T.Kovacs, N.R.
Franks, On optimal decision making in brains and social insect colonies,
J.R. Soc Interface 6,1065 (2009).
[9] I.D. Couzin, Collective Cognition in Animal Groups, Trends Cogn. Sci.
13,36 (2008).
[10] Seeley, Thomas D., Visscher, P. Kirk. Choosing a home: How the scouts
in a honey bee swarm perceive the completion of their group decision
making. Behavioral Ecology and Sociobiology 54 (5) 511-520.
[11] Seeley, Thomas D. Honeybee Democracy. Princeton University Press,
2010.
[12] Scheibehenne, Benjamin, Rainer Greifeneder, and Peter M. Todd. "Can
there ever be too many options? A meta-analytic review of choice
overload." Journal of Consumer Research 37.3 (2010): 409-425.
[13] ESPN College Bowl Picks, published on December 7, 2016.
http://espn.go.com/college-football/story/_/id/14306141/forty-games-
forty-one-predictions
... Recently, a new method has been developed that is not based on aggregating input from isolated individuals but involves synchronous groups of forecasters working together as real-time systems. Known as Artificial Swarm Intelligence (ASI) or Swarm AI, this method has been shown in numerous studies to significantly increase the accuracy of group forecasts [5][6][7][8][9][10][11][12][13]. ...
... In the natural world, swarming organisms establish real-time feedback loops among group members. To achieve this among groups of networked humans, ASI technology allows distributed users to form closed-loop systems moderated by swarming algorithms [5][6][7][8][9]. The goal is to enable groups of distributed users to work in parallel to (a) integrate noisy evidence, (b) weigh competing alternatives, and (c) converge on decisions in synchrony, while also allowing all participants to perceive and react to the changing system in real-time, thereby closing a feedback loop around the full population of participants. ...
... These results add to previous research demonstrating that human groups can use Swarm AI to make better collective assessments across a wide range of domains, from subjective judgements and medical diagnoses to market forecasting [5][6][7][8][9][10][11][12][13][14]. While this study was limited in that it only involved forecasting volatile cult stocks with groups of MBA students, the results support prior research showing success amplifying the accuracy of group financial forecasts using Swarm AI [12]. ...
Conference Paper
Full-text available
Swarm Intelligence (SI) is a natural process that has been shown to amplify decision-making accuracy in many social species, from schools of fish to swarms of bees. Artificial Swarm Intelligence (ASI) is a technology that enables similar benefits in networked human groups. The present research tests whether ASI enables human groups to reach more accurate financial forecasts. Specifically, a group of MBA candidates at Cambridge University was tasked with forecasting the three-day price change of 12 highly volatile assets, a majority of which were cult (or meme) stocks. Over a period of 9 weeks, human forecasters who averaged +0.96% ROI as individuals amplified their ROI to +2.3% when predicting together in artificial swarms (p=0.128). Further, a $5,000 bankroll was managed by investing in the top three buy recommendations produced each week by ASI, which yielded a 2.0% ROI over the course of the 9-week study. This suggests that swarm-based forecasting has the potential to boost the performance of financial traders in real-world settings.
... This reveals a range of behavioral characteristics within the swarm population and weights their contributions accordingly, from entrenched participants to flexible participants to fickle participants. Already, human swarms using this platform have significantly increased the predictive accuracy of groups across a variety of tasks, from betting on sporting events to forecasting financial markets [28][29][30][31][32]. Successful swarms have included as low as three to over 40 participants. ...
... As social perceptiveness is the greatest known predictor of collective intelligence, this suggests that swarms can perform better on a wide range of tasks and decisions. This interpretation is supported by other successful applications of human swarms to make surprisingly accurate decisions, from predicting sporting event outcomes to forecasting financial markets [28][29][30][31][32]. Future research can examine further the kinds of decisions and tasks that are best suited for human swarming. ...
Article
Swarm Intelligence is natural phenomenon that enables social animals to make group decisions in real-time systems. This process has been deeply studied in fish schools, bird flocks, and bee swarms, where collective intelligence has been observed to emerge. The present paper describes swarm.ai—a collaborative technology that enables swarms of humans to collectively converge upon a decision as a real-time system. Then we present the results of a study investigating if groups working as “human swarms” can amplify their social perceptiveness, a key predictor of collective intelligence. Results showed that groups reduced their social perceptiveness errors by more than half when operating as a swarm. A statistical analysis revealed with 99.9% confidence that groups working as swarms had significantly higher social perceptiveness than either individuals working alone or through plurality vote.
... However, only recently have platforms become available to effectively combine the power of human intelligence with swarm intelligence.[11] By creating a closedloop feedback system for humans to interact with, human swarm intelligence tightly connects the individual brains in the group, reducing noise in the group decision making process. ...
Article
In recent years, new Artificial Intelligence technologies have mimicked examples of collective intelligence occurring in the natural world including flocks of birds, schools of fish, and swarms of bees. One company in particular, Unanimous AI, built a platform (UNU Swarm) that enables a group of humans to make decisions as a single mind by forming a real-time closed-loop feedback system for individuals. This platform has proven the ability to amplify the predictive ability of groups of humans in realms including sports, medicine, politics, finance, and entertainment. Previous research has demonstrated it is possible to further enhance knowledge accumulation within a crowd through curation and bias methods applied to individuals in the crowd.\newline This study explores the efficacy of applying a machine learning pipeline to identify the top performing individuals in the crowd based on a structural profile of survey responses. The ultimate goal is to select these users as Swarm participants to improve the accuracy of the overall system. Unanimous AI provided 24 weeks of survey data collection consisting of 1,139 users from the NHL 2017-2018 season. By applying a machine learning pipeline, this study able to curate a crowd consisting of users that had an average z-score 0.309 and Wisdom of the Crowd prediction accuracy of 61.5%, which is 4.1% higher than a randomly selected crowd and 1.4% lower than Vegas favorite picks.
... Group decision making is an active area of research across behavioral sciences, psychology and neuroscience 43 , ecology 44 and collective (or swarm) robotics 45 ; but its dynamics and optimality principles are still incompletely known. We show that sensorimotor (haptic) communication can optimise group decisions. ...
Article
Full-text available
Group decisions can outperform the choices of the best individual group members. Previous research suggested that optimal group decisions require individuals to communicate explicitly (e.g., verbally) their confidence levels. Our study addresses the untested hypothesis that implicit communication using a sensorimotor channel—haptic coupling—may afford optimal group decisions, too. We report that haptically coupled dyads solve a perceptual discrimination task more accurately than their best individual members; and five times faster than dyads using explicit communication. Furthermore, our computational analyses indicate that the haptic channel affords implicit confidence sharing. We found that dyads take leadership over the choice and communicate their confidence in it by modulating both the timing and the force of their movements. Our findings may pave the way to negotiation technologies using fast sensorimotor communication to solve problems in groups.
... To enable swarming among groups of networked humans, ASI technology allows distributed groups of users to form closed-loop systems [5][6][7] and (a) integrate noisy evidence, (b) weigh competing alternatives, and (c) converge on decisions in synchrony, while also allowing all participants to perceive and react to the changing system in real-time. ...
Conference Paper
Full-text available
Groups often struggle to reach decisions, especially when populations are strongly divided by conflicting views. Traditional methods for collective decision-making involve polling individuals and aggregating results. In recent years, a new method called Artificial Swarm Intelligence (ASI or Swarm AI) has been developed that enables networked human groups to deliberate in real-time systems, moderated by artificial intelligence algorithms. While traditional voting methods aggregate input provided by isolated participants, Swarm-based methods enable participants to influence each other and converge on solutions together. In this study we compare the output of traditional methods such as Majority vote and Borda count to the Swarm method on a set of divisive policy issues. We find that the rankings generated using ASI and the Borda Count methods are often rated as significantly more satisfactory than those generated by the Majority vote system (p<0.05). This result held for both the population that generated the rankings (the "in-group") and the population that did not (the "out-group"): the in-group ranked the Swarm prioritizations as 9.6% more satisfactory than the Majority prioritizations, while the out-group ranked the Swarm prioritizations as 6.5% more satisfactory than the Majority prioritizations. This effect also held even when the out-group was subject to a demographic sampling bias of 10% (i.e. the out-group was composed of 10% more Labour voters than the in-group). The Swarm method was the only method to be perceived as more satisfactory to the "out-group" than the voting group.
Article
Full-text available
Machine learning has become the state-of-the-art technique for many tasks including computer vision, natural language processing, speech processing tasks, etc. However, the unique challenges posed by machine learning suggest that incorporating user knowledge into the system can be beneficial. The purpose of integrating human domain knowledge is also to promote the automation of machine learning. Human-in-the-loop is an area that we see as increasingly important in future research due to the knowledge learned by machine learning cannot win human domain knowledge. Human-in-the-loop aims to train an accurate prediction model with minimum cost by integrating human knowledge and experience. Humans can provide training data for machine learning applications and directly accomplish tasks that are hard for computers in the pipeline with the help of machine-based approaches. In this paper, we survey existing works on human-in-the-loop from a data perspective and classify them into three categories with a progressive relationship: (1) the work of improving model performance from data processing, (2) the work of improving model performance through interventional model training, and (3) the design of the system independent human-in-the-loop. Using the above categorization, we summarize the major approaches in the field; along with their technical strengths/ weaknesses, we have a simple classification and discussion in natural language processing, computer vision, and others. Besides, we provide some open challenges and opportunities. This survey intends to provide a high-level summarization for human-in-the-loop and to motivate interested readers to consider approaches for designing effective human-in-the-loop solutions.
Chapter
Full-text available
Artificial intelligence (AI) that can augment human intelligence in teamwork has been addressed in many studies; however, the state-of-the art of scholarly knowledge of the topic itself is missing. Thus, this paper provides a systematic review of the current knowledge on AI in augmenting human teams. The systematic literature review shows that AI working as teammate could augment human teams in important ways in enhancing team coordination, enhancing knowledge sharing and learning, supporting decision making, as well as evaluation and team performance. Further, the review also reveals that there are concerns related with social and machine teammate interaction, design, privacy, and ethics, that need further research to unleash AI technologies’ benefits in increasingly knowledge-intensive and diverse team collaboration.KeywordsArtificial intelligenceHumansTeamsAI-human collaborationAI-augmenting human intelligence (HI)AI-human teamwork (HT)
Article
Full-text available
Research in metaheuristics for global optimization problems are currently experiencing an overload of a wide range of available metaheuristic-based solution approaches. Since the commencement of the first set of classical metaheuristic algorithms namely genetic, particle swarm optimization, ant colony optimization, simulated annealing and tabu search in the early 70s to late 90s, several new advancements have been recorded with an exponential growth in the novel proposals of new generation metaheuristic algorithms. Because these algorithms are neither entirely judged based on their performance values nor according to the useful insight they may provide, but rather the attention is given to the novelty of the processes they purportedly models, this area of study will continue to periodically see the arrival of several new similar techniques in the future. However, there is an obvious reason to keep track of the progressions of these algorithms by collating their general algorithmic profiles in terms of design inspirational source, classification based on swarm or evolutionary search concept, existing variation from the original design, and application areas. In this paper, we present a relatively new taxonomic classification list of both classical and new generation sets of metaheuristic algorithms available in the literature, with the aim of providing an easily accessible collection of popular optimization tools for the global optimization research community who are at the forefront in utilizing these tools for solving complex and difficult real-world problems. Furthermore, we also examined the bibliometric analysis of this field of metaheuristic for the last 30 years.
Article
Artificial intelligence (AI) is fundamentally changing organizational decision-making processes. With the ability to self-learn and improve decision quality, AI is now taking over many decision responsibilities that were formally assigned to humans alone. However, the effectiveness of AI for ill-structured and uncertain decision environments is still in question. In such decision contexts that have no precedent to base a solution, humans have historically relied on their intuition to make decisions. Yet, intuition too has been found to have weaknesses that impact decision quality. Therefore, this article introduces a decision-making model that effectively integrates the strengths of both intuition and AI while minimizing the vulnerabilities of each method. The model specifies when and how both modes should be combined for effective organizational decision-making. In addition, the article presents important future research considerations relating to AI that impact both practitioners and academics.
Conference Paper
Full-text available
Much research has been done in the field of collective intelligence to aggregate input from human populations with the goal of amplifying the abilities of the groups. Nearly all prior research follows a similar model where input is collected from human participants in isolation and then aggregated statistically after the fact. The paper introduces a radically different approach in which the human participants is not aggregated statistically, but through a real-time dynamic control in which the participants act, react, and interact as a part of a system modeled after swarms in nature. Early testing of these "human swarms" suggest great potential for amplifying the intelligence of human groups, exceeding traditional aggregation methods. on the simulation of collaborative systems as it relates to the emergence of real-time collective intelligence. While theoretical studies are of great research value, there’s a growing need for real-world platforms that test the emergence of collective intelligence among human users. This short paper introduces such a platform. It enables networks of online collaborators to converge on questions, decisions, and dilemmas in real-time, functioning as a unified dynamic system. The dynamic system has been modeled after biological swarms, which is why refer to the process as “social swarming” or "human swarming". Early testing of human swarms suggests a great potential for harnessing collective intelligence.
Article
Full-text available
We synthesize findings from neuroscience, psychology, and behavioral biology to show that some key features of cognition in the neuron-based brains of vertebrates are also present in the insect-based swarm of honey bees. We present our ideas in the context of the cognitive task of nest-site selection by honey bee swarms. After reviewing the mechanisms of distributed evidence gathering and processing that are the basis of decision making in bee swarms, we point out numerous similarities in the functional organization of vertebrate brains and honey bee swarms. These include the existence of interconnected subunits, parallel processing of information, a spatially distributed memory, layered processing of information, lateral inhibition, and mechanisms of focusing attention on critical stimuli. We also review the performance of simulated swarms in standard psychological tests of decision making: tests of discrimination ability and assessments of distractor effects.
Article
Full-text available
This study considers the mystery of how the scout bees in a honey bee swarm know when they have completed their group decision making regarding the swarm's new nest site. More specifically, we investigated how the scouts sense when it is appropriate for them to begin producing the worker piping signals that stimulate their swarm-mates to prepare for the flight to their new home. We tested two hypotheses: "consensus sensing," the scouts noting when all the bees performing waggle dances are advertising just one site; and "quorum sensing," the scouts noting when one site is being visited by a sufficiently large number of scouts. Our test involved monitoring four swarms as they discovered, recruited to, and chose between two nest boxes and their scouts started producing piping signals. We found that a consensus among the dancers was neither necessary nor sufficient for the start of worker piping, which indicates that the consensus sensing hypothesis is false. We also found that a buildup of 10–15 or more bees at one of the nest boxes was consistently associated with the start of worker piping, which indicates that the quorum sensing hypothesis may be true. In considering why the scout bees rely on reaching a quorum rather than a consensus as their cue of when to start preparing for liftoff, we suggest that quorum sensing may provide a better balance between accuracy and speed in decision making. In short, the bees appear to begin preparations for liftoff as soon as enough of the scout bees, but not all of them, have approved of one of the potential nest sites.
Article
Full-text available
Stop, Don't Go There Decision-making in complex brains requires the reaching of a consensus based on information relayed by multiple neurons. A similar situation occurs in the decision-making process of swarming bees, where multiple individuals relay information about suitable hive sites, but a single site is chosen by the swarm. Seeley et al. (p. 108 , published online 8 December; see the Perspective by Niven ) show that consensus in this system is reached as information from scouting bees accumulates within the hive. Stop signals were given by scout bees from a particular site to those scout bees signaling from other sites. As the signals accumulate, scouting ceases and the bees prepare to swarm to the site that was best represented among the scouts. When this process was simulated, the results indicated that cross-inhibition among the bees functions similarly to that which occurs among neurons within complex brains. In both cases, such cross-inhibition prevents the overall system from coming to an impasse.
Article
Full-text available
The choice overload hypothesis states that an increase in the number of options to choose from may lead to adverse consequences such as a decrease in the motivation to choose or the satisfaction with the finally chosen option. A number of studies found strong instances of choice overload in the lab and in the field, but others found no such effects or found that more choices may instead facilitate choice and increase satisfaction. In a meta-analysis of 63 conditions from 50 published and unpublished experiments (N = 5,036), we found a mean effect size of virtually zero but considerable variance between studies. While further analyses indicated several potentially important preconditions for choice overload, no sufficient conditions could be identified. However, some idiosyncratic moderators proposed in single studies may still explain when and why choice overload reliably occurs; we review these studies and identify possible directions for future research. (c) 2010 by JOURNAL OF CONSUMER RESEARCH, Inc..
Article
Full-text available
The problem of how to compromise between speed and accuracy in decision-making faces organisms at many levels of biological complexity. Striking parallels are evident between decision-making in primate brains and collective decision-making in social insect colonies: in both systems, separate populations accumulate evidence for alternative choices; when one population reaches a threshold, a decision is made for the corresponding alternative, and this threshold may be varied to compromise between the speed and the accuracy of decision-making. In primate decision-making, simple models of these processes have been shown, under certain parametrizations, to implement the statistically optimal procedure that minimizes decision time for any given error rate. In this paper, we adapt these same analysis techniques and apply them to new models of collective decision-making in social insect colonies. We show that social insect colonies may also be able to achieve statistically optimal collective decision-making in a very similar way to primate brains, via direct competition between evidence-accumulating populations. This optimality result makes testable predictions for how collective decision-making in social insects should be organized. Our approach also represents the first attempt to identify a common theoretical framework for the study of decision-making in diverse biological systems.
Conference Paper
Twelve radiologists independently diagnosed 74 medical images. We use two approaches to combine their diagnoses: a collective algorithmic strategy and a social consensus strategy using swarm techniques. The algorithmic strategy uses weighted averages and a geometric approach to automatically produce an aggregate diagnosis. The social consensus strategy used visual cues to quickly impart the essence of the diagnoses to the radiologists as they produced an emergent diagnosis. Both strategies provide access to additional useful information from the original diagnoses. The mean number of correct diagnoses from the radiologists was 50 and the best was 60. The algorithmic strategy produced 63 correct diagnoses and the social consensus produced 67. The algorithm’s accuracy in distinguishing normal vs. abnormal patients (0.919) was significantly higher than the radiologists’ mean accuracy (0.861; p = 0.047). The social consensus’ accuracy (0.951; p = 0.007) showed further improvement.
Article
In recent years there has been a growing interest in the relationship between individual behavior and population-level properties in animal groups. One of the fundamental problems is related to spatial scale; how do interactions over a local range result in population properties at larger, averaged, scales, and how can we integrate the properties of aggregates over these scales? Many group-living animals exhibit complex, and coordinated, spatio-temporal patterns which despite their ubiquity and ecological importance are very poorly understood. This is largely due to the difficulties associated with quantifying the motion of, and interactions among, many animals simultaneously. It is on how these behaviors scale to collective behaviors that I will focus here. Using a combined empirical approach (using novel computer vision techniques) and individual-based computer models, I investigate pattern formation in both invertebrate and vertebrate systems, including - Collective memory and self-organized group structure in vertebrate groups (Couzin, I.D., Krause, J., James, R., Ruxton, G.D. & Franks, N.R. (2002) Journal of Theoretical Biology 218, 1-11. (2) Couzin, I.D. & Krause, J. (2003) Advances in the Study of Behavior 32, 1-75. (3) Hoare, D.J., Couzin, I.D. Godin, J.-G. & Krause, J. (2003) Animal Behaviour, in press.) - Self-organized lane formation and optimized traffic flow in army ants (Couzin, I.D. & Franks, N.R. (2003) Proceedings of the Royal Society of London, Series B 270, 139-146) - Leadership and information transfer in flocks, schools and swarms. - Why do hoppers hop? Hopping and the generation of long-range order in some of the largest animal groups in nature, locust hopper bands.