Conference PaperPDF Available

Forecasting of Volatile Assets using Artificial Swarm Intelligence

Authors:
  • Unanimous AI

Abstract and Figures

Swarm Intelligence (SI) is a natural process that has been shown to amplify decision-making accuracy in many social species, from schools of fish to swarms of bees. Artificial Swarm Intelligence (ASI) is a technology that enables similar benefits in networked human groups. The present research tests whether ASI enables human groups to reach more accurate financial forecasts. Specifically, a group of MBA candidates at Cambridge University was tasked with forecasting the three-day price change of 12 highly volatile assets, a majority of which were cult (or meme) stocks. Over a period of 9 weeks, human forecasters who averaged +0.96% ROI as individuals amplified their ROI to +2.3% when predicting together in artificial swarms (p=0.128). Further, a $5,000 bankroll was managed by investing in the top three buy recommendations produced each week by ASI, which yielded a 2.0% ROI over the course of the 9-week study. This suggests that swarm-based forecasting has the potential to boost the performance of financial traders in real-world settings.
Content may be subject to copyright.
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Forecasting of Volatile Assets using Artificial Swarm
Intelligence
Louis Rosenberg
Unanimous AI
San Francisco, USA
louis@unanimous.ai
Gregg Willcox
Unanimous AI
San Francisco, USA
gregg@unanimous.ai
Martti Palosuo
Judge Business School
University of Cambridge
Cambridge, UK
prm50@jbs.cam.ac.uk
Ganesh Mani
Carnegie Mellon University
Pittsburgh, USA
ganeshm@andrew.cmu.edu
Abstract Swarm Intelligence (SI) is a natural process that has
been shown to amplify decision-making accuracy in many social
species, from schools of fish to swarms of bees. Artificial Swarm
Intelligence (ASI) is a technology that enables similar benefits in
networked human groups. The present research tests whether ASI
enables human groups to reach more accurate financial forecasts.
Specifically, a group of MBA candidates at Cambridge University
was tasked with forecasting the three-day price change of 12
highly volatile assets, a majority of which were cult (or meme)
stocks. Over a period of 9 weeks, human forecasters who averaged
+0.96% ROI as individuals amplified their ROI to +2.3% when
predicting together in artificial swarms (p=0.128). Further, a
$5,000 bankroll was managed by investing in the top three buy
recommendations produced each week by ASI, which yielded a
2.0% ROI over the course of the 9-week study. This suggests that
swarm-based forecasting has the potential to boost the
performance of financial traders in real-world settings.
KeywordsArtificial Swarm Intelligence, Swarm Intelligence,
Human Forecasting, Financial Forecasting, Investing, Group
Forecasting, Cult Stocks, Meme Stocks, Collective Intelligence,
Wisdom of Crowds, Human-Machine Teaming.
I. INTRODUCTION
It is well known that groups of forecasters can outperform
individuals by aggregating estimates using statistical methods
[1-3]. Often called the Wisdom of Crowds (WoC) or Collective
Intelligence (CI), this phenomenon was first observed over a
century ago and has been applied to many fields. The most
common methods involve polling human groups and then
aggregating their input as a simple or weighted mean [4].
Recently, a new method has been developed that is not based
on aggregating input from isolated individuals but involves
synchronous groups of forecasters working together as real-time
systems. Known as Artificial Swarm Intelligence (ASI) or
Swarm AI, this method has been shown in numerous studies to
significantly increase the accuracy of group forecasts [5-13].
In a recent study at the Stanford University School of
Medicine, groups of doctors were asked to review chest X-rays
and predict the likelihood that each patient had pneumonia.
When working together in artificial swarm, diagnostic errors
were reduced by over 30% [14]. In another study, groups of
financial traders were asked to predict common market
indicators including the price of gold, oil, and the S&P 500.
Results showed a 36% increase in forecasting accuracy when
participants used ASI as compared to traditional methods [20].
While prior studies have shown ASI to significantly amplify
the group accuracy in controlled settings, the present work
assesses whether swarm-based forecasting of highly volatile
assets (mostly so-called cult or meme stocks), achieves similar
improvements. To address this, a nine-week pilot study tasked a
group of MBA candidates at Cambridge University with making
weekly forecasts of 12 high-volatility assets, comparing
individual forecasts to swarm-based predictions. Performance
was also compared to traditional Wisdom of Crowd methods.
II. BACKGROUND
A. Swarm Intelligence (SI)
The decision-making process that governs honeybee swarms
has been researched since the 1950s and has been shown at a
high level to be quite similar to decision-making in neurological
brains [15,16]. Both employ populations of simple excitable
units (i.e., neurons and bees) that work in parallel to integrate
noisy evidence, weigh competing alternatives, and converge on
decisions in real-time. In both brains and swarms, outcomes are
arrived at through competition among groups of excitable units.
In honeybees, this enables hundreds of scout bees to collect
information about their local environment and then deliberate in
synchrony, converging on a single optimal decision [17-20].
In the natural world, swarming organisms establish real-time
feedback loops among group members. To achieve this among
groups of networked humans, ASI technology allows distributed
users to form closed-loop systems moderated by swarming
algorithms [5-9]. The goal is to enable groups of distributed
users to work in parallel to (a) integrate noisy evidence, (b)
weigh competing alternatives, and (c) converge on decisions in
synchrony, while also allowing all participants to perceive and
react to the changing system in real-time, thereby closing a
feedback loop around the full population of participants.
B. Swarm Software
The software used to enable ASI in this study is called the
Swarm® platform from Unanimous AI and is shown in Figure 1.
Using this software, groups answer questions in real time by
collaboratively moving a graphical puck to select among a set of
answer options. Each participant provides input by moving a
graphical magnet to pull on the puck, thereby imparting their
personal intent on the system as a whole. The input from each
user is not a discrete vote, but a stream of time-varying vectors.
Fig. 1. Users engaging Swarm software to rank assets.
Because all users can adjust their intent continuously in real-
time, the puck moves based on interactions among all members,
empowering the group to converge in synchrony. Participants
must continuously update their input throughout the real-time
process or lose their influence over the outcome. This enables
the intelligence algorithms to continuously monitor the changing
behaviors of all members, modulating the aggregation. Figure 2
shows an example of the underlying human behaviors. More
details on the Swarm software can be found in [21, 22].
Fig. 2. Behavioral plot of the real-time decision making process. Darker
areas convey higher conviction. The dotted line shows the puck trajectory.
III. METHODOLOGY
To assess the ability of human groups to forecast cult stocks,
we conducted a nine-week study using volunteers from the
Cambridge Judge School MBA program. Volunteers self-
identified as interested in cult stocks and followed at least one
stock closely. In other words, they were all representative of the
high-level demographic driving the cult-stock movement. Each
weekly group of between 8 and 16 participants came from the
same pool of volunteers. To establish a baseline, all participants
provided their weekly forecasts as individuals using a standard
online survey. The group then congregated online in real-time
and used the Swarm platform and make collective forecasts.
In each week of the study, participants first predicted the
price change of the 12 assets over the next three days in a survey.
The survey asked participants to buy or short up to 2 units of
their virtual bankroll for each asset, and to predict which asset
would increase the most and which would decrease the most
respectively. The Wisdom of the Crowd (WoC) response to each
question was calculated as the most popular response provided
by the survey participants (i.e. the statistical Mode).
Next, participants logged into the Swarm platform to
synchronously answer these same questions as a groupfirst
allocating their virtual bankroll for each stock, and then creating
two rankings of assets: the most likely to decrease and the most
likely to increase over the next three days. For these rankings,
the top 5 longs and top 5 shorts were considered. All individuals
were anonymous to one another while swarming.
Swarm sessions started approximately 15 minutes after the
close of the market and lasted approximately half an hour. The
price of each security was recorded at the open of the market the
day after the swarm, and also at the close of the market three
days after the swarm. The price of bitcoin (BTC) was recorded
as soon as the swarm ended, as BTC trades continuously.
The percentage price change in each stock was calculated
using the price of the equity upon market open the day after the
swarm and the price at close of market three days after the
swarm. The top three individuals whose virtual bankroll showed
the highest ROI over the three-day period were awarded
bonuses: $15 for first place, $10 for second place, and $5 for
third place. This bonus was to incentivize participants to use
their best efforts in the forecasting surveys.
Finally, we managed a real bankroll over the duration of this
experiment: starting with $5,000, we invested in the three stocks
the Swarm ranked as most likely to increase in price. Bonuses
were awarded to participants based on the overall performance
of the swarm-managed bankroll. This was done to motivate best
efforts from members during the swarming portion of the study.
A. Data Analysis
Of the 108 asset movements collected, the mean movement
was 1.62% upwards, which was skewed higher by the presence
of a small number of outliers that increased in price by more than
20%. No price ever decreased by more than 20%.
Fig. 3. Distribution of Observed Price Movements
Such extreme outliers distort the analysis and interpretation
of these results by biasing towards a handful of data and may be
unrealistic in practice: a trader or hedge fund manager would
likely reduce exposure to these wild events by using stop losses
(and perhaps profit targets). As a result, we consider a clipping
function that restricts the maximum movement of these stocks
in the three-day window to a fixed interval: either 10% or 20%.
For reference, the 12 assets forecast consisted of stocks
(GME, AMC, RKT, TLRY, PLTR, TSLA, PTON, AAPL and
SPCE), ETFs (ARKK and social-media driven BUZZ) and
Bitcoin (BTC, a volatile cryptocurrency). A chart of the average
price change of each asset is provided in Figure 4. The vast
majority of price movements were under 5% during each trading
period, though some assets exhibited larger volatility.
Fig. 4. Average Movement across 12 Assets with 95% Confidence Interval
shown as black bars.
Finally, to meaningfully compare data, we ran statistical
significance tests comparing each investment strategy to each
other investment strategy by bootstrapping 1,000 times over the
data points produced by each metric. For example, to bootstrap
the Swarm Top Picks metric over the 9 weeks of the study, we
randomly selected 18 of the swarm’s Top Long and Top Short
picks with replacement from the set of 18 data points (9 weeks,
one Long and one Short per week) we have for this metric.
B. Results
When using a 20% clipping function, the swarm’s top-
ranked picks netted an average ROI of 2.3% week-over-week,
which outperformed the Individual (0.96%, p=0.13) and WoC
(1.6%, p=0.18) top-ranked picks, as shown in Figure 5. As a
result, we can be more than 80% confident that the top-ranked
Swarm picks outperformed the WoC and Individual rankings
due to more than random chance. Over the 9-week course of this
study, the Swarm Top Ranking strategy’s weekly average ROI
corresponds to an estimated cumulative ROI of 22.7%.
Fig. 5. Investment Performance using 20% Clipping.
To examine how sensitive these results are to changes in the
clipping, we next limited the stock movements further using a
10% clipping function. In this context, the swarm achieves a
1.77% ROI, which outperforms the average individual (-0.26%
ROI, p=0.110) and the WoC (-0.39% ROI, p=0.106). As a result,
we can be more than 85% confident that the Swarm Rankings
outperformed both the average individual and the median
individual response in this respect due to more than random
chance alone. We also see that there’s a reasonable range of
clipping limits for which the Swarm Top Rankings outperform
the WoC and Individual top rankings.
Fig. 6. Investment Performance using 10% Clipping.
We also find that the Swarm Top Rankings achieved a
significantly positive ROI in both the 20% clipping condition
(p<0.1) and the 10% clipping condition (p<0.1). Neither the
WoC nor Individual Top Rankings for either of these clipping
conditions yielded a significantly positive ROI (p>0.25 in all
cases). The distribution of all 1000 bootstrapped performances
for each investment strategy is shown in Figure 7.
Fig. 7. Distribution of Bootstrapped Performance using 10% Clipping
To more accurately evaluate the quality of these strategies
compared to the market, we can examine the performance of a
benchmark index, the S&P 500 (SPY), over the same time
intervals. We tabulated the price changes of SPY across the
length of this study in the same way as the cult stocks. SPY on
an average increased by +0.4% over the same time period we
considered. The Swarm Top Rankings outperformed this
benchmark metric using both the 10% clipping (p<0.2) and 20%
clipping (p<0.2), while the Individual and WoC Top Rankings
did not. As a result, we can conclude that Artificial Swarm
Intelligence allowed this group to achieve a higher ROI in
forecasting cult stocks than they would have achieved by
investing in a market benchmark.
C. Real Bankroll Analysis
Before the experiment began, we allocated a real bankroll of
$5,000 and invested weekly by splitting the full bankroll across
the top three swarm-ranked buy picks for that week. At the end
of the nine-week experiment, this bankroll had grown by 2.0%.
IV. CONCLUSIONS
This study highlights a promising technology for amplifying
the real-world forecasting power of groups: Artificial Swarm
Intelligence (ASI). In this study, ASI enabled a group of MBA
candidates to forecast the price movements of 12 high-volatility
financial assets (chiefly, equities) more accurately than if the
group were forecasting as individuals or as a crowd aggregating
their input statistically. The swarm-based forecasts yielded an
impressive 22.7% cumulative ROI over the nine-week study by
selecting one “top long” and one “top short” from the 12 assets
under consideration each week. Further, the top three long picks
were used each week to manage a real-world bankroll (with
fees) and achieved +2.0% ROI over the nine-week test.
These results add to previous research demonstrating that
human groups can use Swarm AI to make better collective
assessments across a wide range of domains, from subjective
judgements and medical diagnoses to market forecasting [5-14].
While this study was limited in that it only involved forecasting
volatile cult stocks with groups of MBA students, the results
support prior research showing success amplifying the accuracy
of group financial forecasts using Swarm AI [12].
ACKNOWLEDGMENT
The authors thank the MBA candidates at the University of
Cambridge, for their time. Thanks also to Culley Deisinger and
Chris Hornbostel for moderating and organizing the sessions.
REFERENCES
[1] Galton, F. (1907). Vox Populi. Nature, 75, 450-451.
[2] Steyvers, M., Lee, M.D., Miller, B., & Hemmer, P. (2009). The Wisdom
of Crowds in the Recollection of Order Information. NIPS. 1785-93.
[3] Philip E. Tetlock and Dan Gardner. 2015. Superforecasting: The Art and
Science of Prediction. Crown Publishing Group, New York, NY, USA.
[4] J Dana, P Atanasov, P Tetlock, B Mellers (2019), Are markets more
accurate than polls. The surprising informational value of “just asking.”
Judgment and Decision Making 14 (2), 135-147.
[5] Rosenberg, L.B., “Human Swarms, a real-time method for collective
intelligence.” Proceedings of the European Conference on Artificial Life
2015, pp. 658-659.
[6] Metcalf L, Askay DA, Rosenberg LB. Keeping Humans in the Loop:
Pooling Knowledge through Artificial Swarm Intelligence to Improve
Business Decision Making. California Management Review.
2019;61(4):84-109.
[7] Patel, B.N., Rosenberg, L., Willcox, G. et al. Humanmachine partnership
with artificial intelligence for chest radiograph diagnosis. npj Digital
Medicine 2, 111 (2019). https://doi.org/10.1038/s41746-019-0189-7
[8] Rosenberg, Louis. (2016) “Artificial Swarm Intelligence vs Human
Experts,” International Joint Conference on Neural Networks (IJCNN).
[9] Rosenberg, Louis. Baltaxe, David and Pescetelli, Nicollo. "Crowds vs
Swarms, a Comparison of Intelligence," IEEE 2016 Swarm/Human
Blended Intelligence (SHBI), Cleveland, OH, 2016, pp. 1-4.
[10] Baltaxe, David, Rosenberg, Louis and N. Pescetelli, “Amplifying
Prediction Accuracy using Human Swarms”, Collective Intelligence
2017. New York, NY.
[11] Willcox G., Rosenberg L., Askay D., Metcalf L., Harris E., Domnauer C.
(2020) Artificial Swarming Shown to Amplify Accuracy of Group
Decisions in Subjective Judgment Tasks. In: Arai K., Bhatia R. (eds)
Advances in Information and Communication. FICC 2019. Lecture Notes
in Networks and Systems, vol 70. Springer, Cham.
[12] L. Rosenberg, N. Pescetelli and G. Willcox, "Artificial Swarm
Intelligence amplifies accuracy when predicting financial markets," 2017
IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile
Communication Conference (UEMCON), NY, NY, 2017, pp. 58-62.
[13] L. Rosenberg and G. Willcox, "Artificial Swarm Intelligence vs Vegas
Betting Markets," 2018 11th International Conference on Developments
in eSystems Engineering (DeSE), Cambridge, UK, 2018, pp. 36-39.
[14] L. Rosenberg, M. Lungren, S. Halabi, G. Willcox, D. Baltaxe and M.
Lyons, "Artificial Swarm Intelligence employed to Amplify Diagnostic
Accuracy in Radiology," 2018 IEEE 9th Annual Information Technology,
Electronics and Mobile Communication Conference (IEMCON),
Vancouver, BC, 2018, pp. 1186-1191.
[15] Seeley T.D, Buhrman S.C 2001 “Nest-site selection in honey bees: how
well do swarms implement the ‘best-of-N’ decision rule?” Behavioral
Ecology and Sociobiology. 49(5), 416427.
[16] Marshall, James. Bogacz, Rafal. Dornhaus, Anna. Planqué, Robert.
Kovacs, Tim. Franks, Nigel. “On optimal decision-making in brains and
social insect colonies.” Soc. Interface 2009.
[17] Seeley, Thomas D., et al. "Stop signals provide cross inhibition in
collective decision-making by honeybee swarms." Science 335.6064
(2012): 108-111.
[18] Seeley, Thomas D. Honeybee Democracy. Princeton Univ. Press, 2010.
[19] Seeley, Thomas D., Visscher, P. Kirk. “Choosing a home: How the scouts
in a honey bee swarm perceive the completion of their group decision
making.” Behavioural Ecology and Sociobiology 54 (5) 511-520.
[20] Schumann, H., et. al. (2019) “‘Human Swarming Amplifies Accuracy
and ROI when Forecasting Financial Markets. IEEE International
Conference on Humanized Computing and Communication (HCC), 77-
82.
[21] Rosenberg L., Willcox G. (2020) Artificial Swarm Intelligence. In: Bi Y.,
Bhatia R., Kapoor S. (eds) Intelligent Systems and Applications.
IntelliSys 2019. Advances in Intelligent Systems and Computing, vol
1037. Springer, Cham. https://doi.org/10.1007/978-3-030-29516-5_79
[22] ASI White Paper - https://unanimous.ai/whitepaper
... Similarly, swarm computing can be understood as an expression or instance of socio-technical computing. One exciting quality of socio-technical computing is that, unlike conventional AI (which relies on deep learning only), it manifests as a quasi-living thing with the potential to seem more self-directed, hence more intelligent, than earlier AI paradigms, e.g., centrally coordinated AI. Rosenberg et al. (2021) highlight that AI can be harnessed in ways that mimic ants and birds in their manifest group intelligence to perform a range of tasks. In their trials this approach was able to obtain around a 20% improvement in system performance in new areas such as stock trading. ...
... A well-known example is core Internet technologies like Wikipedia. Research by Rosenberg et al. (2021) has shown how swarm-based computing can be harnessed to produce better outcomes than either AI alone or human experts. Another well-known example is Google's use of AI's experts to direct their research into fusion (Norman 2017). ...
Chapter
In this chapter we invite the reader to consider the importance of context in the history and evolution of both artificial intelligence and the virtual university. There are histories in the terminology as much as what they refer to. Thus, the virtual universities that emerged late last century are very different in scope and vision compared to contemporary practice. There are likewise different theoret- ical perspectives with Gartner’s “Hype Cycle,” one among many methods that provides perspective on the stages of AI-driven advances in technological inno- vation. An imperative to foreground questioning of the domain is what frames our discussion. One key question among many concerns the affordances and chal- lenges that AI presents for a virtual university in the coming decades. A university must prepare for AI-based auditing of the integrity of its student records. AI can help spot plagiarism and fake IDs and support academic integrity. Policy-makers might use AI to spot potentially impossible situations that could give rise to big penalties. But as AI merges into our environment, it is no longer just a tool with new ethical and social issues arising. There already exist forms of colonial biases embedded into AI and machine learning systems. What practical choices exist now? Every virtual university might consider collaborating on how to ensure trustworthiness of their operational systems that use AI. Emerging public policy such as the EU’s Artificial Intelligence Act provides a legal mechanism for building a reliable AI ecosystem. Research and innovation into both AI and the kinds of services virtual universities will need into the future are now proliferat- ing. Many futures are possible. From an educational perspective developing human agency through this next era should be a priority. Students could be taught how to use AI-supported tools to solve increasingly challenging questions. Even more imperative will be the questions they ask.
... Algorithm-assisted prediction and auction markets can facilitate negotiation around limited resources and maximization of joint outcomes via market-like bidding systems (Malone et al., 2017). Recent developments incorporate models of "swarm intelligence" to gauge collective sentiment and provide a high-level view to all members to help facilitate consensus (Rosenberg, Willcox, Palosuo, & Mani, 2021). However, there are additional opportunities for AI to initiate such exchanges and prompt collective reasoning based on indicators that members may have misaligned goals, such as when the level of member engagement declines based on the pace of activity and emotional states (Van Kleef, Homan, & Cheshin, 2012). ...
Article
Full-text available
Artificial Intelligence (AI) powered machines are increasingly mediating our work and many of our managerial, economic, and cultural interactions. While technology enhances individual capability in many ways, how do we know that the sociotechnical system as a whole, consisting of a complex web of hundreds of human–machine interactions, is exhibiting collective intelligence? Research on human–machine interactions has been conducted within different disciplinary silos, resulting in social science models that underestimate technology and vice versa. Bringing together these different perspectives and methods at this juncture is critical. To truly advance our understanding of this important and quickly evolving area, we need vehicles to help research connect across disciplinary boundaries. This paper advocates for establishing an interdisciplinary research domain—Collective Human‐Machine Intelligence (COHUMAIN). It outlines a research agenda for a holistic approach to designing and developing the dynamics of sociotechnical systems. In illustrating the kind of approach, we envision in this domain, we describe recent work on a sociocognitive architecture, the transactive systems model of collective intelligence, that articulates the critical processes underlying the emergence and maintenance of collective intelligence and extend it to human–AI systems. We connect this with synergistic work on a compatible cognitive architecture, instance‐based learning theory and apply it to the design of AI agents that collaborate with humans. We present this work as a call to researchers working on related questions to not only engage with our proposal but also develop their own sociocognitive architectures and unlock the real potential of human–machine intelligence.
... It will take advantage of both the large in-sample datasets and the regime-switching strategy which dominates static strategies out-of-sample (Ang and Bekaert, 2015). Contemporary data science techniques from the field of machine learning as well as swarm modeling (Rosenberg et al., 2021) will also be explored. ...
Article
Full-text available
Cryptoassets have experienced dramatic volatility in their prices, especially during the COVID-19 pandemic era. This pilot study explores volatility asymmetry of and correlations among three popular cryptoassets (Bitcoin, Ethereum, and Dogecoin) as well as Gold. Multiple Generalized Autoregressive Conditional Heteroskedasticity (GARCH) models are analyzed. We find that positive shocks have a greater impact on the volatility of these financial assets than negative shocks of the same magnitude, perhaps a manifestation of the fear of missing out (FOMO) effect. Our research is the first time to use COVID-19-period volatility of financial assets (in-sample data) to forecast their later COVID-19-period volatility (out-of-sample data). This forecast accuracy is compared to that produced by forecasts using the same out-of-sample data and a longer in-sample data. Our results indicate that generally, the larger in-sample dataset gives a higher forecast accuracy though the smaller in-sample dataset is from the same regime as the out-of-sample data. We also evaluate the correlations among the assets using the Dynamic Conditional Correlation (DCC) framework and find that there is an elevated positive correlation between Gold and Bitcoin during the past two years. The Gold-Bitcoin correlation hit its peak during the peak of the COVID-19 pandemic and then fell back to around zero in July 2021 when the pandemic crisis eased. Unsurprisingly, there is a strong positive correlation among the cryptocurrencies. Pairwise correlation among all four assets was stronger during the COVID-19 pandemic. Such continuing analysis can inform portfolio asset allocation as well as general financial policy decisions.
Article
Full-text available
Human-in-the-loop (HITL) AI may enable an ideal symbiosis of human experts and AI models, harnessing the advantages of both while at the same time overcoming their respective limitations. The purpose of this study was to investigate a novel collective intelligence technology designed to amplify the diagnostic accuracy of networked human groups by forming real-time systems modeled on biological swarms. Using small groups of radiologists, the swarm-based technology was applied to the diagnosis of pneumonia on chest radiographs and compared against human experts alone, as well as two state-of-the-art deep learning AI models. Our work demonstrates that both the swarm-based technology and deep-learning technology achieved superior diagnostic accuracy than the human experts alone. Our work further demonstrates that when used in combination, the swarm-based technology and deep-learning technology outperformed either method alone. The superior diagnostic accuracy of the combined HITL AI solution compared to radiologists and AI alone has broad implications for the surging clinical AI deployment and implementation strategies in future practice.
Conference Paper
Full-text available
Many social species amplify their decision-making accuracy by deliberating in real-time closed-loop systems. Known as Swarm Intelligence (SI), this natural process has been studied extensively in schools of fish, flocks of birds, and swarms of bees. The present research looks at human groups and tests their ability to make financial forecasts by working together in systems modeled after natural swarms. Specifically, groups of financial traders were tasked with forecasting the weekly trends of four common market indices (SPX, GLD, GDX, and Crude Oil) over a period of 19 consecutive weeks. Results showed that individual forecasters, who averaged 56.6% accuracy when predicting weekly trends on their own, amplified their accuracy to 77.0% when predicting together as real-time swarms. This reflects a 36% increase in forecasting accuracy and shows high statistical significance (p<0.001). Further, if investments had been made according to these swarm-based forecasts, the group would have netted a 13.3% return on investment (ROI) over the 19 weeks, compared to the individual's 0.7% ROI. This suggests that enabling groups of traders to form real-time systems online, governed by swarm intelligence algorithms, has the potential to significantly increase the accuracy and ROI of financial forecasts.
Article
Full-text available
Psychologists typically measure beliefs and preferences using self-reports, whereas economists are much more likely to infer them from behavior. Prediction markets appear to be a victory for the economic approach, having yielded more accurate probability estimates than opinion polls or experts for a wide variety of events, all without ever asking for self-reported beliefs. We conduct the most direct comparison to date of prediction markets to simple self-reports using a within-subject design. Our participants traded on the likelihood of geopolitical events. Each time they placed a trade, they first had to report their belief that the event would occur on a 0-100 scale. When previously validated aggregation algorithms were applied to self-reported beliefs, they were at least as accurate as prediction-market prices in predicting a wide range of geopolitical events. Furthermore, the combination of approaches was significantly more accurate than prediction-market prices alone, indicating that self-reports contained information that the market did not efficiently aggregate. Combining measurement techniques across behavioral and social sciences may have greater benefits than previously thought.
Conference Paper
Full-text available
In the natural world, Swarm Intelligence (SI) is a commonly occurring process in which biological groups amplify their collective intelligence by forming closed-loop systems. It is well known in schools of fish, flocks of bird, and swarms of bees. In recent years, new AI technologies have enabled networked human groups to form systems modeled after natural swarms. Known as Artificial Swarm Intelligence (ASI), the technique has been shown to amplify the effective intelligence of human groups. This study compares the predictive ability of ASI systems against large betting markets when forecasting sporting events. Groups of average sports fans were tasked with predicting the outcome of 200 hockey games (10 games per week for 20 weeks) in the NHL. The expected win rate for Vegas favorites was 62% across the 200 games based on the published odds. The ASI system achieved a win rate of 85%. The probability that the system outperformed Vegas by chance was extremely low (p = 0.0057), indicating a significant result. In addition, researchers compared the winnings from two betting models - one that wagered weekly on the Vegas favorite, and one that wagered weekly on the ASI favorite. At the end of 20 weeks, the Vegas model generated a 41% financial loss, while the ASI model generated a 170% financial gain.
Chapter
Full-text available
New technologies enable distributed human teams to form real-time systems modeled after natural swarms. Often referred to as Artificial Swarm Intelligence (ASI) or simply “human swarming”, these real-time systems have been shown to amplify group intelligence across a wide range of tasks, from handicapping sports to forecasting financial markets. While most prior research has studied human swarms with 20–100 members, the present study explores the ability of ASI to amplify accuracy in small teams of 3–6 members. The present study also explores if conducting multiple swarms and aggregating by taking a “vote of swarms” can further amplify the accuracy. A large set of 66 small teams were engaged in this study. Each team was given a standard subjective judgement test. Participants took the test both as individuals and real-time swarms. The average individual scored 69% correct, while the average swarm scored 84% correct (p < 0.001). In addition, aggregation of multiple swarms revealed additional amplifications of accuracy. For example, by randomly selecting sets of 3 swarms and aggregating by plurality vote, average accuracy increased to 91% (p < 0.001). These results suggest that when small teams make subjective judgements as real-time swarms, they can be significantly more accurate than individual members, and that their accuracy can be further amplified by aggregating the output across small sets of swarms.
Conference Paper
Full-text available
Swarm Intelligence (SI) is a biological phenomenon in which groups of organisms amplify their combined brainpower by forming real-time systems. It has been studied for decades in fish schools, bird flocks, and bee swarms. Recent advances in networking and AI technologies have enabled distributed human groups to form closed-loop systems modeled after natural swarms. The process is referred to as Artificial Swarm Intelligence (ASI) and has been shown to significantly amplify group performance. The present research applies ASI technology to the field of medicine, exploring if small groups of networked radiologists can improve their diagnostic accuracy when reviewing chest X-rays for the presence of pneumonia. Performance data was collected for individual radiologists generating diagnoses alone, as well as for small groups of radiologists working together to generate diagnoses as a real-time ASI system. Diagnoses were also collected from a state-of-the-art deep learning system (CheXNet) developed at Stanford University. Results showed that small groups of networked radiologists, when working as a real-time ASI system, were significantly more accurate than the individual radiologists on their own, reducing diagnostic errors by 33%. Results also showed that small groups of networked radiologists, when working as an ASI system, were significantly more accurate (22%) than a state-of-the-art deep learning system (CheXNet).
Chapter
Swarm Intelligence (SI) is a natural phenomenon that enables social species to quickly converge on optimized group decisions by interacting as real-time closed-loop systems. This process, which has been shown to amplify the collective intelligence of biological groups, has been studied extensively in schools of fish, flocks of birds, and swarms of bees. This paper provides an overview of a new collaboration technology called Artificial Swarm Intelligence (ASI) that brings the same benefits to networked human groups. Sometimes referred to as "human swarming" or building "hive minds," the process involves groups of networked users being connected in real-time by AI algorithms modeled after natural swarms. This paper presents the basic concepts of ASI and reviews recently published research that shows its effectiveness in amplifying the collective intelligence of human groups, increasing accuracy when groups make forecasts, generate assessments, reach decisions, and form predictions. Examples include significant performance increases when human teams generate financial predictions, business forecasts, subjective judgments, and medical diagnoses.
Article
This article explores how a collaboration technology called Artificial Swarm Intelligence (ASI) addresses the limitations associated with group decision making, amplifies the intelligence of human groups, and facilitates better business decisions. It demonstrates of how ASI has been used by businesses to harness the diverse perspectives that individual participants bring to groups and to facilitate convergence upon decisions. It advances the understanding of how artificial intelligence (AI) can be used to enhance, rather than replace, teams as they collaborate to make business decisions.