Content uploaded by Rahul Jain
Author content
All content in this area was uploaded by Rahul Jain on Jan 09, 2024
Content may be subject to copyright.
Citation: Jain, R.; Vanzara, R.
Emerging Trends in AI-Based Stock
Market Prediction: A Comprehensive
and Systematic Review. Eng. Proc.
2023,56, 254. https://doi.org/
10.3390/ASEC2023-15965
Academic Editor: Nunzio Cennamo
Published: 10 November 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
Proceeding Paper
Emerging Trends in AI-Based Stock Market Prediction: A
Comprehensive and Systematic Review †
Rahul Jain 1, * and Rakesh Vanzara 2
1Department of Computer Engineering, UVPCE, Ganpat University, Mehsana 384012, Gujarat, India
2Department of Information Technology, UVPCE, Ganpat University, Mehsana 384012, Gujarat, India;
rakesh.vanzara@ganpatuniversity.ac.in
*Correspondence: rahuljaincse51@gmail.com; Tel.: +91-9993671809
†
Presented at the 4th International Electronic Conference on Applied Sciences, 27 October–10 November 2023;
Available online: https://asec2023.sciforum.net/.
Abstract: This research paper provides a comprehensive review of the emerging trends in AI-based
stock market prediction. The paper highlights the key concepts, approaches, and techniques employed
in AI-based stock market prediction and discusses their strengths and limitations. Key topics covered
include deep learning, natural language processing, sentiment analysis, and reinforcement learning.
This paper also presents case studies and evaluates the performance of different AI-based models in
predicting stock market trends. Overall, this research paper provides valuable insights into the latest
advancements in AI-based stock market prediction and their potential implications for investors,
financial analysts, and policy makers.
Keywords: AI; machine learning; stock market; prediction; deep learning; natural language processing;
sentiment analysis; reinforcement learning
1. Introduction
The stock market, a complex and dynamic system vital to the global economy, is influ-
enced by politics, economics, and investor sentiment. This unpredictability has challenged
financial experts. Traditionally, fundamental and technical analyses were used for predic-
tions, but they had limitations. Artificial Intelligence (AI) is reshaping the stock market
landscape through predictive analytics, trading algorithms, risk management, fraud detec-
tion, and portfolio optimization. Artificial Intelligence (AI) employs predictive analytics
to analyze extensive financial data, revealing patterns and predicting market movements
and empowering traders and investors with well-informed decisions. Real-time trading
algorithms driven by AI adjust strategies based on market conditions, bolstering prof-
itability. AI aids in identifying and mitigating risks by analyzing varied data sources like
financial news, social media, and economic indicators. AI’s pattern recognition capabilities
are instrumental in detecting fraudulent trading and market manipulation activities. Port-
folio optimization benefits from AI’s analysis of market trends and risk factors, reshaping
decision-making, risk management, and profit realization. The study on Emerging Trends
in AI-based Stock Market Prediction aims to illuminate recent breakthroughs and their
impact on investors, financial analysts, and policymakers. AI-driven forecasts outshine
conventional methods, holding potential for informed investment choices. Acknowledging
AI’s strengths and limitations, this research strives to heighten prediction accuracy and
reliability, ultimately benefiting the financial sector. In recent times, AI techniques have
gained traction for stock market prediction due to their superior performance compared
to traditional methods. With ample financial data and advanced machine learning, AI
models are revolutionizing the finance industry. Approaches like fundamental analysis,
technical analysis, quantitative analysis, machine learning, sentiment analysis, and expert
opinion categorize prediction methods based on methodology and data sources, reflecting
Eng. Proc. 2023,56, 254. https://doi.org/10.3390/ASEC2023-15965 https://www.mdpi.com/journal/engproc
Eng. Proc. 2023,56, 254 2 of 6
AI’s evolving impact on stock prediction. This paper delivers valuable insights into recent
strides in AI-based stock market prediction and their potential repercussions for investors,
financial analysts, and policymakers. Its aim is to foster a comprehensive comprehension
of the present AI-based stock market prediction landscape, spotlighting areas warranting
further exploration. These findings empower better-informed investment choices and foster
more precise and dependable AI models for stock market prediction.
2. Literature Review
The research topic “Emerging Trends in AI-based Stock Market Prediction: A Review”
has garnered substantial interest, prompting various studies to explore AI and machine
learning’s potential in forecasting stock trends and prices. Presented here is a concise
literature overview on this subject. Based on our review study, Figure 1reflects % use in
stock market prediction vs. ML Techniques.
Eng. Proc. 2023, 56, x FOR PEER REVIEW 2 of 6
machine learning, sentiment analysis, and expert opinion categorize prediction methods
based on methodology and data sources, reflecting AI’s evolving impact on stock pre-
diction. This paper delivers valuable insights into recent strides in AI-based stock market
prediction and their potential repercussions for investors, financial analysts, and poli-
cymakers. Its aim is to foster a comprehensive comprehension of the present AI-based
stock market prediction landscape, spotlighting areas warranting further exploration.
These findings empower better-informed investment choices and foster more precise and
dependable AI models for stock market prediction.
2. Literature Review
The research topic “Emerging Trends in AI-based Stock Market Prediction: A Re-
view” has garnered substantial interest, prompting various studies to explore AI and
machine learning’s potential in forecasting stock trends and prices. Presented here is a
concise literature overview on this subject. Based on our review study, Figure 1 reflects %
use in stock market prediction vs. ML Techniques.
Figure 1. Stock market prediction in % vs. ML Techniques used for it (LSTM: long short-term
memory, GBM: gradient boosted models, CNN: convolutional neural network, RNN: recurrent
neural networks, RF: random forest, SVM: support vector machine, and LR: linear regression).
Machine learning models offer investors a potential edge in predicting stock prices
and trends by swiftly processing extensive data, though their accuracy is variable due to
the intricate and multifaceted nature of the stock market [1]. Unraveling intricate
non-linear stock market patterns requires careful technique selection, with feature choice
playing a pivotal role in revealing crucial insights and ensuring reliable predictions [2].
Identifying complex stock market patterns requires intricate techniques, with feature se-
lection being pivotal for robust predictions, as features unveil crucial insights from in-
tricate data [3]. In their 2023 research, Vitor Azevedo et al. investigated the prediction of
capital market anomalies, using diverse machine learning techniques and models on a
massive dataset, with top models showing significant monthly returns of 1.8% to 2.0%
[4]. The authors investigate ANN, SVM, and LSTM neural networks, highlighting their
distinct characteristics and practical applications, underscoring the transformative role of
machine learning in shaping investment strategies [5]. The study highlights data label-
ing’s significance in trading system creation, introducing N-Period Min-Max (NPMM)
labeling to tackle information loss and noise issues and showing its superior effectiveness
in stock price trend prediction compared to other methods [6]. A novel algorithm merg-
ing deep reinforcement learning with portfolio theory, utilizing a 3D convolutional neu-
Figure 1. Stock market prediction in % vs. ML Techniques used for it (LSTM: long short-term
memory, GBM: gradient boosted models, CNN: convolutional neural network, RNN: recurrent neural
networks, RF: random forest, SVM: support vector machine, and LR: linear regression).
Machine learning models offer investors a potential edge in predicting stock prices
and trends by swiftly processing extensive data, though their accuracy is variable due to the
intricate and multifaceted nature of the stock market [
1
]. Unraveling intricate non-linear
stock market patterns requires careful technique selection, with feature choice playing a
pivotal role in revealing crucial insights and ensuring reliable predictions [
2
]. Identifying
complex stock market patterns requires intricate techniques, with feature selection being
pivotal for robust predictions, as features unveil crucial insights from intricate data [
3
].
In their 2023 research, Vitor Azevedo et al. investigated the prediction of capital market
anomalies, using diverse machine learning techniques and models on a massive dataset,
with top models showing significant monthly returns of 1.8% to 2.0% [
4
]. The authors in-
vestigate ANN, SVM, and LSTM neural networks, highlighting their distinct characteristics
and practical applications, underscoring the transformative role of machine learning in
shaping investment strategies [
5
]. The study highlights data labeling’s significance in trad-
ing system creation, introducing N-Period Min-Max (NPMM) labeling to tackle information
loss and noise issues and showing its superior effectiveness in stock price trend prediction
compared to other methods [
6
]. A novel algorithm merging deep reinforcement learning
with portfolio theory, utilizing a 3D convolutional neural network for feature extraction
and Deep Deterministic Policy Gradient (DDPG) for portfolio optimization, consistently
demonstrates improved performance [
7
]. Michele Costola et al. reveals a statistical connec-
tion between the stock market and COVID-19 emotions, showcasing how pandemic-related
Eng. Proc. 2023,56, 254 3 of 6
news influenced investor expectations and financial market trends [
8
]. The study introduces
a novel approach using an LSTM predictor and AC-SFLA optimizer for enhanced efficiency
and is demonstrated through testing on a real stock market dataset [
9
]. LSTM effectively
processes temporal data by utilizing input, output, and forgetting gates to manage infor-
mation flow [
10
]. Yanrui Li et al. (2022) present a comprehensive method that enhances
company ranking for investment in China’s A market, yielding a remarkable 9.2% increase
in P-return compared to averages and indicating enhanced accuracy and performance [
11
].
In their 2022 study, VMalti Bansal et al. found that deep learning algorithms outperformed
traditional machine learning in predicting stock prices of Indian companies, using various
algorithms and data from 2015 to 2021 [
12
]. Shamima Ahmed et al. investigate AI and
machine learning (ML) in finance, examining 348 articles from 2011 to 2021. It reveals a
growing trend since 2015, showcasing applications in bankruptcy prediction, stock prices,
portfolio management, and more. Leading contributors are the US, China, and the UK [
13
].
Different graph neural networks like Graph Convolutional Network (GCN), Graph Atten-
tion Network (GAT), and Gaussian and Neural Accelerator (GNA) are used, and GCN
and GAT are found to be the predominant choices for stock prediction [
14
]. Abdulhamit
Subasi et al.’s 2021 study evaluates seven classification methods for stock market datasets,
revealing strong predictive performance by random forest and Bagging (93% accuracy) with
leaked data, followed by AdaBoost (82%), while Decision Trees perform less effectively (49%
accuracy) [
15
]. Utilizing ensemble deep learning, the model predicts next-day stock prices
with around 85% accuracy, enhanced by various deep learning methods, outperforming
other techniques and yielding error rates of 0.0% to 13.3% [
16
]. Deep learning algorithms,
notably LSTM and BI-LSTM, significantly influence modern technology, particularly in
time series-based prediction models like stock price prediction, where accurate parameter
adjustment is pivotal for enhancing prediction accuracy [
17
]. Vachhani et al. (2020) provide
a succinct overview of machine learning’s impact on stock market analysis, addressing
limitations, exploring regression models, decision trees, neural networks, and support
vector machines, and emphasizing feature selection and future research directions [18].
AI and ML methods demonstrate the potential of forecasting stock market trends and
prices, necessitating thorough assessment and comparison of models for optimal selection
in distinct situations.
3. Results
The systematic review of AI-based stock market prediction examines performance
metrics, data sources, feature selection, timeframes, and comparison with traditional
methods, and challenges. It offers valuable insights into AI’s effectiveness in predicting
stock prices and guides future research in the field. Table 1provides the comparative
studies of selected papers based on various parameters.
Table 1. Comparative study among selected papers on AI-based stock market predictions.
Year Objective Dataset F* Techniques PT* Metrics Results RG* R*
2023 SM* prediction NA NA ML, ANN, SVM,
NN, LSTM NA NA High accuracy NA* [1]
2023 Stock price,
trend prediction NA Price, Trend BPNN, CNN,
GRU, LSTM NA Accuracy,
Error High accuracy NA [2]
2023 Stock prediction News Price ML, MLP, SVM,
LSTM, ANN M* Accuracy High accuracy
Market data and
text data can lead
to more accuracy
[3]
2023
Enhancing stock
market
anomalies
NA Profit ML M Profit
margin Fitness of model The factor zoo [4]
2022
ML models’
stock market
prediction
NA NA ML, ANN, SVM,
LSTM NA NA NN working
efficiently in depth NA [5]
2023
ML trading
system for
the SM
NASDAQ Price, Trend ML, NPMM,
XGBoost A* Accuracy Labelling is found
productive NA [6]
Eng. Proc. 2023,56, 254 4 of 6
Table 1. Cont.
Year Objective Dataset F* Techniques PT* Metrics Results RG* R*
2023
DRL for stock
portfolio
optimization
NA Portfolio Op-
timization DL, RL, DDPG A, M, W* Sharpe
ratio
Algorithms
outperformed by
the suggested
methods
Dynamically
modifies the
weight
[7]
2023
ML sentiment
analysis and SM
reactions
COVID-19
News, S&P
500
Correlation ML, NLP, BERT A Sentiment
scores
Positively correlated
and statistically
significant
Brief window
examining only [8]
2022
AI-based
day-ahead SM
forecasting
China Stock
Market Profit LSTM, SFLA D Profit
margin
LSTM, AC-SFLA
has high efficiency NA [9]
2023 Stock prediction
and analysis SSE Price LSTM D* Error
MAE of 0.029,
MAPE of 0.61%,
and RMSE of 0.037
Refining the
model
architecture
[
10
]
2022
Automatic stock
selection like
fund managers
China’s A
Share
Market
Profit ML, Scoring,
Screening Model NA Profit
margin
P-return is a notable
increase NA
[
11
]
2022 SM prediction BSE, NSE Profit ML, K-NN, LR,
SVR, DTR, LSTM M Accuracy The LSTM is
outperforming
In time series
data, ML appears
to produce less
reliable
[
12
]
2022 GNN in SMP NA Price GNN, GCN, GAT,
GNA NA NA
GCN and GAT are
the most frequently
utilized
NA
[
14
]
2021
SM prediction
based on ML
Algorithms
NASDAQ,
NYSE,
NIKKEI,
FTSE
Accuracy
DM, RF, SVM,
ANN, Bagging,
AdaBoost, Decision
Trees, K-NN
NA Accuracy
RF, Bagging with a
leaked dataset
results in high
performance
NA
[
15
]
PT*: prediction type, R*: reference, D*: daily, M*: monthly, W*: weekly, A*: annually, RG*: research gaps,
F*: features, SM*: stock market, and NA*: not available.
Indian broking companies are now following these AI and ML trends. Zerodha:
Zerodha may collect and analyze customer data on trading patterns, portfolio performance,
and market sentiment [
19
]. Upstox: Upstox may collect and analyze customer data on
investment preferences, portfolio performance, and market trends [
20
]. ICICI Direct: ICICI
Direct may collect and analyze customer data on investment goals, risk tolerance, and
portfolio diversification [
21
]. HDFC Securities: HDFC Securities may collect and analyze
customer data on investment preferences, portfolio performance, and market trends [
22
].
Motilal Oswal: Motilal Oswal may collect and analyze customer data on investment
preferences, portfolio performance, and market sentiment [23]. 5Paisa: 5Paisa may collect
and analyze customer data on investment preferences, portfolio performance, and market
trends [
24
]. Grow: Grow may collect and analyze customer data on investment goals, risk
tolerance, and portfolio performance [
25
]. The AI-based stock market prediction market is
set to reach USD 7.3 billion by 2024, growing at a CAGR of 32.9%, while AI models like
deep learning and ensemble learning enhance accuracy. Feature selection methods such as
MI-based approaches optimize efficiency, yet challenges include data complexity, model
transparency, and overfitting risks [26].
4. Observations and Discussion
The global AI market is projected to reach USD 267 billion by 2027, contributing USD
15.7 trillion to the global economy by 2030. Around 37% of businesses utilize AI, while
the rise of AI will generate 97 million new jobs and replace 85 million by 2025. Voice
assistant usage is increasing, with over three billion in use and eight billion expected to
be in use by 2023. The AI industry is forecasted to earn USD 126 billion annually by
2025. Additionally, 67% of Americans trust self-driving cars, and 25 countries are actively
developing autonomous vehicles, which could lead to a USD 600 billion industry within
five years [
27
]. Key observations from the studies include the prevalent use of machine
learning techniques like neural networks and support vector machines, diverse data sources
such as news articles and historical prices, varying prediction accuracy, and the demand
for more research to enhance models by incorporating market sentiment and addressing
data imbalance. These findings underscore the increasing interest in machine learning for
Eng. Proc. 2023,56, 254 5 of 6
stock prediction, emphasizing the necessity for refining existing models and exploring
alternative data sources to improve prediction accuracy.
5. Limitations, Future Scope, and Challenges
AI-based stock market prediction faces challenges due to the stock market’s uncer-
tainty and reliance on historical data quality, complex models, and overfitting. Future
opportunities include improving data quality, making AI models interpretable, integrating
with other technologies, and providing personalized recommendations. Challenges involve
regulatory issues, human bias, ethical concerns, and high costs for AI implementation.
6. Conclusions
AI- and ML-based methods hold potential for stock market prediction by analyzing
historical data, but challenges like market unpredictability, data quality, bias, and overfitting
persist. Despite these hurdles, progress is seen in transparent models and the integration of
diverse data sources like social media. With ongoing research, improved accuracy could
aid investor decisions and enhance market performance.
Author Contributions: Conceptualization, R.J.; methodology, R.V.; software, R.J. and R.V.; validation,
R.V.; formal analysis, R.J. and R.V.; investigation, R.V.; resources, R.V.; data curation, R.J. and R.V.;
writing— R.J.; writing—review and editing, R.J.; visualization, R.J. and R.V.; supervision, R.V.; project
administration, R.V. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: No new data created, all the data that is used in this paper is available
on internet as per the citations and their references.
Conflicts of Interest: The authors declare no conflict of interest.
References
1.
Mintarya, L.N.; Halim, J.N.; Angie, C.; Achmad, S.; Kurniawan, A. Machine learning approaches in stock market prediction: A
systematic literature review. Procedia Comput. Sci. 2023,216, 96–102. [CrossRef]
2.
Chaudhari, K.; Thakkar, A. Neural network systems with an integrated coefficient of variation-based feature selection for stock
price and trend prediction. Expert Syst. Appl. 2023,219. [CrossRef]
3.
Maqbool, J.; Aggarwal, P.; Kaur, R.; Mittal, A.; Ganaie, I.A. Stock Prediction by Integrating Sentiment Scores of Financial News
and MLP-Regressor: A Machine Learning Approach. Procedia Comput. Sci. 2023,218, 1067–1078. [CrossRef]
4.
Azevedo, V.; Hoegner, C. Enhancing stock market anomalies with machine learning. Rev. Quant. Finance Account. 2022,60,
195–230. [CrossRef]
5.
Chhajer, P.; Shah, M.; Kshirsagar, A. The applications of artificial neural networks, support vector machines, and long–short term
memory for stock market prediction. Decis. Anal. J. 2021,2, 100015. [CrossRef]
6.
Han, Y.; Kim, J.; Enke, D. A machine learning trading system for the stock market based on N-period Min-Max labeling using
XGBoost. Expert Syst. Appl. 2023,211, 118581. [CrossRef]
7.
Jang, J.; Seong, N. Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory.
Expert Syst. Appl. 2023,218, 119556. [CrossRef]
8. Costola, M.; Hinz, O.; Nofer, M.; Pelizzon, L. Machine learning sentiment analysis, COVID-19 news and stock market reactions.
Res. Int. Bus. Finance 2023,64, 101881. [CrossRef] [PubMed]
9.
Luo, J.; Zhu, G.; Xiang, H. Artificial Intelligent based day-ahead stock market profit forecasting. Comput. Electr. Eng. 2022,99,
107837. [CrossRef]
10.
Lin, H.; Chen, X.; Chui, S.Y. Stock prediction and analysis using LSTM network. In Proceedings of the SPIE 12510, International
Conference on Statistics, Data Science, and Computational Intelligence (CSDSCI 2022), Zhengzhou, China, 25–27 August 2022.
[CrossRef]
11.
Li, Y.; Fu, K.; Zhao, Y.; Yang, C. How to make machine select stocks like fund managers? Use scoring and screening model. Expert
Syst. Appl. 2022,196, 116629. [CrossRef]
12.
Bansal, M.; Goyal, A.; Choudhary, A. Stock Market Prediction with High Accuracy using Machine Learning Techniques. Procedia
Comput. Sci. 2022,215, 247–265. [CrossRef]
Eng. Proc. 2023,56, 254 6 of 6
13.
Ahmed, S.; Alshater, M.M.; El Ammari, A.; Hammami, H. Artificial intelligence and machine learning in finance: A bibliometric
review. Res. Int. Bus. Finance 2022,61, 101646. [CrossRef]
14.
Zhang, W.; Chen, Z.; Miao, J.; Liu, X. Research on Graph Neural Network in Stock Market. Procedia Comput. Sci. 2022,214,
786–792. [CrossRef]
15.
Subasi, A.; Amir, F.; Bagedo, K.; Shams, A.; Sarirete, A. Stock Market Prediction Using Machine Learning. Procedia Comput. Sci.
2021,194, 173–179. [CrossRef]
16.
Vaishali, I.; Sachin, D. SPDL-Stock Price Prediction with Deep Learning (June 26, 2021). In Proceedings of the 3rd International
Conference on Communication & Information Processing (ICCIP) The Nutan College of Engineering and Research (NCER),
Talegaon Dabhade Pune, India, 26–27 June 2021; Available online: https://ssrn.com/abstract=3911033 (accessed on 1 July 2023).
17.
Sunny, A.I.; Maswood, M.M.S.; Alharbi, A.G. Deep Learning-Based Stock Price Prediction Using LSTM and Bi-Directional LSTM
Model. In Proceedings of the 2020 2nd Novel Intelligent and Leading Emerging Sciences Conference (NILES), Giza, Egypt, 24–26
October 2020; pp. 87–92. [CrossRef]
18.
Vachhani, H.; Obaidat, M.S.; Thakkar, A.; Shah, V.; Sojitra, R.; Bhatia, J.; Tanwar, S. Machine Learning Based Stock Market
Analysis: A Short Survey. In Innovative Data Communication Technologies and Application. ICIDCA 2019. Lecture Notes on Data
Engineering and Communications Technologies; Raj, J., Bashar, A., Ramson, S., Eds.; Springer: Cham, Switzerland, 2020; Volume 46.
[CrossRef]
19. Zerodha Technology. Available online: https://zerodha.com/technology (accessed on 18 August 2023).
20.
Upstox Pro: Trading App for the New Generation. Available online: https://upstox.com/products/upstox-pro/ (accessed on 18
August 2023).
21.
ICICI Direct: Online Share Trading and Investment Platform. Available online: https://www.icicidirect.com/idirectcontent/
Home/Home.aspx (accessed on 18 August 2023).
22.
HDFC Securities: Online Trading and Stock Broking Services. Available online: https://www.hdfcsec.com/ (accessed on 18
August 2023).
23.
Motilal Oswal Research and Analytics. Available online: https://www.motilaloswal.com/research-and-analytics (accessed on 18
August 2023).
24.
5Paisa.com: Best Online Share Trading & Stock Broking App in India. Available online: https://www.5paisa.com (accessed on 18
August 2023).
25. Groww: Invest in Mutual Funds & Stocks. Available online: https://groww.in/ (accessed on 18 August 2023).
26. Market Watch: Stock Market News. Available online: https://www.marketwatch.com/ (accessed on 18 August 2023).
27.
55 Fascinating AI Statistics and Trends for 2023. Available online: https://dataprot.net/statistics/ai-statistics (accessed on 18
August 2023).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
