Conference PaperPDF Available

Predicting Customer Churn in E-commerce Subscription Services using RNN with Attention Mechanisms

Authors:
  • Indian Institute of Banking & Finance

Abstract and Figures

In e-commerce subscription services, predicting client attrition is essential for businesses trying to boost sales and enhance client retention. This article proposes a unique approach for accurate churn prediction: recurrent neural networks (RNNs) plus attention mechanisms. By taking advantage of the temporal dependencies found in subscription-based interactions, the RNN model dynamically learns and concentrates on important features. This enables it to detect minute patterns that can point to potential churn behavior in the future. This study thoroughly examines churn characteristics in e-commerce subscription services in order to determine the effectiveness of the proposed RNN architecture in anticipating client attrition. Experiments on real datasets show that the model works better than traditional methods, achieving notable accuracy and predictive power. Moreover, the attention processes in the model allow it to rank the informative traits in order of importance, providing insight into the fundamental causes of customer attrition. The results of this study establish a solid framework for proactive churn control and long-term growth, which has important ramifications for e-commerce companies. Businesses may maximize resource allocation, raise customer happiness, and develop enduring connections with their subscribers by utilizing the predictive powers of RNNs with attention mechanisms. Python is used to implement the suggested work. The accuracy of the suggested study is 97%.
Content may be subject to copyright.
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Predicting Customer Churn in E-commerce
Subscription Services using RNN with Attention
Mechanisms
1Ch. Anudeep
GITAM School of Business
GITAM University
Visakhapatnam, India
ch.anudeep379@gmail.com
4Thiruma Valavan A
Deputy Director,
Indian Institute of Banking & Finance,
Mumbai, India
thirumavalavan63744@gmail.com
2R. Venugopal
Dept of Marketing
GITAM School of Business,
GITAM (Deemed to be) University,
Visakhapatnam, India
vreddi@gitam.edu
5Veera Ankalu. Vuyyuru
Department of CSE
Koneru Lakshmaiah Education Foundation,
Vaddeswaram, Andhra Pradesh, India.
veerankalu14@kluniversity.in
3Mohd Aarif
Department of Commerce
Aligarh Muslim University,
Aligarh, Uttar Pradesh, India
arifgc6036@gmail.com
6S Muthuperumal
Department of Mathematics,
K.Ramakrishnan College of Engineering,
Trichy, India
muthuperumal@krce.ac.in
Abstract In e-commerce subscription services, predicting
client attrition is essential for businesses trying to boost sales
and enhance client retention. This article proposes a unique
approach for accurate churn prediction: recurrent neural
networks (RNNs) plus attention mechanisms. By taking
advantage of the temporal dependencies found in subscription-
based interactions, the RNN model dynamically learns and
concentrates on important features. This enables it to detect
minute patterns that can point to potential churn behavior in
the future. This study thoroughly examines churn
characteristics in e-commerce subscription services in order to
determine the effectiveness of the proposed RNN architecture in
anticipating client attrition. Experiments on real datasets show
that the model works better than traditional methods, achieving
notable accuracy and predictive power. Moreover, the attention
processes in the model allow it to rank the informative traits in
order of importance, providing insight into the fundamental
causes of customer attrition. The results of this study establish a
solid framework for proactive churn control and long-term
growth, which has important ramifications for e-commerce
companies. Businesses may maximize resource allocation, raise
customer happiness, and develop enduring connections with
their subscribers by utilizing the predictive powers of RNNs
with attention mechanisms. Python is used to implement the
suggested work. The accuracy of the suggested study is 97%.
Keywords Attention Mechanisms, Churn Prediction, E-
commerce, Recurrent Neural Networks, Subscription Services.
I. INTRODUCTION
The manner that customers obtain goods and services
online has been completely transformed by e-commerce
subscription services. For a subscription fee, these services
allow users to conveniently get things on a regular basis,
frequently on a repeating basis. The selection of subscription
items is enormous and is growing, ranging from software and
streaming services to meal packages and cosmetics.
Personalizationthe ability to customize subscriptions to a
user's tastes and needsis one of the main draws of e-
commerce subscription services. Consumers may take
advantage of personalized product recommendations,
automated delivery, and frequently lower prices as compared
to in-store shopping [1]. Furthermore, subscription models
encourage a feeling of commitment and loyalty between
companies and clients, which may result in increased lifetime
value and recurrent income streams. Long-term client
retention is a major difficulty for e-commerce subscription
services, notwithstanding their benefits. One of the most
important metrics for subscription-based organizations is
customer churn, or the rate at which users cancel their
memberships. It is critical to anticipate client attrition for a
number of reasons. First of all, it enables businesses to foresee
at-risk clients and take proactive measures to retain them
before they go. Businesses may address underlying issues like
product discontent, price concerns, or bad customer
experiences by recognizing the variables that contribute to
churn. Furthermore, lowering churn lowers expenses related
to bringing in new clients while simultaneously protecting
current revenue. Keeping up a high rate of client retention
might also eventually result in increased profitability and
steady growth. Therefore, accurate churn prediction models
are essential for e-commerce subscription services to optimize
customer retention efforts and maximize long-term
profitability [2]. RNNs have become highly effective
instruments for sequential data processing, which makes them
ideal for modelling time-series data, including consumer
interactions in e-commerce subscription services.
Recognizing temporal relationships within sequences, RNNs
can successfully simulate patterns and trends over time,
unlike standard feedforward neural networks [3]. This feature
is very helpful for churn prediction, as past customer
behaviour is a key factor in predicting upcoming churn
occurrences. Moreover, RNN architectures like GRU and
LSTM are made to deal with the vanishing gradient issue,
which makes it possible for them to remember information
across extended sequences and improve upon previous
encounters [4]. Attention methods provide a valuable
enhancement to RNNs in the context of customer churn
prediction by allowing the model to concentrate on the most
pertinent segments of the input sequence. Through the
assignment of relevance weights to various input data pieces,
attention methods imitate human attention, enabling the model
to selectively pay to important qualities while ignoring
irrelevant information. This capacity is particularly useful for
e-commerce subscription services because client behaviour
might differ greatly in terms of how frequently, recently, and
intensely they connect. Churn prediction models
can dynamically adjust to the trajectories of individual
customers by integrating attention mechanisms into RNN
architectures. This allows the models to detect tiny signals that
may indicate an approaching churn. Because of this, attention-
enhanced RNNs provide a framework for forecasting
customer churn in e-commerce subscription services that is
easier to understand and use, enabling companies to prevent
revenue loss and actively retain important customers.
The structure of the paper is as follows. The beginning of
the first part highlights the significance of customer churn
prediction in e-commerce subscription services. The literature
review is covered in Section 2. The available approaches'
limitations are enumerated in Section 3. In Section 4, the
RNNs with attention mechanisms technique for improved
churn prediction is covered. The findings and suggestions are
given in Section 5. Conclusion and more research in Section
6.
II. LITERATURE REVIEW
Artificial intelligence is a critical answer to the rising
rates of customer attrition in sectors such as banking, where
recruiting new users is more expensive than maintaining
current ones. In this study, RNNs and convolutional neural
networks are integrated in conjunction using BiLSTM-CNN
[5]. This combination fixes the supervision problem in
DLCNN models and overcomes the drawbacks of individual
RNNs and CNNs too. This study uses bank data for
comparison analysis to investigate the possible advantages of
the AttnBLSTM-CNN over BiLSTM-CNN. The integration
of attention mechanisms enhances model performance,
enabling advanced warning and pre-emptive measures
against user churn. This not only safeguards against customer
loss but also enhances the core competitiveness of financial
institutions [6]. Transformers are an excellent choice for NPD
prediction because of their ability to handle a variety of
patterns seen in e-commerce data, like as trends, seasonality,
and abnormalities. The transformer model outperforms
baseline techniques in terms of prediction accuracy.
Moreover, a clustered transformer model is presented, which
groups comparable clients according to their purchase
patterns in order to improve accuracy. By highlighting the
possibilities of transformer topologies for NPD prediction,
this method provides e-commerce companies looking to
enhance inventory management and marketing tactics with a
flexible and scalable solution. The study's conclusions
advance the field of predictive modelling in e-commerce and
offer useful information to companies looking to increase
revenue and improve customer happiness through focused
marketing campaigns and customized customer experiences
[7]. Predicting churn rate is crucial for the success of the
telecommunications industry, as it directly impacts
profitability. This study addresses the challenge of accurately
predicting customer churn while maximizing profits. Two
publicly available datasets that reflect the telecom markets in
Southeast Asia and America are used in the study. The
outcomes show that on both datasets, CNN [8] and ANN
perform better than other methods. In comparison, ANN
obtains 98% accuracy for the Southeast Asian dataset and
99% accuracy for the American dataset, whereas CNN
achieves 99% accuracy for both datasets. These results
demonstrate the efficacy of deep learning methods, in
particular CNN and ANN, in precisely forecasting customer
attrition in telecoms. For telecom businesses looking to
improve profitability and optimize client retention tactics, the
suggested models provide insightful information. Telecom
operators may proactively identify at-risk clients and perform
tailored interventions to minimize churn and boost customer
satisfaction by utilizing powerful machine learning
algorithms [9]. The literature review covers papers focusing
on advanced machine learning techniques for predictive
analytics in customer churn. By combining bidirectional
LSTM-CNN models with attention mechanisms for customer
loss prediction, the study shows enhanced performance and
competitive advantage for financial organizations. In
contrast, Leon et al., introduces a transformer-based model
for predicting customers' next purchase day in e-commerce,
showcasing superior accuracy and scalability compared to
traditional methods. Finally, explores other approaches to
learning, such as deep learning and ensemble methods, for
anticipating churn in the telecom sector. Top-performing
models CNN and ANN provide high accuracy and provide
useful insights for telecom firms seeking to improve revenue
and optimize client retention tactics. The review underscores
the growing significance of advanced machine learning
techniques in predictive analytics, enabling businesses to
effectively tackle customer issues and enhance operational
efficiency.
III. PROBLEM STATEMENT
The existing systems for predicting customer churn in e-
commerce subscription services often face several limitations
that hinder their effectiveness. Traditional methods may rely
solely on static features, such as demographics or purchase
history, which fail to capture the dynamic nature of customer
behaviour over time. This approach overlooks important
temporal patterns and trends that could be indicative of churn
risk [10]. To overcome these limitations, the proposed work
utilizes RNNs with Attention Mechanisms, providing several
advantages. More precise forecasts of churn events are made
possible by the model's ability to grasp temporal relationships
and sequential patterns in customer data by integrating RNNs.
The model may prioritize significant characteristics and
disregard noise or extraneous information by focusing on
pertinent portions of the input sequence, which is made
possible by the attention processes. Through the identification
of critical churn contributing components, this attention
mechanism enhances the interpretability of the model and
helps solve the issue of information saturation in e-commerce
data. The proposed study is to give e-commerce firms a strong
tool for proactive churn control by solving these shortcomings
of current systems, which will eventually increase customer
retention and company sustainability.
IV. RNN WITH ATTENTION MECHANISMS FOR
PREDICTING CUSTOMER CHURN
The proposed work presents a new method for predicting
customer attrition in e-commerce subscription services,
combining RNNs with attention mechanisms. It shows better
predictive performance than conventional techniques,
enabling better churn mitigation plans and improved client
retention. The framework is validated using actual
subscription data. The RNN model with attention mechanisms
for forecasting customer churn in e-commerce subscription
services is developed and evaluated using some essential
phases in the methodology of the proposed study. First, data
collection entails gathering relevant datasets containing
information about subscriber interactions, such as transaction
histories, browsing activity, and engagement metrics.
Preprocessing steps follow, creation of time-based features, to
prepare the dataset for model training. Next, the RNN
architecture with attention mechanisms is designed and
implemented. The attention mechanism enables the model to
concentrate on significant characteristics and patterns
throughout the input sequences, while the RNN component
records temporal relationships in the sequential data. The
prepared dataset is then used to train the model, and
backpropagation and gradient descent are used to optimize its
parameters. Several measures are used to assess the model's
performance. It may be possible to do experiments to evaluate
the suggested model's predictive power of customer attrition
by contrasting it with baseline and cutting-edge approaches.
Sensitivity assessments can also be carried out to investigate
how various hyperparameters and architectural decisions
affect the functionality of the model. Finally, the results are
interpreted, and insights are drawn regarding the factors
influencing customer churn in e-commerce subscription
services, along with implications for business strategies and
future research directions. Fig. 1 depicts the proposed work
framework.
Fig. 1. Proposed Framework for RNN with Attention Mechanisms for
Predicting Customer Churn
A. Data Collection
For the proposed work on churn prediction in an
Ecommerce subscription services, the dataset used is the
churn-modelling dataset. This dataset is collected to analyse
customer behaviour and predict churn rates within the
ecommerce industry. It contains various attributes related to
customers, including their demographics, banking activity,
and account status. The dataset comprises several key
features, including Customer Id, Surname, Credit Score,
Geography, Gender, Age, Tenure, Balance, Num of Products,
Has Cr Card, Is Active Member, Estimated Salary, Exited
[11].
B. Data Pre-processing
For the proposed study, a unique strategy is to capture the
time dynamics of consumer behaviour by integrating temporal
aspects in addition to conventional data pretreatment
approaches. To accomplish this, time-related features that
encapsulate temporal patterns and trends must be created.
These characteristics can offer important insights on the
frequency and timing of customer churn events. The churn
prediction model incorporates time-based features like Time
since Last Interaction, Seasonality Indicators, and Temporal
Aggregations to better understand customer behaviour. These
features capture the duration of customer interactions,
seasonal variations, and trends over different time intervals.
This enhances predictive accuracy and robustness in
identifying at-risk customers, thereby improving the model's
predictive accuracy.
C. RNN with Attention Mechanism for Churn prediction
Designed to manage sequential input by maintaining
information across time, RNNs are a kind of neural networks.
RNNs can display temporal dynamic behaviors because they
feature connections that create directed cycles, in contrast to
feedforward neural networks, whereby information travels in
a single direction. An RNN's basic building block is a series
of repeated neural network phases, each of which processes
just one component of the process sequence while preserving
a hidden state that holds data from earlier elements. Every
phase , the hidden state of an RNN cell is computed using
the equation (1).
󰇛  󰇜 (1)
Where, is the input vector at phase ,  is the
hidden state vector from the previous time step,  and 
are weight matrices for the input and hidden state,
respectively, is the bias vector,  is the hyperbolic
tangent activation function. In churn prediction, RNNs can be
applied to sequential data representing customer interactions
or behaviors over time. The objective is to build an RNN
model to determine, from past behavior, whether a customer
will be inclined to return or not. The churn-modelling dataset,
containing customer attributes and historical transaction data,
is pre-processed to create sequences of input features and
corresponding target labels. The input sequences may include
features such as customer demographics, transactional
history, account activity, and temporal information. The
target labels indicate whether a customer has churned (1) or
not (0) within a certain time frame. The RNN model
architecture consists of multiple recurrent layers, typically
implemented using LSTM or GRU cells. During the training
process, the RNN model learns to predict churn based on
patterns and dependencies in the input sequences. The model
is trained using BPTT, an extension of backpropagation that
unfolds the computational graph over time steps. After
training, the trained RNN model is used to make predictions
on new sequences of input features. The predicted churn
probabilities are compared to a threshold (e.g., 0.5) to
determine the predicted churn class (churn or not churn).
Three major parts constitute an RNN's architecture are, the
input layer, the recurrent layer, and the output layer. The
input layer receives sequential data representing customer
interactions with the bank over time. This could include
features such as transaction histories, account activity,
customer demographics, and temporal information [12]. Each
phase at a time, the recurrent layer examines the input
sequence, changing its hidden state based on the input at that
point and its earlier hidden state. A crucial element of the
model design is the attention mechanism, which enables the
network to concentrate on pertinent segments of the sequence
of inputs during prediction. The attention mechanism
determines the relevance or importance of each time step's
information for the prediction task by computing attention
weights for each time step in the input sequence. The most
informative portions of the input sequence are essentially
highlighted by using these attention weights to produce a
weighted sum of the sequence. Predictions are generated by
the output layer using the data contained in the attention-
weighted input sequence. A binary classifier that uses the
learnt representation of the input sequence to determine
whether or not a customer is likely to churn (1) or not (0) may
be used as the output layer in the framework of customer
retention forecasting. The attention weight  for element
at time step is given in eqn. (2).
 󰇛󰇜
󰇛󰇜
 (2)
Where, is the length of the input sequence. Attention
mechanisms play a crucial role in capturing relevant temporal
patterns and features in sequential data, such as customer
transaction histories or account activity. Time steps with
higher attention weights receive more emphasis during the
computation of the final prediction, allowing the model to
prioritize informative features and ignore irrelevant or noisy
inputs. Attention mechanisms in RNNs create interpretable
representations of input sequences, highlighting important
temporal patterns and features. This transparency helps
stakeholders understand customer behaviors driving
predictions and aids decision-making in customer retention
strategies. Attention-based models achieve better prediction
accuracy, precision, and recall rates compared to traditional
RNN architectures without attention mechanisms. Attention
weights are computed using a learned function, assigning
higher weights to relevant time steps, allowing the model to
focus on important temporal patterns and features. In the
proposed work, RNNs with attention mechanisms are
integrated to utilize the strengths of both approaches for
customer churn prediction. The RNN processes the
sequential customer behaviors data, capturing temporal
dependencies and learning a representation of the input
sequence. The attention mechanism then dynamically focuses
on relevant parts of the input sequence, enhancing the model's
ability to capture informative temporal patterns and features
for churn prediction. By combining RNNs with attention
mechanisms, the proposed model can provide accurate
predictions of customer churn and valuable insights into the
factors driving churn behaviors in the banking industry. This
integrated approach offers a powerful framework for
developing proactive customer retention strategies and
reducing churn rates, ultimately leading to improved
customer satisfaction and business performance. The model
architecture of an RNN with attention mechanisms for
customer churn prediction involves integrating attention
mechanisms into the standard RNN architecture to capture
relevant temporal patterns and features more effectively. In
the proposed work for predicting customer churn in e-
commerce subscription services using RNN with attention
mechanisms, the attention mechanism plays a crucial role in
determining the importance or relevance of each element in
the input sequence. The attention mechanism computes
attention weights for each element of the sequence, allowing
the model to focus more on informative features while
making predictions. By integrating attention mechanisms into
the RNN architecture, the model gains the ability to
dynamically focus on informative features or interactions
within the customer's historical data. This allows the model
to allocate more attention to relevant events or patterns that
may precede churn events, leading to improved prediction
accuracy and interpretability. Additionally, attention weights
provide insights into the factors driving churn, enabling
businesses to identify actionable strategies for retaining at-
risk customers. RNNs with attention mechanisms offer a
powerful framework for predicting customer churn in e-
commerce subscription services. By leveraging sequential
data on customer interactions and behaviors, RNN models
can effectively capture temporal dependencies and identify
patterns indicative of potential churn events. Integration of
attention mechanisms further enhances the model's predictive
capabilities by allowing it to focus on the most relevant
features within the input sequence. By applying RNNs with
attention mechanisms, businesses can gain valuable insights
into customer churn behaviors and implement targeted
retention strategies to mitigate churn and improve customer
satisfaction.
V. RESULTS AND DISCUSSION
In the proposed work, the results of applying the
RNN model with attention mechanisms for customer churn
prediction in e-commerce subscription services are highly
promising. The model demonstrates significant
improvements in predictive accuracy and effectiveness
compared to traditional methods. Through extensive
experimentation and evaluation on real-world datasets, the
RNN model achieves superior performance in identifying
customers at risk of churn. The incorporation of attention
mechanisms into the RNN architecture enhances the model's
interpretability and ability to capture subtle temporal patterns
in customer behaviors. The attention weights provide
valuable insights into the features and time points that are
most influential in predicting churn, empowering businesses
to understand the underlying drivers of customer attrition and
implement targeted retention strategies. In general, the
findings demonstrate how well the suggested strategy
predicts user attrition in e-commerce subscription services
and provide useful information for enhancing client retention
and boosting revenue.
A. Performance Evaluation
Performance evaluation for the proposed work involves
assessing the effectiveness and robustness of the RNN model
with attention mechanisms for predicting customer churn in
e-commerce subscription services. Several evaluation
metrics are used to measure the model's performance and
compare it against baseline methods. Evaluation metrics such
as accuracy, precision, recall, and F1-score show the model's
ability to accurately predict churn events and distinguish
between churners and non-churners.
TABLE I. COMPARISON OF EXISTING SYSTEM PERFORMANCE WITH
PROPOSED RNN WITH ATTENTION MECHANISMS
Model
Accuracy
Precision
Recall
F1-
score
RF [13]
91%
90%
78%
83%
AttnBLSTM-
CNN [14]
95%
95%
96%
95%
Proposed RNN
with Attention
Mechanism
97%
97%
96%
96%
Table I shows performance metrics of three models for
predicting customer churn in e-commerce subscription
services: RF, AttnBLSTM-CNN, and RNN with Attention
Mechanism. RF has a 91% accuracy, but may miss some
instances. AttnBLSTM-CNN outperforms RF with 95%
accuracy, precision, recall, and F1-score. RNN with Attention
Mechanism has the highest performance with 97% accuracy,
precision, recall, and F1-score. It effectively identifies churn
instances and minimizes false positives.
Fig. 2. Performance Evaluation of Proposed Model with Existing Model
The performance metrics of three distinct modelsRF,
AttnBLSTM-CNN, and RNN with Attention Mechanism
for forecasting customer attrition in e-commerce subscription
services are shown in Figure 2. Higher accuracy numbers
denote superior performance. Accuracy quantifies the total
correctness of forecasts. The percentage of accurately
anticipated churn instances among all projected churn
instances is known as precision, whereas the percentage of
properly predicted churn instances among all real churn events
is known as recall. The F1-score offers a fair assessment of a
model's performance as it is the harmonic mean of accuracy
and recall. The RNN with Attention Mechanism beats the
other models in every statistic, including accuracy, precision,
recall, and F1-score, as shown in the image. This shows that
the RNN model with attention mechanisms may more
accurately forecast customer turnover in e-commerce
subscription services by capturing temporal relationships and
focusing on pertinent elements within the input sequence.
B. Discussion
The proposed work on predicting customer churn in e-
commerce subscription services using RNN with attention
mechanisms, it's important to address both the limitations of
existing systems and the advantages of the proposed
approach, as well as identify potential areas for future
research. Existing churn prediction systems often rely on
traditional machine learning algorithms or simple heuristics,
which may struggle to capture complex temporal
dependencies and subtle patterns in customer behaviors [15].
Additionally, these systems may overlook the importance of
feature attention, leading to suboptimal performance in
identifying key factors driving churn. Moreover, scalability
and interpretability issues may arise when dealing with large-
scale datasets and complex models. The suggested study has
a number of benefits over current systems. The model may
more accurately forecast customer turnover by focusing on
pertinent aspects within the sequential data and capturing
temporal relationships through the use of RNNs with
attention mechanisms. By include attention processes, the
model becomes easier to understand and gives organizations
more insight into the variables that affect churn behavior.
Moreover, the suggested methodology exhibits great
scalability and adeptness in managing extensive datasets,
rendering it appropriate for practical implementations in e-
commerce subscription services. Notwithstanding its
benefits, the suggested job has several drawbacks. One
limitation is the potential for overfitting, especially when
dealing with limited training data or highly imbalanced
datasets. Addressing this limitation may require techniques
such as data augmentation, regularization, or ensemble
methods. The proposed approach may be computationally
intensive, particularly when training deep neural networks on
large-scale datasets. Future work could focus on optimizing
model architectures and training procedures to improve
efficiency and scalability. Furthermore, exploring the impact
of additional features or external factors on customer churn
prediction could enhance the model's performance and
provide deeper insights into customer behaviors. Finally,
evaluating the proposed approach on diverse datasets from
different e-commerce domains and subscription services
could further validate its effectiveness and generalizability.
VI. CONCLUSION AND FUTURE SCOPE
The proposed work on leveraging RNN with attention
mechanisms to predict client churn in e-commerce
subscription services has led to a substantial advancement in
client relationship management and retention strategies. The
model performs better in capturing temporal relationships
and detecting critical elements driving churn behavior by
utilizing RNNs with attention mechanisms. By incorporating
attention processes, the model becomes more interpretable
and offers insightful information about the most significant
elements in the input sequence. This enables companies to
develop focused retention campaigns and successfully reduce
customer attrition. The suggested method's efficiency and
scalability make it ideal for real-world applications in e-
commerce subscription services, where complicated
consumer interactions and big datasets are commonplace.
The model is a useful tool for companies looking to increase
revenue and improve customer retention because of its
0
20
40
60
80
100
120
Accuracy Precision Recall F1-score
Percentage
Performance Evaluation
RF
AttnBLSTM-CNN
Proposed RNN with Attention Mechanism
versatility in handling various data sources and capacity to
adjust to shifting customer behavior patterns. There are
numerous directions in which research and development can
go. First off, investigating more complex RNN and attention
mechanism architectures and methodologies may help to
boost model performance and prediction accuracy even more.
Furthermore, carrying out longitudinal research to assess the
long-term efficacy of retention tactics put into place in
accordance with the model's predictions may offer insightful
and helpful criticism for continued optimization. In addition
to opening the door for further study and advancement in this
crucial field of customer relationship management, the
suggested work establishes the groundwork for efficient
churn prediction and customer retention methods in e-
commerce subscription services.
REFERENCES
[1] S. F. Bischof, T. M. Boettger, and T. Rudolph, “Curated subscription
commerce: A theoretical conceptualization,” Journal of Retailing and
Consumer Services, vol. 54, p. 101822, 2020.
[2] P. Lalwani, M. K. Mishra, J. S. Chadha, and P. Sethi, “Customer churn
prediction system: a machine learning approach,” Computing, vol. 104,
no. 2, pp. 271294, 2022.
[3] R. Sudharsan and E. Ganesh, “A Swish RNN based customer churn
prediction for the telecom industry with a novel feature selection
strategy,” Connection Science, vol. 34, no. 1, pp. 18551876, 2022.
[4] A. Khodadadi, S. A. Hosseini, E. Pajouheshgar, F. Mansouri, and H.
R. Rabiee, “ChOracle: A unified statistical framework for churn
prediction,” IEEE Transactions on Knowledge and Data Engineering,
vol. 34, no. 4, pp. 16561666, 2020.
[5] Y. Liu, M. Shengdong, G. Jijian, and N. Nedjah, “Intelligent prediction
of customer churn with a fused attentional deep learning model,”
Mathematics, vol. 10, no. 24, p. 4733, 2022.
[6] G. Mena, K. Coussement, K. W. De Bock, A. De Caigny, and S.
Lessmann, “Exploiting time-varying RFM measures for customer
churn prediction with deep neural networks,” Annals of Operations
Research, pp. 123, 2023.
[7] A. Grigoraș and F. Leon, “Transformer-Based Model for Predicting
Customers’ Next Purchase Day in e-Commerce,” Computation, vol.
11, no. 11, p. 210, 2023.
[8] M. U. Tariq, M. Babar, M. Poulin, and A. S. Khattak, “Distributed
model for customer churn prediction using convolutional neural
network,” Journal of Modelling in Management, vol. 17, no. 3, pp.
853863, 2022.
[9] L. Saha, H. K. Tripathy, T. Gaber, H. El-Gohary, and E.-S. M. El-
kenawy, “Deep churn prediction method for telecommunication
industry,” Sustainability, vol. 15, no. 5, p. 4543, 2023.
[10] L. Geiler, S. Affeldt, and M. Nadif, “A survey on machine learning
methods for churn prediction,” International Journal of Data Science
and Analytics, vol. 14, no. 3, pp. 217242, 2022.
[11] “Churn_Modelling_dataset.” Accessed: Apr. 22, 2024. [Online].
Available: https://www.kaggle.com/datasets/nazishjaveed/churn-
modelling-dataset?resource=download
[12] Z. Cuiyan, Z. Manman, X. Xiaoling, M. Yiwei, and C. Hao, “Customer
Churn Prediction Model Based on User Behavior Sequences.,” Journal
of Donghua University, vol. 39, no. 6, 2022.
[13] C. Ling, T. Zhang, and Y. Chen, “Customer purchase intent prediction
under online multi-channel promotion: A feature-combined deep
learning framework,” IEEE Access, vol. 7, pp. 112963112976, 2019.
[14] K. Weterings, S.-L. Kimelman, S. Bromuri, and M. van Eekelen,
“Interpreting Attention Models: LSTM vs. CNN: A case study on
customer activation,” in 2020 19th IEEE International Conference on
Machine Learning and Applications (ICMLA), IEEE, 2020, pp. 336
343.
[15] J. Ahn, J. Hwang, D. Kim, H. Choi, and S. Kang, “A survey on churn
analysis in various business domains,” IEEE Access, vol. 8, pp.
220816220839, 2020.
Conference Paper
Abstract: In recent years, customer churn prediction in subscription based-business is more essential for improving retention policies. Moreover, various traditional methods are failed to address difficulties such as data quality and struggled to understand customer behavior dynamics. To resolve these issues Embedding Language Model with Chi Square Test (ELMo-CST) is proposed to predict customer churns in subscription based-business. Initially, data imputation and z-score normalization techniques are used to preprocess data for improving the performance of the model. Further, Term Frequency-Inverse Document Frequency (TF-IDF) is employed to extract Call Detail Record (CDR) features by estimating the important terms in the customer feedback. Then, the proposed ELMo-CST is used for predicting the customer churns effectively where ELMo is used to create feature embeddings for contextual representation by capturing sematic meanings on the basis of context. Moreover, CST is utilized for evaluating the relationships among CDR features and churn status by assessing the difference among the categorical variables for efficient prediction. From the results, proposed ELMo-CST established better results including accuracy (ACC) of 98.52%, precision (PR) of 93.86%, recall (RE) of 90.15% and F1-score of 0.93 respectively than existing Random Forest (RF).
Article
Full-text available
The paper focuses on predicting the next purchase day (NPD) for customers in e-commerce, a task with applications in marketing, inventory management, and customer retention. A novel transformer-based model for NPD prediction is introduced and compared to traditional methods such as ARIMA, XGBoost, and LSTM. Transformers offer advantages in capturing long-term dependencies within time series data through self-attention mechanisms. This adaptability to various time series patterns, including trends, seasonality, and irregularities, makes them a promising choice for NPD prediction. The transformer model demonstrates improvements in prediction accuracy compared to the baselines. Additionally, a clustered transformer model is proposed, which further enhances accuracy, emphasizing the potential of this architecture for NPD prediction.
Article
Full-text available
Deep neural network (DNN) architectures such as recurrent neural networks and transformers display outstanding performance in modeling sequential unstructured data. However, little is known about their merit to model customer churn with time-varying data. The paper provides a comprehensive evaluation of the ability of recurrent neural networks and transformers for customer churn prediction (CCP) using time-varying behavioral features in the form of recency, frequency, and monetary value (RFM). RFM variables are the backbone of CCP and, more generally, customer behavior forecasting. We examine alternative strategies for integrating time-varying and non-variant customer features in one network architecture. In this scope, we also assess hybrid approaches that incorporate the outputs of DNNs in conventional CCP models. Using a comprehensive panel data set from a large financial services company, we find recurrent neural networks to outperform transformer architectures when focusing on time-varying RFM features. This finding is confirmed when time-invariant customer features are included, independent of the specific form of feature integration. Finally, we find no statistical evidence that hybrid approaches (based on regularized logistic regression and extreme gradient boosting) improve predictive performance—highlighting that DNNs and especially recurrent neural networks are suitable standalone classifiers for CCP using time-varying RFM measures.
Article
Full-text available
Being able to predict the churn rate is the key to success for the telecommunication industry. It is also important for the telecommunication industry to obtain a high profit. Thus, the challenge is to predict the churn percentage of customers with higher accuracy without comprising the profit. In this study, various types of learning strategies are investigated to address this challenge and build a churn predication model. Ensemble learning techniques (Adaboost, random forest (RF), extreme randomized tree (ERT), xgboost (XGB), gradient boosting (GBM), and bagging and stacking), traditional classification techniques (logistic regression (LR), decision tree (DT), and k-nearest neighbor (kNN), and artificial neural network (ANN)), and the deep learning convolutional neural network (CNN) technique have been tested to select the best model for building a customer churn prediction model. The evaluation of the proposed models was conducted using two pubic datasets: Southeast Asian telecom industry, and American telecom market. On both of the datasets, CNN and ANN returned better results than the other techniques. The accuracy obtained on the first dataset using CNN was 99% and using ANN was 98%, and on the second dataset it was 98% and 99%, respectively.
Article
Full-text available
In recent years, churn rates in industries such as finance have increased, and the cost of acquiring new users is more than five times the cost of retaining existing users. To improve the intelligent prediction accuracy of customer churn rate, artificial intelligence is gradually used. In this paper, the bidirectional long short-term memory convolutional neural network (BiLSTM-CNN) model is integrated with recurrent neural networks (RNNs) and convolutional neural networks (CNNs) in parallel, which well solves the defective problem that RNNs and CNNs run separately, and it also solves the problem that the output results of a long short-term memory network (LSTM) layer in a densely-connected LSTM-CNN (DLCNN) model will ignore some local information when input to the convolutional layer. In order to explore whether the attention bidirectional long short-term memory convolutional neural network (AttnBLSTM-CNN) model can perform better than BiLSTM-CNN, this paper uses bank data to compare the two models. The experimental results show that the accuracy of the AttnBiLSTM-CNN model is improved by 0.2%, the churn rate is improved by 1.3%, the F1 value is improved by 0.0102, and the AUC value is improved by 0.0103 compared with the BLSTM model. Therefore, introducing the attention mechanism into the BiLSTM-CNN model can further improve the performance of the model. The improvement of the accuracy of the user churn prediction model can warn of the possibility of user churn in advance and take effective measures in advance to prevent user churn and improve the core competitiveness of financial institutions.
Article
Full-text available
Owing to saturated markets, fierce competition, dynamic criteria, along with introduction of new attractive offers, the considerable issue of customer churn was faced by the telecommunication industry. Thus, an efficient Churn Prediction (CP) model is required for monitoring customer churn. Therefore, this work proposes a novel framework to predict customer churn through a deep learning model namely Swish Recurrent Neural Network (S-RNN). Finally, SRNN is adapted to classify the Churn Customer (CC) and a normal customer. If the result is a churn customer, network utilisation history is analysed for retention process. Whereas, the number of churn customers based on the area network usage is not recognised in this frameworkOwing to saturated markets, fierce competition, dynamic criteria, along with introduction of new attractive offers, the considerable issue of customer churn was faced by the telecommunication industry. Thus, an efficient Churn Prediction (CP) model is required for monitoring customer churn. Therefore, this work proposes a novel framework to predict customer churn through a deep learning model namely Swish Recurrent Neural Network (S-RNN). Finally, S-RNN is adapted to classify the Churn Customer (CC) and a normal customer. If the result is a churn customer, network utilisation history is analysed for retention process.
Article
Full-text available
The diversity and specificities of today’s businesses have leveraged a wide range of prediction techniques. In particular, churn prediction is a major economic concern for many companies. The purpose of this study is to draw general guidelines from a benchmark of supervised machine learning techniques in association with widely used data sampling approaches on publicly available datasets in the context of churn prediction. Choosing a priori the most appropriate sampling method as well as the most suitable classification model is not trivial, as it strongly depends on the data intrinsic characteristics. In this paper, we study the behavior of eleven supervised and semi-supervised learning methods and seven sampling approaches on sixteen diverse and publicly available churn-like datasets. Our evaluations, reported in terms of the Area Under the Curve (AUC) metric, explore the influence of sampling approaches and data characteristics on the performance of the studied learning methods. Besides, we propose Nemenyi test and Correspondence Analysis as means of comparison and visualization of the association between classification algorithms, sampling methods and datasets. Most importantly, our experiments lead to a practical recommendation for a prediction pipeline based on an ensemble approach. Our proposal can be successfully applied to a wide range of churn-like datasets.
Article
Full-text available
The purpose of the proposed model is to assist the e-business to predict the churned users using machine learning. This paper aims to monitor the customer behavior and to perform decision-making accordingly. Design/methodology/approach The proposed model uses the 2-D convolutional neural network (CNN; a technique of deep learning). The proposed model is a layered architecture that comprises two different phases that are data load and preprocessing layer and 2-D CNN layer. In addition, the Apache Spark parallel and distributed framework is used to process the data in a parallel environment. Training data is captured from Kaggle by using Telco Customer Churn. Findings The proposed model is accurate and has an accuracy score of 0.963 out of 1. In addition, the training and validation loss is extremely less, which is 0.004. The confusion matric results show the true-positive values are 95% and the true-negative values are 94%. However, the false-negative is only 5% and the false-positive is only 6%, which is effective. Originality/value This paper highlights an inclusive description of preprocessing required for the CNN model. The data set is addressed more carefully for the successful customer churn prediction.
Article
Full-text available
The customer churn prediction (CCP) is one of the challenging problems in the telecom industry. With the advancement in the field of machine learning and artificial intelligence, the possibilities to predict customer churn has increased significantly. Our proposed methodology, consists of six phases. In the first two phases, data pre-processing and feature analysis is performed. In the third phase, feature selection is taken into consideration using gravitational search algorithm. Next, the data has been split into two parts train and test set in the ratio of 80% and 20% respectively. In the prediction process, most popular predictive models have been applied, namely, logistic regression, naive bayes, support vector machine, random forest, decision trees, etc. on train set as well as boosting and ensemble techniques are applied to see the effect on accuracy of models. In addition, K-fold cross validation has been used over train set for hyperparameter tuning and to prevent overfitting of models. Finally, the obtained results on test set have been evaluated using confusion matrix and AUC curve. It was found that Adaboost and XGboost Classifier gives the highest accuracy of 81.71% and 80.8% respectively. The highest AUC score of 84%, is achieved by both Adaboost and XGBoost Classifiers which outperforms over others.
Article
Full-text available
In this paper, we present churn prediction techniques that have been released so far. Churn prediction is used in the fields of Internet services, games, insurance, and management. However, since it has been used intensively to increase the predictability of various industry/academic fields, there is a big difference in its definition and utilization. In this paper, we collected the definitions of churn used in the fields of business administration, marketing, IT, telecommunications, newspapers, insurance and psychology, and described their differences. Based on this, we classified and explained churn loss, feature engineering, and prediction models. Our study can be used to select the definition of churn and its associated models suitable for the service field that researchers are most interested in by integrating fragmented churn studies in industry/academic fields.