ArticlePDF Available

Enterprise cloud resource optimization and management based on cloud operations

Authors:
Binbin Wu
Binbin Wu
Binbin Wu
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Yulu Gong
Yulu Gong
Yulu Gong
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Haotian Zheng at New York University
Haotian Zheng
Yifan Zhang at Institute of Acoustics, Chinese Academy of Sciences
Yifan Zhang
Show all 6 authorsHide
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

The so-called automated operation and maintenance refers to a large number of repetitive tasks in daily IT operations (from simple daily checks, configuration changes and software installation to organizational scheduling of the entire change process) from manual execution in the past to standardized, streamlined and automated operations. This article delves into the realm of enterprise cloud resource optimization and management, leveraging automated operations (autoOps) as a fundamental strategy. As industries like banking witness exponential growth and innovation in IT systems, the complexity of managing resources escalates. Automated operations have emerged as a critical component, transitioning from manual interventions to encompass standardization, workflow optimization, and architectural enhancements. Through real-world deployments and theoretical frameworks, it elucidates effective strategies for optimizing and governing enterprise cloud resources, thereby enhancing efficiency, security, and resilience in IT operations.
Figures - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Available via license: CC BY 4.0
Content may be subject to copyright.
Enterprise cloud resource optimization and management
based on cloud operations
Binbin Wu1,*, Yulu Gong2, Haotian Zheng3, Yifan Zhang4, Jiaxin Huang5, Jingyu
Xu6
1Heating Ventilation and Air Conditioning Engineering, Tsinghua University, Beijing
China
2Computer & Information Technology, Northern Arizona University, Flagstaff, AZ,
USA
3Electrical & Computer Engineering, New York University,New York, NY, USA
4Executive Master of Business Administration, Amazon Connect Technology Services
(Beijing) Co., Ltd. Xi’an, Shaanxi, China
5Information Studies, Trine University, Phoenix USA
6Computer Information Technology, Northern Arizona University, Flagstaff, AZ,
USA
*Corresponding author: wubinbin.1@gmail.com
Abstract. The so-called automated operation and maintenance refers to a large number of
repetitive tasks in daily IT operations (from simple daily checks, configuration changes and
software installation to organizational scheduling of the entire change process) from manual
execution in the past to standardized, streamlined and automated operations. This article delves
into the realm of enterprise cloud resource optimization and management, leveraging automated
operations (autoOps) as a fundamental strategy. As industries like banking witness exponential
growth and innovation in IT systems, the complexity of managing resources escalates.
Automated operations have emerged as a critical component, transitioning from manual
interventions to encompass standardization, workflow optimization, and architectural
enhancements. Through real-world deployments and theoretical frameworks, it elucidates
effective strategies for optimizing and governing enterprise cloud resources, thereby enhancing
efficiency, security, and resilience in IT operations.
Keywords: Digitization, IT Operations (ITOps), Cloud Computing, Resource Management,
Automation.
1. Introduction
In industries like banking, characterized by high levels of digitization, the relentless growth and
innovation in business operations have led to the expansion and sophistication of IT systems. Within
this landscape, IT operations (ITOps) have emerged as a vital component of IT service delivery, tasked
with navigating increasingly complex business requirements and diverse user demands. As the scale of
data center infrastructure continues to swell, encompassing servers, storage, databases, and network
resources, the need for efficient resource management becomes more pronounced. Moreover, stringent
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
© 2024 The Authors. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0
(https://creativecommons.org/licenses/by/4.0/).
8
regulatory mandates, combining on-site inspections with remote audits, place greater emphasis on
standardization, compliance, and governance in ITOps practices.
The escalating demands of [1]IT applications necessitate a more nuanced approach to ensure that IT
services remain flexible, secure, and resilient.As a result, automated operations (autoOps) have garnered
significant attention as a cornerstone of modern IT service assurance.This article aims to explore the
intersection of automated operations and enterprise cloud resource management, specifically focusing
on optimization and governance strategies. By examining real-world deployments and theoretical
frameworks, we seek to provide insights for experts and practitioners engaged in the development and
implementation of autoOps platforms within cloud environments. Through an analysis of industry best
practices and emerging trends, this research endeavors to elucidate effective approaches for enterprise
cloud resource optimization and management, enhancing efficiency, security, and resilience in IT
operations.
2. Related work
2.1. Traditional operation and maintenance mode
Generally speaking, enterprise IT can be divided into three stages, the first stage is the mainframe era,
characterized by the recording and processing of core financial data, IT system problems do not affect
the operation of the business. The second stage is the information age, characterized by the recording
and processing of core production data, and the failure of IT systems will cause some businesses to stop
running. The third stage is the digital era, which is characterized by the recording and processing of
comprehensive data of enterprises, the growth of data volume by tens of times, and the management of
production and operation of enterprises by digital means, and the problem of IT system will seriously
affect the normal operation of enterprises. The traditional operation and maintenance mode is mainly
the maintenance management in the first and second phases, and the new operation and maintenance
mode must be used in the third phase. The traditional O&M mode mainly maintains applications based
on chimney architecture, while the new O&M mode maintains applications based on microservices
distributed architecture. [2]The concept of microservice distribution has been put forward for 20 or 30
years, and the theory and practice are very mature. In recent years, driven by the digital economy, it has
flourished and become the mainstream mode of running applications in production environments.
The original purpose is to improve the efficiency of operation and maintenance with the help of
automated tools. [3]Unexpectedly, it hindered the improvement of operation and maintenance
efficiency.In the past two years, many enterprises have chosen to re-implement the operation and
maintenance mode more in line with modern automatic intelligence methods, and to achieve the upgrade
from ordinary operation and maintenance to CloudOps Technology and maintenance, we must first do
the following basic skills, otherwise automated operation and maintenance will only be a flash in the
pan and cannot continue to support operation and maintenance work.
2.2. CloudOps Technology
CloudOps is actually automated operation and maintenance on the cloud,CloudOps = Cloud x DevOps,
emphasizing it is to make full use of the characteristics of the cloud itself to better practice [6]DevOps,
accelerate the rapid and stable delivery of business value, its core point is to emphasize the
characteristics of the cloud itself, without the need for our repeated development. The characteristics of
the cloud itself include the high elasticity, high standardization, high automation and self-service mode
of the cloud, which means that users can access it according to their own needs, without relying on any
other capability support.
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
9
Figure 1. CloudOps implementation process architecture diagram
CloudOps defines the five dimensions that enterprises focus on in the process of cloud and cloud
management, and it echoes the five common pain points of cloud customers we mentioned earlier. They
are Cost, Automation, Reliability, Elasticity and Security, abbreviated as CARES[4]. For example, the
cost optimization tool solves the cost problem, the automation capability solves the problem of
automated operation and maintenance efficiency, the reliability capability can be used to improve the
stability of the business and shorten the time of service loss, the elasticity capability solves the problem
of application availability, and the security compliance capability improves the security of the business.
[5]Therefore, CloudOps is not only an operation and maintenance concept, but also represents the
general term for Cloud Vendors to provide you with a set of standardized tools around the operation and
maintenance experience.
2.3. Enterprise cloud resource management
Corporate strategy is the anchor of all corporate activities. Strategic goals are long-term goals, while
business goals are short-term tasks. This requires that enterprise resources must match the goals of the
organization, not only to ensure the long-term strategy of resource supply compliance and rationality,
but also to provide enough flexibility to ensure the short-term goals. Under the premise of clear and
clear consensus, the responsibilities and authority of the organization should be clarified, and the
capacity budget indicators of the business system should be implemented. Therefore, organizational
resource management must be able to construct multi-level organizations based on roles or users, and
can allocate resource quotas and expense quotas for each organization. At the same time, when the
organization or resources change due to business reasons, it must be able to flexibly adjust the
organizational structure and resource allocation.
Therefore, the core of the enterprise cloud is to use cloud resources, and the resources on the cloud
have "out-of-the-box" and "elastic payment" ways. How to manage resources is to consider how to use
resources conveniently, and how to use resources safely. [6]Once the resources are abused, or there may
be a series of serious consequences such as out-of-control situations and data loss. In the scope of IT
management and governance, resources are in a very core position, and resources are closely related to
identity, authority, financial costs, audit compliance, etc.This sets the stage for exploring CloudOps
technology and maintenance, which leverages automated operations on the cloud to optimize resource
management and enhance operational efficiency in the enterprise cloud environment.
3. CloudOps Enterprise cloud resource optimization practices
In the field of enterprise cloud resources, the phenomenon of inefficient and costly manual configuration
is very common. A large number of irregular processes compete for resources with scheduled tasks and
forget to be shut down after they are used. Although this phenomenon is reasonable, the long running
of virtual machine instances also brings a lot of unnecessary cost and waste. But the challenges go far
beyond that. [7]Despite the flexibility of cloud computing, we still need to balance resources to ensure
that critical business processes are prioritized, while less important processes such as database imports
or file transfers are prioritized. If not, critical workflows will experience delays and even failures due to
this mismatch or over-fragmentation of cloud and virtual resources.
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
10
3.1. Efficient Resource Management
In the cloud native technology system, containerization has become the first choice for developers to
deploy applications, and [8]Kubernetes is the preferred container orchestration and scheduling system.
While containerization and Kubernetes have greatly simplified application deployment, service
governance still requires deep developer involvement. The core concept of a service grid is to route
requests between microservices in the infrastructure layer using proxies that run in parallel with each
service, form a network-format network, and interact with microservices.
Figure 2. Cloud resource management server
Service Mesh, as an infrastructure layer dealing with communication between services, helps
developers get rid of the dilemma of service communication problems, and gives the heavy work of
communication control logic to the grid, so some people call it the second generation of microservices.
Load balancing distributes access traffic to multiple back-end servers based on forwarding policies to
achieve high concurrency and improve processing performance. In the process of enterprise O&M
automation, load balancing becomes a necessary part to take into account the increasing number of
services, users, and services. [9]Massive traffic is distributed to multiple servers in the background for
processing to cope with high concurrency challenges.
In the operation and maintenance management of enterprise equipment, the effective management
of manpower, spare parts, technology and data resources is the key to reduce the operation and
maintenance cost and improve the comprehensive operation efficiency. Human, spare parts, technical
and data resources play an important role in the effective management of automated cloud assets, and
through reasonable planning and management of these resources, the efficient, intelligent and
sustainable development of equipment operation and maintenance can be achieved.
3.2. Prioritization of Critical Workflows
In a cloud-native ecosystem characterized by high concurrency and distributed architectures, prioritizing
critical workflows is essential to maintain business continuity and performance. Without proper resource
allocation and workload management, essential processes may experience delays or failures due to
resource contention or over-fragmentation.
A large part of daily operation and maintenance work involves configuration management and state
maintenance of services. At present, configuration management based on state (system state, code state,
configuration state and process state) has been greatly developed, and has made great progress in
operation and maintenance. The emergence of new tools is endless and dazzling, and in practical
applications, whether these tools are replaced or combined, the understanding of each specific scene and
selection will be different, and eventually may be displayed in a completely different form.Many large
It companies use puppet to manage and deploy software in clusters. The advantages and disadvantages
are that the Web UI generates processing reports, resource lists, real-time node management, and the
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
11
push command can trigger changes immediately. The disadvantages are that the installation process is
more complex than other tools, and requires learning Puppet's DSL or Ruby. The installation process
lacks error verification and generates error reports.
Each O&M[10] tool is only used to help personnel with O&M. Each tool has its own advantages.
Puppet is applicable to automatic software configuration and deployment. SaltStack is designed for
infrastructure management and can be up and running in minutes, easily managing thousands of servers,
and fast enough. Ansible is used for batch operating system configuration, batch program deployment,
and batch command running.
3.3. Automated monitoring and tuning
The basic principle of enterprise IT application O&M monitoring architecture is to comprehensively
monitor and manage enterprise IT systems by collecting, storing, analyzing, and displaying various
monitoring data. Monitoring data includes system, network, and application indicator data, event data,
and log data, which can be collected by various data collectors.
The collected data can be stored in storage systems such as distributed databases, [11]NoSQL
databases, or data warehouses, and transformed into visual monitoring indicators through data analysis
and processing, and displayed through dashboards, charts, and reports. At the same time, IT can also
monitor and alarm the monitoring data in real time through the alarm system, and automate the
management and optimization of the IT system through automated operation and maintenance.
Generally speaking, there must be monitoring where there is an IT system, and the distribution of IT
systems in different enterprises is not the same. Some enterprises have a large number of edge systems,
such as: computer, industrial computer and so on; Some enterprises have their own IDC room, and their
own IT system is built in the IDC room; Some enterprises build their IT systems on the public cloud;
Some enterprises establish a hybrid cloud architecture, IDC room and public cloud both.
IT monitoring system is attached to the above, for the edge system, there is a similar IOT monitoring
system; The IDC room has a monitoring system for network equipment (this is generally provided by
the network provider); The system on the public cloud provides a complete monitoring system by the
cloud provider, such as: cloudwatch on [12]AWS; If there is a hybrid cloud architecture, it is necessary
for the monitoring system construction team to integrate the monitoring system on the cloud to provide
unified monitoring.
Application layer monitoring
Application layer monitoring refers to the process of real-time monitoring and management of
application performance, availability, and security. Usually include:
A. Application performance monitoring: Monitoring application performance indicators, including
request response time, throughput, error rate saturation rate and other golden four indicators, in order to
find application performance problems and bottlenecks in a timely manner.
B. Availability monitoring: Monitors the availability of applications, including the running status,
access times, and error rate of applications, to ensure the normal running and availability of applications.
C. Security monitoring: Monitoring the security of applications, including application firewalls,
intrusion detection, security events, etc., to protect applications from security threats. Generally, this is
the responsibility of the security team, and operation and maintenance personnel are rarely involved.
D. Log management: Collect, analyze, and visualize application log information to help users quickly
discover and resolve application problems and anomalies.
3.4. Operation and maintenance data analysis and automated decision-making
The biggest difference between O&M automation and monitoring automation is whether there is a
subsequent action. After we have collected all the operation and maintenance data and processed it
according to the established logic, the next step is the extraction of value information and the
implementation of effective actions. If we want to check whether the storage device is properly loaded,
do we need to adjust the resource configuration? In traditional mode, engineers need to collect various
performance logs on storage (port performance, storage controller CPU and cache, storage volume I/O
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
12
performance, throughput, and so on), and collect the average and peak data within a certain period. Then,
after the comprehensive judgment, the decision needs to adjust resources or does not need to adjust. If
it is automated operations, we need to design the logic and experience of how engineers analyze and
judge into the script. Logic can be distilled into algorithms, and experience can be distilled into
probabilistic algorithms of historical data.
Most financial enterprises are faced with special holidays or periods of time for shopping and swipe-
card consumption. If the situation is predictable, many relevant engineers will be on duty in advance,
application engineers will observe the situation of application processing business in real time, and IT
engineers will observe the usage of various resources such as computing, network and storage. IT
engineers install and configure server resources prepared in advance and throw them into the resource
pool. This is a common emergency solution. If it is automated operation and maintenance, we need to
standard solidify the logic for engineers to judge the information monitored by applications, systems,
networks, and other aspects in real time, and script a series of operations for preparing resources,
initializing resources, allocating resources, and providing external services.
In summary, operation and maintenance automation is an important goal pursued by enterprise IT
operations. The prerequisite for the realization of O&M automation is standardization, and the key to
ensure the quality of O&M automation is the utilization of O&M data. The idea of landing O&M
automation project lies in the design idea from the bottom up.
4. Conclusion
Enterprise cloud resource management plays a pivotal role in aligning organizational goals with resource
utilization, ensuring both long-term strategic alignment and short-term operational flexibility. As
businesses navigate the complexities of digitization, effective resource management becomes paramount,
necessitating clear organizational structures, role-based access controls, and flexible resource allocation
mechanisms. [20]The evolution of O&M practices from traditional to modern automated methods
reflects the changing demands of the digital era. While initial enthusiasm for automated O&M projects
yielded valuable insights, challenges such as tool fragmentation and scalability issues necessitated a
shift towards more intelligent and efficient approaches. CloudOps technology emerges as a key enabler,
leveraging automated operations on the cloud to optimize resource management and enhance operational
efficiency in the enterprise cloud environment.
Furthermore, effective O&M management relies on leveraging human, spare parts, technical, and
data resources to achieve efficient, intelligent, and sustainable development. By establishing
standardized processes, utilizing advanced technologies, and harnessing data-driven insights, enterprises
can enhance the efficiency and effectiveness of O&M operations, paving the way for continued
innovation and growth in the digital landscape.
In conclusion, the pursuit of O&M automation is integral to realizing the full potential of enterprise
IT operations. By embracing standardized approaches, leveraging advanced technologies, and
harnessing the power of data, organizations can navigate the complexities of modern cloud environments
while driving efficiency, resilience, and competitiveness in their operations.
References
[1] Mann, Zoltán Ádám. "Resource optimization across the cloud stack." IEEE Transactions on
Parallel and Distributed Systems29.1 (2017): 169-182.
[2] Mireslami, S., Rakai, L., Far, B. H., & Wang, M. (2017). Simultaneous cost and QoS optimization
for cloud resource allocation. IEEE Transactions on Network and Service Management, 14(3),
676-689.
[3] Sun, Y., White, J., Eade, S., & Schmidt, D. C. (2016). ROAR: A QoS-oriented modeling
framework for automated cloud resource allocation and optimization. Journal of Systems and
Software, 116, 146-161.
[4] Muhammad, Tayyab, et al. "Elevating Business Operations: The Transformative Power of Cloud
Computing." International Journal of Computer Science and Technology 2.1 (2018): 1-21.
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
13
[5] Rimal, Bhaskar Prasad, et al. "Architectural requirements for cloud computing systems: an
enterprise cloud approach." Journal of Grid Computing 9 (2011): 3-26.
[6] Singh, Sukhpal, and Inderveer Chana. "Cloud resource provisioning: survey, status and future
research directions." Knowledge and Information Systems 49 (2016): 1005-1069.
[7] Christiaanse, W. R., and A. H. Palmer. "A technique for the automated scheduling of the
maintenance of generating facilities." IEEE Transactions on Power Apparatus and Systems 1
(1972): 137-144.
[8] Devriendt, C., Magalhães, F., Weijtjens, W., De Sitter, G., Cunha, Á., & Guillaume, P. (2014).
Structural health monitoring of offshore wind turbines using automated operational modal
analysis. Structural Health Monitoring, 13(6), 644-659.
[9] Bajrić, A., Høgsberg, J., & Rüdinger, F. (2018). Evaluation of damping estimates by automated
operational modal analysis for offshore wind turbine tower vibrations. Renewable Energy, 116,
153-163.
[10] Cord‐Ruwisch, R., Mercz, T. I., Hoh, C. Y., & Strong, G. E. (1997). Dissolved hydrogen
concentration as an on‐line control parameter for the automated operation and optimization of
anaerobic digesters. Biotechnology and bioengineering, 56(6), 626-634.
[11] Fenton, R. E., & Mayhan, R. J. (1991). Automated highway studies at the Ohio State University-
an overview. IEEE transactions on Vehicular Technology, 40(1), 100-113.
[12] Rad, S. R., Farah, H., Taale, H., van Arem, B., & Hoogendoorn, S. P. (2020). Design and
operation of dedicated lanes for connected and automated vehicles on motorways: A
conceptual framework and research agenda. Transportation research part C: emerging
technologies, 117, 102664.
Proceedings of the 2nd International Conference on Software Engineering and Machine Learning
DOI: 10.54254/2755-2721/67/20240667
14
... Tsantekidis et al. proposed a deep convolutional neural network approach for high-frequency time series forecasting in limit order books [27] . Their model captured spatial and temporal dependencies in order book data, outperforming traditional time series models in predicting short-term price movements. ...
Optimization of High-Frequency Trading Strategies Using Deep Reinforcement Learning
Article
Full-text available
  • Nov 2024
This study presents a new method for optimising high-risk trading (HFT) strategies using deep learning (DRL). We propose a multi-time DRL framework integrating advanced neural network architectures with sophisticated business data processing techniques. The framework employs a combination of convolutional neural networks for manual order analysis, short-term memory networks for time series processing, and a multi-head listening mechanism for body fusion. We formulate the HFT problem based on Markov Decision Processes and use the Proximal Policy Optimization algorithm for training. The model is evaluated using tick-by-tick data from the NASDAQ exchange, including ten liquid stocks in 6 months. The experimental results show the superiority of our method, achieving a Sharpe ratio of 3.42, outperforming the learning model and machine learning based on benchmarks up to 33%. The proposed strategy has demonstrated strong performance across a wide range of regulatory markets and has shown potential for strategic objectives. Sensitivity analysis confirms the model's stability across a range of hyperparameters. Our findings suggest that the DRL-based approach can improve HFT performance and provide better market adaptation and risk management. This research leads to the continuous evolution of algorithmic trading strategies and shows the potential of AI-driven approaches in financial markets.
View
... Regarding dynamic budget adjustment, in the budget preparation stage, RPA can simulate the financial performance under different business scenarios, such as slowing sales growth and rising costs, and help enterprises develop flexible budget plans through hypothesis analysis and sensitivity testing. In the implementation process, once the actual performance deviates from the budget target, the IPA can give real-time warnings and recommend adjustments to budget allocation to achieve optimal allocation of resources [36] . ...
Driving Efficiency and Risk Management in Finance through AI and RPA
Preprint
Full-text available
  • Oct 2024
This article examines the integration of artificial intelligence (AI) and robotic process automation (RPA) in financial accounting and management, underscoring their role in driving the digital transformation of corporate finance. It discusses the shortcomings of traditional financial processes and highlights the potential of AI and RPA technologies to enhance efficiency, accuracy, and cost-effectiveness. The paper also explores the limitations of RPA, such as its challenges in processing unstructured data and handling complex decision-making scenarios. Looking forward, it considers the future trends in AI and RPA, emphasizing the benefits of cloud technology in scaling automated systems and addressing associated challenges.
View
... This study utilizes the MyAnimeList dataset, a comprehensive collection of user interactions and anime metadata [21]. The dataset comprises 112,000 user reviews for 1,000 distinct anime titles, providing a rich source of textual data and user preferences. ...
Personalized Recommendation Systems Powered By Large Language Models: Integrating Semantic Understanding and User Preferences
Article
  • Aug 2024
This study proposes a novel personalized recommendation system leveraging Large Language Models (LLMs) to integrate semantic understanding with user preferences [1]. The system addresses critical challenges in traditional recommendation approaches by harnessing LLMs' advanced natural language processing capabilities. We introduce a framework combining a fine-tuned Roberta semantic analysis model with a multi-modal user preference extraction mechanism.The LLM component undergoes domain adaptation using Masked Language Modeling on a corpus of 112,000 user reviews from the MyAnimeList dataset, followed by task-specific fine-tuning using contrastive learning. User preferences are modeled through a weighted combination of explicit ratings, review sentiment, and implicit feedback, incorporating temporal dynamics through a time-decay function. Experimental results demonstrate significant improvements over state-of-the-art baselines, including Matrix Factorization, Neural Collaborative Filtering, BERT4Rec, and LightGCN. Our LLM-powered system achieves an 8.6%increase in NDCG@10 and a 10.5% improvement in Mean Reciprocal Rank compared to the best-performing baseline. Ablation studies reveal the synergistic effect of integrating LLM-based semantic understanding with user preference modeling. Case studies highlight the system's ability to recommend long-tail items and provide cross-genre suggestions, showcasing its capacity for nuanced content understanding. Scalability analysis indicates that while the LLM-based approach has higher initial computational costs, its performance scales comparably to other deep learning models for larger datasets. This research contributes to the field by demonstrating the effectiveness of LLMs in enhancing recommendation accuracy and diversity. Future work will explore advanced LLM architectures, multi-modal data integration, and techniques to improve computational efficiency and interpretability of recommendations.
View
Revolutionizing Cloud Infrastructure Scaling: A Gap Detection Approach
Article
Full-text available
  • Nov 2024
In the dynamic landscape of cloud computing, efficiently scaling compute and storage resources remains critical for organizations striving to optimize costs while meeting fluctuating demand. Traditional auto-scaling methods, which rely on setting minimum and maximum limits, often fail to account for nuanced growth patterns and can lead to either resource shortages or excess capacity. This comprehensive article introduces a novel approach to cloud infrastructure scaling by distinguishing between organic and inorganic growth while implementing an innovative gap detection methodology. The article examines how integrating real-time metrics with predictive analytics enables proactive resource management, allowing organizations to secure long-term reservations and optimize costs. By incorporating both organic and inorganic growth factors into the scaling strategy, this research demonstrates the creation of a holistic resource management system that enhances operational efficiency while significantly reducing costs through advanced techniques such as machine learning, dynamic resource allocation, automated reservation systems, and policy-driven scaling decisions.
View
Prediction of financial customer buying behavior based on machine learning
Article
Full-text available
  • Jun 2024
This study investigates the prediction of online purchasing behavior on the palm life APP through comprehensive analysis of customer operation logs, attribute sets, and purchase labels. Utilizing advanced feature engineering techniques, including time-based metrics, frequency analyses, and category-specific operations, we constructed a robust feature system comprising 61 associated features. The predictive models, leveraging Logistic Regression and LightGBM algorithms, were evaluated using cross-validation and AUC scores, demonstrating strong generalization capabilities and effectiveness in predicting customer behavior. Findings highlight the significance of personalized recommendations and targeted marketing strategies in enhancing customer engagement and optimizing operational efficiencies for e-commerce platforms. This research contributes to both theoretical advancements in consumer behavior prediction and practical implications for enhancing customer experience and service personalization in fintech applications.
View
Enhancing User Engagement through Adaptive UI/UX Design: A Study on Personalized Mobile App Interfaces
Article
  • Sep 2024
This paper presents a comprehensive study on developing and evaluating an adaptive UI/UX framework to enhance user engagement in mobile applications through personalized interfaces. The research investigates key factors influencing user engagement, including demographics, cognitive abilities, and contextual variables. A context-aware adaptation engine was designed to adjust interface elements based on real-time user data dynamically. The proposed framework was implemented in a mobile learning application and subjected to rigorous usability testing and user engagement analysis. Results demonstrated significant improvements in task completion rates, user satisfaction, and overall engagement metrics compared to non-adaptive interfaces. This study contributes valuable insights into the design and optimization of adaptive mobile interfaces, emphasizing the importance of personalization in creating compelling user experiences.
View
Transforming User Experience (UX) through Artificial Intelligence (AI) in Interactive Media Design
Article
  • Sep 2024
This paper discusses the application and advantages of artificial intelligence technology in digital media interactive product design. Firstly, the development background of artificial intelligence technology and its promoting effect on design innovation is introduced, and the application of new technologies, such as generative adversarial networks in art creation and design personalization, is definitely analyzed. It then explores in detail the ability of AI to break traditional constraints in interactive design, innovate design, and optimize user experience, especially in digital media. Finally, the case study of Microsoft Cortana shows the method of classifying and processing user queries by machine learning system and its experimental results. The research of this paper provides the theoretical basis and empirical support for the application of artificial intelligence technology in the future interactive design of digital media and has important academic and practical significance.
View
Federated Learning for Privacy-Preserving Medical Data Sharing in Drug Development
Preprint
Full-text available
  • Oct 2024
This study explores the potential of Federated Learning (FL) to facilitate the sharing and collaboration of medical data in drug development under the premise of privacy protection. While traditional centralized data processing methods limit effective collaboration across agencies due to data privacy and compliance concerns, federated learning avoids the risk of privacy breaches through a distributed architecture that allows participants to train artificial intelligence (AI) models together without sharing raw data. This paper systematically describes the core mechanism of federated learning, including the key technologies such as model parameter updating, differential privacy and homomorphic encryption, and their applications in drug development and medical data processing. Examples, such as NVIDIA Clara's Federated learning application and COVID-19 resource prediction, show that federated learning improves the efficiency of multi-party collaboration and model performance while ensuring data privacy. In addition, this study explores the scalability and generality of federated learning in the medical field, and points out that the technology is not only suitable for drug development, but also has broad cross-industry application potential, especially in areas such as finance and insurance, where data privacy is critical.
View
Deep Belief Networks (DBN) for Financial Time Series Analysis and Market Trends Prediction
Article
  • Jul 2024
Deep Belief Networks (DBNs) represent a transformative approach in financial time series analysis, addressing the complexities of market dynamics through advanced deep learning techniques. By leveraging hierarchical layers of unsupervised Restricted Boltzmann Machines (RBMs), DBNs excel in extracting intricate patterns from vast datasets, enabling accurate prediction of market trends and fluctuations. This capability not only enhances traditional financial analysis methods but also facilitates informed decision-making in dynamic and uncertain financial environments.
View
Blockchain Based Reverse Logistics Data Tracking: An Innovative Approach to Enhance E - Waste Recycling Efficiency
Article
  • Jul 2024
This study explores the application of blockchain technology in e-waste recycling, focusing on enhancing reverse logistics data tracking. A blockchain-based system integrating IoT sensors, smart contracts, and a token-based incentive mechanism was designed and implemented. The case study in Metropolis demonstrated significant improvements in e-waste management efficiency. Recycling rates increased by 27%, material recovery efficiency improved by 18%, and stakeholder participation doubled. The system processed an average of 50,000 transactions daily, proving its scalability. The blockchain implementation addressed key challenges in e-waste management, including lack of transparency and inefficient processes. The immutable audit trail enhanced traceability, fostering trust among participants. The token-based incentive system drove behavioral changes, increasing consumer participation by 119%. The study contributes to the theoretical understanding of blockchain applications in environmental management and extends literature on reverse logistics. Practical implications include a blueprint for implementing blockchain-based e-waste management systems, insights for policymakers, and opportunities for technology developers. The research demonstrates blockchain's potential to address environmental challenges, offering a promising path towards sustainable resource management practices. Future research directions include exploring cross-border e-waste management and integrating artificial intelligence for predictive analytics.
View
Elevating Business Operations: The Transformative Power of Cloud Computing
Article
Full-text available
  • Jan 2018
The rapid advancement of technology has led to the emergence of cloud computing as a game-changer in the world of business operations. This research paper aims to explore the transformative power of cloud computing and its impact on elevating business operations. The study delves into the various aspects of cloud computing, including cost-efficiency, scalability, flexibility, collaboration, accessibility, disaster recovery, and security. It also examines the broader impacts of cloud computing on innovation, competitiveness, environmental sustainability, and the ability of businesses to focus on their core operations. Through a series of case studies, the research highlights the practical applications and benefits of cloud computing across businesses of different sizes and industries. Additionally, the paper discusses the challenges associated with cloud computing, such as data privacy, compliance, and bandwidth limitations, and proposes solutions to overcome these hurdles. Finally, the study explores emerging trends in cloud computing and provides recommendations for businesses looking to leverage the power of the cloud to elevate their operations. The findings of this research underscore the transformative potential of cloud computing in enhancing the efficiency, competitiveness, and sustainability of business operations.
View
Design and operation of dedicated lanes for connected and automated vehicles on motorways: A conceptual framework and research agenda
Article
Full-text available
  • Aug 2020
  • TRANSPORT RES C-EMER
Dedicated Lanes (DLs) have been proposed as a potential scenario for the deployment of Automated and/or Connected Vehicles (C/AVs) on the road network. However, evidence-based knowledge regarding the impacts of different design configurations, utilization policies, and the design of their access/egress on traffic safety and efficiency is limited. In order to develop an adequate design for DLs, first, a conceptual framework describing the relations and interrelations between these factors and traffic safety and efficiency is needed. Therefore, the main aim of this paper is to develop a conceptual framework accounting for the factors that could affect the safety and efficiency of DLs. This conceptual framework is underpinned based on relevant literature on how the deployment of C/AVs, driver behaviour, and DL design and operation affect traffic safety and efficiency. Based on the conceptual framework, the knowledge gaps on DL design for C/AVs were identified and a research agenda, including prioritization of the research needs, is proposed. Following the developed conceptual framework, the necessary building blocks for investigating the impacts of different design configurations, utilization policies, and the design of their access/egress on traffic safety and efficiency are: (1) to specify the types of vehicles with certain capabilities allowed to drive on DLs; (2) to incorporate existing algorithms of C/AVs, which reflect more realistically their behaviour, in both driving simulator experiments and microscopic simulation; (3) to translate the empirical data regarding human behavioural adaptation collected from field tests and driving simulator studies to mathematical models and implement them in traffic flow simulation platform. It is also recommended to develop automated lane change algorithms, taking into account connectivity between C/AVs which can be also implemented in driving simulators and traffic flow simulation platforms. Finally, it is recommended that future research investigate the combined effects of traffic safety and efficiency in designing DLs while considering driver behaviour adaptation and control transitions between manual and automated operation.
View
Resource Optimization Across the Cloud Stack
Article
Full-text available
  • Aug 2017
Previous work on optimizing resource provisioning in virtualized environments focused either on mapping virtual machines (VMs) to physical machines (PMs) or mapping application components to VMs. In this paper, we argue that these two optimization problems influence each other significantly and in a highly non-trivial way. We define a sophisticated problem model for the joint optimization of the two mappings, taking into account sizing aspects, colocation constraints, license costs, and hardware affinity relations. As demonstrated by the empirical evaluation on a real-world workload trace, the combined optimization leads to significantly better overall results than considering the two problems in isolation.
View
Cloud resource provisioning: survey, status and future research directions
Article
Full-text available
  • Dec 2016
  • KNOWL INF SYST
Cloud resource provisioning is a challenging job that may be compromised due to unavailability of the expected resources. Quality of Service (QoS) requirements of workloads derives the provisioning of appropriate resources to cloud workloads. Discovery of best workload–resource pair based on application requirements of cloud users is an optimization problem. Acceptable QoS cannot be provided to the cloud users until provisioning of resources is offered as a crucial ability. QoS parameters-based resource provisioning technique is therefore required for efficient provisioning of resources. This research depicts a broad methodical literature analysis of cloud resource provisioning in general and cloud resource identification in specific. The existing research is categorized generally into various groups in the area of cloud resource provisioning. In this paper, a methodical analysis of resource provisioning in cloud computing is presented, in which resource management, resource provisioning, resource provisioning evolution, different types of resource provisioning mechanisms and their comparisons, benefits and open issues are described. This research work also highlights the previous research, current status and future directions of resource provisioning and management in cloud computing.
View
Structural health monitoring of offshore wind turbines using automated operational modal analysis
Article
Full-text available
This article will present and discuss the approach and the first results of a long-term dynamic monitoring campaign on an offshore wind turbine in the Belgian North Sea. It focuses on the vibration levels and modal parameters of the fundamental modes of the support structure. These parameters are crucial to minimize the operation and maintenance costs and to extend the lifetime of offshore wind turbine structure and mechanical systems. In order to perform a proper continuous monitoring during operation, a fast and reliable solution, applicable on an industrial scale, has been developed. It will be shown that the use of appropriate vibration measurement equipment together with state-of-the art operational modal analysis techniques can provide accurate estimates of natural frequencies, damping ratios, and mode shapes of offshore wind turbines. The identification methods have been automated and their reliability has been improved, so that the system can track small changes in the dynamic behavior of offshore wind turbines. The advanced modal analysis tools used in this application include the poly-reference least squares complex frequency-domain estimator, commercially known as PolyMAX, and the covariance-driven stochastic subspace identification method. The implemented processing strategy will be demonstrated on data continuously collected during 2 weeks, while the wind turbine was idling or parked.
View
Simultaneous Cost and QoS Optimization for Cloud Resource Allocation
Article
  • Sep 2017
Cloud computing is a new era of computing that offers resources and services for web applications. Selection of optimal cloud resources is the main goal in cloud resource allocation. Sometimes, customers pay more than required since cloud providers’ pricing strategy is designed for the interest of the providers. Nonetheless, cloud customers are interested in selecting cloud resources to meet their QoS requirements. Thus, for the interest of both providers and customers, it is vital to balance the two conflicting objectives of deployment cost and QoS performance. In this paper, we present a cost-effective and runtime friendly algorithm that minimizes the deployment cost while meeting the QoS performance requirements. In other words, the algorithm offers an optimal choice, from customers’ point of view, for deploying a web application in cloud environment. The multi-objective optimization algorithm minimizes cost and maximizes QoS performance simultaneously. The proposed algorithm is verified by a series of experiments on different workload scenarios deployed in two distinct cloud providers. The results show that the proposed algorithm finds the optimal combination of cloud resources that provides a balanced trade-off between deployment cost and QoS performance in relatively low runtime.
View
Evaluation of damping estimates by automated Operational Modal Analysis for offshore wind turbine tower vibrations
Article
  • Mar 2017
  • RENEW ENERG
Reliable predictions of the lifetime of offshore wind turbine structures are influenced by the limited knowledge concerning the inherent level of damping during downtime. Error measures and an automated procedure for covariance driven Operational Modal Analysis (OMA) techniques has been proposed with a particular focus on damping estimation of wind turbine towers. In the design of offshore structures the estimates of damping are crucial for tuning of the numerical model. The errors of damping estimates are evaluated from simulated tower response of an aeroelastic model of an 8 MW offshore wind turbine. In order to obtain algorithmic independent answers, three identification techniques are compared: Eigensystem Realization Algorithm (ERA), covariance driven Stochastic Subspace Identification (COV-SSI) and the Enhanced Frequency Domain Decomposition (EFDD). Discrepancies between automated identification techniques are discussed and illustrated with respect to signal noise, measurement time, vibration amplitudes and stationarity of the ambient response. The best bias-variance error trade-off of damping estimates is obtained by the COV-SSI. The proposed automated procedure is validated by real vibration measurements of an offshore wind turbine in non-operating conditions from a 24-h monitoring period.
View
ROAR: A QoS-Oriented Modeling Framework for Automated Cloud Resource Allocation and Optimization
Article
  • Aug 2015
  • J SYST SOFTWARE
Cloud computing offers a fast, easy and cost-effective way to configure and allocate computing resources for web applications, such as consoles for smart grid applications, medical records systems, and security management platforms. Although a diverse collection of cloud resources (e.g., servers) is available, choosing the most optimized and cost-effective set of cloud resources for a given web application and set of quality of service (QoS) goals is not a straightforward task. Optimizing cloud resource allocation is a critical task for offering web applications using a software as a service model in the cloud, where minimizing operational cost while ensuring QoS goals are met is critical to meeting customer demands and maximizing profit. Manual load testing with different sets of cloud resources, followed by comparison of test results to QoS goals is tedious and inaccurate due to the limitations of the load testing tools, challenges characterizing resource utilization, significant manual test orchestration effort, and challenges identifying resource bottlenecks. This paper introduces our work using a modeling framework – ROAR (Resource Optimization, Allocation and Recommendation System) to simplify, optimize, and automate cloud resource allocation decisions to meet QoS goals for web applications, including complex multi-tier application distributed in different server groups. ROAR uses a domain-specific language to describe the configuration of the web application, the APIs to benchmark and the expected QoS requirements (e.g., throughput and latency), and the resource optimization engine uses model-based analysis and code generation to automatically deploy and load test the application in multiple resource configurations in order to derive a cost-optimal resource configuration that meets the QoS goals.
View
Architectural Requirements for Cloud Computing Systems: An Enterprise Cloud Approach
Article
  • Mar 2011
Cloud Computing is a model of service delivery and access where dynamically scalable and virtualized resources are provided as a service over the Internet. This model creates a new horizon of opportunity for enterprises. It introduces new operating and business models that allow customers to pay for the resources they effectively use, instead of making heavy upfront investments. The biggest challenge in Cloud Computing is the lack of a de facto standard or single architectural method, which can meet the requirements of an enterprise cloud approach. In this paper, we explore the architectural features of Cloud Computing and classify them according to the requirements of end-users, enterprises that use the cloud as a platform, and cloud providers themselves. We show that several architectural features will play a major role in the adoption of the Cloud Computing paradigm as a mainstream commodity in the enterprise world. This paper also provides key guidelines to software architects and Cloud Computing application developers for creating future architectures.
View
Dissolved hydrogen concentration as on-line control parameter for the automated operation and optimization of anaerobic digesters
Article
  • Dec 1997
The use of dissolved hydrogen as an early warning signal of digester failure and a control parameter to operate anaerobic digesters was investigated. A sensitive, on-line method was developed for measuring trace levels of dissolved hydrogen in a semi-permeable membrane, situated within the biomass of a 1 L laboratory anaerobic digester, using trace reduction gas analysis. At normal operating conditions, the dissolved hydrogen partial pressure (2 to 8 Pa) was found to be linearly correlated with the loading rate of the digester, and was a sensitive indicator of the effect of shockloads as well as gradual overloading. An increase in hydrogen partial pressure above a critical concentration of 6.5-7 Pa indicated the initial stage of digester overloading (i.e., volatile fatty acids accumulation). A H(2)-based computer control system, using a critical hydrogen partial pressure of 6.5 Pa as the setpoint, was found to be effective for the safe operation of a laboratory digester close to its maximum sustainable loading rate. The existence of a relationship between hydrogen level and organic loading rate was also confirmed on a 600 m(3) industrial digester, with digester overloading occurring at hydrogen concentrations above 7 Pa. The results suggest that the dissolved hydrogen concentration is capable of being a sensitive on-line parameter for the automated management of anaerobic digesters near their maximum sustainable loading capacity. (c) 1997 John Wiley & Sons, Inc. Biotechnol Bioeng 56: 626-634, 1997.
View