Figure 6 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Kubernetes (K8s) is expected to be a key container orchestration tool for edge computing infrastructures owing to its various features for supporting container deployment and dynamic resource management. For example, its horizontal pod autoscaling feature provides service availability and scalability by increasing the number of replicas. kube-proxy...
Contexts in source publication
Context 1
... shown in Figure 6a, the throughput of iptables was higher than that of userspace in all test cases. This is because the performance of userspace is limited by the bottleneck problem explained in Section 3.2. ...
Context 2
... the throughput cannot increase if the processing times for requests are long. By contrast, the throughput in the RAP mode tends to be constant regardless of increasing network delay, and it remained stable at approximately 6000 reqs/s, as shown in Figure 6. This is because the requests are handled locally as much as possible instead of being forwarded to other worker nodes. ...
Context 3
... addition, if a local worker needs to offload requests, RAP forwards them to the best node to minimize the latency during load-balancing. The latency of the three kube-proxy modes in Figure 6b was measured with the average latency based on forwarding and processing times. Notably, the latency of RAP was constant at approximately 5 ms. ...
Citations
... What sets our approach apart from related work, such as [7]- [9], is our intentional design of the proxy to distribute the workload evenly across instances capable of meeting the QoS requirements specified for a particular service, rather than identifying instances that optimize QoS or perform tradeoffs between QoS and load balancing, which can lead to instance overload or QoS degradation. With this technique, we minimize the risk of overloading service instances while ensuring that clients continuously receive satisfactory QoS. ...
... The routing decision is based on an adjustable parameter that represents the desired trade-off between pure load-balancing and pure proximity-based routing. Nguyen et al. [9] present Resource Adaptive Proxy (RAP), which performs regular checks on the resource status of individual pods 1 and the network status between worker nodes to make load-balancing decisions, prioritizing local request handling. RAP and proxy-mity require replacing the default Kubernetes functionality (kube-proxy). ...
... This strategy ensures continuous QoS adherence while minimizing the risk of overloading service instances, unlike alternative solutions that rigidly distribute requests within the same set of nodes in the proximity of the source node to optimize QoS, or occasionally randomly select nodes, potentially leading to QoS violations. Second, QEdgeProxy promptly reacts to QoS degradation, also differentiating itself from solutions [7]- [9] that use QoS approximations predominantly based on network latency, thus not capturing processing delays or the waiting time induced by the target instance's queue. By taking into account the actual measurements of QoS parameters, our approach ensures more effective routing decisions, as shown by our experiments. ...
While various service orchestration aspects within Computing Continuum (CC) systems have been extensively addressed, including service placement, replication, and scheduling, an open challenge lies in ensuring uninterrupted data delivery from IoT devices to running service instances in this dynamic environment, while adhering to specific Quality of Service (QoS) requirements and balancing the load on service instances. To address this challenge, we introduce QEdgeProxy, an adaptive and QoS-aware load balancing framework specifically designed for routing client requests to appropriate IoT service instances in the CC. QEdgeProxy integrates naturally within Kubernetes, adapts to changes in dynamic environments, and manages to seamlessly deliver data to IoT service instances while consistently meeting QoS requirements and effectively distributing load across them. This is verified by extensive experiments over a realistic K3s cluster with instance failures and network variability, where QEdgeProxy outperforms both Kubernetes built-in mechanisms and a state-of-the-art solution, while introducing minimal computational overhead.
... A container [15] binds all libraries, dependencies, and required configuration files into a unit of software that is required to run an application on an operating system. Kubernetes [8,9] is a container orchestration tool released by Google, for containerized applications. ...
Containerization has become popular these days because of its easy way of deploying and managing applications in the cloud. Containers are packages that are lightweight with application code, dependencies, programming language with versions and run times, and libraries that are required to run software services. Kubernetes is one of the most popular container orchestration tools which can automate the deploying, scaling, and other operations of containers thus reducing human error and cost. Kubernetes cluster refers to a group of nodes in which one is the master node and the others are worker nodes. Nodes in the Kubernetes cluster run Pods. Pods contain one or more containers and volumes. Volumes are dependent on the life cycle of Pod. Hence we use Persistent volume for storage which is independent of Pod’s life cycle. One of the significant limitations of Kubernetes is that Pods are mortal. Migration of running Pods in Kubernetes is done by stopping the running Pod, killing it, and then redeploying it from scratch. We proposed a modified Pod migration technique that uses Persistent volume and migration controller to capture the state of the containers. We show that redeploying a new Pod in Pod migration with optimized containers is a better strategy that minimizes the Pod’s cold start time and minimizes the use of CPU in nano CPU and minimizes utilization of memory in bytes.
... Several architectures are based on combining different computing models, such as edge and cloud, to enable distributed computing and task-offloading [19,45,46]. They focus on distributing tasks efficiently and optimizing resource usage. ...
... The edge servers can reduce the communication delay, but they are limited in size and computational capacity and not only cost-effective solutions. Nguyen et al. present a resource adaptive proxy [46] in an edge-computing environment consisting of multiple components, including the controller manager, scheduler, master server, cloud controller manager, and cloud-edge client. The resource adaptive proxy component is implemented in each worker node of the Kubernetes cluster and is integrated into every worker node within the cluster. ...
The management of decentralized energy resources and smart grids needs novel data-driven low-latency applications and services to improve resilience and responsiveness and ensure closer to real-time control. However, the large-scale integration of Internet of Things (IoT) devices has led to the generation of significant amounts of data at the edge of the grid, posing challenges for the traditional cloud-based smart-grid architectures to meet the stringent latency and response time requirements of emerging applications. In this paper, we delve into the energy grid and computational distribution architectures, including edge–fog–cloud models, computational orchestration, and smart-grid frameworks to support the design and offloading of grid applications across the computational continuum. Key factors influencing the offloading process, such as network performance, data and Artificial Intelligence (AI) processes, computational requirements, application-specific factors, and energy efficiency, are analyzed considering the smart-grid operational requirements. We conduct a comprehensive overview of the current research landscape to support decision-making regarding offloading strategies from cloud to fog or edge. The focus is on metaheuristics for identifying near-optimal solutions and reinforcement learning for adaptively optimizing the process. A macro perspective on determining when and what to offload in the smart grid is provided for the next-generation AI applications, offering an overview of the features and trade-offs for selecting between federated learning and edge AI solutions. Finally, the work contributes to a comprehensive understanding of edge offloading in smart grids, providing a Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis to support cost–benefit analysis in decision-making regarding offloading strategies.
... However, the approach is not "online", and the scheduling decision is not taken for every task. More technological approaches instead, like the one proposed in [9], design algorithms specifically targeting well-known frameworks like Kubernetes. In that case, the authors propose a proxy-based approach that periodically monitors the pods' state, and according to the load, it forwards the requests to balance it; however, the approach does not consider node heterogeneity which can have the same load but generates different service latency. ...
When dealing with distributed applications in Edge or Fog computing environments, the service latency that the user experiences at a given node can be considered an indicator of how much the node itself is loaded with respect to the others. Indeed, only considering the average CPU time or the RAM utilisation, for example, does not give a clear depiction of the load situation because these parameters are application- and hardware-agnostic. They do not give any information about how the application is performing from the user's perspective, and they cannot be used for a QoS-oriented load balancing. In this paper, we propose a load balancing algorithm that is focused on the service latency with the objective of levelling it across all the nodes in a fully decentralised manner. In this way, no user will experience a worse QoS than the other. By providing a differential model of the system and an adaptive heuristic to find the solution to the problem in real settings, we show both in simulation and in a real-world deployment, based on a cluster of Raspberry Pi boards, that our approach is able to level the service latency among a set of heterogeneous nodes organised in different topology.
... quests/demands equally to all Pods in a K8s cluster. Taking into consideration that these requests might be forwarded to remote worker nodes, this approach can result in long delays, especially in Edge computing environments where worker nodes are geographically dispersed [49]. ...
Container orchestration handles the semi-automated management of applications across Edge-Cloud, providing features such as autoscaling, high availability, and portability. Having been developed for Cloud-based applications, container orchestration faces challenges in the context of decentralized Edge-Cloud environments, requiring a higher degree of adaptability in the verge of mobility, heterogeneous networks, and constrained devices. In this context, this perspective paper aims at igniting discussion on the aspects that a dynamic orchestration approach should integrate to support an elastic orchestration of containerized applications. The motivation for the provided perspective focuses on proposing directions to better support challenges faced by next-generation IoT services, such as mobility or privacy preservation, advocating the use of context awareness and a cognitive, cross-layer approach to container orchestration to be able to provide adequate support to next-generation services. A proof of concept (available open source software) of the discussed concept has been implemented in a testbed composed of embedded devices.
... Kube-proxy: Kube-proxy is a network proxy that runs on each worker node, managing network routing and load balancing for the containers. It ensures that traffic is correctly routed between services and pods within the cluster, implementing the Service abstraction by maintaining IPVS or iptables rules (Nguyen et al., 2022). ...
Containerization has revolutionized software development and deployment by
accelerating the implementation and adoption of Microservices Architecture through
different technologies such as Docker and Kubernetes. In the core of the practice stands the container, a lightweight, and portable unit that is used to package an application together with all its configurations and libraries, offering consistency and flexibility in a wide set of systems. Containers are the perfect tools for implementing a microservice architecture by providing a framework where services can be isolated, each with their own functionality but at the same time properly coordinated. Docker is the most prominent and straightforward tool for building and running containers, allowing building and testing services without issues across many environments. Kubernetes on the other hand is another complementary technology which orchestrates the containers, abstracting their tedious management and turning them into a scalable cluster. The implementation of these technologies should be carefully accompanied with the adaptation of some practices concentrated mostly on security and optimization. In this
study, I have demonstrated an effective implementation of Kubernetes and microservices by deploying Apache Airflow as a platform for provisioning data workflows. The synergy of Kubernetes with different tools such as Terraform (Infrastructure as Code), Helm Charts (Kubernetes Templates) and ArgoCD (GitOps), in addition to Azure Kubernetes Service (Platform-as-a-Service) resulted in a fully functional Apache Airflow deployment.
... The following Figure 3 depicts the architecture of Light Weight Native Edge Load Balancer. These load balancers typically comprise several components that collaborate to distribute the workload across edge devices [15]. When deploying Light Weight Native Edge Load Balancers at the network edge, they are responsible for distributing traffic among multiple servers or services. ...
Edge computing has become an essential aspect of modern computing systems. Edge computing involves processing data at the edge of the network, closer to where the data is generated. The ability to process data in real-time at the edge provides various benefits such as lower latency, improved response times, and reduced network congestion. Load balancing is a critical component of edge computing, which distributes the workload across multiple edge devices, ensuring that the workload is evenly distributed. This paper discusses current trends in edge computing load balancing techniques, including static, dynamic, and hybrid load balancing approaches.
... A pod is the smallest executable unit in Kubernetes. Pod [ 5] is a set of one or more containers with data volumes. Kubernetes assures that the containers which belong to one pod execute in the same machine and share the same set of resources such as a single private IP address within the Kubernetes cluster [ 6]. ...
... C 0 = [1, 1, 1, 1] here we are considering a pod with four containers// 4 Step2: CREATE (P, C 0 , node1)//we assume a process CREATE creates the pod P on node 1 with C 0 = (δ0, δ1, ..., δn). 5 Step 3: KILL (P, C 0 , node1)//we assume that due to some reason a pod is killed// 6 Step 4: After checking the state of the container now edit the pod by removing the containers with ag c, i.e. Edit the number of containers in the pod's yaml le and save the yaml le in a temporary location; 7 Step 5: DELETE (P, C 0 , node1)//we assume that DELETE procedure removes the pod from node1//; 8 Step 6: CREATE (P, C t , nodex)//we assume a process CREATE creates the sub-pod on some node x. ...
Recreation of a Sub-pod for a Killed
Pod with Optimized Containers
in Kubernetes
... Consequently, they validated that the proposed approach outperforms other benchmark approaches and provides low latency and optimal resource consumption. Taherizadeh et al. [26] proposed a dynamic multi-level auto-scaling technique for container-based application services, and [27][28][29] proposed Kubernetes-based resource provisioning and service quality improvement measures. Le et al. [27] address the limitation of the Kubernetes horizontal pod autoscaler, in that it is not suitable for different traffic distribution environments with real-time service demand in edge computing environments. ...
... They proposed the traffic-aware horizontal pod autoscaler to improve service quality by dynamically adjusting cluster resources according to the network traffic distribution. Nguyen et al. [28] proposed a proxy for an improved Kubernetes, referred to as RAP, which offloads latency caused by the load during load balancing to other optimal nodes. Gupta et al. [29] proposed a method to containerize and deploy deep-learning models to learn from edges and improve service quality by reducing data latency and traffic. ...
KubeEdge is an open-source platform that orchestrates containerized Internet of Things (IoT) application services in IoT edge computing environments. Based on Kubernetes, it supports heterogeneous IoT device protocols on edge nodes and provides various functions necessary to build edge computing infrastructure, such as network management between cloud and edge nodes. However, the resulting cloud-based systems are subject to several limitations. In this study, we evaluated the performance of KubeEdge in terms of the computational resource distribution and delay between edge nodes. We found that forwarding traffic between edge nodes degrades the throughput of clusters and causes service delay in edge computing environments. Based on these results, we proposed a local scheduling scheme that handles user traffic locally at each edge node. The performance evaluation results revealed that local scheduling outperforms the existing load-balancing algorithm in the edge computing environment.
... These Pods are used to induce chaos to our experiment which is essential for performing OAT. In [3] Nguyen, Q. M., et al contemplates on Kubernetes about how the requests send to pods have equal weightage and for it to be a key container orchestration tool for edge computing infrastructures. OAT is used as a best practice and is considered as a common type of non-functional software testing. ...
How much confidence we can have in the interconnected complex systems that we put into production environment? In this paper we will provide the solution for operational readiness of a platform strengthening the backup, restore, network file transfer, failover capabilities and overall security. We provide the evaluation of inducing chaos to a Kubernetes environment which terminates random pods with data from edge devices in data centers while processing analytics on Big Data network and infer the recovery time of pods to calculate an estimated response time. In this research, we discuss the Operational Acceptance Testing through Chaos Engineering at all layers and more precisely on the modern architecture and practices like Microservices Architecture and Cloud computing which have changed our IT landscape in recent times.
