Soumyajit Dey

• Indian Institute of Technology Kharagpur

Multi-Agent Reinforcement Learning (MARL) has gained significant traction for solving complex real-world tasks, but the inherent stochasticity and uncertainty in these environments pose substantial challenges to efficient and robust policy learning. While Distributional Reinforcement Learning has been successfully applied in single-agent settings t...

The next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems realize this integration for 6G applications such...

With increasing transistor density, modern heterogeneous embedded processors often exhibit high temperature gradients due to complex application scheduling scenarios which may have missed design considerations. In many use cases, off-chip ”active” cooling solutions are considered prohibitive in such reduced form factors. Core frequency throttling b...

Smart grids are designed to efficiently handle variable power demands, especially for large loads, by real-time monitoring, distributed generation and distribution of electricity. However, the grid's distributed nature and the internet connectivity of large loads like Heating Ventilation, and Air Conditioning (HVAC) systems introduce vulnerabilitie...

Modern cyber-physical systems (CPSs) employ an increasingly large number of software control loops to enhance their autonomous capabilities. Such large task sets and their dependencies may lead to deadline misses caused by platform-level timing uncertainties, resource contention, etc. To ensure the schedulability of the task set in the embedded pla...

The distributed nature of smart grids, combined with sophisticated sensors, control algorithms, and data collection facilities at Supervisory Control and Data Acquisition (SCADA) centers, makes them vulnerable to strategically crafted cyber-physical attacks. These malicious attacks can manipulate power demands using high-wattage Internet of Things...

Modern Cyber-Physical Systems (CPSs) consist of numerous control units interconnected by communication networks. Each control unit executes multiple safety-critical and non-critical tasks in real-time. Most of the safety-critical tasks are executed with a fixed sampling period to ensure deterministic timing behaviour that helps in its safety and pe...

Low-power edge devices equipped with Graphics Processing Units (GPUs) are a popular target platform for real-time scheduling of inference pipelines. Such application-architecture combinations are popular in Advanced Driver-Assistance Systems (ADAS) for aiding in the real-time decision-making of automotive controllers. However, the real-time through...

Safe Reinforcement Learning (SRL) algorithms aim to learn a policy that maximizes the reward while satisfying the safety constraints. One of the challenges in SRL is that it is often difficult to balance the two objectives of reward maximization and safety constraint satisfaction. Existing algorithms utilize constraint optimization techniques like...

Smart meters provide fine-grained power usage profiles of consumers to utility providers to facilitate various grid functionalities such as load monitoring, real-time pricing, etc. However, information leakage from these usage profiles can potentially reveal sensitive aspects of consumers’ daily routines and their home absence, as state-of-the-art...

Future intelligent transportation will involve significant fleet based autonomy driven by cost and efficiency concerns. To achieve such goals, usual control theoretic methods for Cooperative Adaptive Cruise Control (CACC) for vehicle platooning needs to be aware of non-idealities like packet drops and delays during exchange of periodic time-critica...

Active deployment of Deep Reinforcement Learning (DRL) based controllers on safety-critical embedded platforms require model compaction. Neural pruning has been extensively studied in the context of CNNs and computer vision, but such approaches do not guarantee the preservation of safety in the context of DRL. A pruned network converging to high re...

Reinforcement Learning (RL) needs sufficient exploration to learn an optimal policy. However, exploratory actions could lead the learning agent to safety hazards, not necessarily in the next state but in the future. Therefore, it is essential to evaluate each action beforehand to ensure safety. The exploratory actions and the actions proposed by th...

Neural network pruning is a technique of network compression by removing weights of lower importance from an optimized neural network. Often, pruned networks are compared in terms of accuracy, which is realized in terms of rewards for Deep Reinforcement Learning (DRL) networks. However, networks that estimate control actions for safety-critical tas...

Cyber-Physical Systems (CPSs) have become an essential ingredient of today’s digital world. They operate in a deeply embedded manner inside infrastructures for transportation, power grids, smart buildings, and several other application domains. The implementation of most CPSs involves components like sensors, real-time computing platforms, actuator...

Neural Network (NN) based real-time inferencing tasks are often co-scheduled on GPGPU-style edge platforms. Existing works advocate using different NN parameters for the same detection task in different environments. However, realizing such approaches remains challenging, given accelerator devices’ limited on-chip memory capacity. As a solution, we...

Smart metering is a mechanism through which fine-grained power consumption profiles of the consumers are collected periodically in a Smart grid. However, a growing concern in this regard is that the leakage of consumers’ consumption data may reveal their daily life patterns as the state-of-the-art metering strategies lack adequate security and priv...

Adjusting the remote electrical tilt (RET) of antennas is one of the important actions targeting run-time optimization of key performance indicators (KPIs) related to service quality in wireless self-organizing networks (SONs). Reinforcement learning (RL) is one of the preferred Machine Learning methods for automating the choice of RET for all the...

Modern vehicles contain a multitude of electronic control units (ECUs) that implement software features controlling most of the operational, entertainment, connectivity, and safety aspects of the vehicle. However, with security requirements often being an afterthought in automotive software development, incorporation of such software features with...

Modern Cyber-Physical Systems (CPSs) are often designed as networked, software-based controller implementations which have been found to be vulnerable to network-level and physical level attacks. A number of research works have proposed CPS-specific attack detection schemes as well as techniques for attack resilient controller design. However, such...

In the past decade, heterogeneous multicore architectures with support for Single Instruction Multiple Thread (SIMT) style computing have become the standard platform of choice for scheduling HPC applications. Here, applications are typically modelled as a set of data-parallel tasks with dependencies represented in the form of a directed acyclic gr...

Recent developments in the smart mobility domain have transformed automobiles into networked transportation agents helping realize new age, large-scale intelligent transportation systems (ITS). The motivation behind such networked transportation is to improve road safety as well as traffic efficiency. In this setup, vehicles can share information a...

Deep Reinforcement Learning (DRL) has been successfully applied to learn policies for safety-critical systems with unknown model dynamics in simulation. DRL controllers though optimal in terms of reward, do not provide any safety and stability guarantees. With reliance on model information, safety conditions can be expressed as Control Barrier Func...

In the past decade, high performance compute capabilities exhibited by heterogeneous GPGPU platforms have led to the popularity of data parallel programming languages such as CUDA and OpenCL. Developing high performance parallel programming solutions using such languages involve a steep learning curve due to the complexity of the underlying heterog...

Recent trends in real-time applications have raised the demand for high-throughput embedded platforms with integrated CPU-GPU based Systems-On-Chip (SoCs). The enhanced performance of such SoCs, however, comes at the cost of increased power consumption, resulting in significant heat dissipation and high on-chip temperatures. The prolonged occurrenc...

Building loads consume roughly 40% of the energy produced in developed countries, a significant part of which is invested towards building temperature-control infrastructure. Therein, renewable resource-based microgrids offer a greener and cheaper alternative. This communication explores the possible co-design of microgrid power dispatch and buildi...

In recent years, the transitioning of conventional power grid system into the smart grid infrastructure has made the power distribution network more susceptible towards faults and physical attacks. In this context, we discuss recently proposed Manipulation-of-Demand via IoT attack, False Data Injection Attacks and Electric Fault Attacks. These atta...

Increased dependence on networked, software based control has escalated the vulnerabilities of Cyber Physical Systems (CPSs). Detection and monitoring components developed leveraging dynamical systems theory are often employed as lightweight security measures for protecting such safety critical CPSs against false data injection attacks. However, ex...

A large fraction of bugs discovered in the design flow of Embedded Control Software (ECS) embedded control software arises from the control software’s interaction with the plant it controls. Traditional formal analysis approaches using interleaved controller-plant reach-set analysis grossly over-approximate the reachable states and does not scale....

The increasing demand for mapping diverse embedded features onto shared electronic control units has brought about novel ways to co-design control tasks and their schedules. These techniques replace traditional implementations of control with new methods, such as pattern-based scheduling of control tasks and adaptive sharing of bandwidth among cont...

Bug traces serve as references for patching a microprocessor design after a bug has been found. Unless the root cause of a bug has been detected and patched, variants of the bug may return through alternative bug traces, following a different sequence of micro‐architectural events. To avoid such a situation, the verification engineer must think of...

In the past decade, high performance compute capabilities exhibited by heterogeneous GPGPU platforms have led to the popularity of data parallel programming languages such as CUDA and OpenCL. Such languages, however, involve a steep learning curve as well as developing an extensive understanding of the underlying architecture of the compute devices...

In recent years, the conventional power grid system has been streamlined towards Smart grid infrastructure that empowers two-way communication between the consumers and the utility providers. This however also makes the grid more susceptible towards faults as well as physical and cyber attacks. In this work, we propose a Physically Unclonable Funct...

We consider the problem of provably securing a given control loop implementation in the presence of adversarial interventions on data exchange between plant and controller. Such interventions can be thwarted using continuously operating monitoring systems and also cryptographic techniques, both of which consume network and computational resources....

This contribution introduces the notion of cyber-physical systems (CPSs) and presents the questions which the corresponding Special Issue on CPS Education is trying to answer. For this special issue, it provides an overview of the article selection process and the finally selected papers. — Peter Marwedel, TU Dortmund</italic

Heating ventilation and air conditioning (HVAC) systems usually account for the highest percentage of overall energy usage in large-sized smart building infrastructures. The performance of HVAC control systems for large buildings strongly depend on the outside environment, building architecture, and (thermal) zone usage pattern of the building. In...

Given the widespread deployment of cyber-physical systems and their safety-critical nature, reliability and security guarantees offered by such systems are of paramount importance. While the security of such systems against sensor attacks have garnered significant attention from researchers in recent times, improving the reliability of a control so...

In the energy sector, IoT manifests in the form of next-generation power grids that provide enhanced electrical stability, efficient power distribution and utilization. The primary feature of a Smart Grid is the presence of an advanced bi-directional communication network between the Smart meters at the consumer end and the servers at the Utility O...

Advanced Driver-Assistance Systems (ADAS) is one of the primary drivers behind increasing levels of autonomy, driving comfort in this age of connected mobility. However, the performance of such systems is a function of execution rate which demands on-board platform-level support. With GPGPU platforms making their way into automobiles, there exists...

We consider the problem of securing a given control loop implementation of a cyber-physical system (CPS) in the presence of Man-in-the-Middle attacks on data exchange between plant and controller over a compromised network. To this end, there exist various detection schemes that provide mathematical guarantees against such attacks for the theoretic...

Executing a set of control loops over a shared multi-hop (wireless) control network (MCN) requires careful co-scheduling of the control tasks and the routing of sensory/actuation messages over the MCN. In this work, we establish pattern guided aperiodic execution of control loops as a resource-aware alternative to traditional fully periodic executi...

Modern electronic systems become evermore complex, yet remain modular, with integrated circuits (ICs) acting as versatile hardware components at their heart. Electronic design automation (EDA) for ICs has focused traditionally on power, performance, and area. However, given the rise of hardware-centric security threats, we believe that EDA must als...

Embedded control systems are prevalent in a multitude of domains such as automotive, avionics, industrial control etc. For such systems, robustness against non‐idealities of the compute platform created by situations such as hardware level transient faults (memory errors, sensor reading errors), network packet drops, late arrival of messages etc.,...

This work summarizes established vulnerability analysis techniques for cyber physical control software and highlights the possibility of employing lightweight security measures in conjunction with reachability analysis for such systems. We propose a possible cyber physical control system implementation methodology which leverages the dependence of...

With the emerging migration of automotive and other distributed control platforms from federated to integrated architectures, the need for optimal utilization of ECU (electronic control unit) bandwidth will become a key requirement in the implementation of embedded control features. This paper advocates the partitioning of the operating space of th...

We present an intelligent scheduling framework which takes as input a set of OpenCL kernels and distributes the workload across multiple CPUs and GPUs in a heterogeneous multicore platform. The framework relies on a Machine Learning (ML) based frontend that analyzes static program features of OpenCL kernels and predicts the ratio in which kernels a...

We propose pattern based execution of control loops as a preferable alternative to traditional fully periodic execution for implementing a set of embedded control systems where the sensors, actuators and control nodes are connected via a shared wireless network. We use a theoretically sound model for skipping loop executions without compromising st...

Software controllers are increasingly prevalent in embedded applications controlling safety-critical physical processes. The actual stability guarantee offered by such software implementations depend on errors induced in plant state information due to sensor level faults caused either by transient errors in sensing hardware or adversaries with the...

Software implementation of multiple embedded control loops often share compute resources. The control performance of such implementations have been shown to improve if the sharing of bandwidth between control loops can be dynamically regulated in response to input disturbances. In the absence of a structured methodology for planning such measures,...

With the increase in the complexity of safety-critical embedded applications, the reliability analysis of such systems have also become increasingly difficult. For such complex system specifications, if the reliability provisions are declared upfront in the design flow then the overall system level reliability can be easily inferred given that the...

Behavioral specifications are often employed for modeling complex systems at high levels of abstraction. Failure conditions of such systems can naturally be specified as assertions defined over system variables. In that way, such behavioral descriptions can be transformed to imperative programs with annotated failure assertions. In this paper, we p...

We consider the problem of incorporating control tasks on top of a partially loaded shared computing resource, whose current task execution pattern is characterizable using a window based pattern. We consider that the control task to be scheduled is allowed to switch between multiple controllers, each with different associated sampling rate, in ord...

Heterogeneous partitioning is a key step for efficient mapping and scheduling of data parallel applications on multi-core computing platforms involving both CPUs and GPUs. Over the last few years, several automated partitioning methodologies, both static as well as dynamic, have been proposed for this purpose. The present work provides an in-depth...

Recent control theoretic studies have shown that the sharing of electronic control unit (ECU) bandwidth among multiple controllers can be adaptively regulated to achieve better control performance as opposed to static bandwidth allocation to each controller. Unfortunately the ideal controllers cannot always be implemented, since the computational p...

The increasing complexity of safety-critical embedded applications have made it imperative to specify and analyze reliability upfront in the design flow so that reliable systems can be automatically synthesized adhering to such descriptions. This paper develops a framework, RELSPEC, to express the reliability of a safety-critical embedded applicati...

Recent studies have shown that adaptively regulating the sampling rate results in significant reduction in the computational resources of embedded software based control. Selecting a uniform sampling rate for a control loop is robust, but pessimistic for sharing processors among multiple control loops. Fine-grained regulation of periodicity achieve...

Recent studies have shown that adaptively regulating the sampling rate results in significant reduction in computational resources in embedded software based control. Selecting a uniform sampling rate for a control loop is robust, but overtly pessimistic for sharing processors among multiple control loops. Fine grained regulation of periodicity ach...

The tagged signal model (TSM) is a formal framework for modeling heterogeneous embedded systems. In the present work, we provide a representation of tagged systems using the semantics of Kleene algebra. We further illustrate mechanisms for both behavioral transformational verification through equivalence checking and property verification of hetero...

Embedded multimedia systems often run multiple time-constrained applications simultaneously. To meet the throughput constraints given in the specification, each application must be provided with enough resources by the underlying architecture, which is generally a multiprocessor system-on-chip (MPSoC). For this purpose, a mechanism for task binding...

Around 2.1 million people in India are affected by severe speech and motor impairments (SSMI). Unfortunately only a minute fraction of them have access to Advanced Assistive Devices (AAD) which are still very expensive and beyond affordability for a major portion of the people in India. Moreover, such devices generate text messages or speech mostly...

Approximately 60 million people in India fall under the category of physically or mentally disabled out of which 2.1 million people are affected with severe speech and motor impairments (SSMI). Unfortunately only a minute fraction of them have any access whatsoever to Advanced Assistive Devices (AAD) which are still very expensive and beyond the af...

Evaluation of software performance on a given cus- tomized embedded processor is an important step in the design space exploration of embedded system architec- tures. Such evaluations help system designers in taking early design decisions regarding the hardware architec- ture most suitable for the target application. Simulation based performance ev...