Ivana Dusparic

Ivana Dusparic
Trinity College Dublin | TCD · School of Computer Science and Statistics

PhD

About

102
Publications
16,994
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
917
Citations

Publications

Publications (102)
Chapter
Despite notable results in various fields over the recent years, deep reinforcement learning (DRL) algorithms lack transparency, affecting user trust and hindering deployment to high-risk tasks. Causal confusion refers to a phenomenon where an agent learns spurious correlations between features which might not hold across the entire state space, pr...
Preprint
Full-text available
Explainability of Reinforcement Learning (RL) policies remains a challenging research problem, particularly when considering RL in a safety context. Understanding the decisions and intentions of an RL policy offer avenues to incorporate safety into the policy by limiting undesirable actions. We propose the use of a Boolean Decision Rules model to c...
Article
In a system with energy-constrained sensors, each transmitted observation comes at a price. The price is the energy the sensor expends to obtain and send a new measurement. The system has to ensure that sensors’ updates are timely, i.e., their updates represent the observed phenomenon accurately, enabling services to make informed decisions based o...
Preprint
Fast identification of new network attack patterns is crucial for improving network security. Nevertheless, identifying an ongoing attack in a heterogeneous network is a non-trivial task. Federated learning emerges as a solution to collaborative training for an Intrusion Detection System (IDS). The federated learning-based IDS trains a global model...
Preprint
In this letter, we study the energy efficiency (EE) optimisation of unmanned aerial vehicles (UAVs) providing wireless coverage to static and mobile ground users. Recent multi-agent reinforcement learning approaches optimise the system's EE using a 2D trajectory design, neglecting interference from nearby UAV cells. We aim to maximise the system's...
Preprint
Full-text available
Despite notable results in various fields over the recent years, deep reinforcement learning (DRL) algorithms lack transparency, affecting user trust and hindering their deployment to high-risk tasks. Causal confusion refers to a phenomenon where an agent learns spurious correlations between features which might not hold across the entire state spa...
Preprint
Full-text available
Deep Reinforcement Learning (DRL) solutions are becoming pervasive at the edge of the network as they enable autonomous decision-making in a dynamic environment. However, to be able to adapt to the ever-changing environment, the DRL solution implemented on an embedded device has to continue to occasionally take exploratory actions even after initia...
Article
Full-text available
Context-oriented Programming ( COP) first appeared in 2005 as a way to enable the dynamic adaptation of software systems to specific situations in their surrounding environment. Multiple COP languages have since been proposed, and used in numerous adaptive systems areas, enabling dynamic swapping and composition of adaptive behavior at run-time. Ho...
Preprint
Full-text available
In complex tasks where the reward function is not straightforward and consists of a set of objectives, multiple reinforcement learning (RL) policies that perform task adequately, but employ different strategies can be trained by adjusting the impact of individual objectives on reward function. Understanding the differences in strategies between pol...
Preprint
Full-text available
Reinforcement learning (RL) has been used in a range of simulated real-world tasks, e.g., sensor coordination, traffic light control, and on-demand mobility services. However, real world deployments are rare, as RL struggles with dynamic nature of real world environments, requiring time for learning a task and adapting to changes in the environment...
Article
Full-text available
The prevailing variable speed limit (VSL) systems as an effective strategy for traffic control on motorways have the disadvantage that they only work with static VSL zones. Under changing traffic conditions, VSL systems with static VSL zones may perform suboptimally. Therefore, the adaptive design of VSL zones is required in traffic scenarios where...
Article
Unmanned Aerial Vehicles (UAVs) are emerging as important users of next-generation cellular networks. By operating in the sky, UAV users experience very different radio conditions than terrestrial users, due to factors such as strong Line-of-Sight (LoS) channels (and interference) and Base Station (BS) antenna misalignment. As a consequence, the UA...
Preprint
Unmanned Aerial Vehicles (UAVs) promise to become an intrinsic part of next generation communications, as they can be deployed to provide wireless connectivity to ground users to supplement existing terrestrial networks. The majority of the existing research into the use of UAV access points for cellular coverage considers rotary-wing UAV designs (...
Preprint
Full-text available
Millions of sensors, cameras, meters, and other edge devices are deployed in networks to collect and analyse data. In many cases, such devices are powered only by Energy Harvesting(EH) and have limited energy available to analyse acquired data. When edge infrastructure is available, a device has a choice: to perform analysis locally or offload the...
Preprint
Unmanned aerial vehicles serving as aerial base stations (UAV-BSs) can be deployed to provide wireless connectivity to ground devices in events of increased network demand, points-of-failure in existing infrastructure, or disasters. However, it is challenging to conserve the energy of UAVs during prolonged coverage tasks, considering their limited...
Article
Creating sustainable urban futures partly requires reducing car-use and transport induced stresses on the environment and society. New transport technologies such as autonomous vehicles are increasingly assuming prominence in debates about the transition toward sustainable urban futures. Yet, enormous uncertainties currently exist on how autonomous...
Preprint
Full-text available
Self-adaptive software systems continuously adapt in response to internal and external changes in their execution environment, captured as contexts. The COP paradigm posits a technique for the development of self-adaptive systems, capturing their main characteristics with specialized programming language constructs. COP adaptations are specified as...
Preprint
Full-text available
Self-adaptive systems continuously adapt to changes in their execution environment. Capturing all possible changes to define suitable behaviour beforehand is unfeasible, or even impossible in the case of unknown changes, hence human intervention may be required. We argue that adapting to unknown situations is the ultimate challenge for self-adaptiv...
Preprint
Full-text available
Unmanned Aerial Vehicle (UAV) technology is becoming more prevalent and more diverse in its application. 5G and beyond networks must enable UAV connectivity. This will require the network operator to consider this new type of user in the planning and operation of the network. This work presents the challenges an operator will encounter and should c...
Article
Full-text available
Enabling Ride-sharing (RS) in Mobility-on-demand (MoD) systems allows reduction in vehicle fleet size while preserving the level of service. This, however, requires an efficient vehicle to request assignment, and a vehicle rebalancing strategy, which counteracts the uneven geographical spread of demand and relocates unoccupied vehicles to the areas...
Conference Paper
Full-text available
Suspension control systems in cars are a vital component in modern vehicles tasked with enhancing ride comfort for passengers. However, these systems rely on system controllers to dictate the damping rate. Commonly PID controllers are used for this purpose, but these controllers have a number of drawbacks such as their linearity and inability to ad...
Preprint
Unmanned Aerial Vehicle (UAV) technology is becoming increasingly used in a variety of applications such as video surveillance and deliveries. To enable safe and efficient use of UAVs, the devices will need to be connected into cellular networks. Existing research on UAV cellular connectivity shows that UAVs encounter significant issues with existi...
Preprint
Full-text available
Self-adaptive systems continuously adapt to internal and external changes in their execution environment. In context-based self-adaptation, adaptations take place in response to the characteristics of the execution environment, captured as a context. However, in large-scale adaptive systems operating in dynamic environments, multiple contexts are o...
Preprint
Unmanned Aerial Vehicles (UAVs) are emerging as important users of next-generation cellular networks. By operating in the sky, these UAV users experience very different radio conditions than terrestrial users, due to factors such as strong Line-of-Sight (LoS) channels (and interference) and Base Station (BS) antenna misalignment. The consequence of...
Conference Paper
Full-text available
Variable Speed Limit (VSL) is a traffic control approach that optimises the mainstream traffic on motorways. Reinforcement Learning approach to VSL has been shown to achieve improvements in controlling the mainstream traffic bottleneck on motorways. However, single-agent VSL, applied to a shorter motorway segment, can produce a discontinuity in tra...
Conference Paper
Full-text available
Connected and Autonomous Vehicles (CAVs) are expected to bring major transformations to transport efficiency and safety. Studies show a range of possible impacts, from worse efficiency of CAVs at low penetration rates, to significant improvements in both efficiency and safety at high penetration rates and loads. However, these studies tend to explo...
Conference Paper
Full-text available
Mobility-on-Demand (MoD) systems offer a flexible mobility alternative to classical public transportation services in urban areas. However, a significant part of MoD vehicles operating time can be spent waiting empty or driving to reach new potential ride requests. Improving vehicle fleet operation is an extremely challenging problem, as the number...
Preprint
Full-text available
With the increasing number of \acp{uav} as users of the cellular network, the research community faces particular challenges in providing reliable \ac{uav} connectivity. A challenge that has limited research is understanding how the local building and \ac{bs} density affects \ac{uav}'s connection to a cellular network, that in the physical layer is...
Preprint
Full-text available
Collective Adaptive Systems (CAS) are increasingly used as a model for building and managing large-scale cyber-physical systems, highlighting the need for a flexible approach for their definition and engineering. CAS consist of multiple individual context-aware self-adaptive systems, which need to adapt to their own environment, but also interact w...
Preprint
Variable Speed Limit (VSL) is a traffic control approach that optimises the mainstream traffic on motorways. Reinforcement Learning approach to VSL has been shown to achieve improvements in controlling the mainstream traffic bottleneck on motorways. However, single-agent VSL, applied to a shorter motorway segment, can produce a discontinuity in tra...
Conference Paper
Full-text available
Enabling Ride-sharing (RS) in existing Mobility-on-demand (MoD) systems allows to reduce the operating vehicle fleet size while achieving a similar level of service. This however requires an efficient vehicle to multiple requests assignment, which is the focus of most RS-related research, and an adaptive fleet rebalancing strategy, which counteract...
Conference Paper
Full-text available
Deep Learning (DL) is powerful family of algorithms used for a wide variety of problems and systems, including safety critical systems. As a consequence, analyzing, understanding, and testing DL models is attracting more practitioners and researchers with the purpose of implementing DL systems that are robust, reliable, efficient, and accurate. Fir...
Article
Full-text available
Autonomous cars controlled by an artificial intelligence are increasingly being integrated in the transport portfolio of cities, with strong repercussions for the design and sustainability of the built environment. This paper sheds light on the urban transition to autonomous transport, in a threefold manner. First, we advance a theoretical framewor...
Article
Full-text available
Mobility-on-demand systems consisting of shared autonomous vehicles (SAVs) are expected to improve the efficiency of urban mobility through reduced vehicle ownership and parking demand. However, several issues in their implementation remain open, such as unifying the vehicle and ride-sharing assignment with rebalancing non-occupied vehicles. Furthe...
Article
Full-text available
Participatory sensing is a process whereby mobile device users (or participants) collect environmental data on behalf of a service provider who can then build a service based upon these data. To attract submissions of such data, the service provider will often need to incentivize potential participants by offering a reward. However, for the privacy...
Conference Paper
Full-text available
With increasing applications of reinforcement learning in real life problems, it is becoming essential that agents are able to update their knowledge continually. Lifelong learning approaches aim to enable agents to retain the knowledge they learn and to selectively transfer knowledge to new tasks. Recent techniques for lifelong reinforcement learn...
Conference Paper
Full-text available
Multi-agent Reinforcement Learning (RL) is frequently used in large-scale autonomous systems to learn the behaviours that best suit the system's operating environment. Learning can take a significant amount of time during which an RL system's performance is necessarily suboptimal. Transfer learning (TL), a method of reusing knowledge which has been...
Conference Paper
Full-text available
Reinforcement Learning (RL) is increasingly used to achieve adaptive behaviours in Internet of Things systems relying on large amounts of sensor data. To address the need for self-adaptation in such environments, techniques for detecting environment changes and re-learning behaviours appropriate to those changes have been proposed. However, with th...
Preprint
Full-text available
Recent use of Reinforcement Learning (RL), and specifically Deep RL (DRL), in real-world scenarios is increasing the need for a structured approach to systematically test applications incorporating DRL. While no specific testing approaches exist for DRL, we discuss the applicability of testing approaches for deep neural networks to DRL applications...
Article
Full-text available
Agents frequently collaborate to achieve a shared goal or to accomplish a task that they cannot do alone. However, collaboration is difficult in open multi-agent systems where agents share constrained resources to achieve both individual and shared goals. In current approaches to collaboration, agents are organised into disjoint groups and social r...
Conference Paper
Full-text available
Reinforcement Learning (RL) has been extensively used in Urban Traffic Control (UTC) optimization due its capability to learn the dynamics of complex problems from interactions with the environment. Recent advances in Deep Reinforcement Learning (DRL) have opened up the possibilities for extending this work to more complex situations due to it over...
Article
Full-text available
Provision of smart city services often relies on users contribution, e.g., of data, which can be costly for the users in terms of privacy. Privacy risks, as well as unfair distribution of benefits to the users, should be minimized as they undermine user participation, which is crucial for the success of smart city applications. This paper investiga...
Preprint
Full-text available
Advances in renewable energy generation and introduction of the government targets to improve energy efficiency gave rise to a concept of a Zero Energy Building (ZEB). A ZEB is a building whose net energy usage over a year is zero, i.e., its energy use is not larger than its overall renewables generation. A collection of ZEBs forms a Zero Energy Co...
Article
Full-text available
Participatory sensing is a paradigm through which mobile device users (or participants) collect and share data about their environments. The data captured by participants is typically submitted to an intermediary (the service provider) who will build a service based upon this data. For a participatory sensing system to attract the data submissions...
Conference Paper
Full-text available
Recent availability of large amounts of sensor data from Internet of Things devices opens up the possibility for software systems to dynamically provide fine-grained adaptations to the observed environment conditions, rather than executing only static hard-coded behaviors. However, in current adaptive systems such adaptations still need to be speci...
Preprint
Full-text available
Several smart city services rely on users contribution, e.g., data, which can be costly for the users in terms of privacy. High costs lead to reduced user participation, which undermine the success of smart city technologies. This work develops a scenario-independent design principle, based on public good theory, for resource management in smart ci...
Poster
Full-text available
Poster presentation for the on-going research project "Surpass: how shared autonomous cars will transform cities". Presented at the launch of ENABLE- a new research programme in Ireland which aims to connect communities with smart urban environments through the Internet of Things
Article
Full-text available
Residential demand response (DR) has gained a significant increase in interest from industrial and academic communities as a means to contribute to more efficient operation of smart grids, with numerous techniques proposed to implement residential DR programmes. However, the proposed techniques have been evaluated in scenarios addressing different...
Article
Full-text available
In multi-agent systems, agents coordinate their behaviour and work together to achieve a shared goal through collaboration. However, in open multi-agent systems, selecting qualified participants to form effective collaboration communities is challenging. In such systems, agents do not have access to complete domain knowledge, they leave and join th...
Conference Paper
Full-text available
Context-oriented Programming enables dynamic behavioral adaptations with the purpose of presenting the most appropriate behavior to the situations in software systems' surrounding execution environment. However, as multiple situations may be sensed simultaneously, different such adaptations may be applicable, generating conflicts in systems' execut...
Article
Full-text available
Multi-agent reinforcement learning (MARL) is a widely researched technique for decentralised control in complex large-scale autonomous systems. Such systems often operate in environments that are continuously evolving and where agents’ actions are non-deterministic, so called inherently non-stationary environments. When there are inconsistent resul...