Marc Hanheide

• PhD
• Professor (Full) at University of Lincoln

Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system's complexity, requiring the integration of several nontrivial components for the operation of each robot but also...

In autonomous navigation, trajectory replanning, refinement, and control command generation are essential for effective motion planning. This paper presents a resilient approach to trajectory replanning addressing scenarios where the initial planner's solution becomes infeasible. The proposed method incorporates a hybrid A* algorithm to generate fe...

The study of cause and effect is of the utmost importance in many branches of science, but also for many practical applications of intelligent systems. In particular, identifying causal relationships in situations that include hidden factors is a major challenge for methods that rely solely on observational data for building causal models. This art...

The introduction of mobile robots to assist pickers in transporting crops during fruit harvesting operations is a promising solution to mitigate the impacts of the current labour shortage. However, having robots sharing the workspace with humans involves solving new challenges related to Human-Robot Interaction (HRI). For instance, effective Human-...

The study of cause-and-effect is of the utmost importance in many branches of science, but also for many practical applications of intelligent systems. In particular, identifying causal relationships in situations that include hidden factors is a major challenge for methods that rely solely on observational data for building causal models. This pap...

In recent years, camera-based object detection models have been highly used to allow robots to perceive and understand what is in their surroundings. Although modern detection models have been demonstrated to be reliable solutions in certain controlled environments, their performance can be reduced when applied to outdoor robotic applications where...

Selective harvesting by autonomous robots will be a critical enabling technology for future farming. Increases in inflation and shortages of skilled labor are driving factors that can help encourage user acceptability of robotic harvesting. For example, robotic strawberry harvesting requires real‐time high‐precision fruit localization, three‐dimens...

Selective harvesting by autonomous robots will be a critical enabling technology for future farming. Increases in inflation and shortages of skilled labour are driving factors that can help encourage user acceptability of robotic harvesting. For example, robotic strawberry harvesting requires real-time high-precision fruit localisation, 3D mapping...

Autonomous mobile robots can rely on several human motion detection and prediction systems for safe and efficient navigation in human environments, but the underline model architectures can have different impacts on the trustworthiness of the robot in the real world. Among existing solutions for context-aware human motion prediction, some approache...

Deploying robots in human-shared environments requires a deep understanding of how nearby agents and objects interact. Employing causal inference to model cause-and-effect relationships facilitates the prediction of human behaviours and enables the anticipation of robot interventions. However, a significant challenge arises due to the absence of im...

Current technology has made it possible to automate a number of agricultural processes that were traditionally carried out by humans and now can be entirely performed by robotic platforms. However, there are certain tasks like soft fruit harvesting, where human skills are still required. In this case, the robot's job is to cooperate/collaborate wit...

Field-deployed robotic fleets can provide solutions that improve operational efficiency, control operational costs, and provide farmers with transparency over day-to-day scouting operations. The topology of agricultural environments, such as polytunnels, provides a basic configuration that can be exploited to create topological maps aiding operatio...

Mobile robots increasingly operate in real-world environments that are subject to change over time. Accurate and robust localization is, however, crucial for the effective operation of autonomous mobile systems. In this paper, we tackle the challenge of developing a generalizable learned filter for long-term localization based on scan-to-map matchi...

Driven by the increasing food demand and the need for higher-quality cultivation, precision agriculture grows steadily during the last decade. It involves the application of mobile robots and intelligent robotic technologies in various agricultural field tasks, concerning a variety of crop types. Aiming at compensating for the lack of selective rob...

A typical commercial berry farm is spread across quite a few hectares and has over hundreds of rows and tens of pickers operating on a daily basis. The berries in these farms are grown in a poly-tunnel parallel row based environment. This work develops a heuristic assignment technique called Smart Parking. Smart Parking is used for allocating parki...

Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system’s complexity, requiring the integration of several non-trivial components for the operation of each robot but also...

Robotic mapping provides spatial information for autonomous agents. Depending on the tasks they seek to enable, the maps created range from simple 2D representations of the environment geometry to complex, multilayered semantic maps. This survey article is about maps of dynamics (MoDs), which store semantic information about typical motion patterns...

Achieving a robust long‐term deployment with mobile robots in the agriculture domain is both a demanded and challenging task. The possibility to have autonomous platforms in the field performing repetitive tasks, such as monitoring or harvesting crops, collides with the difficulties posed by the always‐changing appearance of the environment due to...

In the last decades, robotic solutions have been introduced in agriculture to improve the efficiency of tasks such as spraying, plowing, and seeding. However, for a more complex task like soft-fruit harvesting, the efficiency of experienced human pickers has not been surpassed yet by robotic solutions. Thus, in the immediate future, human labor wil...

Deploying service robots in our daily life, whether in restaurants, warehouses or hospitals, calls for the need to reason on the interactions happening in dense and dynamic scenes. In this paper, we present and benchmark three new approaches to model and predict multi-agent interactions in dense scenes, including the use of an intuitive qualitative...

Reasoning on the context of human beings is crucial for many real-world applications especially for those deploying autonomous systems (e.g. robots). In this paper, we present a new approach for context reasoning to further advance the field of human motion prediction. We therefore propose a neuro-symbolic approach for human motion prediction (Neur...

Identifying the main features and learning the causal relationships of a dynamic system from time-series of sensor data are key problems in many real-world robot applications. In this paper, we propose an extension of a state-of-the-art causal discovery method, PCMCI, embedding an additional feature-selection module based on transfer entropy. Start...

Exploiting robots for activities in human-shared environments, whether warehouses, shopping centres or hospitals, calls for such robots to understand the underlying physical interactions between nearby agents and objects. In particular, modelling cause-and-effect relations between the latter can help to predict unobserved human behaviours and antic...

3D point cloud semantic classification is an important task in robotics as it enables a better understanding of the mapped environment. This work proposes to learn the long-term stability of the 3D objects using a neural network based on PointNet++, where the long-term stable object refers to a static object that cannot move on its own (e.g. tree,...

Current intralogistics services require keeping up with e-commerce demands, reducing delivery times and waste, and increasing overall flexibility. As a consequence, the use of automated guided vehicles (AGVs) and, more recently, autonomous mobile robots (AMRs) for logistics operations is steadily increasing.

Long-term autonomy is one of the most demanded capabilities looked into a robot. The possibility to perform the same task over and over on a long temporal horizon, offering a high standard of reproducibility and robustness, is appealing. Long-term autonomy can play a crucial role in the adoption of robotics systems for precision agriculture, for ex...

Yield forecasting is a critical first step necessary for yield optimisation, with important consequences for the broader food supply chain, procurement, price-negotiation, logistics, and supply. However yield forecasting is notoriously difficult, and oft-inaccurate. Premonition Net is a multi-timeline, time sequence ingesting approach towards proce...

Exploiting robots for activities in human-shared environments, whether warehouses, shopping centres or hospitals, calls for such robots to understand the underlying physical interactions between nearby agents and objects. In particular, modelling cause-and-effect relations between the latter can help to predict unobserved human behaviours and antic...

The majority of motion planning strategies developed over the literature for reaching an object in clutter are applied to two dimensional (2-d) space where the state space of the environment is constrained in one direction. Fewer works have been investigated to reach a target in 3-d cluttered space, and when so, they have limited performance when a...

We present automatically parameterised Fully Homomorphic Encryption (FHE) for encrypted neural network inference and exemplify our inference over FHE-compatible neural networks with our own open-source framework and reproducible examples. We use the fourth generation Cheon, Kim, Kim, and Song (CKKS) FHE scheme over fixed points provided by the Micr...

In this work, we propose a framework for allowing autonomous robots deployed for extended periods of time in public spaces to adapt their own behaviour online from user interactions. The robot behaviour planning is embedded in a Reinforcement Learning (RL) framework, where the objective is maximising the level of overall user engagement during the...

The majority of motion planning strategies developed over the literature for reaching an object in clutter are applied to two dimensional (2-d) space where the state space of the environment is constrained in one direction. Fewer works have been investigated to reach a target in 3-d cluttered space, and when so, they have limited performance when a...

In this work, we propose a framework for allowing autonomous robots deployed for extended periods of time in public spaces to adapt their own behaviour online from user interactions. The robot behaviour planning is embedded in a Reinforcement Learning (RL) framework, where the objective is maximising the level of overall user engagement during the...

Tactile sensing provides essential information about the state of the world for the robotic system to perform a successful and robust manipulation task. Integrating and exploiting tactile sensation enables the robotic systems to perform wider variety of manipulation tasks in unstructured environments relative to pure vision based systems. While sli...

In this work, we present a comparative analysis of the trajectories estimated from various Simultaneous Localization and Mapping (SLAM) systems in a simulation environment for vineyards. Vineyard environment is challenging for SLAM methods, due to visual appearance changes over time, uneven terrain, and repeated visual patterns. For this reason, we...

We present automatically parameterised Fully Homomorphic Encryption (FHE), for encrypted neural network inference. We present and exemplify our inference over FHE compatible neural networks with our own open-source framework and reproducible step-by-step examples. We use the 4th generation Cheon, Kim, Kim and Song (CKKS) FHE scheme over fixed point...

In this work, we present a comparative analysis of the trajectories estimated from various Simultaneous Localization and Mapping (SLAM) systems in a simulation environment for vineyards. Vineyard environment is challenging for SLAM methods, due to visual appearance changes over time, uneven terrain, and repeated visual patterns. For this reason, we...

The agricultural domain offers a working environment where many human laborers are nowadays employed to maintain or harvest crops, with huge potential for productivity gains through the introduction of robotic automation. Detecting and localizing humans reliably and accurately in such an environment, however, is a prerequisite to many services offe...

The agricultural domain offers a working environment where many human laborers are nowadays employed to maintain or harvest crops, with huge potential for productivity gains through the introduction of robotic automation. Detecting and localizing humans reliably and accurately in such an environment, however, is a prerequisite to many services offe...

This paper describes a hazard analysis for an agricultural scenario where a crop is treated by a robot using UV-C light. Although human-robot interactions are not expected, it may be the case that unauthorized people approach the robot while it is operating. These potential human-robot interactions have been identified and modeled as Markov Decisio...

This paper describes our work to assure safe autonomy in soft fruit production. The first step was hazard analysis, where all the possible hazards in representative scenarios were identified. Following this analysis, a three-layer safety architecture was identified that will minimise the occurrence of the identified hazards. Most of the hazards are...

Soft robotic grippers are increasingly desired in applications that involve grasping of complex and deformable objects. However, their flexible nature and non-linear dynamics makes the modelling and control difficult. Numerical techniques such as Finite Element Analysis (FEA) present an accurate way of modelling complex deformations. However, FEA a...

The volume LNAI 13054 constitutes the refereed proceedings of the 22th Annual Conference Towards Autonomous Robotic Systems, TAROS 2021, held in Lincoln, UK, in September 2021.* The 45 full papers were carefully reviewed and selected from 66 submissions. Organized in the topical sections "Algorithms" and "Systems", they discuss significant findings...

For adoption of Autonomous Mobile Robots (AMR) across a breadth of industries, they must navigate around humans in a way which is safe and which humans perceive as safe, but without greatly compromising efficiency. This work aims to classify the Human-Robot Spatial Interaction (HRSI) situation of an interacting human and robot, to be applied in Hum...

Estimating accurate forward and inverse dynamics models is a crucial component of model-based control for sophisticated robots such as robots driven by hydraulics, artificial muscles, or robots dealing with different contact situations. Analytic models to such processes are often unavailable or inaccurate due to complex hysteresis effects, unmodell...

Estimating accurate forward and inverse dynamics models is a crucial component of model-based control for sophisticated robots such as robots driven by hydraulics, artificial muscles, or robots dealing with different contact situations. Analytic models to such processes are often unavailable or inaccurate due to complex hysteresis effects, unmodell...

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way toward in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behavior optimization. However, engagement is often considered very multi-faceted and difficult to capture in a worka...

We propose a haptic-guided shared control system that provides an operator with force cues during reach-to-grasp phase of tele-manipulation. The force cues inform the operator of grasping configuration which allows collision-free autonomous post-grasp movements. Previous studies showed haptic guided shared control significantly reduces the complexi...

Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cas...

Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cas...

Automated exploration is one of the most relevant applications for autonomous robots. In this letter, we propose a novel online coverage algorithm called Next-Best-Sense (NBS), an extension of the Next-Best-View class of exploration algorithms which optimizes the exploration task balancing multiple criteria. NBS is applied to the problem of localiz...

We propose a new pipeline to facilitate deep learning at scale for agriculture and food robotics, and exemplify it using strawberry tabletop. We use this multimodal, autonomously self-collected, distributed dataset for predicting strawberry tabletop yield, aiming at informing both agronomists and creating a robotic attention system. We call this sy...

We develop a taxonomy that categorizes HRI failure types and their impact on trust to structure the broad range of knowledge contributions. We further identify research gaps in order to support fellow researchers in the development of trustworthy robots. Studying trust repair in HRI has only recently been given more interest and we propose a taxono...

The autonomous landing of an Unmanned Aerial Vehicle (UAV) on a marker is one of the most challenging problems in robotics. Many solutions have been proposed, with the best results achieved via customized geometric features and external sensors. This paper discusses for the first time the use of deep reinforcement learning as an end-to-end learning...

In this paper, we present a novel grasp planning algorithm for unknown objects given a registered point cloud of the target from different views. The proposed methodology requires no prior knowledge of the object, nor offline learning. In our approach, the gripper kinematic model is used to generate a point cloud of each finger workspace, which is...

In this paper, enhancement to the novel grasp planning algorithm based on gripper workspace spheres is presented. Our development requires a registered point cloud of the target from different views, assuming no prior knowledge of the object, nor any of its properties. This work features a new set of metrics for grasp pose candidates evaluation, as...

Continuously measuring the engagement of users with a robot in a Human-Robot Interaction (HRI) setting paves the way towards in-situ reinforcement learning, improve metrics of interaction quality, and can guide interaction design and behaviour optimisation. However, engagement is often considered very multi-faceted and difficult to capture in a wor...

The use of robots in educational and STEM engagement activities is widespread. In this paper we describe a system developed for engaging learners with the design of dialogue-based interactivity for mobile robots. With an emphasis on a web-based solution that is grounded in both a real robot system and a real application domain – a museum guide robo...

This work addresses the performance of several local planners for navigation of autonomous pallet trucks in the presence of humans in a simulated warehouse as well as a complementary approach developed within the ILIAD project. Our focus is to stress the open problem of a safe manoeuvrability of pallet trucks in the presence of moving humans. We pr...

The papers in this special section focus on the use of artificial intelligence (AI) for long term autonomy. Autonomous systems have a long history in the fields of AI and robotics. However, only through recent advances in technology has it been possible to create autonomous systems capable of operating in long-term, real-world scenarios. Examples i...

In this paper, we present a human-in-the-loop learning framework for mobile robots to generate effective local policies in order to recover from navigation failures in long-term autonomy. We present an analysis of failure and recovery cases derived from long-term autonomous operation of a mobile robot, and propose a two-layer learning framework tha...

Autonomous systems will play an essential role in many applications across diverse domains including space, marine, air, field, road, and service robotics. They will assist us in our daily routines and perform dangerous, dirty and dull tasks. However, enabling robotic systems to perform autonomously in complex, real-world scenarios over extended ti...

Long-term studies with autonomous robots "in the wild" (deployed in real-world human-inhabited environments) are among the most laborious and resource-intensive endeavours in Human-Robot Interaction. Even if a robot system itself is robust and well-working, the analysis of the vast amounts of user data one aims to collect and analyse poses a signif...

Autonomous systems will play an essential role in many applications across diverse domains including space, marine, air, field, road, and service robotics. They will assist us in our daily routines and perform dangerous, dirty and dull tasks. However, enabling robotic systems to perform autonomously in complex, real-world scenarios over extended ti...

The soft fruit industry is facing unprecedented challenges due to its reliance of manual labour. We are presenting a newly launched robotics initiative which will help to address the issues faced by the industry and enable automation of the main processes involved in soft fruit production. The RASberry project (Robotics and Autonomous Systems for B...

Robots in agricultural contexts are finding increased numbers of applications with respect to (partial) automation for increased productivity. However, this presents complex technical problems to be overcome, which are magnified when these robots are intended to work side-by-side with human workers. In this contribution we present an exploratory pi...

Table-top object manipulation is a well-established test bed on which to study both basic foundations of general human-robot interaction and more specific collaborative tasks. A prerequisite, both for studies and for actual collaborative or assistive tasks, is the robust perception of any objects involved. This paper presents a real-time capable an...

Human-Robot Collaboration is an area of particular current interest, with the attempt to make robots more generally useful in contexts where they work side-by-side with humans. Currently, efforts typically focus on the sensory and motor aspects of the task on the part of the robot to enable them to function safely and effectively given an assigned...

This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the wor...

In this paper, we describe a navigation system requiring very few computational resources, but still providing performance comparable with commonly used tools in the ROS universe. This lightweight navigation system is thus suitable for robots with low computational resources and provides interfaces for both ROS and NAOqi middlewares. We have succes...

With use cases that range from external localisation of single robots or robotic swarms to self-localisation in marker-augmented environments and simplifying perception by tagging objects in a robot's surrounding, fiducial markers have a wide field of application in the robotic world. We propose a new family of circular markers which allow for both...

This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the wor...

We present a system for automated benchmarking of robotics experiments. The system is based on open source, freely-available tools commonly used in software development. While it allows for a seamless and fair comparison of a newly developed method with the original one, it does not require disclosure of neither the original codes nor evaluation da...

Fiducial markers have a wide field of applications in robotics, ranging from external localisation of single robots or robotic swarms, over self-localisation in marker-augmented environments, to simplifying perception by tagging objects in a robot's surrounding. We propose a new family of circular markers allowing for a computationally efficient de...

Adapting to users' intentions is a key requirement for autonomous robots in general, and in care settings in particular. In this paper, a comprehensive long-term study of a mobile robot providing information services to residents, visitors, and staff of a care home is presented, with a focus on adapting to the when and where the robot should be off...

Older adults represent a new user group of robots that are deployed in their private homes or in care facilities. In the presented study tangible aspects of older adults' interaction with an autonomous robot were focused. The robot was deployed as a companion in physical therapy for older adults with progressed dementia. Interaction was possible vi...

In this paper, we present a case study to investigate the effects of educational robotics on a formal undergraduate Computer Science education in a developing country. The key contributions of this paper include a longitudinal study design, spanning the whole duration of one taught course, and its focus on continually assessing the effectiveness an...