Patrick van der Smagt

• Prof. Dr.
• Professor at Eötvös Loránd University

We study the choice of action space in robot manipulation learning and sim-to-real transfer. We define metrics that assess the performance, and examine the emerging properties in the different action spaces. We train over 250 reinforcement learning (RL) agents in simulated reaching and pushing tasks, using 13 different control spaces. The choice of...

Neural networks are powerful machine learning models, but their reliability and trust are often criticized due to the unclear nature of their internal learned relationships. We explored neural network learning behavior in wheat yield prediction using game theory-based methods (SHapley Additive exPlanations, Shapley-like, cohort Owen), examined data...

In offline reinforcement learning, a policy is learned using a static dataset in the absence of costly feedback from the environment. In contrast to the online setting, only using static datasets poses additional challenges, such as policies generating out-of-distribution samples. Model-based offline reinforcement learning methods try to overcome t...

The central nervous system adapts the gain of short-latency reflex loops to changing conditions. Experiments on biomimetic robots showed that reflex modulation could substantially increase energy efficiency and stability of periodic motions if, unlike known mechanisms, the reflex modulation both acted precisely on the muscles involved and lasted af...

Regulatory frameworks for the use of AI are emerging. However, they trail behind the fast-evolving malicious AI technologies that can quickly cause lasting societal damage. In response, we introduce a pioneering Assistive AI framework designed to enhance human decision-making capabilities. This framework aims to establish a trust network across var...

The Finite Element Method (FEM) is a widely used technique for simulating crash scenarios with high accuracy and reliability. To reduce the significant computational costs associated with FEM, the Finite Element Method Integrated Networks (FEMIN) framework integrates neural networks (NNs) with FEM solvers. However, this integration can introduce er...

Objective Subcutaneous Immunotherapy (SCIT) is the long-lasting causal treatment of allergic rhinitis (AR). How to enhance the adherence of patients to maximize the benefit of allergen immunotherapy (AIT) plays a crucial role in the management of AIT. This study aims to leverage novel machine learning models to precisely predict the risk of non-adh...

Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuable in real-world applications. Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objec...

Robotic manipulation requires accurate motion and physical interaction control. However, current robot learning approaches focus on motion-centric action spaces that do not explicitly give the policy control over the interaction. In this paper, we discuss the repercussions of this choice and argue for more interaction-explicit action spaces in robo...

We study the choice of action space in robot manipulation learning and sim-to-real transfer. We define metrics that assess the performance, and examine the emerging properties in the different action spaces. We train over 250 reinforcement learning (RL) agents in simulated reaching and pushing tasks, using 13 different control spaces. The choice of...

This paper introduces a novel computational framework, Finite Element Method Integrated Networks (FEMIN), designed to accelerate crash simulations significantly. The core innovation is that large regions of the mesh in a Finite Element Method (FEM) crash simulation are replaced by a Neural Network (NN). The NN is directly integrated into the FEM so...

The growing adoption and deployment of Machine Learning (ML) systems came with its share of ethical incidents and societal concerns. It also unveiled the necessity to properly audit these systems in light of ethical principles. For such a novel type of algorithmic auditing to become standard practice, two main prerequisites need to be available: A...

In robotics and biomechanics, accurately determining joint parameters and computing the corresponding forward and inverse kinematics are critical yet often challenging tasks, especially when dealing with highly individualized and partly unknown systems. This paper unveils a cutting-edge kinematic optimizer, underpinned by an autoencoder-based archi...

This study presents an innovative test rig engineered to explore the kinematic and viscoelastic characteristics of human specimen hands. The rig features eight force-controlled motors linked to muscle tendons, enabling precise stimulation of hand specimens. Hand movements are monitored through an optical tracking system, while a force-torque sensor...

Multi-modality magnetic resonance imaging data with various sequences facilitate the early diagnosis, tumor segmentation, and disease staging in the management of nasopharyngeal carcinoma (NPC). The lack of publicly available, comprehensive datasets limits advancements in diagnosis, treatment planning, and the development of machine learning algori...

Type Ia supernovae (SNe) remain poorly understood despite decades of investigation. Massive computationally intensive hydrodynamic simulations have been developed and run to model an ever-growing number of proposed progenitor channels. Further complicating the matter, a large number of subtypes of Type Ia SNe have been identified in recent decades....

Objective: Subcutaneous Immunotherapy (SCIT) is the long-lasting causal treatment of allergic rhinitis. How to enhance the adherence of patients to maximize the benefit of allergen immunotherapy (AIT) plays a crucial role in the management of AIT. This study aims to leverage novel machine learning models to precisely predict the risk of non-adheren...

Many astrophysical applications require efficient yet reliable forecasts of stellar evolution tracks. One example is population synthesis, which generates forward predictions of models for comparison with observations. The majority of state-of-the-art rapid population synthesis methods are based on analytic fitting formulae to stellar evolution tra...

Type Ia supernovae remain poorly understood despite decades of investigation. Massive computationally intensive hydrodynamic simulations have been developed and run to model an ever-growing number of proposed progenitor channels. Further complicating the matter, a large number of sub-types of Type Ia supernovae have been identified in recent decade...

Partially-observable problems pose a trade-off between reducing costs and gathering information. They can be solved optimally by planning in belief space, but that is often prohibitively expensive. Model-predictive control (MPC) takes the alternative approach of using a state estimator to form a belief over the state, and then plan in state space....

We introduce PRISM, a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. Previous approaches either lack uncertainty estimates for the map and agent state, do not run in real-time, do not have a dense scene representation or do not model agent dynamics. Our solution reconciles all of these aspe...

Multi-robot manipulation tasks involve various control entities that can be separated into dynamically independent parts. A typical example of such real-world tasks is dual-arm manipulation. Learning to naively solve such tasks with reinforcement learning is often unfeasible due to the sample complexity and exploration requirements growing with the...

Supernova spectral time series can be used to reconstruct a spatially resolved explosion model known as supernova tomography. In addition to an observed spectral time series, a supernova tomography requires a radiative transfer model to perform the inverse problem with uncertainty quantification for a reconstruction. The smallest parametrizations o...

Auto-encoder models that preserve similarities in the data are a popular tool in representation learning. In this paper we introduce several auto-encoder models that preserve local distances when mapping from the data space to the latent space. We use a local distance preserving loss that is based on the continuous k-nearest neighbours graph which...

Understanding the evolution of massive binary stars requires accurate estimates of their masses. This understanding is critically important because massive star evolution can potentially lead to gravitational-wave sources such as binary black holes or neutron stars. For Wolf–Rayet (WR) stars with optically thick stellar winds, their masses can only...

The use of machine learning in artistic music generation leads to controversial discussions of the quality of art, for which objective quantification is nonsensical. We therefore consider a music-generating algorithm as a counterpart to a human musician, in a setting where reciprocal improvisation is to lead to new experiences, both for the musicia...

Understanding the evolution of massive binary stars requires accurate estimates of their masses. This understanding is critically important because massive star evolution can potentially lead to gravitational wave sources such as binary black holes or neutron stars. For Wolf-Rayet stars with optically thick stellar winds, their masses can only be d...

Intrinsic motivation is vital for living beings. It enables skill acquisitions, triggers explorative behaviour, and hence enhances cognitive capabilities. One way of formalising the variety of behaviours induced by intrinsic motivation is empowerment, an information-theoretic measure that encodes the influence an agent exerts on its environment. Fo...

We introduce a method for real-time navigation and tracking with differentiably rendered world models. Learning models for control has led to impressive results in robotics and computer games, but this success has yet to be extended to vision-based navigation. To address this, we transfer advances in the emergent field of differentiable rendering t...

Knowledge about body motion kinematics and underlying muscle contraction dynamics usually derives from electromyographic (EMG) recordings. However, acquisition of such signals in snakes is challenging because electrodes either attached to or implanted beneath the skin may unintentionally be removed by force or friction caused from undulatory motion...

Placing robots outside controlled conditions requires versatile movement representations that allow robots to learn new tasks and adapt them to environmental changes. The introduction of obstacles or the placement of additional robots in the workspace and the modification of the joint range due to faults or range-of-motion constraints are typical c...

This paper presents a novel mechatronic exoskeleton architecture for finger rehabilitation. The system consists of an underactuated kinematic structure that enables the exoskeleton to act as an adaptive finger stimulator. The exoskeleton has sensors for motion detection and control. The proposed architecture offers three main advantages. First, the...

Manual fits to spectral times series of Type Ia supernovae have provided a method of reconstructing the explosion from a parametric model but due to lack of information about model uncertainties or parameter degeneracies direct comparison between theory and observation is difficult. In order to mitigate this important problem we present a new way t...

With the increased focus on quantum circuit learning for near-term applications on quantum devices, in conjunction with unique challenges presented by cost function landscapes of parametrized quantum circuits, strategies for effective training are becoming increasingly important. In order to ameliorate some of these challenges, we investigate a lay...

Manual fits to spectral times series of Type Ia supernovae have provided a method of reconstructing the explosion from a parametric model but due to lack of information about model uncertainties or parameter degeneracies direct comparison between theory and observation is difficult. We present a probabilistic reconstruction of the normal Type Ia su...

Supernova spectral time series contain a wealth of information about the progenitor and explosion process of these energetic events. The modeling of these data requires the exploration of very high dimensional posterior probabilities with expensive radiative transfer codes. Even modest parameterizations of supernovae contain more than 10 parameters...

Versatile movement representations allow robots to learn new tasks and rapidly adapt them to environmental changes, e.g. introduction of obstacles, placing additional robots in the workspace, modification of the joint range due to faults or range of motion constraints due to tool manipulation. Probabilistic movement primitives (ProMP) model robot m...

Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. T...

Current neuroethological experiments require sophisticated technologies to precisely quantify the behavior of animals. In many studies, solutions for video recording and subsequent tracking of animal behavior form a major bottleneck. Three-dimensional (3D) tracking systems have been available for a few years but are usually very expensive and rarel...

Supernova spectral time series contain a wealth of information about the progenitor and explosion process of these energetic events. The modeling of these data requires the exploration of very high dimensional posterior probabilities with expensive radiative transfer codes. Even modest parametrizations of supernovae contain more than ten parameters...

With the increased focus on quantum circuit learning for near-term applications on quantum devices, in conjunction with unique challenges presented by cost function landscapes of parametrized quantum circuits, strategies for effective training are becoming increasingly important. In order to ameliorate some of these challenges, we investigate a lay...

We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep generative approach which combines learned with engineered models. This principled treatment of uncertainty and probabilistic inference overcomes the shortcoming of current state-of-the-art solutions to rely on...

In the early 1990s, we thought neural networks would be commonplace soon and used everywhere. It took 30 years longer, but that point has now been reached. Neural networks can learn any function from complex and high-dimensional data sets, image or pattern recognition is solved, and probability theory serves as a sound mathematical basis for it all...

Learning to control robots without requiring models has been a long-term goal, promising diverse and novel applications. Yet, reinforcement learning has only achieved limited impact on real-time robot control due to its high demand of real-world interactions. In this work, by leveraging a learnt probabilistic model of drone dynamics, we achieve hum...

Measuring the similarity between data points often requires domain knowledge. This can in parts be compensated by relying on unsupervised methods such as latent-variable models, where similarity/distance is estimated in a more compact latent space. Prevalent is the use of the Euclidean metric, which has the drawback of ignoring information about si...

Learning a model of dynamics from high-dimensional images can be a core ingredient for success in many applications across different domains, especially in sequential decision making. However, currently prevailing methods based on latent-variable models are limited to working with low resolution images only. In this work, we show that some of the i...

We address tracking and prediction of multiple moving objects in visual data streams as inference and sampling in a disentangled latent state-space model. By encoding objects separately and including explicit position information in the latent state space, we perform tracking via amortized variational Bayesian inference of the respective latent pos...

The study of dexterous manipulation has provided important insights into human sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here, we describe a method using the deformation and color distribution of the fingernail and its surrounding sk...

We address the problem of one-to-many mappings in supervised learning, where a single instance has many different solutions of possibly equal cost. The framework of conditional variational autoencoders describes a class of methods to tackle such structured-prediction tasks by means of latent variables. We propose to incentivise informative latent r...

Recently, it has been shown that many functions on sets can be represented by sum decompositions. These decompositons easily lend themselves to neural approximations, extending the applicability of neural nets to set-valued inputs—Deep Set learning. This work investigates a core component of Deep Set architecture: aggregation functions. We suggest...

The length of the geodesic between two data points along a Riemannian manifold, induced by a deep generative model, yields a principled measure of similarity. Current approaches are limited to low-dimensional latent spaces, due to the computational complexity of solving a non-convex optimisation problem. We propose finding shortest paths in a finit...

The study of dexterous manipulation has provided important insights in humans sensorimotor control as well as inspiration for manipulation strategies in robotic hands. Previous work focused on experimental environment with restrictions. Here we describe a method using the deformation and color distribution of the fingernail and its surrounding skin...

We address the problem of one-to-many mappings in supervised learning, where a single instance has many different solutions of possibly equal cost. The framework of conditional variational autoencoders describes a class of methods to tackle such structured-prediction tasks by means of latent variables. We propose to incentivise informative latent r...

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learni...

System identification of complex and nonlinear systems is a central problem for model predictive control and model-based reinforcement learning. Despite their complexity, such systems can often be approximated well by a set of linear dynamical systems if broken into appropriate subsequences. This mechanism not only helps us find good approximations...

We propose to learn a hierarchical prior in the context of variational autoencoders. Our aim is to avoid over-regularisation resulting from a simplistic prior like a standard normal distribution. To incentivise an informative latent representation of the data by learning a rich hierarchical prior, we formulate the objective function as the Lagrangi...

Recently, it has been shown that many functions on sets can be represented by sum decompositions. These decompositons easily lend themselves to neural approximations, extending the applicability of neural nets to set-valued inputs---Deep Set learning. This work investigates a core component of Deep Set architecture: aggregation functions. We sugges...

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while fully stochastic networks can b...

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learni...

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while fully stochastic networks can b...

The length of the geodesic between two data points along the Riemannian manifold, induced by a deep generative model, yields a principled measure of similarity. Applications have so far been limited to low-dimensional latent spaces, as the method is computationally demanding: it constitutes to solving a non-convex optimisation problem. Our approach...

Learning from multiple sources of information is an important problem in machine-learning research. The key challenges are learning representations and formulating inference methods that take into account the complementarity and redundancy of various information sources. In this paper we formulate a variational autoencoder based multi-source learni...

Robots can rapidly acquire new skills from demonstrations. However, during generalisation of skills or transitioning across fundamentally different skills, it is unclear whether the robot has the necessary knowledge to perform the task. Failing to detect missing information often leads to abrupt movements or to collisions with the environment. Acti...

We propose to learn a stochastic recurrent model to solve the problem of simultaneous localisation and mapping (SLAM). Our model is a deep variational Bayes filter augmented with a latent global variable---similar to an external memory component---representing the spatially structured environment. Reasoning about the pose of an agent and the map of...

Functional Magnetic Resonance Imaging (fMRI) is a powerful tool for neuroscience. It allows the visualization of active areas in the human brain. Combining this method with haptic interfaces allows one to conduct human motor control studies with an opportunity for standardized experimental conditions. However, only a small number of specialized MR-...

We propose stochastic, non-parametric activation functions that are fully learnable and individual to each neuron. Complexity and the risk of overfitting are controlled by placing a Gaussian process prior over these functions. The result is the Gaussian process neuron, a probabilistic unit that can be used as the basic building block for probabilis...

We propose stochastic, non-parametric activation functions that are fully learnable and individual to each neuron. Complexity and the risk of overfitting are controlled by placing a Gaussian process prior over these functions. The result is the Gaussian process neuron, a probabilistic unit that can be used as the basic building block for probabilis...

Neural samplers such as variational autoencoders (VAEs) or generative adversarial networks (GANs) approximate distributions by transforming samples from a simple random source---the latent space---to samples from a more complex distribution represented by a dataset. While the manifold hypothesis implies that the density induced by a dataset contain...

Kjolstad et. al. proposed a tensor algebra compiler. It takes expressions that define a tensor element-wise, such as $f_{ij}(a,b,c,d) = \exp\left[-\sum_{k=0}^4 \left((a_{ik}+b_{jk})^2\, c_{ii} + d_{i+k}^3 \right) \right]$, and generates the corresponding compute kernel code. For machine learning, especially deep learning, it is often necessary to c...

We introduce a methodology for efficiently computing a lower bound to empowerment, allowing it to be used as an unsupervised cost function for policy learning in real-time control. Empowerment, being the channel capacity between actions and states, maximises the influence of an agent on its near future. It has been shown to be a good model of biolo...

Industry 4.0 factories become more and more complex with increased maintenance costs. Reducing costs by cyber-physical (CP) controllers should ensure the commercialization of the CPS for smart factory project results. We implement multi-adaptive CP controllers in the following domains: industrial robot arms, car manufacturing, steel industry, and a...

The study of human motor control using functional Magnetic Resonance Imaging gives rises to many challenges. One of them is the design of haptic interfaces that are compatible with the magnetic field. To achieve this, the existing haptic interfaces employ parallel kinematics. However, they are limited to three Degrees of Freedom (DoF). When trying...

ELife digest Each nerve cell in the brain of a mammal communicates with about 1,000 other nerve cells in a complex network. Nerve cells ‘talk’ to each other via structures called synapses that connect the nerve cells together. The number of synapses in the brain is enormous – for example, a human brain contains about one quadrillion synapses. One t...

DOI: http://dx.doi.org/10.7554/eLife.26414.015

Source data for plot in panel 4b.DOI: http://dx.doi.org/10.7554/eLife.26414.017

(Table ) 1 Overview of methods for automated synapse detection. Res. Fac: Image voxel volume of SBEM data used in this study relative to the voxel volume in the reported studies. Note that most studies employ data of substantially higher image resolution. DOI: http://dx.doi.org/10.7554/eLife.26414.031

Synapse gallery. Document describing the criteria by which synapses in 3D SBEM data were detected by human expert annotators. These criteria are exemplified for synapses from the test set of the SynEM classifier. DOI: http://dx.doi.org/10.7554/eLife.26414.034

Source data for plots in panels 1b, 1c.DOI: http://dx.doi.org/10.7554/eLife.26414.004

DOI: http://dx.doi.org/10.7554/eLife.26414.022

DOI: http://dx.doi.org/10.7554/eLife.26414.024

Source data for plots in panels 6b, 6c, 6d.DOI: http://dx.doi.org/10.7554/eLife.26414.026

Source data for plots in panels 7a, 7b.DOI: http://dx.doi.org/10.7554/eLife.26414.030

(Table) 2 Number of synapses between connected neurons obtained from published studies of paired recordings of excitatory neurons in rodent cortex. These distributions were used in Figure 5 for prediction of connectome precision and recall. DOI: http://dx.doi.org/10.7554/eLife.26414.032

(Table) 3 Number of synapses between connected neurons obtained from published studies of paired recordings of inhibitory neurons in rodent cortex. DOI: http://dx.doi.org/10.7554/eLife.26414.033

Source data for plot in panel 2d.DOI: http://dx.doi.org/10.7554/eLife.26414.006