Gabriel De Oliveira Ramos

Gabriel De Oliveira Ramos
Universidade do Vale do Rio dos Sinos | UNISINOS · Graduate Program in Applied Computing

PhD

About

60
Publications
6,697
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
271
Citations
Introduction
I am an Assistant Professor of the Graduate Program in Applied Computing at Unisinos, Brazil. Before that, I was a Postdoctoral Research Fellow at the AI-Lab of VUB, Belgium. I received my PhD (2018, with highest honours) and MSc (2013) degrees in Computer Science from INF-UFRGS, Brazil. My research focuses on machine learning, reinforcement learning, multiagent systems, game theory, and search, with applications in smart cities, traffic, smart grids, industry 4.0, and healthcare.

Publications

Publications (60)
Conference Paper
Full-text available
Glaucoma é a principal causa mundial de perda irreversível de visão. Afim de viabilizar a implantação de uma ferramenta de diagnóstico de glaucoma para a clínica médica, um trabalho base foi selecionado e otimizado. Ao unificar duas redes de segmentação reduzimos o tempo de processamento em 24,24%, e adicionando uma segunda rede de classificação di...
Article
Full-text available
Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear co...
Article
The COVID-19 pandemic has rapidly spread around the world. The rapid transmission of the virus is a threat that hinders the ability to contain the disease propagation. The pandemic forced widespread conversion of in-person to virtual care delivery through telemedicine. Given this gap, this article aims at providing a literature review of machine le...
Article
Full-text available
In the field of agricultural research, Machine Learning (ML) has been used to increase agricultural productivity and minimize its environmental impact, proving to be an essential technique to support decision making. Accurate harvest time prediction is a challenge for fruit production in a sustainable manner, which could eventually reduce food wast...
Preprint
Full-text available
Nowadays, data has become an invaluable asset to entities and companies, and keeping it secure represents a major challenge. Data centers are responsible for storing data provided by software applications. Nevertheless, the number of vulnerabilities has been increasing every day. Managing such vulnerabilities is essential for building a reliable an...
Article
Industry 4.0 (I4.0) provides connectivity, data volume, new devices, miniaturization, inventory reduction, personalization, and controlled production. In this new era, customization and data availability are essential to generate information that allows decision-making. The possibility of predicting the need for maintenance in the future and using...
Article
Cardiovascular diseases represent the number one cause of death in the world, including the most common disorders in the heart’s health, namely coronary artery disease (CAD). CAD is mainly caused by fat accumulated in the arteries’ internal walls, creating an atherosclerotic plaque that impacts the blood flow functional behavior. Anatomical plaque...
Chapter
Industry 4.0 introduces several changes to the original approach of industrial automation. Internet of Things (IoT) and cyber-physical system (CPS) technologies play huge roles in this context introducing cognitive automation and consequently implementing the concept of intelligent production, leading to smart products and services. This approach l...
Article
Through the Internet of Things (IoT), the generation of data, Cyber-Physical Systems (CPS) has shown a steady increase. The search for approaches in order to take advantage of generated data is a recurring theme on several managers’ agenda. To this end, data mining techniques, combined with asset health management, contribute to Industry 4.0 releva...
Chapter
Early identification of patients with COVID-19 is essential to enable adequate treatment and to reduce the burden on the health system. The gold standard for COVID-19 detection is the use of RT-PCR tests. However, due to the high demand for tests, these can take days or even weeks in some regions of Brazil. Thus, an alternative for detecting COVID-...
Chapter
The extensive exploration of the Low Earth Orbit (LEO) has created a dangerous spacial environment, where space debris has threatened the feasibility of future operations. In this sense, Active Debris Removal (ADR) missions are required to clean up the space, deorbiting the debris with a spacecraft. ADR mission planning has been investigated in the...
Preprint
Full-text available
The recent paper `"Reward is Enough" by Silver, Singh, Precup and Sutton posits that the concept of reward maximisation is sufficient to underpin all intelligence, both natural and artificial. We contest the underlying assumption of Silver et al. that such reward can be scalar-valued. In this paper we explain why scalar rewards are insufficient to...
Article
Full-text available
Background The second wave of the COVID-19 pandemic was more aggressive in Brazil compared to other countries around the globe. Considering the Brazilian peculiarities, we analyze the in-hospital mortality concerning socio-epidemiological characteristics of patients and the health system of all states during the first and second waves of COVID-19....
Preprint
Full-text available
Early identification of patients with COVID-19 is essential to enable adequate treatment and to reduce the burden on the health system. The gold standard for COVID-19 detection is the use of RT-PCR tests. However, due to the high demand for tests, these can take days or even weeks in some regions of Brazil. Thus, an alternative for detecting COVID-...
Article
Monitoring the degradation of machines to anticipate potential failures represents a significant challenge. In Industry 4.0, this task is critical when the costs associated with maintenance and stoppages on the productive processes are high. Nowadays, many preventive maintenance techniques employ supervised or unsupervised machine learning algorith...
Conference Paper
Atherosclerosis represents the restriction of blood flow in the heart muscle and is one of the main causes of death in the world. The assessment of atherosclerosis is challenging and is currently evaluated by the Fractional Flow Reserve (FFR) and the Quantitative Flow Ratio (QFR). Both exams are based on angiography, which is the gold standard for g...
Preprint
Full-text available
Background: The second wave of the COVID-19 pandemic was more aggressive in Brazil compared to other countries around the globe. Considering the Brazilian peculiarities, we analyze the in-hospital mortality concerning socio-epidemiological characteristics of patients and the health system of all states during the first and second waves of COVID-19....
Conference Paper
Full-text available
Early identification of patients with COVID-19 is essential to enable adequate treatment and to reduce the burden on the health system. The gold standard for COVID-19 detection is the use of RT-PCR tests. However, due to the high demand for tests, these can take days or even weeks in some regions of Brazil. Thus, an alternative for the detection of...
Article
The gold standard for breast cancer diagnosis, treatment, and management is the histological analysis of a suspected section. Histopathology consists in analyzing the characteristics of the lesions using tissue sections stained with hematoxylin and eosin. However, pathologists are currently subjected to high workloads, mainly due to the fundamental...
Article
Full-text available
The World Health Organization (WHO) reported that more than 1 billion people live with some form of disability. Moreover, the number of elderly is increasing in recent years. According to the United Nations (UN), in 2050, there will be 2.1 billion people above 60 years of age worldwide. Many of these people live alone in their homes or clinics and...
Conference Paper
Full-text available
Este artigo descreve a criação de um template no formato openEHR para interoperabilidade semântica de dados de saúde – um padrão aberto para informações clínicas. As informações coletadas através da ficha de notificação SRAG (síndrome respiratória aguda grave) foram utilizadas como conceitos base para composição do template. O template SARS event n...
Preprint
Full-text available
Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination....
Preprint
Full-text available
The interpretation of seismic data is vital for characterizing sediments' shape in areas of geological study. In seismic interpretation, deep learning becomes useful for reducing the dependence on handcrafted facies segmentation geometry and the time required to study geological areas. This work presents a Deep Neural Network for Facies Segmentatio...
Conference Paper
A cardiac ischemia is a restriction of the blood flow in the heart muscle caused by narrowed heart arteries. The most common narrowing process is called atherosclerosis. The strategies to evaluate its significance are the Fractional Flow Reserve (FFR) and the Quantitative Flow Ratio (QFR) which evaluate the local impact of the atherosclerosis. This...
Conference Paper
Early diagnosis usually results in a better prognosis in cancer cases. Currently, histopathological analysis is the gold standard for the diagnosis, staging, and definition of the treatment of breast neoplasms. However, the technique has some restrictions that hinder the process of investigation by the pathologist, such as different staining protoc...
Article
The business world is continually changing. Dynamic environments, full of uncertainties, complexities, and ambiguities, demand faster and more confident decisions. To compete in this environment, Industry 4.0 emerges as an essential alternative. In this context, the reliability of manufacturing is an essential aspect for companies to make successfu...
Conference Paper
Full-text available
Multiagent reinforcement learning has shown its potential for tackling real world problems, like traffic. We consider the toll-based route choice problem, where self-interested agents repeatedly commute attempting to minimise their travel costs. In this paper, we introduce Generalised Toll-based Q-learning (GTQ-learning), a mul-tiagent reinforcemen...
Article
The problem of traffic congestion incurs numerous social and economical repercussions and has thus become a central issue in every major city in the world. For this work we look at the transportation domain from a multiagent system perspective, where every driver can be seen as an autonomous decision-making agent. We explore how learning approaches...
Article
Hospitals play an important role towards ensuring proper health treatment to human beings. One of the major challenges faced in this context refers to the increasingly overcrowded patients queues, which contribute to a potential deterioration of patients health conditions. One of the reasons of such an inefficiency is a poor allocation of health pr...
Preprint
Full-text available
The idea of experience sharing between cooperative agents naturally emerges from our understanding of how humans learn. Our evolution as a species is tightly linked to the ability to exchange learned knowledge with one another. It follows that experience sharing (ES) between autonomous and independent agents could become the key to accelerate learn...
Conference Paper
Traffic congestions present a major challenge in large cities. Consid- ering the distributed, self-interested nature oftraffic we tackle congestions using multiagent reinforcement learning (MARL). In this thesis, we advance the state- of-the-art by delivering the first MARL convergence guarantees in congestion- like problems. We introduce an algori...
Conference Paper
Full-text available
Multiagent reinforcement learning has shown its potential for tackling real world problems, like traffic. We consider the toll-based route choice problem, in which self-interested drivers (agents) need to repeatedly choose (commuting) routes that minimise the travel costs between their origins to their destinations. One of the major challenges here...
Conference Paper
Full-text available
We consider the route choice problem using multiagent reinforcement learning. In this problem, agents individually learn which routes minimise their expected travel costs. Such a selfish behaviour results in the so-called User Equilibrium (UE), which is inefficient from the system's perspective. In order to reduce the impact of selfishness , we dev...
Article
Full-text available
In the route choice problem, self-interested drivers aim at choosing routes that minimise travel costs between their origins and destinations. We model this problem as a multiagent reinforcement learning scenario. Here, since agents must adapt to each others’ decisions, the minimisation goal is seen as a moving target. Regret is a well-known perfor...
Thesis
Full-text available
Multiagent reinforcement learning (MARL) is a challenging task, where self-interested agents concurrently learn a policy that maximise their utilities. Learning here is difficult because agents must adapt to each other, which makes their objective a moving target. As a side effect, no convergence guarantees exist for the general MARL setting. This...
Chapter
In this chapter, we introduce the use of self-organised coalitions in smart grid scenarios for finding a coalition structure that maximises the systems’ utility. The complexity of such a task is exponential with the number of agents, and optimal coalition formation has been considered impractical. Several heuristic alternatives have been proposed i...
Conference Paper
Full-text available
Reinforcement learning (RL) is a challenging task, especially in highly competitive multiagent scenarios. We consider the route choice problem, in which self-interested drivers aim at choosing routes that minimise their travel times. Employing RL here is challenging because agents must adapt to each others' decisions. In this paper, we investigate...
Conference Paper
Full-text available
The use of reinforcement learning (RL) in multiagent scenarios is challenging. I consider the route choice problem, where drivers must choose routes that minimise their travel times. Here, selfish RL-agents must adapt to each others' decisions. In this work, I show how the agents can learn (with performance guarantees) by minimising the regret asso...
Conference Paper
Full-text available
Evaluating multiagent reinforcement learning (MARL) approaches in real world problems, such as traffic, is a challenging task. In general, such approaches cannot be deployed before extensive validation. Hence, simulating the impact of these approaches represents an essential step towards its deployment. Existing MARL tools make this process easier...
Conference Paper
Full-text available
The notion of regret has been extensively employed to measure the performance of reinforcement learning agents. The regret of an agent measures how much worse it performs following its current policy in comparison to following the best possible policy. As such, measuring regret requires complete knowledge of the environment. However, such an assump...
Conference Paper
The traffic assignment problem (TAP) plays a key role in the context of efficient urban mobility. The TAP can be approached from various perspectives. One of the fundamental models to solve the TAP is the so-called User Equilibrium (UE), which assumes that drivers behave rationally aiming at minimising their travel costs. However, this is a complex...
Article
Full-text available
The Vehicle-To-Grid (V2G) concept is a key feature towards the integration of electric vehicles into smart grids. Through V2G sessions, plug-in electric vehicles (PEVs) can sell their surplus energy to the grid. However, profiting from V2G sessions is not trivial for singletons. Thereby, the formation of coalitions among PEVs has been proposed to t...
Conference Paper
Full-text available
Solving the traffic assignment problem (TAP) is an important step towards an efficient usage of the traffic infrastructure. A fundamental assignment model is the so-called User Equilibrium (UE), which may turn into a complex optimisation problem. In this paper, we present the use of the GRASP metaheuristic to approximate the UE of the TAP. A path r...
Conference Paper
Full-text available
Finding an optimal coalition structure is a hard problem. In order to simplify this process, it is possible to explore some characteristics of the agents organization. In this paper we propose an algorithm that deals with a particular family of games in characteristic function, but is able to search in a much smaller space by considering organizati...
Chapter
Full-text available
Urban mobility is a major challenge in modern societies. Increasing the infrastructure’s physical capacity has proven to be unsustainable from a socio-economical perspective. Intelligent transportation systems (ITS) emerge in this context, aiming to make a more efficient use of existing road networks by means of new technologies. In this paper we a...
Conference Paper
Full-text available
In transportation systems, drivers usually choose their routes based on their own knowledge about the network. Such a knowledge is obtained from drivers' previous trips. When drivers are faced with jams they may change their routes to take a faster path. But this re-routing may not be a good choice because other drivers can proceed in the same way....
Article
Full-text available
The use of electric vehicles (EVs) and vehicle-to-grid (V2G) technologies have been advocated as an efficient way to reduce the intermittency of renewable energy sources in smart grids. However, operating on V2G sessions in a cost-effective way is not a trivial task for EVs. The formation of coalitions among EVs has been proposed to tackle this pro...
Conference Paper
Full-text available
This paper presents a tool developed in Java to perform traffic assignment and route learning in road networks. For traffic assignment, the tool provides the following methods: all-or-nothing assignment, incremental assignment, and successive averages. As for route learning, Q-learning and learning automata methods are available. In addition to the...
Chapter
Full-text available
In this paper we analyse the trade-off between privacy-preservation methods and the quality of data mining applications, within the specific context of the smart grid. The use of smart meters to automate data collection is set to solve the problem of electricity theft, which is a serious concern in developing nations. Nevertheless, the unlimited us...
Conference Paper
Full-text available
In the last years, the need for using multiple energy sources made the concept of smart grids emerge. A smart grid is a fully automated electricity network, which monitors and controls all its elements being able to supply energy in an efficient and reliable way. Within this context, the use of electric vehicles (EVs) and Vehicle-To-Grid (V2G) tech...
Conference Paper
Full-text available
Smart grids have received great attention in recent years. Among many technologies that are used in smart grids, the concept of vehicle-to-grid (V2G) arises, which allows the use of electric vehicles' batteries to provide energy back to the grid when needed. An interesting research approach is the formation of coalitions among electric vehicles to...

Projects

Project (1)
Project
To evaluate the potential of using self-organized coalitions to manage multiple complex system scenarios in areas like: optimization, prediction, smart grids, evolutionary game theory, reputation models, neural networks, cellular automata, etc.