Jose Pinto

• Master of Engineering
• Senior Researcher at OceanScan - Marine Systems & Technology

This paper describes the open-source software toolchain developed by the Underwater Systems and Technology Laboratory (LSTS) for supporting networked heterogeneous air and ocean vehicle systems. The toolchain supports the deployment of air and ocean vehicles interacting over limited acoustic and wireless networks combined with disruption-tolerant n...

This paper describes a command and control infrastructure that provides mixed-initiative user interfaces for the coordination and control of networked vehicle systems composed by heterogeneous robots, satellite trackers, manned ships and communication gateways. The presented infrastructure was tested in a field experiment for tracking and sampling...

Study of ocean processes is important to understanding climatic variability especially on the productive upper water-column. Ocean currents regulate the climate, it captures CO\(_2\) from the atmosphere and oxygen is generated by its plankton communities, all of which are part of the global environmental cycle which are being impacted by anthropoge...

The study of dynamic features of the ocean, in which complex physical, chemical, and biological interactions evolve on multiple time scales, poses significant sampling challenges because the required spatial and temporal resolutions are not possible by ship or satellite studies alone. Satellite remote sensing captures only surface effects while exp...

With the growing interest in underwater exploration and monitoring, autonomous underwater vehicles have become essential. The recent interest in onboard deep learning (DL) has advanced real-time environmental interaction capabilities relying on efficient and accurate vision-based DL models. However, the predominant use of sonar in underwater enviro...

With the growing interest in underwater exploration and monitoring, Autonomous Underwater Vehicles (AUVs) have become essential. The recent interest in onboard Deep Learning (DL) has advanced real-time environmental interaction capabilities relying on efficient and accurate vision-based DL models. However, the predominant use of sonar in underwater...

Digital Twins and Marine Spatial Planning play an important role by enabling what-if scenarios and promoting the efficient use of maritime space. Informed decisions require sound knowledge about the current state of the sea, including nearshore areas, where data acquisition requires innovative approaches. A drifting sensor, originally named WAVY, w...

The development of increasingly autonomous underwater vehicles has long been a research focus in underwater robotics. Recent advances in deep learning have shown promising results, offering the potential for fully autonomous behavior in underwater vehicles. However, its implementation requires improvements to the current vehicles. This paper propos...

Over the past decades, underwater robotics has enjoyed growing popularity and relevance. While performing a mission, one crucial task for Autonomous Underwater Vehicles (AUVs) is bottom tracking, which should keep a constant distance from the seabed. Since static obstacles like walls, rocks, or shipwrecks can lie on the sea bottom, bottom tracking...

Accurate identification of an uncertain underwater environment is one of the challenges of underwater robotics. Autonomous Underwater Vehicle (AUV) needs to understand its environment accurately to achieve autonomous tasks. The method proposed in this paper is a real-time automatic target recognition based on Side Scan Sonar images to detect and lo...

div>The REP(MUS)19 is an exercise that takes place annually since 2010 in the south of continental Portugal. LSTS (Underwater Systems and Technology Laboratory) has co-organized these events, since the beginning, together with the Portuguese Navy and, in more recent years, with NATO-CMRE. NATO’s MUS (Maritime Unmanned Systems Initiative) initiativ...

div>The REP(MUS)19 is an exercise that takes place annually since 2010 in the south of continental Portugal. LSTS (Underwater Systems and Technology Laboratory) has co-organized these events, since the beginning, together with the Portuguese Navy and, in more recent years, with NATO-CMRE. NATO’s MUS (Maritime Unmanned Systems Initiative) initiativ...

Internal waves (IWs) in the ocean span across a wide range of time and spatial scales and are now acknowledged as important sources of turbulence and mixing, with the largest observations having 200 m in amplitude and vertical velocities close to 0.5 m s−1. Their origin is mostly tidal, but an increasing number of non-tidal generation mechanisms ha...

Study of ocean processes is important to understanding climatic variability especially on the productive upper water-column. Ocean currents regulate the climate, it captures CO2 from the atmosphere and oxygen is generated by its plankton communities, all of which are part of the global environmental cycle which are being impacted by anthropogenic c...

The ocean is an essential support system for most if not all Earth ecosystems but, in spite of that, it still remains for the most part unexplored. It covers more than 70 per cent of Earth's surface and has a big impact, for example, in the global weather patterns by distributing the absorbed sun's heat around the world via its ever-moving ocean cu...

Previous front-tracking (front-following) algorithms for marine robots have been developed for-and applied to-coastal fronts such as wind-driven coastal upwelling fronts or river plume fronts. During the R/V Falkor cruise FK180528 in May-June 2018 we developed a new, universal front-following algorithm based on a novel concept of tracking front's a...

Ocean fronts are dynamic three-dimensional (3D) structures that can be effectively and efficiently sampled with robots, particularly with autonomous underwater vehicles (AUVs). Search, mapping and tracking of large-scale open ocean fronts require a mix of different types of robots, each with certain advantages and constraints. The limited endurance...

Our research concerns the coordination and control of robotic vehicles for upper water-column oceanographic observations. In such an environment, operating multiple vehicles to observe dynamic oceanographic phenomena, such as ocean processes and marine life, from fronts to cetaceans, has required that we design, implement and operate software, meth...

In the last few years, the technological development of underwater robotics has been a key factor to improve oceanographic research. Autonomous underwater vehicles (AUVs) are especially well suited for coastal studies because they can provide near-synoptic spatial observations. Their high spatial and temporal resolutions combined with exceptional m...

This paper presents a report from a cruise onboard the R/V Falkor oceanographic vessel from the Schmidt Ocean Institute. The goal of this cruise was to demonstrate a novel approach to observe the ocean with multiple underwater, surface, and aerial vehicles, as well with the R/V Falkor also used as the base and control center for all assets. We desc...

We describe the 2018 edition of the Recognised Environmental Picture exercise (REP18). Every year, the Portuguese Navy hosts an international exercise where challenging and novel operational scenarios are tested in a controlled environment. This is done to achieve a readiness level that allows the use of those developments in future real-world scen...

Marine Protected Areas classification can help in the protection and monitoring of important ecosystems that might be sheltered by their habitats. New instrumentation techniques are being applied to study this type of environments, performing various scientific surveys in the water. In this paper we introduce a practical approach for habitat mappin...

This paper presents recent software developments which allow the coordination of multiple AUVs over high-latency and low-bandwidth satellite communications. These developments span over multiple components of the LSTS toolchain: protocols, onboard plan execution and adaptation, satellite communications and temporal planning. The developments have b...

Habitat mapping is an important task to manage ecosystems. This task becomes most challenging when it comes to marine habitats as it is hard to get good images in underwater conditions and to precisely locate them. In this paper we present a novel technique for performing habitat mapping automating all phases, from data collection to classification...

In this paper, we present a control strategy for trajectory tracking and path following of generic paths for underactuated marine vehicles. Our work is inspired and motivated by previous works on ground vehicles. In particular, we extend the definition of the hand position point, introduced for ground vehicles, to autonomous surface vehicles and au...

We present Dolphin, an extensible programming language for autonomous vehicle networks. A Dolphin program expresses an orchestrated execution of tasks defined compositionally for multiple vehicles. Building upon the base case of elementary one-vehicle tasks, the built-in operators include support for composing tasks in several forms, for instance a...

Propagation of river plumes along coast strongly depends on several physical processes, which determine the plume fate over shelf. Traditional methods of in situ is often very difficult and expensive due to great spatial and temporal variability of its dominant drivers. Here we report a method and preliminary results of using autonomous underwater...

Unmanned vehicles are usually run in unknown environments where additional information can help improving the vehicle's path and safety. Neptus is a command and control framework targeting the operation of unmanned vehicles and leveraging simulation in order to do so. Neptus uses a set of simulation tools allowing a better understanding of the site...

Design of simple and efficient routing protocols for underwater acoustic sensor networks is a difficult subject because they have to cope with the challenging and variable characteristics of the water medium. With the participation of autonomous underwater vehicles in underwater networks, mobility has become a more important matter to deal with in...

Our research concerns the mixed-initiative coordination of air and underwater vehicles interacting over inter-operated radio and underwater communication networks for novel oceanographic field studies. In such an environment, operating multiple vehicles to observe dynamic oceanographic events such as fronts, plumes, blooms and cetaceans has require...

Our research concerns the mixed-initiative coordination of air and underwater vehicles interacting over inter-operated radio and underwater communication networks for novel oceanographic field studies. In such an environment, operating multiple vehicles to observe dynamic oceanographic events such as fronts, plumes, blooms and cetaceans has require...

We often take Web connectivity for granted. This can be the case in our houses where we rely on more and more devices that connect to the Internet and exchange information with remote servers or other nearby devices around using cheap Wi-Fi radios. However, the world remains for the most part disconnected. There are still uncountable remote areas w...

Over the last decade, ocean sunfish movements have been monitored worldwide using various satellite tracking methods. This study reports the near-real time monitoring of fine-scale (< 10 m) behaviour of sunfish. The study was conducted in southern Portugal in May 2014 and involved satellite tags and underwater and surface robotic vehicles to measur...

Northward (left) and Eastward (right) daily component provided by the Wave Glider (line) and model output (dots) averaged over two days and registered at 18 m depth. (TIF)

a) Example of a drifter (red line, with dates) and particles (blue lines) tracks, b) Atlantic section of the drifter track before entering the Mediterranean Sea (red line) and mean trajectory of the 200 particles released in the Lagrangian model (green line). (TIF)

Supplementary Material for integrated monitoring of Mola mola behaviour in space and time. (DOCX)

Example of a drifter track superimposed on a MODIS SST map composite from May 25th to June 1th (top) and June 2th to 9th June (bottom), both 2014. (TIF)

Archive of Drifter data. (ZIP)

Archive of AUV tracks data. (ZIP)

This paper deals with the problem of autonomous exploration of unknown areas using teams of Autonomous X Vehicles (AXVs)—with X standing for Aerial, Underwater or Sea-surface—where the AXVs have to autonomously navigate themselves so as to construct an accurate map of the unknown area. Such a problem can be transformed into a dynamic optimization p...

Supervision and control of Autonomous underwater vehicles (AUVs) has traditionally been focused on an operator determining a priori the sequence of waypoints of a single vehicle for a mission. As AUVs become more ubiquitous as a scientific tool, we envision the need for controlling multiple vehicles which would impose less cognitive burden on the o...

The use of unmanned vehicle networks for diverse applications is becoming widespread. It is generally hard to program unmanned vehicle networks as a "whole", however. The coordination of multiple vehicles requires careful planning through intricate human intervention, and a high degree of informality is implied in what concerns the specification of...

The coordinated use of multiple unmanned vehicles over a network can be employed for numerous real-world applications. However, multi-vehicle operations are often deployed through a patchwork of separate components that informally "glue" together during operation, as they are hard to program as a "whole". With this aim, we developed the Networked V...

The coordinated use of multiple unmanned vehicles over a net-work can be employed for numerous real-world applications. How-ever, multi-vehicle operations are often deployed through a patch-work of separate components that informally "glue" together dur-ing operation, as they are hard to program as a "whole". With this aim, we developed the Network...

The use of unmanned vehicle networks for diverse applications is becoming widespread. It is generally hard to program unmanned vehicle networks as a " whole " , however. The coordination of multiple vehicles requires careful planning through intricate human intervention, and a high degree of informality is implied in what concerns the specification...

Within the project NOPTILUS, a fully functional system/methodology had been developed that allows the cooperative, fully-autonomous navigation of teams of AUVs when deployed in Static or Dynamic Underwater Map Construction (SDUMC) or Dynamic Underwater Phenomena Tracking (DUPT) missions. The key ingredient of this fully functional system/methodolog...

Obtaining synoptic observations of dynamic ocean phenomena such as fronts, eddies, oxygen minimum zones and blooms has been challenging primarily due to the large spatial scales involved. Traditional methods of observation with manned ships are expensive and unless the vessel can survey at high-speed, is unrealistic. Autonomous underwater vehicles...

This paper describes scientific and engineering efforts towards remote sensing, tagging and continuous tracking of sunfish specimens (Mola mola, world largest bony fish) in the open sea. This ongoing experiment is done in the context of a collaboration between MBARI and Porto University and bridges biology, fisheries research, autonomy and control...

This paper presents the LAUV (Light Autonomous Underwater Vehicle), with special emphasis on the most recent developments and discusses how the LAUV enables networked operations. The LAUV is a lightweight, one-man-portable vehicle specially designed to be a highly operational and effective surveying tool for oceanographic, hydrographic, security an...

An experiment with different AUVs was carried out in the Mar Menor Coastal Lagoon from October 31 to November 5 in order to measure and assess the influence of the water from the Mar Menor on the adjacent area of the Mediterranean. This was carried out as a result of the meeting held between several institutions from the Iberian Peninsula and EEUU...

This paper describes the layered control architecture and its software implementation developed and used at the Underwater Systems and Technology Laboratory. The architecture is implemented as a toolchain which consists on three main entities: DUNE onboard software, Neptus command and control software and a common IMC message-based communication pr...

This paper describes the layered control architecture and its software implementation developed and used at the Underwater Systems and Technology Laboratory. The architecture is implemented as a toolchain which consists on three main entities: DUNE onboard software, Neptus command and control software and a common IMC message-based communication pr...

We describe the process of improving the onboard autonomy of LAUV-Seacon AUVs with an in-situ planning agent. Deliberative planning is achieved by extending the existing control architecture with T-REX and a domain model description which is specific to the Seacon AUVs and typical mission scenarios. We discuss the required architectural changes as...

The global market offers an extensive catalog of commercial Autonomous Underwater Vehicles with different shapes, sizes and capabilities. This paper addresses the innovative concept which led to the development of a new commercial AUV product to compete in such a dynamic market. The LAUV was planned to be a lightweight vehicle that can be easily la...

We study various issues related to robust real world underwater mission planning, to obtain a global view of the involved challenges. We explore various maneuvers that are required for underwater missions, and the issues involved in real world cooperative mission planning with several vehicles in the presence of sporadic communication disruptions,...

In this paper, we formalize the problem of human interaction with bio-inspired robot teams (HuBIRT). The formalism applies to a large class of bio-inspired team dynamics and uses simple algebraic graph theory representations to distinguish between interagent influence, environmental influence, and operator influence. These representations lead to m...

In this paper we discuss a conceptual framework that supports operational scenarios with multiple UAVs and operators. These UAVs possess different levels of autonomy while the operators have variable skill sets. The scenarios themselves encompass different missions, with different phases (requiring different levels of attention from the operator) a...

Although much work has been done on designing autonomy and user interfaces for managing small teams of independent robots, much less is known about managing large-scale bio-inspired robot (BIRT) teams. In this paper, we explore human interaction with BIRT teams in an information foraging task. We summarize results from two small experiments that us...

Operation of networks of heterogeneous vehicles and sensors imposes many technical and operational challenges. Simultaneous control of multiple vehicle types requires abstraction any device-specific details and to keep human operators in the loop by providing them a good overall picture of the current system state. In this paper, we present the Nep...

In this paper we report our research on the development of an open web-based architecture for controlling the behavior of systems composed of static sensors, mobile sensors and unmanned vehicles. We document the first implementation of the architecture and describe its application to our vehicles and sensors. We discuss the validation of this imple...

This paper presents a command and control graphical user interfacefor teams of Unmanned Air Vehicles compliant with the NATO’s STANAG 4586standardization agreement for unmanned aerial vehicles (UAVs) interoperability.The framework is deployed with Neptus, a software nfrastructure for hetero-geneous teams of autonomous vehicles. We describe our impl...

A team of networked UAVs are deployed in an unknown region to search and destroy targets. To successfully destroy a target, a coalition of UAVs with sufficient cumulative resources needs to be assigned. Forming coalitions under networks with dynamic topology is difficult and the type of coalition formation strategy adopted affects the mission perfo...

A viability algorithm is developed to compute the constrained minimum time function for general dynamical systems. The algorithm is instantiated for a specific dynamics (Dubin's vehicle forced by a flow field) in order to numerically solve the minimum time problem. With the specific dynamics considered, the framework of hybrid systems enables us to...

This paper presents recent developments from the Underwater Systems and Technology Laboratory in order to integrate data received simultaneously from static and free-moving wireless sensors as well as unmanned vehicles, allowing real-time access of data using ubiquitous technologies. Live data is communicated using pre-established XML data formats...

This paper presents the Inter-Module Communication (IMC) protocol, a message-oriented protocol designed and implemented in the Underwater Systems and Technology Laboratory (LSTS) to build interconnected systems of vehicles, sensors and human operators that are able to pursue common goals cooperatively by exchanging real-time information about the e...

In this paper, we present a cooperative control design for multiple underwater vehicles performing an ocean exploration mission. Underwater vehicles have limited communication range and low bandwidth for coordinated maneuvers and for human-in-loop missions. Autonomous surface vehicles (ASV) can be configured and tasked to act as mobile communicatio...

A viability algorithm is developed to compute the constrained minimum time function for general dynamical systems. The algorithm is instantiated for a specific dynamics(Dubin’s vehicle forced by a flow field) in order to numerically solve the minimum time problem. With the specific dynamics considered, the framework of hybrid systems enables us to solv...

The design and development of the swordfish autonomous surface vehicle (ASV) system is discussed. Swordfish is an ocean capable 4.5 m long catamaran designed for network centric operations (with ocean and air going vehicles and human operators). In the basic configuration, Swordfish is both a survey vehicle and a communications node with gateways f...

Underwater acoustic networks can be quite effective to establish communication links between autonomous underwater vehicles (AUVs) and other vehicles or control units, enabling complex vehicle applications and control scenarios. A communications and control framework to support the use of underwater acoustic networks and sample application scenario...

This video shows a brief overview over Neptus, a command and control infrastructure for heterogeneous teams of autonomous vehicles. Having different types of vehicles at our laboratory and from our partners, there was an increasing need to create a common infrastructure to all these systems. Additionally, a tool to support the entire mission life c...

This paper describes the design and development of the Swordfish Autonomous Surface Vehicle (ASV) system. The work focuses the sensors, actuators, communications and C4I of an unmanned vehicle for marine operations. SWORDFISH is an autonomous surface vehicle used as the central communications link between air, undersea, and terrestrial robotic vehi...