Conference PaperPDF Available

Autonomous Ship Navigation Under Deep Learning and the Challenges in COLREGs

Authors:

Abstract and Figures

A general framework to support the navigation side of autonomous ships is discussed in this study. That consists of various maritime technologies to achieve the required level of ocean autonomy. Decision-making processes in autonomous vessels will play an important role under such ocean autonomy, therefore the same technologies should consist of adequate system intelligence. Each onboard application in autonomous vessels may require localized decision-making modules, therefore that will introduce a distributed intelligence type strategy. Hence, future ships will be agent-based systems with distributed intelligence throughout vessels. The main core of this agent should consist of deep learning type technology that has presented promising results in other transportation systems, i.e. self-driving cars. Deep learning can capture helmsman behavior, therefore that type system intelligence can be used to navigate autonomous vessels. Furthermore, additional decision supporting layers should also be developed to facilitate deep learning type technology including situation awareness and collision avoidance. Ship collision avoidance is regulated by the Convention on the International Regulations for Preventing Collisions at Sea, 1972 (COLREGs) under open sea areas. Hence, a general overview of the COLREGs and its implementation challenges, i.e. regulatory failures and violations, under autonomous ships are also discussed with the possible solutions as the main contribution of this study. Furthermore, additional considerations, i.e. performance standards with the applicable limits of liability, terms, expectations and conditions, towards evaluating ship behavior as an agent-based system on collision avoidance situations are also illustrated in this study.
Content may be subject to copyright.
1 Copyright © 2018 by ASME
Proceedings of the 37th International Conference on Ocean, Offshore and Arctic Engineering
OMAE2018
June 17-22, 2018, Madrid, Spain
OMAE2018-77672
AUTONOMOUS SHIP NAVIGATION UNDER DEEP LEARNING AND THE
CHALLENGES IN COLREGS
Lokukaluge P. Perera
UiT The Arctic University of Norway, Tromso, Norway
Prasad.Perera@uit.no
ABSTRACT
A general framework to support the navigation side of
autonomous ships is discussed in this study. That consists of
various maritime technologies to achieve the required level of
ocean autonomy. Decision-making processes in autonomous
vessels will play an important role under such ocean autonomy,
therefore the same technologies should consist of adequate
system intelligence. Each onboard application in autonomous
vessels may require localized decision-making modules,
therefore that will introduce a distributed intelligence type
strategy. Hence, future ships will be agent-based systems with
distributed intelligence throughout vessels. The main core of this
agent should consist of deep learning type technology that has
presented promising results in other transportation systems, i.e.
self-driving cars. Deep learning can capture helmsman behavior,
therefore that type system intelligence can be used to navigate
autonomous vessels. Furthermore, an additional decision support
layer should also be developed to facilitate deep learning type
technology including situation awareness and collision
avoidance. Ship collision avoidance is regulated by the
Convention on the International Regulations for Preventing
Collisions at Sea, 1972 (COLREGs) under open sea areas.
Hence, a general overview of the COLREGs and its
implementation challenges, i.e. regulatory failures and
violations, under autonomous ships are also discussed with the
possible solutions as the main contribution of this study.
Furthermore, additional considerations, i.e. performance
standards with the applicable limits of liability, terms,
expectations and conditions, towards evaluating ship behavior as
an agent-based system on collision avoidance situations are also
illustrated in this study.
INTRODUCTION
Ocean Autonomy
Autonomy can be expressed as a situation, where one can have
freedom from external controls or influences. However, the same
should have adequate intelligence to make appropriate decisions
in relation to possible internal and external variations. The same
concept has been adopted towards mechanical and electrical
systems by introducing machine intelligence-based decision-
making facilities to operate themselves. Such self-operating
systems, i.e. autonomous systems, should consist of advanced
decisions making facilities, therefore various technologies to
support the same should also be developed by the respective
industries. Recent technological advancements in autonomous
systems, i.e. self-driving cars, robots, etc. [1], can be such
examples consisting of adequate decision-making facilities.
Furthermore, these autonomous systems are also supported by
internal and external IoT (i.e. internet of things), big data and
communication infrastructure to overcome the respective
challenges. However, the success of decision-making features in
these autonomous systems is yet to be evaluated,
comprehensively by the respective authorities [2].
Autonomous systems will be an important role in the
future transportation systems, even though there are many
challenges [3]. Since self-driving vehicles have already been
introduced on public roads, similar systems will also be
introduced by air and maritime transport systems. This study
focusses on the navigation side of maritime transport systems
with autonomous and remote-controlled facilities, i.e.
Autonomous and remote-controlled ships. That will also be a
part of ocean autonomy as a transportation system. However,
ocean autonomy into the offshore sector yet to be developed.
Modern vessels are facilitated with onboard and
onshore IoT to support various digitalization applications in the
shipping industry. Industrial digitalization converts conventional
paper-based information handling approaches into data driven
applications. A comprehensive overview of vessels and ship
systems can be captured under industrial digitalization and that
information should use towards decision-making facilities of
future ships. Hence, industrial digitalization should support
ocean autonomy under the maritime transportation, where
human inference can be minimal by transforming big data
collected by IoT into intelligent decisions [4].
2 Copyright © 2018 by ASME
Fig 1: A general framework for autonomous ship navigation
There are several millstones that should be achieved by
the shipping industry to make autonomous ship navigation a
reality. "Remote Controlled Ship" can be an important milestone
in this route [5] and this same state, i.e. remote controlled
navigation facilities, can always be a part of autonomous ship
navigation. In general, each voyage can have both autonomous
and remote controlled navigation sector that should be
segmented by considering the operational requirements of future
vessels. The required maritime infrastructure to support both
remote controlled and autonomous ship operations should be
developed and that can also be another important milestone in
the same route [6]. The success in ship intelligence, i.e. artificial
intelligence to navigate and operate vessels and ship systems,
will make the most important milestone in this journey.
Appropriate decision support facilities to support ship
intelligence should also be developed and that will be another
important milestone. Finally, adequate tools and techniques to
evaluate vessel behavior under ship intelligence with decision
support facilities should also be developed, where the acceptable
risk and limits of liability, terms, expectations and conditions
under autonomous ship navigation should be defined.
Agent Based Systems
This study proposes the autonomous ship as an agent based
system. It is believed that this approach can also satisfy the
respective milestones, appropriately. An agent can be defined
as a system located in a specific environment, therefore it
interacts with the environment by intelligent decisions and
actions to satisfy its design objectives. However, other similar
and/or different agents can also be located within the same
environment, where these agents should interact. Therefore,
various cooperative and non-cooperative interactions among
these agents are expected in this same environment. However,
each agent may have its own design objectives, therefore
adequate intelligent to fully or partially satisfy the same should
be facilitated. If individual design objectives cannot be achieved
(i.e. unsatisfactory) in this environment, adequate compromising
strategies to satisfy appropriate group objectives should be
considered. Such situations can be categorized as a cooperative
multi-agent learning approach with machine learning approaches
(i.e. reinforcement learning) [7]. Therefore, adequate system
3 Copyright © 2018 by ASME
intelligence in each agent should be facilitated to handle rather
complex interactions among agents in the environment.
It is expected that autonomous ships will be agent based
systems, therefore various cooperative and non-cooperative
interactions among vessels in open sea areas and traffic lanes are
expected. In general, such intelligent agent should have the
following basic properties [7]:
Autonomy: Each agent should operate by its own
actions and/or internal states without the direct
inference of humans or others.
Social-ability: Each agent should interact with other
agents (i.e. including humans) by appropriate agent-
communication language.
Reactivity: Each agent should not only interact with the
environment but also respond to a timely fashion for the
respective environmental changes and challenges.
Pro-activeness: Each agent should not only interact
with the environment but also take appropriate
initiatives to exhibit goal-oriented behavior to satisfy its
design objectives.
One should note that these properties should also be a
part of future autonomous vessels. Therefore, the interactions
among vessels and ocean environmental conditions can be
facilitated by agent based systems. However,, vessels and ship
systems should have adequate ship intelligence and decision
support facilities to support these agent functionalities and that
can overcome the respective challenges in autonomous ship
navigation.
AUTONOMOUS VESSELS AND SHIP SYSTEMS
Future Vessels and Onboard Systems
It is expected that the shapes and types of future vessels will be
changed due to remote-controlled and autonomous operational
conditions. Remote-controlled vessels that will open the path
towards autonomous vessels will create the next generation
maritime transport systems. Human presence will be limited in
such vessels, where ship systems that supports humans onboard,
i.e. accommodation systems, air-conditioning, etc., can be
eliminated. Hence, the cost of installing human support systems
can be reduced, considerably and their maintenance can be
eliminated. Since remote-controlled and autonomous vessels
may not require to have adequate comfort for humans, the shapes
of such vessels can be designed in a way that reduce the
respective ship resistance, further.
However, additional vessel stability issues may rise
under such ship designs and that should also be addressed.
Though on the other hand, new ship systems, i.e. modern
decision support facilities, to support remote-controlled and
autonomous operations should be installed in these vessels.
Furthermore, the respective maintenance processes for these new
systems should be initiated. The overall reliability of these
vessels and ship systems should considerably be higher to cope
with harsh environmental conditions, therefore adequate
condition monitoring and condition based maintenance
approaches onboard as well as onshore should be available.
Future vessels will be facilitated with various onboard and
onshore IoT to monitor health conditions of individual systems
in real-time, i.e. possibly at component levels. Such network
connectivity will create big data sets and the respective system
health conditions, i.e. system information, can be extracted from
the same data sets. Hence, the respective system and component
failures can often be identified at an early stage due to their
health information. A considerable amount of recent research and
development activities are focused on conditions monitoring
(CM) and condition based maintenance (CBM) applications in
the shipping industry. There are onshore condition monitoring
centers, i.e. intelligent asset management centers, are introduced
to support future vessels by various industrial players [9].
System Intelligence
The navigation side of Autonomous vessels to achieve the
required level of ocean autonomy is discussed in this section. It
is expected that autonomous vessels will be trained by humans,
i.e. to navigate like an experienced helmsman. Similar training
case studies have successfully been implemented by other
transport systems, i.e. autonomous car, drones [10], therefore the
same approaches can be adopted by the maritime transport
systems. Furthermore, adequate system intelligence should be
developed within these vessels to absorb such knowledge from
the navigators during their training processes. Such intelligence
can be transferred from one vessel to another and that same can
further be improved during future ship operations. System
intelligence in autonomous vessels may consist of a complex
systems of system supported by additional decision supporting
facilities. A general framework to support the navigation side of
autonomous ships with the required technologies (i.e. navigation
and automation systems) is presented in Figure 1.
Initially, ship navigators will train these vessels to
achieve the respective system intelligence levels, i.e. to operate
both navigation and automation systems. The training process
can be done by onboard human presence and onshore remote-
controlled centers by introducing various navigation decisions
(i.e. human inference). That will be converted into navigation
actions. The navigation actions that required to achieve the
required ship route vs actual ship route can be considered as the
first level inputs to the training process (i.e. navigation training)
of ship intelligence (see Figure 1). Such navigation actions will
be implemented on the ship automation system, i.e.
propeller/thruster and rudder control systems, of the vessel to
achieve the required ship behavior i.e. ship speed, heading and
course.
One should note that these propeller/thruster and rudder
control system configurations can vary from one vessel to
another. Therefore, the navigator actions on propeller (i.e.
propeller pitch and speed) and rudder (i.e. rudder angle) control
systems can vary from one vessel to another. Furthermore, vessel
seakeeping and maneuvering behavior can be influenced by
vessel structures, ship systems and external environmental
conditions other than navigator’s actions. Both vessel behavior
4 Copyright © 2018 by ASME
and external environmental conditions should be monitored by
onboard and onshore IoT to provide required navigation
information. Each voyage consists of a required (i.e. expected)
ship route, therefore appropriate navigation actions should be
taken by the helmsman to achieve the same with respect to the
actual ship route.
Ship navigation systems consist of various IoT
including ECDIS (Electronic chart display and information
system), Radar and APRA (Automatic radar plotting aid),
Conning and additional systems and sensors. A combination of
such systems is classified as an integrated bridge system (IBS)
[11]. IBSs can collect and visualize the most important vessel
navigation and automation information including vessel
seakeeping and maneuvering behavior in the actual ship route
(i.e. vessel position, speed, course, heading and draft). This
information is collected as big data sets that should further be
analyzed to extract the respective seakeeping and maneuvering
behavior of the vessel. The accumulation of seakeeping and
maneuvering behavior and external environmental conditions
(i.e. navigation information) collected by IoT along with the
navigator’s actions are considered as the second level inputs to
the training process of ship intelligence. These two inputs, i.e.
navigation actions and information, complete the typical training
cycle, i.e. Training Process in Figure 1, of an autonomous vessel.
However, there are additional system layers that should support
system intelligence to succeed autonomous ship navigation.
These system layers are further illustrated in Figure 1 as:
Decision support, information sources and supporting services
and authorities.
Deep Learning in Shipping
A considerable section of ship intelligence will consist of a deep
learning based framework, i.e. artificial neural network. The
same framework will create the respective agent behavior within
autonomous vessels. Similar frameworks have been
implemented by other transport systems, i.e. autonomous
navigation systems of drive-less cars, and that have achieved
promising results in terms of navigating with the required safety
levels [12]. In general, deep leaning based frameworks
transform a self-driving vessel problem into a data classification
problem. e.g. convolutional neural networks (CNN, or ConvNet)
are a class of deep learning framework [13] that can solve
complex image classification problems and that have also been
used for self-driving vehicles. The same classification approach
can provide an elegant mechanism to capture helmsman
behavior, i.e. agent behavior in ship navigation. Initially, such
deep learning frameworks should be the observers to manual or
remote controlled vessels that are operated by human navigators.
That step is previously categorized as a training process (see
Figure 1) and the main objective of this phase is to train the
respective neural networks to capture ship behavior with respect
to navigator’s actions. Therefore, adequate features of vessel
seakeeping and maneuvering behavior can be accommodated
into these neural networks.
When such neural networks are adequately trained, that
technology can be used to navigate the first level autonomous
vessels. In addition, the same deep learning based frameworks
can be distributed (i.e. shared knowledge) among several vessels
and that can also be further trained during their operations. These
networks are often trained by image based information and
navigator actions rather than system parameters. A similar
approach can also be adopted towards the training phase of
autonomous vessels. These networks can transfer from one
vessel to another as mentioned before, however additional
training periods may require to capture highly accurate ship
behavior in some situations. If these vessels are standardized
during their ship design phase, then additional training
requirements can be eliminated. After a successful navigation
training period, ship intelligence can navigate such vessel as an
experienced helmsman, i.e. execution process in Figure 1.
Several technological challenges in controlling such
vessels, specially under rough weather conditions, should also be
expected. Conventional vessels are often categorized as under-
actuated systems, i.e. rudder and propeller control systems may
not able to control vessels completely during some sea going
situations. This ship controllability issue, i.e. under-actuation,
can complicate the training process of autonomous ships. One
should note that drones or under water vehicles navigate in a
single environmental media (i.e. land, air or water). Ships are
navigating between two environmental media (i.e. air and water)
and the interactions between both media influence on vessel
seakeeping and maneuvering behavior. That can further
complicate the training process of ship intelligence in under-
actuated vessels. The inertia in heavy vessels makes ship
controllability an extremely difficult challenge especially under
rough weather conditions.
The rudder and propeller control systems, i.e. only
available control units for vessel actuation, may fail to control
vessels under rough sea going conditions. When ships are
navigating under moderate or high speeds (i.e. over 3-4 knots),
the capabilities of thrusters are negligible. The control solutions
developed for autonomous surface vehicles (ASVs) and
autonomous underwater vehicles (AUVs) are not acceptable for
large vessels due to the same reasons that are mentioned
previously. Similarly, the control solutions developed for
autonomous land vehicles, i.e. driver-less car, are not acceptable
due to the same reasons, i.e. land vehicles are light weight
transportation units compared to ships and have a better
controllability over the roads. Hence, the controllability of under
actuated vessels under various navigation conditions should be
investigated in the near future. It is also expected that deep
learning frameworks supported by the decision support layer
(see Figure 1) may overcome some challenges in ship
controllability.
Decision Support Facilities
The decision support layer with various onboard and onshore IoT
supports ship intelligence. Each ship route may divide into
several voyage segments during voyage planning, i.e. possible
autonomous and remote-controlled navigation segments. Some
voyage segments of ship navigation may execute as remote-
controlled routes due to the respective safety and security
5 Copyright © 2018 by ASME
reasons. Global and local digital maps including navigation and
emission control rules and regulations should support the same
voyage planning phase. Furthermore, additional decision support
facilities such as weather routing and pilotage can also be a part
of voyage planning. Even though global maps are already
included under ECDISs, local maps (i.e. harbor areas and
confined waters) with the respective navigation rules and
regulations can be supplied by local maritime and port
authorities during the ship operation phase. Such local
information improves the safety of autonomous ship navigation,
therefore the pilotage type activities, i.e. humans with local
knowledge employed onboard ships to guide vessels, can be
eliminated. In addition, the respective maritime authorities can
further enforce energy efficiency and emission control rules and
regulations [14] on these vessels by distributing the respective
information. The energy efficiency and emission control rules
and regulations are enforced, extensively on the designated
emission control areas (ECAs). Hence, local digital maps can
provide of those information, accurately to enforce the respective
energy efficiency and emission control rules and regulations in
future vessels.
Local digital maps can be integrated with global maps
to support autonomous ship navigation under SLAM
(Simultaneous localization and mapping) type applications [15].
The SLAM type applications support intelligent agents, i.e.
autonomous vessels, to locate themselves within global and local
maps by considering the information collected from onboard and
onshore IoT. The agents can also learn about the environment in
some situations by executing possible actions. Therefore,
additional sensors, i.e. Lidar and Laser, should also be available
in autonomous vessels to support SLAM type applications.
Additional information resources such as VTI (vessel traffic
information), TSS (Traffic separation Schemes), AIS (Automatic
identification system) and LRIT (Long-Range Identification and
Tracking) can also support the same SLAM type applications in
future vessels. Some information sources are provided by vessel
traffic management and information systems (i.e. VTMIS).
Weather routing and safe ship handling should also
support ship intelligence under the decision support layer to
improve the safety and efficiency of ship navigation (see Figure
1). The required global and local weather information can be
obtained from weather centers by autonomous vessels. Weather
routing facilitates autonomous vessels by providing the
recommended ship routes prior to and during each voyage under
various navigational constraints and global weather forecast
[14]. Safe ship handling facilitates autonomous vessels by
providing recommended ship position, orientation and speed
conditions on the recommended route under similar navigational
constrains and local weather conditions [16]. In general, safe
ship handling applications have often been used for unexpected
rough weather conditions in ship routes. Therefore, both weather
routing and safe ship handling should support future vessels
under global and local weather forecast and that can also be an
important part of the voyage planning phase. Furthermore,
weather routing and safe ship handling can support each other
during the operation phase to achieve the required energy
efficient and safe ship navigational levels of autonomous vessels.
Conventional vessels consist of stability calculation
systems to estimate the respective ship loading conditions. Since
cargo loading and unloading activities at ports, vessels stability
should be calculated at the begging of each voyage. It is
expected that autonomous vessels should have similar decision
support facilities with additional IoT to verify the respective
loading calculations, accurately. Since future cargo loading and
unloading conditions in posts will also be automated, that
information can be shared with these decisions supporting layer.
Situation awareness and collision avoidance among
stationary and moving objects will play an important role in
autonomous ship navigation. The stationary objects, i.e. land
masks, shipwrecks, etc., are marked in global and local digital
maps and unexpected ones should be detected by onboard IoT.
The avoidance of both objects may relate to a path planning type
problem, especially under harbor or confined water navigation.
Situation awareness and collision avoidance facilities should
consist of adequate intelligence to detect and identify stationary
and moving objects. Such stationary and moving object detection
and classification has been done under deep learning frameworks
with successful results [17], therefore similar approach can be
adopted towards autonomous ship navigation. A general
overview in situation awareness and collision avoidance of
autonomous vessels in relation to the respective rules and
regulations is discussed in the following sections.
Situation Awareness and Collision Avoidance
Future vessels should be facilitated with appropriate collision
avoidance technologies. However, the behaviors of such
technologies should also be evaluated to guarantee the respective
safety levels of ship navigation [18, 19]. This section illustrates
the respective challenges in evaluating situation awareness and
collision avoidance technologies in autonomous vessels. In
general, such evaluation procedure for autonomous vessels
should consist of the following basic units:
Legal frameworks and their regulatory failures
Autonomous and target vessels and their behavior
Testable systems to evaluate ship behavior
Legal Frameworks and Their Regulatory Failures
All sea going vessels should follow the law of the sea. The
International Maritime Organization (IMO) in 1972 by the
Convention on the International Regulations for Preventing
Collisions at Sea (COLREGs) has introduced a legal framework
to regulate ship encounter situations [20]. The respective studies
of ship collisions indicate that 75%–96% of maritime collisions
and causalities caused by some types of human errors and 56%
of major maritime collisions involved one or more violations of
the COLREGs rules and regulations [21]. Future ships should
also be regulated by the same rules and regulations during their
vessel encounter situations and that behavior should be evaluated
by testable systems under acceptable performance standards.
Three distinct ship encounter situations that involve the risk of
6 Copyright © 2018 by ASME
collisions can be considered: overtaking, head-on, and crossing.
In general, the decision space of ship encounter situations with
two vessels can be categorized into the following stages in open
sea:
When none of the vessels are at a collision risk range,
both vessels have the option to take appropriate actions
to avoid a collision situation.
When both vessels are at a collision risk range, the
“give-way vessel should take appropriate actions to
achieve safe passing distance in accordance with the
COLREGs rules and regulations, and the “stand on
vessel should maintain course and speed conditions.
When both vessels are at a critical collision risk range,
and the “give way vessel does not take appropriate
actions to achieve a safe passing distance in accordance
with the COLREGs rules and regulations, then the
“stand on” vessel has the option to take appropriate
actions to avoid the collision.
However, local navigation rules and regulations, traffic
lanes, offshore operations and special types of vessels
(i.e. fishing vessels) can override some of the same
decision space especially under overtaking and head-on
situations.
The COLREGs may have some regulatory failure situations
under the same decision space and that may lead towards
collision situations [22]. Such situations have been reported in
the previous studies [23], while applying the COLREGs rules
and regulations into if-then-else type computer codes under
Fuzzy Logic. Those situations are further illustrated in this
section to illustrate COLREGs regulatory failure situations.
Figure 2 represents a situation, where the target vessel is in
a head-on situation from the port, slightly with the own vessel.
That can create a regulatory failure situation within the
COLREGs rules and regulations, i.e. the directions these vessels
should pass each other. The respective vessel positions, O(k) &
Pi(k), course-speed vectors, Vo(k) & Vi(k) and relative
navigational trajectory of the target vessel are also presented in
the figure. Figure 3 represents a situation, where the target vessel
relative navigational trajectory varies from head-on to crossing
situations with the own vessel. Therefore, such situations can
also create a regulatory failure situation within the COLREGs
rules and regulations, i.e. which rules in head-on or crossing
situations should apply. Similarly, the respective vessel
positions, i.e. O(k) & Pi(k), course-speed vectors, i.e. Vo(k) &
Vi(k) and relative navigational trajectory of the target vessel are
also presented in the same figure. Figure 4 represents a situation,
where multiple target vessels are approaching the own vessel
from different directions. Therefore, such situations can also
create a regulatory failure situation within the COLREGs rules
and regulations, i.e. which vessel avoidance should have the
priority.
One should note that these are selected regulatory failure
situations of the COLREGs rules and regulations that are
observed by previous studies. Therefore, additional regulatory
Fig 2: Ship encounter situation 1
Fig 3: Ship encounter situation 2
Fig 4: Ship encounter situation 3
failure situations are yet to be discovered in the future, when
these rules and regulations are implemented under the decision
support layer (see Figure 1). Adequate measures to overcome
such regulatory failure situations within the COLREGs legal
framework should be considered and that will eventually
7 Copyright © 2018 by ASME
improve the required safety levels of autonomous ship
navigation.
Autonomous and Target Vessel Behavior
Vessels may not honor rules and regulations in some
navigation situations and that may create high collision risk
situations. One should note that such ship encounters are clear
regulatory violations within the COLREGs legal framework and
cannot be categorized as regulatory failure situations. That may
result in ship collisions and close encounter situations in open
sea and the COLREGs legal framework may not provide clear
guidance to overcome for such situations. In general, ship
navigators use their experiences to avoid such situations and that
may also lead to “crash stoppingtype maneuvers of the “stand
on vessel due to a lack of distance for speed reduction. One
should note that the “stand onvessel represents the ship that has
the priority for navigating in an encounter situation.
The characteristics of the “stand onvessel with respect
to its “stopping distance and “turning circle should be
considered for developing appropriate collision avoidance
strategies for future vessels in such situations. Since the expert
knowledge has been used for such ship encounter situations, it is
expected that the same knowledge can be absorbed by the deep
learning framework of ship intelligence. These high collision
risk situations can be intentional or unintentional, however the
respective situation awareness and collision avoidance facilities
under ship intelligence should take appropriate actions to find
appropriate navigational solutions. On the other hand, adequate
tools and techniques to predict the intensions of vessels should
also be considered to support ship intelligence.
High Collision Risk Ship Encounters
The responses (i.e. vessel behavior) of autonomous vessels, i.e.
ship intelligence, in such high collision risk encounters should
also be evaluated and the acceptability of their navigational
actions should be investigated. That can be done by proper
testable systems supported by appropriate performance
standards to evaluate ship intelligence. These performance
standards are likely to be defined by the respective maritime
authorities or classification societies and evaluates the behavior
for autonomous vessels. However, the evaluation process of
autonomous vessels under critical collision situations should be
carefully formulated, since collision avoidance approaches also
depend on vessel maneuverability characteristics. The
maneuverability characteristics may be captured by ship
intelligence under deep learning type frameworks. Therefore,
ship intelligence may provide initial performance standards to
evaluate the first generation autonomous vessels.
Various ship navigation situations with high collision
risks should be recreated to evaluate vessel behavior, i.e. ship
intelligence, under the testable platforms. The results, i.e. vessel
behavior under ship collision or near collision situations, should
be communicated towards system developers in autonomous
vessels to improve their ship intelligence and decision support
facilities. Furthermore, testable systems should suggest the
required autonomous ship behavior, i.e. to reduce the collision
risk, with the appropriate reasoning about compliance
requirements of the legal frameworks, i.e. the COLREGs and
local navigational rules and regulations. Any unusual vessel
behavior or collision avoidance failures, i.e. regulatory failure
situations, may lead to introduce additional modifications into
the legal frameworks. One should note that collision worthiness
and survivability can also be important features for transport
systems. However, autonomous vessels may focus on collision
avoidance features for ship encounter situations rather than
collision worthiness and collision survivability, extensively.
Testable System for Vessel Behavior
Fig 5: Testable system for evaluating autonomous vessel behavior
A testable system with required components to evaluate collision
avoidance behavior of autonomous vessels is presented in Figure
5. That is influenced by two or more legal frameworks i.e.
COLREGs and other local navigation rules and regulations. This
system evaluates autonomous ship behavior, i.e. as an agent
based system, under various encounter situations with target
vessels. Therefore, several navigational ship routes with varying
weather conditions should be introduced in the same to create
realistic ship navigation conditions. Autonomous vessel
behavior can further be detailed into navigation and automation
system levels by monitoring the respective systems. Therefore,
vessel and ship system behavior should be evaluated by the
testable system with pre-defined performance standards. That
provides the specific performance expectations that should be
satisfied by autonomous vessels for various ship encounter
situations.
Appropriate course and/or speed changes, at any cost,
must be taken by the vessels to avoid collision situations, as
highlighted in the COLREGs. Hence, that unique feature can
introduce the minimum expectation level for the performance
standards of the testable systems. However, the ultimate
expectation of the performance standards may relate to selected
policy options and/or legal frameworks. Furthermore, that will
also be influenced by the applicable limits of liability, terms,
expectations and conditions that are introduced by the insurance
8 Copyright © 2018 by ASME
industry. However, existing performance standards [24, 25] for
ship navigation can also be adopted by the same testable systems.
The initial performance standards for situation
awareness and collision avoidance should be developed in
relation to a two vessel encounter situation and that can be
expanded towards multiple vessel encounter situations. Hence,
the performance standards should illustrate the following
navigation features to evaluate vessel encounter situations:
Autonomous and target vessel domains
The collision risk between the vessels
The distance and time to a possible collision/close
encounter situation
Autonomous and target vessels course-speed vectors
The bearing vector between two vessels
Autonomous vessel decisions and the time to execute
the same decisions into actions
Autonomous and target vessel predicted and actual
behavior.
The satisfactory level (i.e. under the performance
standards) of the decisions and actions that have taken
by the vessels.
If the autonomous vessel fails to satisfy the expected
performance standards, then the recommendations on navigation
and automation system improvements should be forwarded to
the respective manufacture. Therefore, adequate modifications
not only in a system level but also a ship intelligence and
decision support level should be considered (see Figure 5). It is
also recommended to implement the following testing levels in
the proposed platform to evaluate situation awareness and
collision avoidance behavior in autonomous vessels under ship
encounter situations:
Testing level 1: Both autonomous and target vessels are
simulated by software programs.
Testing level 2: Target vessels are simulated by a
software program and the autonomous ship is
represented by a full-scale/model-scale vessel. The
autonomous vessel is navigating in approved waters.
Testing level 3: Both autonomous and target vessels are
represented by full-scale/model-scale vessels. That are
navigating in open sea.
It is expected that testable systems will initially evaluate
autonomous vessels under the first testing level. Both
autonomous and target vessels can be considered as agent based
systems in this situation, where appropriate mathematical
models to simulate realistic ship behavior are required. In
addition, several weather conditions can also be introduced in
this situation to observe the variations in vessel behavior. The
respective sea trial data that are collected from ocean going
vessels can be used to develop such mathematical models for
these vessels [26]. Such mathematical model of ship
maneuvering should consist of: ship course-speed vector,
heading vector and turning rates. That can also be estimated from
time-series data sets, i.e. AIS data, and such data sets can also
influence on the respective performance standards.
The lessons learned from the first testing level should
consider for the second testing level and that represents realistic
ocean-going conditions. The implementation of the third testing
level can be a challenge due to the difficulties in creating vessel
collision situations in open sea, intentionally (i.e. due to
maneuverability difficulties in vessels). Therefore, the first and
second testing levels can be considered as possible situations that
can be implemented in the testable system. Such two vessel
encounter situations can be accumulated towards multi-vessel
encounter situations, as mentioned previously.
These evaluation procedures of ship behavior in the
testable systems should consider realistic ocean-going
conditions. That include all possible navigation scenarios
including various ship encounter situations, varying
environmental conditions, and random ship system & sensor
failures. However, the main objective in this testable system is to
evaluate vessel behavior of autonomous and target ships rather
than the reliability of hardware and software systems. Even
though system hardware and software failures can eventually
influence on ship intelligence and decision support layers of
future vessels, a considerable amount to tools and techniques to
regulate such failures have been developed by the respective
industries. In general, both hardware and software developments
are often guided by system development (i.e. the V model) [27]
and agile software development approaches [28] by various
manufactures. Therefore, future ship systems will have higher
reliability levels due to the maturity of the respective
technologies. Hence, adequate research focuses should be aimed
towards ship intelligence and decision support facilities of
autonomous vessels, rather than individual system and
component failures.
CONCLUSIONS
A general framework to support the navigation side of
autonomous vessels is discussed in this paper and that consists
of various technologies to achieve the required level of ocean
autonomy. Since decision-making processes in autonomous
vessels will play an important role under ocean autonomy, the
same technologies should consist of adequate system
intelligence. Each onboard application in autonomous ships may
require a localized decision-making process, where a distributed
intelligence type strategy should be considered. Hence, the ship
should be an agent-based system with distributed intelligence
throughout the vessel. The main core of such agent consists of
deep learning type frameworks, i.e. ship intelligence, to simulate
the helmsman actions in ship navigation. Furthermore, an
additional decision supporting layer should also available to
facilitate situation awareness and collision avoidance among
vessels. Hence, a considerable amount of research and
development work will be required to achieve required ship
intelligence within deep learning frameworks and machine
learning applications for decision support of autonomous
vessels.
9 Copyright © 2018 by ASME
The required technologies to implement and evaluate
ship intelligence in transport systems are still in a preliminary
stage. Therefore, a considerable amount of such knowledge yet
to be created. The same knowledge should be shared among
research communities to develop safer maritime transportation
systems. Autonomous system developers will play a crucial role
in developing and sharing the knowledge on ship intelligence
and decision support facilities and that may push machine
learning into a more regulated industry. However, the human
interactions and their outcome under artificial intelligence yet to
be investigated by the research community.
Collision avoidance among autonomous and target
vessels is focused in this study. The same ship intelligence with
decision support facilities in collision avoidance should also be
evaluated under various ship encounter situations. That can be
done by testable systems of situation awareness and collision
avoidance and the outcome should be compared with the
applicable limits of liability, terms, expectations and conditions
in ship navigation. That will be resulted in appropriate
performance standards to evaluate vessel behavior as an agent
based system in various ship encounter situations. It is also noted
that the respective navigation rules and regulations may have
regulatory failure and violation situations under ship intelligence
and decision support facilities, therefore adequate measures to
overcome such challenges should be considered.
REFERENCES
[1] Li, T. H. S., Lee, M. H., Lin, C. W., Liou, G. H. and Chen, W. C.,
(2016) "Design of Autonomous and Manual Driving System for
4WIS4WID Vehicle," in IEEE Access, vol. 4, pp. 2256-2271.
[2] Dokic, J., Müller, B., and Meyer, G., 2015, European Roadmap
Smart Systems for Automated Driving, European Technology
Platform on Smart Systems Integration (EPoSS).
[3] Rahman, A. A., Hamid, U. Z. A., Rahman, T. A. C., (2017),
"Emerging Technologies with Disruptive Effects: A Review"
PERINTIS eJournal. 7 (2).
[4] Perera, L. P., "Industrial IoT to Predictive Analytics: A Reverse
Engineering Approach from Shipping," In Proceedings of the 3rd
Norwegian Big Data Symposium (NOBIDS), Trondheim, Norway,
November, 2017.
[5] Perera, L. P., Ferrari, V., Santos, F. P., Hinostroza, M. A., and
Guedes Soares, C., (2015) "Experimental Evaluations on Ship
Autonomous Navigation & Collision Avoidance by Intelligent
Guidance," IEEE Journal of Oceanic Engineering, vol. 40, no. 2,
pp. 374-387.
[6] Rolls-Royce, (2016) “Remote and Autonomous ship: the next
step,AAWA position paper.
[7] Panait, L. and Luke, S., (2005), "Cooperative Multi-Agent
Learning: The State of the Art", Autonomous Agents and Multi-
Agent Systems, vol. 11, no. 3, pp. 387-434
[8] Wooldridge, M., and Jennings, N., (1995), “Intelligent agents:
Theory and practice,” The Knowledge Engineering Review, vol.
10, no. 2, pp. 115-152.
[9] Rolls-Royce, 2016, Ship intelligence transforming future marine
operations, Norway.
[10] Giusti, A., Guzzi, J., Ciresan, D.C., He, F., Rodriguez, J.P.,
Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Di Caro, G.,
Scaramuzza, D., and Gambardella, L.M., (2016), "A Machine
Learning Approach to Visual Perception of Forest Trails for Mobile
Robots, " IEEE Robotics and Automation Letters (RA-L), pages
661 – 667.
[11] Perera, L.P., and Guedes Soares, C., (2015), "Collision Risk
Detection and Quantification in Ship Navigation with Integrated
Bridge Systems," Journal of Ocean Engineering, vol. 109, pp. 344-
354.
[12] Liu, S., Tang, J., Zhang, Z., and Gaudiot, J. L., (2017), "Computer
Architectures for Autonomous Driving," in Computer, vol. 50, no.
8, pp. 18-25.
[13] Collobert, R., and Weston, J., (2008), "A Unified Architecture for
Natural Language Processing: Deep Neural Networks with
Multitask Learning". Proceedings of the 25th International
Conference on Machine Learning. ICML '08. New York, USA:
ACM: 160–167.
[14] Perera, L.P., and Guedes Soares, C., (2017), "Weather Routing and
Safe Ship Handling in the Future of Shipping," Journal of Ocean
Engineering, vol. 130, pp. 684-695.
[15] Bresson, G., Alsayed, Z., Yu, L., and Glaser, S., (2017),
"Simultaneous Localization and Mapping: A Survey of Current
Trends in Autonomous Driving," in IEEE Transactions on
Intelligent Vehicles, vol. 2, no. 3, pp. 194-220.
[16] Perera, L. P., Rodrigues, J. M., Pascoal, R. and C. Guedes Soares,
C., (2012),"Development of an onboard decision support system
for ship navigation under rough weather conditions,", Sustainable
Maritime Transportation and Exploitation of Sea Resources, E.
Rizzuto & C. Guedes Soares (Eds.), Taylor & Francis Group,
London, UK, pp. 837-844.
[17] Szegedy, C., Toshev, A., and Erhan, D., (2013). "Deep neural
networks for object detection". Advances in Neural Information
Processing Systems.
[18] The Munin Project, [online] Available: http://www.unmanned-
ship.org/munin/wp.
[19] Burmeister, H.-C., Bruhn, W., Rødseth, Ø. J., Porathe, T.,
“Autonomous Unmanned Merchant Vessel and its Contribution
towards the e-Navigation Implementation: The MUNIN
Perspective,International Journal of e-Navigation and Maritime
Economy, vol. 1, 2014, pp. 1-13.
[20] IMO, (1972) “Convention on the International Regulations for
Preventing Collisions at Sea (COLREGs),[Online]. Available:
http://www.imo.org/conventions
[21] Rothblum, A.M., Wheal, D., Withington, S., Shappell, S. A.,
Wiegmann, D. A., Boehm, W., and Chaderjian, M., (2002), “Key
to successful incident inquiry,in Proc. 2nd Int. Workshop Human
Factors Offshore Oper., pp. 1–6.
[22] Perera, L.P., Carvalho, J.P., and Guedes Soares, C., (2012),
"Intelligent ocean navigation & Fuzzy-Bayesian decision-action
formulation," IEEE Journal of Oceanic Engineering, vol 37, no 2,
pp. 204-219.
[23] Perera, L.P., Carvalho, J.P., and Guedes Soares, C., (2014),
"Solutions to the Failures and Limitations of Mamdani Fuzzy
Inference in Ship Navigation," IEEE Transactions on Vehicular
Technology, vol. 63, no. 4, pp. 1539-1554.
[24] IMO, (2002), ’Standards for Ship Manoeuvrability, Resolution
MSC.137 (76).
[25] IMO, (2002), ‘Explanatory Notes to the Standards for Ship
Manoeuvrability,’ MSC/Circ.1053.
[26] International standard ISO 15016 and ISO 19019, (2002), “Ship
and marine technology Guidelines for the assessment of speed
and power performance by analysis of speed trial data”, First
edition.
10 Copyright © 2018 by ASME
[27] Forsberg, K. and Mooz, H., (1991), "The Relationship of System
Engineering to the Project Cycle", in Proceedings of the First
Annual Symposium of National Council on System Engineering,
pp. 57–65.
[28] Abrahamson, P., Salo, O., Ronkainen, J., and Warsta, J., (2002).
Agile software development methods: Review and analysis
(Technical report). VTT. 478.
... In the present study, we used data from a spaceflight analog balancing task that reliably led to spatial disorientation and loss of control. The deep learning and AI communities have explored problems related to crash avoidance for conventional ground vehicles (Peng et al., 2019), autonomous vehicles (Perumal et al., 2021), unmanned aerial vehicles (Gandhi et al., 2017), ships (Perera, 2018), swarming systems (Lan et al., 2020), and aircraft collisions with other aircraft (Julian et al., 2019). However, no one to our knowledge has used deep learning to predict the occurrence of crashes in a novel analog condition where participants experience disorientation similar to what pilots and astronauts may experience. ...
Article
Full-text available
Were astronauts forced to land on the surface of Mars using manual control of their vehicle, they would not have familiar gravitational cues because Mars’ gravity is only 0.38 g. They could become susceptible to spatial disorientation, potentially causing mission ending crashes. In our earlier studies, we secured blindfolded participants into a Multi-Axis Rotation System (MARS) device that was programmed to behave like an inverted pendulum. Participants used a joystick to stabilize around the balance point. We created a spaceflight analog condition by having participants dynamically balance in the horizontal roll plane, where they did not tilt relative to the gravitational vertical and therefore could not use gravitational cues to determine their position. We found 90% of participants in our spaceflight analog condition reported spatial disorientation and all of them showed it in their data. There was a high rate of crashing into boundaries that were set at ± 60 ° from the balance point. Our goal was to see whether we could use deep learning to predict the occurrence of crashes before they happened. We used stacked gated recurrent units (GRU) to predict crash events 800 ms in advance with an AUC (area under the curve) value of 99%. When we prioritized reducing false negatives we found it resulted in more false positives. We found that false negatives occurred when participants made destabilizing joystick deflections that rapidly moved the MARS away from the balance point. These unpredictable destabilizing joystick deflections, which occurred in the duration of time after the input data, are likely a result of spatial disorientation. If our model could work in real time, we calculated that immediate human action would result in the prevention of 80.7% of crashes, however, if we accounted for human reaction times (∼400 ms), only 30.3% of crashes could be prevented, suggesting that one solution could be an AI taking temporary control of the spacecraft during these moments.
... The autonomous vessel navigation employs various models accounting for a range of obstacles, with varying degrees of dynamics (Geng et al., 2019), and adopting multiple techniques and methods, such as deep learning - (Perera, 2018), collision potentiality of a location (He et al., 2017), or virtual force field (VFF) method for track-keeping in case of a collision. Overall, AGN regulates ship routing to avoid collisions and preventing damages (Lee et al., 2004;Perera et al., 2011;Thieme et al., 2018). ...
Article
Full-text available
Maritime shipping, with a significant role in global trade, confronts various accidents leading to loss of lives, properties, and the environment. Shipping 4.0 technologies are scaling up to address this problem by employing real-time data-driven technologies, including cyber-physical systems, advanced tracking and tracing, intelligent systems, and big data analytics. Despite growing attention, there is a general lack of clarity on the level and direction of progress in this field. Accordingly, this study aims to identify critical shipping accident risks, analyze the role of relevant shipping 4.0 technologies in controlling these risks, and consolidate the findings into a conceptual guiding framework directing future developments. Accordingly, a systematic review is performed that reveals how shipping 4.0 approaches address critical accident risks and the gaps that still exist. Overall, we found that the collision is the most frequent accident referred to, while the most frequent technology to control the accidents is the Automatic Identification System. In contrast, we see an evident lack of cloud computing, internet-of-things, and big data analytics, which play crucial roles in current industry 4.0 developments.
... As a major advance in ML, the deep learning (DL) approach is becoming a powerful technique for autonomy. DL methodologies are applied in various fields in the maritime industry such as detecting anomalies, ship classification, collision avoidance, risk detection of cyberattacks, navigation in ports, etc. [60]. A diverse range of methods are available in the literature for USV and MASS autonomy and their applications in maritime navigation. ...
Article
Autonomy and unmanned systems have evolved significantly in recent decades, becoming a key routine component for various sectors and domains as an intrinsic sign of their improvement, the ocean not being an exception. This paper shows the transition from the research concept to the commercial product and related services for Unmanned Surface Vehicles (USV). Note that it has not always been easy in most cases due to the limitations of the technology, business, and policy framework. An overview of current trends in USV technology looking for a baseline to understand the sector where some experiences of the authors are shown in this work. The analysis presented shows a multidisciplinary approach to the field. USV's capabilities and applications today include a wide range of operations and services aimed at meeting the specific needs of the maritime sector. This important consideration for USV has yet to be fully addressed, but progress is being made.
... To comply with COLREGs rules, [10] presented the application of neuroevolutionary methods in vessel overtaking, head-on, and crossing scenarios using artificial potential field (APF). However, [33] argued that COLREGs regulation might fail in some instances when both of the ships decide to take the same decision, which may lead to ship collision. Therefore it is suggested that any two vessels; (1) should not be overtaking when there exists a crossing vessel, and (2) an overtaking can be done by a faster vessel when there is no crossing vessel in the area of proximity [23]. ...
Article
Full-text available
Safe, accurate, and predictable autonomous systems in marine vehicles are paramount. An understanding of an intelligent system fitted inside a ship is critical to ensure an autonomous ship is safe to be operated. Although the use of artificial intelligence in the design of the road-based vehicle has arrived at the self-driving level, there exists a significant gap within the research of autonomous ship to operate in restricted water (riverine and ports). Hence, this article shall discuss the relevant works of literature to set a preliminary guiding principle for the design of an autonomous ship. We present a simple illustrative framework as a starting point for ship designers to begin working in a simulated environment, which can be used as a foundation before the physical autonomous-ships are constructed and tested in a real-world situation. The framework consists of a virtual 3D environment and a surface vehicle with distance sensors, controlled by a neuroevolution-based autonomous piloting system. In this work, two scenarios will be presented: navigation in restricted waters, and obstacle avoidance capability of an autonomous ship. Results show that the resulting autonomous surface vehicle (ASV) is also capable of performing obstacle avoidance in the test track, albeit not being trained to do so in the training track. The work demonstrated in this paper is useful to the ship designers and can be extended for scenario-based planning for autonomous ship design.
... For certain operations, e.g. collision avoidance at sea [28], it can be argued that the possible number of situations that need to be tested for is too large to get sufficient significance levels from testing. This means that it may be impossible to prove that the system works sufficiently well in all situations [29]. ...
Preprint
Full-text available
The concept of autonomous mobile robots (AMR) has gained much popularity in recent years, particularly in commercial settings where the name industrial autonomous mobile robot (IAMR) is proposed. In addition to automatic guided vehicles and automated mining trucks, IAMR also includes autonomous merchant ships. AMR is an old concept which was first introduced in the 1980s. Although the concept of AMRs is old and broadly used, there is still no common definition of autonomy when mobile robots are concerned. This paper will review some of the most known definitions and develop a taxonomy for autonomy in mobile autonomous robots. This will be used to compare the different definitions of robotic autonomy. This paper will mainly look at industrial autonomous mobile robots, i.e. systems that are designed to operate with a clear commercial objective in mind and which are normally supported by a remote control centre. This means that the robot is not fully autonomous, but to varying degrees dependent on humans in some control and monitoring functions.
... MUNIN [35] also identifies several relevant hazards in MASS operations. Finally, much research focuses also on collision avoidance and guidance systems; see for example Perera et al. [37] who propose a ship collision situation avoidance algorithm based on fuzzy logic to support decision-making systems in autonomous vessels and Xu et al. [63] who use AIS data to propose a path generation system. ...
Article
Full-text available
Maritime Autonomous Surface Ships (MASS) are attracting increasing attention in the maritime industry. Despite the expected benefits in reducing human error and significantly increasing the overall safety level, the development of autonomous ships would undoubtedly introduce new risks. The overall goal of this work is to develop an approach to evaluate the risk level of major hazards associated with MASS. To that extent, a Failure Modes and Effects Analysis (FMEA) method is used in conjunction with Evidential Reasoning (ER) and Rule-based Bayesian Network (RBN) to quantify the risk levels of the identified hazards. The results show that ‘interaction with manned vessels and detection of objects’ contributes the most to the overall risk of MASS operations, followed by ‘cyber-attacks’, ‘human error’ and ‘equipment failure’. The findings provide useful insights on the major hazards and can aid the overall safety assurance of MASS.
Technical Report
Full-text available
Highly Autonomous Warship Technologies: Underpinning Safe, Secure and Cost-effective lean-crewed warships. Embracing the future of maritime autonomy this paper explores the core enabling technologies that will facilitate lean crewing on warships. There are many ethical and practical constraints that currently make a completely unmanned surface combatant unfeasible; therefore the near future solutions may utilise highly autonomous vessels alongside smaller unmanned counterparts to create an optimal force structure. In this thought leadership piece, BMT breaks down the challenges of autonomy in warship design into seven core development areas and creates a vision for how a warship in 2040 could operate.
Article
Full-text available
In the shipping digitalisation process, the peak will be reached with the advent of a wholly autonomous and at the same time safe and reliable ship. Full autonomy could be obtained by two linked Artificial-Intelligence systems representing the ship navigator and the ship engineer that possess sensing and analysis skills, situational awareness, planning, and control capabilities. Many efforts have been made in developing onboard systems; however, the shore facilities are not ready yet to deal with these new technologies. The paper aims to present the innovative technologies and methodologies needed to develop a futuristic Vessel Traffic System. The proposed systems will aim at faultless data acquisition and processing, provide input to decision-making systems, and suggest evasive manoeuvre; to deal with hazards and systems failure without human intervention onboard. The system is composed of three different and interacting layers. The first is an artificially intelligent tool to detect and control autonomous ships, thanks to situation recognition and obstacle avoidance strategies. The second is an orchestration and management platform designed to coordinate the sensing/actuation infrastructure and the AI algorithms’ results made available by multiple ships, mustering edge, and distributed computing techniques to fulfil the specific harsh requirements of the sea environment. The final part is a holistic guidance-navigation-control framework to manage autonomous ships’ navigation in a crowded area. Eventually, a cyber-physical scenario, using both a ship digital-twin and a real model-scale ship, is suggested to test and validate the innovative system without the availability of a full-scale scenario.
Article
Full-text available
In this paper, we describe a preliminary experiment of citizenship engagement in the context of marine robotics using imitation learning to train a controller that mimics human behavior. The experiment has been carried out during the Festival della Comunicazione in Camogli, Italy, in September 2019. In more detail, citizens have been asked to pilot a small, light, and safe autonomous surface vehicle in front of a crowded public beach with the goal of performing an S-shaped path. The trajectories and controls performed by non-expert human operators have been recorded with the aim of training an imitation system that, after collecting a sufficient number of trajectory-control pairs, has been able to drive the vehicle without human intervention. To learn the human behavior, echo state networks have been employed as approximating architectures. The resulting controller turned out to be very effective in successfully performing the considered experiment with a reduced amount of training trajectories by imitating the human behavior also in unknown situations. The success of this experiment may pave the way to new research processes where citizens are actively engaged.
Article
Full-text available
Since the last decade, several technologies have been abruptly influencing our lives. Among the notable major changes in the human society is the way humans interact with each other, which now involves the social media heavily where this is previously unknown to the masses. This type of technology is called Disruptive Innovation, a type of technology which has the potential to alter how human lives, market trends as well as other aspects including transportation and communications. Studies show that in the next few years, there are plenty of technology with the disruptive features. The new technologies, part of the domino effect of previous emerging technologies, include autonomous vehicle, the blockchain as well as Internet-of-Things. Despite the rapid advent, little discussion has been done to discuss and summarizes the emerging technologies in a single work in Malaysia. This brief survey is written as an effort to initiate the discussion, particularly in Malaysia about the disruptive innovation. It gives an introductory idea to the general audience, as well as academician and practitioners about the examples of the innovation, as well as brief discussion about the future studies which need to be commended in relation to the current development. This work is hoped to speed up the implementation of the emerging technologies in Malaysia as well helping and guiding policymakers, practitioners as well as academicians in understanding the phenomenon.
Conference Paper
Full-text available
A novel mathematical framework to support industrial digitization of shipping is presented in this study. The framework supports a data flow path, i.e. from Industrial IoT (i.e. with Big Data) to Predictive Analytics, where digital models with advanced data analytics are introduced. The digital models are derived from ship performance and navigation data sets and a combination of such models facilitates towards the proposed Predictive Analytics. Since the respective data sets are used to derive the Predictive Analytics, this mathematical framework is also categorized as a reverse engineering approach. Furthermore, a data anomaly detection and recover procedure that is associated with the same framework to improve the respective data quality is also described in this study.
Article
Full-text available
Deep Neural Networks (DNNs) have recently shown outstanding performance on image classification tasks [14]. In this paper we go one step further and address the problem of object detection using DNNs, that is not only classifying but also precisely localizing objects of various classes. We present a simple and yet powerful formulation of object detection as a regression problem to object bounding box masks. We define a multi-scale inference procedure which is able to produce high-resolution object detections at a low cost by a few network applications. State-of-the-art performance of the approach is shown on Pascal VOC.
Article
Full-text available
While IMO's e-Navigation project's scope is to enhance safety of navigation by improved ship-to-shore-cooperation, the EU's FP7 project MUNIN's aim is to develop a concept for an autonomous dry bulk carrier, that is at least as safe as a manned vessel. As e-Navigation has a strong focus on improving the human element in shipping and MUNIN tends towards an unmanned bridge, a common baseline might look quite contradictory at first, but they share the need to ensure and enhance the safety of navigation. After an introduction into e-Navigation and the MUNIN project, this paper will demonstrate with two examples, how MUNIN's results address identified e-Navigation's gaps and addresses e-Navigation's user needs. Thus, MUNIN contributes to the development and implementation of the prioritized e-Navigation solutions.
Article
In this article, we propose a survey of the Simultaneous Localization And Mapping field when considering the recent evolution of autonomous driving. The growing interest regarding self-driving cars has given new directions to localization and mapping techniques. In this survey, we give an overview of the different branches of SLAM before going into the details of specific trends that are of interest when considered with autonomous applications in mind. We first present the limits of classical approaches for autonomous driving and discuss the criteria that are essential for this kind of application. We then review the methods where the identified challenges are tackled. We mostly focus on approaches building and reusing long-term maps in various conditions (weather, season, etc.). We also go through the emerging domain of multi-vehicle SLAM and its link with self-driving cars. We survey the different paradigms of that field (centralized and distributed) and the existing solutions. Finally, we conclude by giving an overview of the various large-scale experiments that have been carried out until now and discuss the remaining challenges and future orientations.
Article
We describe the computing tasks involved in autonomous driving, examine existing autonomous driving computing platform implementations. To enable autonomous driving, the computing stack needs to simultaneously provide high performance, low power consumption, and low thermal dissipation, at low cost. We discuss possible approaches to design computing platforms that will meet these needs.
Article
This paper focuses on an overview of weather routing and safe ship handling approaches in the future of shipping. Weather routing provides the recommendations on transportation routes prior to and during ship sailing under various navigation constraints under global weather forecasts. Safe ship handling provides the recommendations on vessel position, orientation and speed conditions that should execute at the previously recommended route (with safe navigation conditions) under local weather conditions. Both approaches that complement each other should be implemented simultaneously to achieve optimal and safe ship navigation conditions. That will facilitate towards future navigation tools in integrated bridge systems, where the respective environmental pollution due to the shipping industry should be minimized.
Article
In this paper, we propose a driving system consisting of an autonomous and manual system for a four-wheel independent steering and four-wheel independent driving (4WIS4WID) vehicle. The autonomous driving system consists of three applications, lane following, reverse parking, and parallel parking, and is based on machine vision and fuzzy control theory. The vehicle implements these functions by using four webcams to detect lines on the ground and by using related control technologies to command all the server motors. The webcams are installed on all sides of the 4WIS4WID vehicle for all around viewing, but the vehicle still has many blind spots and the view is not wide enough. In additon, a parking space cannot be viewed completely in one image. Therefore, the integrating judgements of vision with the fuzzy control methods is necessary to make sure that the 4WIS4WID vehicle can perform the correct motions. By using the proposed fuzzy rules, the 4WIS4WID vehicle can change its velocity in a timely way under any condition and can successfully move in a narrow and curved lane. Moreover, the manual driving system is designed based on the traditional driving system, so that people can easily adapt to it. In order to verify the feasibility of these applications for the 4WIS4WID vehicle, an indoor real-time experiment is conducted.
Article
This study focuses on a collision detection methodology (i.e. collision risk assessment) in an integrated bridge system accounting for vessel state uncertainties in complex ship maneuvers. Modern technology solutions in integrated bridge systems (IBSs) to improve the navigation safety under vessel close encounters are discussed and ship navigation tools that would detect potential collision situations ahead of time are presented. The proposed vessel relative distance based collision risk detection and quantification methodology is simulated and evaluated under a two vessel encounter in a collision or near-collision situation. Furthermore, the relative navigation trajectory, the relative course-speed vector and the relative bearing vector of one vessel with respect to the other vessel are estimated by an extended Kalman filter. Then, the respective course-speed and relative bearing vectors are used to detect and quantify the collision risk between the vessels. Hence, the proposed collision risk detection and quantification methodology can be implemented in modem integrated bridge system (IBSs), where the potential risk among vessels ahead of a collision situation can be detected and that is also an important part of e-navigation.