Content uploaded by Lokukaluge Prasad Perera
Author content
All content in this area was uploaded by Lokukaluge Prasad Perera on Feb 15, 2019
Content may be subject to copyright.
Content uploaded by Lokukaluge Prasad Perera
Author content
All content in this area was uploaded by Lokukaluge Prasad Perera on Jun 29, 2018
Content may be subject to copyright.
1 Copyright © 2018 by ASME
Proceedings of the 37th International Conference on Ocean, Offshore and Arctic Engineering
OMAE2018
June 17-22, 2018, Madrid, Spain
OMAE2018-77672
AUTONOMOUS SHIP NAVIGATION UNDER DEEP LEARNING AND THE
CHALLENGES IN COLREGS
Lokukaluge P. Perera
UiT The Arctic University of Norway, Tromso, Norway
Prasad.Perera@uit.no
ABSTRACT
A general framework to support the navigation side of
autonomous ships is discussed in this study. That consists of
various maritime technologies to achieve the required level of
ocean autonomy. Decision-making processes in autonomous
vessels will play an important role under such ocean autonomy,
therefore the same technologies should consist of adequate
system intelligence. Each onboard application in autonomous
vessels may require localized decision-making modules,
therefore that will introduce a distributed intelligence type
strategy. Hence, future ships will be agent-based systems with
distributed intelligence throughout vessels. The main core of this
agent should consist of deep learning type technology that has
presented promising results in other transportation systems, i.e.
self-driving cars. Deep learning can capture helmsman behavior,
therefore that type system intelligence can be used to navigate
autonomous vessels. Furthermore, an additional decision support
layer should also be developed to facilitate deep learning type
technology including situation awareness and collision
avoidance. Ship collision avoidance is regulated by the
Convention on the International Regulations for Preventing
Collisions at Sea, 1972 (COLREGs) under open sea areas.
Hence, a general overview of the COLREGs and its
implementation challenges, i.e. regulatory failures and
violations, under autonomous ships are also discussed with the
possible solutions as the main contribution of this study.
Furthermore, additional considerations, i.e. performance
standards with the applicable limits of liability, terms,
expectations and conditions, towards evaluating ship behavior as
an agent-based system on collision avoidance situations are also
illustrated in this study.
INTRODUCTION
Ocean Autonomy
Autonomy can be expressed as a situation, where one can have
freedom from external controls or influences. However, the same
should have adequate intelligence to make appropriate decisions
in relation to possible internal and external variations. The same
concept has been adopted towards mechanical and electrical
systems by introducing machine intelligence-based decision-
making facilities to operate themselves. Such self-operating
systems, i.e. autonomous systems, should consist of advanced
decisions making facilities, therefore various technologies to
support the same should also be developed by the respective
industries. Recent technological advancements in autonomous
systems, i.e. self-driving cars, robots, etc. [1], can be such
examples consisting of adequate decision-making facilities.
Furthermore, these autonomous systems are also supported by
internal and external IoT (i.e. internet of things), big data and
communication infrastructure to overcome the respective
challenges. However, the success of decision-making features in
these autonomous systems is yet to be evaluated,
comprehensively by the respective authorities [2].
Autonomous systems will be an important role in the
future transportation systems, even though there are many
challenges [3]. Since self-driving vehicles have already been
introduced on public roads, similar systems will also be
introduced by air and maritime transport systems. This study
focusses on the navigation side of maritime transport systems
with autonomous and remote-controlled facilities, i.e.
Autonomous and remote-controlled ships. That will also be a
part of ocean autonomy as a transportation system. However,
ocean autonomy into the offshore sector yet to be developed.
Modern vessels are facilitated with onboard and
onshore IoT to support various digitalization applications in the
shipping industry. Industrial digitalization converts conventional
paper-based information handling approaches into data driven
applications. A comprehensive overview of vessels and ship
systems can be captured under industrial digitalization and that
information should use towards decision-making facilities of
future ships. Hence, industrial digitalization should support
ocean autonomy under the maritime transportation, where
human inference can be minimal by transforming big data
collected by IoT into intelligent decisions [4].
2 Copyright © 2018 by ASME
Fig 1: A general framework for autonomous ship navigation
There are several millstones that should be achieved by
the shipping industry to make autonomous ship navigation a
reality. "Remote Controlled Ship" can be an important milestone
in this route [5] and this same state, i.e. remote controlled
navigation facilities, can always be a part of autonomous ship
navigation. In general, each voyage can have both autonomous
and remote controlled navigation sector that should be
segmented by considering the operational requirements of future
vessels. The required maritime infrastructure to support both
remote controlled and autonomous ship operations should be
developed and that can also be another important milestone in
the same route [6]. The success in ship intelligence, i.e. artificial
intelligence to navigate and operate vessels and ship systems,
will make the most important milestone in this journey.
Appropriate decision support facilities to support ship
intelligence should also be developed and that will be another
important milestone. Finally, adequate tools and techniques to
evaluate vessel behavior under ship intelligence with decision
support facilities should also be developed, where the acceptable
risk and limits of liability, terms, expectations and conditions
under autonomous ship navigation should be defined.
Agent Based Systems
This study proposes the autonomous ship as an agent based
system. It is believed that this approach can also satisfy the
respective milestones, appropriately. An agent can be defined
as a system located in a specific environment, therefore it
interacts with the environment by intelligent decisions and
actions to satisfy its design objectives. However, other similar
and/or different agents can also be located within the same
environment, where these agents should interact. Therefore,
various cooperative and non-cooperative interactions among
these agents are expected in this same environment. However,
each agent may have its own design objectives, therefore
adequate intelligent to fully or partially satisfy the same should
be facilitated. If individual design objectives cannot be achieved
(i.e. unsatisfactory) in this environment, adequate compromising
strategies to satisfy appropriate group objectives should be
considered. Such situations can be categorized as a cooperative
multi-agent learning approach with machine learning approaches
(i.e. reinforcement learning) [7]. Therefore, adequate system
3 Copyright © 2018 by ASME
intelligence in each agent should be facilitated to handle rather
complex interactions among agents in the environment.
It is expected that autonomous ships will be agent based
systems, therefore various cooperative and non-cooperative
interactions among vessels in open sea areas and traffic lanes are
expected. In general, such intelligent agent should have the
following basic properties [7]:
Autonomy: Each agent should operate by its own
actions and/or internal states without the direct
inference of humans or others.
Social-ability: Each agent should interact with other
agents (i.e. including humans) by appropriate agent-
communication language.
Reactivity: Each agent should not only interact with the
environment but also respond to a timely fashion for the
respective environmental changes and challenges.
Pro-activeness: Each agent should not only interact
with the environment but also take appropriate
initiatives to exhibit goal-oriented behavior to satisfy its
design objectives.
One should note that these properties should also be a
part of future autonomous vessels. Therefore, the interactions
among vessels and ocean environmental conditions can be
facilitated by agent based systems. However,, vessels and ship
systems should have adequate ship intelligence and decision
support facilities to support these agent functionalities and that
can overcome the respective challenges in autonomous ship
navigation.
AUTONOMOUS VESSELS AND SHIP SYSTEMS
Future Vessels and Onboard Systems
It is expected that the shapes and types of future vessels will be
changed due to remote-controlled and autonomous operational
conditions. Remote-controlled vessels that will open the path
towards autonomous vessels will create the next generation
maritime transport systems. Human presence will be limited in
such vessels, where ship systems that supports humans onboard,
i.e. accommodation systems, air-conditioning, etc., can be
eliminated. Hence, the cost of installing human support systems
can be reduced, considerably and their maintenance can be
eliminated. Since remote-controlled and autonomous vessels
may not require to have adequate comfort for humans, the shapes
of such vessels can be designed in a way that reduce the
respective ship resistance, further.
However, additional vessel stability issues may rise
under such ship designs and that should also be addressed.
Though on the other hand, new ship systems, i.e. modern
decision support facilities, to support remote-controlled and
autonomous operations should be installed in these vessels.
Furthermore, the respective maintenance processes for these new
systems should be initiated. The overall reliability of these
vessels and ship systems should considerably be higher to cope
with harsh environmental conditions, therefore adequate
condition monitoring and condition based maintenance
approaches onboard as well as onshore should be available.
Future vessels will be facilitated with various onboard and
onshore IoT to monitor health conditions of individual systems
in real-time, i.e. possibly at component levels. Such network
connectivity will create big data sets and the respective system
health conditions, i.e. system information, can be extracted from
the same data sets. Hence, the respective system and component
failures can often be identified at an early stage due to their
health information. A considerable amount of recent research and
development activities are focused on conditions monitoring
(CM) and condition based maintenance (CBM) applications in
the shipping industry. There are onshore condition monitoring
centers, i.e. intelligent asset management centers, are introduced
to support future vessels by various industrial players [9].
System Intelligence
The navigation side of Autonomous vessels to achieve the
required level of ocean autonomy is discussed in this section. It
is expected that autonomous vessels will be trained by humans,
i.e. to navigate like an experienced helmsman. Similar training
case studies have successfully been implemented by other
transport systems, i.e. autonomous car, drones [10], therefore the
same approaches can be adopted by the maritime transport
systems. Furthermore, adequate system intelligence should be
developed within these vessels to absorb such knowledge from
the navigators during their training processes. Such intelligence
can be transferred from one vessel to another and that same can
further be improved during future ship operations. System
intelligence in autonomous vessels may consist of a complex
systems of system supported by additional decision supporting
facilities. A general framework to support the navigation side of
autonomous ships with the required technologies (i.e. navigation
and automation systems) is presented in Figure 1.
Initially, ship navigators will train these vessels to
achieve the respective system intelligence levels, i.e. to operate
both navigation and automation systems. The training process
can be done by onboard human presence and onshore remote-
controlled centers by introducing various navigation decisions
(i.e. human inference). That will be converted into navigation
actions. The navigation actions that required to achieve the
required ship route vs actual ship route can be considered as the
first level inputs to the training process (i.e. navigation training)
of ship intelligence (see Figure 1). Such navigation actions will
be implemented on the ship automation system, i.e.
propeller/thruster and rudder control systems, of the vessel to
achieve the required ship behavior i.e. ship speed, heading and
course.
One should note that these propeller/thruster and rudder
control system configurations can vary from one vessel to
another. Therefore, the navigator actions on propeller (i.e.
propeller pitch and speed) and rudder (i.e. rudder angle) control
systems can vary from one vessel to another. Furthermore, vessel
seakeeping and maneuvering behavior can be influenced by
vessel structures, ship systems and external environmental
conditions other than navigator’s actions. Both vessel behavior
4 Copyright © 2018 by ASME
and external environmental conditions should be monitored by
onboard and onshore IoT to provide required navigation
information. Each voyage consists of a required (i.e. expected)
ship route, therefore appropriate navigation actions should be
taken by the helmsman to achieve the same with respect to the
actual ship route.
Ship navigation systems consist of various IoT
including ECDIS (Electronic chart display and information
system), Radar and APRA (Automatic radar plotting aid),
Conning and additional systems and sensors. A combination of
such systems is classified as an integrated bridge system (IBS)
[11]. IBSs can collect and visualize the most important vessel
navigation and automation information including vessel
seakeeping and maneuvering behavior in the actual ship route
(i.e. vessel position, speed, course, heading and draft). This
information is collected as big data sets that should further be
analyzed to extract the respective seakeeping and maneuvering
behavior of the vessel. The accumulation of seakeeping and
maneuvering behavior and external environmental conditions
(i.e. navigation information) collected by IoT along with the
navigator’s actions are considered as the second level inputs to
the training process of ship intelligence. These two inputs, i.e.
navigation actions and information, complete the typical training
cycle, i.e. Training Process in Figure 1, of an autonomous vessel.
However, there are additional system layers that should support
system intelligence to succeed autonomous ship navigation.
These system layers are further illustrated in Figure 1 as:
Decision support, information sources and supporting services
and authorities.
Deep Learning in Shipping
A considerable section of ship intelligence will consist of a deep
learning based framework, i.e. artificial neural network. The
same framework will create the respective agent behavior within
autonomous vessels. Similar frameworks have been
implemented by other transport systems, i.e. autonomous
navigation systems of drive-less cars, and that have achieved
promising results in terms of navigating with the required safety
levels [12]. In general, deep leaning based frameworks
transform a self-driving vessel problem into a data classification
problem. e.g. convolutional neural networks (CNN, or ConvNet)
are a class of deep learning framework [13] that can solve
complex image classification problems and that have also been
used for self-driving vehicles. The same classification approach
can provide an elegant mechanism to capture helmsman
behavior, i.e. agent behavior in ship navigation. Initially, such
deep learning frameworks should be the observers to manual or
remote controlled vessels that are operated by human navigators.
That step is previously categorized as a training process (see
Figure 1) and the main objective of this phase is to train the
respective neural networks to capture ship behavior with respect
to navigator’s actions. Therefore, adequate features of vessel
seakeeping and maneuvering behavior can be accommodated
into these neural networks.
When such neural networks are adequately trained, that
technology can be used to navigate the first level autonomous
vessels. In addition, the same deep learning based frameworks
can be distributed (i.e. shared knowledge) among several vessels
and that can also be further trained during their operations. These
networks are often trained by image based information and
navigator actions rather than system parameters. A similar
approach can also be adopted towards the training phase of
autonomous vessels. These networks can transfer from one
vessel to another as mentioned before, however additional
training periods may require to capture highly accurate ship
behavior in some situations. If these vessels are standardized
during their ship design phase, then additional training
requirements can be eliminated. After a successful navigation
training period, ship intelligence can navigate such vessel as an
experienced helmsman, i.e. execution process in Figure 1.
Several technological challenges in controlling such
vessels, specially under rough weather conditions, should also be
expected. Conventional vessels are often categorized as under-
actuated systems, i.e. rudder and propeller control systems may
not able to control vessels completely during some sea going
situations. This ship controllability issue, i.e. under-actuation,
can complicate the training process of autonomous ships. One
should note that drones or under water vehicles navigate in a
single environmental media (i.e. land, air or water). Ships are
navigating between two environmental media (i.e. air and water)
and the interactions between both media influence on vessel
seakeeping and maneuvering behavior. That can further
complicate the training process of ship intelligence in under-
actuated vessels. The inertia in heavy vessels makes ship
controllability an extremely difficult challenge especially under
rough weather conditions.
The rudder and propeller control systems, i.e. only
available control units for vessel actuation, may fail to control
vessels under rough sea going conditions. When ships are
navigating under moderate or high speeds (i.e. over 3-4 knots),
the capabilities of thrusters are negligible. The control solutions
developed for autonomous surface vehicles (ASVs) and
autonomous underwater vehicles (AUVs) are not acceptable for
large vessels due to the same reasons that are mentioned
previously. Similarly, the control solutions developed for
autonomous land vehicles, i.e. driver-less car, are not acceptable
due to the same reasons, i.e. land vehicles are light weight
transportation units compared to ships and have a better
controllability over the roads. Hence, the controllability of under
actuated vessels under various navigation conditions should be
investigated in the near future. It is also expected that deep
learning frameworks supported by the decision support layer
(see Figure 1) may overcome some challenges in ship
controllability.
Decision Support Facilities
The decision support layer with various onboard and onshore IoT
supports ship intelligence. Each ship route may divide into
several voyage segments during voyage planning, i.e. possible
autonomous and remote-controlled navigation segments. Some
voyage segments of ship navigation may execute as remote-
controlled routes due to the respective safety and security
5 Copyright © 2018 by ASME
reasons. Global and local digital maps including navigation and
emission control rules and regulations should support the same
voyage planning phase. Furthermore, additional decision support
facilities such as weather routing and pilotage can also be a part
of voyage planning. Even though global maps are already
included under ECDISs, local maps (i.e. harbor areas and
confined waters) with the respective navigation rules and
regulations can be supplied by local maritime and port
authorities during the ship operation phase. Such local
information improves the safety of autonomous ship navigation,
therefore the pilotage type activities, i.e. humans with local
knowledge employed onboard ships to guide vessels, can be
eliminated. In addition, the respective maritime authorities can
further enforce energy efficiency and emission control rules and
regulations [14] on these vessels by distributing the respective
information. The energy efficiency and emission control rules
and regulations are enforced, extensively on the designated
emission control areas (ECAs). Hence, local digital maps can
provide of those information, accurately to enforce the respective
energy efficiency and emission control rules and regulations in
future vessels.
Local digital maps can be integrated with global maps
to support autonomous ship navigation under SLAM
(Simultaneous localization and mapping) type applications [15].
The SLAM type applications support intelligent agents, i.e.
autonomous vessels, to locate themselves within global and local
maps by considering the information collected from onboard and
onshore IoT. The agents can also learn about the environment in
some situations by executing possible actions. Therefore,
additional sensors, i.e. Lidar and Laser, should also be available
in autonomous vessels to support SLAM type applications.
Additional information resources such as VTI (vessel traffic
information), TSS (Traffic separation Schemes), AIS (Automatic
identification system) and LRIT (Long-Range Identification and
Tracking) can also support the same SLAM type applications in
future vessels. Some information sources are provided by vessel
traffic management and information systems (i.e. VTMIS).
Weather routing and safe ship handling should also
support ship intelligence under the decision support layer to
improve the safety and efficiency of ship navigation (see Figure
1). The required global and local weather information can be
obtained from weather centers by autonomous vessels. Weather
routing facilitates autonomous vessels by providing the
recommended ship routes prior to and during each voyage under
various navigational constraints and global weather forecast
[14]. Safe ship handling facilitates autonomous vessels by
providing recommended ship position, orientation and speed
conditions on the recommended route under similar navigational
constrains and local weather conditions [16]. In general, safe
ship handling applications have often been used for unexpected
rough weather conditions in ship routes. Therefore, both weather
routing and safe ship handling should support future vessels
under global and local weather forecast and that can also be an
important part of the voyage planning phase. Furthermore,
weather routing and safe ship handling can support each other
during the operation phase to achieve the required energy
efficient and safe ship navigational levels of autonomous vessels.
Conventional vessels consist of stability calculation
systems to estimate the respective ship loading conditions. Since
cargo loading and unloading activities at ports, vessels stability
should be calculated at the begging of each voyage. It is
expected that autonomous vessels should have similar decision
support facilities with additional IoT to verify the respective
loading calculations, accurately. Since future cargo loading and
unloading conditions in posts will also be automated, that
information can be shared with these decisions supporting layer.
Situation awareness and collision avoidance among
stationary and moving objects will play an important role in
autonomous ship navigation. The stationary objects, i.e. land
masks, shipwrecks, etc., are marked in global and local digital
maps and unexpected ones should be detected by onboard IoT.
The avoidance of both objects may relate to a path planning type
problem, especially under harbor or confined water navigation.
Situation awareness and collision avoidance facilities should
consist of adequate intelligence to detect and identify stationary
and moving objects. Such stationary and moving object detection
and classification has been done under deep learning frameworks
with successful results [17], therefore similar approach can be
adopted towards autonomous ship navigation. A general
overview in situation awareness and collision avoidance of
autonomous vessels in relation to the respective rules and
regulations is discussed in the following sections.
Situation Awareness and Collision Avoidance
Future vessels should be facilitated with appropriate collision
avoidance technologies. However, the behaviors of such
technologies should also be evaluated to guarantee the respective
safety levels of ship navigation [18, 19]. This section illustrates
the respective challenges in evaluating situation awareness and
collision avoidance technologies in autonomous vessels. In
general, such evaluation procedure for autonomous vessels
should consist of the following basic units:
Legal frameworks and their regulatory failures
Autonomous and target vessels and their behavior
Testable systems to evaluate ship behavior
Legal Frameworks and Their Regulatory Failures
All sea going vessels should follow the law of the sea. The
International Maritime Organization (IMO) in 1972 by the
Convention on the International Regulations for Preventing
Collisions at Sea (COLREGs) has introduced a legal framework
to regulate ship encounter situations [20]. The respective studies
of ship collisions indicate that 75%–96% of maritime collisions
and causalities caused by some types of human errors and 56%
of major maritime collisions involved one or more violations of
the COLREGs rules and regulations [21]. Future ships should
also be regulated by the same rules and regulations during their
vessel encounter situations and that behavior should be evaluated
by testable systems under acceptable performance standards.
Three distinct ship encounter situations that involve the risk of
6 Copyright © 2018 by ASME
collisions can be considered: overtaking, head-on, and crossing.
In general, the decision space of ship encounter situations with
two vessels can be categorized into the following stages in open
sea:
When none of the vessels are at a collision risk range,
both vessels have the option to take appropriate actions
to avoid a collision situation.
When both vessels are at a collision risk range, the
“give-way” vessel should take appropriate actions to
achieve safe passing distance in accordance with the
COLREGs rules and regulations, and the “stand on”
vessel should maintain course and speed conditions.
When both vessels are at a critical collision risk range,
and the “give way” vessel does not take appropriate
actions to achieve a safe passing distance in accordance
with the COLREGs rules and regulations, then the
“stand on” vessel has the option to take appropriate
actions to avoid the collision.
However, local navigation rules and regulations, traffic
lanes, offshore operations and special types of vessels
(i.e. fishing vessels) can override some of the same
decision space especially under overtaking and head-on
situations.
The COLREGs may have some regulatory failure situations
under the same decision space and that may lead towards
collision situations [22]. Such situations have been reported in
the previous studies [23], while applying the COLREGs rules
and regulations into if-then-else type computer codes under
Fuzzy Logic. Those situations are further illustrated in this
section to illustrate COLREGs regulatory failure situations.
Figure 2 represents a situation, where the target vessel is in
a head-on situation from the port, slightly with the own vessel.
That can create a regulatory failure situation within the
COLREGs rules and regulations, i.e. the directions these vessels
should pass each other. The respective vessel positions, O(k) &
Pi(k), course-speed vectors, Vo(k) & Vi(k) and relative
navigational trajectory of the target vessel are also presented in
the figure. Figure 3 represents a situation, where the target vessel
relative navigational trajectory varies from head-on to crossing
situations with the own vessel. Therefore, such situations can
also create a regulatory failure situation within the COLREGs
rules and regulations, i.e. which rules in head-on or crossing
situations should apply. Similarly, the respective vessel
positions, i.e. O(k) & Pi(k), course-speed vectors, i.e. Vo(k) &
Vi(k) and relative navigational trajectory of the target vessel are
also presented in the same figure. Figure 4 represents a situation,
where multiple target vessels are approaching the own vessel
from different directions. Therefore, such situations can also
create a regulatory failure situation within the COLREGs rules
and regulations, i.e. which vessel avoidance should have the
priority.
One should note that these are selected regulatory failure
situations of the COLREGs rules and regulations that are
observed by previous studies. Therefore, additional regulatory
Fig 2: Ship encounter situation 1
Fig 3: Ship encounter situation 2
Fig 4: Ship encounter situation 3
failure situations are yet to be discovered in the future, when
these rules and regulations are implemented under the decision
support layer (see Figure 1). Adequate measures to overcome
such regulatory failure situations within the COLREGs legal
framework should be considered and that will eventually
7 Copyright © 2018 by ASME
improve the required safety levels of autonomous ship
navigation.
Autonomous and Target Vessel Behavior
Vessels may not honor rules and regulations in some
navigation situations and that may create high collision risk
situations. One should note that such ship encounters are clear
regulatory violations within the COLREGs legal framework and
cannot be categorized as regulatory failure situations. That may
result in ship collisions and close encounter situations in open
sea and the COLREGs legal framework may not provide clear
guidance to overcome for such situations. In general, ship
navigators use their experiences to avoid such situations and that
may also lead to “crash stopping” type maneuvers of the “stand
on” vessel due to a lack of distance for speed reduction. One
should note that the “stand on” vessel represents the ship that has
the priority for navigating in an encounter situation.
The characteristics of the “stand on” vessel with respect
to its “stopping distance” and “turning circle” should be
considered for developing appropriate collision avoidance
strategies for future vessels in such situations. Since the expert
knowledge has been used for such ship encounter situations, it is
expected that the same knowledge can be absorbed by the deep
learning framework of ship intelligence. These high collision
risk situations can be intentional or unintentional, however the
respective situation awareness and collision avoidance facilities
under ship intelligence should take appropriate actions to find
appropriate navigational solutions. On the other hand, adequate
tools and techniques to predict the intensions of vessels should
also be considered to support ship intelligence.
High Collision Risk Ship Encounters
The responses (i.e. vessel behavior) of autonomous vessels, i.e.
ship intelligence, in such high collision risk encounters should
also be evaluated and the acceptability of their navigational
actions should be investigated. That can be done by proper
testable systems supported by appropriate performance
standards to evaluate ship intelligence. These performance
standards are likely to be defined by the respective maritime
authorities or classification societies and evaluates the behavior
for autonomous vessels. However, the evaluation process of
autonomous vessels under critical collision situations should be
carefully formulated, since collision avoidance approaches also
depend on vessel maneuverability characteristics. The
maneuverability characteristics may be captured by ship
intelligence under deep learning type frameworks. Therefore,
ship intelligence may provide initial performance standards to
evaluate the first generation autonomous vessels.
Various ship navigation situations with high collision
risks should be recreated to evaluate vessel behavior, i.e. ship
intelligence, under the testable platforms. The results, i.e. vessel
behavior under ship collision or near collision situations, should
be communicated towards system developers in autonomous
vessels to improve their ship intelligence and decision support
facilities. Furthermore, testable systems should suggest the
required autonomous ship behavior, i.e. to reduce the collision
risk, with the appropriate reasoning about compliance
requirements of the legal frameworks, i.e. the COLREGs and
local navigational rules and regulations. Any unusual vessel
behavior or collision avoidance failures, i.e. regulatory failure
situations, may lead to introduce additional modifications into
the legal frameworks. One should note that collision worthiness
and survivability can also be important features for transport
systems. However, autonomous vessels may focus on collision
avoidance features for ship encounter situations rather than
collision worthiness and collision survivability, extensively.
Testable System for Vessel Behavior
Fig 5: Testable system for evaluating autonomous vessel behavior
A testable system with required components to evaluate collision
avoidance behavior of autonomous vessels is presented in Figure
5. That is influenced by two or more legal frameworks i.e.
COLREGs and other local navigation rules and regulations. This
system evaluates autonomous ship behavior, i.e. as an agent
based system, under various encounter situations with target
vessels. Therefore, several navigational ship routes with varying
weather conditions should be introduced in the same to create
realistic ship navigation conditions. Autonomous vessel
behavior can further be detailed into navigation and automation
system levels by monitoring the respective systems. Therefore,
vessel and ship system behavior should be evaluated by the
testable system with pre-defined performance standards. That
provides the specific performance expectations that should be
satisfied by autonomous vessels for various ship encounter
situations.
Appropriate course and/or speed changes, at any cost,
must be taken by the vessels to avoid collision situations, as
highlighted in the COLREGs. Hence, that unique feature can
introduce the minimum expectation level for the performance
standards of the testable systems. However, the ultimate
expectation of the performance standards may relate to selected
policy options and/or legal frameworks. Furthermore, that will
also be influenced by the applicable limits of liability, terms,
expectations and conditions that are introduced by the insurance
8 Copyright © 2018 by ASME
industry. However, existing performance standards [24, 25] for
ship navigation can also be adopted by the same testable systems.
The initial performance standards for situation
awareness and collision avoidance should be developed in
relation to a two vessel encounter situation and that can be
expanded towards multiple vessel encounter situations. Hence,
the performance standards should illustrate the following
navigation features to evaluate vessel encounter situations:
Autonomous and target vessel domains
The collision risk between the vessels
The distance and time to a possible collision/close
encounter situation
Autonomous and target vessels course-speed vectors
The bearing vector between two vessels
Autonomous vessel decisions and the time to execute
the same decisions into actions
Autonomous and target vessel predicted and actual
behavior.
The satisfactory level (i.e. under the performance
standards) of the decisions and actions that have taken
by the vessels.
If the autonomous vessel fails to satisfy the expected
performance standards, then the recommendations on navigation
and automation system improvements should be forwarded to
the respective manufacture. Therefore, adequate modifications
not only in a system level but also a ship intelligence and
decision support level should be considered (see Figure 5). It is
also recommended to implement the following testing levels in
the proposed platform to evaluate situation awareness and
collision avoidance behavior in autonomous vessels under ship
encounter situations:
Testing level 1: Both autonomous and target vessels are
simulated by software programs.
Testing level 2: Target vessels are simulated by a
software program and the autonomous ship is
represented by a full-scale/model-scale vessel. The
autonomous vessel is navigating in approved waters.
Testing level 3: Both autonomous and target vessels are
represented by full-scale/model-scale vessels. That are
navigating in open sea.
It is expected that testable systems will initially evaluate
autonomous vessels under the first testing level. Both
autonomous and target vessels can be considered as agent based
systems in this situation, where appropriate mathematical
models to simulate realistic ship behavior are required. In
addition, several weather conditions can also be introduced in
this situation to observe the variations in vessel behavior. The
respective sea trial data that are collected from ocean going
vessels can be used to develop such mathematical models for
these vessels [26]. Such mathematical model of ship
maneuvering should consist of: ship course-speed vector,
heading vector and turning rates. That can also be estimated from
time-series data sets, i.e. AIS data, and such data sets can also
influence on the respective performance standards.
The lessons learned from the first testing level should
consider for the second testing level and that represents realistic
ocean-going conditions. The implementation of the third testing
level can be a challenge due to the difficulties in creating vessel
collision situations in open sea, intentionally (i.e. due to
maneuverability difficulties in vessels). Therefore, the first and
second testing levels can be considered as possible situations that
can be implemented in the testable system. Such two vessel
encounter situations can be accumulated towards multi-vessel
encounter situations, as mentioned previously.
These evaluation procedures of ship behavior in the
testable systems should consider realistic ocean-going
conditions. That include all possible navigation scenarios
including various ship encounter situations, varying
environmental conditions, and random ship system & sensor
failures. However, the main objective in this testable system is to
evaluate vessel behavior of autonomous and target ships rather
than the reliability of hardware and software systems. Even
though system hardware and software failures can eventually
influence on ship intelligence and decision support layers of
future vessels, a considerable amount to tools and techniques to
regulate such failures have been developed by the respective
industries. In general, both hardware and software developments
are often guided by system development (i.e. the V model) [27]
and agile software development approaches [28] by various
manufactures. Therefore, future ship systems will have higher
reliability levels due to the maturity of the respective
technologies. Hence, adequate research focuses should be aimed
towards ship intelligence and decision support facilities of
autonomous vessels, rather than individual system and
component failures.
CONCLUSIONS
A general framework to support the navigation side of
autonomous vessels is discussed in this paper and that consists
of various technologies to achieve the required level of ocean
autonomy. Since decision-making processes in autonomous
vessels will play an important role under ocean autonomy, the
same technologies should consist of adequate system
intelligence. Each onboard application in autonomous ships may
require a localized decision-making process, where a distributed
intelligence type strategy should be considered. Hence, the ship
should be an agent-based system with distributed intelligence
throughout the vessel. The main core of such agent consists of
deep learning type frameworks, i.e. ship intelligence, to simulate
the helmsman actions in ship navigation. Furthermore, an
additional decision supporting layer should also available to
facilitate situation awareness and collision avoidance among
vessels. Hence, a considerable amount of research and
development work will be required to achieve required ship
intelligence within deep learning frameworks and machine
learning applications for decision support of autonomous
vessels.
9 Copyright © 2018 by ASME
The required technologies to implement and evaluate
ship intelligence in transport systems are still in a preliminary
stage. Therefore, a considerable amount of such knowledge yet
to be created. The same knowledge should be shared among
research communities to develop safer maritime transportation
systems. Autonomous system developers will play a crucial role
in developing and sharing the knowledge on ship intelligence
and decision support facilities and that may push machine
learning into a more regulated industry. However, the human
interactions and their outcome under artificial intelligence yet to
be investigated by the research community.
Collision avoidance among autonomous and target
vessels is focused in this study. The same ship intelligence with
decision support facilities in collision avoidance should also be
evaluated under various ship encounter situations. That can be
done by testable systems of situation awareness and collision
avoidance and the outcome should be compared with the
applicable limits of liability, terms, expectations and conditions
in ship navigation. That will be resulted in appropriate
performance standards to evaluate vessel behavior as an agent
based system in various ship encounter situations. It is also noted
that the respective navigation rules and regulations may have
regulatory failure and violation situations under ship intelligence
and decision support facilities, therefore adequate measures to
overcome such challenges should be considered.
REFERENCES
[1] Li, T. H. S., Lee, M. H., Lin, C. W., Liou, G. H. and Chen, W. C.,
(2016) "Design of Autonomous and Manual Driving System for
4WIS4WID Vehicle," in IEEE Access, vol. 4, pp. 2256-2271.
[2] Dokic, J., Müller, B., and Meyer, G., 2015, European Roadmap
Smart Systems for Automated Driving, European Technology
Platform on Smart Systems Integration (EPoSS).
[3] Rahman, A. A., Hamid, U. Z. A., Rahman, T. A. C., (2017),
"Emerging Technologies with Disruptive Effects: A Review"
PERINTIS eJournal. 7 (2).
[4] Perera, L. P., "Industrial IoT to Predictive Analytics: A Reverse
Engineering Approach from Shipping," In Proceedings of the 3rd
Norwegian Big Data Symposium (NOBIDS), Trondheim, Norway,
November, 2017.
[5] Perera, L. P., Ferrari, V., Santos, F. P., Hinostroza, M. A., and
Guedes Soares, C., (2015) "Experimental Evaluations on Ship
Autonomous Navigation & Collision Avoidance by Intelligent
Guidance," IEEE Journal of Oceanic Engineering, vol. 40, no. 2,
pp. 374-387.
[6] Rolls-Royce, (2016) “Remote and Autonomous ship: the next
step,” AAWA position paper.
[7] Panait, L. and Luke, S., (2005), "Cooperative Multi-Agent
Learning: The State of the Art", Autonomous Agents and Multi-
Agent Systems, vol. 11, no. 3, pp. 387-434
[8] Wooldridge, M., and Jennings, N., (1995), “Intelligent agents:
Theory and practice,” The Knowledge Engineering Review, vol.
10, no. 2, pp. 115-152.
[9] Rolls-Royce, 2016, Ship intelligence transforming future marine
operations, Norway.
[10] Giusti, A., Guzzi, J., Ciresan, D.C., He, F., Rodriguez, J.P.,
Fontana, F., Faessler, M., Forster, C., Schmidhuber, J., Di Caro, G.,
Scaramuzza, D., and Gambardella, L.M., (2016), "A Machine
Learning Approach to Visual Perception of Forest Trails for Mobile
Robots, " IEEE Robotics and Automation Letters (RA-L), pages
661 – 667.
[11] Perera, L.P., and Guedes Soares, C., (2015), "Collision Risk
Detection and Quantification in Ship Navigation with Integrated
Bridge Systems," Journal of Ocean Engineering, vol. 109, pp. 344-
354.
[12] Liu, S., Tang, J., Zhang, Z., and Gaudiot, J. L., (2017), "Computer
Architectures for Autonomous Driving," in Computer, vol. 50, no.
8, pp. 18-25.
[13] Collobert, R., and Weston, J., (2008), "A Unified Architecture for
Natural Language Processing: Deep Neural Networks with
Multitask Learning". Proceedings of the 25th International
Conference on Machine Learning. ICML '08. New York, USA:
ACM: 160–167.
[14] Perera, L.P., and Guedes Soares, C., (2017), "Weather Routing and
Safe Ship Handling in the Future of Shipping," Journal of Ocean
Engineering, vol. 130, pp. 684-695.
[15] Bresson, G., Alsayed, Z., Yu, L., and Glaser, S., (2017),
"Simultaneous Localization and Mapping: A Survey of Current
Trends in Autonomous Driving," in IEEE Transactions on
Intelligent Vehicles, vol. 2, no. 3, pp. 194-220.
[16] Perera, L. P., Rodrigues, J. M., Pascoal, R. and C. Guedes Soares,
C., (2012),"Development of an onboard decision support system
for ship navigation under rough weather conditions,", Sustainable
Maritime Transportation and Exploitation of Sea Resources, E.
Rizzuto & C. Guedes Soares (Eds.), Taylor & Francis Group,
London, UK, pp. 837-844.
[17] Szegedy, C., Toshev, A., and Erhan, D., (2013). "Deep neural
networks for object detection". Advances in Neural Information
Processing Systems.
[18] The Munin Project, [online] Available: http://www.unmanned-
ship.org/munin/wp.
[19] Burmeister, H.-C., Bruhn, W., Rødseth, Ø. J., Porathe, T.,
“Autonomous Unmanned Merchant Vessel and its Contribution
towards the e-Navigation Implementation: The MUNIN
Perspective,” International Journal of e-Navigation and Maritime
Economy, vol. 1, 2014, pp. 1-13.
[20] IMO, (1972) “Convention on the International Regulations for
Preventing Collisions at Sea (COLREGs),” [Online]. Available:
http://www.imo.org/conventions
[21] Rothblum, A.M., Wheal, D., Withington, S., Shappell, S. A.,
Wiegmann, D. A., Boehm, W., and Chaderjian, M., (2002), “Key
to successful incident inquiry,” in Proc. 2nd Int. Workshop Human
Factors Offshore Oper., pp. 1–6.
[22] Perera, L.P., Carvalho, J.P., and Guedes Soares, C., (2012),
"Intelligent ocean navigation & Fuzzy-Bayesian decision-action
formulation," IEEE Journal of Oceanic Engineering, vol 37, no 2,
pp. 204-219.
[23] Perera, L.P., Carvalho, J.P., and Guedes Soares, C., (2014),
"Solutions to the Failures and Limitations of Mamdani Fuzzy
Inference in Ship Navigation," IEEE Transactions on Vehicular
Technology, vol. 63, no. 4, pp. 1539-1554.
[24] IMO, (2002), ’Standards for Ship Manoeuvrability,’ Resolution
MSC.137 (76).
[25] IMO, (2002), ‘Explanatory Notes to the Standards for Ship
Manoeuvrability,’ MSC/Circ.1053.
[26] International standard ISO 15016 and ISO 19019, (2002), “Ship
and marine technology – Guidelines for the assessment of speed
and power performance by analysis of speed trial data”, First
edition.
10 Copyright © 2018 by ASME
[27] Forsberg, K. and Mooz, H., (1991), "The Relationship of System
Engineering to the Project Cycle", in Proceedings of the First
Annual Symposium of National Council on System Engineering,
pp. 57–65.
[28] Abrahamson, P., Salo, O., Ronkainen, J., and Warsta, J., (2002).
Agile software development methods: Review and analysis
(Technical report). VTT. 478.
