ChapterPDF Available

Towards an Optimal Design for Ecosystem-Level Ocean Observatories

Authors:
  • The Fish Listener
  • Institut de Ciències del Mar (ICM-CSIC); Stazione Zoologica of Naples (SZN) "Anton Dohrn"

Abstract and Figures

Four operational factors, together with high development cost, currently limit the use of ocean observatories in ecological and fisheries applications: 1) limited spatial coverage, 2) limited integration of multiple types of technologies, 3) limitations in the experimental design for in situ studies, and 4) potential unpredicted bias in monitoring outcomes due to the infrastructure's presence and functioning footprint. To address these limitations, we propose a novel concept of a standardised 'ecosystem observatory module' structure composed of a central node and three tethered satellite pods together with permanent mobile platforms. The module would be designed with a rigid spatial configuration to optimise overlap among multiple observation technologies, each providing 360° coverage of a cylindrical or hemi-spherical volume around the module, including permanent stereo video cameras, acoustic imaging sonar cameras, horizontal multibeam echosounders, and a passive acoustic array. The incorporation of multiple integrated observation technologies would enable unprecedented quantification of macrofaunal composition, abundance, and density surrounding the module, as well as the ability to track the movements of individual fishes and macroinvertebrates. Such a standardised modular design would allow for the hierarchical spatial connection of observatory modules into local module clusters and larger geographic module networks providing synoptic data within and across linked ecosystems suitable for fisheries and ecosystem-level monitoring on multiple scales.
Schematic 3-dimensional illustration (not to scale) of the spatial configuration of the central node and satellites of a module and the overlap among video, acoustic imaging sonar, and echosounder spatial coverage areas (inset provides a bird's-eye-view of the spatial configuration). Integration of multiple 3-dimensional observation modalities in a module provides cross-referencing data to identify and track organisms. (1-6) a single individual of species A is tracked as it moves through the module area. Changes in echosounder target strength due to orientation changes can be quantified by comparison of different observations at each location, enhancing our ability to determine fish identification from target strength data. (1) Silent individual detected by echosounder and acoustic imaging sonar, provides target strength, orientation, location, and size estimates. (2) Same individual produces sound loud enough to be localised by the passive acoustic array, actual location and identification confirmed by stereo-video, acoustic image, and echosounder. Sound received level can then be corrected for exact location to obtain sound source level and detection range. (3) Now silent, it is detected by stereo video, acoustic imaging sonar, and echosounder from the central node, as well as stereo-video from a satellite. (4) The individual moves out of camera range but continues to be tracked by acoustic imaging sonar and echosounder. (5) As the individual leaves the module area, it continues to be tracked by echosounder. (6) Separate individual of species A is detected on echosounder only, but target strength consistency of species A at other locations permits accurate attribution to species within the wider area covered by the echosounder. (7) A second unknown species is detected by acoustic imaging sonar and echosounder and is identified based on localisation on a known sound. (8) The reaction of multiple fish of species C to light is quantified by acoustic imaging sonar and echosounder.
… 
Content may be subject to copyright.
Oceanography and
Marine Biology
An Annual Review
Volume 58
Edited by
S. J. Hawkins, A. L. Allcock, A. E. Bates, A. J. Evans, L. B. Firth,
C. D. McQuaid, B. D. Russell, I. P. Smith, S. E. Swearer, P. A. Todd
First edition published 2021
ISBN: 978-0-367-36794-7 (hbk)
ISBN: 978-0-429-35149-5 (ebk)
Chapter 2
Towards an Optimal Design for Ecosystem-Level Ocean
Observatories
Rodney A. Rountree, Jacopo Aguzzi, Simone Marini, Emanuela Fanelli,
Fabio C. De Leo, Joaquin Del Rio & Francis Juanes
(CC BY-NC-ND 4.0)
79
Oceanography and Marine Biology: An Annual Review, 2020, 58, 79 –106
© S. J. Hawkins, A. L. Allcock, A. E. Bates, A. J. Evans, L. B. Firth, C. D. McQuaid, B. D. Russell,
I. P. Smith, S. E. Swearer, P.A. Todd, Editors
Taylor & Francis
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-
LEVEL OCEAN OBSERVATORIES
RODN EY A. ROU NTREE1,2, JACOPO AGUZZ I3, SIMONE MARINI4, EMANUELA
FANE LLI 5, FABIO C. DE LEO6,2 , JOAQUIN DEL RIO7 & FRANCIS JUANES2
1The Fish Listener, 23 Joshua Lane, Waquoit, Massachusetts, USA
2Department of Biology, University of Victoria, Victoria, Canada
3Instituto de Ciencias del Mar (ICM-CSIC), Barcelona, Spain
4National Research Council of Italy (CNR), Institute of Marine Sciences, La Spezia, Italy
5Department of Life and Environmental Sciences,
PolytechnicUniversity of Marche, Ancona, Italy
6Ocean Networks Canada, University of Victoria, Victoria, Canada
7OBSEA, SARTI, Universitat Politecnica de Catalunya (UPC), Barcelona, Spain
Abstract Four operational factors, together with high development cost, currently limit the use of
ocean observatories in ecological and sheries applications: 1) limited spatial coverage, 2) limited
integration of multiple types of technologies, 3) limitations in the experimental design for in situ
studies, and 4) potential unpredicted bias in monitoring outcomes due to the infrastructure’s presence
and functioning footprint. To address these limitations, we propose a novel concept of a standardised
‘ecosystem observatory module’ structure composed of a central node and three tethered satellite
pods together with permanent mobile platforms. The module would be designed with a rigid spatial
conguration to optimise overlap among multiple observation technologies, each providing 360°
coverage of a cylindrical or hemi-spherical volume around the module, including permanent stereo
video cameras, acoustic imaging sonar cameras, horizontal multibeam echosounders, and a passive
acoustic array. The incorporation of multiple integrated observation technologies would enable
unprecedented quantication of macrofaunal composition, abundance, and density surrounding the
module, as well as the ability to track the movements of individual shes and macroinvertebrates. Such
a standardised modular design would allow for the hierarchical spatial connection of observatory
modules into local module clusters and larger geographic module networks providing synoptic
data within and across linked ecosystems suitable for sheries and ecosystem-level monitoring on
multiple scales.
Keywords: Ocean observatories, Ocean technology, Ecological monitoring, Networks, Coenoclines,
Deep-sea, Behaviour, Optoacoustic technologies, Passive acoustic, Fish sounds, Cyber interfaces
Introduction
Ocean observatories have become important resources for oceanographic observations around the
world and consist of networks of instruments primarily designed to collect data on oceanographic
and geophysical conditions in real time over long durations (Tunnicliffe etal. 2003, Schoeld
& Glenn 2004, Aguzzi etal. 2012, Gould etal. 2013). However, increasingly, observatories are
80
RODNEY A. ROUNTREE ET AL.
becoming useful tools for biologists interested in animal behaviour, ecosystem ecology, and sheries
applications (ACT 2007, Aguzzi etal. 2011b, 2015a, Barns etal. 2013). Four operational factors,
besides development costs, limit applicability of existing ocean observations systems for use as
tools in sheries and ecosystem level applications: 1) limited spatial coverage, 2) limited integration
of multiple types of technologies (i.e. multiple modalities of observation), 3) limitations in the
experimental design for in situ studies, and 4) potential unpredicted bias in monitoring outcomes
due to the infrastructure’s presence and functioning footprint. These limitations have slowed the
spread of ocean observatory use for sheries and other ecological applications (e.g. benthopelagic
connectivity), highlighting the need for efforts to improve observatory design (e.g. Handegard etal.
2013, Locascio etal. 2018).
The objective of this review is to show how ocean observatories, combined with other
observational sampling technologies, can be better designed from sheries and ecology perspectives
for the monitoring of marine ecosystems and their connectivity through coenoclines (i.e. gradients
of communities) formed along depth, latitude, and geographic gradients. What is unique about
the suggested approach is that systems would be designed from the beginning for ecosystem-level
observations on large spatial and temporal scales and would be replicated in many locations for
global coverage. In order to meet these objectives, observatories need to be highly standardised
and produce quantitative observations that are comparable among locations and over time. The
present review presents a concept of standardised modular platform design that is intended to
stimulate discussion and renement within the scientic community. The ecosystem observatory
module (EOM) concept (hereafter simply referred to as the ‘module’) consists of a central node
and three tethered satellite pods (hereafter referred to as ‘satellites’). A modular design means that
the platforms should be designed so that they can be prefabricated, and therefore produced at lower
cost, but be exible enough to allow customisation and implementation in different habitats. Such
a design serves two purposes: rst, it provides directly comparable data among different locations,
and second, it will encourage wider implementation of observatories around the globe. Most of
the instrumentation proposed for each module has already been developed and implemented in
some existing cabled observatories, though signicant improvements in capabilities and reduction
in cost are needed (see review in Aguzzi etal. 2019). In addition, much of the software needed to
realise large-scale observatory networks that are useful to sheries scientists, resource managers, and
ecologists are still in the early stages of development (Allken etal. 2018, Juanes 2018, Marini etal.
2018a,b). Therefore, the development of data delivery systems that are accessible to a wide range of
stakeholders from different disciplines and backgrounds is of vital importance for the effective use
of ocean observatories for sheries and ecological applications (Pearlman etal. 2019). It is therefore
important to design data packaging and delivery systems in concert with the observatory structural
and instrumentation design, rather than as an afterthought.
The implementation of permanent monitoring systems should deliver data on animal movement
across habitat gradients (Aguzzi etal. 2015a) and energy ux interchange (Thomsen etal. 2017),
providing measures of ecosystem functioning (Aguzzi etal. 2019). Time series of visual counts
for different species by different monitoring modules and their satellites of a network may provide
spatially meaningful representations of a population’s abundance uctuations when data are summed
(i.e. scaled) together (Aguzzi et al. 2019). Spatiotemporal variations in population abundances
could then be used to track the status of ecosystem services such as sheries resources. Major
requirements for ecosystem-level ocean observatory networks include 1) spatial quantication of
organism abundance, density, and biomass through cross-referencing of data obtained from multiple
observation technologies; 2) quantication of the impact of the observatory structure and operation
of its instruments on the local biota; 3) a design for use of observatories as in situ laboratories; 4)
spatial clustering of observatories/devices to optimise observation on multiple spatial scales over
appropriate coenoclines; 5) integration of ocean observatory data with observational data collected
through other sampling methodologies (e.g. ship, satellite, drifter, and buoy-based surveys and
81
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
animal-borne devices); 6) automatic data processing, such as detection of sh images or sounds to
enhance data analysis by end-users; and 7) seamless presentation of multiple data streams to end-
users that are synchronised in time across all instruments within a module and ultimately across all
module locations.
This review starts with a brief summary of ocean habitat connectivity to provide context, followed
by a description of a proposed ecosystem observatory system, including modules, components, and
ways of combining modules into clusters and networks to allow monitoring along habitat gradients
and coenoclines. A description of monitoring modelling and forecasting of observatory data is
followed by an explanation of how observatories could be integrated with animal-borne technologies
and the cyber developments needed to support monitoring networks accessible to users of different
backgrounds. The nal section is a rationale for the incorporation of observatory systems into
commercial developments such as windfarms and oil platforms (e.g. Fujii & Jamieson 2016) to serve
as partial mitigation for potential ecosystem impacts by extending marine ecosystem monitoring
capability beyond what would otherwise be nancially feasible.
Background: Ecosystem connectivity
Current ocean observatories have limited applicability to sheries and ecosystem monitoring, in
part because marine habitats exhibit complex linkages that operate on many different scales. A
brief summary of ocean ecosystem connectivity helps to provide context for the rationale for highly
standardised observatories that are organised in hierarchical spatial congurations to enhance
quantication of ecosystem attributes along habitat gradients or coenoclines.
Researchers have long known that marine ecosystems are intricately linked through passive and
active mechanisms for matter and energy transference. For example, estuaries serve as an important
direct and indirect source of nutrients for coastal marine waters and thereby help to sustain coastal
and deep-water sheries (e.g. Teal 1962, Haines 1979, Nixon 1980, Odum 1980, Pomeroy & Wiegert
1981, Dame etal. 1986). Passive processes involve bi-directional uxes of nutrients, pollutants,
and plankton carried by water movements such as runoff, river ow, tides, up- and down-welling,
storm events, and dense shelf-water cascading, all acting along a habitat gradient from freshwater to
coastal areas and to the deep sea (Figure 1; Canals etal. 2006, Afonso etal. 2014, Puig etal. 2014,
Rogers 2015, Thomsen etal. 2017). Active processes also contribute to energy/matter transference
in the form of rhythmic and arrhythmic population movements across seabed and water column
depth gradients, such as diel vertical migrations (DVMs), which represent the largest natural daily
movement of biomass on the planet (e.g. Graeme etal. 2010, Doya etal. 2014, Aguzzi etal. 2015b,
De Leo etal. 2018; Figures 1 and 2).
Mechanisms that regulate nekton distribution and movements along bathymetric and latitudinal
coenoclines are similar and involve interactions between environmental (e.g. temperature gradients
and cyclic uctuations) and biological conditions (e.g. food and shelter availability and predation
risk) (see reviews in Rountree 1992, Deegan etal. 2000, Rountree & Able 2007, Aguzzi & Company
2010, Aguzzi et al. 2011a). Horizontal linkages have been referred to as the ‘chain-of-migration’
(Rountree 1992, Deegan etal. 2000, Rountree & Able 2007), while vertical migrations have been
referred to as the ‘ladder-of-migration’ (Vinogradov 1953, 1955, 1971). Mechanisms for linkages
along a depth coenocline from the photic to dysphotic pelagic zones and the deep-sea benthos include
‘organic rain’ (Vinogradov 1971, McCave 1975, Honjo 1980, Alldredge & Silver 1988, Thomsen
etal. 2017), ontogenetic (i.e. with size or life-stage) vertical migration of organisms (e.g. Merrett
1978, Wakeeld & Smith 1990, Kobari etal. 2008, De Leo etal. 2018), and cyclic vertical migrations
such as observed in the deep scattering layers (DSLs, Vinogradov 1953, Marshall 1971, Longhurst
1976, Mauchline 1980, Aguzzi & Company 2010, Naylor 2010, Aguzzi etal. 2017). In particular,
rhythmic depth strata movements similar to DVMs also occur within the benthic boundary layer
across shelves and slopes by endobenthic burrowing organisms (Aguzzi & Company 2010). Indirect
82
RODNEY A. ROUNTREE ET AL.
Figure 1 Exa mple of some major linkages among habitats dist ributed along horizonta l and vertical coenoclines
connecting terrestrial to deep-sea ecosystems along a large river system. Major linkages are provided by a
chain-of-migration connecting habitats horizontally, while a ladder-of-migration connects vertical habitats
through ontogenetic and cyclical movements of organisms (see Figure 2). Other mechanisms of linkage include:
(A) run-off from land to sea; (B) nutrient, detritus, and organism ‘outwelling’ and corresponding ‘inwelling’;
and (C) upwelling/downwelling occur largely due to water movements such as tides and storms; (D) deposition
occurs where water velocity slows to allow precipitation of suspended materials and entrapment and mortality
of organisms, as well as faecal deposition of migrating organism; and, nally, (E) organic and inorganic rain.
Figure 2 Example of mechanisms of energetic linkages among adjacent habitats or ecosystems through the
distribution and movements of organisms. Major mechanisms include diffusion, ontogenetic migration, and
chain-of-migration. Diffusion results from trophic transfer of energy among overlapping assemblages and
is poorly understood. Ontogenetic migration results from movements of organisms among habitats as they
grow and can be size, environmental condition (such as temperature), or seasonally mediated. The chain-of-
migration (and analogous ladder-of-migration) results from rhythmic movements of organisms among habitats
on seasonal, lunar, diel, or tidal cycles. The smallest links in the chain are between adjacent habitats, but
links from direct movements of organisms can occur on any spatial scale among habitats located along the
same coenocline. Major mechanisms of energy transfer include predator–prey interactions, spawning, faecal
deposition, and local mortality.
83
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
day-night synchronisation of biological activity in deep-sea aphotic realms may also occur due
to the movements of deep-scattering layer organisms (e.g. Irigoien etal. 2014). These rhythmic
movements may also be accompanied by changes in background illumination at the seabed, when
species making up the scattering layers are bioluminescent (i.e. bioluminescence panoramas; Aguzzi
etal. 2017).
All these types of ontogenetic and rhythmic (e.g. diel and seasonal) movements produce energy
uxes that affect the functioning of ecosystems connected through a coenocline (Rountree & Able
2007, Aguzzi etal. 2011a; Figures 1 and 2), which are difcult to quantify with isolated ocean
observatories. Accordingly, any technological development dedicated to ecosystem exploration,
monitoring, and ultimately management (sensu Danovaro etal. 2017) should be planned by combining
Lagrangian sampling strategies (i.e. capable of tracking individuals and population movements)
as well as Eulerian approaches (i.e. a ‘snapshot’ capable of characterising locally the community
changes produced by species displacements). For the former strategy, large-scale movements of
animals are being studied through telemetry via satellite (Hussey etal. 2015). Nevertheless, only a
few environmental parameters (e.g. depth and salinity) and no other ecological features (e.g. species
interactions) are measured as explanatory factors of behaviour. For the latter strategy, a virtually
holistic environmental monitoring approach is possible, but typically at a ne scale, which can be
difcult to scale up to larger systems (Aguzzi etal. 2019). Accordingly, a merger of both strategies
would be possible by the establishment of networks of monitoring stations that allow animal and
population tracking at a high rate in a simultaneous fashion across large geographic scales and across
latitudinal and depth gradients.
In this context, sheries scientists have recognised the need to move from single species to
ecosystem-based management approaches, but progress has been slow due to the complexity of
coenoclines and the difculty of obtaining synoptic data on appropriate scales (e.g. Marshall etal.
2018). Fishery management agencies can simultaneously advocate tracking and quantifying stocks
as a monitoring action required to inform management measures and implement no-take zones
(Maxwell etal. 2015). This point is crucial, as many essential sh habitats (EFHs, e.g. spawning or
nursery areas) are not permanent; thus, the establishment of shery restricted areas (FAO 2018) or
other spatial management measures for sh and habitat protection could follow an adaptive approach
(Walters 2007). Such a spatially dynamic approach will require different pathways for technological
development in species and ecosystem monitoring. Such an approach is currently being pursued in
the development of a cross-communication capability between cabled observatories and animal-
borne technologies (e.g. hydrophones for acoustic tag recognition; Hussey etal. 2015).
Marine strategic areas are dened as ecologically iconic zones where multiannual surveying, as
carried out by vessel-oriented technologies, is strongly recommended for scientic or management
purposes (Aguzzi etal. 2019). Data on species demographic indicators (e.g. density, size and biomass),
community composition (i.e. richness), and the effects of environmental controls on biodiversity
obtained in this way for one iconic zone could be scaled to other areas with similar geomorphologic
and oceanographic features as similar seascapes (Danovaro etal. 2017). Relevant areas have been and
continue to be instrumented with different types of pelagic and benthic multiparametric platforms
as part of observational networks (Tunnicliffe etal. 2003, Barnes etal. 2013), providing different
levels of monitoring capability and manipulative interventions (e.g. ONC 2019, OOI 2019). However,
such large networks could be improved by the development of the ecosystem observatory module
design, with its increased focus on obtaining temporally and spatially overlapping data from multiple
observation technologies.
Observatories are invasive technologies that produce noise, lighting, and motions that can be
foreign to the habitat under study. In addition, it is important that observatories be designed to better
understand their invasive impact to comply with international legislation (e.g. underwater noise
as an ecological descriptor; Audoly etal. 2016, 2017). Therefore, there is a need for observatories
to have built-in capabilities to monitor their own effect on the surrounding habitat and biota. The
84
RODNEY A. ROUNTREE ET AL.
EOM design seeks to provide self-monitoring capabilities for two reasons: 1) measurement bias and
2) degree of impact by the structures’ presence and functioning on the local environment (typical
sizes of the main components of observatory systems are likely to be around 3–5 m on each side and
2–4 m in height). Since any observatory will function as an articial reef and thereby modify the
local habitat characteristics that are being measured (Vardaro etal. 2007, Blanco etal. 2013), more
attention is needed to understand the attraction, repulsion, and residency effects of the structures
and their operations (e.g. pan-tilt camera motor noise, mobile platform noise, and illumination at
imaging) on sessile and motile species and their interactions with each other (e.g. the establishment
of fouling communities on the structure could inuence the local trophic structure). Over time, such
developments can result in enough changes that the observatory data will no longer reect the habitat
that it was designed to observe.
The ecosystem observatory module
A conceptual schematic of a proposed ecosystem observatory module and its components is provided
in Figure 3, and the function of each sensor and component device is outlined in Table 1. Standard
components of each module would include: 1) central node and associated instruments, 2) mobile
platforms, 3) three satellite pods, 4) a passive acoustic array, 5) a spatial conguration and software
to optimise cross-referencing among observational data, and 6) autonomous instruments. Optionally,
some modules would be enhanced with the addition of a pelagic satellite to collect data on sea-
surface and water-column organisms and conditions.
The central node and its instruments
The central node serves as the primary instrumentation platform, power supply, and data link for
the module. It also houses dockage, data transfer links, and power supply for three types of mobile
Figure 3 Schematic illustration of the proposed standard ecosystem observatory module consisting of
a central node, three satellites, AUV, ROV and crawler mobile platforms and their dockage, and various
autonomous devices. Hydrophones on the central node and each satellite form a 3-dimensional passive acoustic
array. Crawlers would operate on predetermined tracks lines to reduce their impact on the substrate.
85
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Table 1 Optoacoustic-image and passive acoustic sensors installed on the standard cabled
ecosystem observatory module and its associated mobile docked platforms
Components Instruments Purpose
Central node Multiple Module power supply, data deposition and transmission, instrument
platform, and mobile dockage platform.
Hydrophones Passive acoustic monitoring, including recording of environmental noise,
system noise, and biological sounds over the biologically relevant
frequency range of 1 Hz to 150 kHz. Component of the module’s
3-dimensional passive acoustic array for sound source location;
cross-reference with video, acoustic imaging sonar, and echosounder for
species identication, tracking, and target strength quantication.
Stereo-video cameras Video recording of conditions and organisms over 360° around the central
node to determine the size and spatial location of individual organisms.
Use for identifying or conrming the source of sounds, targets in the
acoustic image, and bioacoustic echosounder targets.
Pan-tilt HD cameras User-controlled video cameras with pan-tilt control, zoom capability, and
lighting control, for use in investigating selected eld of view areas,
infrastructure elements, and to zoom in on selected passive acoustic,
acoustic image, and echosounder targets for identication and
behavioural observations.
Acoustic imaging
sonar cameras
Recording the presence and movements of animals in a 360° cylindrical
volume surrounding the central node during all visibility conditions;
cross-reference with passive acoustic array source location, stereo camera
location, pan-tilt cameras, and echosounder targets for species
identication, tracking, and target strength quantication.
Rotary horizontal
multibeam
echosounder
Bioacoustic echosounder to quantify distribution of organisms in the water
column within a 360° zone surrounding the central node and extending
outward for a radius of 100–800 m. Cross-reference with passive acoustic
array source location, stereo camera localisation, and pan-tilt cameras,
for species identication, tracking, and target strength quantication.
Environmental sensor
package
Continuous recording of habitat variables, for example, pressure,
temperature, salinity, current speed and direction, methane, oxygen,
nitrates, pH, chlorophyll, and turbidity.
Acoustic and optic
receivers and
transponders
Acoustic receivers for animal- and instrument-borne telemetry signals.
Also including receivers for acoustic modem-based or optical
communication and data transmission. In some cases, transponders can
be used for two-way communication with animal- and instrument-borne
devices.
Crawler and dockage Placing and servicing autonomous devices and satellite experimental
payloads; conduct physical and biological sampling in the area
surrounding central node along xed and predetermined tracks.
ROV and dockage Central node servicing; place and service autonomous devices and satellite
experimental payloads, conduct physical and biological sampling in area
surrounding node, conduct video transect surveys, document fouling
organism and species associations with infrastructure, investigate
unknown targets detected by observation technologies.
AUV and dockage Conduct benthic habitat and biota distribution mapping transects around
the central node and throughout area between modules within an
observatory cluster. Investigate unknown echosounder targets beyond the
range of the crawler and ROV and of video and acoustic imaging sonar
ranges.
(Continued)
86
RODNEY A. ROUNTREE ET AL.
platforms (Figure 3, Table 1). Standard observation instruments on the central node would include
stereo video cameras, acoustic imaging sonar cameras (e.g. dual-frequency identication sonar:
DIDSON), and bioacoustic echosounders, as well as a passive acoustic system capable of recording
sounds over a biologically relevant bandwidth (1 Hz to 150 kHz). In order for these systems to
provide observations useful for ecosystem-level monitoring, they must provide spatially and
temporally quantiable data. For example, pan-tilt high-denition cameras that are often standard
on observatories are not conducive to the collection of occurrence data on even a presence/absence
level because the direction, depth, and angle of the eld of view are constantly changing and hence
the absence of organisms cannot be determined.
To achieve the desired quantication, the module should be designed so that each technology
provides 3-dimensional data over 360° around the module and overlaps with others to the
maximum degree possible (Figure 4). However, each device will have different ranges, beam
angles, and time resolutions which must be integrated to provide seamless views to the end-
user (see section on cyber developments subsequently). Comparison of data from the overlapping
3-dimensional views provides the ability to cross-reference data to improve identication and
measurement accuracy (Figure 4). Stereo video cameras should be used to obtain the 360° view
around the central node because they also provide 3-dimensional location and organism size
data (Bosch etal. 2019). Although we are not aware of previous stereo video camera applications
on existing observatories, they have been widely used in sheries and ecological applications,
including deep-sea applications (e.g. Harvey & Shortis 1998, Shortis etal. 2008, Williams etal.
2010, 2018, Bonin etal. 2011, Merritt etal. 2011, Shortis & Abdo 2016). It is important that these
devices not be under user control, because they must provide the maximum stability of views
over time (i.e. constant eld of view within the device’s limits). However, it is advisable for each
module’s central node to contain at least one pan-tilt video camera under user control to allow
the examination of specic phenomena (e.g. burrow emergence of different individuals or rate
of access to carrion) and to help validate the identication of organisms observed with the xed
video or other observation instrument.
Table 1 (Continued) Optoacoustic-image and passive acoustic sensors installed on the standard
cabled ecosystem observatory module and its associated mobile docked platforms
Components Instruments Purpose
Satellite pods Hydrophones Passive acoustic recording of ambient sounds (see previously).
Components of the module’s passive acoustic array for sound
localisation.
Stereo-video cameras 360° calibrated visual recording of organisms (see previously) around the
satellite and cross-reference with observational data from the central
node.
Pan-tilt video
cameras
User-controlled video cameras (see previously). Also, to supplement and
cross-reference observational data from the central node instruments.
Environmental
sensors
Record microdistribution of physical parameters (see previously) expected
to vary within the module area.
Experiment or
observation payload
Exchangeable ‘plug-and-play’ payload containing instruments for
user-designed data collection or experimentation, such as settlement trays
with different substrates (e.g. carbon, wood, or bones and even litter),
experimentation on light effect on species, tagging, and so on.
Autonomous
devices
Mission dependent Stand-alone sound recorders, cameras, cages, mesocosms, and other
devices to be placed by ROV or crawler to monitor short- and long-term
conditions at a specic location such as monitoring a sh nest or sessile
invertebrates. Other possible devices include animal collection traps and
stand-alone small-scale experimental packages.
87
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Mobile platforms
Three types of mobile platforms would be docked at the central node of each module, including a
seaoor ‘crawler’, neutrally buoyant remotely operated vehicle (ROV), and autonomous underwater
vehicle (AUV). Although there is some redundancy among ROV, AUV, and crawler platforms, each
provides unique capabilities and disadvantages. All three types of platforms are useful for surveying
habitat and organism distribution surrounding the module, and each can be used to investigate
Figure 4 Schematic 3-dimensional illustration (not to scale) of the spatial conguration of the central node and
satellites of a module and the overlap among video, acoustic imaging sonar, and echosounder spatial coverage
areas (inset provides a bird’s-eye-view of the spatial conguration). Integration of multiple 3-dimensional
observation modalities in a module provides cross-referencing data to identify and track organisms. (1–6) a
single individual of species A is tracked as it moves through the module area. Changes in echosounder target
strength due to orientation changes can be quantied by comparison of different observations at each location,
enhancing our ability to determine sh identication from target strength data. (1) Silent individual detected
by echosounder and acoustic imaging sonar, provides target strength, orientation, location, and size estimates.
(2) Same individual produces sound loud enough to be localised by the passive acoustic array, actual location
and identication conrmed by stereo-video, acoustic image, and echosounder. Sound received level can then
be corrected for exact location to obtain sound source level and detection range. (3) Now silent, it is detected
by stereo video, acoustic imaging sonar, and echosounder from the central node, as well as stereo-video from
a satellite. (4) The individual moves out of camera range but continues to be tracked by acoustic imaging sonar
and echosounder. (5) As the individual leaves the module area, it continues to be tracked by echosounder. (6)
Separate individual of species A is detected on echosounder only, but target strength consistency of species A
at other locations permits accurate attribution to species within the wider area covered by the echosounder. (7)
A second unknown species is detected by acoustic imaging sonar and echosounder and is identied based on
localisation on a known sound. (8) The reaction of multiple sh of species C to light is quantied by acoustic
imaging sonar and echosounder.
88
RODNEY A. ROUNTREE ET AL.
specic phenomena observed around the module and can aid in the identication of unknown targets
detected by the video, acoustic imaging sonar, echosounder, and passive acoustic array.
ROVs and AUVs are both mobile assets; the AUV has a much greater range and is not limited
by its tether. In contrast, although hampered in some ways by a tether, the ROV has manipulative
capabilities (i.e. by robotic arms), can carry larger payloads, and can be directly controlled by a user
in real time. The AUV provides the best mechanism for mapping and monitoring habitat and biota
(benthic and pelagic) of the area surrounding a module and the larger area encompassed by the
module’s satellites. In addition to providing habitat-mapping capabilities of the area immediately
surrounding the central node, the ROV can also be used to place autonomous instruments, exchange
satellite payload packages, and service all infrastructure components of the module (Sivčev etal.
2018). The ROV can also be equipped with push-corers in order to sample sediments. The ROV’s
high mobility also allows important functions such as the monitoring of the fouling community,
interactions of organisms with the infrastructure and its instruments, and faunal residence (e.g.
sheltering).
A drawback of both AUVs and ROVs is that thrusters must be in operation even when hovering
at a station, thus creating high levels of noise and turbulence that limit their ability to conduct
unbiased sampling and observations at a specic location for any period of time (Rountree & Juanes
2010; Durden etal. 2016a). An important, but often overlooked, noise problem with ROVs is that
their acoustic tracking and guidance systems produce intense broadband noise that may inuence
animal behaviour and can also bias measurements of the acoustic properties of biological sounds
(Rountree & Juanes 2010). In addition, the intense tracking pings make it harder for a human user
to process soundscape data (R.A. Rountree pers. obs.). Another drawback of ROVs is the need for
lights for operations (Rountree & Juanes 2010). The main limitations of AUVs are related to the
development of suitable docking infrastructures that can provide for data downloading and fast
inductive recharging of batteries to increase AUV operating time.
A crawler can more effectively conduct point-census surveys that can provide data at specic
locations for extended time periods (minutes to hours), during which noise production and turbulence
can be substantially reduced compared with the other mobile platforms. A drawback to the crawler is
its physical disturbance of the benthic habitat and impact on benthic organisms along its movement
track, but this can be reduced to a narrow strip of seabed by limiting the crawler to a constant
corridor for displacement (Chatzievangelou etal. 2016). Since such potential impacts would be
magnied in the area around a permanent observatory, we recommend that crawlers be operated on
predetermined and constant tracks to minimise habitat disturbance (Figure 3).
Tethered satellite pods
The three standardised satellites of each module would have several functions: 1) provide
observational redundancy and spatial overlap of observations with the central node observations to
assist in organism detection, identication, and development of 3-dimensional distribution maps in
the area surrounding the module (Figure 4), 2) provide observation of biotic responses to the central
node and its mobile platform presence and operations; and 3) serve as platforms for changeable
instrument packages designed to address specic research hypotheses.
A central premise of the proposed ecosystem observatory module design is that it includes
multiple modalities of observation that are synchronised in time and provide the maximum spatial
overlap. Therefore, the spatial conguration is dependent on optimising the overlap among the systems
under local conditions, as well as limitations of tethering with regard to ROV and crawler access to
the satellites. In many locations, satellites placed at 120° intervals and at distances on the order of
10 m from the central node would be most suitable (Figure 4). Minimally, each satellite would be
equipped with stereo video cameras capable of capturing a 360° cylindrical or hemispherical view
around the satellite. Ideally, they would also include the same acoustic imaging sonar and bioacoustic
89
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
echosounder instruments as those on the central node, but at the present time, these systems are
prohibitively costly to achieve the ideal redundancy and overlap within the module area. As these
technologies advance sufciently to allow cost-effective 360° coverage, they should be added to the
satellites to improve spatial overlap over a larger area surrounding the central node.
Observational data obtained by the satellites of the area surrounding the central node and by the
central node of the area surrounding each satellite would provide a powerful means of determining
faunal interactions with the structures, including behavioural reactions to instrument operations (e.g.
lights and sounds, Figure 4).
Ocean observatories should be thought of as permanently instrumented areas where scientists of
different backgrounds have an opportunity to perform manipulative experiments, favouring iconic
environments such as the deep sea, for example, and resulting in a transition from a still largely
descriptive science toward a more experimental, hypothesis-driven, approach. In order to better serve
as platforms for hypothesis-driven research objectives, the satellites need to be designed with an
infrastructure that allows for ‘slide-in slide-out’ exchange of experimental payloads for hook-up to
power and data transfer. Examples of potential payloads might include settlement trays, experiments
on the response of biota to articial light regimes (useful for behaviour studies but also to examine
the impact of observatory lights), observation of biota response to bioluminescent light, response
to various baits, response to sound playback experiments (useful to understand behaviour and also
the impact of observatory generated noise on the biota), experimental attempts to mark or tag biota
through ingestion of tags or automatic capture, tag and release mechanisms (having the dual purpose
of studying sh movements and residency and using the observatory structure as habitat), the effects
of new colonised substrates on species and succession experiments, habitat manipulation experiments
such as predator exclusions, microcosm and mesocosm experiments, and many other possibilities.
Passive acoustic array
Passive acoustic monitoring of shes and invertebrates has become an important tool in sheries and
ecosystem studies (Rountree etal. 2006, Luczkovich etal. 2008); however, inherent problems have
slowed its more widespread application, including lack of catalogues of sh sound data (Rountree
etal. 2002), lack of information on source levels and detection ranges, and lack of sufciently
developed autodetection software (Rountree etal. 2006, Luczkovich etal. 2008). The use of multiple
observation technologies to aid in the in situ validation of sound source identity, source level, and
detection ranges is in its infancy (Rountree etal. 2003, Rountree 2008, Rountree & Juanes 2010), but
a combination of using a passive acoustic array with video for the in situ identication of unknown
sh sounds has recently been demonstrated (Mouy etal. 2018). The application of passive acoustic
arrays for localisation and cross-reference with other forms of observation on ocean observatories
are particularly promising, especially in the deep sea where many shes possess sonic muscles
that are presumably used for sound production (Rountree et al. 2012, Wall et al. 2013). Calls for
the increased use of passive acoustics for shes and invertebrates to be incorporated into ocean
observing systems have been made at workshops for decades (Rountree etal. 2003, ACT 2007,
R.A. Rountree pers. obs.), but implementation has been slow (Locascio etal. 2018). It should be
emphasised that incidental sounds produced by shes and invertebrates as by-products of movement,
feeding, or physiological processes can be important markers of species identity and useful for
monitoring temporal and spatial patterns in the associated behaviour (Rountree etal. 2006, 2018).
Thus, passive acoustics can be a useful tool for monitoring both vocal and non-vocal organisms and
their behaviours at observatories.
Because of the potential importance of passive acoustic monitoring as an important tool in ocean
observatories, it is essential to include a hydrophone array in the EOM design. At the minimum,
hydrophones should be placed on the central node and each satellite to create a four-element
3-dimensional array that can localise sounds originating near the central node. However, a greatly
90
RODNEY A. ROUNTREE ET AL.
improved ability to localise the low-amplitude sounds created by many shes and invertebrates
could be achieved by placing compact arrays of six hydrophones on each element (sensu Mouy etal.
2018) or by placing additional hydrophones at intervals along the tethers from the central node to
each satellite (Figure 3).
Stand-alone sensors and other devices
Autonomous instruments and recording devices (e.g. Corgnati et al. 2016, Marini et al. 2018a)
deployed and serviced by the mobile platforms would be incorporated into the area surrounding the
module to provide unique data on biota in the surrounding habitat and additional opportunities for
in situ experimentation (Figure 3). For example, autonomous video recorders could be placed close
enough to individual sh nest sites, or individual sessile invertebrates, to use short-range infra-red
lighting to make long-term observations on microhabitat use, behaviour, and species associations.
Autonomous instruments could also be used to measure gradients in conditions at increasing
distances from the central node or specic satellites in an effort to quantify the effects of habitat
heterogeneity on animal presence and habitat use and the observatory’s inuence on environmental
conditions, habitat structure, and organism distribution (i.e. to distinguish between natural variation
and artefacts resulting from effects of the module). Many other types of autonomous devices can be
envisioned to carry out hypothesis-driven experiments, such as small mesocosms, settlement trays,
exclusion cages, and benthic animal traps.
Importance of observation data overlap
Time synchronisation and spatial overlap of all observation data, within the resolution limits of each
type of instrument, within a standardised spatial conguration, is one of the most important attributes
of the proposed ecosystem observatory module design, as it allows for the cross-referencing needed
for species detection, identication, and tracking (Figure 4). Consideration of how best to optimise
the spatial coverage and overlap of observation data and how it can be packaged for users should be
part of the design process for implementation of the EOM concept.
Ideal spacing between the central node and satellites is determined by optimising overlap
among spatial coverage of all instruments for local conditions. Stereo video cameras provide the
highest accuracy of species identication, size, and location in the area surrounding the module
but are limited to periods of natural or articial lighting during hours of darkness or in the aphotic
zone. Acoustic imaging sonar provides accurate location of targets but poorer species identication
(Aguzzi etal. 2019). However, it is not limited by lighting. Horizontal multibeam echosounders
provide highly accurate 3-dimensional location over a large spatial area surrounding the module, but
identication is limited by the accuracy of back-scatter target strength data, which are inuenced by
sh size and orientation to the acoustic beam, creating uncertainty in multispecies scenarios (Juanes
2018). Sounds detected by the passive acoustic array can be used to identify species when sounds
are well known, but until detailed catalogues of sh and invertebrate sounds become available, most
sounds detected and localised will be from unknown sources.
Cross-referencing of echosounder data with acoustic imaging sonar, video, and passive acoustic
data can provide valuable validation of target strength data for organisms and thereby enhance
biomass estimations around the observatory, as well as providing target strength data for other
independent conventional bioacoustics surveys (e.g. traditional sheries pelagic surveys that rely on
accurate target strength data for bioacoustic assessment of sh stocks). Similarly, cross-referencing of
acoustic imaging sonar with echosounder, video, and passive acoustic data can provide identication
validation of acoustic image targets in the near eld (ranges of up to the limit of visibility) and
echosounder targets in the far eld (ranges up to 1000 m). Finally, cross-references of unknown
sounds localised by the passive acoustic array with video, acoustic imaging sonar, and echosounder
91
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
data can provide sound source identication and quantication of source level and detection range
(Rountree 2008, Mouy etal. 2018).
Simultaneous observations from all technologies would make it possible to track organisms
continuously as they move around the module (Figure 4). Therefore, at each location, data on changes
in sh orientation and location can be used to quantify their inuence on echosounder target strength.
As more and more data are compiled, accuracy of identication and tracking and estimates of
sh size, abundance, density, sound source level, and sound detection range can be improved. A
360° view around the observatory by multiple observation technologies allows users to estimate
the abundance of biota per unit volume while correcting for movements of individual sh and
other organisms. A sh swimming in circles around the structure can be counted accurately as one
individual, rather than multiple individuals moving in and out of a video eld of view. To obtain
these type of data, modules must be congured with satellites in close enough proximity to provide
adequate coverage of mobile biota (Figure 4).
Optional pelagic satellite
The ecological monitoring of modules can be signicantly enhanced by the addition of surface
and water-column assets that can combine benthic observations with water-column and surface
observations to monitor both surface-associated organisms and conditions as well as those of the
water column (Figure 5). Besides providing a monitoring capability of the pelagic ecosystems,
ecosystem observatory modules enhanced with a pelagic satellite can provide unprecedented
information on pelagic-benthic ecosystem connectivity. This can be accomplished by placing a
buoyed surface platform in contact with a module via an instrumented mooring line. Surface buoys
and mooring lines have the potential capacity for numerous instruments to be distributed throughout
the water column to synoptically monitor ne-scale hydrographic and biogeochemical parameters as,
for example, corrosive (i.e. low pH, high pCO2) oxygen minimum zone waters that intrude seasonally
onto continental shelf-edge zones (Juniper etal. 2016). Instruments can be either xed (e.g. Bahamon
etal. 2011) or movable as yo-yo systems for sh monitoring from decommissioned platforms (Fujii
& Jamieson 2016). They can also serve as access platforms to allow some types of maintenance of
observatories placed on the seabed (depending on depth and conditions).
Each buoy would be tted with a weather station, microphone, and video camera to monitor
surface conditions and shipping activity (e.g. Aguzzi etal. 2011b, OBSEA 2019). Recordings of aerial
noises associated with weather, sea state, and shipping can be validated by the video and compared
with simultaneous acoustic recordings from hydrophones to provide important insight into the source
of underwater sounds and help to quantify noise impacts on the aquatic soundscape. The surface
buoy would also support downward-projecting video, acoustic imaging sonar, and echosounder
instruments to provide similar capabilities to those of the bottom mounted instruments and hence
valuable data on pelagic components of the ecosystem. All instruments would be connected to the
cabled observatory for data transmission and power supply, with no need for satellite communication.
The development of a new cargo elevator technology (Figure 5) would allow the rapid delivery
and retrieval of instruments and materials to and from the module. For example, in combination
with the module’s ROV or crawler, scientists could deliver a new experimental payload to one of the
satellites and remove the old unit. Another example would be to deliver fresh bait to a baited camera
system or to retrieve organisms captured by instruments at the module. An elevator system could
dramatically increase our ability to deploy and retrieve materials to the module because it would no
longer depend solely on the use of expensive ship-based submersibles or ROV bottom time.
Pelagic satellites can also be used as docking and communication stations for specially adapted
aerial drones (Figure 5). One of the most important applications of drones would be to map spatial
and temporal distributions of marine birds, mammals, turtles, and large pelagic shes (e.g. Toonen
& Bush 2018). Pleustonic and neustonic components of the ecosystem could also be mapped,
92
RODNEY A. ROUNTREE ET AL.
including distributions of jellysh, Sargassum, and other surface organisms and the development of
windrows. In addition, they can map the distribution of organic matter subsidies, including kelp and
marine mammal carcasses, and also track pollution, such as oating plastics, oil slicks, and other
buoyant pollutants. This can also be crucial to monitor alien species and forecast potential areas of
invasion, as plastic debris and other oating materials contribute to the transfer of non-native species
(Vetger etal. 2014). Drone systems are already being successfully developed to conduct passive
acoustic surveys (Lloyd etal. 2017). A communication tower on the buoy would enable researchers
to communicate with the drones through a relay from the cabled observatory and also provide short-
range communication with research ships and aircraft.
Satellite remote sensing has become an important tool in oceanography and sheries monitoring
(e.g. Santos 2000, Blondeau-Patissier etal. 2014), but ground-truthing of data is critical for accurate
Figure 5 Schematic of an ecosystem observatory module enhanced with a pelagic satellite composed of a
surface buoy and associated instruments to monitor vertical distribution of organisms and physical properties.
The surface buoy would be equipped with downward-looking video ca mera, acoustic imaging sonar system, and
echosounder similar to those deployed on the benthic module components. It would also include a microphone
and 360° video to capture above-water audio and video data of weather and shipping conditions for correlation
with underwater recordings. The mooring line would be variously equipped with monitoring instruments at
different depths and a cargo elevator system to transport materials, such as new scientic payloads for satellite
nodes between benthic and surface systems. The pelagic satellite includes a drone system to map aerial (e.g.
birds) and aquatic megafauna (mammals, sh, turtles), as well as neustonic and pleustonic organisms and
pollutants. Drones can also be used to carry an instrument payload such as a hydrophone or uorometer and
other instrumentation for spatial mapping.
93
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
interpretation and modelling (Congalton 1991). The Southeast Atlantic Coastal Ocean Observing
System (SEACOOS) included a pilot study of the potential for integration of satellite remote sensing
and ocean observation systems (Nelson & Weisberg 2008), which found that coordination among
data providers, management, modellers, and users was a critical bottleneck. Field validation efforts
are important but expensive and difcult to coordinate. Observatory-based drone sampling can also
be used to enhance satellite remote sensing programs by conducting some types of coordinated eld
validation sampling. Ecosystem observatory-based drones could provide a more cost-effective tool
for obtaining oceanographic data for a wide range of measurements from sea surface temperature to
primary production in order to tune satellite data interpretation and modelling. Some drones could
be equipped with a payload of specialised equipment for specic projects, such as a chlorophyll
uorometer, or for deployment of sonobuoys, drifters, and expandable vertical prolers. Thus,
integration of ocean observatories with remote sensing satellite systems can improve the accuracy
of spatial mapping of large-scale environmental conditions.
Ecosystem observatory module clusters and networks
To be able to provide meaningful ecological data at different spatial scales (i.e. from local conditions
to geographic areas) accounting for key factors such as habitat heterogeneity along a coenocline
(e.g. Rex & Etter 2010, Lecours etal. 2015, Zeppilli etal. 2016), local modules should be associated
into a spatial hierarchy of clusters and networks, called ecosystem observatory module clusters
and ecosystem observatory module networks. Adopting a highly reproducible module design for
observatories should reduce costs and allow for replication of data at different locations.
The spatial conguration of modules within clusters and clusters within networks is critical to
providing spatial and temporal overlap among the various observation technologies required for
cross-referencing and validation. Experiments are needed to determine the optimal conguration
under local conditions. In these experiments, a minimum of three modules within clusters and three
clusters within networks are needed to ensure at least minimal coverage and overlap. A cluster design
of three modules separated on the order of hundreds of metres would be an effective way to scale
up data collection from individual sites to habitat (Figure 6). At distances of hundreds of metres,
bioacoustics coverage among the modules in a cluster would overlap to provide the ability to estimate
water-column biota density in a homogenous fashion over a large area (0.5–1 km2 or more; Figure6)
and to quantify the effect of module structure and operations on biota occurrence and behaviour.
Cross-reference data from each module would greatly improve the accuracy of the identication and
density estimation of biota within the cluster area, but well outside of individual modules, and allow
for detailed benthic habitat mapping over the larger area encompassed by the cluster. Such coverage
would facilitate accurate faunal abundance and density estimates necessary for sheries and other
applications and reduce observatory bias on measurements due to attraction and avoidance responses
of organisms to the observatory structures.
Finally, advanced AUV capabilities would enable the AUV to be used to map habitat and benthic
biota distributions between and among modules within the cluster. In some scenarios, all modules
within a cluster might share one AUV that patrols among them and can dock at any module. In
other scenarios, AUVs provided by each module would provide the cluster with multiple AUVs for
more rapid and detailed mapping. Observational data obtained from the AUV tracks can further
increase our ability to validate the identity of bioacoustic and passive acoustic targets outside of
the modules but within the cluster area. In some cases, AUVs might be programmed to investigate
passive acoustic or echosounder targets beyond the range of the other observational instruments
within a cluster area to improve identication and density estimates. Where feasible, an observatory
cluster would include one module equipped with a pelagic satellite that could provide drone support
for the entire cluster to enhance studies of vertical connectivity from the surface to the benthos at
the cluster location.
94
RODNEY A. ROUNTREE ET AL.
Sentinel system
Observatory systems combining multiple EOM clusters along a coenocline form a ‘sentinel system’
observatory network (Figure 7). A minimum of three EOM clusters (i.e. nine modules arranged
in a spatial hierarchy) would be needed to elevate the monitoring network from examination of
local habitats to ecosystems and large geographic regions (Figure 7). It should be clear that such
a sentinel system would ideally be one component of a larger monitoring effort that coordinates
data from conventional ship, satellite, buoy-based, and animal-borne survey programmes. For
example, establishing a sentinel system composed of clusters (each of which provides high-resolution
monitoring on a scale of 0.5–1 km2) in the upper and lower sections of a major estuary (e.g. the
Chesapeake Bay) and another on the continental shelf just offshore would be effective at monitoring
movements of coastal shes that utilise the estuary as seasonal feeding or nursery grounds. Similarly,
deployment along coastlines can provide information on the timing of seasonal movements of shes
and habitat connectivity along migration corridors. Sentinel systems would be useful to monitor
migration patterns of shes and invertebrates by documenting rst detection, last detection, and
residence period at different points along the gradient. Such a system would also be useful for
monitoring the invasion of organisms into new territories (Juanes 2018) by placing clusters along
the predicted invasion pathway.
Figure 6 Schematic illustration of an ecosystem observatory module cluster designed to provide synoptic
data on differing spatial scales within the cluster area. Three or more modules should be arranged in geometric
clusters to allow detailed spatial comparisons within a larger spatial array. Spacing between modules is
dependent on local conditions, AUV range, and optimal echosounder coverage. Clusters with module spacing
allowing for overlap among bioacoustics echosounders, with greatest overlap in the centre of the cluster, enable
highly accurate identication of water column organisms over a large spatial area. One or more AUVs would
be designed to navigate among modules in the cluster to map habitat and organism distributions within the
cluster area and provide additional ground-truth data for organism identication based on their target strength.
Demersal and benthic organism and habitat mapping resolution is greatest around the modules but is also high
within the wider area encompassed by the observatory module cluster.
95
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Ecosystem surveillance, modelling, and forecasting
Fixed and mobile platforms allow for an experimental approach to the study and monitoring
of ecosystem functioning at different spatiotemporal scales (over kilometres and years). The
combination of stereo video, acoustic imaging, and echosounder imaging provides the ability to
quantify abundance, size, and biomass of organisms over a wide size range, as well as to identify
multiple types of behavioural reactions to natural or articial stimuli. In addition, the simultaneous
acquisition of biochemical and oceanographic data can inform researchers of potential causative
factors for observed behaviour and abundance patterns. However, automatic processing of the
high volumes of data generated by the observatories would be essential. Automated detection and
classication methodologies based on the various observation technologies are rapidly advancing
(e.g. Allken etal. 2018, Juanes 2018, Marini etal. 2018b). However, we suggest that the concept of
an ecosystem observatory user data interface would greatly enhance the application, testing, and
quality control of detection algorithms by providing a simple computer interface for user-aided
system learning (see ‘Cyber developments in support of monitoring networks’ section subsequently;
Figure 8).
Ecosystem observatory networks can be used to estimate local species abundances derived from
the image-based identication and counting of individuals, made possible through integration of
multiple observation technologies (see Figures 4 and 8). In addition, the methodology provides an
ability to develop size-class frequency data and species biomass estimation based on the estimated
size and counts of individuals (Durden etal. 2016b). Cross-referencing of data from ROV, crawler,
AUV, and echosounder data with validation data from each module provides the ability to obtain
standardised abundance and biomass data for the entire observatory network area (Figures 4, 6
and 7). Simultaneous monitoring of a large suite of environmental factors such as temperature,
turbidity, chlorophyll concentration, and other biochemical factors, together with ne-scale temporal
and spatial distribution patterns of organisms, would provide important data on environmental
regulators of species population structure and behavioural patterns. Temporal patterns in species
Figure 7 Sentinel ecosystem observatory networks (not to scale) composed of multiple module clusters
distributed across a habitat gradient or coenocline occurring from estuarine/riverine areas to coastal zones and
the shelf, down to the deep-continental margin of the slope and abyssal plain.
96
RODNEY A. ROUNTREE ET AL.
richness, abundance, biomass, size-class structure, and role of environmental regulators within an
observatory network, supplemented with data from other monitoring programmes, could provide the
raw data needed to develop ecosystem modelling and forecasting programs for the habitat or region
surrounding the network.
Data from multiple observatory networks could then be linked to make comparisons among
areas, populations, and environmental regulatory factors to develop regional and ultimately global
monitoring programmes. Spatially representative and long-term monitoring provides the ability
to distinguish between population/community regulation by repetitive phenomena (e.g. rhythmic
abundance variations due to seasonal environmental changes and ontogenetic migrations, spawning
Figure 8 Hypothetical user interface of data from an ecosystem observatory module, composed of the central
node plus the three satellites. All data windows (A–E) play in time simultaneously, as indicated by the time
cursor in the scientic data and passive acoustic sound windows (D–E). Video, acoustic image, and echosounder
displays show only the portion of the 360° area surrounding the module which has been selected by the user,
with the angular view selection bar common to all three (below C). However, when the playback is paused,
the user can simultaneously scroll through all 360° surrounding the module in the video, acoustic image,
and echosounder windows (A–C). The overlapping observation modalities and integrated visual displays are
a powerful tool for examining correspondences among environmental conditions, observatory operations,
and animal behaviour. When autodetection is available for one or more of the observation technologies, the
user can validate detections in other windows, for example, targets ‘a–d’ are detected in video (A), acoustic
image (B), and echosounder data (C). Data from each instrument can then be compiled to provide the most
accurate information on species identication, 3-dimensional location, size, and target strength together with
environmental conditions at the time of detection. In addition, sound source targets localised by the passive
acoustic array and shown in the sound window (sound labelled ‘d’ in E) can be identied by its corresponding
location in the other windows (A–C).
97
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
migrations; Aguzzi & Company 2010, Aguzzi etal. 2011a) from long-term (decadal and longer)
processes such as shifts in species distributions due to climate change and changes in resource
exploitation. If the information obtained from the observatory network system and associated
modelling and forecasting programs is automated, it may be possible to develop ecosystem alarm
protocols that detect anomalies in ecosystem parameters that might signal undesired environmental
states such as impending population collapse of keystone species.
Integration of benthopelagic networks
with animal-borne technologies
Cross-connection of ecosystem observatory module networks with free-moving animal-borne sensor
(ABS) technologies can also be envisaged. Inclusion of technology into the module design that
allows communication with independent ABS (Figure 3) is particularly promising for obtaining
data on animal behaviour as well as data from animal-borne environmental monitoring programmes
(see, for example, the Animal Telemetry Network Implementation Plan 2016–2021, NOC 2016).
Presently, data loggers connected to animals are getting ever more miniaturised (e.g. Nassar etal.
2018) and still primarily store oceanographic information about travelled seascapes (Wilmer etal.
2015, Fehlmann & King 2016) but only limited ecological information on intra- and interspecic
interactions experienced by the traveller. This weakness is being corrected in part by the development
of animal-borne cameras. Animal-borne video collection directly allows the derivation of ecological
information based on what is seen by individuals during their displacements (Moll etal. 2007).
Moreover, the progressive miniaturisation of implant components will eventually allow camera
installation on animals of very different sizes (although lming may be constrained at night or in
deep water).
If both the observatory module and animal-borne technologies are capable of two-way
communication, then data-intensive video-sampling by animal-borne technologies can be enhanced
by dumping data to the observatory, thereby freeing up data storage and increasing their useful
lifespan. Similarly, modules can be tuned to receive telemetric data from tagged animals freely
moving across depths and basins (Hussey etal. 2015). This cross-communication can complement
the monitoring capability of already existing pelagic and coastal-shallow networks (e.g. OTN 2019).
Presently, for the development of technological tracking of epibenthic animals carrying an acoustic
emitter, displacements can be measured into a network of moored receiving hydrophone stations
(Rotllant etal. 2014, Tuck etal. 2015). Such development is necessarily limited by the range of
hydrophone detection capabilities and could be potentially expanded when animal tracking is
assisted by moving platforms, delivering real-time data on their positioning. Tracking expansion
is presently pursued by using wave-gliders and AUVs (e.g. Lin etal. 2016, Masmitja etal. 2017).
Cyber developments in support of monitoring networks
Networks of xed and mobile units for coordinated ecological monitoring require not only hardware
development but a concomitant suitable cyber architecture for data communication, processing,
storage, and visualisation of interrelated multidisciplinary data of different types (Florea & Buiu
2017). Moreover, cyber infrastructures should provide proper ‘virtual research environments’ (VREs),
which can be described as online collaborative environments that allow open access and program
development for best science practices (Martin etal. 2019, Morris etal. 2019, Pearlman etal. 2019).
These VREs should be built on top of interrelated multiparametric data access platforms similar to
those developed for the Ocean Networks Canada Web services application program interface (API)
and Sandbox tool set (Rempel & Cabrera 2018). It is critical that such VREs serve as libraries of
multiparametric data (e.g. imaging, acoustics, physical, biochemical) derived from the observatories,
as well as open-source automated classication and statistical analysis programs.
98
RODNEY A. ROUNTREE ET AL.
As ecology researchers increasingly deploy embedded sensor networks, they are being
confronted with an array of challenges in capturing, organising, and managing large amounts of
data (Borgman etal. 2007). User navigation into network data banks and analysis capability requires
the design of efcient interfaces between people and computers. Such a design should include all
steps of information ow, from data collection at each sensor and platform to its global elaboration.
This type of information ow framework is well described by ecoinformatics (Michener & Jones
2012), which arose from the need to integrate environmental and information sciences to provide
the language tools and standardisation practices necessary to access and analyse massive amounts
of heterogeneous data (e.g. by developing data banking).
Data integration would include several disciplines related to information technology that allow control
of data collection, processing, integration, and use in VRE systems by multiple sensor technologies. The
sensor web enablement (SWE) approach dened by the Open Geospatial Consortium (OGC) standards
(Del Río etal. 2018, Chaturvedi & Kolbe 2019) is a low-level specication of functionalities that allow
any kind of compliant sensor to interact with other sensors, with human users, or with properly dened
intelligent services. Networks of SWE-compliant sensors allow for a remote interaction by simply
triggering them on and off or by changing their acquisition conguration in order to adapt the monitoring
activities for specic purposes. The intelligent services capable of interacting with the SWE-compliant
sensors are generally dened according to the Internet of Things (IoT) technology paradigm (Qin etal.
2016, Čolaković & Hadžialić 2018), which refers to the capability of making content and services
understandable by devices without human involvement. To achieve this goal within the marine science
and technology community, data science methodologies (Skiena 2017) based on articial intelligence
should be capable of extracting the relevant content from the acquired data, then using this content
for interacting with the SWE-compliant observatory or for populating appropriate data repositories
(e.g. the Copernicus or the SeaDataNet initiatives). For example, data acquired by SWE sensors and
managed by intelligent services could be of the biophony (sounds of known shes, cetaceans, birds,
unknown biological sounds, etc.), the geophony (natural sounds like wind, rain, thunder, waves, etc.),
and the anthropophony (noise from ships, seismic surveys, and the observatory itself), which would then
be utilised by sound type classication software to document spatial and temporal patterns in sound
occurrence and correlations between biophony and anthrophony to assess noise impacts. SWE sensors
could similarly be used for biogeochemical data or visual data acquired by stand-alone devices capable
of communicating the relevant acquired information (Marini etal. 2018a).
Since all marine monitoring networks are increasingly service- and end-user oriented, their
data management cyber infrastructures are also being upgraded to retrieve, store, and process data
in real time, acting as a cognitive system for data interpretation for humankind (Shenoi etal. 2015).
Systems should enable any end-user worldwide to investigate ecological processes via interactive
web interfaces, allowing navigation into banks of multiparametric ‘big’ biological and environmental
data (Figure 8). Responses should be visualised in the form of synthetic graphic outputs, highlighting
signicant global trends and cause–effect relationships. Such visualisation would be based on high-
level data science activities performed within VRE capable of allowing non-expert users to compose
complex workows based on tools with high technological and scientic content (Buck etal. 2019).
Data output could be based on automated time series analysis (Aguzzi etal. 2012, Skiena 2017,
Recknagel & Michener 2018) as well as on multivariate statistics, which would then allow modelling
of biological responses to key environmental variables. The use of such powerful software tools
on big biological and environmental data will transition ocean observatory systems from a largely
observational to a more quantitative monitoring platform for ecological and sheries applications.
Data ow management from multiple observation technologies
It is critical that data streams from all the observation instruments and sensors be synchronised and
maintained as relationally integrated data that are interoperable with other observation networks
99
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
(e.g. the ONC’s Oceans 2.0 program; ONC 2019). Data should be enriched with the appropriate
semantic information that allows their retrieval by semantic-based search engines (Aguzzi et al.
2015a). A user annotating events in one dataset should be able to seamlessly populate the same
annotation in all other data streams (Figure 8). For example, a user marking the location of a sound
in the hydrophone recording should be able to locate the corresponding data position automatically
in video, acoustic image, echosounder, and environmental datasets (e.g. ‘d’ in Figure 8). Although
observatories currently provide metadata containing information on observatory instrumentation
functioning performance, maintenance status and functioning history, data quality assurance and
control, calibration, and other aspects (e.g. ONC 2019), this may not be sufcient for end-users
who are not capable of cross-referencing all this information automatically, because it must rst be
downloaded and integrated by the users themselves.
A user interface that provides all module data integrated together in an interactive visual
display would be a powerful tool for researchers (Figure 8). For example, a user viewing a video
would immediately see not only environmental and other observational data but also the activity
state of all instrumentation (e.g. lights on, rotary motor active, ADCP active, ROV thrusters on or
off). Comparison of data from the overlapping 3-dimensional views in video, acoustic image, and
echosounder windows provides the ability to cross-reference data to improve identication and
measurement accuracy. For example, if a video detector identies targets ‘a’ to ‘d’, its ‘ghost’ target
can be displayed in the acoustic image and echosounder windows to look for matches or to compare
with automatic detections in those datasets. That will help a user determine if some detections are
valid or to identify unknown detection targets. The user could then download a dataset containing
all the attributes of the target based on the different observation types as well as corresponding
environmental and operational state data. Such information can provide valuable clues to understand
species response to the observatory and potential biases in behavioural observations, in addition
to providing data on biotic responses to environmental conditions and the raw data necessary to
compile species abundance and volume density maps. The ability to download data seamlessly in
these kinds of relational datasets is of the utmost importance to encouraging widespread utilisation
of observatory data among scientists, resource managers, and educators.
Observatory integration within
commercial development projects
Scientists around the world struggle to obtain funding for even small observatory systems. The cost
of observatory infrastructure, such as the platform and dedicated data/power transmission cables
to shore, often constitutes the largest expense and greatly limits observatory capabilities. Offshore
development projects (e.g. telecommunication cables, wind farms, tidal/current turbines, and oil/
gas platforms) provide a unique opportunity for advancement of ocean science if government
and industry leaders have the foresight to integrate ocean observatory systems into offshore
development design (e.g. Danovaro etal. 2017). It is hoped that current large scientic actions
are being conceived at higher institutional levels to combine the two visions and design offshore
energy systems that can provide both much-needed renewable energy and also much-needed ocean
observatory systems (e.g. see DELOS and LoVe initiatives as reviewed by Aguzzi etal. 2019).
Offshore energy development can provide platforms for many EOMs at a location and thus the
ability to construct large EOM cluster networks capable of delivering an unprecedented view of
underwater life to scientists, shers, and the public. Further, because power and data cables are a
necessary part of the energy delivery system, scientists could have a fully functional data transfer
network to shore already in place. If commercial industries incorporated observatory systems into
project design from the beginning, rather than post-construction, and consider the advantages of
an improved public image, the cost of required environmental monitoring and mitigation would
likely be more palatable.
100
RODNEY A. ROUNTREE ET AL.
Acknowledgements
This work was developed within the framework of the Tecnoterra (ICM-CSIC/UPC) and the
following project activities: ARIM (Autonomous Robotic sea-oor Infrastructure for benthopelagic
Monitoring; MartTERA ERA-Net Cofound) and RESBIO (TEC2017-87861-R; Ministerio de
Ciencia, Innovación y Universidades). Inspiration for our modular observatory design and some
of its components resulted from discussions of participants at an international workshop on ocean
observation technology hosted by Oceans Network Canada, in Barcelona, Spain on October 4–5,
2018. A special thanks is also devoted to Dr T. Fujii, OceanLab University of Aberdeen for his
helpful comments during the manuscript preparation.
References
ACT (Alliance for Coastal Technologies). 2007. Underwater Passive Acoustic Monitoring for Remote Regions.
A Workshop of Research Scientists, Technology Developers, and Resource Managers. Hawaii Institute
of Marine Biology, Coconut Island, Hawaii, Feb. 7–9, 2007. Alliance for Coastal Technologies Ref. No.
ACT-07-02. Solomons, Maryland: Alliance for Coastal Technologies.
Afonso, P., McGinty, N., Graça, G., Fontes, J., Inácio, M., Totland, A. & Menezes, G. 2014. Vertical migrations
of a deep-sea sh and its prey. PLOS ONE 9(5), e9 788 4.
Aguzzi, J., Chatzievangelou, D., Marini, S. et al. 2019. New high-tech interactive and exible networks
for the future monitoring of deep-sea ecosystems. Environmental Science and Technology 53,
6616 –6631.
Aguzzi, J. & Company, J.B. 2010. Chronobiology of deep-water decapod crustaceans on continental margins.
Advances in Marine Biology 58, 155 –225.
Aguzzi, J., Company, J.B., Costa, C. etal. 2012. Challenges to assessment of benthic populat ions and biodiversity
as a result of rhythmic behaviour: Video solutions from cabled observatories. Oceanography and Marine
Biology: An Annual Review 50, 235–286.
Aguzzi, J., Company, J.B., Costa, C., Menesatti, P., Garcia, J.A., Bahamon, N., Puig, P. & Sardà, F. 2011a.
Activity rhythms in the deep-sea: A chronobiological approach. Frontiers in Bioscience 16, 131–150 .
Aguzzi, J., Doya, C., Tecchio, S. etal. 2015a. Coastal observatories for monitoring of sh behaviour and their
responses to environmental changes. Reviews in Fish Biology and Fisheries 25, 463– 483.
Aguzzi, J., Fanelli, E., Ciuffardi, T., Schirone, A., Craig, G. & the NEMO Consortium. 2017. Inertial
bioluminescence rhythms at the Central Mediterranean KM3NeT deep-sea neutrino telescope. Scientic
Reports 7, 44938.
Aguzzi, J., Manuél, A., Condal, F. etal. 2011b. The new SEAoor OBservatory (OBSEA) for remote and long-
term coastal ecosystem monitoring. Sensors 11, 5850 –5872.
Aguzzi, J., Sbragaglia, V., Tecchio, S., Navarro, J. & Company, J.B. 2015b. Rhythmic behaviour of marine
benthopelagic species and the synchronous dynamics of benthic communities. Deep-Sea Research I
95, 1–11.
Alldredge, A.L. & Silver, M.W. 1988. Characteristics, dynamics and signicance of marine snow. Progress in
Oceanography 20, 41–82.
Allken, V., Handegard, N.O., Rosen, S., Schreyeck, T., Mahiout, T. & Malde, K. 2018. Fish species identication
using a convolutional neural network trained on synthetic data. ICES Journal of Marine Science 76(1),
342–349.
Audoly, C., Gaggero, T., Baudin, E., Folegot, T., Rizzuto, E., Mullor, R.S., André, M., Rousset, C. & Kellett, P.
2017. Mitigation of underwater radiated noise related to shipping and its impact on marine life: A practical
approach developed in the scope of AQUO Project. Journal of Oceanic Engineering IEEE 42(2), 373–387.
Audoly, C., Rousset, C., Baudin, E. & Folegot, T. 2016. AQUO Project. Research on solutions for the mitigation
of shipping noise and its impact on marine fauna: Synthesis of guidelines. In Proceedings of the 23rd
International Congress on Sound Vibrations, 10–14 July, 2016. Athens, Greece: ICSV23, pp. 1–8.
Bahamon, N., Aguzzi, J., Bernardello, R., Ahumada-Sempoal, M.A., Puigdefabregas, J., Cateura, J., Muñoz,
E., Velásquez, Z. & Cruzado, A. 2011. The new pelagic Operational Observatory of the Catalan Sea
(OOCS) for the multisensor coordinated measurement of atmospheric and oceanographic conditions.
Sensors 11(12), 11251–11272.
101
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Barnes, C., Best, M.M.R., Johnson, F.R., Pautet, L. & Pirenne, B. 2013. Challenges, benets, and opportunities
in installing and operating cabled ocean observatories: Perspectives from NEPTUNE Canada. IEEE
Journal of Oceanic Engineering 38(1), 144 –157.
Blanco, R., Shields, M.A. & Jamieson, A.J. 2013. Macrofouling of deep-sea instrumentation after three years
at 3690 m depth in the Charlie Gibbs fracture zone, mid-Atlantic ridge, with emphasis on hydroids
(Cnidaria: Hydrozoa). Deep Sea Research Part II: Topical Studies in Oceanography 98, 370–373.
Blondeau-Patissier, D., Gower, J.F., Dekker, A.G., Phinn, S.R. & Brando, V.E. 2014. A review of ocean
color remote sensing methods and statistical techniques for the detection, mapping and analysis of
phytoplankton blooms in coastal and open oceans. Progress in Oceanography 123, 123 –14 4.
Bonin, F., Burguera, A. & Oliver, G. 2011. Imaging systems for advanced underwater vehicles. Journal of
Maritime Research 8(1), 65–86.
Borgman, C.L., Wallis, J.C. & Enyedy, N. 2007. Little science confronts the data deluge: Habitat ecology,
embedded sensor networks, and digital libraries. International Journal on Digital Libraries 7, 17–30.
Bosch, J., Istenič, K., Gracias, N., Garcia, R. & Ridao, P. 2019. Omnidirectional multicamera video stitching
using depth maps. IEEE Journal of Oceanic Engineering. doi: 10.1109/JOE.2019.2924276
Buck, J.J., Bainbridge, S.J., Burger, E.F. etal. 2019. Ocean data product integration through innovation – The
next level of data interoperability. Frontiers in Marine Science 6, 1–19.
Canals, M., Puig, P., de Madron, X.D., Heussner, S., Palanques, A. & Fabres, J. 2006. Flushing submarine
canyons. Nature 444( 7117), 3 54.
Chaturvedi, K. & Kolbe, T.H. 2019. Towards establishing cross-platform interoperability for sensors in smart
cities. Sensors 19, 562. doi:10.3390/s19030562
Chatzievangelou, D., Doya, C., Mihály, S., Sastri, A.R., Thomsen, L. & Aguzzi, J. 2016. High-frequency
patterns in the abundance of benthic species near a cold-seep: An Internet Operated Vehicle application.
PLOS ONE 11(10), e0163808.
Čolaković, A. & Hadžialić, M. 2018. Internet of Things (IoT): A review of enabling technologies, challenges,
and open research issues. Computer Networks 144, 17–39.
Congalton, R.G. 1991. A review of assessing the accuracy of classications of remotely sensed data. Remote
Sensing of Environment 37, 35–46.
Corgnati, L., Marini, S., Mazzei, L., Ottaviani, E., Aliani, S., Conversi, A. & Griffa, A. 2016. Looking inside the
ocean: Toward an autonomous imaging system for monitoring gelatinous zooplankton. Sensors 16, 2124.
Dame, R.F., Chrzanowski, T.H., Bildstein, K. etal. 1986. The outwelling hypothesis and North Inlet, South
Carolina. Marine Ecology Progress Series 33, 217–2 29.
Danovaro, R., Aguzzi, J., Fanelli, E. etal. 2017. A new international ecosystem-based strategy for the global
deep ocean. Science 355, 452–454.
Deegan, L.A., Hughes, J.E. & Rountree, R.A. 2000. Salt marsh suppor t of marine transient species. In Concepts
and Controversies in Tidal Marsh Ecology, M. Weinstein & D. Kreeger (eds). Boston, Massachusetts:
Kluwer Aca demic Publishers, 333–365.
De Leo, F., Ogata, B., Sastri, A., Heeseman n, M., Mi hály, S., Galbraith, M. & Morley, M.G. 2018. High-frequency
observations from a deep-sea cabled observatory reveal seasonal overwintering of Neocalanus spp. in
Barkley Canyon, NE Pacic: Insights into particulate organic carbon ux. Progress in Oceanography
169, 120–13 7.
Del Río, J., Toma, D.M., Martínez, E., O’Reilly, T.C., Delory, E., Pearlman, J.S., Waldmann, C. & Jirka, S.
2018. A Sensor Web Architecture for integrating smart oceanographic sensors into the semantic sensor
Web. IEEE Journal of Oceanic Engineering 43(4), 830–842.
Doya, C., Aguzzi, J., Pardo, M., Matabos, M., Company, J.B., Costa, C. & Milhaly, S. 2014. Diel behavioral
rhythms in the sablesh (Anoplopoma mbria) and other benthic species, as recorded by deep-sea cabled
observatories in Barkley canyon (NEPTUNE-Canada). Journal of Marine Systems 130, 69–78.
Durden, J.M., Bett, B.J., Horton, T., Serpell-Stevens, A., Morris, K.J., Billet, D.S.M. & Ruhl, H.A. 2016b.
Improving the estimation of deep-sea megabenthos biomass: Dimension to wet weight conversions for
abyssal invertebrates. Marine Ecology Progress Series 552, 71–79.
Durden, J.M., Schoening, T., Althaus, F. etal. 2016a. Perspectives in visual imaging for marine biology and
ecology: From acquisition to understanding. Oceanography and Marine Biology Annual Review 54, 1–72.
FAO (Food and Agriculture Organization of the United Nations). 2018. The State of Mediterranean and
Black Sea Fisheries. General Fisheries Commission for the Mediterranean. Rome: Food and Agriculture
Organization.
102
RODNEY A. ROUNTREE ET AL.
Fehlmann, G. & King, A.J. 2016. Bio-logging. Current Biology 26, 830– 831.
Florea, A.G. & Buiu, C. (eds) 2017. Membrane Computing for the Distributed Control of Robotic Swarms.
Advances in Computational Intelligence and Robotics (ACIR) Book Series. IGI Global. doi:
10.4018/978-1-5225 -2280-5.
Fujii, T. & Jamieson, A.J. 2016. Fine-scale monitoring of sh movements and multiple environmental
parameters around a decommissioned offshore oil platform: A pilot study in the North Sea. Ocean
Engineering 126, 481–487.
Gould, J., Sloyan, B., Visbeck, M. 2013. In situ ocean observations: A brief history, present status, and future
directions. International Geophysics 103, 59–81.
Graeme, C.H., Ferreira, L.C., Sequeira, A.M.M. etal. 2010. Key questions in marine megafauna movement
ecology. Trends in Ecology and Evolution 31, 4 63 475.
Haines, E.B. 1979. Interactions between Georgia salt marshes and coastal waters: A changing paradigm. In
Ecological Processes in Coastal and Marine Systems, R.J. Livingston (ed.). New York: Plenum, 35– 46.
Handegard, N.O., Du Buisson, L., Brehmer, P. etal. 2013. Towards an acoustic-based coupled observation
and modelling system for monitoring and predicting ecosystem dynamics of the open ocean. Fish and
Fisheries 14(4), 605–615.
Harvey, E.S. & Shortis, M.R. 1998. Calibration stability of an underwater stereo-video system: Implications
for measurement accuracy and precision. Marine Technology Society Journal 32(2) , 3 –17.
Honjo, S. 1980. Material uxes and modes of sedimentation in the mesopelagic and bathypelagic zones.
Journal of Marine Research 38, 53 97.
Hussey, N.E., Kessel, S.T., Aarestrup, K. etal. 2015. Aquatic animal telemetry: A panoramic window into the
underwater world. Science 348, 1221–1231.
Irigoien, X., Klevjer, T.A., Røstad, A. etal. 2014. Large mesopelagic shes’ biomass and trophic efciency in
the open ocean. Nature Communications 5, 3271.
Juanes, F. 2018. Visual and acoustic sensor for early detection of biological invasions: Current uses and future
potential. Journal for Nature Conservation 42, 7–11.
Juniper, S.K., Sastri, A., Mihály, S., Whitehead, J., Brent, E., Helmuth, T. & Miller, L. 2016. Continuous pCO2
time series from Ocean Networks Canada cabled observatories on the northeast Pacic shelf-edge/upper
slope and in the sub-tidal Arctic. European Geophysical Union, Geophysical Research Abstracts 18,
EGU2016-9426.
Kobari, T., Steinberg, D.K., Ueda, A., Tsuda, A., Silver, M.W. & Kitamura, M. 2008. Impacts of ontogenetically
migrating copepods on downward carbon ux in the western subarctic Pacic Ocean. Deep-Sea Research
II 55, 1648 –1660.
Lecours, V., Devillers, R., Schneider, D.C., Lucieer, V.L., Brown, C.J. & Edinger, E.N. 2015. Spatial scale and
geographic context in benthic habitat mapping: Review and future directions. Marine Ecology Progress
Series 535, 259–284.
Lin, Y., Hsiung, S.-C., Piersal, R., Whitem, C., Lowe, C.G. & Clark, C.M. 2016. A multi-autonomous underwater
vehicle system for autonomous tracking of marine life. Journal of Field Robotics 34, 757–774.
Lloyd, S., Lepper, P. & Pomeroy, S. 2017. Evaluation of UAVs as an underwater acoustics sensor deployment
platform. International Journal of Remote Sensing 38, 2808–2817.
Locascio, J., Mann, D., Wilcox, K. & Luther, M. 2018. Incorporation of acoustic sensors on a coastal ocean
monitoring platform for measurements of biological activity. Marine Technology Society Journal 52(3),
64 –70.
Longhurst, A.R. 1976. Vertical migration. In The Ecology of the Seas, D.H. Cushing & J.J. Walsh (eds). Oxford:
Blackwell Scientic Publications, 116–137.
Luczkovich, J.J., Mann, D.A. & Rountree, R.A. 2008. Passive acoustics as a tool in sheries science.
Transactions of the American Fisheries Society 137, 533–541.
Marini, S., Corgnati, L., Manotovani, C., Bastianini, M., Ottaviani, E., Fanelli, E., Aguzzi, J., Griffa, A. &
Poulain, P.M. 2018a. Automated estimate of sh abundance through the autonomous imaging device
GUA RD1. Measurement 126, 72 –75.
Marini, S., Fanelli, E., Sbragaglia, V., Azzurro, E., Del Rio, J. & Aguzzi, J. 2018b. Tracking sh abundance by
underwater image recognition. Scientic Reports 8, 13748.
Marshall, K.N., Levin, P.S., Essington, T.E. etal. 2018. Ecosystem-based sheries management for social–
ecological systems: Renewing the focus in the United States with next generation shery ecosystem
plans. Conservation Letters 11(1), e12 367.
103
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Marshall, N.B. 1971. Explorations in the Life of Fishes. Cambridge, Massachusetts: Harvard University Press.
Martin, P., Remy, L., Theodoridou, M., Jeffery, K. & Zhao, Z. 2019. Mapping heterogeneous research
infrastructure metadata into a unied catalogue for use in a generic virtual research environment. Future
Generation Computer Systems 101, 327–340.
Masmitja, I., Bouvet, P.J., Gomarítz, S., Aguzzi, J. & Del Río, J. 2017. Accuracy and precision studies for range-
only underwater target tracking in shallow waters. In Supporting World Development Through Electrical
and Electronic Measurements, 22nd IMEKO TC4 International Symposium and 20th International
Workshop on ADC Modelling and Testing, 14–15 September 2017. Iasi, Romania. Budapest, Hungary:
International Measurement Confederation (IMEKO), 94–99.
Mauchline, J. 1980. The biology of mysids and euphausiids. Part 2. The biology of euphausiids. Advances in
Marine Biology 18, 372– 623.
Maxwell, S.M., Hazen, E.L., Lewison, R.L. et al. 2015. Dynamic ocean management: Dening and
conceptualizing real-time management of the ocean. Marine Policy 58, 42–50.
McCave, I.N. 1975. Vertical ux of carbon in the ocean. Deep-Sea Research 22, 491–502.
Merrett, N.R. 1978. On the identity and pelagic occurrence of larval and juvenile stages of rattail shes (family
Macrouridae) from 60°N, 20°W and 53°N, 20°W. Deep-Sea Research I 25, 147–160.
Merritt, D., Donovan, M.K., Kelley, C., Waterhouse, L., Parke, M., Wong, K. & Drazen, J.C. 2011. BotCam: A
baited camera system for nonextractive monitoring of bottomsh species. Fishery Bulletin 109(1), 56 –67.
Michener, W.K. & Jones, M.B. 2012. Ecoinformatics: Supporting ecology as a data-intensive science. Tren ds
in Ecology and Evolution 27, 85–93.
Moll, R.J., Millspaugh, J.J., Beringer, J., Sartwell, J. & He, Z. 2007. A new ‘view’ of ecology and conservation
through animal-borne video systems. Trends in Ecology and Evolution 22, 660–668.
Morris, C., Andreetto, P., Banci, L. etal. 2019. West-Life: A virtual research environment for structural biology.
Journal of Structural Biology: X 1 (January–March 2019), 100006. doi: 10.1016/j.yjsbx.2019.100006.
Mouy, X., Rountree, R., Juanes, F. & Dosso, S.E. 2018. Cataloguing sh sounds in the wild using combined
acoustic and video recordings. The Journal of the Acoustical Society of America 143 (5), 333–339.
Nassar, J.M., Khan, S.M., Velling, S.J., Diaz-Gaxiola, A., Shaikh, S.F., Geraldi, N.R., Sevilla, G.A.T., Duarte,
C.M. & Hussain, M.M. 2018. Compliant lightweight non-invasive standalone ‘Marine Skin’ tagging
system. npj Flexible Electronics 2(1), 13, doi: 10.1038/s41528-018-0025-1
Naylor, E. 2010. Chronobiology of Marine Organisms. Cambridge: Cambridge University Press.
Nelson, J.R. & Weisberg, R.H. 2008. In situ observations and satellite remote sensing in SEACOOS: Program
development and lessons learned. Marine Technology Society Journal 42(3) , 41–54 .
Nixon, S.W. 1980. Between coastal marshes and coastal waters: A review of twenty years of speculation and
research on the role of salt marshes in estuarine productivity and water chemistry. In Estuarine and
Wetland Processes, P. Hamilton & K.B. MacDonald (eds). New York: Plenum, 437–525.
NOC (National Ocean Council). 2016. Animal Telemetry Network Implementation Plan 2016–2021.
Washington, DC: National Ocean Council.
OBSEA. 2019. OBSEA Expandable Seaoor Observatory. Barcelona: SARTI Technological Development
Center of Remote Acquisition and Data Processing Systems. Online. https://obsea.es/ (accessed 18 April
2019).
Odum, E.P. 1980. The status of three ecosystem-level hypotheses regarding salt marsh estuaries: Tidal subsidy,
outwelling, and detritus-based food chains. In Estuarine Perspectives, V.S. Kennedy (ed.). New York:
Academic Press, 485– 495.
ONC (Ocean Networks Canada). 2019. Discover the Ocean. Understand the Planet. Victoria, British Columbia:
Oean Networks Canada. Online. http://www.oceannetworks.ca/sights-sounds/images/maps (accessed
18 April 2019).
OOI (Ocean Observatories Initiative). 2019. The Ocean Observatories Initiative. Woods Hole, Massachusetts:
Ocean Observatories Initiative. Online. https://oceanobservatories.org/ (accessed 18 April 2019).
OTN (Ocean Tracking Network). 2019. Ocean Tracking Network. Halifax, Nova Scotia: Ocean Tracking
Network. Online. http://oceantrackingnetwork.org/ (accessed 18 April 2019).
Pearlman, J.S., Bushnell, M., Coppola, L. etal. 2019. Evolving and sustaining ocean best practices and
standards for the next decade. Frontiers in Marine Science 6, 277.
Pomeroy, L.R. & Wiegert, R.G. (eds) 1981. The Ecology of a Salt Marsh. New York: Springer.
Puig, P., Palanques, A. & Martín, J. 2014. Contemporary sediment-transport processes in submarine canyons.
Annual Review of Marine Science 6, 53–77.
104
RODNEY A. ROUNTREE ET AL.
Qin, Y., Sheng, Q.Z., Falkner, N.J.G., Dustdar, S., Wang, H. & Vasilakos, A.V. 2016. When things matter: A
survey on data-centric internet of things. Journal of Network and Computer Applications 64, 137–153.
Recknagel, F. & Michener, W.K. (eds) 2018. Ecological Informatics: Data Management and Knowledge
Discovery. Cham, Switzerland: Springer Nature, 3rd edition.
Rempel, A. & Cabrera, D.A. 2018. The Oceans 2.0 Sandbox. Victoria, British Columbia: Ocean Networks Canada.
Online. https://wiki.oceannetworks.ca/display/O2A/The+Oceans + 2.0+Sandbox (accessed 18 April 2019).
Rex, M.A. & Etter, R.J. (eds) 2010. Deep-Sea Biodiversity: Pattern and Scale. Cambridge, Massachusetts:
Har vard University Press.
Rogers, A.D. 2015. Environmental change in the deep ocean. Annual Review of Environment and Resources
40, 1–38.
Rotllant, G., Aguzzi, J., Sarria, D. etal. 2014. Pilot acoustic tracking study on adult spiny lobsters (Palinurus
mauritanicus) and spider crabs (Maja squinado) within an articial reef. Hydrobiologia 742, 27–38.
Rountree, R.A. 1992. Fish and macroinvertebrate community structure and habitat use patterns in salt marsh
creeks of southern New Jersey, with a discussion of marsh carbon export. Ph.D. Dissertation, Rutgers,
The State University of New Jersey, United States of America.
Rountree, R.A. 2008. Do you hear what I hear? Future technological development and needs in passive
acoustics underwater observation. Marine Technology Reporter 51(9), 40– 46.
Rountree, R.A. & Able, K.W. 2007. Spatial and temporal habitat use patterns for salt marsh nekton: Implications
for functions. Aquatic Ecology 41, 25– 45.
Rountree, R.A., Gilmore, R.G., Goudey, C.A., Hawkins, A.D., Luczkovich, J. & Mann, D.A. 2006. Listening
to sh: applications of passive acoustics to sheries science. Fisheries 31(9), 433– 446.
Rountree, R.A., Goudey, C. & Hawkins, T. (eds) 2003. Listening to Fish. Proceedings of the International
Workshop on the Applications of Passive Acoustics to Fisheries. April 8–10, 2002, Dedham,
Massachusetts. MIT Sea Grant Technical Report MITSG 03-2. Online. http://web.mit.edu/seagrant/
aqua/cfer/acoustics/PAprocBrFINAL.pdf (accessed 3 October 2019).
Rountree, R.A. & Juanes, F. 2010. First attempt to use a remotely operated vehicle to observe soniferous sh
behaviour in the Gulf of Maine, Western Atlantic Ocean. Current Zoology 56(1), 90 –99.
Rountree, R.A., Juanes, F. & Bolgan, M. 2018. Air movement sound production by alewife, white sucker, and
four salmonid shes suggests the phenomenon is widespread among freshwater shes. PLOS ONE 13(9),
e02 0 42 47.
Rountree, R.A., Juanes, F., Goudey, C.A. & Ekstrom, K.E. 2012. Is biological sound production important
in the deep sea? In The Effects of Noise on Aquatic Life, A.N. Popper & A. Hawkins (eds). New York:
Springer Science+Business Media, LLC, 118–183. doi: 10.1007/978-1-4419-7311-5_41
Rountree, R.A., Perkins, P.J., Kenney, R.D. & Hinga, K.R. 2002. Sounds of western North Atlantic shes: Data
rescue. Bioacoustics 12, 242–244.
Santos, A.M.P. 2000. Fisheries oceanography using satellite and airborne remote sensing methods: A review.
Fisheries Research 49(1), 1–20.
Schoeld, O. & Glenn, S. 2004. Introduction to special section: Coastal ocean observatories. Journal of
Geophysical Research: Oceans, 109(C12 S01). doi: 10.1029/200 4JC 00 2577
Shenoi, R.A., Bowker, J.A., Dzielendziak, A.S. etal. 2015. Global marine technology trends 2030. Southampton:
Lloyd’s Register, Farnborough: QinetiQ, Southampton: University of Southampton. Online. https://issuu.
com/lr_marine/docs/55046_lr2030_web-lr_25mb/1 (accessed 18 April 2019).
Shortis, M. & Abdo, E.H.D. 2016. A review of underwater stereo-image measurement for marine biology and
ecology applications. Oceanography and Marine Biology: An Annual Review 47, 257–292.
Shortis, M.R., Seager, J.W., Williams, A., Barker, B.A. & Sherlock, M. 2008. Using stereo-video for deep water
benthic habitat surveys. Marine Technology Society Journal 42(4), 28 –37.
Sivčev, S., Coleman, J., Omerdić, E., Dooly, G. & Toal, D. 2018. Underwater manipulators: A review. Ocean
Engineering 163, 431– 450.
Skiena, S.S. 2017. The Data Science Design Manual. Cham, Switzerland: Springer Nature.
Teal, J.M. 1962. Energy ow in the salt marsh ecosystem of Georgia. Ecology 43, 614–6 24 .
Thomsen, L., Aguzzi, J., Costa, C., De Leo, F., Ogston, A. & Purser, A. 2017. The oceanic biological pump:
Rapid carbon transfer to depth at continental margins during winter. Scientic Reports 7(1), 10763.
Toonen, H.M. & Bush, S.R. 2018. The digital frontiers of sheries governance: Fish attraction devices,
drones and satellites. Journal of Environmental Policy and Planning 22(1), 125–137. doi:
10.108 0/1523908X.2018.1461084
105
TOWARDS AN OPTIMAL DESIGN FOR ECOSYSTEM-LEVEL OCEAN OBSERVATORIES
Tuck, I.D., Parsons, D.M., Hartill, B.W. & Chiswell, S.M. 2015. Scampi (Metanephrops challengeri) emergence
patterns and catchability. ICES Journal of Marine Science 72, 199–210 .
Tunnicliffe, V., Dewey, R. & Smith, D. 2003. Research plans for a mid-depth cabled seaoor observatory in
Western Canada. Oceanography 16(4), 53 –59.
Vardaro, M.F., Parmley, D. & Smith, K.L. Jr. 2007. A study of possible “reef effects” caused by a long-term
time-lapse camera in the deep North Pacic. Deep Sea Research Part I: Oceanographic Research Papers
54(8), 1231–1240.
Vetger, A.C., Barletta, M., Beck, C. etal. 2014. Global research priorities to mitigate plastic pollution impacts
on marine wildlife. Endangered Species Research 25, 225 –24 7.
Vinogradov, M.E. 1953. The role of vertical migration of the zooplankton in the feeding of deep sea animals.
Priroda 6, 95–96.
Vinogradov, M.E. 1955. Vertical migrations of zooplankton and their importance for the nutrition of abyssal
pelagic faunas. Transactions of the Institute of Oceanology (USSR) 13, 71–76.
Vinogradov, M.E. (ed.) 1971. Life Activit y of Pelagic Communities in the Ocean Tropics (Based on Data of the
44th Cruise of the R/V Vityaz). Institute of Oceanography, Academy of Science of the USSR. Izdatel’stvo
“Nauka.” Moskva. (Kaner, N., Transl., 1973. Israel Program for Scientic Translations, Jerusalem).
Wakeeld, W.W. & Smith, K.L. Jr. 1990. Ontogenetic vertical migration in Sebastolobus altivelis as a
mechanism for transport of particulate organic matter at continental slope depths. Limnology and
Oceanography 35(6), 1314–1328.
Wall, C.C., Rountree, R.A., Pomerleau, C. & Juanes, F. 2013. An exploration for deep-sea sh sounds off
Vancouver Island from the NEPTUNE Canada ocean observing system. Deep-Sea Research I 83, 57–64.
Walters, C.C. 2007. Is adaptive management helping to solve sheries problems? AMBIO: A Journal of the
Human Environment 36(4), 304–308.
Williams, K., Rooper, C.N., De Robertis, A., Levine, M. & Towler, R. 2018. A method for computing volumetr ic
sh density using stereo cameras. Journal of Experimental Marine Biology and Ecology 508, 21–26.
Williams, K., Rooper, C.N. & Towler, R. 2010. Use of stereo camera systems for assessment of rocksh
abundance in untrawlable areas and for recording pollock behavior during midwater trawls. Fishery
Bulletin 108(3), 352 –362.
Wilmer, C.C., Nickel, B., Bryce, C.M., Smith, J.A., Wheat, R.E. & Yovovich, V. 2015. The golden age of
bio-logging: How animal-borne sensors are advancing the frontiers of ecology. Ecology 96, 1741–1753.
Zeppilli, D., Pusceddu, A., Trincardi, F. & Danovaro, R. 2016. Seaoor heterogeneity inuences the
biodiversity–ecosystem functioning relationships in the deep sea. Scientic Reports 6, 26352.
... Vessel-independent monitoring technologies such as cabled observatories allow for prolonged and autonomous data collection that can be implemented at virtually any depth of the continental margin [26][27][28]. Their deployment implies very high initial costs that can be progressively depreciated over years of continuous data collection [29]. Although their fixed imaging has been extensively used to assess the animals' abundance, behaviour, biodiversity, and even community successions in many marine areas (e.g., [25,[30][31][32][33]), they lack an extensive Field of View (FOV), therefore, the acquired data are not representative of the ecological heterogeneity that surrounds the observatories [27]. ...
... Their deployment implies very high initial costs that can be progressively depreciated over years of continuous data collection [29]. Although their fixed imaging has been extensively used to assess the animals' abundance, behaviour, biodiversity, and even community successions in many marine areas (e.g., [25,[30][31][32][33]), they lack an extensive Field of View (FOV), therefore, the acquired data are not representative of the ecological heterogeneity that surrounds the observatories [27]. ...
... Depending on the specific needs, different platforms could be eligible for that task. Docked propeller-driven pelagic robots, such as ROVs, AUVs, and Autonomous Underwater Helicopters (AUHs) are preferred when medium-long range mobility through rough seabed morphologies must be met [27,34]. However, ROVs will result in a shorter range, due to the limitation of the cable, but provide a higher degree of control over the operations [35][36][37]. ...
Article
Full-text available
The use of marine cabled video observatories with multiparametric environmental data collection capability is becoming relevant for ecological monitoring strategies. Their ecosystem surveying can be enforced in real time, remotely, and continuously, over consecutive days, seasons, and even years. Unfortunately, as most observatories perform such monitoring with fixed cameras, the ecological value of their data is limited to a narrow field of view, possibly not representative of the local habitat heterogeneity. Docked mobile robotic platforms could be used to extend data collection to larger, and hence more ecologically representative areas. Among the various state-of-the-art underwater robotic platforms available, benthic crawlers are excellent candidates to perform ecological monitoring tasks in combination with cabled observatories. Although they are normally used in the deep sea, their high positioning stability, low acoustic signature, and low energetic consumption, especially during stationary phases, make them suitable for coastal operations. In this paper, we present the integration of a benthic crawler into a coastal cabled observatory (OBSEA) to extend its monitoring radius and collect more ecologically representative data. The extension of the monitoring radius was obtained by remotely operating the crawler to enforce back-and-forth drives along specific transects while recording videos with the onboard cameras. The ecological relevance of the monitoring-radius extension was demonstrated by performing a visual census of the species observed with the crawler’s cameras in comparison to the observatory’s fixed cameras, revealing non-negligible differences. Additionally, the videos recorded from the crawler’s cameras during the transects were used to demonstrate an automated photo-mosaic of the seabed for the first time on this class of vehicles. In the present work, the crawler travelled in an area of 40 m away from the OBSEA, producing an extension of the monitoring field of view (FOV), and covering an area approximately 230 times larger than OBSEA’s camera. The analysis of the videos obtained from the crawler’s and the observatory’s cameras revealed differences in the species observed. Future implementation scenarios are also discussed in relation to mission autonomy to perform imaging across spatial heterogeneity gradients around the OBSEA.
... An efficient method for the automatic detection of some of these behaviours holds an enormous biological value, as it allows for a specific interpretation of long-term video recordings from offshore observatories [2], [13], e.g., identifying predation events of a given species, rather than just its presence. As a result, this capability represents a change in the level of abstraction of machine learning-and computer visionbased interpretation of underwater data [14]: from a narrow and context-agnostic counting of specimens to a broader and biologically complex identification of specific behaviours. ...
... More specifically, our slowest configurations of TempNet (with Wavelet Down Sampling) can process 4second videos in 42 ms. This efficiency enables quick processing of large existing marine video datasets and the realtime processing of multiple key data streams for collecting ecologically relevant information [13]. ...
... An efficient method for the automatic detection of some of these behaviours holds an enormous biological value, as it allows for a specific interpretation of long-term video recordings from offshore observatories [2], [13], e.g., identifying predation events of a given species, rather than just its presence. As a result, this capability represents a change in the level of abstraction of machine learning-and computer visionbased interpretation of underwater data [14]: from a narrow and context-agnostic counting of specimens to a broader and biologically complex identification of specific behaviours. ...
... More specifically, our slowest configurations of TempNet (with Wavelet Down Sampling) can process 4second videos in 42 ms. This efficiency enables quick processing of large existing marine video datasets and the realtime processing of multiple key data streams for collecting ecologically relevant information [13]. ...
Preprint
Full-text available
Recent advancements in cabled ocean observatories have increased the quality and prevalence of underwater videos; this data enables the extraction of high-level biologically relevant information such as species' behaviours. Despite this increase in capability, most modern methods for the automatic interpretation of underwater videos focus only on the detection and counting organisms. We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos. TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder. TempNet also presents temporal attention during spatial encoding as well as Wavelet Down-Sampling pre-processing to improve model accuracy. Although our system is designed for applications to diverse fish behaviours (i.e, is generic), we demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events. We compare the proposed approach with a state-of-the-art end-to-end video detection method (ReMotENet) and a hybrid method previously offered exclusively for the detection of sablefish's startle events in videos from an existing dataset. Results show that our novel method comfortably outperforms the comparison baselines in multiple metrics, reaching a per-clip accuracy and precision of 80% and 0.81, respectively. This represents a relative improvement of 31% in accuracy and 27% in precision over the compared methods using this dataset. Our computational pipeline is also highly efficient, as it can process each 4-second video clip in only 38ms. Furthermore, since it does not employ features specific to sablefish startle events, our system can be easily extended to other behaviours in future works.
... 3 Selected observations and monitored events 3.1 Morphological and structural features As a mobile, resident monitoring vehicle performing transects along marked seafloor stations, the crawler has been able to make spatially geo-referenced observations and record features on the seabed which might have been missed by other mobile, sporadically deployed vehicles (e.g., ROVs, AUVs, towed or drift-cameras) which are vessel-dependent and are not capable of spatiotemporally repetitive and intensive monitoring across the extended temporal scales achievable with the crawler (Bicknell et al., 2016;Dominguez-Carrioé t al., 2021). Similarly, fixed platforms (e.g., lander cameras) have a restricted field of view which limits the ecological representation capability of acquired data spatially, though achieving the temporality capable with the crawler platform (Rountree et al., 2020). ...
... This solution is especially suited for increasing imaging monitoring penetrability of crawlers of several meters beyond the usual reach of HD camera systems (Aguzzi et al., 2019). Additional solutions may include the use of new, low-light cameras and motors with reduced sound emissions could tackle the issue of light and noise in the future (Phillips et al., 2016;Rountree et al., 2020), though the crawler is already a far quieter system than thruster equipped AUVs or ROVs. ...
Article
Full-text available
Scientific, industrial and societal needs call urgently for the development and establishment of intelligent, cost-effective and ecologically sustainable monitoring protocols and robotic platforms for the continuous exploration of marine ecosystems. Internet Operated Vehicles (IOVs) such as crawlers, provide a versatile alternative to conventional observing and sampling tools, being tele-operated, (semi-) permanent mobile platforms capable of operating on the deep and coastal seafloor. Here we present outstanding observations made by the crawler “Wally” in the last decade at the Barkley Canyon (BC, Canada, NE Pacific) methane hydrates site, as a part of the NEPTUNE cabled observatory. The crawler followed the evolution of microhabitats formed on and around biotic and/or abiotic structural features of the site (e.g., a field of egg towers of buccinid snails, and a colonized boulder). Furthermore, episodic events of fresh biomass input were observed (i.e., the mass transport of large gelatinous particles, the scavenging of a dead jellyfish and the arrival of macroalgae from shallower depths). Moreover, we report numerous faunal behaviors (i.e., sablefish rheo- and phototaxis, the behavioral reactions and swimming or resting patterns of further fish species, encounters with octopuses and various crab intra- and interspecific interactions). We report on the observed animal reactions to both natural and artificial stimuli (i.e., crawler’s movement and crawler light systems). These diverse observations showcase different capabilities of the crawler as a modern robotic monitoring platform for marine science and offshore industry. Its long deployments and mobility enable its efficiency in combining the repeatability of long-term studies with the versatility to opportunistically observe rarely seen incidents when they occur, as highlighted here. Finally, we critically assess the empirically recorded ecological footprint and the potential impacts of crawler operations on the benthic ecosystem of the Barkley Canyon hydrates site, together with potential solutions to mitigate them into the future.
... tichannel acoustic recorders and video cameras to existing cabled observatories where storage and power are not a limitation(Aguzzi et al., 2019;Rountree et al., 2020). Since the beginning of F I G U R E 1 5 Illustration of the different constraints, strengths and weaknesses of each audio-video array proposed in this study.Downloaded from https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.14095 ...
Article
Full-text available
Associating fish sounds to specific species and behaviours is important for making passive acoustics a viable tool for monitoring fish. While recording fish sounds in tanks can sometimes be performed, many fish do not produce sounds in captivity. Consequently, there is a need to identify fish sounds in situ and characterise these sounds under a wide variety of behaviours and habitats. We designed three portable audio-video platforms capable of identifying species-specific fish sounds in the wild: a large array, a mini array and a mobile array. The large and mini arrays are static autonomous platforms than can be deployed on the seafloor and record audio and video for one to two weeks. They use multichannel acoustic recorders and low-cost video cameras mounted on PVC frames. The mobile array also uses a multichannel acoustic recorder, but mounted on a remotely operated vehicle with built-in video, which allows remote control and real-time positioning in response to observed fish presence. For all arrays, fish sounds were localised in three dimensions and matched to the fish positions in the video data. We deployed these three platforms at four locations off British Columbia, Canada. The large array provided the best localisation accuracy and, with its larger footprint, was well suited to habitats with a flat seafloor. The mini and mobile arrays had lower localisation accuracy but were easier to deploy, and well suited to rough/uneven seafloors. Using these arrays, we identified, for the first time, sounds from quillback rockfish Sebastes maliger, copper rockfish Sebastes caurinus and lingcod Ophiodon elongatus. In addition to measuring temporal and spectral characteristics of sounds for each species, we estimated mean source levels for lingcod and quillback rockfish sounds (115.4 and 113.5 dB re 1 μPa, respectively) and maximum detection ranges at two sites (between 10.5 and 33 m). All proposed array designs successfully identified fish sounds in the wild and were adapted to various budget, logistical and habitat constraints. We include here building instructions and processing scripts to help users replicate this methodology, identify more fish sounds around the world and make passive acoustics a more viable way to monitor fish.
... Cabled video-observatory monitoring technology is considered as the core of growing in situ and robotized marine ecological laboratories in coastal and deep-sea areas 14,15 . International initiatives about marine observatories infrastructures, like for example the European Multidisciplinary Seafloor and water column Observatory (EMSO-ERIC), the Joint European Research Infrastructure of Coastal Observatories (JERICO-RI), or the Ocean Network Canada (ONC) are becoming widespread all over the world 16 , and increasingly install multiparametric sensors that, beside the imaging depicting biological information, also acquire oceanographic and geo-chemical data 13,17 . ...
Article
Full-text available
Multiparametric video-cabled marine observatories are becoming strategic to monitor remotely and in real-time the marine ecosystem. Those platforms can achieve continuous, high-frequency and long-lasting image data sets that require automation in order to extract biological time series. The OBSEA, located at 4 km from Vilanova i la Geltrú at 20 m depth, was used to produce coastal fish time series continuously over the 24-h during 2013–2014. The image content of the photos was extracted via tagging, resulting in 69917 fish tags of 30 taxa identified. We also provided a meteorological and oceanographic dataset filtered by a quality control procedure to define real-world conditions affecting image quality. The tagged fish dataset can be of great importance to develop Artificial Intelligence routines for the automated identification and classification of fishes in extensive time-lapse image sets.
... This is true especially for the benthic fauna, where systematic observations repeated in time provide information useful for a detailed understanding and predictions of their dynamics. Furthermore, structure and composition of the benthic fauna are used as an effective tool to identify the impacts of environmental factors as well as the impact of human activities 5,6,[19][20][21] . ...
Article
Full-text available
Antarctica is a remote place, the continent is covered by ice and its surrounding coastal areas are frozen for the majority of the year. Due to its peculiarity the observation of the underwater organisms is particularly difficult, complicated by logistic factors. We present a long-term dataset consisting of 755 images acquired by using a non-invasive, autonomous imaging device and encompassing both the Antarctic daylight and dark periods, including the corresponding transition phases. All images have the same field of view showing the benthic fauna and part of the water column above, including fishes present in the monitored period. All the images are manually annotated after a visual inspection performed by expert biologists. The extended monitoring period and the annotated images make the dataset a valuable benchmark suitable for studying the dynamics of the long-term Antarctic underwater fauna as well as for developing and testing algorithms for automated image analysis focused on the recognition and classification of the Antarctic organisms and the automated analysis of their long-term dynamics.
... There is therefore an opportunity to improve current stock assessment methods through the use of novel tools such as optoacoustic imaging, telemetry, environmental DNA (eDNA) metabarcoding, and Artificial Intelligence (AI) (Aguzzi et al., 2020b). Moreover, with recent development in ecological monitoring technologies (e.g., Aguzzi et al., 2019;Rountree et al., 2020) these tools could be implemented using fishery independent, autonomous and remote robotic platforms. We consider that many of the proposed technologies are expected to work in tandem, as their operational ranges and monitoring targets make them complementary rather than directly competing/comparable. Thus, we believe that ranking these technological solutions is not something within the scope of this manuscript. ...
Article
Full-text available
The Norway lobster, Nephrops norvegicus, supports a key European fishery. Stock assessments for this species are mostly based on trawling and UnderWater TeleVision (UWTV) surveys. However, N. norvegicus are Frontiers in Marine Science burrowing organisms and these survey methods are unable to sample or observe individuals in their burrows. To account for this, UWTV surveys generally assume that "1 burrow system = 1 animal", due to the territorial behavior of N. norvegicus. Nevertheless, this assumption still requires in-situ validation. Here, we outline how to improve the accuracy of current stock assessments for N. norvegicus with novel ecological monitoring technologies, including: robotic fixed and mobile camera-platforms, telemetry, environmental DNA (eDNA), and Artificial Intelligence (AI). First, we outline the present status and threat for overexploitation in N. norvegicus stocks. Then, we discuss how the burrowing behavior of N. norvegicus biases current stock assessment methods. We propose that state-of-the-art stationary and mobile robotic platforms endowed with innovative sensors and complemented with AI tools could be used to count both animals and burrows systems in-situ, as well as to provide key insights into burrowing behavior. Next, we illustrate how multiparametric monitoring can be incorporated into assessments of physiology and burrowing behavior. Finally, we develop a flowchart for the appropriate treatment of multiparametric biological and environmental data required to improve current stock assessment methods.
Article
The demand for ocean exploration down to the deep seafloor, for instance, ocean resource exploration and ocean observations, has promoted innovations in underwater technology. Underwater unmanned vehicles are urgently required to be deployed and operate on the seafloor. Because most underwater vehicles are unable to operate on the seafloor and even have difficulty reaching there, we developed a seafloor-resident autonomous underwater helicopter (AUH). This paper introduces the new idea and design of AUHs and discusses the pros and cons of AUHs in comparison with other underwater vehicles. Afterwards, we verify the importance of developing new facilities to enable mankind to easily operate close to the seafloor.
Article
This paper presents a cooperative fault-tolerant mission planner system for a team of unmanned surface vehicles (USVs) employed for monitoring and inspection of ocean sensor networks. The problem of monitoring and inspection of ocean sensor networks is summarized as verifying the healthiness of the buoy sensors via USVs in a pre-determined zone and within a pre-defined mission time. A realistic map of the SOFAR's global Spotter network, including Spotter buoys used for observation, forecast, and hindcast of ocean and weather data is used. The mission planner system is then developed to provide a cooperative inspection of the Spotter buoys to the maximum possible numbers in an assigned mission time while the uncertainty with failure of USVs due to internal /external faults is incorporated. Extensive simulation studies together with statistical analysis are conducted to verify the performance of the proposed cooperative fault-tolerant mission planner system. The simulation results confirm the fidelity, effectiveness, and robustness of the proposed system to be implemented onboard USVs' host computers for different monitoring and inspection missions.
Article
Full-text available
The oceans play a key role in global issues such as climate change, food security, and human health. Given their vast dimensions and internal complexity, efficient monitoring and predicting of the planet’s ocean must be a collaborative effort of both regional and global scale. A first and foremost requirement for such collaborative ocean observing is the need to follow well-defined and reproducible methods across activities: from strategies for structuring observing systems, sensor deployment and usage, and the generation of data and information products, to ethical and governance aspects when executing ocean observing. To meet the urgent, planet-wide challenges we face, methods across all aspects of ocean observing should be broadly adopted by the ocean community and, where appropriate, should evolve into “Ocean Best Practices.” While many groups have created best practices, they are scattered across the Web or buried in local repositories and many have yet to be digitized. To reduce this fragmentation, we introduce a new open access, permanent, digital repository of best practices documentation (oceanbestpractices.org) that is part of the Ocean Best Practices System (OBPS). The new OBPS provides an opportunity space for the centralized and coordinated improvement of ocean observing methods. The OBPS repository employs user-friendly software to significantly improve discovery and access to methods. The software includes advanced semantic technologies for search capabilities to enhance repository operations. In addition to the repository, the OBPS also includes a peer reviewed journal research topic, a forum for community discussion and a training activity for use of best practices. Together, these components serve to realize a core objective of the OBPS, which is to enable the ocean community to create superior methods for every activity in ocean observing from research to operations to applications that are agreed upon and broadly adopted across communities. Using selected ocean observing examples, we show how the OBPS supports this objective. This paper lays out a future vision of ocean best practices and how OBPS will contribute to improving ocean observing in the decade to come.
Article
Full-text available
In the next decade the pressures on ocean systems and the communities that rely on them will increase along with impacts from the multiple stressors of climate change and human activities. Our ability to manage and sustain our oceans will depend on the data we collect and the information and knowledge derived from it. Much of the uptake of this knowledge will be outside the ocean domain, for example by policy makers, local Governments, custodians, and other organizations, so it is imperative that we democratize or open the access and use of ocean data. This paper looks at how technologies, scoped by standards, best practice and communities of practice, can be deployed to change the way that ocean data is accessed, utilized, augmented and transformed into information and knowledge. The current portal-download model which requires the user to know what data exists, where it is stored, in what format and with what processing, limits the uptake and use of ocean data. Using examples from a range of disciplines, a web services model of data and information flows is presented. A framework is described, including the systems, processes and human components, which delivers a radical rethink about the delivery of knowledge from ocean data. A series of statements describe parts of the future vision along with recommendations about how this may be achieved. The paper recommends the development of virtual test-beds for end-to-end development of new data workflows and knowledge pathways. This supports the continued development, rationalization and uptake of standards, creates a platform around which a community of practice can be developed, promotes cross discipline engagement from ocean science through to ocean policy, allows for the commercial sector, including the informatics sector, to partner in delivering outcomes and provides a focus to leverage long term sustained funding. The next 10 years will be “make or break” for many ocean systems. The decadal challenge is to develop the governance and co-operative mechanisms to harness emerging information technology to deliver on the goal of generating the information and knowledge required to sustain oceans into the future.
Article
Full-text available
The West-Life project (https://about.west-life.eu/) is a Horizon 2020 project funded by the European Commission to provide data processing and data management services for the international community of structural biologists, and in particular to support integrative experimental approaches within the field of structural biology. It has developed enhancements to existing web services for structure solution and analysis, created new pipelines to link these services into more complex higher-level workflows, and added new data management facilities. Through this work it has striven to make the benefits of European e-Infrastructures more accessible to life-science researchers in general and structural biologists in particular.
Article
Full-text available
Typically, smart city projects involve complex distributed systems having multiple stakeholders and diverse applications. These applications involve a multitude of sensor and IoT platforms for managing different types of timeseries observations. In many scenarios, timeseries data is the result of specific simulations and is stored in databases and even simple files. To make well-informed decisions, it is essential to have a proper data integration strategy, which must allow working with heterogeneous data sources and platforms in interoperable ways. In this paper, we present a new lightweight web service called InterSensor Service allowing to simply connect to multiple IoT platforms, simulation specific data, databases, and simple files and retrieving their observations without worrying about data storage and the multitude of different APIs. The service encodes these observations “on-the-fly” according to the standardized external interfaces such as the OGC Sensor Observation Service and OGC SensorThings API. In this way, the heterogeneous observations can be analyzed and visualized in a unified way. The service can be deployed not only by the users to connect to different sources but also by providers and stakeholders to simply add further interfaces to their platforms realizing interoperability according to international standards. We have developed a Java-based implementation of the InterSensor Service, which is being offered free as open source software. The service is already being used in smart city projects and one application for the district Queen Elizabeth Olympic Park in London is shown in this paper.
Article
Full-text available
We sought to describe sounds of some of the common fishes suspected of producing unidentified air movement sounds in soundscape surveys of freshwater habitats in the New England region of North America. Soniferous behavior of target fishes was monitored in real time in the field in both natural and semi-natural environments by coupling Passive Acoustic Monitoring (PAM) with direct visual observation from shore and underwater video recording. Sounds produced by five species including, alewife (Alosa pseudoharengus, Clupeidae), white sucker (Catastomus commersonii, Catostomidae), brook trout (Salvelinus fontinalis, Salmonidae), brown trout (Salmo trutta, Salmonidae), and rainbow trout (Oncorhynchus mykiss, Salmonidae) were validated and described in detail for the first time. In addition, field recordings of sounds produced by an unidentified salmonid were provisionally attributed to Atlantic salmon (Salmo salar, Salmonidae). Sounds produced by all species are of the air movement type and appear to be species specific. Our data based on fishes in three distinct orders suggest the phenomenon may be more ecologically important than previously thought. Even if entirely incidental, air movement sounds appear to be uniquely identifiable to species and, hence, hold promise for PAM applications in freshwater and marine habitats.
Article
Omnidirectional vision has recently captured plenty of attention within the computer vision community. The popularity of cameras able to capture 360 $^{\circ }$ has increased in the last few years. A significant number of these cameras are composed of multiple individual cameras that capture images or videos, which are stitched together at a later postprocess stage. Stitching strategies have the complex objective of seamlessly joining the images, so that the viewer has the feeling the panorama was captured from a single location. Conventional approaches either assume that the world is a simple sphere around the camera, which leads to visible misalignments on the final panoramas, or use feature-based stitching techniques that do not exploit the rigidity of multicamera systems. In this paper, we propose a new stitching pipeline based on state-of-the-art techniques for both online and offline applications. The goal is to stitch the images taking profit of the available information on the multicamera system and the environment. Exploiting the spatial information of the scene helps to achieve significantly better results. While for the online case, sparse data can be obtained from a simultaneous localization and mapping process, for the offline case, it is estimated from a 3-D reconstruction of the scene. The information available is represented in depth maps, which provide all information in a condensed form and allow easy representation of complex shapes. The new pipelines proposed for both online and offline cases are compared, visually and numerically, against conventional approaches, using a real data set. The data set was collected in a challenging underwater scene with a custom-designed multicamera system. The results obtained surpass those of conventional approaches.
Article
Virtual Research Environments (VREs), also known as science gateways or virtual laboratories, assist researchers in data science by integrating tools for data discovery, data retrieval, workflow management and researcher collaboration, often coupled with a specific computing infrastructure. Recently, the push for better open data science has led to the creation of a variety of dedicated research infrastructures (RIs) that gather data and provide services to different research communities, all of which can be used independently of any specific VRE. There is therefore a need for generic VREs that can be coupled with the resources of many different RIs simultaneously, easily customised to the needs of specific communities. The resource metadata produced by these RIs rarely all adhere to any one standard or vocabulary however, making it difficult to search and discover resources independently of their providers without some translation into a common framework. Cross-RI search can be expedited by using mapping services that harvest RI-published metadata to build unified resource catalogues, but the development and operation of such services pose a number of challenges. In this paper, we discuss some of these challenges and look specifically at the VRE4EIC Metadata Portal, which uses X3ML mappings to build a single catalogue for describing data products and other resources provided by multiple RIs. The Metadata Portal was built in accordance to the e-VRE Reference Architecture, a microservice-based architecture for generic modular VREs, and uses the CERIF standard to structure its catalogued metadata. We consider the extent to which it addresses the challenges of cross-RI search, particularly in the environmental and earth science domain, and how it can be further augmented, for example to take advantage of linked vocabularies to provide more intelligent semantic search across multiple domains of discourse.
Article
Increasing interest in the acquisition of biotic and abiotic resources from within the deep sea (e.g. fisheries, oil-gas extraction, and mining) urgently imposes the development of novel monitoring technologies, beyond the traditional vessel-assisted, time-consuming, high-cost sampling surveys. The implementation of permanent networks of seabed and water-column cabled (fixed) and docked mobile platforms is presently enforced, to cooperatively measure biological features and environmental (physico-chemical) parameters. Video and acoustic (i.e. optoacoustic) imaging are becoming central approaches for studying benthic fauna (e.g. quantifying species presence, behaviour, and trophic interactions) in a remote, continuous, and prolonged fashion. Imaging is also being complemented by in situ environmental-DNA sequencing technologies, allowing the traceability of a wide range of organisms (including prokaryotes) beyond the reach of optoacoustic tools. Here, we describe the different fixed and mobile platforms of those benthic and pelagic monitoring networks, proposing at the same time an innovative roadmap for the automated computing of hierarchical ecological information of deep-sea ecosystems (i.e. from single species’ abundance and life traits, to community composition, and overall biodiversity).
Article
Acoustic-trawl surveys are an important tool for marine stock management and environmental monitoring of marine life. Correctly assigning the acoustic signal to species or species groups is a challenge, and recently trawl camera systems have been developed to support interpretation of acoustic data. Examining images from known positions in the trawl track provides high resolution ground truth for the presence of species. Here, we develop and deploy a deep learning neural network to automate the classification of species present in images from the Deep Vision trawl camera system. To remedy the scarcity of training data, we developed a novel training regime based on realistic simulation of Deep Vision images. We achieved a classification accuracy of 94% for blue whiting, Atlantic herring, and Atlantic mackerel, showing that automatic species classification is a viable and efficient approach, and further that using synthetic data can effectively mitigate the all too common lack of training data. © International Council for the Exploration of the Sea 2018. All rights reserved.
Article
Cameras are an important tool for sampling marine environments, providing estimates of fish size and abundance for ecological studies and resource management. In addition to length estimates, calibrated stereo-cameras can be used to provide precise locations (i.e. range and angle) of objects within the view frame of both cameras. We present a general method to exploit this information to estimate the density of fishes from stereo-camera measurements. This includes deriving the jointly sampled volume from stereo-camera images, accounting for the presence of the sea floor within the camera view, and accounting for detectability of fishes as a function of range. The method was demonstrated on a set of paired still images collected by stationary camera platforms deployed on the seafloor. Estimates of fish range, camera sampling volume, and seafloor position were used to model volumetric density as a function of range from the camera. A comparison of volumetric density estimates and traditional count metrics found that these metrics differ substantially in inferred relative species composition. The potential value of volumetric density for camera-based marine surveys, the potential for deriving absolute abundance, and the possibility of range-induced bias in count data are discussed.