ArticlePDF Available

Machine learning enables improved runtime and precision for bio-loggers on seabirds


Abstract and Figures

Unravelling the secrets of wild animals is one of the biggest challenges in ecology, with bio-logging (i.e., the use of animal-borne loggers or bio-loggers) playing a pivotal role in tackling this challenge. Bio-logging allows us to observe many aspects of animals’ lives, including their behaviours, physiology, social interactions, and external environment. However, bio-loggers have short runtimes when collecting data from resource-intensive (high-cost) sensors. This study proposes using AI on board video-loggers in order to use low-cost sensors (e.g., accelerometers) to automatically detect and record complex target behaviours that are of interest, reserving their devices’ limited resources for just those moments. We demonstrate our method on bio-loggers attached to seabirds including gulls and shearwaters, where it captured target videos with 15 times the precision of a baseline periodic-sampling method. Our work will provide motivation for more widespread adoption of AI in bio-loggers, helping us to shed light onto until now hidden aspects of animals’ lives. Joseph Korpela et al. demonstrate the use of machine-learning assisted bio-loggers on black-tailed gulls and streaked shearwaters. As video recording is only activated through variations in movement detected by low-cost accelerometers, this method represents improvements to runtime and precision over existing bio-logging technology.
This content is subject to copyright. Terms and conditions apply.
Machine learning enables improved runtime and
precision for bio-loggers on seabirds
Joseph Korpela1, Hirokazu Suzuki2, Sakiko Matsumoto2, Yuichi Mizutani2, Masaki Samejima1,
Takuya Maekawa 1, Junichi Nakai3& Ken Yoda 2
Unravelling the secrets of wild animals is one of the biggest challenges in ecology, with bio-
logging (i.e., the use of animal-borne loggers or bio-loggers) playing a pivotal role in tackling
this challenge. Bio-logging allows us to observe many aspects of animalslives, including their
behaviours, physiology, social interactions, and external environment. However, bio-loggers
have short runtimes when collecting data from resource-intensive (high-cost) sensors. This
study proposes using AI on board video-loggers in order to use low-cost sensors (e.g.,
accelerometers) to automatically detect and record complex target behaviours that are of
interest, reserving their deviceslimited resources for just those moments. We demonstrate
our method on bio-loggers attached to seabirds including gulls and shearwaters, where it
captured target videos with 15 times the precision of a baseline periodic-sampling method.
Our work will provide motivation for more widespread adoption of AI in bio-loggers, helping
us to shed light onto until now hidden aspects of animalslives. OPEN
1Graduate School of Information Science and Technology, Osaka University, Suita, Osaka 565-0871, Japan. 2Graduate School of Environmental Studies,
Nagoya University, Nagoya, Aichi 464-8601, Japan. 3Graduate School of Dentistry, Tohoku University, Sendai, Miyagi 980-8575, Japan.
COMMUNICATIONS BIOLOGY | (2020) 3:633 | | 1
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Animal-borne data loggers, i.e., bio-loggers, have revolu-
tionised the study of animal behaviour in the animals
natural environments, allowing researchers to gain great
insights into various aspects of the animalslives, such as their
social interactions and interactions with their environments13.
Although there have been extraordinary improvements in the
sensors and storage capacities of bio-loggers since the rst logger
was attached to a Weddell seal49, their data collection strategies
have remained relatively simple: record data continuously, record
data in bursts (e.g., periodic sampling), or use manually deter-
mined thresholds to detect basic collection criteria such as a
minimum depth, acceleration threshold, or illumination level10
17. However, these data collection strategies fall short when
attempting to collect data using resource-intensive sensors (e.g.,
video cameras) from specic animal behaviours, as they tend to
deplete all of the bio-loggersresources on non-target
behaviours18,19. This is especially true when working with ani-
mals such as birds, since the mass of a bio-logger is restricted to a
small fraction of the birds mass8, e.g., the video bio-loggers used
in this study weighed as little as 27 g (Methods), which greatly
restricts the maximum battery capacity.
In this study, we propose the concept of AI-assisted bio-log-
gers, which we will refer to as AI on Animals (AIoA), that can use
low-cost (i.e., non-resource-intensive) sensors to automatically
detect behaviours of interest in real time, allowing them to con-
ditionally activate high-cost (i.e., resource-intensive) sensors to
target those behaviours. Using AIoA, these bio-loggers can limit
their use of high-cost sensors to times when they are most likely
to capture the target behaviour, increasing their chances of suc-
cess by extending their runtime (e.g., from 2 h when continuously
recording video to up to 20 h when using AIoA). To the best of
our knowledge, the bio-loggers used in this study are the rst AI-
enabled bio-loggers ever to have been used in the wild. They
include an integrated video camera (high-cost sensor) along with
several low-cost sensors including an accelerometer and GPS unit.
AI-assisted bio-logging. Figure 1shows an example of how AIoA
can be used to capture videos of foraging behaviour when used on
a bio-logger attached to a black-tailed gull (Larus crassirostris). In
this example, the bio-logger pictured in Fig. 1a can detect fora-
ging activity using acceleration data (Fig. 1c) in order to extend its
runtime by only recording video during periods of that activity,
which are indicated by the green segments shown in Fig. 1d,
allowing it to capture target behaviours such as those pictured in
Fig. 1e, f (see also Supplementary Movies 1 and 2). See Methods
for a description of the machine learning algorithm used when
recognising behaviours on board the bio-loggers.
Experiment of black-tailed gulls. We evaluated the effectiveness
of our method by using AIoA-based camera control on board ten
bio-loggers (Supplementary Fig. 1) that were attached to black-
tailed gulls from a colony located on Kabushima Island near
Hachinohe City, Japan18, with the AI trained to detect possible
foraging behaviour based on acceleration data. Figure 2illustrates
the results based on an evaluation of the videos by the ecologists
participating in this study (see also Supplementary Movie 3). Of
the 95 videos collected using the naive method, only 2 contained
any target behaviour (2 possible foraging) while 93 videos con-
tained non-target behaviour (11 ying and 82 stationary), giving a
precision of about 0.02. In contrast, of the 184 videos collected
using the proposed method, 55 contained target behaviour (4
conrmed foraging and 51 possible foraging) and 129 contained
non-target behaviour (86 ying and 43 stationary), giving the
proposed method a precision of about 0.30, which is about 15
times the expected precision of random sampling given that the
target comprises only 1.6% of the dataset (see Fig. 2d and the
following investigation). This high precision also illustrates
the robustness of the proposed method, considering that it was
achieved in the wild using AI trained from data collected from
different hardware (Axy-Trek bio-loggers; see Methods) and in
some cases different positions on the animals (the back vs. the
abdomen). A two-sided Fishers exact test was performed to
compare the results for the proposed method and the naive
method (p=1.618 × 109, odds ratio =19.694, 95% condence
interval: [4.975170.549]).
Along with the video-based evaluation, we also analysed the
results for the black-tailed gulls by rst fully labelling the low-cost
sensor data (i.e., accelerometer data) collected by the bio-loggers
and then computing the precision, recall, and f-measure for 1-s
windows of sensor data, using the presence or absence of video
data to determine the classier output. Based on this full labelling
of the data, the proposed method achieved a precision of 0.27, a
recall of 0.56, and an f-measure of 0.37, while the naive method
achieved a precision of 0.00, a recall of 0.00 and an f-measure of
0.00. Using this full labelling of the data, we were also able to
compute an estimated distribution of the behaviours in the
collected data and found that the target behaviour (foraging)
comprised only about 1.62% of the 116 total hours of data
collected, with 9.96% corresponding to ying behaviour and the
remaining 88.43% corresponding to stationary behaviour. Thus,
the proposed method was able to capture about half of the target
behaviour and well outperformed the expected precision of
random sampling when the target comprises only 1.62% of the
Experiment of streaked shearwaters. Along with the above
evaluation that used acceleration data to train the machine
learning models, we also evaluated the proposed method when
training the models with GPS data. The bio-loggers were attached
to streaked shearwaters (Calonectris leucomelas) from a colony
located on Awashima Island in Niigata Prefecture, Japan. The AI
on these loggers was trained to detect area restricted search
(ARS), i.e., local ight activity in which the birds respond to
patchily distributed prey, based on features extracted from the
GPS data such as the distance travelled and average speed
(Supplementary Fig. 2). Along with the AIoA-based loggers, we
also deployed one naive logger that controlled the camera by
simply turning it on once every 30 min (periodic sampling). Both
the naive and AIoA-based loggers recorded 5-min duration
videos. Supplementary Fig. 3 shows an overview of these results
based on an evaluation of the videos by the ecologists partici-
pating in the study. See also Supplementary Movie 4. Of the 15
videos collected using the naive method, only 1 contained target
behaviour (ARS) while 14 videos contained only non-target
behaviour (1 transit and 13 stationary), giving a precision of
about 0.07. In contrast, of the 32 videos collected using the
proposed method, 19 contained target behaviour (ARS) and 13
contained non-target behaviour (7 transit and 6 stationary),
giving the proposed method a precision of about 0.59. A two-
sided Fishers exact test was performed to compare the results for
the proposed method and the naive method (p=1.085 × 103,
odds ratio =19.277, 95% condence interval: [2.390904.847]).
Along with the video-based evaluation, we also fully labelled
the low-cost sensor data (i.e., GPS data) collected by the loggers
and computed the precision, recall, and f-measure for 1-s
windows of sensor data, with the presence or absence of video
again used in lieu of classier output for each 1-s window. Based
on this evaluation, the proposed method achieved a precision of
0.65, a recall of 0.14, and an f-measure of 0.23, while the naive
2COMMUNICATIONS BIOLOGY | (2020) 3:633 | |
Content courtesy of Springer Nature, terms of use apply. Rights reserved
method achieved a precision of 0.13, a recall of 0.13 and an f-
measure of 0.13. We also computed an estimated distribution of
the behaviours in the dataset based on this full labelling, with the
target behaviour (ARS) found to constitute about 23.20% of the
approximately 59 h of data collected while transit behaviour
constituted about 34.36% and stationary behaviour constituted
about 42.43%. These results again indicate that the proposed
method was able to well outperform the expected precision of
random sampling given that the target behaviour comprises only
23.20% of the dataset.
In addition to our investigation of the effectiveness of the
proposed method, we also used the video captured by our bio-
loggers to analyse the relationship between ARS behaviour and
group formation in streaked shearwaters20.Acountwasdone
of the number of birds visible in each frame of video captured,
with each frame also labelled with the animals ID and its
current behaviour, i.e., ARS vs. non-ARS (transit or stationary).
Since the counts came from consecutive video frames, we
chose to analyse this data using a generalised linear mixed
model (GLMM) with Gaussian error distribution, with the
individual factors (based on the animal collecting the video)
treated as random effects. Using GLMM analysis, we found a
signicant difference in the number of birds visible during ARS
behaviour (t=41.919; df =4.820 × 104;p=2×10
16 (two-
sided), effect size (r2)=0.070). The number of data points for
ARS was 29,195 and for non-ARS was 23,741 (see Supplemen-
tary Fig. 4 for a chart of the dataset distribution). These results
support the hypothesis that streaked shearwaters forage in
ocks during ARS.
While several previous studies have proposed energy-efcient
machine learning algorithms for use with larger wearable devi-
ces2123, few have explored space-efcient models for use on
devices with extreme memory constraints, such as an ATme-
ga328P MCU2426. Even in these latter studies, the assumed
memory constraints were based on the total RAM available on the
devices (i.e., 2 KB), with two studies mostly generating models
larger than 2 KB in size25,26 and only one attempting to optimise
its model sizes to below 1 KB24. Furthermore, the models used in
that nal study were based on precomputed feature vectors24,
meaning that the space needed for feature extraction from raw
time-series data was not considered when calculating their
memory use.
In addition, several previous studies involving bio-loggers have
introduced trigger mechanisms that can be used to control when
high-cost sensors are activated based on coarse-level character-
isations of behaviour, e.g., underwater vs. surface activity, with
many of these studies focusing on controlling animal-borne
cameras such as the one used in this study10,11,13,2732. In con-
trast, our method can be used to distinguish between complex
behaviours at a ner scale, allowing us to target a specic beha-
viour; thereby greatly increasing the likelihood that interesting
behaviours will be captured. A case in point is the video captured
by AIoA of intraspecic kleptoparasitism at sea by a black-tailed
gull (Fig. 1e, Supplementary Movie 1). In addition, three of the
foraging videos captured using AIoA included footage of the gulls
foraging for ying insects over the sea (Supplementary Movie 3,
Fig. 2e, f), a previously unreported behaviour. Until now, insects
Target behavior
Fig. 1 Example use of our AI-assisted bio-logger in the eld. a A bio-logger attached to the abdomen of a black-tailed gull. The bird is shown with its
abdomen facing upward, with the bio-loggers camera lens facing towards the birds head. bAfter attaching the bio-logger, the bird is then released to roam
freely in its natural environment. cAn accelerometer (low-cost sensor) can be used to control the bio-loggers video camera (high-cost sensor) by
detecting body movements that are characteristic of the target behaviour (e.g., diving), activating the camera upon detection. dA GPS track captured by
the bio-logger as the bird was ying off the coast of Aomori Prefecture, Japan. The portion of the track highlighted in green shows where videos (e) and (f)
were captured, i.e., predicted target behaviour. (Map tiles by Stamen Design, under CC BY 3.0. Data by OpenStreetMap, under CC BY SA.) eFrames taken
from a video captured using AI on Animals (AIoA) that show intraspecic kleptoparasitism by a black-tailed gull. fFrames taken from a video captured
using AIoA that show a black-tailed gull catching a sh. Supplementary Data 1 provides source data of this gure.
COMMUNICATIONS BIOLOGY | (2020) 3:633 | | 3
Content courtesy of Springer Nature, terms of use apply. Rights reserved
found in traditionally used stomach-content analyses have been
considered to be preyed upon over land18. As this example shows,
by focusing the bio-loggersdata collection on a target behaviour,
we can increase the probability with which new ndings related to
that behaviour are discovered.
While the need for intelligent methods for supporting data
collection in the wild has motivated a wide range of previous
studies1,2,33, this is the rst study to our knowledge to use
machine learning on board animal-borne data loggers to support
data collection in the wild. Using machine learning, we can focus
data collection by high-cost sensors on interesting but infrequent
behaviours (e.g., 1.6% occurrence rate), greatly reducing the
number of bio-loggers required to collect the same amount of
data from interesting behaviours when compared to naive data
collection methods. Furthermore, since reducing or limiting the
weight of data loggers is an important aspect of experimental
design34, our approach can be used by researchers to reduce
battery requirements in order to either reduce device mass or
4COMMUNICATIONS BIOLOGY | (2020) 3:633 | |
Content courtesy of Springer Nature, terms of use apply. Rights reserved
increase the amount of data collected using a given device mass.
We anticipate this work will provide motivation for more wide-
spread research into AIoA, further improving its ability to control
resource-intensive sensors such as video cameras, microphones
and EEGs. Furthermore, AIoA could even be applied to other
applications such as controlling what data is transmitted from
devices over the low-bandwidth connections used with satellite
relay tags35 and detecting the poaching of endangered species
with real-time anti-poaching tags36.
Video bio-logger hardware. Supplementary Fig. 1a shows an example of the video
bio-loggers used during this study. It measures 85 mm length × 35 mm width × 15
mm height. The bio-loggers were attached to either the birds back or abdomen by
taping them to the birds feathers using waterproof tape. When attaching the bio-
logger to a birds abdomen, a harness made of Teon ribbon (Bally Ribbon Mills,
USA) was also used. When working with streaked shearwaters, the bio-loggers used
a 3.7 V 600 mAh battery and weighed approximately 2627 g. When working with
black-tailed gulls, the bio-loggers used a 3.7 V 720 mAh battery and weighed
approximately 30 g.
The bio-loggers are controlled by an ATmega328 MCU (32 KB programme
memory, 2 KB RAM) and have an integrated video camera (640 × 480, 15 FPS) that
can be controlled by the MCU, with the video data streamed directly to its own
dedicated storage. Note that digital cameras such as the one used in this bio-logger
have a delay of several seconds from powering on to when they can begin
recording, which in the case of our bio-logger resulted in a 2- to 3-s delay between
when the MCU signals the start of recording and the actual start of recording when
attempting to save energy by powering off the camera when not in use (see also
Yoshino et al.13 for another example of this camera delay). Our bio-loggers also
include several low-cost sensors that are controlled by the MCU (see
Supplementary Fig. 1b). Each of these sensors can be used by the MCU as input for
AIoA applications (e.g., camera control) and can be archived to long-term storage
for analysis upon device retrieval. The bio-loggers had an average battery life of
approximately 2 h when continuously recording video and approximately 20 h
when recording from all other (i.e., only low-cost) sensors.
Activity recognition method. We achieve AIoA by employing machine learning
to conduct activity (behaviour) recognition on board the bio-logging devices. We
do this by training an activity recognition model in advance using low-cost sensor
data that has been labelled by an ecologist to identify the behaviours that he/she
wants to capture. In the case of the black-tailed gulls, we use accelerometer-based
features since they can be used to detect the body movements of the animals with
only a small (e.g., 1-s) delay between when data collection begins and when
behaviours can rst be detected. Such features are often used when detecting body
movements in human activity recognition studies in order to recognise activities
such as running and eating37. For animal-based AI, such body movements can be
useful to detect similar types of behaviours, such as ying and foraging38. See Fig. 3
for an example of how such accelerometer-based features can be extracted from
raw data and used to train a decision tree classier model. The features were
extracted from 1-s windows of 25 Hz acceleration data, with the raw 3-axis
acceleration data converted to net magnitude of acceleration data prior to feature
extraction. The activity recognition processes were run once per second on the 1-s
windows of data, allowing us to detect target behaviours within about 1 s of their
start. See Supplementary Table 1 for descriptions and estimated sizes for all the
features used in this study. In addition, Supplementary Fig. 5a shows the
acceleration-based features ranked by their Normalised Gini Importance when
used to classify behaviours for black-tailed gulls.
The energy-saving microcontroller units (MCUs) in bio-loggers tend to have
limited memory and low computing capability, which makes it difcult to run the
computationally expensive processes needed for the pretrained machine learning
models on board the bio-loggers. Therefore, this study proposes a method for
generating space-efcient, i.e., programme memory efcient, decision tree classier
models that can be run on such MCUs. Decision trees are well suited for use on
MCUs, since the tree itself can be implemented as a simple hierarchy of conditional
logic statements, with most of the space needed for the tree being used by the
algorithms needed to extract meaningful features from the sensor data, such as the
variance or kurtosis of 1-s windows of data. In addition, since each data segment is
classied by following a single path through the tree from the root node to the leaf
node that represents that data segments estimated class, an added benet of using
a tree structure is that the MCU only needs to extract features as they are
encountered in the path taken through the tree, allowing the MCU to run only a
subset of the feature extraction processes for each data segment. However, the
feature extraction algorithms needed by the tree can be prohibitively large, e.g.,
kurtosis requires 680 bytes (Supplementary Table 1), when implemented on MCUs
that typically have memory capacities measured in kilobytes, e.g., 32 KB, which is
already largely occupied by the functions needed to log sensor data to storage.
Standard decision tree algorithms, e.g., scikit-learns default algorithm, build
decision trees that maximise classication accuracy with no option to weight the
features used in the tree based on a secondary factor such as memory usage39. The
trees are built starting from the root node, with each node in the tree choosing
from among all features the one feature that can best split the training data passed
to it into subsets that allow it to differentiate well between the different target
classes. A new child node is then created for each of the subsets of training data
output from that node, with this process repeating recursively until certain
stopping conditions are met, e.g., the subsets generated by a node reach a minimum
size. Figure 4b shows an example of a decision tree built using scikit-learns default
decision tree classier algorithm using the black-tailed gull data, which results in an
estimated memory footprint of 1958 bytes (Supplementary Table 1). Note that
since the basic system functions needed to record sensor data to long-term storage
already occupy as much as 95% of the bio-loggersash memory, incorporating a
decision tree with this large of a memory footprint can cause the programme to
exceed the bio-loggers memory capacity (see the bio-logger source code distributed
as Supplementary Software for more details).
In this study, we propose a method for generating decision tree classiers that
can t in bio-loggers with limited programme memory (e.g., 32 KB) that is based
on the random forests algorithm40, which is a decision tree algorithm that injects
randomness into the trees it generates by restricting the features compared when
creating the split at each node in a tree to a randomly selected subset of the
features, as opposed to the standard decision tree algorithm that compares all
possible features at each node, as was described above. Our method modies the
original random forests algorithm by using weighted random selection when
choosing the subset of features to compare when creating each node. Figure 4a
illustrates the weighted random selection process used by our method. We start by
assigning each feature a weight that is proportional to the inverse of its size. We
then use these weights to perform weighted random selection when selecting a
group of features to consider each time we create a new node in the tree, with the
feature used at that node being the best candidate from amongst these randomly
selected features.
Using our method for weighted random selection of nodes described above, we
are then able to generate randomised trees that tend to use less costly features.
When generating these trees, we can estimate the size of each tree based on the
sizes of the features used in the tree and limit the overall size by setting a threshold
and discarding trees above that threshold. Figure 4c shows an example batch of
trees output by our method where we have set a threshold size of 1000 bytes. We
can then select a single tree from amongst these trees that gives our desired balance
of cost to accuracy. In this example, we have selected the tree shown in Fig. 4d
based on its high estimated accuracy. Comparing this tree to Fig. 4b, we can see
that our method generated a tree that is 42% the size of the default tree while
maintaining close to the same estimated accuracy. We developed our method based
on scikit-learns (v.0.20.0) RandomForestClassier.
In addition, in order to achieve robust activity recognition, our method also has
the following features: (i) robustness to sensor positioning, (ii) robustness to noise,
and (iii) reduction of sporadic false positives. Note that robustness to noise and
positioning are extremely important when deploying machine learning models on
bio-loggers, as the models will likely be generated using data collected in previous
years, possibly using different hardware and methods of attachment. While there is
a potential to improve prediction accuracy by removing some of these variables,
Fig. 2 Results of AI video control for black-tailed gulls. a GPS tracks marked with the locations of videos collected by bio-loggers using the proposed
method. The letters e and f indicate the locations where the video frames shown in (e) and (f) were collected. bGPS tracks marked with the locations of
videos collected by bio-loggers using the naive method (periodic sampling). cExamples of acceleration data (shown as magnitude of acceleration)
collected around the time of video camera activation on bio-loggers using the proposed method. Cells Foraging (e) and Foraging (f) show the acceleration
data that triggered the camera to record the video frames shown in (e) and (f). Note that the camera is activated based on a 1-s window of data, which
corresponds to a window extracted from around the 2- to 4-s mark for each example. As shown in these charts, while acceleration data can be used to
detect the target behaviour, it is difcult to avoid false positives due to the similarity between the target behaviour and other anomalous movements in the
sensor data. dEstimated distribution of behaviours based on the 116 h of acceleration data collected. eFrames taken from video captured using AIoA of a
black-tailed gull catching an insect in mid-air while ying over the ocean. fFrames taken from video captured using AI on Animals (AIoA) of a black-tailed
gull plucking an insect off the ocean surface. Supplementary Data 1 provides source data of this gure.
COMMUNICATIONS BIOLOGY | (2020) 3:633 | | 5
Content courtesy of Springer Nature, terms of use apply. Rights reserved
e.g., by collecting from the same animal multiple times using the same hardware,
moving to more animal-dependent models is generally not practical as care must
be taken to minimise the handling of each animal along with the amount of time
the animals spend with data loggers attached34. See Robust activity recognition
for more details.
GPS features. Due to the low resolution of GPS data (e.g., metre-level accuracy),
GPS-based features cannot detect body movements with the same precision as
acceleration-based features, but are useful when analysing patterns in changes in an
animals location as it traverses its environment. For animal-based AI, these fea-
tures can be used to differentiate between different large-scale movement patterns,
such as transit versus ARS. In this study, we used GPS-based features to detect ARS
by streaked shearwaters. These features were extracted once per minute from 1/
60th Hz GPS data using 10-min windows. Supplementary Fig. 5b shows these
features ranked by their Normalised Gini Importance when used to classify
behaviours for streaked shearwaters. Supplementary Fig. 2 shows an example of
two such 10-min windows that correspond to target (ARS) and non-target (transit)
behaviour, along with several examples of GPS-based features extracted from those
windows. Supplementary Table 1 describes all the GPS features used in this study
along with their estimated sizes when implemented on board our bio-logger. Note
that the variance and mean cross features were extracted after rst rotating the GPS
positions around the mean latitude and longitude values at angles of 22.5°, 45.0°,
67.5°, and 90.0° in order to nd the orientation that maximised the variance in the
longitude values (see Supplementary Table 1, feature Y: rotation). This was done to
provide some measure of rotational invariance to these values without the need for
a costly procedure such as principal component analysis. The primary and sec-
ondary qualiers for these features refer to whether the feature was computed on
the axis with maximised variance vs. the perpendicular axis, respectively.
Robust activity recognition. In this study, we also incorporated two methods for
improving the robustness of the recognition processes in the eld. First, we
addressed how loggers can be attached to animals at different positions and
orientations, such as on the back to maximise GPS reception or on the abdomen to
improve the cameraseld of view during foraging. For example, during our case
study involving black-tailed gulls, the AI models were trained using data collected
from loggers mounted on the birdsbacks, but in many cases were used to detect
target behaviour on board loggers mounted on the birdsabdomens. We achieved
this robustness to positioning by converting the three-axis accelerometer data to
net magnitude of acceleration values, thereby removing the orientation information
from the data. To test the robustness of the magnitude data, we evaluated the
difference in classication accuracy between raw three-axis acceleration data and
magnitude of acceleration data when articial rotations of the collection device
were introduced into the test data. In addition, we also evaluated the effectiveness
of augmenting the raw three-axis data with articially rotated data as an alternative
to using the magnitude of acceleration data. These results are shown in Supple-
mentary Fig. 6a. Note that the results for the magnitude of acceleration data are
constant across all levels of test data rotation, since the magnitudes are unaffected
by the rotations. Based on these results, we can see that extracting features from
magnitude of acceleration data allows us to create features that are robust to the
rotations of the device that can result from differences in how the device is attached
to an animal.
Next, we addressed the varying amount of noise that can be introduced into the
sensor data stemming from how loggers are often loosely attached to a birds
feathers. We achieved this noise robustness by augmenting our training dataset
with copies of the dataset that were altered with varying levels of random articial
noise, with this noise added by multiplying all magnitude of acceleration values in
each window of data by a random factor. We tested the effect of this augmentation
by varying the amount of articial noise added to our training and testing data and
observing how the noise levels affected performance (see Supplementary Fig. 6b).
Based on these results, the training data used for eldwork was augmented using
the 0.2 level. Note that at higher levels of simulated noise (test noise greater than
0.15) the training noise settings of 0.25 and 0.3 both appear to outperform the
0.2 setting. However, since these results are based solely on laboratory simulations,
we chose to use the more conservative setting of 0.2 in the eld.
Reduction of sporadic false positives. When activating the camera to capture
target behaviour, it is possible to reduce the number of false positives (i.e., increase
our condence in the classiers output) by considering multiple consecutive
outputs from the classier before camera activation. We accomplish this using two
crest <= 0.69
mean cross <= 6.5
crest <= 0.149 stationary
energy <= 1.141 foraging
stationary foraging
Fig. 3 Generating decision trees from acceleration data. a We start by converting the raw three-axis data (row one) into magnitude of acceleration values
(row two) and segmenting the data into 1-s windows. We then extract the ACC features listed in Supplementary Table 1 from each window. Rows three and
four show examples of the features extracted, which can be used to differentiate between the behaviours. For example, Crest can be used to identify Flying
behaviour, since its values are higher for Flying than for the other two behaviours. bAn example decision tree generated from the feature values shown in
the lower two rows of (a), with each leaf (grey) node representing a nal predicted class for a 1-s segment of data. Supplementary Data 1 provides source
data of this gure.
6COMMUNICATIONS BIOLOGY | (2020) 3:633 | |
Content courtesy of Springer Nature, terms of use apply. Rights reserved
methods. In the rst, we assume that the classier can reliably detect the target
behaviour throughout its duration, allowing us to increase our condence in the
classiers output by requiring multiple consecutive detections of the target
behaviour before activating the camera. We employed this method when detecting
ARS behaviour for streaked shearwaters using GPS data, since the characteristics of
the GPS data that allow for detection of the target behaviour were expected to be
consistent throughout its duration, with the number of consecutive detections
required set to 2.
In the second method, we assume that the classier cannot reliably detect the
target behaviour throughout its duration, since the actions corresponding to the
target behaviour that the classier can reliably detect occur only sporadically
throughout its duration. In this case, we can instead consider which behaviours
were detected immediately prior to the target behaviour. When controlling the
camera for black-tailed gulls, we assume that detection of the target behaviour
(foraging) is more likely to be a true positive after detecting ying behaviour, since
the birds typically y when searching for their prey. Therefore, we required that the
logger rst detect ve consecutive windows of ying behaviour to enter a ying
state in which it would activate the camera immediately upon detecting foraging.
This ying state would time out after ten consecutive windows of stationary
behaviour. Note that in this case, while the intervals of detectable target behaviour
may be short and sporadic, the overall duration of the target behaviour is still long
enough that we can capture video of the behaviour despite the delay between
behaviour detection and camera activation (see Video bio-logger hardwarefor
Procedures of experiment of black-tailed gulls. We evaluated the effectiveness of
our method by using AIoA-based camera control on board ten bio-loggers that
were attached to black-tailed gulls (on either the birds abdomen or back) from a
colony located on Kabushima Island near Hachinohe City, Japan18, with the AI
trained to detect possible foraging behaviour based on acceleration data. The
possible foraging events were identied based on dips in the acceleration data. The
training data used for the AI was collected at the same colony in 2017 using Axy-
Trek bio-loggers (TechnoSmArt, Roma, Italy). These Axy-Trek bio-loggers were
mounted on the animalsbacks when collecting data. Along with the AIoA-based
bio-loggers, three bio-loggers were deployed using a naive method (periodic
sampling), with the cameras controlled by simply activating them once every 15
min. All 13 loggers recorded 1-min duration videos.
Sample size was determined by the time available for deployment and the
availability of sensor data loggers. The birds were captured alive at their nests by
Resulng subset
Weighted random selecon
Features (size in bytes)
Fig. 4 Generating space-efcient trees. a Our process for the weighted random selection of features. We start with a list of features along with their
required programme memory sizes in bytes (rst panel). Each feature is assigned a weight proportional to the inverse of its size, illustrated using a pie
chart where each feature has been assigned a slice proportional to its weight (second panel). We then perform weighted random selection to choose the
subset of features that will be used when creating a new node in the tree. In this example, we have randomly placed four dots along the circumference of
the circle to simulate the selection of four features (second panel). The resulting subset of features will then be compared when making the next node in
the decision tree (third panel). bExample decision tree built using scikit-learns default decision tree classier algorithm using the black-tailed gull data
described in Methods. Each node is coloured based on its corresponding features estimated size in bytes when implemented on board the bio-logger
(scale shown in the colour bar). cSeveral space-efcient decision trees generated using the proposed method from the same data used to create the tree in
(b). dExample space-efcient tree selected from the trees shown in (c) that costs much less than the default tree in (b) while maintaining almost the same
COMMUNICATIONS BIOLOGY | (2020) 3:633 | | 7
Content courtesy of Springer Nature, terms of use apply. Rights reserved
hand prior to logger deployment and subsequent release. Loggers were tted
externally within 10 min to minimise disturbance. Logger deployment was
undertaken by the ecologists participating in this study. Loggers that suffered
hardware failures (e.g., due to the failure of the waterproong material used on
some loggers) were excluded.
Ethics statement. All experimental procedures were approved by the Animal
Experimental Committee of Nagoya University. Black-tailed gulls: the procedures
were approved by the Hachinohe city mayor, and the Aomori Prefectural Gov-
ernment. Streaked shearwaters: the study was conducted with permits from the
Ministry of the Environment, Japan.
Statistics and reproducibility. Fishers exact tests were done using the exact2x2
package (v. 1.6.3) of R (v. 3.4.3). The GLMM analysis was conducted using the
lmerTest package (v. 2.036) of R (v. 3.4.3). In regards to reproducibility, no
experiments as such were conducted, rather our data are based on tracked
movements of individual birds.
Reporting summary. Further information on research design is available in the Nature
Research Reporting Summary linked to this article.
Data availability
The data from this study are available from the corresponding author upon reasonable
Code availability
Our code run on the bio-loggers is written in C++. Code needed to replicate our
ndings and hardware diagrams of the bio-loggers used in this study can be found at:
Received: 9 May 2020; Accepted: 7 October 2020;
1. Hussey, N. E. et al. Aquatic animal telemetry: a panoramic window into the
underwater world. Science 348, 1255642 (2015).
2. Kays, R., Crofoot, M. C., Jetz, W. & Wikelski, M. Terrestrial animal tracking as
an eye on life and planet. Science 348, aaa2478 (2015).
3. Yonehara, Y. et al. Flight paths of seabirds soaring over the ocean surface
enable measurement of ne-scale wind speed and direction. Proc. Natl Acad.
Sci. USA 113, 90399044 (2016).
4. Kooyman, G. L. Techniques used in measuring diving capacities of Weddell
Seals. Polar Rec. 12, 391394 (1965).
5. Fehlmann, G. & King, A. J. Bio-logging. Curr. Biol. 26, R830R831 (2016).
6. Cooke, S. J. et al. Biotelemetry: a mechanistic approach to ecology. Trends
Ecol. Evol. 19, 334343 (2004).
7. Ropert-Coudert, Y. & Wilson, R. P. Trends and perspectives in animal-
attached remote sensing. Front. Ecol. Environ. 3, 437444 (2005).
8. Yoda, K. Advances in bio-logging techniques and their application to study
navigation in wild seabirds. Adv. Robot. 33, 108117 (2019).
9. Rutz, C. & Troscianko, J. Programmable, miniature video-loggers for
deployment on wild birds and other wildlife. Methods Ecol. Evol. 4, 114122
10. Watanuki, Y. et al. Underwater images from bird-borne cameras provide clue
to poor breeding success of Shags in 2005. Br. Birds 100, 466470 (2007).
11. Volpov, B. L. et al. Identication of prey captures in Australian fur seals
(Arctocephalus pusillus doriferus) using head-mounted accelerometers: eld
validation with animal-borne video cameras. PLoS ONE 10, e0128789 (2015).
12. Rutz, C., Bluff, L. A., Weir, A. A. S. & Kacelnik, A. Video cameras on wild
birds. Science 318, 765 (2007).
13. Yoshino, K. et al. Acceleration-triggered animal-borne videos show a
dominance of sh in the diet of female northern elephant seals. J. Exp. Biol.
223, jeb212936 (2020).
14. Gómez-Laich, A., Yoda, K. & Quintana, F. Insights into the foraging behavior
of Magellanic penguins (Spheniscus magellanicus). Waterbirds 41, 332336
15. Austin, R. E. et al. A sex-inuenced exible foraging strategy in the
magnicent frigatebird Fregata magnicens.Mar. Ecol. Prog. Ser. 611,
203214 (2019).
16. Thiebot, J. B. et al. Jellysh and other gelata as food for four penguin species
insights from predator-borne videos. Front. Ecol. Environ. 15, 437441 (2017).
17. Volpov, B. L. et al. Dive characteristics can predict foraging success in
Australian fur seals (Arctocephalus pusillus doriferus) as validated by animal-
borne video. Biol. Open 5, 262271 (2016).
18. Yoda, K., Tomita, N., Mizutani, Y., Narita, A. & Niizuma, Y. Spatio-temporal
responses of black-tailed gulls to natural and anthropogenic food resources.
Mar. Ecol. Prog. Ser. 466, 249259 (2012).
19. Watanabe, Y. Y. & Takahashi, A. Linking animal-borne video to
accelerometers reveals prey capture variability. Proc. Natl Acad. Sci. USA 110,
21992204 (2013).
20. Thiebault, A. & Tremblay, Y. Splitting animal trajectories into ne-scale
behaviorally consistent movement units: breaking points relate to
external stimuli in a foraging seabird. Behav. Ecol. Sociobiol. 67, 10131026
21. Korpela, J. et al. An energy-aware method for the joint recognition of activities
and gestures using wearable sensors. In Proc. 2015 ACM International
Symposium on Wearable Computers.
22. Le Roux, S. P., Wolhuter, R. & Niesler, T. Energy-aware feature and model
selection for onboard behavior classication in low-power animal borne
sensor applications. IEEE Sens. J. 19, 27222734 (2019).
23. Rault, T., Bouabdallah, A., Challal, Y. & Marin, F. A survey of energy-efcient
context recognition systems using wearable sensors for healthcare
applications. Pervasive Mob. Comput. 37,2344 (2017).
24. Kumar, A., Goyal, S. & Varma, M. Resource-efcient machine learning in 2
KB RAM for the Internet of Things. In Proc. 34th International Conference on
Machine Learning, 19351944 (2017).
25. Gupta, C. et al. ProtoNN: compressed and accurate kNN for resource-scarce
devices. In Proc. 34th International Conference on Machine Learning,
13311340 (2017).
26. Kusupati, A. et al. FastgRNN: a fast, accurate, stable and tiny kilobyte sized
gated recurrent neural network. Adv. Neural Inf. Process. Syst. 90179028
27. Troscianko, J. & Rutz, C. Activity proles and hook-tool use of New
Caledonian crows recorded by bird-borne video cameras. Biol. Lett. 11,
20150777 (2015).
28. Beringer, J., Millspaugh, J. J., Sartwell, J. & Woeck, R. Real-time video
recording of food selection by captive white-tailed deer. Wildl. Soc. Bull. 32,
648654 (2004).
29. Goldbogen, J. A. et al. Using digital tags with integrated video and inertial
sensors to study moving morphology and associated function in large aquatic
vertebrates. Anat. Rec. 300, 19351941 (2017).
30. Boness, D. J., Bowen, W. D., Buhleier, B. M. & Marshall, G. J. Mating tactics
and mating system of an aquatic-mating pinniped: the harbor seal, Phoca
vitulina. Behav. Ecol. Sociobiol. 61, 119130 (2006).
31. Nishiumi, N. et al. A miniaturized threshold-triggered acceleration data-logger
for recording burst movements of aquatic animals. J. Exp. Biol. 221, jeb172346
32. Brown, D. D. et al. Accelerometer-informed GPS telemetry: reducing the
trade-off between resolution and longevity. Wildl. Soc. Bull. 36, 139146
33. Le Maho, Y. et al. Rovers minimize human disturbance in research on wild
animals. Nat. Methods 11, 12421244 (2014).
34. Williams, H. J. et al. Optimizing the use of biologgers for movement ecology
research. J. Anim. Ecol. 89, 186206 (2020).
35. Cox, S. L. et al. Processing of acceleration and dive data on-board satellite
relay tags to investigate diving and foraging behaviour in free-ranging marine
predators. Methods Ecol. Evol. 9,6477 (2018).
36. ODonoghue, P. & Rutz, C. Real-time anti-poaching tags could help prevent
imminent species extinctions. J. Appl. Ecol. 53,510 (2016).
37. Bao, L. & Intille, S. S. Activity recognition from user-annotated acceleration
data. Int. Conf. Pervasive Comput.
6_1 (2004).
38. Yoda, K. et al. A new technique for monitoring the behaviour of free-ranging
Adelie penguins. J. Exp. Biol. 204, 685690 (2001).
39. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn.
Res. 12, 28252830 (2011).
40. Breiman, L. Random Forests. Mach. Learn. 45,532 (2001).
41. AI-enabled bio-logger codes. GitHub
We thank Rory P. Wilson, Flavio Quintana, Agustina Gómez Laich, Takashi Yamamoto,
Yasue Kishino, and Kazuya Murao for suggestions and comments on this work. This
study is partially supported by JSPS KAKENHI JP16H06539, JP16H06541 and
8COMMUNICATIONS BIOLOGY | (2020) 3:633 | |
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Author contributions
J.K. performed the method design, software implementation, data collection, data ana-
lysis and paper writing. H.S., S.M. and Y.M. performed the data collection and data
analysis. M.S. performed the software implementation and data collection. T.M. con-
ceived and directed the study, and performed method design, data collection, data
analysis and paper writing. J.N. performed the hardware design. K.Y. performed data
collection, data analysis and paper writing.
Competing interests
The authors declare no competing interests.
Additional information
Supplementary information is available for this paper at
Correspondence and requests for materials should be addressed to T.M.
Reprints and permission information is available at
Publishers note Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional afliations.
Open Access This article is licensed under a Creative Commons
Attribution 4.0 International License, which permits use, sharing,
adaptation, distribution and reproduction in any medium or format, as long as you give
appropriate credit to the original author(s) and the source, provide a link to the Creative
Commons license, and indicate if changes were made. The images or other third party
material in this article are included in the articles Creative Commons license, unless
indicated otherwise in a credit line to the material. If material is not included in the
articles Creative Commons license and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this license, visit
© The Author(s) 2020
COMMUNICATIONS BIOLOGY | (2020) 3:633 | | 9
Content courtesy of Springer Nature, terms of use apply. Rights reserved
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
... Usually, unsupervised approaches are used when validation data are not available, in contrast to a supervised approach making use of pre-labelled known behavioural activities recorded. While unsupervised approaches (e.g., Expectation Maximisation, k-means) independently detect behaviours and allow for the detection of unknown behaviours 20,26 , supervised approaches (e.g., Random Forest, Support Vector Machine) are fast and reliable on known behaviours 27,28 . Whilst both approaches have their strengths, they also have their individual and shared weaknesses and can be complementary. ...
... Finally, training datasets accounting for behavioural variability could be applied to increasingly larger databases and/or used for real-time processing of accelerometer data on board of bio-logging devices to facilitate transmission via satellite systems 27 . While our framework has been tested on two penguin species, it is transferable to other species and systems. ...
Full-text available
Animal-borne tagging (bio-logging) generates large and complex datasets. In particular, accelerometer tags, which provide information on behaviour and energy expenditure of wild animals, produce high-resolution multi-dimensional data, and can be challenging to analyse. We tested the performance of commonly used artificial intelligence tools on datasets of increasing volume and dimensionality. By collecting bio-logging data across several sampling seasons, datasets are inherently characterized by inter-individual variability. Such information should be considered when predicting behaviour. We integrated both unsupervised and supervised machine learning approaches to predict behaviours in two penguin species. The classified behaviours obtained from the unsupervised approach Expectation Maximisation were used to train the supervised approach Random Forest. We assessed agreement between the approaches, the performance of Random Forest on unknown data and the implications for the calculation of energy expenditure. Consideration of behavioural variability resulted in high agreement (> 80%) in behavioural classifications and minimal differences in energy expenditure estimates. However, some outliers with < 70% of agreement, highlighted how behaviours characterized by signal similarity are confused. We advise the broad bio-logging community, approaching these large datasets, to be cautious when upscaling predictions, as this might lead to less accurate estimates of behaviour and energy expenditure.
... For example, one could downsample the high-resolution image-based data to determine the optimal sampling frequency for bio-logging studies, using the videos to verify that the resulting data capture the target behaviours. Recording the behaviour of instrumented animals could also provide sensor ground truth data and aid in the development of more energy efficient, behaviourally activated sensors (Korpela et al., 2020;Yu et al., 2021). (Koci et al., 2020) or resource quality and accessibility (Jennewein et al., 2021) which would also be informative for analysing recorded animal behaviour. ...
Full-text available
Methods for collecting animal behaviour data in natural environments, such as direct observation and biologging, are typically limited in spatiotemporal resolution, the number of animals that can be observed and information about animals' social and physical environments. Video imagery can capture rich information about animals and their environments, but image-based approaches are often impractical due to the challenges of processing large and complex multi-image datasets and transforming resulting data, such as animals' locations, into geographical coordinates. We demonstrate a new system for studying behaviour in the wild that uses drone-recorded videos and computer vision approaches to automatically track the location and body posture of free-roaming animals in georeferenced coordinates with high spatiotemporal resolution embedded in contemporaneous 3D landscape models of the surrounding area. We provide two worked examples in which we apply this approach to videos of gelada monkeys and multiple species of group-living African ungulates. We demonstrate how to track multiple animals simultaneously, classify individuals by species and age-sex class, estimate individuals' body postures (poses) and extract environmental features, including topography of the landscape and animal trails. By quantifying animal movement and posture while reconstructing a detailed 3D model of the landscape, our approach opens the door to studying the sensory ecology and decision-making of animals within their natural physical and social environments.
... One way to increase ecological velocity is to use artificial intelligence (AI), with machine learning algorithms processing continuous streams of data to provide inference and insights (Christin et al., 2019;Goodwin et al., 2022;Makiola et al., 2020). Early applications in this realm include real-time event identification in biologgers for tracking animal movement (Korpela et al., 2020), automatic identification and counting of animals from camera trap images (Norouzzadeh et al., 2021) and automated analysis of animal behaviour from video data (Williams & DeLeon, 2020). ...
Full-text available
Ecological research and monitoring need to be able to rapidly convey information that can form the basis of scientifically sound management. Automated sensor systems, especially if combined with artificial intelligence, can contribute to such rapid high resolution data retrieval. Here, we explore the prospects of automated methods to generate insights for seabirds, which are often monitored for their high conservation value and for being sentinels for marine ecosystem changes. We have developed a system of video surveillance combined with automated image processing, which we apply to common murres Uria aalge. The system uses a deep learning algorithm for object detection (YOLOv5)that has been trained on annotated images of adult birds, chicks and eggs, and outputs time, location, size and confidence level of all detections, frame-by-frame, in the supplied video material. A total of 144 million bird detections were generated from a breeding cliff over three complete breeding seasons(2019–2021). We demonstrate how object detection can be used to accurately monitor breeding phenology and chick growth. Our automated monitoring approach can also identify and quantify rare events that are easily missed in traditional monitoring, such as disturbances from predators. Further, combining automated video analysis with continuous measurements from a temperature logger allows us to study impacts of heat waves on nest attendance in high detail. Our automated system thus produces comparable, and in several cases significantly more detailed, data than those generated from observational fieldstudies. By running in real time on the camera streams, it has the potential to supply researchers and managers with high-resolution up-to-date information on seabird population status. We describe how the system can be modified to fit various types of ecological research and monitoring goals and thereby provide up-to-date support for conservation and ecosystem management.
... One way to solve this problem is to use recently developed video loggers with eventtrigger function. In marine animals, efficient recording of feeding behavior has been achieved by acceleration-triggered video loggers (Yoshino et al. 2020) and AI-assisted video loggers (Korpela et al. 2020). Such devices would be helpful in observing low-frequency predation events and examining pursuit and antipredator strategies in various animals. ...
... For example, one could downsample the high resolution image-based data to determine the optimal sampling frequency for bio-logging studies, using the videos to verify that the resulting data capture the target behaviors. Recording and quantifying the behavior of instrumented animals could also aid in the development of behaviorally activated "smart" sensors (Korpela et al., 2020;Yu, 2021). ...
... Recently, intelligent biologgers, termed Logbots, have been developed. They are operated by machine learning algorithms implemented in a microcontroller that autonomously regulates the timing of logging, thereby minimizing battery consumption [45]. An intelligent Logbot-controlled neurologger is expected to unveil the neural correlate of long-distance movements in the future. ...
Full-text available
Simultaneous monitoring of animal behavior and neuronal activity in the brain enables us to examine the neural underpinnings of behaviors. Conventionally, the neural activity data are buffered, amplified, multiplexed, and then converted from analog to digital in the head-stage amplifier, following which they are transferred to a storage server via a cable. Such tethered recording systems, intended for indoor use, hamper the free movement of animals in three-dimensional (3D) space as well as in large spaces or underwater, making it difficult to target wild animals active under natural conditions; it also presents challenges in realizing its applications to humans, such as the Brain–Machine Interfaces (BMI). Recent advances in micromachine technology have established a wireless logging device called a neurologger, which directly stores neural activity on ultra-compact memory media. The advent of the neurologger has triggered the examination of the neural correlates of 3D flight, underwater swimming of wild animals, and translocation experiments in the wild. Examples of the use of neurologgers will provide an insight into understanding the neural underpinnings of behaviors in the natural environment and contribute to the practical application of BMI. Here we outline the monitoring of the neural underpinnings of flying and swimming behaviors using neurologgers. We then focus on neuroethological findings and end by discussing their future perspectives.
... Cela ne nécessiterait plus de recapturer l'animal et cette avancée ouvrirait de nouveaux horizons dans l'étude des animaux migrateurs. L'implémentation d'algorithme d'apprentissage dans des balises commence à peine à se développer avec une première étude très encourageante sur les oiseaux où un CART (Annexe 2 pour définition) a pu être implémenté à un accéléromètre afin d'activer une caméra embarquée en fonction des comportements exprimés(Korpela et al., 2020). Ainsi, l'implémentation d'algorithme basé sur du deep learning pourrait bien être les nouveaux enjeux du bio-logging. ...
La Plateforme intergouvernementale sur la biodiversité et les services écosystémiques (IPBES) alertait en 2019 qu’un million d’espèces animales et végétales sont désormais menacées d’extinction. L’étude du comportement animal peut apporter une contribution significative à la conservation en améliorant les connaissances sur l’écologie des espèces et permettre l’élaboration de mesures de protection adaptées et effectives. Ainsi, l’objectif de cette thèse était de développer une méthode d’identification automatique des comportements à partir de bio-loggers pour des espèces menacées difficiles à observer ; les tortues marines. A partir du déploiement de caméras embarquées couplées à des accéléromètres, gyroscopes et capteurs de pression sur les tortues vertes en Martinique, un puissant réseau de neurones a été entrainé pour répondre à cet objectif. L’application de cette méthode sur cette même population via le déploiement de bio-loggers sur plusieurs jours nous a permis d’identifier des mesures de protection concrètes et adaptées aux enjeux économiques de la région.
This paper presents the sensitivity characteristics of a microelectromechanical systems (MEMS) piezoresistive cantilever and nanohole array used in a waterproof airflow sensor. Previously, a Pitot tube-type waterproof airflow sensor was developed for seabird biologging. Built-in MEMS piezoresistive cantilevers are used as the differential pressure sensing elements. The waterproof function is achieved using nanohole arrays via Laplace pressure. However, the mechanism underlying sensitivity reduction when nanohole arrays are attached is unclear. Here, we experimentally and theoretically verified that the specific constant, which determines the airflow rate through the cantilever and nanohole array, affects sensitivity reduction. An airflow sensor with a small sensitivity reduction was achieved based on the calculated results using an appropriate cantilever and nanohole arrays. We demonstrated that the proposed method for estimating the sensitivity reduction is useful for designing waterproof airflow sensors using cantilever-type differential pressure sensor elements.
Areas at which seabirds forage intensively can be discriminated by tracking the individuals' at‐sea movements. However, such tracking data may not accurately reflect the birds' exact foraging locations. In addition to tracking data, gathering information on the dynamic body acceleration of individual birds may refine inferences on their foraging activity. Our aim was to classify the foraging behaviors of surface‐feeding seabirds using data on their body acceleration and use this signal to discriminate areas where they forage intensively. Accordingly, we recorded the foraging movements and body acceleration data from seven and ten black‐tailed gulls Larus crassirostris in 2017 and 2018, respectively, using GPS loggers and accelerometers. By referring to video footage of flying and foraging individuals, we were able to classify flying (flapping flight, gliding and hovering), foraging (surface plunging, hop plunging and swimming) and maintenance (drifting, preening, etc.) behaviors using the speed, body angle and cycle and amplitude of body acceleration of the birds. Foraging areas determined from acceleration data corresponded roughly with sections of low speed and area‐restricted searching (ARS) identified from the GPS tracks. However, this study suggests that the occurrence of foraging behaviors may be overestimated based on low‐speed trip sections, because birds may exhibit long periods of reduced movement devoted to maintenance. Opposite, the ARS‐based approach may underestimate foraging behaviors since birds can forage without conducting an ARS. Therefore, our results show that the combined use of accelerometers and GPS tracking helps to adequately determine the important foraging areas of black‐tailed gulls. Our approach may contribute to better discriminate ecologically or biologically significant areas in marine environments.
Full-text available
To successfully exploit resources, animals must be adapted to operate under phenotypic and environmental constraints. The strategies that predators use to locate prey are therefore diverse, particularly for breeding central-place foragers that must balance investment in reproduction and self-maintenance. Magnificent frigatebirds Fregata magnificens are tropical seabirds with intriguing morphology and feeding ecology, which display strikingly unequal levels of parental care (males deserting offspring months before females). These unusual traits can better help us understand the links between movement behaviour and breeding strategies in this poorly studied species. Using archival GPS, GPS-GSM loggers, bird-borne cameras and dietary data, we investigated the foraging ecology of chick-rearing magnificent frigatebirds from a breeding population in the Cayman Islands. This population engages in 2 main foraging strategies: (1) coastal trips over the continental shelf, where individuals target reef species and engage in kleptoparasitism, and (2) offshore trips during which birds feed on schooling pelagic prey. Differences in strategy use were partially linked to sex, with males (which invest less in offspring) roaming further from nests, and showing a higher propensity to forage offshore. Video data further indicated differences in social information use between strategies: foraging with conspecifics was more prevalent in coastal environments than pelagic. We suggest that observed variation in at-sea behaviour may partially be mediated by sex-based differences in parental roles, and/or size differences leading to intraspecific competition. Our study provides evidence of bimodal foraging and sheds new light on the importance of both pelagic and coastal feeding in this enigmatic species.
Full-text available
Between 25-30 November 2013, 2014 and 2015, miniaturized video cameras were attached to Magellanic Penguins (Spheniscus magellanicus; n = 14) in Punta Norte/San Lorenzo, Península Valdés, Chubut, Argentina. The objective was to examine prey selection, consumption of untraceable prey, and inter- and intraspecific interactions. During 56.3 hr of video footage, 1,621 dives from 14 individuals were recorded. Magellanic Penguins swam through shoals of lobster krill (Munida gregaria morph subrugosa), selectively consuming the fish, primarily anchovies (Engraulis anchoita), that were dispersed along the shoal, but did not consume the lobster krill. Magellanic Penguins captured fish on dives of less than 2 m in depth. The tagged individuals foraged with conspecifics in 2% (n = 33) of the total recorded dives. In addition, a multispecies feeding association was also documented (n = 1). Results were constrained to the upper 40 m of the water column; below this depth light level was too low for detections by video. The development of cameras with a light source and wider-angle lens are crucial to improve our understanding of Magellanic Penguin foraging behavior.
Full-text available
Although animal-borne accelerometers are effective tools for quantifying the kinematics of animal behaviors, quantifying burst movements of small and agile aquatic animals remains challenging. To capture the details of burst movements, accelerometers need to sample at a very high frequency, which will inevitably shorten the recording duration or increase the device size. To overcome this problem, we developed a high-frequency acceleration data-logger that can be triggered by a manually-defined acceleration threshold, thus allowing the selective measurement of burst movements. We conducted experiments under laboratory and field conditions to examine the performance of the logger. The laboratory experiment using red seabream (Pagrus major) showed that the new logger could measure the kinematics of their escape behaviors. The field experiment using free-swimming yellowtail kingfish (Seriola lalandi) showed that the loggers trigger correctly. We suggest that this new logger can be applied to measure the burst
Conference Paper
Full-text available
Several real-world applications require real-time prediction on resource-scarce devices such as an Internet of Things (IoT) sensor. Such applications demand prediction models with small storage and computational complexity that do not compromise significantly on accuracy. In this work, we propose ProtoNN, a novel algorithm that addresses the problem of real-time and accurate prediction on resource-scarce devices. Pro-toNN is inspired by k-Nearest Neighbor (KNN) but has several orders lower storage and prediction complexity. ProtoNN models can be deployed even on devices with puny storage and computational power (e.g. an Arduino UNO with 2kB RAM) to get excellent prediction accuracy. ProtoNN derives its strength from three key ideas: a) learning a small number of prototypes to represent the entire training set, b) sparse low dimensional projection of data, c) joint discrim-inative learning of the projection and prototypes with explicit model size constraint. We conduct systematic empirical evaluation of ProtoNN on a variety of supervised learning tasks (binary, multi-class, multi-label classification) and show that it gives nearly state-of-the-art prediction accuracy on resource-scarce devices while consuming several orders lower storage, and using minimal working memory.
Knowledge of the diet of marine mammals is fundamental to understanding their role in marine ecosystems and response to environmental change. Recently, animal-borne video cameras have revealed the diet of marine mammals that make short foraging trips. However, novel approaches that allocate video time to target prey capture events is required to obtain diet information for species that make long foraging trips over great distances. We combined satellite telemetry and depth recorders with newly developed date/time-, depth-, and acceleration-triggered animal-borne video cameras to examine the diet of female northern elephant seals during their foraging migrations across the eastern North Pacific. We obtained 48.2 hours of underwater video, from cameras mounted on the head (n=12) and jaw (n=3) of seals. Fish dominated the diet (78% of 697 prey items recorded) across all foraging locations (range: 37-55°N, 122-150°W), diving depths (range: 238-1167 m) and water temperatures (range: 3.2-7.4 °C), while squid comprised only 7% of the diet. Identified prey included fishes such as myctophids, Merluccius sp., and Icosteus aenigmaticus, and squids such as Histioteuthis sp., Octopoteuthis sp., and Taningia danae. Our results corroborate fatty acid analysis, which also found that fish are more important in the diet and contrasts to stomach content analyses that found cephalopods to be the most important component of the diet. Our work shows that in-situ video observation is a useful method for studying the at-sea diet of long-ranging marine predators.
1.The paradigm‐changing opportunities of bio‐logging sensors for ecological research, especially movement ecology, are vast, but the crucial questions of how best to match the most appropriate sensors and sensor combinations to specific biological questions, and how to analyse complex bio‐logging data, are mostly ignored. 2.Here, we fill this gap by reviewing how to optimise the use of bio‐logging techniques to answer questions in movement ecology and synthesise this into an Integrated Bio‐logging Framework (IBF). 3.We highlight that multi‐sensor approaches are a new frontier in bio‐logging, whilst identifying current limitations and avenues for future development in sensor technology. 4.We focus on the importance of efficient data exploration, and more advanced multi‐dimensional visualisation methods, combined with appropriate archiving and sharing approaches, to tackle the big data issues presented by bio‐logging. We also discuss the challenges and opportunities in matching the peculiarities of specific sensor data to the statistical models used, highlighting at the same time the large advances which will be required in the latter to properly analyse bio‐logging data. 5.Taking advantage of the bio‐logging revolution will require a large improvement in the theoretical and mathematical foundations of movement ecology, to include the rich set of high‐frequency multivariate data, which greatly expand the fundamentally limited and coarse data that could be collected using location‐only technology such as GPS. Equally important will be the establishment of multi‐disciplinary collaborations to catalyse the opportunities offered by current and future bio‐logging technology. If this is achieved, clear potential exists for developing a vastly improved mechanistic understanding of animal movements and their roles in ecological processes, and for building realistic predictive models. This article is protected by copyright. All rights reserved.
We present an energy-aware feature and classifier selection technique for low-power sensor applications. The aim is to minimise the combined energy consumed by feature extraction and statistical classification while minimising the associated loss in classifier accuracy. Our particular application is the development of animal borne sensors with onboard behaviour classification to support conservation efforts of endangered wildlife. Our technique makes use of cross-validated sequential forward feature selection to identify a shortlist of classifiers and feature sets that offer a favourable tradeoff between energy consumption and classification accuracy. This shortlist is subsequently re-ranked by incorporating the energy cost of the classifier itself, which is often disregarded in related studies. Our technique, therefore, favours classifiers and features which are in combination less energy expensive to compute at runtime. We apply this technique to datasets of accelerometer data we have complied for sheep and rhinoceros. For the sheep dataset, we are able to achieve a 6.8-fold reduction in energy consumption relative to a baseline while suffering a loss in classification accuracy among five behavioural classes from 89.6% to 88.4 %. For the rhinoceros dataset, we achieve a 363-fold reduction in energy requirements while suffering a loss in classification accuracy among three behavioural classes from 99.6% to 99.3 %. We conclude that the presented technique offers a feasible and successful means to achieve the principled design of statistical classifiers intended for low-power embedded sensor applications.
This review summarizes the advances in bio-logging technology that enables us to monitor foraging behavior, movement, behavioral performance, physiological performance, and sociality in a wide range of bird species, as well as their habitat. Subsequently, navigation is discussed, using long-distance movements in streaked shearwaters as a case study. Moreover, challenges and future research directions in bio-logging science are presented, with focus on: multimodal recording, big data analysis, feedback logging, low-power consumption and power generation systems, logger effects, and capture–recapture methods.
The anatomy of large cetaceans has been well documented, mostly through dissection of dead specimens. However, the difficulty of studying the world's largest animals in their natural environment means the functions of anatomical structures must be inferred. Recently, non-invasive tracking devices have been developed that measure body position and orientation, thereby enabling the detailed reconstruction of underwater trajectories. The addition of cameras to the whale-borne tags allows the sensor data to be matched with real-time observations of how whales use their morphological structures, such as flukes, flippers, feeding apparatuses, and blowholes for the physiological functions of locomotion, feeding, and breathing. Here, we describe a new tag design with integrated video and inertial sensors and how it can be used to provide insights to the function of whale anatomy. This technology has the potential to facilitate a wide range of discoveries and comparative studies, but many challenges remain to increase the resolution and applicability of the data. Anat Rec, 300:1935–1941, 2017. © 2017 Wiley Periodicals, Inc.
Biologging technologies are changing the way in which the marine environment is observed and monitored. However, because device retrieval is typically required to access the high‐resolution data they collect, their use is generally restricted to those animals that predictably return to land. Data abstraction and transmission techniques aim to address this, although currently these are limited in scope and do not incorporate, for example, acceleration measurements which can quantify animal behaviours and movement patterns over fine‐scales. In this study, we present a new method for the collection, abstraction and transmission of accelerometer data from free‐ranging marine predators via the Argos satellite system. We test run the technique on 20 juvenile southern elephant seals Mirounga leonina from the Kerguelen Islands during their first months at sea following weaning. Using retrieved archival data from nine individuals that returned to the colony, we compare and validate abstracted transmissions against outputs from established accelerometer processing procedures. Abstracted transmissions included estimates, across five segments of a dive profile, of time spent in prey catch attempt (PrCA) behaviours, swimming effort and pitch. These were then summarised and compared to archival outputs across three dive phases: descent, bottom and ascent. Correlations between the two datasets were variable but generally good (dependent on dive phase, marginal R² values of between .45 and .6 to >.9) and consistent between individuals. Transmitted estimates of PrCA behaviours and swimming effort were positively biased to those from archival processing. Data from this study represent some of the first remotely transmitted quantifications from accelerometers. The methods presented and analysed can be used to provide novel insight towards the behaviours and movements of free‐ranging marine predators, such as juvenile southern elephant seals, from whom logger retrieval is challenging. Future applications could however benefit from some adaption, particularly to reduce positive bias in transmitted PrCA behaviours and swimming effort, for which this study provides useful insight.