ArticlePDF Available

VideoHacking: Automated Tracking and Quantification of Locomotor Behavior with Open Source Software and Off-the-Shelf Video Equipment


Abstract and Figures

Differences in nervous system function can result in differences in behavioral output. Measurements of animal locomotion enable the quantification of these differences. Automated tracking of animal movement is less labor-intensive and bias-prone than direct observation, and allows for simultaneous analysis of multiple animals, high spatial and temporal resolution, and data collection over extended periods of time. Here, we present a new video-tracking system built on Python-based software that is free, open source, and cross-platform, and that can analyze video input from widely available video capture devices such as smartphone cameras and webcams. We validated this software through four tests on a variety of animal species, including larval and adult zebrafish (Danio rerio), Siberian dwarf hamsters (Phodopus sungorus), and wild birds. These tests highlight the capacity of our software for long-term data acquisition, parallel analysis of multiple animals, and application to animal species of different sizes and movement patterns. We applied the software to an analysis of the effects of ethanol on thigmotaxis (wall-hugging) behavior on adult zebrafish, and found that acute ethanol treatment decreased thigmotaxis behaviors without affecting overall amounts of motion. The open source nature of our software enables flexibility, customization, and scalability in behavioral analyses. Moreover, our system presents a free alternative to commercial video-tracking systems and is thus broadly applicable to a wide variety of educational settings and research programs.
Content may be subject to copyright.
The Journal of Undergraduate Neuroscience Education (JUNE), Summer 2015, 13(3):A120-A125
JUNE is a publication of Faculty for Undergraduate Neuroscience (FUN) www.funjo
VideoHacking: Automated Tracking and Quantification of Locomotor Behavior
with Open Source Software and Off-the-Shelf Video Equipment
Emily E. Conklin, Kathyann L. Lee, Sadie A. Schlabach, Ian G. Woods
Department of Biology, Ithaca College, Ithaca, NY 14850
Differences in nervous system function can result in
differences in behavioral output. Measurements of animal
locomotion enable the quantification of these differences.
Automated tracking of animal movement is less labor-
intensive and bias-prone than direct observation, and
allows for simultaneous analysis of multiple animals, high
spatial and temporal resolution, and data collection over
extended periods of time. Here, we present a new video-
tracking system built on Python-based software that is free,
open source, and cross-platform, and that can analyze
video input from widely available video capture devices
such as smartphone cameras and webcams. We validated
this software through four tests on a variety of animal
species, including larval and adult zebrafish (Danio rerio),
Siberian dwarf hamsters (Phodopus sungorus), and wild
birds. These tests highlight the capacity of our software for
long-term data acquisition, parallel analysis of multiple
animals, and application to animal species of different
sizes and movement patterns. We applied the software to
an analysis of the effects of ethanol on thigmotaxis (wall-
hugging) behavior on adult zebrafish, and found that acute
ethanol treatment decreased thigmotaxis behaviors without
affecting overall amounts of motion. The open source
nature of our software enables flexibility, customization,
and scalability in behavioral analyses. Moreover, our
system presents a free alternative to commercial video-
tracking systems and is thus broadly applicable to a wide
variety of educational settings and research programs.
Key words: video tracking; zebrafish; ethanol;
thigmotaxis; open source; Python
The quantification of behavior is a powerful, non-invasive
method for studying nervous system function, as
differences in behavior arise from differences in neural
activity. For example, locomotor output is a robustly
quantifiable behavior that is sensitive to genetic and
pharmacological perturbations of nervous system activity
and to differences in sensory input (Rothenfluh and
Heberlein, 2002; Lebestky et al., 2009; Rihel et al., 2010;
Swierczek et al., 2011; Woods et al., 2014). Locomotion
can be quantified via direct observation, but these assays
are time- and labor-intensive, require extensive training,
and can be confounded by subjective differences between
observers, and drift due to observer fatigue. Automated
assays of animal movement enable simultaneous tracking
of multiple animals and objective quantification of
locomotor behaviors. Furthermore, video-based analyses
enable high sampling frequency, high spatial resolution,
and long periods of data collection. These characteristics
eliminate potential observer biases and enable
quantification of behaviors that might be missed via manual
observation, such as rapid movements, rare behaviors, or
patterns of locomotion that emerge over extended periods
of time (Noldus et al., 2001).
Though commercial systems exist for video tracking of
locomotor behaviors, the significant costs of the hardware
and software used in these systems limit their broad
applicability. In addition, customization of these systems
often requires hardware modifications and additional
software that can increase costs further. To overcome
these limitations, video tracking tools have been developed
that use freely available software and inexpensive video
equipment (Togasaki et al., 2005; Ramazani et al., 2007;
Aguiar et al., 2007). Similarly, technologies already in
widespread use have been creatively repurposed to
quantify animal locomotion. For example, Pittman and
Ishikawa (2013) employed smartphone applications
originally designed for slow-motion analysis of golf club
swings and other sports movements to quantify locomotor
behaviors in zebrafish.
Here, we have developed video tracking software to
quantify locomotor behaviors, based on the widely
available programming language Python and the OpenCV
set of open source computer vision tools. Our system is
designed to process and analyze video acquired from
commonly available image capture devices, including
smartphones, laptop computers, and inexpensive video
cameras. Briefly, our system (1) allows users to delineate
regions of interest in a field of view, (2) quantifies
locomotion in each region of interest by calculating frame-
to-frame pixel changes in the video feed, (3) allows the
user to group regions of interest according to treatment or
genotype if appropriate, and (4) displays the results of the
experiment in a variety of user-specified ways. Our video
tracking system thus facilitates quantification of locomotor
behavior for any organism, is flexible in terms of video
input, is based on free and cross-platform software, and
can be run on mobile devices and personal computers
using a variety of operating systems. This flexibility will
enable widespread implementation in undergraduate
neuroscience and animal behavior labs, and will facilitate
rapid and inexpensive prototyping of behavioral assays.
Experimental setup. Because motion is detected by
quantifying pixel differences between successive video
frames, care must be taken to eliminate potential sources
of motion artifacts, such as moving shadows, glare,
The Journal of Undergraduate Neuroscience Education (JUNE), Summer 2015, 13(3):A120-A125 A121
Figure 1. Quantification of movements via pixel differences.
A, B. Frames from a movie acquired on a smartphone showing
two adult zebrafish in circular plastic containers. Color movies
are converted to grayscale and blurred slightly to reduce
background motion artifacts. Each movie frame is then
represented mathematically by an array of numbers representing
pixel intensities. C. Subtraction of successive video frames
highlights regions that are different between the two frames and
represent movement (white), while background regions and
stationary animals are omitted (black), enabling quantification of
the timing and magnitude of movements.
changes in light intensity, or warping in the image resulting
from lens curvature. Similarly, when analyzing the
behavior of multiple animals simultaneously, confounding
variables such as enclosure size, age, or sex should be
minimized. Though the selection of the regions of interest
to analyze is flexible, the software presented here is
especially suited for samples arranged in a grid, such as in
multi-well cell culture plates or multiple adjacent enclosures
of similar size and shape.
Software acquisition and installation. The core
computational modules require Python and the Python
libraries matplotlib ( and NumPy
( These are included as part of the
current OS X operating system, and can be easily installed
on computers using Windows or Linux. The image
analysis components of the system require OpenCV
(, which is compatible with all operating
systems in common use. The optional graphical user
interface requires installation of PyQt4
Instructions for use and details regarding how the software
functions are available as part of the software package
download at
Software operation. The video tracking software is based
upon three core modules, as outlined below. These
modules can be accessed via the command line interface,
or via the Python-based graphical user interface included
in the software package. This module enables demarcation of
regions of interest (ROIs) by retrieving the first frame of a
video file or from the live video feed of an attached or built-
in camera if no file is specified. The video frame is
displayed on screen, and the user may select ROIs on the
image by drawing rectangles of any size. If only one
rectangle is drawn, the user is prompted to partition the
large rectangle into smaller ROIs by entering the desired
number of rows and columns. This partitioning step
enables the generation of gridded ROIs, which are useful
for animals arrayed in a regular pattern. Each region of
interest is numbered by position and is assigned a
corresponding color label. This module quantifies motion in the video
feed by reading the video stream frame-by-frame and
calculating the differences in pixels between adjoining
frames. The resulting difference image highlights where
motion has occurred between the two frames. In this
manner, both the background and non-moving animals
appear black, whereas moving objects appear white
(Figure 1). The amount of movement in each ROI is
recorded as the number of pixels that are different in the
two subsequent frames, and the time at which this
difference occurs is also recorded. Thus, both the
magnitude and timing of each movement are quantified
and saved in a matrix, with the rows corresponding to pixel
differences for each frame, and the columns corresponding
to each ROI, with an additional column for time. This is the data analysis module,
which can display the results of in a variety of
ways as specified by the user (see Figures 2-5 for
examples). Results can be displayed for each individual
ROI, or for groupings of ROIs according to differences in
treatment or genotype. Data display is customizable for
each experiment, and with additional coding can be applied
to more complex analyses of movements at high temporal
resolution (Woods et al., 2014).
Ethanol treatment of adult zebrafish. Two hours before
data collection, male zebrafish at six months of age were
placed into 2-liter tanks at a density of six fish per tank.
One hour before data collection, 10 mL of ethanol was
added to the treatment tanks (for a final concentration of
0.5% vol/vol), and 10 mL of system water was added to the
control tanks. Five minutes before data collection, the fish
were transferred to a rounded square plastic container (23
cm x 23 cm x 7 cm, 3.07 L volume, GLAD) containing 2 L
Conklin et al. Automated open source video tracking of animal locomotion A122
Figure 2. Quantification of locomotion in a prerecorded movie. A.
A frame of a movie showing three rows of a simple 15-row maze.
The circled region (dotted white line) denotes a Siberian dwarf
hamster navigating one of the rows of this maze. B. Timing of
movement through the rows of the maze. Each frame that
contained movement was represented by a dot on the plot;
continuous movements were thus represented as lines. C.
Quantification of motion in each row of the maze. The bars show
the time spent by the hamster in each row. After a slow start (~11
seconds in the first row, ~7 seconds in the second row), the
hamster moved relatively quickly through each row (~2-3
seconds/row) and completed the maze in about 51 seconds.
of system water. Locomotor behavior was recorded with a
Panasonic HX-WA03 video camera affixed to a tripod. An
umbrella was fastened to the top of the tripod to minimize
glare from overhead lights. Data were collected between
10 am and 2 pm. Experiments were performed on fish
from the TL strain, and on fish from a TL x AB cross;
results from both strains were similar.
The video tracking software described here can be applied
to a wide range of animal species, and can be customized
to fit a variety of educational programs and experimental
interests. Sample experiments are described below to
demonstrate the flexibility of data acquisition and
visualization provided by the software.
Timing of maze navigation. To test the ability of our
program to quantify animal locomotion from pre-recorded
Figure 3. Simultaneous tracking of 48 larval zebrafish with a
smartphone. A. Regions of interest in the multi-well plate. The
image is a frame from a movie taken with a smartphone, and the
different colors represent each region of interest (ROI). The
arrayed ROIs are generated automatically, by partitioning a
rectangle drawn over the entire plate into a grid based on user-
entered numbers of rows and columns. B. Quantification of
movement differences. In the twenty minutes prior to treatment
with tricaine, movements of control and experimental larvae were
statistically indistinguishable (p = 0.4; n=24 for each condition).
Larvae treated with tricaine became largely immotile compared to
their control siblings (p < 0.0001). Upon exposure to warm water,
control larvae exhibited increased locomotion (p < 0.001
comparing controls before and after heat exposure), whereas
tricaine-treated fish remained immotile (p < 0.0001 comparing
tricaine-treated fish with controls after heat exposure). All
statistical comparisons are via two-tailed t-tests. C. Overview of
locomotion. The average locomotion of 24 tricaine-treated larvae
(red) and 24 untreated siblings (black) is shown by the dark lines,
while the shaded regions represent +/- s.e.m.
The Journal of Undergraduate Neuroscience Education (JUNE), Summer 2015, 13(3):A120-A125 A123
videos, we analyzed a movie of a single Siberian dwarf
hamster navigating through a simple maze. The fifteen
rows of the maze were partitioned by into
rectangular regions of interest (ROIs) of equal size (Figure
2A). Motion in each row was recorded by, and was used to visualize data in two
different ways. First, the timing of motion in each ROI (i.e.,
each row of the maze) was represented by plotting a dot
whenever a pixel difference was detected in that ROI
(Figure 2B). Second, the total time spent in each ROI was
plotted in a bar graph (Figure 2C). The hamster spent a
longer period of time navigating through the first two rows
of the maze. By analyzing multiple videos of this type, the
average time spent in each row or the time to complete the
entire maze can be objectively calculated and compared
between animals of different genotypes or different
treatment regimes. Thus, our video tracking program is
capable of automated, rapid, bias-free analysis of animal
motion at high temporal resolution, and visualizing the data
in ways meaningful to the user.
Response of larval zebrafish to tricaine and heat. To
test the ability of our video tracking system to process
video acquired from a smartphone, and to assess motion
detection of a large number of small animals
simultaneously, we quantified the effects of an anesthetic
and heat on larval zebrafish (size ~ 4 mm, Kimmel et al.,
1995). Larvae at six days post fertilization were placed
one per well in a 48-well plate (Falcon) in 1 mL of E3
medium. After an hour of acclimation to the plate, the plate
was placed in a small chamber with recirculating water
from a water bath set at 28.5 ºC. Each of the wells on the
48-well plate was designated as an individual ROI (Figure
3A). Video was acquired using an iPhone, and baseline
motion of the larvae was recorded for 20 minutes. Wells in
the dish were then partitioned into two groups. Larvae in
odd-numbered wells received a dose of tricaine anesthetic
(40 µL of 4 mg/mL; Cf: 160 µg/mL), while an equal volume
of water was added to larvae in even-numbered wells.
Tricaine-treated larvae exhibited a striking decrease in
locomotion compared to their untreated siblings (Figure 3B,
C). The water source for the recirculating bath was then
switched to a reservoir at 46 ºC. Water temperature in
wells reached maximum of 36 ºC after 10 minutes, as
measured by a handheld thermometer. After 15 minutes of
warm water treatment, the source for the recirculating bath
was returned to 28.5 ºC. Untreated larvae exhibited a
robust increase in locomotion, whereas their tricaine-
treated siblings did not respond to the change in
temperature (Figure 3B, C). Thus, video acquired from a
mobile device was sufficient to quantify differences in
motion of numerous microscopic animals simultaneously.
Long-term quantification of live video. To test the ability
of our software to analyze a live video feed, to record data
over an extended time period, and to quantify locomotion in
real time, we performed a simple test of food preference in
wild birds. A Logitech webcam (model C920) was placed
on tripod and connected to an iMac computer, and the
video stream was focused upon four small (12.5 cm x 8.5
Figure 4. Long-term quantification of movement via a live
webcam feed. A. Regions of interest defined in an image taken
with a Logitech C920 webcam. The camera was situated behind
a window with a screen mesh, and was pointed down at the
exterior windowsill. ROI 1 = raisins, ROI 2 = sunflower seeds,
ROI 3 = miscellaneous small seeds, ROI 4 = peanuts. B-C.
Timing of movement recorded in each ROI. Frames with motion
were represented by a dot at the appropriate time. All four trays
of food were visited regularly during daylight hours (B). Between
14:00 (2pm) and 15:30 (3:30pm), significant motion was recorded
in ROI 3. D. Quantification of movement in each ROI over entire
experiment. E-F. Examples of birds enjoying a treat provided
during the recording session.
cm) trays containing various kinds of bird food (from
Pennington Ultra Nut & Fruit Blend), including peanuts,
raisins, sunflower seeds, and miscellaneous small seeds (<
15 mm on longest axis, Figure 4A). The camera began
recording at 9 am, and motion was recorded for a total of
27 hours. The ability of the camera to detect instances of
feeding was confirmed via a live display of the pixel
differences on the computer screen and simultaneous
observations of visiting birds (Figure 4E, F). A variety of
species were observed visiting the trays, including black-
capped chickadees, house finches, and a pair of northern
cardinals (Figure 4E, F). Significant motion was detected
at all four trays during daylight hours (Figure 4B).
Differences in feeding preference, however, were
observable on a scale of hours and minutes (Figure 4C).
Over the course of the experiment, the most motion was
detected at the tray containing the small seeds, an
intermediate amount of motion was detected at the trays
containing sunflower seeds and peanuts, and the smallest
amount of motion was detected at the tray containing the
raisins (Figure 4D). These trends were recapitulated in an
additional experiment of comparable length, in which the
Conklin et al. Automated open source video tracking of animal locomotion A124
Figure 5. Ethanol decreases thigmotaxis behaviors in adult
zebrafish. A-B. Frames taken from a movie recorded by a
Panasonic HX-WA03 video camera, showing adult zebrafish in
individual enclosures, filmed from above. ROIs were divided into
inner and outer regions of equal area. Because of the rounded
corners of the enclosures, the navigable outer region is smaller
than the inner square. The white circles denote fish detected in
the outer region (A), and the inner region (B) of the enclosures. C.
Fish treated for one hour with 0.5% (vol/vol) ethanol exhibited
less time in outer regions of their enclosures than do their control
siblings (p < 0.0001 by two-tailed t-test; control: n=10; ethanol:
n=10). D. Control fish exhibited a preference for the edge regions
(black) of their enclosures. E. Ethanol-treated fish moved mostly
in the center regions (blue). Plots are mean (dark line) +/- s.e.m.
(shaded region), n=10 for each.
order of tray positioning was scrambled (not shown). Thus,
our software is capable of long-term quantification of
locomotor behavior, and can record and analyze behaviors
in a live video feed. Therefore, our system enables
analysis of behavioral patterns that emerge over extended
periods of time, such as circadian behaviors or differences
in locomotion that arise from long-term exposure to
pharmacological agents.
Thigmotaxis behavior in adult zebrafish. To test the
ability of our software to perform more complicated
analyses of locomotion, we analyzed thigmotaxis behaviors
in adult zebrafish exposed to a novel environment.
Thigmotaxis, also known as wall-hugging or edge-seeking
behavior, is the tendency of an animal to avoid the center
region of a novel enclosure, and to exhibit preference for
the boundary regions. Increased thigmotaxis has been
associated with anxiety-like behaviors in a variety of
species, including zebrafish (Champagne et al., 2010). We
placed individual adult zebrafish into containers arrayed in
a grid, and specified edge and center ROIs in each
container with (Figure 5 A, B). Untreated
control fish exhibited a strong preference for the edge
region of the enclosure during the seven-minute video
(Figure 5D).
We then investigated the effects of acute ethanol
exposure on thigmotaxis behavior. Prior to video recording,
siblings of the control fish were treated for one hour with
ethanol at a concentration of 0.5% (vol/vol). In previous
work, this dose of ethanol decreased the response of fish
to a predator-like stimulus, suggesting that ethanol
treatment may be anxiolytic in adult fish (Gerlai et al.,
2006). Here, the ethanol-treated fish, unlike their untreated
siblings, lacked the strong preference for the container
boundaries, and instead moved most often in the center
region of their enclosures (Figure 5E). Thus, ethanol-
treated fish exhibited a significant reduction in thigmotaxis
(Figure 5C); the overall amount of movement, however,
was not significantly different between ethanol-treated and
control fish (not shown).
Automated video tracking enables objective, robust, and
reproducible quantification of animal behaviors. Video-
based quantification of behavior can provide significant
advantages over manual data collection. For example,
manual observations often require extensive training, and
can be subject to experimenter bias and observer fatigue.
In addition, the presence of an experimenter in the same
room as the animals being studied can induce changes in
the way animals behave (Sorge et al., 2014). The video
tracking system we have developed here is flexible in
terms of animals and experimental settings. Users can
define regions of interest that are amenable to tracking
large animals through different parts of their enclosures,
quantifying place or feeding preferences in an open
environment, and detecting motion simultaneously in
animals arrayed in multi-well plates. Similarly, our software
is flexible in terms of video input, and is able to analyze
both live and recorded video from a variety of sources,
including smartphone cameras, inexpensive webcams, and
small handheld video cameras. The open source
programming tools that are the foundation of our software
are cross-platform and can be run on any personal
computer. Furthermore, the software can be controlled via
an optional Graphical User Interface (GUI), facilitating ease
of use. Thus, the system we present here enables
sophisticating video tracking analysis using inexpensive
and nearly ubiquitous image capture devices and computer
platforms. Moreover, because the software is free and
open source, it is easily extendable, scalable, and
The Journal of Undergraduate Neuroscience Education (JUNE), Summer 2015, 13(3):A120-A125 A125
customizable to a variety of research questions and
educational settings.
Aguiar P, Mendonça L, Galhardo V (2007) OpenControl: A free
opensource software for video tracking and automated control
of behavioral mazes. J Neurosci Methods 166:66-72.
Champagne DL, Hoefnagels CCM, de Kloet RE, Richardson MK
(2010) Translating rodent behavioral repertoire to zebrafish
(Danio rerio): relevance for stress research. Behav Brain Res
Gerlai R, Lee V, Blaser R (2006) Effects of acute and chronic
ethanol exposure on the behavior of adult zebrafish (Danio
rerio). Pharmacol Biochem Behav 85:752761.
Kimmel CB, Ballard WW, Kimmel SR, Ullmann B, Schilling TF
(1995) Stages of embryonic development of the zebrafish. Dev
Dyn 203:253310.
Lebestky T, Chang JC, Dankert H, Zelnik L, Kim YC, Han KA,
Wolf FW, Perona P, Anderson DJ (2009) Two different forms of
arousal in Drosophila are oppositely regulated by the dopamine
D1 receptor ortholog DopR via distinct neural circuits. Neuron
Noldus LP, Spink AJ, Tegelenbosch RA (2001) EthoVision: a
versatile video tracking system for automation of behavioral
experiments. Behav Res Methods Instrum Comput 33:398414.
Pittman JT, Ichikawa KM (2013) iPhone® applications as versatile
video tracking tools to analyze behavior in zebrafish (Danio
rerio). Pharmacol Biochem Behav 106:137142.
Ramazani RB, Krishnan HR, Bergeson SE, Atkinson NS (2007)
Computer automated movement detection for the analysis of
behavior. J Neurosci Methods 162:171179.
Rihel J, Prober DA, Arvanites A, Lam K, Zimmerman S, Jang S,
Haggarty SJ, Kokel D, Rubin LL, Peterson RT, Schier AF
(2010) Zebrafish behavioral profiling links drugs to biological
targets and rest/wake regulation. Science 327:348351.
Rothenfluh A, Heberlein U (2002) Drugs, flies, and videotape: the
effects of ethanol and cocaine on Drosophila locomotion. Curr
Opin Neurobiol 12:639645.
Sorge RE et al. (2014) Olfactory exposure to males, including
men, causes stress and related analgesia in rodents. Nat
Methods 11:629632.
Swierczek NA, Giles AC, Rankin CH, Kerr RA (2011) High-
throughput behavioral analysis in C. elegans. Nat Methods
Togasaki DM, Hsu A, Samant M, Farzan B, DeLanney LE,
Langston JW, Di Monte DA, Quik M (2005) The Webcam
system: a simple, automated, computer-based video system for
quantitative measurement of movement in nonhuman primates.
J Neurosci Methods 145:159166.
Woods IG, Schoppik D, Shi VJ, Zimmerman S, Coleman HA,
Greenwood J, Soucy ER, Schier AF (2014) Neuropeptidergic
signaling partitions arousal behaviors in zebrafish. J Neurosci
Received March 30, 2015; Revised May 01, 2015; Accepted May 14 ,
We thank the organizers of the 2014 Faculty for Undergraduate
Neuroscience conference for the invitation to present this work. K.L.L and
S.A.S. were supported by DANA research fellowships from Ithaca College .
We thank Leann Kanda for the hamster maze video.
Address correspondence to: Ian G. Woods, Dept. of Biology, 953 Danby
Road, CNS 160, Ithaca, NY 14850. Email:
Copyright © 2015 Faculty for Undergraduate Neuroscience
www.funj our nal .org
... IdTracker 5 even explicitly defined the smallest acceptable size ratio between the zebrafish and the tank for creating the clear background environment required for the video data. In addition, these larvae tracking systems 1,4,8 use a petri dish plate to separate individual zebrafish larvae, allowing only one zebrafish larvae in each petri dish to avoid overlapped and swapped trajectories that can result from multiple zebrafish larvae housed in one container (as shown in Fig. 1c). However, limiting experiments to one zebrafish per dish strictly constrains the research application as interaction and grouping behaviour cannot be studied. ...
... 8), labels (seq. 4,[8][9] Segmentation evaluation metrics. This paper uses three standard metrics and proposes a new metric to numerically quantify the segmentation performance by calculating dissimilarity with the manually generated ground truth. ...
Full-text available
The accurate tracking of zebrafish larvae movement is fundamental to research in many biomedical, pharmaceutical, and behavioral science applications. However, the locomotive characteristics of zebrafish larvae are significantly different from adult zebrafish, where existing adult zebrafish tracking systems cannot reliably track zebrafish larvae. Further, the far smaller size differentiation between larvae and the container render the detection of water impurities inevitable, which further affects the tracking of zebrafish larvae or require very strict video imaging conditions that typically result in unreliable tracking results for realistic experimental conditions. This paper investigates the adaptation of advanced computer vision segmentation techniques and multiple object tracking algorithms to develop an accurate, efficient and reliable multiple zebrafish larvae tracking system. The proposed system has been tested on a set of single and multiple adult and larvae zebrafish videos in a wide variety of (complex) video conditions, including shadowing, labels, water bubbles and background artifacts. Compared with existing state-of-the-art and commercial multiple organism tracking systems, the proposed system improves the tracking accuracy by up to 31.57% in unconstrained video imaging conditions. To facilitate the evaluation on zebrafish segmentation and tracking research, a dataset with annotated ground truth is also presented. The software is also publicly accessible.
... Existing automatic tracking systems for multiple organisms (Zhou et al., 2014;Conklin et al., 2015) either use adult subjects (Pérez-Escudero et al., 2014) in a large container to limit object interaction, or use a petri dish plate to separate individual objects, allowing only one object in each dish well to avoid overlapped and swapped trajectories. The machine learning based CNNTracker (Zhiping & Cheng, 2017) attempts to optimise individual identification accuracy by creating zebrafish head feature maps. ...
Full-text available
Accurately tracking a group of small biological organisms using algorithms to obtain their movement trajectories is essential to biomedical and pharmaceutical research. However, object mis-detection, segmentation errors and overlapped individual trajectories are particularly common issues that restrict the development of automatic multiple small organism tracking research. Extending on previous work, this paper presents an accurate and generalised Multiple Small Biological Organism Tracking System (MSBOTS), whose general feasibility is tested on three types of organisms. Evaluated on zebrafish, Artemia and Daphnia video datasets with a wide variety of imaging conditions, the proposed system exhibited decreased overall Multiple Object Tracking Precision (MOTP) errors of up to 77.59%. Moreover, MSBOTS obtained more reliable tracking trajectories with a decreased standard deviation of up to 47.68 pixels compared with the state-of-the-art idTracker system. This paper also presents a behaviour analysis module to study the locomotive characteristics of individual organisms from the obtained tracking trajectories. The developed MSBOTS with the locomotive analysis module and the tested video datasets are made freely available online for public research use.
... Automated video-tracking of adult zebrafish behavior has become a standard, objective and efficient methodology for CNS drug screening in this promising and sensitive model organism (Bailoo et al., 2010;Cachat et al., 2011;Conklin et al., 2015;Henry et al., 2019;Wang et al., 2017). In addition to minimizing experimental variance, it also allows large sets of raw data to be stored for subsequent analyses and re-analyses (Henry et al., 2019;Mikut et al., 2013). ...
Zebrafish (Danio rerio) are rapidly emerging in biomedicine as promising tools for disease modelling and drug discovery. The use of zebrafish for neuroscience research is also growing rapidly, necessitating novel reliable and unbiased methods of neurophenotypic data collection and analyses. Here, we applied the artificial intelligence (AI) neural network-based algorithms to a large dataset of adult zebrafish locomotor tracks collected previously in vivo experiments with multiple established psychotropic drugs. We first trained AI to recognize various drugs from a wide range of psychotropic agents tested, and then confirmed prediction accuracy of trained AI by comparing several agents with known similar behavioral and pharmacological profiles. Presenting a framework for innovative neurophenotyping, this proof-of-concept study aims to improve AI-driven movement pattern classification in zebrafish, thereby fostering drug discovery and development utilizing this key model organism.
... Moreover, it contains a graphical interface to view summary data. 26 Another advantage of VideoHacking is the ability to analyze live or prerecorded videos of several hours with highquality tracking and tracking of several animals simultaneously (as long as they do not overlap). Outcome parameters of locomotor activity, such as velocity, acceleration, total length, average speed, and time spent in an ROI (e.g., thigmotaxis behavior), are generated by the software. ...
The analysis of behavior in animal models is an important objective in many research fields, including neu- roscience, psychology, toxicology, and neuropsychopharmacology. Animal models have been used for many years, and several behavioral paradigms, such as locomotor activity, social interactions, and cognitive behavior, have been studied in animal models to correlate the behaviors with pharmacological or environmental inter- ventions and with molecular, biochemical, and physiological findings. We reviewed the literature looking for open-source, freely available software to analyze animal behavior and found 12 freely available programs: ToxTrack, EthoWatcher, Mouse Behavior Tracker, Mouse Move, JAABA, wrMTrck, AnimalTracker, id- Tracker, Ctrax, Mousetracker, VideoHacking, and Cowlog, which were developed with different programs, work on different platforms, and have particular types of inputs and outputs and analysis capabilities. We reviewed some examples of their use, tested some of them, and provided several recommendations for the future development of programs for the automated analysis of behavior in animal models. In conclusion, we show freely available software for the automated analysis of behavior in animal models such as adult zebrafish and provide information for researchers and students looking for quick, easy-to-implement, and inexpensive behavior analysis alternatives.
... Another potentially useful technology is computer vision. As with more traditional, electromechanical forms of automation, expensive computer vision tools are purchasable, but some researchers are also creating new open-source solutions (e.g., Aguiar et al., 2007;Conklin, Lee, Schlabach, & Woods, 2015;Kane & Zamani, 2014). For readers interested in developing free computer vision options, we suggest researching the Python programing language (, in conjunction with the scientific analysis package, SciPy (Jones et al., 2001), and computer vision library, OpenCV (Open Source Computer Vision Library, ...
Full-text available
Behavioral research is often enhanced by automated techniques, where experimental parameters and detection of behavior are controlled by electromechanical systems. Automated research promotes refinements in measurement, greater experimental control, longer durations of data collection, reduction of observer fatigue, and may permit new types of research to be conducted. In comparative psychology, use of automated techniques are often restricted to popular model organisms of fields such as behavior analysis and behavioral neuroscience. One factor contributing to this species-restriction may be the availability of automated research equipment, as most commercial research equipment is designed for rodents, and many researchers lack the skills required to create their own automated equipment. However, there are alternatives to commercial equipment, as some behavioral scientists have made available their own species-flexible, low-cost research equipment. In this paper, we provide three reviews. We first review recent trends in automated comparative psychology research, and then relate this to a second review on currently available automated research equipment. We also review affordable alternatives to commercial equipment that have been designed by behavioral scientists. Finally, we discuss useful technological skills that may allow comparative psychologists to take automation into their own hands and design equipment specific to their species and research topic.
... These software programs have been introduced to zebrafish researchers to study behavior; however, the selling price was expensive and thus unaffordable to most laboratories. Therefore, several open-access software packages compiled by Matlab [33][34][35] or OpenCV scripts have been programed [36,37]. With either commercial or home-made software, two or more cameras were required to capture the movement simultaneously. ...
Full-text available
Generally, the measurement of three-dimensional (3D) swimming behavior in zebrafish relies on commercial software or requires sophisticated scripts, and depends on more than two cameras to capture the video. Here, we establish a simple and economic apparatus to detect 3D locomotion in zebrafish, which involves a single camera capture system that records zebrafish movement in a specially designed water tank with a mirror tilted at 45 degrees. The recorded videos are analyzed using idTracker, while spatial positions are calibrated by ImageJ software and 3D trajectories are plotted by Origin 9.1 software. This easy setting allowed scientists to track 3D swimming behavior of multiple zebrafish with low cost and precise spatial position, showing great potential for fish behavioral research in the future.
Full-text available
Background: Atlantoaxial (AA) cerebrospinal fluid (CSF) collection in standing horses utilizes the controlled narcotic morphine, thereby limiting feasibility in field practice settings. Objectives: To compare AA CSF collection time and reaction scores in horses sedated with morphine-containing and opioid-free sedation protocols: detomidine + morphine (DM), detomidine + xylazine (DX), detomidine + detomidine (DD), detomidine alone (D0); To develop a novel method for assessing sedation in standing horses using open-source motion-tracking software. Animals: Six healthy adult horses. Methods: Randomized crossover. Atlantoaxial CSF collections were performed weekly for 4 weeks. Horses received sedation protocols in random order. Procedure time and procedure reaction scores were compared between groups using Friedman test. Associations between procedure reaction scores and motion tracking variables (total distance and farthest excursion traveled by horse's head) were examined using scatter diagrams. Results: Procedure times were D0 (median = 120 seconds, range XXXX), DM (48, range xxx to yy), DX (36 range), or DD (49, range); (P = .25). Procedure reaction scores were lower in horses sedated with DX (median score = 1.0, range xx to yy), compared to DD (2.8; range cc to vv, P = .04) or DO (3.0 range; P = .01). Reactions to dura mater puncture were recorded in 3 of 6 horses in D0 and DD groups, and 0 of 6 horses in DX and DM groups. Positive associations were observed between reaction score vs total distance or farthest excursion distance from baseline. Conclusions and clinical importance: Both opioid-free and morphine-containing sedation protocols are acceptable for AA CSF collection. Motion-tracking software represents a novel method for assessing sedation in standing horses.
Computational Neuroethology comprises a wide variety of devices, computational tools and techniques used in studies aiming to understand the neural substrate of the observable behavior. In this short review we focus on the description of available computational tools in a landscape of resources that is steadily growing as the scientific community recognizes this Computational Neuroethology as one of the frontiers of scientific endeavor. We comment on the biological basis and some examples of studies reported in the literature before providing a description and taxonomy of resources and tools.
Genomic studies have identified hundreds of candidate genes near loci associated with risk for schizophrenia. To define candidates and their functions, we mutated zebrafish orthologs of 132 human schizophrenia-associated genes. We created a phenotype atlas consisting of whole-brain activity maps, brain structural differences, and profiles of behavioral abnormalities. Phenotypes were diverse but specific, including altered forebrain development and decreased prepulse inhibition. Exploration of these datasets identified promising candidates in more than 10 gene-rich regions, including the magnesium transporter cnnm2 and the translational repressor gigyf2, and revealed shared anatomical sites of activity differences, including the pallium, hypothalamus, and tectum. Single-cell RNA sequencing uncovered an essential role for the understudied transcription factor znf536 in the development of forebrain neurons implicated in social behavior and stress. This phenotypic landscape of schizophrenia-associated genes prioritizes more than 30 candidates for further study and provides hypotheses to bridge the divide between genetic association and biological mechanism. Analysis of zebrafish deficient for human schizophrenia-associated genes generates an atlas of brain and behavior phenotypes for the study of psychiatric disorders.
Conference Paper
The accurate tracking of zebrafish larvae movement is essential to many biomedical and neural science applications. This paper develops an accurate and reliable multiple zebrafish larvae tracking system resilient to detection and segmentation errors due to object misdetection and occlusion. The proposed system can therefore be applied to microscopic videos in unconstrained, realistic imaging conditions. Evaluated on a set of single and multiple adult and larvae zebrafish videos, a wide variety of (complex) video conditions were tested, including shadowing, labels, water bubbles and background artefacts. The proposed system obtains decreased overall MOTP error of up to 44.49 pixels compared to the commercial LoliTrack system, and increased MOTA accuracy by 31.57% compared with the state-of-the-art idTracker approach. The results offer an additional advantage of improved position detection, increased accuracy and unique identification compared to current techniques.
Full-text available
We found that exposure of mice and rats to male but not female experimenters produces pain inhibition. Male-related stimuli induced a robust physiological stress response that results in stress-induced analgesia. This effect could be replicated with T-shirts worn by men, bedding material from gonadally intact and unfamiliar male mammals, and presentation of compounds secreted from the human axilla. Experimenter sex can thus affect apparent baseline responses in behavioral testing.
Full-text available
Zebrafish (Danio rerio) are emerging as a promising model organism for experimental studies relevant to biological psychiatry. The objective of this study was to develop a novel video-based movement tracking and analysis system to quantify behavioral changes following psychoactive drug exposure in zebrafish. We assessed the effects of withdrawal from chronic ethanol exposure, and subsequent administration of fluoxetine (Prozac®), buspirone (Buspar®), diazepam (Valium) using two behavioral paradigms; the Novel Tank Diving Test and the Light/Dark Choice Assay. A video tracking system was developed using two Apple® applications (Apps) to quantify these behaviors. Data from zebrafish exposed to the above treatments are presented in this paper to exemplify not only behavioral alterations associated with chronic exposure, but more importantly, to validate the video tracking system. Following withdrawal from chronic ethanol exposure, zebrafish exhibited dose/time-dependent anxiogenic effects; including reduced exploration and freezing behavior in the Novel Tank Diving Test, and preference for the dark area for the Light/Dark Choice Assay. In contrast, the above drug treatments had significant anxiolytic effects. We have developed a simple and cost-effective method of measuring zebrafish behavioral responses. The iPhone® Apps outlined in this study offer numerous flexible methods of data acquisition; namely, ease of identification and tracking of multiple animals, tools for visualization of the tracks, and calculation of a range of analysis parameters. Furthermore, the limited amount of time required for interpretation of the video data makes this a powerful high-throughput tool with potential applications for pre-clinical drug development.
Full-text available
We designed a real-time computer vision system, the Multi-Worm Tracker (MWT), which can simultaneously quantify the behavior of dozens of Caenorhabditis elegans on a Petri plate at video rates. We examined three traditional behavioral paradigms using this system: spontaneous movement on food, where the behavior changes over tens of minutes; chemotaxis, where turning events must be detected accurately to determine strategy; and habituation of response to tap, where the response is stochastic and changes over time. In each case, manual analysis or automated single-worm tracking would be tedious and time-consuming, but the MWT system allowed rapid quantification of behavior with minimal human effort. Thus, this system will enable large-scale forward and reverse genetic screens for complex behaviors.
Full-text available
A major obstacle for the discovery of psychoactive drugs is the inability to predict how small molecules will alter complex behaviors. We report the development and application of a high-throughput, quantitative screen for drugs that alter the behavior of larval zebrafish. We found that the multidimensional nature of observed phenotypes enabled the hierarchical clustering of molecules according to shared behaviors. Behavioral profiling revealed conserved functions of psychotropic molecules and predicted the mechanisms of action of poorly characterized compounds. In addition, behavioral profiling implicated new factors such as ether-a-go-go–related gene (ERG) potassium channels and immunomodulators in the control of rest and locomotor activity. These results demonstrate the power of high-throughput behavioral profiling in zebrafish to discover and characterize psychotropic drugs and to dissect the pharmacology of complex behaviors.
Full-text available
The need for automating behavioral observations and the evolution of systems developed for that purpose is outlined. Video tracking systems enable researchers to study behavior in a reliable and consistent way and over longer time periods than if they were using manual recording. To overcome limitations of currently available systems, we have designed EthoVision, an integrated system for automatic recording of activity, movement, and interactions of animals. The EthoVision software is presented, highlighting some key features that separate EthoVision from other systems: easy file management, independent variable definition, flexible arena and zone design, several methods of data acquisition allowing identification and tracking of multiple animals in multiple arenas, and tools for visualization of the tracks and calculation of a range of analysis parameters. A review of studies using EthoVision is presented, demonstrating the system's use in a wide variety of applications. Possible future directions for development are discussed.
Animals modulate their arousal state to ensure that their sensory responsiveness and locomotor activity match environmental demands. Neuropeptides can regulate arousal, but studies of their roles in vertebrates have been constrained by the vast array of neuropeptides and their pleiotropic effects. To overcome these limitations, we systematically dissected the neuropeptidergic modulation of arousal in larval zebrafish. We quantified spontaneous locomotor activity and responsiveness to sensory stimuli after genetically induced expression of seven evolutionarily conserved neuropeptides, including adenylate cyclase activating polypeptide 1b (adcyap1b), cocaine-related and amphetamine-related transcript (cart), cholecystokinin (cck), calcitonin gene-related peptide (cgrp), galanin, hypocretin, and nociceptin. Our study reveals that arousal behaviors are dissociable: neuropeptide expression uncoupled spontaneous activity from sensory responsiveness, and uncovered modality-specific effects upon sensory responsiveness. Principal components analysis and phenotypic clustering revealed both shared and divergent features of neuropeptidergic functions: hypocretin and cgrp stimulated spontaneous locomotor activity, whereas galanin and nociceptin attenuated these behaviors. In contrast, cart and adcyap1b enhanced sensory responsiveness yet had minimal impacts on spontaneous activity, and cck expression induced the opposite effects. Furthermore, hypocretin and nociceptin induced modality-specific differences in responsiveness to changes in illumination. Our study provides the first systematic and high-throughput analysis of neuropeptidergic modulation of arousal, demonstrates that arousal can be partitioned into independent behavioral components, and reveals novel and conserved functions of neuropeptides in regulating arousal.
We describe a series of stages for development of the embryo of the zebrafish, Danio (Brachydanio) rerio. We define seven broad periods of embryogenesis—the zygote, cleavage, blastula, gastrula, segmentation, pharyngula, and hatching periods. These divisions highlight the changing spectrum of major developmental processes that occur during the first 3 days after fertilization, and we review some of what is known about morphogenesis and other significant events that occur during each of the periods. Stages subdivide the periods. Stages are named, not numbered as in most other series, providing for flexibility and continued evolution of the staging series as we learn more about development in this species. The stages, and their names, are based on morphological features, generally readily identified by examination of the live embryo with the dissecting stereomicroscope. The descriptions also fully utilize the optical transparancy of the live embryo, which provides for visibility of even very deep structures when the embryo is examined with the compound microscope and Nomarski interference contrast illumination. Photomicrographs and composite camera lucida line drawings characterize the stages pictorially. Other figures chart the development of distinctive characters used as staging aid signposts. ©1995 Wiley-Liss, Inc.
The current study provides a detailed description of the pattern of exploratory behaviors encountered in adult zebrafish when exposed to a novel/unfamiliar environment using the light/dark box and open field tests. We also document the impact of an acute stressor (restraint stress) given just prior the onset of behavioral testing. We report the following main findings: (1) zebrafish display anxiety-like behaviors including dark-avoidance (in light/dark box test) and thigmotaxis (in open field test), (2) upon exposure to a novel environment (first 2 min), zebrafish display place preference for the outer zone of the testing apparatus where they seek escape via the transparent wall, and exhibit high locomotor activity accompanied by high swimming speed, (3) thigmotaxis, behavioral hyperactivity, and swimming speed habituate (decrease) over time, (4) prior history of stress attenuates the natural tendency to engage in dark-avoidance behavior and thigmotaxis, reduces attempts to escape via the transparent wall, and greatly increased behavioral hyperactivity and swimming speed. Stress-induced patterns of behavior normalize to levels comparable to that of non-stressed controls by the end of the 5-min test session. Taken together, these findings suggest that novel environment can elicit anxiety-like behaviors in zebrafish such as dark-avoidance and thigmotaxis and the prior history of stress greatly affects patterns of exploration, defensive behaviors, and coping strategies in the light/dark box and open field tests. These findings are consistent with previous findings in rodents and support the usefulness of such behavioral paradigms in zebrafish.
Arousal is fundamental to many behaviors, but whether it is unitary or whether there are different types of behavior-specific arousal has not been clear. In Drosophila, dopamine promotes sleep-wake arousal. However, there is conflicting evidence regarding its influence on environmentally stimulated arousal. Here we show that loss-of-function mutations in the D1 dopamine receptor DopR enhance repetitive startle-induced arousal while decreasing sleep-wake arousal (i.e., increasing sleep). These two types of arousal are also inversely influenced by cocaine, whose effects in each case are opposite to, and abrogated by, the DopR mutation. Selective restoration of DopR function in the central complex rescues the enhanced stimulated arousal but not the increased sleep phenotype of DopR mutants. These data provide evidence for at least two different forms of arousal, which are independently regulated by dopamine in opposite directions, via distinct neural circuits.
Drosophila melanogaster has been introduced recently as a model organism in which to study the mechanisms by which drugs of abuse change behavior and by which the nervous system changes upon repeated drug exposure. Surprising similarities between flies and mammals have begun to emerge at the behavioral, neurochemical and molecular levels.