Video Monitoring of Honey Bee Colonies at the Hive Entrance
ABSTRACT The flight activity of a honey bee colony is an impor-tant indicator of its strength and condition. We propose the use of video sensing to monitor arrivals and depar-tures at the hive entrance. We describe the challenges of tracking and counting bees visually in the uncontrolled outdoor environment of an apiary, and detail a hard-ware platform and software algorithms we have devel-oped to meet many of those challenges. Finally, we dis-cuss our preparations for rigorously evaluating the pro-posed monitoring system and describe early results.
- SourceAvailable from: Frank Dellaert[show abstract] [hide abstract]
ABSTRACT: Subspace representations have been a popular way to model appearance in computer vision. In Jepson and Black's influential paper on EigenTracking, they were successfully applied in tracking. For noisy targets, optimization-based algorithms (including EigenTracking) often fail catastrophically after losing track. Particle filters have recently emerged as a robust method for tracking in the presence of multi-modal distributions. To use subspace representations in a particle filter, the number of samples increases exponentially as the state vector includes the subspace coefficients. We introduce an efficient method for using subspace representations in a particle filter by applying Rao-Blackwellization to integrate out the subspace coefficients in the state vector. Fewer samples are needed since part of the posterior over the state vector is analytically calculated. We use probabilistic principal component analysis to obtain analytically tractable integrals. We show experimental results in a scenario in which we track a target in clutter. 1.05/2004;
- [show abstract] [hide abstract]
ABSTRACT: A sensor has been developed to monitor objects passing through tunnels using a capacitance bridge. While the sensor concept is easily adaptable to a wide range of objects or organisms which pass through an enclosed area, our version of the sensor was designed specifically for monitoring bumblebee colonies. Other bee sensors have been developed based on optical methods of detection. The capacitance sensor provides all the information of the optical sensors and additional information on the bee size and velocity. The sensor is expected to provide entomologists with more efficient methods of studying the foraging activities of bees.Measurement Science and Technology 11/2005; 16(12):2503. · 1.44 Impact Factor
- Apidologie 01/1994; 25(4):384-395. · 2.16 Impact Factor
Video Monitoring of Honey Bee Colonies at the Hive Entrance
Jason Campbell, Lily Mummert, Rahul Sukthankar
Intel Research Pittsburgh
The flight activity of a honey bee colony is an impor-
tant indicator of its strength and condition. We propose
the use of video sensing to monitor arrivals and depar-
tures at the hive entrance. We describe the challenges of
tracking and counting bees visually in the uncontrolled
outdoor environment of an apiary, and detail a hard-
ware platform and software algorithms we have devel-
oped to meet many of those challenges. Finally, we dis-
cuss our preparations for rigorously evaluating the pro-
posed monitoring system and describe early results.
Honey bees have been a focus of recent attention be-
cause of their vital role in pollinating agricultural crops,
and because of long term and recent precipitous de-
clines in the number of colonies . Beekeepers assess
colony health by manually inspecting hives, beginning
with visual observation of flight activity. A rough es-
timate of traffic, along with knowledge of local condi-
tions and prior behavior, can indicate if closer inspec-
tion or intervention is warranted.
Attempts to automate activity monitoring at the hive
entrance date back nearly a century . Early meth-
ods are reviewed in Pham-Del´ egue et al. . Today,
commercially-available activity counters are based on
infrared sensing [10, 14]. A device is placed in the
hive entrance that consists of up to 32 bidirectional,
bee-sized tunnels, each equipped with a dual photore-
ceptor to determine direction of movement. Drawbacks
of this approach include obstruction of bee movement
and spurious counts caused by debris in the tunnels,
which must be cleaned regularly. More recent work
on a capacitance-based sensor to monitor bumblebees
passing through a tunnel addresses some of these draw-
backs . Counting systems have been developed for
tracking specific bees using techniques such as metal
detection  and bar code scanning [3, 13].
methods require manipulation of individual bees, and
Figure 1. Instrumented bee hive
are thus infeasible in a practical setting.
Flight activity is affected by many factors both inter-
nal and external to a colony, such as population, pres-
ence of an egg-laying queen, weather conditions, avail-
ability of nectar and pollen, disease, and exposure to
toxins. Despite this complexity, activity counters pro-
vide useful practical information. For example, they
reveal phenomena such as swarming, an event of im-
portance to beekeepers that is brief and difficult to pre-
dict, but is marked by large numbers of bees leaving
the hive. Other events that cause significant changes in
short-term activity include thunderstorms and exposure
to pesticides. Bromenshenk et al. found two measures
derivedfromactivitycountsmostuseful: netlosses, and
the coefficient of variation for activity between hives at
the same site . Spikes in activity, large losses, un-
sustainable loss rates over time, less active colonies rel-
ative to their peers, and colonies whose behavior devi-
ates substantially from their established norms would
all merit investigation by the beekeeper.
The use of video for monitoring bees is relatively
recent, now feasible because of the availability of com-
modity digital cameras and high-performance comput-
ing devices. Several studies have aimed at assisting be-
havioral analysis of activity inside the hive, such as bee
dance communication , by automatically tracking
bees [7, 8] and labeling their behaviors [5, 15]. Outside
the hive, video has been used to track flying bees in an
orchard to study pollination .
In this paper, we propose the use of video for ac-
tivity monitoring at the hive entrance in a practical set-
ting. To our knowledge, this is the first use of video
for this purpose. The main advantage of video is that
it does not interfere with normal colony activity, though
accurate interpretation of the video stream poses a num-
ber of challenges. In the rest of this paper, we describe
our sensing platform, and discuss techniques to detect,
track, and count arriving and departing bees. Our goal
is to measure hive interaction with the outside world in
a way that enables beekeepers to focus their attention
where it is needed.
2. Sensing Platform
Our approach monitors flights to and from the a bee
colony using a camera positioned over the unmodified
entrance of a standard hive, as shown in Figure 1. We
chose a downward-facing view in order to maximize the
benefit of the white background offered by the landing
platform just outside the hive entrance, and to mini-
mize the scale and perspective differences that would
be caused by larger distances.
We use a Unibrain Fire-iTMdigital board camera
equipped with a wide angle lens (f = 2.1mm,80◦hor-
izontal view angle). This wide angle view covers the
entire 40cm (16 in) width of the entrance platform while
mounting the camera only 20cm (8 in) above that plat-
form. Thusthe cameracanbe attachedwithinthe height
of a single primary hive body itself rather than relying
on the presence of at least one expansion box (“super”
in apiarist terminology). We can hence monitor hives
from the very earliest stages.
shield custom fabricated using fused-deposition model-
ing (see Fig. 2, dimensions are in centimeters). A port
on the side of the enclosure permits access for an IEEE
1394 bus cable and, optionally, for a power cable. The
enclosure is attached to the hive with removable adhe-
The camera acquires color 640x480 video frames
at 30 frames/sec. We have gathered video data from
Figure 2. Camera enclosure
Figure 3. Sample video frames
two different hives, the first established from a 3-pound
package in April 2007, and the second established with
four frames of brood from two existing colonies in May
2008. Sample video frames from the second hive, taken
in July 2008, are shown in Figure 3.
3. Measuring Activity
Measuring flight activity at the hive entrance from
video involves detecting bees and tracking their motion
through a sequence of frames.
There are several reasons why bee detection and
tracking is a difficult computer vision problem.
First, as seen in Figure 3, in a typical image ac-
quired from the hive-mounted camera a single bee oc-
cupies only a very small portion of the image (approx-
imately 6 × 14 pixels). Bee detection might be easier
with higher-resolution cameras or with multiple cam-
eras each placed closer to the hive entrance, but only
at a substantial increase in cost as well as physical and
computational complexity, limiting utility in practical
Second, because bee hives are outdoors, lighting
conditions can vary significantly with the time of day,
season and weather. Shadows are cast by the camera
enclosure, moving bees, and moving foliage overhead.
These all hinder naive approaches to background sub-
traction. Although it is possible to arrange clear light-
ing in the hive entry area this demands onerous hive-
placement constraints vis-a-vis trees, buildings, and
compass points. Artificial lighting, such as routinely
used in industrial machine vision, would be difficult to
place in the field and could affect bee behavior.
Third, even at 30 frames/second, flying bees can
ment complicates frame-to-frame matching as worker
bees from a hive are virtually identical in appearance.
Furthermore, bees transition quickly between loitering,
rections unpredictably; this makes it impossible to track
them using one unimodal motion model.
Fourth and finally, the scene is often cluttered. Bees
can group or occlude each other in ways that challenge
simple segmentation and tracking approaches.
problem has also been observed for video taken inside
the hive [7, 5, 15].
Our current bee detector performs adaptive back-
ground subtraction using a background model derived
from a running average of the most recent 300 video
frames. We then match an elliptical, graduated template
at 16 orientations across each background-subtracted
video frame. We presently consider only one size tem-
plate. Adding sizes would be a straightforward exten-
sion. The graduated template encourages centering of
the detection region on each bee and penalizes oval ob-
jects which do not exhibit bees’ characteristic round ap-
pearance in depth as well as outline.
The amount a bee moves between frames depends
on its behavior. We distinguish between four different
behaviors: loitering, crawling, flying out and flying in.
These have significantly different characteristics. For
example, crawling bees typically do not move signifi-
cantly between frames, while flying bees do. Bees fly-
ing towards the hive entrance hover while looking for a
place to land, so their motion is more lateral than lon-
gitudinal. Bees flying away from the hive entrance tend
to exhibit the greatest forward motion per frame.
We model frame-to-frame changes in bee position
and orientation using Gaussian distributions for crawl-
ing and flying. These models are derived from manual
motionanalysisofasmallnumber ofvideoframes. Ori-
entation changes are modeled separately from position.
Loitering is detected with a small distance threshold.
We treat the task of tracking bees from frame-to-
frame as an assignment problem, and solve it using
maximum weighted bipartite graph matching. Specifi-
cally, each detection in frame i is associated with a node
in the first set, A and each detection in frame i+1 with
a node in the second set, B. The weighted edge between
a pair of nodes a ∈ A and b ∈ B is proportional to the
likelihood that a bee with pose a could move to pose b
in the span of a single frame.
Bee motion is expressed by the four motion mod-
els described in Section 3.3. We make the simplifying
assumption that a bee can change its mode of motion in-
stantly among these modes on a frame-by-frame basis,
and that all of them are equally likely a priori.1
We define the weight on each edge of the graph using
the most likely motion model for the given hypothesis.
In general, this means that pairs of detections spaced
far apart acquire a weight from one of the flying models,
those within a pixel or two are associated with loitering,
and the remainder are explained by crawling. Because
bees in one frame may not appear in the next (enter-
ing the hive or flying out of the frame), and new bees
may appear (exiting the hive or flying into the frame),
we augment the graph with dummy nodes and edges
for these hypotheses. Departures and arrivals are de-
termined by examining the tracks of unmatched bees.
Because it is possible to lose track of a bee temporarily
(e.g., occlusions), inconclusive tracks are retained for a
few frames in case they can be reestablished.
4. Bee Activity Data
We have created a manually-annotated dataset of bee
video to train motion models (Section 3.3) and to evalu-
ate the overall system. The training dataset for the mo-
tion models consists of 600 frames annotated with the
motion type and frame-to-frame change in position and
1We plan to refine the models and transitions as additional empir-
ical data on bee motion is collected.
orientation for every visible bee. We have annotated
an additional 1800 frames with ground truth of arrivals,
departures and number of visible bees in each frame.
We plan to annotate additional video from an estab-
lished colony to improve the motion models and pro-
vide a more rigorous basis for evaluation. Because the
surfaces of this hive are more weathered and dirty, and
because the colony is larger and more active it repre-
sents a more complicated test case.
5. Preliminary Results
One of our goals was to not substantially alter the
bees’ environment, and so far impact on hive activity
appears to be minimal. In particular, the bees exhibit no
particular interest in the camera or housing (except for
understandable interest during installation).
We implemented the system described in Section 3
and are testing it on the annotated dataset described in
Section 4. Our initial experience is that the counter
has basic functionality, and is resilient to errors such
as track loss and switching between objects as long as
direction can be discerned.
The counter detects bees with precision 0.94 and re-
call 0.79 on the annotated dataset. False negatives oc-
cur largely at the edges of the frame, where wide an-
gle lens distortions are severe. False positives occur
when bees fly towards the camera and appear larger
than usual, generating multiple detections. Using multi-
ple size templates for detection or an appearance model
trained on bee images from the dataset should improve
detection in these cases.
The counter overcounted arrivals by 2% and under-
counted departures by 7% on the annotated data set.
Undercounting is caused by false negatives in the de-
tector compounded by track aliasing, in which the track
of an arriving bee is incorrectly associated with that of
a nearby departing bee, and vice versa. Incorporating
more detailed models of bee orientation into the mo-
tion model and scoring entire tracks based upon struc-
ture should address this problem.
The most important bit of orientation information —
whether a bee is facing towards or away from the hive
opening — is also the most challenging bit of infor-
mation to obtain. Preliminary analyses of pixel data
suggests it will be possible to distinguish, for instance,
heads from abdomens directly using a more sophisti-
cated appearance model.
This paper presents a video-based approach for mea-
suring flight activity of honey bee colonies. The pri-
mary contribution is a non-invasive system that can be
deployed in an apiary with minimal disruption. Prelimi-
nary results indicate that our methods for adaptive back-
ground subtraction, template-based bee detection and
tracking show promise under real-world conditions.
 J. J. Bromenshenk, G. C. Smith, B. E. King, R. A. Sec-
comb, G. Alnasser, C. B. Henderson, M. R. Loeser, and
C. L. Wrobel. New and Improved Methods for Monitor-
ing Air Quality and the Terrestrial Environment. Tech-
nical Report ADA326262, National Technical Informa-
tion Service, Springfield, VA, November 1996.
 J. M. Campbell, D. C. Dahn, and D. A. J. Ryan.
Capacitance-based sensor for monitoring bees passing
through a tunnel. Measurement Science and Technol-
ogy, 16, November 2005.
 Data2 Corporation. Data2 Creates World’s Smallest
Barcode. Press release, http://www.data2.biz/
 V. Estivill-Castro, D. C. Lattin, F. Suraweera, and
V. Vithanage. Tracking bees — a 3D, outdoor small
object environment. In Proceedings International Con-
ference on Image Processing, 2003.
 A. Feldman and T. Balch. Representing honey bee be-
havior for recognition using human trainable models.
Adaptive Behavior, 12(3–4), 2004.
 J. K. Kaplan. Colony Collapse Disorder: A Complex
Buzz. Agricultural Research, May–June 2008.
 Z. Khan, T. Balch, and F. Dellaert.
Blackwellized Particle Filter for EigenTracking.
Proc. Computer Vision and Pattern Recognition, 2004.
 T. Landgraf and R. Rojas. Tracking honey bee dances
from sparse flow optical fields. Technical Report B 07-
11, Freie Universit¨ at Berlin, Inst. f¨ ur Informatik, 2007.
 L’Institut national de la recherche agronomique
(INRA). Tracking bees to study their behavior.
Press release, http://www.international.inra.
fr/press/tracking_bees, June 2006.
 Lowland Electronics bvba. BeeSCAN Instruction Man-
 A. E. Lundie. The flight activities of the honey bee.
United States Department of Agriculture Bulletin No.
 M.-H. Pham-Del´ egue, A. Decourtye, L. Kaiser, and
J. Devillers. Behavioral methods to assess the effects
of pesticides on honey bees. Apidologie, 33:425–432,
 M. T. Sanford. Bar Codes on Bees and Honey. APIS:
Apicultural Information and Issues, 7(5), 1989.
 M. H. Struye, H. J. Mortier, G. Arnold, C. Miniggio,
and R. Bornek. Microprocessor-controlled monitoring
of honeybee flight activity at the hive entrance. Apidolo-
gie, 25:384–395, 1994.
 A. Veeraraghavan, R. Chellappa, and M. Srinivasan.
Shape and Behavior Encoded Tracking of Bee Dances.
IEEE Transactions on PAMI, 30(3), 2008.
 K. von Frisch. The Dance Language and Orientation of
Bees. Harvard University Press, 1967.