Conference PaperPDF Available

DRONE-BASED TRAFFIC FLOW ESTIMATION AND TRACKING USING COMPUTER VISION

Authors:

Abstract and Figures

Traffic management has become increasingly important with growth in vehicle numbers unmatched by investment in infrastructure. A large part of management is measuring traffic flow. Video footage of traffic flow is normally manually checked to determine key traffic metrics, consuming many human hours. Moreover, installation and maintenance cost of recording equipment and supporting infrastructure is substantial, especially in the Sub-Saharan context. This paper proposes a novel solution to automate traffic flow estimation, using computer vision. The paper also introduces the notion of making the recording equipment mobile by using drone-based equipment, negating the need for fixed recording installations. The results demonstrate measurement accuracies of 100% down to 81% from ideal to worst case conditions, and successful implementation of drone control algorithms.
Content may be subject to copyright.
DRONE-BASED TRAFFIC FLOW ESTIMATION AND TRACKING
USING COMPUTER VISION
A. de Bruin and M.J. Booysen
Department of E&E Engineering, Stellenbosch University, Private Bag X1, Matieland,
7602 Tel: 021 808-4013; Email: mjbooysen@sun.ac.za
ABSTRACT
Traffic management has become increasingly important with growth in vehicle numbers
unmatched by investment in infrastructure. A large part of management is measuring
traffic flow. Video footage of traffic flow is normally manually checked to determine key
traffic metrics, consuming many human hours. Moreover, installation and maintenance
cost of recording equipment and supporting infrastructure is substantial, especially in the
Sub-Saharan context. This paper proposes a novel solution to automate traffic flow
estimation, using computer vision. The paper also introduces the notion of making the
recording equipment mobile by using drone-based equipment, negating the need for fixed
recording installations. The results demonstrate measurement accuracies of 100% down
to 81% from ideal to worst case conditions, and successful implementation of drone
control algorithms.
INTRODUCTION
According to the National Traffic Information System, there are currently around 11 million
registered vehicles on South African roads (National department of transport, 2014). This
number is increasing at an alarming rate, which requires that roads be upgraded
continually. The study of traffic flow estimation is used to evaluate how well a particular
road segment is accommodating traffic, as well as to determine the priority of road
upgrades.
Current traffic monitoring techniques make use of intrusive static sensors in the form of
inductive loop detectors, infrared (IR) detectors and radar guns (Thies et al., 2013). Visual
monitoring is often done manually, with the operator watching hours of video footage while
counting the cars as they pass through an area. Two of the significant problems
associated with the above-mentioned techniques, is that they are both intrusive and time-
consuming. Traffic cameras are mounted around most urban areas and are used primarily
for security reasons. In the City of Cape Town alone, there are around 300 traffic cameras
streaming live video directly to the Transport Management Centre (TMC) database. The
cameras cover the majority of the roads throughout Cape Town, and would therefore
provide unparalleled access to essential video data.
There are some areas throughout Cape Town that are not yet monitored by traffic
cameras. The cameras and related infrastructure are expensive to install, and require
many man hours to complete. A particularly attractive solution to this problem is to erect
simple landing platforms that would allow an autonomous Unmanned Aerial Vehicle (UAV)
to conduct fully autonomous traffic flow analyses.
The work explained in this paper makes use of pure computer vision techniques to
automatically compute traffic metrics along road segments. This paper will focus primarily
on uninterrupted flow in the form of freeways and national highways. A key objective was
to make the system as flexible as possible to maximise the capabilities of the estimation
techniques. One of the main reasons for optimising flexibility originates from the novel
concept of using both pole-mounted traffic camera footage, as well as footage obtained
from other sources such as, but not limited to, the UAV's on-board camera.
The work discussed in this paper proposes a way of autonomously computing key traffic
flow descriptors using pure computer vision techniques. The inclusion of an autonomous
aircraft provides a novel means for obtaining essential video footage.
RELATED WORK
Various methods of intelligent traffic monitoring have been proposed throughout the years,
most of which employ computer vision techniques to detect and track passing vehicles.
This section will discuss similar works in literature and explain their contributions and
shortfalls.
A method proposed by (Koller et al., 1994) uses traffic scene information to optimise traffic
flow during busy periods, to identify stalled vehicles and accidents, and to aid the decision-
making of an autonomous vehicle controller. The system employs a contour tracker and an
affine motion model based on Kalman filters to extract vehicle trajectories over a sequence
of traffic scene images (Koller et al., 1994). This system is slightly different to the one
proposed in this paper as it does not focus on computing traffic flow metrics for
performance analyses, but rather to aid in real-time decision-making for an autonomous
vehicle controller.
A method proposed by (Muthukumar & Chintalacheruvu, 2012) makes use of the Harris-
Stephen corner detector algorithm to efficiently detect vehicles in a video stream. The
system was designed to detect and compute vehicle counts and speeds at arterial
roadways and free-ways. The goal was to develop a system that would eliminate the need
for calibration and have robustness against contrast variations. Similar to the
aforementioned system, the system proposed by Muthukumar and Chintalacheruvu was
designed primarily as an advanced warning and traffic control system.
One of the main inhibiting factors of using computer vision for traffic detection is the fact
that these visual-based systems do not perform well under low-light conditions. A system
proposed by (Kannegulla et al., 2013) makes use of thermal imaging cameras and pure
computer vision techniques to detect and track vehicles under extreme illumination
conditions. The combination of thermal imaging technology and highly optimised computer
vision techniques, allowed for the development of a system that would measure traffic
density extremely accurately. As was the case with the previous two systems, the system
proposed by Kannegulle et al. is not focused on generating traffic flow descriptors for
future road planning, but simply to optimise the immediate flow of traffic.
Table 1 lists some of the features associated with the above-mentioned systems.
Table 1: List of features for each of the aforementioned systems.
Koller et al.
Chintalacheruvu et al.
Kannegulle et al.
Identify stalled vehicles and
accidents.
Compute vehicle count.
Compute vehicle speed.
Designed primarily for
optimising traffic flow.
Used as an autonomous
vehicle controller.
Contour tracking and affine
motion model based on
Kalman filters.
No calibration required.
Performs well under low-light
conditions.
Highly optimised for real-time
performance.
Makes use of thermal
imaging cameras.
Performs extremely well
under low-light conditions.
Highly accurate computation
of traffic density.
SYSTEM DESIGN
The autonomous drone-based traffic flow estimation system can effectively be separated
into two parts. The first part consists of the computer vision system used to detect and
calculate vehicle velocities for calculation of key traffic metrics. The second part involves
the design of an autonomous target tracking and landing system for the UAV.
Figure 1 shows the hardware components and their corresponding methods of
communication. The ground station communicates with a GSM modem via a USB-Serial
connection. Commands are sent from the ground station to the modem as simple AT
strings. The modem interprets these strings, and prepares the IP packets to be sent over
the mobile network. Data is transmitted to a cloud-hosted database via mobile network,
where it is interpreted by Trintel's SMART platform, and displayed graphically on an online
dashboard.
The ground station communicates with the Parrot AR drone via its Wi-Fi module as shown
in figure 1. The SDK network library handles the network interfacing between the ground
station and the drone. Reference angles and angular velocity references are sent to the
drone as fractions of the maximum set-point values.
The design phase is separated into two subsystem designs. The first detailed design is
concerned with the traffic flow estimation process, and has a specific focus on the
supporting computer vision techniques. The second subsystem design focuses on the
control system and additional computer vision techniques used in the automation of the
drone's flight control.
Traffic flow estimation
The main traffic flow algorithm is required to automatically detect the number of vehicles
that pass through a given area, as well as to determine their relative velocities. Once the
Figure 1: Hardware integration
vehicles are detected and their velocities estimated, they are then classified according to
relative size (motorbikes, cars and trucks).
A particularly challenging aspect was to design a system that relied entirely on visual
references. The idea was to design and implement a non-intrusive system that makes use
of existing traffic cameras placed around a city. It is important to note that traffic cameras
need not be the only source of video feed. As mentioned earlier in this paper, the idea is to
eventually incorporate an unmanned aircraft into the system that can autonomously fly to
remote locations which might not currently have an established traffic camera network.
The system is required to be extremely flexible in order to accommodate a variety of
different video sources, and therefore relies heavily on highly adaptive computer vision
techniques to compute all traffic metrics.
Road profile creation
Every road location is unique in the way in which the static traffic cameras are placed. This
causes a potential problem, especially when computing relative vehicle velocities as well
as classifications based on relative vehicle sizes. An elegant and particularly robust
solution was developed to deal with this problem. The idea was to create and save road
profiles that would store all location-specific information. Due to the static nature of the
pole-mounted traffic cameras, road profiles would only need to be generated once for each
location. If the drone is to be used for traffic analysis, a location profile would have to be
generated each time it lands to accommodate for orientation-specific parameters.
In an attempt to make the road profile creation process a more user-friendly experience,
an interactive, self-learning method was designed. The method involved a three-stage
creation process with the first stage being fully autonomous, and the last two requiring
some basic user input. Once a user has input the necessary parameters, the system
stores all location-specific data in a uniquely identifiable DAT file.
Background modelling
The key challenge to realising the system is successfully identify and track objects in a
video stream. The Background Subtraction (BS) technique, for use in computer vision, is
designed to successfully differentiate a moving object from its corresponding static
background scene. The system discussed in this paper makes use of the BS technique for
the detection and tracking of passing vehicles.
In order to conduct background subtraction, it is necessary to obtain a model of the static
background scene. Background modelling consists of two primary phases - phase one is
responsible for background initialisation while phase two is aimed at updating the
background model. A good approximation of the static background scene is obtained by
making use of a running frame average technique. Figure 2 shows the result of the running
average algorithm when approximating the background scene. The approximated
background is a good initialisation point for the background modelling process.
Once the background scene has been approximated, each individual background pixel is
then modelled using a Mixture of Gaussian distributions (MoG modelling technique).
A mixture of N Gaussian distributions is then generated to model each individual
background pixel. Background pixels are characterised based on their persistence and
variance. Equation 1 represents the Gaussian Mixture Model (GMM) equation:
()= (|,)
 (1)
Where the multivariate Gaussian distribution is given by
(|,)=
||/exp [
(  )
(  )] (2)
As the illumination environment changes throughout the operational life of the model,
updates are required to ensure model accuracy. The background model is updated with
each successive frame so as to accommodate for the various illumination effects. The
learning rate parameter specifies the rate at which the model is updated - this parameter
was optimised by means of empirical investigation.
Shadow removal
The use of the BS algorithm does not provide a complete solution with regards to object
detection. A particular disadvantage of using the MoG technique, is that the object
shadows tend to be classified as part of the foreground. The reason for this is that
shadows share the same movement patterns as the objects that create them. Shadows
also tend to exhibit similar pixel intensity characteristics as the corresponding foreground
objects (Lovell et al., 2012). When two vehicles are in close proximity to one another, their
corresponding shadows make them appear as a single object - leading to reduced tracking
and counting accuracy.
The shadow detection technique used in this system is based on the chromaticity
characteristics of shadows. Chromaticity is a measure of colour that is independent of
intensity (Lovell et al., 2012). The idea behind the method of chromaticity is to detect
shadows based on their pixel characteristics. There are three primary characteristics that
distinguish shadow pixels from their non-shadow counterparts. It is known that the
intensity of a shadow pixel (V in HSV) is lower than that of the corresponding background
pixel (Lovell et al., 2012). Furthermore, it known that a shadow cast on the background
does not change the pixel hue (H in HSV), and that a shadow pixel often exhibits lower
saturation (S in HSV) characteristics (Lovell et al., 2012). A pixel p is therefore considered
a shadow pixel based on the above-mentioned criterion. Once the frame coordinates of
the shadow pixels have been identified, the corresponding pixels are subsequently
Figure 2: Running average technique. The left-most image shows the vehicles
moving across the road segment, while the right-most image shows the result of
removed from the foreground mask (result of the background subtraction) before being put
through a bilateral filter to minimise noise.
Vehicle speed detection
Traffic flow estimation theory does not only depend on the number of vehicles passing
through a specific road location, but on the relative velocities of the vehicles as well.
Vehicle velocities are usually obtained using radar guns, inductive loops and IR counters
(National Research Council, 2010). However, these methods are seen as intrusive, as
additional hardware needs to be incorporated into the existing road structure. A particularly
attractive alternative is to use the existing camera infrastructure to automatically compute
relative vehicle velocities.
Velocity is a measure of object displacement per unit time. In order to determine the
velocity of passing vehicles, it is necessary to first obtain the displacement (distance) of
the vehicles between consecutive frames. Once the displacement of the pixels
corresponding to the individual vehicles is known, the frame rate of the video can be used
to compute the relative vehicle velocities. Equation 3 represents the velocity equation:
=     
    (/ ) (3)
Optical flow tracking provides a way of determining pixel displacement between
consecutive frames. Optical flow operates under two primary assumptions. The first
assumption is based on the fact that the pixel intensities of an object should remain
constant between consecutive frames (OpenCV, 2011). The second assumption is that
neighbouring pixels will have a similar motion to that of the pixel under observation
(OpenCV, 2011).
Any optical flow algorithm involves complex mathematical calculations conducted on a
large number of individual pixels. Therefore, the implementation of a highly optimised
algorithm is necessary to ensure real-time performance. The OpenCV platform includes an
optical flow tracking method based specifically on the method proposed by (Lucas &
Kanade, 1981). The method requires unique feature points on the objects in order to track
pixels accurately. According to (Shi & Tomasi, 1994), corners of an object are good
features to track and are therefore used in the optical flow tracking process. Figure 3
shows the optical flow vectors superimposed onto the moving vehicle.
Traffic flow metric computations
Autonomous traffic flow estimation is recognised as the fundamental core of this system.
Determining the total vehicle count and respective vehicle velocities in the previous
sections was a necessary step in computing traffic flow metrics. It was decided that the
following metrics would be useful in describing uninterrupted traffic flow data: Time Mean
Speed (TMS), Volume, Flow Rate, Density, Peak Hour Factor (PHF) and Level of service
(LOS).
Once the system is able to identify the number of vehicles moving through a particular
road segment (background subtraction) and their corresponding velocities (optical flow
tracking), the above-mentioned traffic flow metrics are then autonomously generated
based on predetermined equations described by the Highway Capacity Manual (National
Research Council, 2010).
Autonomous aircraft
The autonomous aircraft was included to provide a novel form of autonomy for future traffic
analyses. The idea is that the drone will eventually fly to pre-determined destinations using
a GPS navigation system. Once the drone is within visual range of the landing platform, a
unique identifier in the form of a checkerboard pattern will be used as a reference for the
visual target tracking system. When the control system has stabilised the drone in front of
the target, an autonomous landing system will land the drone on the platform below. The
drone's front-facing camera (FFC) can then be used as a mobile substitute for the static
pole-mounted traffic cameras.
Target tracking
In order to detect whether a checkerboard shape is currently in the frame, each frame is
converted to a greyscale image to maximise the distinction between the black and white
checker squares. The frame is then put through a binary threshold function before a
pattern recognition algorithm is used to identify the location of the checkerboard. Figure 4
shows the drone On Screen Display (OSD) once the target has been identified. An
algorithm, running on the ground station, determines the translation and rotation of the
checkerboard in 3D space. This information is then used by a feedback PID control system
to automate the drone's flight and ultimately stabilise it at a set distance from the target
position.
Feedback PID control system
The Ziegler-Nichols tuning method was selected as the primary PID tuning methodology. A
particular advantage of using this technique, is that it does not require a mathematical
model of the plant. Instead, the technique is carried out with Hardware In the Loop (HIL)
Figure 4: On Screen Display (OSD)
investigations. This allows for the parameters to be tuned according to the actual plant
dynamics, thereby contributing to the design of a more effective practical controller.
RESULTS
The aim of this section is to discuss and reflect upon the results observed throughout the
system testing procedure. As with the detailed design section, this section once again
deals with the two distinct subsystems individually. The individual subsystems were tested
independently before the final integration and testing was completed.
Traffic flow estimation
System testing and results analysis is an essential part of determining the efficacy of the
methods used in the final system. The performance of the computer vision techniques
were tested using four carefully selected video sources. The test videos were chosen to
test the system under various degrees of tracking difficulty.
Test videos 1 and 2 were chosen as the baseline comparison tests. Test video 3 was
chosen due to the lower overall illumination caused by bad weather conditions. Test video
4 was chosen due to the position of the illumination source during the analysis hour.
In each case, the actual vehicle count is compared to that of the measured vehicle count;
with and without shadow removal. Table 2 shows the accuracy of the vehicle counting
algorithm before and after the shadow removal technique is implemented.
Table 2: Vehicle counting results
With shadows
Shadows removed
Test
Actual count
Count
Accuracy
Count
Accuracy
Video 1
36
48
67%
36
100%
Video 2
81
53
65%
79
98%
Video 3
29
20
69%
25
86%
Video 4
42
25
60%
50
81%
The results from table 2 conclude that shadow removal improves counting accuracy by at
least 15 percentage points. The most obvious reason for this increase is attributed to the
way the system interprets shadow characteristics. Without the shadow removal
functionality, separate vehicles in close proximity to one another are sometimes counted
as a single vehicle. In other cases, a shadow is seen as a completely separate moving
entity, leading to a single vehicle being counted twice.
Figure 5: Shadow removal result. Left-most image shows the foreground mask
before shadow removal. Right
-most image shows the result of the shadow removal
algorithm.
In order to test the accuracy of the velocity computations, the actual vehicle velocities were
compared to that of the system measured velocities. Two test vehicles were driven past a
pole-mounted camera at speeds of 10, 20, 30 and 40km/h (according to speedometer
readings). Table 3 shows the vehicle speed estimation results.
Table 3: Vehicle speed results: Test vehicle 1
Run
Direction
Estimated
Speed
(km/h)
Speedometer
Reading
(km/h)
Difference
(km/h)
Accuracy
1
Right to Left
11.4
10
-1.4
86%
2
Left to Right
21.5
20
-1.5
93%
3
Left to Right
28.6
30
+1.4
95%
4
Right to Left
41.8
40
-1.8
96%
Drone control
The ability of the drone to track a target and minimise the error signal would give an
indication of the tracking system performance. The most important indicator would be
determined by the efficacy of the landing algorithm to successfully land the drone on an
80x80 cm platform.
In order to obtain a quantitative measure of the landing accuracy, 34 test landings were
conducted. After each landing the relative distance from the centre of the platform to the
hull of the drone was measured. This measurement would give an indication of the landing
accuracy, and would facilitate the calculation of a successful landing probability. Figure 6
shows a scatter plot of the aircraft position after each successive landing.
Tests were conducted in semi-ideal conditions, where minimal external disturbances were
experienced. Prop wash from the aircraft did, however, result in some air turbulence. It is
apparent, however, that the control system was able to deal with these disturbances and
ultimately stabilise the aircraft above the landing platform. The results depicted in figure 6
show that out of the 34 test runs conducted, 100% were successful landings.
It is impossible to guarantee that the drone will land on the centre of the platform each and
every time. To deal with this limitation, the traffic tracking algorithm was designed to be as
flexible and as adaptable as possible so that the position of the drone was not of concern.
A background model is generated based on the current position of the camera feed, which
inherently minimises the limitations placed on the position of the source.
Figure 6: Landing coordinate scatter plot.
CONCLUSION
This paper addressed two key challenges in the field of traffic flow estimation laborious
manual vehicle counting and the need for multiple and fixed recording infrastructure. The
former challenge was addressed by automating vehicle detection and automatic
calculation of traffic flow metrics using computer vision techniques. The latter challenge
was addressed by introducing drone-based recording equipment that uses pole-mounted
landing platforms, making it especially useful in for use in remote and fiscally challenged
areas. The results demonstrate that the solutions work with high accuracy, with detection
ranging from 81% to 100%, and 100% landing accuracy for the drone.
REFERENCES
A demonstration video of the complete system is available online at: http://goo.gl/jT7lke
Gopal, M., 2010. Digital control and state variable methods. Mc Graw Hill.
Kannegulla, A., Reddy, A.S., Sudhir, K.V.R.S. & Singh, S., 2013. Thermal Imaging system for Precise Traffic
Control and Surveillance. International Journal of Scientific & Engineering Research, 4(11), pp.464-
67.
Koller, D. et al., 1994. Towards robust automatic traffic scene analysis in real-time. In Computer Vision amp;
Image Processing., Proceedings of the 12th IAPR International Conference., 1994.
Lovell, C., Sanin, A., Sanderson, C. & Brain, C., 2012. Shadow Detection: A Survey and Comparative
Evaluation of Recent Methods. Pattern Recognition, 45(4), pp.1684--1695.
Lucas, B. & Kanade, T., 1981. Proceedings DARPA Image Understanding Workshop., 1981.
Muthukumar, N. & Chintalacheruvu, V., 2012. Video Based Vehicle Detection and its Application in
Intelligent Transportation Systems. Journal of Transportation Technologies, 2(4), pp.305-14.
National department of transport, 2014. Live vehicle population. [Online] Available
at: http://www.enatis.com/index.php/downloads/doc_download/575-
livevehpopulationclassprov20140331 [Accessed 15 September 2014].
National Research Council, 2010. HCM 2010 : highway capacity manual. Washington, D.C.: Transportation
Research Board.
OpenCV, 2011. Optical flow. [Online] Available
at: http://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_lucas_kanade/py_lucas_kanade.ht
ml [Accessed 12 September 2014].
Shi, J. & Tomasi, C., 1994. Good features to track. In IEEE Computer Society Conference., 1994.
Thies, W. et al., 2013. Accurate speed and density measurement for road traffic in india. In Proceedings of
ACM., 2013. ACM.
... Visual object tracking (VOT), the subfield of computer vision, is a process of continuously estimating the target state through video image sequence. In recent years, VOT has become a very active research domain due to its extensive applications in many sorts of fields such as intelligent surveillance [1], automatic driving [2], and traffic flow monitoring [3], to name a few. ...
Article
Full-text available
In recent years, visual object tracking has become a very active research field which is mainly divided into the correlation filter-based tracking and deep learning (e.g., deep convolutional neural network and Siamese neural network) based tracking. For target tracking algorithms based on deep learning, a large amount of computation is required, usually deployed on expensive graphics cards. However, for the rich monitoring devices in the Internet of Things, it is difficult to capture all the moving targets in each device in real time, so it is necessary to perform hierarchical processing and use tracking based on correlation filtering in insensitive areas to alleviate the local computing pressure. In sensitive areas, upload the video stream to a cloud computing platform with a faster computing speed to perform an algorithm based on deep features. In this paper, we mainly focus on the correlation filter-based tracking. In the correlation filter-based tracking, the discriminative scale space tracker (DSST) is one of the most popular and typical ones which is successfully applied to many application fields. However, there are still some improvements that need to be further studied for DSST. One is that the algorithms do not consider the target rotation on purpose. The other is that it is a very heavy computational load to extract the histogram of oriented gradient (HOG) features from too many patches centered at the target position in order to ensure the scale estimation accuracy. To address these two problems, we introduce the alterable patch number for target scale tracking and the space searching for target rotation tracking into the standard DSST tracking method and propose a visual object multimodality tracker based on correlation filters (MTCF) to simultaneously cope with translation, scale, and rotation in plane for the tracked target and to obtain the target information of position, scale, and attitude angle at the same time. Finally, in Visual Tracker Benchmark data set, the experiments are performed on the proposed algorithms to show their effectiveness in multimodality tracking. 1. Introduction Visual object tracking (VOT), the subfield of computer vision, is a process of continuously estimating the target state through video image sequence. In recent years, VOT has become a very active research domain due to its extensive applications in many sorts of fields such as intelligent surveillance [1], automatic driving [2], and traffic flow monitoring [3], to name a few. In fields such as security monitoring and control, the traditional network architecture is difficult to deal with in terms of network delay and security reliability, and thus edge computing technology was born. Tasks with different attributes can be passed to different levels for processing. Zhan [4] shows that the first few feature extraction layers could run on edge device, and the others run on the cloud. And Gao [5, 6] divides tasks into different levels according to the business applications and using edge devices in one level. As Figure 1 shows, for nonsensitive areas, video streams with lower resolutions can be processed on the local device; in the medium area, ordinary-resolution video streams can be used on the edge device; and in high-risk areas, high-resolution video streams can be used on the core cloud server, thereby reducing network bandwidth and improving the overall operating efficiency of the system. This article mainly explores the processing of video streams on edge clouds, and the tracking algorithm used is based on filtering.
... In addition, the capability of geographic information systems was used to represent spatiotemporal phenomena. Hardware components and their communication are described in [14]. ...
Conference Paper
Full-text available
This article provides an overview of existing drone applications in the delivery of goods, planning, design and monitoring of transport infrastructure, as well as an overview of the possibilities of drone applications in the professional and scientific traffic and transportation engineering problems.
... YOLO and SSD are examples of single-shot detectors. YOLO is designed for real time execution and by design provides a trade-off that favors high performance over accuracy [7]. ...
Conference Paper
In recent years Unmanned Aerial Vehicles (UAV s) are used for applications like surveillance, search and rescue, object detection, and target tracking. As an emerging technology, Unmanned Aerial Vehicles (drones) are under extensive study and new applications are being explored. Drones have complex dynamics and to develop an application for real time implementation is a challenging task. In this work, an approach to detect and track the target object (moving or still) for a drone is presented. Convolutional Neural Network (CNN) is used for object detection, and target tracking is performed using simple PID controllers. The system is implemented on a quadcopter equipped with an embedded Artificial Intelligence (AI) computing device NVIDIA Jetson TX2, a USB camera, and a Pixhawk flight controller. Several experiments were conducted in real time and the response of the drone was observed. Experimental results show that the proposed approach for object detection and target tracking works well under normal and varying environmental circumstances.
... YOLO and SSD are examples of single-shot detectors. YOLO is designed for real-time execution and, by design, provides a trade-off favoring high performance over accuracy [7]. ...
Article
Full-text available
Recent advancements in the field of Artificial Intelligence (AI) have provided opportunity to create autonomous devices, robots, and machines characterized particularly with the ability to make decisions and perform tasks without human mediation. One of these devices, Unmanned Aerial Vehicles (UAVs) or drones are widely used to perform tasks like surveillance, search and rescue, object detection and target tracking, parcel delivery (recently started by Amazon), and many more. The sensitivity in performing said tasks demands that drones must be efficient and reliable. For this, in this work, an approach to detect and track the target object, moving or still, for a drone is presented. Parrot AR Drone 2 is used for this application. Convolutional Neural Network (CNN) is used for object detection and target tracking. The object detection results show that CNN detects and classifies object with a high level of accuracy (98%). For real-time tracking, tracking algorithm responds faster than conventionally used approaches, efficiently tracking the detected object without losing it from sight. Calculated based on several iterations, the efficiency achieved for target tracking is 96.5%.
Article
In present road traffic system, drone-network based traffic monitoring using the Internet of Vehicles (IoVs) is a promising solution. However, camera-based traffic monitoring does not collect complete data, cover all areas, provide quick medical services, or take vehicle follow-ups in case of an incident. Drone-based system helps to derive important information (such as commuter's behavior, traffic patterns, vehicle follow-ups) and sends this information to centralized or distributed authorities for making traffic diversions or necessary decisions as per laws. The present approaches fail to meet the requirements such as (i) collision free, (ii) drone navigation, and (iii) less computational and communicational overheads. This work has considered the collision-free drone-based movement strategies for road traffic monitoring using Software Defined Networking (SDN). The SDN controllable drone network results in lesser overhead over drones and provide efficient drone-device management. In simulation, two case studies are simulated using JaamSim simulator. Results show that the zones-based strategy covers a large area in few hours and consume 5 kWs to 25 kWs energy for 150 drones (Case study 1). Zone-less based strategies (case study-2) show that the energy consumption lies between 5 kWs to 18 kWs for 150 drones. Further, the use of SDN-based drones controller reduces the overhead over drone-network and increases the area coverage with a minimum of 1.2% and maximum of 2.6%. Simulation (using AnyLogic simulator) shows the 3D view of successful implementation of collision free strategies.
Article
Full-text available
The awareness of using helmet is still far from the expectation. Often, the motorcycle riders and passengers do not use helmet. It is very dangerous for them and other travelers. However, it is not easy to find and analyze the riders that use or do not use helmet, furthermore to get data of the impact of using or not using helmet in the accident. One of the way to get such data is from “Zebra Operation”. Nowadays, the development of technology of Aerial Photo and geospatial data can be obtained from small format aerial photo. It can be obtained by the means of Unmanned Aerial Vehicle (UAV) with the attached camera on the UAV so that the geospatial data and motor riders without using helmet can be detected. Rider’s behavior can be observed by using UAV or Drone. The aim of this paper is to(1) find the method of observation the travel behavior of riders without using helmet which can endanger other traveler’s safety by using drone (2) find the method of finding the location of accident, type and cause of the accident as quick as possible by using drone. Finding of this paper is a reconstruction method of accident, geometric characteristics of the location, type and cause of the accident as quick as possible. Kesadaran penggunaan helm saat ini masih jauh dari harapan. Seringkali terlihat pengendara sepeda motor dan penumpang yang diboncengnya tidak menggunakan helm. Hal ini sangat membahayakan keselamatannya dan keselamatan banyak orang disekitarnya.Tidak mudah untuk menganalisis jumlah pengguna helm dan pelanggaran lalu lintas, apalagi untuk mengetahui dampak tidak menggunakan helm terhadap korban kecelakaan lalu lintas. Suatu cara termudah untuk mendapatkan data adalah memanfaatkan data dari operasi Zebra. Dengan perkembangan teknologi foto udara, perolehan data permukaan bumi dapat dilakukan dengan menggunakan foto udara format kecil. Foto udara format kecil diperoleh dengan bantuan wahana pesawat udara tanpa awak atau Unmanned Aerial Vehicle dengan meletakkan kamera pada pesawat tersebut sehingga dapat diperoleh data permukaan bumi sesuai dengan yang direncanakan. Perilaku pengendara yang tidak menggunakan helm dan mengabaikan keselamatan diamati dengan menggunakan drone.Tujuan kajian ini adalah untuk:(1) menghasilkan metode pengamatan perilaku pengendara yang tidak menggunakan helm dan yang menyebabkan kecelakaan atau mengabaikan keselamatandengan menggunakan drone,dan (2) menghasilkan metode untuk mendapatkan data kecelakaan, lokasi jenis, dan penyebab kecelakaan secepat mungkin dengan menggunakan drone. Temuan dari paper ini adalah metoda rekonstruksi kecelakaan, karakteristik geometris lokasi, jenis dan penyebab kecelakaan sesegera mungkin.
Preprint
Drones or Unmanned Aerial Vehicles (UAVs) have become a reliable and efficient tool for road traffic monitoring. Compared to loop detectors and bluetooth receivers (with high capital and operational expenditure), drones are a low-cost alternative that offers great flexibility and high quality data. In this work, we derive optimized tour plans that a fleet of drones can follow for rapid traffic monitoring across particular regions of transportation network. To derive these tours, we first identify monitoring locations over which drones should fly through and then compute minimum travel-time tours based on realistic resource constraints. Evaluation results are presented over a real road network topology to demonstrate the applicability of the proposed approach.
Article
Road traffic accidents are one of the leading causes of deaths and injuries in the word resulting in the not only loss of precious human lives but also affect the economic resources. According to the World Health Organization (WHO), over 1.35 million people are killed, and over 50 million are injured due to road accidents throughout the world. Unfortunately, as compared to other developing countries with the same ratio of vehicle possession, in Saudi Arabia, the fatalities and injuries are much higher. Every year around 7000-9000 people die, and over 39000 serious injuries occur in road accidents. There is at least one accident happens every minute in Saudi Arabia. To decrease the road traffic accidents, fatalities, and injuries caused by them, the Saudi Ministry of Interior came up with new rules, regulations, and hefty fines. Also, they introduced a new traffic system called the SAHER system. Still, due to the static nature and other limitations of the system, the drivers found loopholes and ways to deceive the system to avoid the fines and not being caught by the system. The most common violation includes excess speed, abrupt deceleration, and distracted driving. In this paper, we propose a smart traffic surveillance system based on Unmanned Aerial Vehicle (UAV) using 5G technology. This traffic monitoring system covers the existing limitations of the SAHER system deployed in KSA. By overcoming the existing limitations and loopholes of the SAHER system, it is observed that the number of accidents and fatalities can be decreased. The projected results show that those violations when to overcome, the number of accidents per year falls to 299,317 leading to 4,868 deaths and 33,199 injuries for 1st year, and in the next five years the number of deaths and will be decreased to 3,745 and injuries to 16,600 based on the current data available. We aim the system will further reduce the number of accidents and fatalities and injuries caused by it.
Article
Full-text available
Video based vehicle detection technology is an integral part of Intelligent Transportation System (ITS), due to its non-intrusiveness and comprehensive vehicle behavior data collection capabilities. This paper proposes an efficient video based vehicle detection system based on Harris-Stephen corner detector algorithm. The algorithm was used to develop a standalonevehicle detection and tracking system that determines vehicle counts and speeds at arterial roadways and freeways. The proposed video based vehicle detection system was developed to eliminate the need of complex calibration, robustness to contrasts variations, and better performance with low resolutions videos. The algorithm performance for accuracy in vehicle counts and speed was evaluated. The performance of the proposed system is equivalent or better compared to a commercial vehicle detection system. Using the developed vehicle detection and tracking system an advance warning intelligent transportation system was designed and implemented to alert commuters in advance of speed reductions and congestions at work zones and special events. The effectiveness of the advance warning system was evaluated and the impact discussed.
Article
Full-text available
This paper presents a survey and a comparative evaluation of recent techniques for moving cast shadow detection. We identify shadow removal as a critical step for improving object detection and tracking. The survey covers methods published during the last decade, and places them in a feature-based taxonomy comprised of four categories: chromacity, physical, geometry and textures. A selection of prominent methods across the categories is compared in terms of quantitative performance measures (shadow detection and discrimination rates, colour desaturation) as well as qualitative observations. Furthermore, we propose the use of tracking performance as an unbiased approach for determining the practical usefulness of shadow detection methods.The evaluation indicates that all shadow detection approaches make different contributions and all have individual strength and weaknesses. Out of the selected methods, the geometry-based technique has strict assumptions and is not generalisable to various environments, but it is a straightforward choice when the objects of interest are easy to model and their shadows have different orientation. The chromacity based method is the fastest to implement and run, but it is sensitive to noise and less effective in low saturated scenes. The physical method improves upon the accuracy of the chromacity method by adapting to local shadow models, but fails when the spectral properties of the objects are similar to that of the background. The small-region texture based method is especially robust for pixels whose neighbourhood is textured, but may take longer to implement and is the most computationally expensive. The large-region texture based method produces the most accurate results, but has a significant computational load due to its multiple processing steps.
Conference Paper
Full-text available
Automatic symbolic traffic scene analysis is essential to many areas of IVHS (Intelligent Vehicle Highway Systems). Traffic scene information can be used to optimize traffic flow during busy periods, identify stalled vehicles and accidents, and aid the decision-making of an autonomous vehicle controller. Improvements in technologies for machine vision-based surveillance and high-level symbolic reasoning have enabled the authors to develop a system for detailed, reliable traffic scene analysis. The machine vision component of the system employs a contour tracker and an affine motion model based on Kalman filters to extract vehicle trajectories over a sequence of traffic scene images. The symbolic reasoning component uses a dynamic belief network to make inferences about traffic events such as vehicle lane changes and stalls. In this paper, the authors discuss the key tasks of the vision and reasoning components as well as their integration into a working prototype. Preliminary results of an implementation on special purpose hardware using C-40 Digital Signal Processors show that near real-time performance can be achieved without further improvements
Article
Thermal imaging is a concurrent technology used in many applications like power line maintenance, surveillance and intelligent transportation systems. This paper focuses on traffic control and surveillance using thermal imaging cameras. With the combination of two powerful technologies- thermal imaging and image processing, a very accurate measure of traffic density has been achieved, unhindered by any environmental factors like low visibility due to fog or darkness, or other stray objects like animals or humans. The simulations of the gray scale thermal images captured are performed in Matlab, using which the exact count of vehicles on the road is obtained. This paper not only presents a novel method for speeding up traffic flow, but also overcomes the limitations of existing techniques like implementation costs and precision in determination of traffic density. Future research in the same direction consists of fire alarm and detection of accidents and explosives.
Conference Paper
Monitoring traffic density and speed helps to better manage traffic flows and plan transportation infrastructure and policy. In this paper, we present techniques to measure traffic density and speed in unlaned traffic, prevalent in developing countries, and apply those techniques to better understand traffic patterns in Bengaluru, India. Our techniques, based on video processing of traffic, result in about 11% average error for density and speed compared to manually-observed ground truth values. Though we started with intuitive and straight-forward image processing tools, due to a myriad of non-trivial issues posed by the heterogeneous and chaotic traffic in Bengaluru, our techniques have grown to be non-obvious. We describe the techniques and their evaluation, with details of why simpler methods failed under various circumstances. We also apply our techniques to quantify the congestion during peak hours and to estimate the gains achievable by shifting a fraction of traffic to other time periods. Finally, we measure the fundamental curves of transportation engineering, relating speed vs. density and flow vs. speed, which are integral tools for policy makers.
No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments. 1 Introduction IEEE Conference on Computer Vision and Pattern Recognition (CVPR94) Seattle, June 1994 Is feature tracking a solved problem? The extensive studies of image correlation [4], [3], [15], [18], [7], [17] and sum-of-squared-difference (SSD...
HCM 2010 : highway capacity manual
National Research Council, 2010. HCM 2010 : highway capacity manual. Washington, D.C.: Transportation Research Board.
Digital control and state variable methods
  • M Gopal
Gopal, M., 2010. Digital control and state variable methods. Mc Graw Hill.
Optical flow. [Online] Available at: http://docs.opencv.org/trunk
  • Opencv
OpenCV, 2011. Optical flow. [Online] Available at: http://docs.opencv.org/trunk/doc/py_tutorials/py_video/py_lucas_kanade/py_lucas_kanade.ht ml [Accessed 12 September 2014].
Thermal Imaging system for Precise Traffic Control and Surveillance
  • A Kannegulla
  • A S Reddy
  • K V R S Sudhir
  • S Singh
Kannegulla, A., Reddy, A.S., Sudhir, K.V.R.S. & Singh, S., 2013. Thermal Imaging system for Precise Traffic Control and Surveillance. International Journal of Scientific & Engineering Research, 4(11), pp.464-67.