James Ferryman

University of Reading · School of Systems Engineering

Wide area surveillance has become of critical importance, particularly for border control between countries where vast forested land border areas are to be monitored. In this paper, we address the problem of the automatic detection of activity in forbidden areas, namely forested land border areas. In order to avoid false detections, often triggered...

Chapter 16, “FOLDOUT: A Through Foliage Surveillance System for Border Security” was previously published non-open access. It has now been changed to open access under a CC BY 4.0 license and the copyright holder updated to ‘The Author(s)’. The book has also been updated with this change.

This paper investigates the use of Siamese networks for trajectory similarity analysis in surveillance tasks. Specifically, the proposed approach uses an auto‐encoder as a part of training a discriminative twin (Siamese) network to perform trajectory similarity analysis, thus presenting an end‐to‐end framework to perform an online motion pattern ex...

The objective of the European Union (EU) in the field of external border protection is to safeguard the freedom of movement within the Schengen area, and to ensure efficient monitoring of people who cross EU's external borders. To achieve an effective and efficient border management, there is a need for applying enhanced technologies and methods th...

In this work, we present a Fusion and Tracking system developed within the EU project FOLDOUT aimed to facilitate border guards work by fusing separate sensor information and presenting automatic tracking of objects detected in the surveillance area. The focus of FOLDOUT is on through-foliage detection in the inner and outermost regions of the EU....

The objective of the European Union (EU) in the field of external border protection is to safeguard the freedom of movement within the Schengen area, and to ensure efficient monitoring of people who cross EU's external borders. To achieve an effective and efficient border management, there is a need for applying enhanced technologies and methods th...

Improved methods for border surveillance are necessary to ensure an effective and efficient EU border management. In the border control context, as defined by the Schengen Border Code, border surveillance is defined as “the surveillance of borders between border crossing points and the surveillance of border crossing points outside the fixed openin...

Pervasive and useR fOcused biomeTrics bordEr projeCT (PROTECT) is an EU project funded by the Horizon 2020 research and Innovation Programme. The main aim of PROTECT was to build an advanced biometric-based person identification system that works robustly across a range of border crossing types and that has strong user-centric features. This work p...

Several measures for evaluating multi-target video trackers exist that generally aim at providing ‘end performance.’ End performance is important particularly for ranking and comparing trackers. However, for a deeper insight into trackers’ performance it would also be desirable to analyze key contributory factors (false positives, false negatives,...

This work presents the 2nd Cross-Spectrum Iris/Periocular Recognition Competition (Cross- Eyed2017). The main goal of the competition is to promote and evaluate advances in cross-spectrum iris and periocular recognition. This second edition registered an increase in the participation numbers ranging from academia to industry: five teams submitted t...

Privacy protection may be defined as replacing the original content in an image region with a (less intrusive) content having modified target appearance information to make it less recognizable by applying a privacy protection technique. Indeed, the development of privacy protection techniques also needs to be complemented with an established objec...

Spoofing is an act to impersonate a valid user of any biometric systems in order to gain access. In a face biometric system, an imposter might use some fake masks that mimic the real user face. Existing countermeasures against spoofing adopt face texture analysis, motion detection and surface reflection analysis. For the purpose of face anti-spoofi...

Iris localisation and segmentation are challenging and critical tasks in iris biometric recognition. Especially in non-cooperative and less ideal environments, their impact on overall system performance has been identified as a major issue. In order to avoid a propagation of system errors along the processing chain, this paper investigates iris fus...

This work presents a novel dual-spectrum database containing both iris and periocular images synchronously captured from a distance and within a realistic indoor environment. This database was used in the 1st Cross-Spectrum Iris/Periocular Recognition Competition (Cross-Eyed 2016). This competition aimed at recording recent advances in cross-spectr...

In an Internet of Things (IoT) camera-based monitoring application the transmission of images away from the video sensors for processing poses security and privacy risks. Hence, there is a need for an advanced trusted user-centric monitoring system that pushes the application of security and privacy protection closer to the sensor itself and which...

This paper presents an open-source canopy height profile (CHP) toolkit designed for processing small-footprint full-waveform LiDAR data to obtain the estimates of effective leaf area index (LAIe) and CHPs. The use of the toolkit is presented with a case study of LAIe estimation in discontinuous-canopy fruit plantations. The experiments are carried...

In the EU FP7 project IPATCH, we are researching components for a maritime piracy early detection and avoidance system for deployment on merchant vessels. The system combines information from on-board sensors with intelligence from external sources in order to give early warnings about piracy threats. In this paper we present the ongoing work with...

Iris liveness detection methods have been developed to overcome the vulnerability of iris biometric systems to spoofing attacks. In the literature, it is typically assumed that a known attack modality will be perpetrated. Then liveness models are designed using labelled samples from both real/live and fake/spoof distributions, the latter derived fr...

This paper presents an experimental study of different depth sensors. The aim is to answer the question, whether these sensors give accurate data for general depth image analysis. The study examines the depth accuracy between three popularly used depth sensors; ASUS Xtion Prolive, Kinect Xbox 360 and Kinect for Windows v2. The main attention is to...

While a multitude of motion segmentation algorithms have been presented in the literature, there has not been an objective assessment of different approaches to fusing their outputs. This paper investigates the application of 4 different fusion schemes to the outputs of 3 probabilistic pixel-level segmentation algorithms. We performed an extensive...

This paper presents the two datasets (ARENA and P5) and the challenge that form a part of the PETS 2015 workshop. The datasets consist of scenarios recorded by using multiple visual and thermal sensors. The scenarios in ARENA dataset involve different staged activities around a parked vehicle in a parking lot in UK and those in P5 dataset involve d...

This paper presents a quantitative evaluation of a tracking system on PETS 2015 Challenge datasets using well-established performance measures. Using the existing tools, the tracking system implements an end-to-end pipeline that include object detection, tracking and postprocessing stages. The evaluation results are presented on the provided sequen...

Anti-spoofing is attracting growing interest in biometrics, considering the variety of fake materials and new means to attack biometric recognition systems. New unseen materials continuously challenge state-of-the-art spoofing detectors, suggesting for additional systematic approaches to target anti-spoofing. By incorporating liveness scores into t...

Several researchers have presented studies of temporal effects on iris reco gnition accuracy, with varying results on severity of observed effects. The sensitive topic continues to be adversely discussed and the difficulty of isolating performance-impacting factors is immanent. The impact of ageing on segmentation vs. feature extraction has been la...

Multispectral iris recognition uses information from multiple bands of the electromagnetic spectrum to better represent certain physiological characteristics of the iris texture and enhance obtained recognition accuracy. This paper addresses the questions of single versus cross-spectral performance and compares score-level fusion accuracy for diffe...

A Canopy Height Profile (CHP) procedure presented in Harding et al. (2001) for large footprint LiDAR data was tested in a closed canopy environment as a way of extracting vertical foliage profiles from LiDAR raw-waveform. In this study, an adaptation of this method to small-footprint data has been shown, tested and validated in an Australian sparse...

Multibiometrics aims at improving biometric security in presence of spoofing attempts, but exposes a larger availability of points of attack. Standard fusion rules have been shown to be highly sensitive to spoofing attempts - even in case of a single fake instance only. This paper presents a novel spoofing-resistant fusion scheme proposing the dete...

For general home monitoring, a system should automatically interpret people’s actions. The system should be non-intrusive, and able to deal with a cluttered background, and loose clothes. An approach based on spatio-temporal local features and a Bag-of-Words (BoW) model is proposed for single-person action recognition from combined intensity and de...

This paper addresses the issue of activity understanding from video and its semantics-rich description. A novel approach is presented where activities are characterised and analysed at different resolutions. Semantic information is delivered according to the resolution at which the activity is observed. Furthermore, the multiresolution activity cha...

This study has tested Canopy Height Profile (CHP) methodology as a way of effective Leaf Area Index (LAIe) and vertical vegetation profile retrieval at a single-tree level. Waveform and discrete airborne LiDAR data from five swaths as well as from the combined data of five swaths were used to extract LAIe of a single live Callitris glaucophylla tre...

This paper describes the dataset and vision challenges that form part of the PETS 2014 workshop. The datasets are multisensor sequences containing different activities around a parked vehicle in a parking lot. The dataset scenarios were filmed from multiple cameras mounted on the vehicle itself and involve multiple actors. In PETS2014 workshop, 22...

Threat detection is a challenging problem, because threats appear in many variations and differences to normal behaviour can be very subtle. In this paper, we consider threats on a parking lot, where theft of a truck's cargo takes. The theft takes place in very different forms, in the midst of many people who pose no threat. The threats range from...

In this paper we propose an innovative approach for behaviour recognition, from a multicamera environment, based on translating video activity into semantics. First, we fuse tracks from individual cameras through clustering employing soft computing techniques. Then, we introduce a higher-level module able to translate fused tracks into semantic inf...

This paper presents the PETS2009 outdoor crowd image analysis surveillance dataset and the performance evaluation of people counting, detection and tracking results using the dataset submitted to five IEEE Performance Evaluation of Tracking and Surveillance (PETS) workshops. The evaluation was carried out using well established metrics developed in...

We present a method for the recognition of complex actions. Our method combines automatic learning of simple actions and manual definition of complex actions in a single grammar. Contrary to the general trend in complex action recognition, that consists in dividing recognition into two stages, our method performs recognition of simple and complex a...

This paper presents a neuroscience inspired information theoretic approach to motion segmentation. Robust motion segmentation represents a fundamental first stage in many surveillance tasks. As an alternative to widely adopted individual segmentation approaches, which are challenged in different ways by imagery exhibiting a wide range of environmen...

Analysis of human behaviour through visual information has been a highly active research topic in the computer vision community. This was previously achieved via images from a conventional camera, however recently depth sensors have made a new type of data available. This survey starts by explaining the advantages of depth imagery, then describes t...

In this paper we present a set of activity recognition and localization algorithms that together assemble a large amount of information about activities on a parking lot. The aim is to detect and recognize events that may pose a threat to truck drivers and trucks. The algorithms perform zone-based activity learning, individual action recognition an...

Full-waveform laser scanning data acquired with a Riegl LMS-Q560 instrument were used to classify an orange orchard into orange trees, grass and ground using waveform parameters alone. Gaussian decomposition was performed on this data capture from the National Airborne Field Experiment in November 2006 using a custom peak-detection procedure and a...

A number of multi-camera solutions exist for tracking objects of interest in surveillance scenes. Generally, the approach follows the idea of either early fusion (where all cameras are used to make a decision about detection and tracking) or late fusion (where objects are detected and tracked in individual cameras independently, and then the result...

This study has compared preliminary estimates of effective leaf area index (LAI) derived from fish-eye lens photographs to those estimated from airborne full-waveform small-footprint LiDAR data for a forest dataset in Australia. The full-waveform data was decomposed and optimized using a trust-region-reflective algorithm to extract denser point clo...

Video surveillance is a part of our daily life, even though we may not necessarily realize it. We might be monitored on the street, on highways, at ATMs, in public transportation vehicles, inside private and public buildings, in the elevators, in front of our television screens, next to our baby?s cribs, and any spot one can set a camera.

The current state of the art and direction of research in computer vision aimed at automating the analysis of CCTV images is presented. This includes low level identification of objects within the field of view of cameras, following those objects over time and between cameras, and the interpretation of those objects’ appearance and movements with r...

One popular approach for multi-camera detection of pedestrians or other objects of interest in surveillance scenes is to perform background subtraction and project the resulting foreground mask images to a common scene plane using homographies. As the complexity of the scene increases, it is unavoidable that so called "ghost" detections should occu...

Accurate information about vegetation/forest structure, health and growth is needed in many fields of forest management, environmental planning, resource management, fire risk assessment and soil moisture retrievals. Airborne laser scanning has proven over the last nearly two decades to be an invaluable tool in describing vegetation and providing 3...

A method for the estimation of leaf area index, apparent foliage profile and total leaf area of trees from the last significant returns of airborne laser altimetry is presented, and tested, showing the growth of ten individual Eucalypt trees in New South Wales, Australia over a period of five years. Two airborne laser altimetry data sets were acqui...

Forests of decision trees are a popular tool for classification applications. This paper presents an approach to evolving the forest classifier, reducing the time spent designing the optimal tree depth and forest size. This is applied to the task of vehicle classification for purposes of verification against databases at security checkpoints, or ac...

A new generation of advanced surveillance systems is being conceived as a collection of multisensor components such as video, audio, and mobile robots interacting in a cooperating manner to enhance situation awareness capabilities to assist surveillance personnel. The prominent issues that these systems face are the improvement of existing intellig...

In this work we present a novel approach for activity extraction and knowledge discovery from video employing fuzzy relations. Spatial and temporal properties from detected mobile objects are modeled with fuzzy relations. These can then be aggregated employing typical soft-computing algebra. A clustering algorithm based on the transitive closure ca...

There is a rising demand for the quantitative performance evaluation of automated video surveillance. To advance research in this area, it is essential that comparisons in detection and tracking approaches may be drawn and improvements in existing methods can be measured. There are a number of challenges related to the proper evaluation of motion s...

In this paper, we evaluate the Probabilistic Occupancy Map (POM) pedestrian detection algorithm on the PETS 2009 benchmark dataset. POM is a multi-camera generative detection method, which estimates ground plane occupancy from multiple background subtraction views. Occupancy probabilities are iteratively estimated by fitting a synthetic model of th...

This paper presents the results of the crowd image analysis challenge of the PETS2010 workshop. The evaluation was carried out using a selection of the metrics developed in the Video Analysis and Content Extraction (VACE) program and the CLassification of Events, Activities, and Relationships (CLEAR) consortium. The PETS 2010 evaluation was perform...

This paper describes the crowd image analysis challenge that forms part of the PETS 2010 workshop. The aim of this challenge is to use new or existing systems for i) crowd count and density estimation, ii) tracking of individual(s) within a crowd, and iii) detection of separate flows and specific crowd events, in a real-world environment. The datas...

Calibrated cameras are an extremely useful resource for computer vision scenarios. Typically, cameras are calibrated through calibration targets, measurements of the observed scene, or self-calibrated through features matched between cameras with overlapping fields of view. This paper considers an approach to camera calibration based on observation...

The present work presents a new method for activity extraction and reporting from video based on the aggregation of fuzzy relations. Trajectory clustering is first employed mainly to discover the points of entry and exit of mobiles appearing in the scene. In a second step, proximity relations between resulting clusters of detected mobiles and conte...

A new class of shape features for region classification and high-level recognition is introduced. The novel Randomised Region Ray (RRR) features can be used to train binary decision trees for object category classification using an abstract representation of the scene. In particular we address the problem of human detection using an over segmented...

Several pixel-based people counting methods have been developed over the years. Among these the product of scale-weighted pixel sums and a linear correlation coefficient is a popular people counting approach. However most approaches have paid little attention to resolving the true background and instead take all foreground pixels into account. With...

This paper presents the results of the crowd image analysis challenge of the Winter PETS 2009 workshop. The evaluation is carried out using a selection of the metrics developed in the video analysis and content extraction (VACE) program and the classification of events, activities, and relationships (CLEAR) consortium. The evaluation highlights the...

This paper describes the crowd image analysis challenge that forms part of the PETS 2009 workshop. The aim of this challenge is to use new or existing systems for i) crowd count and density estimation, ii) tracking of individual(s) within a crowd, and iii) detection of separate flows and specific crowd events, in a real-world environment. The datas...

Urban surveillance footage can be of poor quality, partly due to the low quality of the camera and partly due to harsh lighting and heavily reflective scenes. For some computer surveillance tasks very simple change detection is adequate, but sometimes a more detailed change detection mask is desirable, eg, for accurately tracking identity when face...