ArticlePDF AvailableLiterature Review

CMOS Image Sensors in Surveillance System Applications

Authors:

Abstract and Figures

Recent technology advances in CMOS image sensors (CIS) enable their utilization in the most demanding of surveillance fields, especially visual surveillance and intrusion detection in intelligent surveillance systems, aerial surveillance in war zones, Earth environmental surveillance by satellites in space monitoring, agricultural monitoring using wireless sensor networks and internet of things and driver assistance in automotive fields. This paper presents an overview of CMOS image sensor-based surveillance applications over the last decade by tabulating the design characteristics related to image quality such as resolution, frame rate, dynamic range, signal-to- noise ratio, and also processing technology. Different models of CMOS image sensors used in all applications have been surveyed and tabulated for every year and application.
Content may be subject to copyright.
A preview of the PDF is not available
Article
Full-text available
Education 4.0 imitates Industry 4.0 in many aspects such as technology, customs, challenges, and benefits. The remarkable advancement in embryonic technologies, including IoT (Internet of Things), Fog Computing, Cloud Computing, and Augmented and Virtual Reality (AR/VR), polishes every dimension of Industry 4.0. The constructive impacts of Industry 4.0 are also replicated in Education 4.0. Real-time assessment, irregularity detection, and alert generation are some of the leading necessities of Education 4.0. Conspicuously, this study proposes a reliable assessment, irregularity detection, and alert generation framework for Education 4.0. The proposed framework correspondingly addresses the comparable issues of Industry 4.0. The proposed study (1) recommends the use of IoT, Fog, and Cloud Computing, i.e., IFC technological integration for the implementation of Education 4.0. Subsequently, (2) the Symbolic Aggregation Approximation (SAX), Kalman Filter, and Learning Bayesian Network (LBN) are deployed for data pre-processing and classification. Further, (3) the assessment, irregularity detection, and alert generation are accomplished over SoTL (the set of threshold limits) and the Multi-Layered Bi-Directional Long Short-Term Memory (M-Bi-LSTM)-based predictive model. To substantiate the proposed framework, experimental simulations are implemented. The experimental outcomes substantiate the better performance of the proposed framework, in contrast to the other contemporary technologies deployed for the enactment of Education 4.0.
Article
This paper presents newly developed two high-precision CMOS proximity capacitance image sensors: Chip A with 12 μm pitch pixels with a large detection area of 1.68 cm2; Chip B with 2.8 μm pitch 1.8 M pixels for a higher resolution. Both fabricated chips achieved a capacitance detection precision of less than 100 zF (10-19 F) at an input voltage of 20 V and less than 10 zF (10-20 F) at 300 V due to the noise cancelling technique. Furthermore, by using multiple input pulse amplitudes, a capacitance detection dynamic range of up to 123 dB was achieved. The spatial resolution improvement was confirmed by the experimentally obtained modulation transfer function for Chip B with various line and space pattens. The examples of capacitance imaging using the fabricated chips were also demonstrated.
Article
Full-text available
Machine and deep learning techniques are two branches of artificial intelligence that have proven very efficient in solving advanced human problems. The automotive industry is currently using this technology to support drivers with advanced driver assistance systems. These systems can assist various functions for proper driving and estimate drivers’ capability of stable driving behavior and road safety. Many studies have proved that the driver’s emotions are the significant factors that manage the driver’s behavior, leading to severe vehicle collisions. Therefore, continuous monitoring of drivers’ emotions can help predict their behavior to avoid accidents. A novel hybrid network architecture using a deep neural network and support vector machine has been developed to predict between six and seven driver’s emotions in different poses, occlusions, and illumination conditions to achieve this goal. To determine the emotions, a fusion of Gabor and LBP features has been utilized to find the features and been classified using a support vector machine classifier combined with a convolutional neural network. Our proposed model achieved better performance accuracy of 84.41%, 95.05%, 98.57%, and 98.64% for FER 2013, CK+, KDEF, and KMU-FED datasets, respectively.
Article
Full-text available
Monitoring drivers’ emotions is the key aspect of designing advanced driver assistance systems (ADAS) in intelligent vehicles. To ensure safety and track the possibility of vehicles’ road accidents, emotional monitoring will play a key role in justifying the mental status of the driver while driving the vehicle. However, the pose variations, illumination conditions, and occlusions are the factors that affect the detection of driver emotions from proper monitoring. To overcome these challenges, two novel approaches using machine learning methods and deep neural networks are proposed to monitor various drivers’ expressions in different pose variations, illuminations, and occlusions. We obtained the remarkable accuracy of 93.41%, 83.68%, 98.47%, and 98.18% for CK+, FER 2013, KDEF, and KMU-FED datasets, respectively, for the first approach and improved accuracy of 96.15%, 84.58%, 99.18%, and 99.09% for CK+, FER 2013, KDEF, and KMU-FED datasets respectively in the second approach, compared to the existing state-of-the-art methods.
Article
Full-text available
Low‐dimensional black phosphorus (LDBP) materials have emerged, with considerable application potential in sensing due to their unique folded structures and remarkable physicochemical properties. These include anisotropies, layer‐dependent and tunable band gaps, high carrier mobilities, high current switching ratios, and excellent electron donor capacities. As a type of supporting material that is favorable to signal transmission or reception, LDBP is widely researched in piezoelectric, flexible, chemical, and biomolecular sensors. This review summarizes the synthetic methods, properties, and modification strategies of LDBP, and then mainly focuses on the research progress in LDBP‐based sensor applications, including physical and chemical sensors and biosensors. The major issues in LDBP‐based sensor applications are also discussed. Finally, the prospects and challenges in the field of LDBP‐based sensors are analyzed. Low‐dimensional black phosphorus (LDBP) opens up new opportunities for sensors development. An overview of LDBP for detection with focus on the structure, properties, protection strategies, preparation methods is presented. Furthermore, the relationship between the characteristics and sensing performance, along with recent progress on various LDBP‐based sensors are elaborated. Finally, the limitations and challenges are discussed.
Article
Full-text available
Recently, the demand of a high resolution complementary metal-oxide semiconductor (CMOS) image sensor is dramatically increasing. As the pixel size reduces to submicron, however, the quality of the sensor image decreases. In particular, the dark current can act as a large noise source resulting in reduction of the quality of the sensor image. Fluorine ion implantation was commonly used to improve the dark current by reducing the trap state density. However, the implanted fluorine diffused to the outside of the silicon surface and disappeared after annealing process. In this paper, we analyzed the effects of carbon implantation on the fluorine diffusion and the dark current characteristics of the CMOS image sensor. As the carbon was implanted with dose of 5.0 × 1014 and 1 × 1015 ions/cm2 in N+ area of FD region, the retained dose of fluorine was improved by more than 131% and 242%, respectively than no carbon implantation indicating that the higher concentration of the carbon implantation, the higher the retained dose of fluorine after annealing. As the retained fluorine concentration increased, the minority carriers of electrons or holes decreased by more Si-F bond formation, resulting in increasing the sheet resistance. When carbon was implanted with 1.0 × 1015 ions/cm2, the defective pixel, dark current, transient noise, and flicker were much improved by 25%, 9.4%, 1%, and 28%, respectively compared to no carbon implantation. Therefore, the diffusion of fluorine after annealing could be improved by the carbon implantation leading to improvement of the dark current characteristics.
Article
Full-text available
Extending CMOS Image Sensors’ dynamic range is of fundamental importance in applications, such as automotive, scientific, or X-ray, where a broad variation of incoming light should be measured. The typical logarithmic pixels suffer from poor performance under low light conditions due to a leakage current, usually referred to as the dark current. In this paper, we propose a logarithmic pixel design capable of reducing the dark current through low-voltage photodiode biasing, without introducing any process modifications. The proposed pixel combines a high dynamic range with a significant improvement in the dark response compared to a standard logarithmic pixel. The reported experimental results show this architecture to achieve an almost 35 dB improvement at the expense of three additional transistors, thereby achieving an unprecedented dynamic range higher than 160 dB.
Article
Full-text available
According to the Center for Disease Control and Prevention (CDC), the average human life expectancy is 78.8 years. Specifically, 3.2 million deaths are reported yearly due to heart disease, cancer, Alzheimer’s disease, diabetes, and COVID-19. Diagnosing the disease is mandatory in the current way of living to avoid unfortunate deaths and maintain average life expectancy. CMOS image sensor (CIS) became a prominent technology in assisting the monitoring and clinical diagnosis devices to treat diseases in the medical domain. To address the significance of CMOS image ‘sensors’ usage in disease diagnosis systems, this paper focuses on the CIS incorporated disease diagnosis systems related to vital organs of the human body like the heart, lungs, brain, eyes, intestines, bones, skin, blood, and bacteria cells causing diseases. This literature survey’s main objective is to evaluate the ‘systems’ capabilities and highlight the most potent ones with advantages, disadvantages, and accuracy, that are used in disease diagnosis. This systematic review used PRISMA workflow for study selection methodology, and the parameter-based evaluation is performed on disease diagnosis systems related to the human body’s organs. The corresponding CIS models used in systems are mapped organ-wise, and the data collected over the last decade are tabulated.
Article
Full-text available
Detecting and striking the critical part of blurred balloon image is an important approach to counterattack the reconnaissance balloons. The existing algorithms of the image target detection task are not able to achieve the high precision and real-time performance in the meanwhile since the critical part of the balloon is tiny, weak and not easy to be segmented. In this paper, a real-time algorithm based curvature feature in the polar coordinate system is proposed to detect the critical part of reconnaissance balloons. We divide the proposed method into three steps: the image is firstly subjected to a gray-scale projection by calculating third-order post-difference, then the balloon boundary is extracted in transformed polar coordinates, and finally the boundary curvature identifies the position of the critical part. The core strategy of the proposed method is to adopt the boundary features of the balloon instead of the general time-consuming image operations (e.g. region labeling, matching) to capture the target part. The experimental results show that the proposed method obtains high precision results with a real-time detection. Our proposed method achieves a processing speed of 200 frames per second on DSP (TMS320C6678) while a state-of-the-art detection precision (>93\(\%\)), which overcomes the existing comparison algorithms.
Article
Full-text available
A minirhizotron is an in situ root imaging system that captures components of root system architecture dynamics over time. Commercial minirhizotrons are expensive, limited to white-light imaging, and often need human intervention. The implementation of a minirhizotron needs to be low cost, automated, and customizable to be effective and widely adopted. We present a newly designed root imaging system called SoilCam that addresses the above mentioned limitations. The imaging system is multi-modal, i.e., it supports both conventional white-light and multispectral imaging, with fully automated operations for long-term in-situ monitoring using wireless control and access. The system is capable of taking 360° images covering the entire area surrounding the tube. The image sensor can be customized depending on the spectral imaging requirements. The maximum achievable image quality of the system is 8 MP (Mega Pixel)/picture, which is equivalent to a 2500 DPI (dots per inch) image resolution. The length of time in the field can be extended with a rechargeable battery and solar panel connectivity. Offline image-processing software, with several image enhancement algorithms to eliminate motion blur and geometric distortion and to reconstruct the 360° panoramic view, is also presented. The system is tested in the field by imaging canola roots to show the performance advantages over commercial systems.
Article
Full-text available
Climate change and global warming are attributed to the increased levels of greenhouse Gases in the atmosphere. Miniature low-cost, lightweight instruments on-board low-cost nanosatellite platforms such as CubeSats could be used to provide precise measurements of greenhouse gases levels. CubeSats, which usually have a narrow field of view, cost a fraction of what more expensive satellites with wide swaths cost. MeznSat is a 3U CubeSat that will carry a shortwave infrared (SWIR) micro-spectrometer as its primary payload, with the aim of deriving greenhouse gas concentrations in the atmosphere by making observations in the 1000–1650 nm wavelength region. The satellite, which is planned for launch in March 2020, is the result of a collaborative project between Khalifa University of Science and Technology (KUST) and the American University of Ras Al-Khaimah (AURAK) with a fund from the United Arab Emirates Space Agency (UAE-SA). The primary payload, Argus 2000, is a miniature, low-cost, space-qualified spectrometer that operates in the shortwave infrared (SWIR) bands. Argus 2000 is a ruggedized unit with a mass of less than 230 g and power consumption of less than 1 W. Also, the Argus 2000 has 0.15 degrees viewing angle and 15 mm fore-optics. The secondary payload will consist of a high definition (HD) camera that will allow post-processing to achieve the high geolocation accuracy required for the SWIR spectrometer data. The RGB combination of visible and SWIR bands setup makes MeznSat a unique CubeSat mission that will generate an interesting dataset to explore atmospheric correction algorithms, which employ SWIR data to process visible channels. This paper describes the mission feasibility, mission analysis, design, and status of MeznSat.
Article
2020. The American Astronomical Society. All rights reserved Arcsecond Space Telescope Enabling Research In Astrophysics (ASTERIA) is a 6U CubeSat space telescope (10 cm 20 cm 30 cm, 10 kg). ASTERIA's primary mission objective was demonstrating two key technologies for reducing systematic noise in photometric observations: high-precision pointing control and high-stability thermal control. ASTERIA demonstrated 0.″5 rms pointing stability and 10 mK thermal control of its camera payload during its primary mission, a significant improvement in pointing and thermal performance compared to other spacecraft in ASTERIA's size and mass class. ASTERIA launched in 2017 August and deployed from the International Space Station in 2017 November. During the prime mission (2017 November-2018 February) and the first extended mission that followed (2018 March-2018 May), ASTERIA conducted opportunistic science observations, which included the collection of photometric data on 55 Cancri, a nearby exoplanetary system with a super-Earth transiting planet. The 55 Cancri data were reduced using a custom pipeline to correct complementary metal-oxide semiconductor (CMOS) detector column-dependent gain variations. A Markov Chain Monte Carlo approach was used to simultaneously detrend the photometry using a simple baseline model and fit a transit model. ASTERIA made a marginal detection of the known transiting exoplanet 55 Cancri e (∼2 R⊕), measuring a transit depth of 374 170 ppm. This is the first detection of an exoplanet transit by a CubeSat. The successful detection of super-Earth 55 Cancri e demonstrates that small, inexpensive spacecraft can deliver high-precision photometric measurements.
Article
Objective: Neonates and infants are patients who would benefit from less invasive vital sign sensing, especially from fewer cables and the avoidance of adhesive electrodes. Photoplethysmography imaging (PPGI) has been studied for medical applications in recent years: it is possible to assess various vital signs remotely, non-invasively, and without contact by using video cameras and light. However, studies on infants and especially on neonates in clinical settings are still rare. Hence, we conducted a single-center study to assess heart activity by estimating the pulse rate (PR) of 19 neonates. Approach: Time series were generated from tracked regions of interest (ROIs) and PR was estimated via a joint time-frequency analysis using a short-time Fourier transform. Artifacts, for example, induced by movement, were detected and flagged by applying a signal quality index in the frequency domain. Main results: The feasibility of PR estimation was demonstrated using visible light and near-infrared light at 850 nm and 940 nm, respectively: the estimated PR was as close as 3 heartbeats per minute in artifact-free time segments. Furthermore, an improvement could be shown when selecting the best performing ROI compared to the ROI containing the whole body. The main challenges are artifacts from motion, light sources, medical devices, and the detection and tracking of suitable regions for signal retrieval. Nonetheless, the PR extracted was found to be comparable to the contact-based photoplethysmography reference and is, therefore, a viable replacement if robust signal retrieval is ensured. Significance: Neonates are seldom measured by PPGI and studies reporting measurements on darker skin tones are rare. In this work, not only a single camera was used, but a synchronized camera setup using multiple wavelengths. Various ROIs were used for signal extraction to examine the capabilities of PPGI. In addition, qualitative observations regarding camera parameters and noise sources were reported and discussed.
Article
Nuclear technology is being promoted and applied worldwide but poses a certain threat to public safety, thus necessitating nuclear radiation detection. In addition to the use of dedicated instruments, the use of surveillance cameras with complementary metal oxide semiconductor (CMOS) sensors is a promising direction for nuclear radiation detection. Currently, such use requires masking the camera lens or working under a static environment, conditions under which a camera cannot perform surveillance monitoring. In this article, we propose a method for detecting nuclear radiation based on an uncovered CMOS camera that enables nuclear radiation detection while the camera is in surveillance monitoring mode. First, we captured videos without covering the camera lens of irradiation from a 99mTc radioactive source. Then, an inter-frame difference algorithm and Gaussian smoothing method were used to reduce the interference of moving objects and visible light on the images. Finally, a series of thresholds were selected to test the validity of our detection method on a validation set. The experimental results indicate that the proposed method can effectively detect nuclear radiation. It is further verified that the surveillance camera can perform surveillance monitoring and radiation detection simultaneously.