No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Demo : Intelligent Vehicular Perception of Non-Line-of-Sight Environment using Visible Light Communication with Stereo Cameras
Abstract
In this work, we explore the use of multi-camera
setups through stereo-vision enabled cameras to perceive immediate and non-line of sight environments of a vehicle. Our
proposed system uses visible light communication between a
transmitter and receiver vehicle to convey information about the
scene in front of the transmitter vehicle. Such a system virtually enables a “see-through-the-vehicle-in-front”, functionality by
combining scene perception with vehicle-vehicle communication.
We posit to demonstrate this functionality using a device-device
communication setup using a stereo camera that does perception
mapping and visible light communication reception from a LED
transmitter.
... In Ref. 11, an experimental demonstration of the vehicle-to-vehicle (V2V)-VLC system has been performed using a stereo camera and an LED light source. The authors suggested a novel approach to enhance the perception of a vehicle surroundings in NLoS scenarios. ...
Visible light communication is seen as a crucial technology within optical wireless communication systems. The technology of vehicular visible light communication holds significant importance in the context of connected vehicles. This technology can serve as a supplementary solution to vehicular systems that are based on radio frequency. In this paper, the authors conduct an analysis of the performance of both line-of-sight and non-line-of-sight vehicle-to-vehicle visible light communication systems under the effect of artificial light source and weather conditions, including clear, hazy, and foggy weather. A practical vehicular laser diode, a street lamp, and an avalanche photodiode are used to design the proposed system model. Performance enhancement for the proposed system is achieved using an optical amplifier at the receiving end. An artificial light source of light-emitting diode Corn-type is used to represent an ambient artificial light source. Different metrics such as quality factor and bit error rate are used to assess the system performance of the non-line-of-sight-vehicular communication system. The proposed line-of-sight model achieves a data rate of 25 Gbps, supporting a distance of 80 m under clear sky and hazy atmospheric conditions. For foggy weather, an attainable link distance of 70 m is achieved. The achieved results emphasize the suitability of the suggested models for vehicular applications in real world environment.
... An LOS link may not be established between vehicles owing to high mobility in cases wherein an NLOS link uses the portion of the light reflected from a particular surface. Furthermore, the optical signal can travel to the receiver using both LOS and NLOS paths [97]. Thus, the channel gain can be substantially increased. ...
Recently, substantial development is observed in the area of Internet of Vehicles owing to the application of wireless communication technologies. Majority of these technologies are based on radio frequency (RF); however, RF spectra are overly congested and regulated, and hence, insufficient to support massive data traffic in the future. In recent times, optical camera communication (OCC) that uses a light-emitting diode (LED) as a transmitter and a camera as a receiver has been deemed an excellent solution for future intelligent transportation systems. As a communication medium, OCC mostly uses visible light, the spectrum of which is vast, completely free, and unregulated. The current outdoor environment is heavily crammed with LED infrastructures, and most vehicles have built-in cameras, rendering OCC immensely promising. OCC is highly secured, supports mobility, and can achieve an excellent bit-error rate. However, the data rate obtained using OCC is not as high as that obtained using other RF-based systems; therefore, its reliability in fast-changing channels is still under research. This review article discusses the applications of the OCC system in vehicle-to-vehicle and vehicle-to-infrastructure (or vice versa) networks; to the best of our knowledge, this is the first extensive review dedicated to the above topic. Herein, we provide a general overview of OCC standardization in IEEE and ISO in recent years. Then, we explain the general principles of OCC, including channel characteristics, region of interest signaling, and modulation schemes particularly considered in vehicular communications. Additionally, we present a comprehensive overview of the effects of mobility, noise, and interference in OCC. Finally, the challenges and future opportunities in OCC are outlined.
Research on intelligent vehicles has been popular in the past decade. To fill the gap between automatic approaches and man-machine control systems, it is indispensable to integrate visual human-computer interactions (VHCIs) into intelligent vehicles systems. In this article, we review existing studies on VHCI in intelligent vehicles from three aspects: 1) visual intelligence; 2) decision making; and 3) macro deployment. We discuss how VHCI evolves in intelligent vehicles and how it enhances the capability of intelligent vehicles. We present several simulated scenarios and cases for future intelligent transportation system.
Visible light communication (VLC) is seen as an interesting technology to complement the IEEE 802.11p-based systems commonly used for vehicle-to-vehicle (V2V) communication. However, the reliability of such a V2V-VLC link in real-world driving scenario had not been demonstrated up to very recently. The results obtained, at that time with orthogonal frequency division multiplexing (OFDM) at 2 kbps, are complemented in this paper with additional results using on-off keying (OOK) at 100 kbps. The overall performances of this second modulation are rather close to those of OFDM. The packet reception rate (PRR) remains more than 90% over a service area of 30 m length-wise and the link is not affected by multipath propagation. Error-free transmission is even demonstrated over 60 m using repetition coding, at the cost of a reduced data rate of 10 kbps. However, it is shown that OFDM is able to cope with the narrowband interferences generated by outdoor lighting such as LED signs whereas the OOK link is completely jammed. Despite its simplicity and good overall performances, OOK is thus not as robust as OFDM to the variety of situations experienced in real-world driving scenario.
As a new technology for next-generation vehicle-to-everything (V2X) communication, visible-light communication (VLC) using light-emitting diode (LED) transmitters and camera receivers has been energetically studied. Toward the future in which vehicles are connected anytime and anywhere by optical signals, the cutting-edge camera receiver employing a special CMOS image sensor, i.e., the optical communication image sensor (OCI), has been prototyped, and an optical V2V communication system applying this OCI-based camera receiver has already demonstrated 10-Mb/s optical signal transmission between real vehicles during outside driving. In this paper, to reach a transmission performance of 54 Mb/s, which is standardized as the maximum data rate in IEEE 802.11p for V2X communication, a more advanced OCI-based automotive VLC system is described. By introducing optical orthogonal frequency-division multiplexing (opticalOFDM), the new system achieves a more than fivefold higher data rate. Additionally, the frequency response characteristics and circuit noise of the OCI are closely analyzed and taken into account in the signal design. Furthermore, the forward-current limitation of an actual LED is also considered for long operational reliability, i.e., the LED is not operated in overdrive. Bit-error-rate experiments verify a system performance of 45 Mb/s without bit errors and 55 Mb/s with BER <; 10
-5
.
This paper presents an analytical performance analysis of a car-to-car visible light communications system under different communication geometries during the daytime. A market-weighted headlamp beam-pattern model, measured dirt effects on light distribution, and the road-surface reflection model are employed. We consider both the line-of-sight and non-line-of-sight links and outline the relationship between the communication range and the system bit error rate (BER) performance. Results show that the wet road surface can help to increase the received optical power and improve the BER performance above a certain distance, and the communications coverage range can reach up to 70 m at a data rate of 50 Mbps when a photodetector is mounted on the car at a height of 0–0.2 m above the road surface.
An optical wireless communication (OWC) system based on a light-emitting-diode (LED) transmitter and a camera receiver has been developed for use in the automotive area. The automotive OWC system will require Mb/s-class data rates and the ability to quickly detect LEDs from an image. The key to achieving this is improvements to the capabilities of the image sensor mounted on the camera receiver. In this paper, we report on a novel OWC system equipped with an optical communication image sensor (OCI), which is newly developed using CMOS technology. To obtain higher transmission rates, the OCI employs a specialized “communication pixel (CPx)” capable of responding promptly to optical intensity variations. Furthermore, a new quick LED detection technique, based on a 1-bit flag image which only reacts to high-intensity objects, is formulated. The communication pixels, ordinary image pixels, and associated circuits (including 1-bit flag image output circuits) are then integrated into the OCI. This paper describes the design, fabrication, and capabilities of the OCI, as well as the development of the LED and image sensor based OWC system, which boasts a 20-Mb/s/pixel data rate without LED detection and a 15-Mb/s/pixel data rate with a 16.6-ms real-time LED detection.
The inherent limitations in RF spectrum availability and susceptibility to interference make it difficult to meet the reliability required for automotive safety applications. To address this challenge, this work explores an alternative communication system called Visual MIMO that uses light emitting arrays as transmitters and cameras as receivers. Visual MIMO applied to vehicular communication proposes to reuse existing LED rear and headlights as transmitters and existing cameras (e.g. those used for parking assistance, rear-view cameras) as receivers. In this work we show a proof of concept based demonstration of the Visual MIMO system consisting of an LED transmitter array and a high-speed camera.
Visible Light Communication (VLC) is a fast-growing technology to provide data communication using low-cost and omni-present LEDs and photodiodes. In this paper, we examine the key proper-ties in enabling vehicular VLC (V2LC) networks as follows. We first develop a custom V2LC research platform on which we expe-rimentally evaluate the feasibility of a V2LC system under working conditions in relation to link resilience to visible light noise and interference. Our experiments show that a receiver's narrow field-of-view angle makes V2LC resilient to visible light noise from sunlight and legacy lighting sources as well as to interference from active VLC transmitters. Then, by leveraging our experimental characterization as the basis of modifications to our simulator, we examine V2LC's performance in providing network services for vehicular applications. Our key findings include: (i) in dense vehicular traffic conditions (e.g., urban highway during peak hours), V2LC takes advantage of multiple available paths to reach vehicles and overcomes the effects of packet collisions; (ii) in the presence of a visible light blockage in traffic, V2LC can still have a significant number of successful transmissions by opportunistically using dynamic inter-vehicle gaps.
Mobile optical communications has so far largely been limited to short ranges of about ten meters, since the highly directional nature of optical transmissions would require costly mechanical steering mechanisms. Advances in CCD and CMOS imaging technology along with the advent of visible and infrared (IR) light sources such as (light emitting diode) LED arrays presents an exciting and challenging concept which we call as visual-MIMO (multiple-input multiple-output) where optical transmissions by multiple transmitter elements are received by an array of photodiode elements (e.g. pixels in a camera). Visual-MIMO opens a new vista of research challenges in PHY, MAC and Network layer research and this paper brings together the networking, communications and computer vision fields to discuss the feasibility of this as well as the underlying opportunities and challenges. Example applications range from household/factory robotic to tactical to vehicular networks as well pervasive computing, where RF communications can be interference-limited and prone to eavesdropping and security lapses while the less observable nature of highly directional optical transmissions can be beneficial. The impact of the characteristics of such technologies on the medium access and network layers has so far received little consideration. Example characteristics are a strong reliance on computer vision algorithms for tracking, a form of interference cancellation that allows successfully receiving packets from multiple transmitters simultaneously, and the absence of fast fading but a high susceptibility to outages due to line-of-sight interruptions. These characteristics lead to significant challenges and opportunities for mobile networking research
In this paper, we present a novel technique for calibrating central omnidirectional cameras. The proposed procedure is very fast and completely automatic, as the user is only asked to collect a few images of a checker board, and click on its corner points. In contrast with previous approaches, this technique does not use any specific model of the omnidirectional sensor. It only assumes that the imaging function can be described by a Taylor series expansion whose coefficients are estimated by solving a four-step least-squares linear minimization problem, followed by a non-linear refinement based on the maximum likelihood criterion. To validate the proposed technique, and evaluate its performance, we apply the calibration on both simulated and real data. Moreover, we show the calibration accuracy by projecting the color information of a calibrated camera on real 3D points extracted by a 3D sick laser range finder. Finally, we provide a Toolbox which implements the proposed calibration procedure.
We present some updates to YOLO! We made a bunch of little design changes to make it better. We also trained this new network that's pretty swell. It's a little bigger than last time but more accurate. It's still fast though, don't worry. At 320x320 YOLOv3 runs in 22 ms at 28.2 mAP, as accurate as SSD but three times faster. When we look at the old .5 IOU mAP detection metric YOLOv3 is quite good. It achieves 57.9 mAP@50 in 51 ms on a Titan X, compared to 57.5 mAP@50 in 198 ms by RetinaNet, similar performance but 3.8x faster. As always, all the code is online at https://pjreddie.com/yolo/
Channel modeling is critical for the design and performance evaluation of visible light communication (VLC). Although a considerable amount of research has focused on indoor VLC systems using singleelement photodiodes, there remains a need for channel modeling of VLC systems for outdoor mobile environments. In this paper, we describe and provide results for modeling image sensor based VLC for automotive applications. In particular, we examine the channel model for mobile movements in the image plane as well as channel decay according to the distance between the transmitter and the receiver. Optical flow measurements were conducted for three VLC situations for automotive use: infrastructure to vehicle VLC (I2V-VLC); vehicle to infrastructure VLC (V2I- VLC); and vehicle to vehicle VLC (V2V-VLC). We describe vehicle motion by optical flow with subpixel accuracy using phase-only correlation (POC) analysis and show that a single-pinhole camera model successfully describes these three VLC cases. In addition, the luminance of the central pixel from the projected LED area versus the distance between the LED and the camera was measured. Our key findings are twofold. First, a single-pinhole camera model can be applied to vehicle motion modeling of a I2V-VLC, V2I-VLC, and V2V-VLC. Second, the DC gain at a pixel remains constant as long as the projected image of the transmitter LED occupies several pixels. In other words, if we choose a pixel with highest luminance among the projected image of transmitter LED, the value remains constant, and the signal-to-noise ratio does not change according to the distance.
Over the past few years, Light Emitting Diode (LED) has become very common in automotive lighting due to its long service life, high resistance to vibration, and better safety performance due to its short rise time. A number of existing works use LEDs that already exist in vehicles, such as brake lights, turn signals, and headlamps, to carry out vehicle-to-vehicle (V2V) communications with visible light communications (VLC). Nonetheless, very few studies derive analytical models for VLC with empirical data obtained from the real world or with realistic assumptions. In addition, experimental works were often conducted in closed environments with lighting systems that are specifically customized, instead of real-life lighting systems. In this paper, we perform an analytical study that attempts to derive VLC channel models in a realistic V2V setting. The proposed model is evaluated against the real-world data obtained with unmodified off-the-shelf scooter taillights, and the results show that the proposed model is able to accurately estimate the received power of the scooter taillight at distances of up to 10 meters. In addition, this paper also discusses several guidelines for modeling VLC radiation behaviors when different types of LED taillights are used.
We propose a multi-layer, real-time vehicle detection and tracking system using stereo vision, multi-view AdaBoost detectors, and optical flow. By adopting a ground plane estimate extracted from stereo information, we generate a sparse set of hypotheses and apply trained AdaBoost classifiers in addition to fast disparity histogramming, for Hypothesis Verification (HV) purposes. Our tracking system employs one Kalman filter per detected vehicle and motion vectors from optical flow, as a means to increase its robustness. An acceptable detection rate with few false positives is obtained at 25 fps with generic hardware.
This paper proposes the driver assistance system based on the omnidirectional stereo vision. This system informs a driver to prepare unexpected situation during driving. This information is a relative distance between a preceding car and driving vehicle and obtained by using the omnidirectional stereo vision. The omnidirectional camera sees 360 degrees unlike a conventional perspective camera. Therefore it is useful to search the road conditions nearby a driving vehicle at every sequence. For detecting a preceding car, we use histogram of oriented gradient (HOG) which has is robustness to for illumination change. Because of stereo vision, the proposed algorithm measures the relative distance between preceding car and moving vehicle whenever preceding car is detected. In experiments, the ratio of detecting preceding car is 99.78% and 94.59% for left and right cameras respectively. The measurement process of relative distance is started when preceding car is detected both left and right cameras and the system detects a center point of tail right. In the measurement process, if a preceding car is not detected from left or right image scene, the system estimates the center point using optical flow consider with previous frame. The maximum error of estimated distance is less than 25cm shown in the Fig 9. Therefore the error of estimated distance is quite ignorable value because the normal relative distance between two moving car is over 2m.
We propose a new tracking method using image sensor (IMG) and photo diode (PD) for visible light road-to-vehicle communication. Proposed method uses an IMG with wide FOV, which receives tracking signal, and a PD with narrow FOV, which receives communication signal. As FOV of the IMG is wide, service range, where communication signal can be received, is larger than conventional methods. Moreover, as FOV of the PD is narrow, PD is less affected by background noise and high speed communication can be possible.
Object detection and tracking are the main research contents of computer vision. One of the methods is based on the estimation of optical flow field, but their calculation result and efficiency are poor. So a multiple object detection and tracking algorithm based on optical flow in polar-log images is proposed in this paper. Optical flow computation is only used in moving area. Firstly, the moving edge is extracted in polar-log coordinate. Secondly, the generalized dynamic image model (GDIM) based method is used and the gradient operator in polar-log coordinate is employed to compute the optical flow directly in every moving area and the object tracking is accomplished. This method has there advantages. One is that the image size reduced and the computing time of optical flow decreased in polar-log coordinate. The other is that the optical flow calculation result is accurate because the GDIM base method is used and the gradient operator in polar-log coordinate is employed. The third is that “excessively smooth” can be resolved by optical flow computation only used in moving area. And this method can be used for multiple objects tracking and real time object tracking. Finally, the experimental results prove that the proposed method in the paper is efficient to multiple objects detection and tracking.
A practical algorithm for automatic chessboard corner detection
- Yu Liu
- Shuping Liu
- Yang Cao
- Zengfu Wang
Yu Liu, Shuping Liu, Yang Cao, and Zengfu Wang. A practical algorithm
for automatic chessboard corner detection. In Image Processing (ICIP),
2014 IEEE International Conference on, pages 3449-3453. IEEE, 2014.
Multiple led arrays acquisition for image-sensor-based i2v-vlc using block matching
- Shintaro Arai
- Yasutaka Shiraki
- Takaya Yamazato
- Hiraku Okada
- Toshiaki Fujii
- Tomohiro Yendo
Shintaro Arai, Yasutaka Shiraki, Takaya Yamazato, Hiraku Okada,
Toshiaki Fujii, and Tomohiro Yendo. Multiple led arrays acquisition
for image-sensor-based i2v-vlc using block matching. In Consumer
Communications and Networking Conference (CCNC), 2014 IEEE 11th,
pages 605-610. IEEE, 2014.
Motion modeling of mobile transmitter for image sensor based i2v-vlc, v2i-vlc, and v2v-vlc
- Masayuki Kinoshita
- Takaya Yamazato
- Hiraku Okada
- Toshiaki Fujii
- Shintaro Arai
- Tomohiro Yendo
- Koji Kamakura
Masayuki Kinoshita, Takaya Yamazato, Hiraku Okada, Toshiaki Fujii,
Shintaro Arai, Tomohiro Yendo, and Koji Kamakura. Motion modeling
of mobile transmitter for image sensor based i2v-vlc, v2i-vlc, and v2v-vlc. In Globecom Workshops (GC Wkshps), 2014, pages 450-455. IEEE,
2014.