Marco Gruteser

Rutgers, The State University of New Jersey | Rutgers · Department of Electrical and Computer Engineering

In Smart City and Vehicle-to-Everything (V2X) systems, acquiring pedestrians' accurate locations is crucial to traffic safety. Current systems adopt cameras and wireless sensors to detect and estimate people's locations via sensor fusion. Standard fusion algorithms, however, become inapplicable when multi-modal data is not associated. For example,...

We introduce ViFiCon, a self-supervised contrastive learning scheme which uses synchronized information across vision and wireless modalities to perform cross-modal association. Specifically, the system uses pedestrian data collected from RGB-D camera footage as well as WiFi Fine Time Measurements (FTM) from a user's smartphone device. We represent...

Visual identification of objects using cameras requires precise detection, localization, and recognition of the objects in the field-of-view. The visual identification problem is very challenging when the objects look identical and features between distinct objects are indistinguishable, even with state-of-the-art computer vision techniques. The pr...

Connected vehicles can benefit from sharing and merging their observations to develop a more complete understanding of the traffic scene and track traffic participants behind obstructions. Although vehicle-to-vehicle(V2V) communications provide a channel for point cloud data sharing, it is challenging to align point clouds from two vehicles with st...

As mobile devices continuously generate streams of images and videos, a new class of mobile deep vision applications are rapidly emerging, which usually involve running deep neural networks on these multimedia data in real-time. To support such applications, having mobile devices offload the computation, especially the neu-ral network inference, to...

To the best of our knowledge, we present the first underground fiber cable position detection methods using distributed fiber optic sensing (DFOS) technology. Meter level localization accuracy is achieved in the results.

The resolution of GPS measurements, especially in urban areas, is insufficient for identifying a vehicle's lane. In this work, we develop a deep LSTM neural network model LaNet that determines the lane vehicles are on by periodically classifying accelerometer samples collected by vehicles as they drive in real time. Our key finding is that even adj...

Head-mounted devices (HMD) for Augmented Reality (AR) are gaining traction thanks to a growing number of applications in the areas of image guided therapy, computer aided design, cargo packing, manufacturing and digital field service. However, providing an always available, intuitive and user friendly input for these devices remains a challenging p...

With the increasing prevalence of mobile and IoT devices (e.g., smartphones, tablets, smart-home appliances), massive private and sensitive information are stored on these devices. To prevent unauthorized access on these devices, existing user verification solutions either rely on the complexity of user-defined secrets (e.g., password) or resort to...

Most existing Augmented Reality (AR) and Mixed Reality (MR) systems are able to understand the 3D geometry of the surroundings but lack the ability to detect and classify complex objects in the real world. Such capabilities can be enabled with deep Convolutional Neural Networks (CNN), but it remains difficult to execute large networks on mobile dev...

Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, we explo...

This chapter focuses on received signal strength (RSS)‐based indoor positioning. It provides an overview of various indoor localization techniques employing RSS, including the lateration methods, machine learning classification, probabilistic approaches, and statistical supervised learning techniques. The chapter compares their performance in terms...

This paper introduces techniques to automatically detect driving corner cases from dashcam video and inertial sensors. Developing robust driver assistance and automated driving technologies requires an understanding of not just common highway and city traffic situations but also a plethora of corner cases that may be encountered in billions of mile...

Due to the extensive use of smart devices using them to study the driving behaviors has attracted a lot of researchers. This work demonstrates the problem of identifying drivers based on their driving style using smart phones. For this purpose the turns done by the drivers are being studied. Different sensors are embedded in the smart phones which...

Academic and industry research has argued for supporting WiFi time-of-flight measurements to improve WiFi localization. The IEEE 802.11-2016 now includes a Fine Time Measurement (FTM) protocol for WiFi ranging, and several WiFi chipsets offer hardware support albeit without fully functional open software. This paper introduces an open platform for...

The growing number of devices we interact with require a convenient yet secure solution for user identification, authorization and authentication. Current approaches are cumbersome, susceptible to eavesdropping and relay attacks, or energy inefficient. In this paper, we propose a body-guided communication mechanism to secure every touch when users...

Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, not just...

This paper introduces an end-to-end untethered VR system design and open platform that can meet virtual reality latency and quality requirements at 4K resolution over a wireless link. High-quality VR systems generate graphics data at a data rate much higher than those supported by existing wireless-communication products such as Wi-Fi and 60GHz wir...

This paper explores the feasibility of localizing and detecting activities of building occupants using visible light sensing across a mesh of light bulbs. Existing Visible Light activity sensing (VLS) techniques require either light sensors to be deployed on the floor or a person to carry a device. Our approach integrates photosensors with light bu...

In-vehicle context sensing can detect many aspects of driver behavior and the environment, such as drivers changing lanes, stop signs, road grade and potholes, and these features can be used to improve driver safety and comfort, and engine efficiency. In general, detecting these features can use either on-board sensors on the vehicle (car sensors),...

As smartphone rooted distractions become commonplace, the lack of compelling safety measures has led to a rise in the number of injuries to distracted walkers. Various solutions address this problem by sensing a pedestrian's walking environment. Existing camera-based approaches have been largely limited to obstacle detection and other forms of obje...

To overcome the drawbacks and security issues of traditional wireless and RF-based communication techniques, multiple alternative "wireless" communication methods have been investigated. Among them, capacitive touch communication which utilizes the touchscreen process as communication channel has been proposed with the promising future since capaci...

Safety messaging for connected vehicles imposes both challenging workloads and stringent robustness requirements on wireless networks. In particular, the design and validation of communication protocols that can operate under high node densities has been a key research focus. The higher robustness requirements for safety communication call for more...

Connected vehicle applications promise to transform traffic management and safety. Evaluating the performance of vehicular communication systems in real-world settings benefits from detailed propagation models. Developing such models can be challenging due to the dynamic nature of the vehicular environment. This paper presents an experimental metho...

The safety of vulnerable road users (VRU) (e.g., pedestrians, bicyclists) can be improved by sharing their position and context information with vehicles over a wireless communication channel. However, challenges exist in managing transmission in densely populated areas with large numbers of VRUs, since these transmissions may overload the wireless...

Traffic estimation has been a long-studied problem, but prior work has mostly provided coarse estimates over large areas. This work proposes effective fine-grained traffic volume estimation using in-vehicle dashboard mounted cameras. Existing work on traffic estimation relies on static traffic cameras that are usually deployed at crowded intersecti...

This paper explores the minimal dataset necessary at vehicular edge nodes, to effectively differentiate drivers using data from existing in-vehicle sensors. This facilitates novel personalization, insurance, advertising, and security applications but can also help in understanding the privacy sensitivity of such data. Existing work on differentiati...

Advanced driver assistance systems and, in particular automated driving offers an unprecedented opportunity to transform the safety, efficiency, and comfort of road travel. Developing such safety technologies requires an understanding of not just common highway and city traffic situations but also a plethora of widely different unusual events (e.g....

Steerable surveillance cameras offer a unique opportunity to support multiple vision applications simultaneously. However, state-of-art camera systems do not support this as they are often limited to one application per camera. We believe that we should break the one-to-one binding between the steerable camera and the application. By doing this we...

Like today's autonomous vehicle prototypes, vehicles in the future will have rich sensors to map and identify objects in the environment. For example, many autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These cameras are used for improving vehicular safety in autonomous driving, but have fundam...

Applications on wearable personal imaging devices, or Smart-glasses as they are called, can largely benefit from accurate and energy-efficient recognition of objects that are within the user's view. Existing solutions such as optical or computer vision approaches are too energy intensive, while low-power active radio tags suffer from imprecise orie...

This paper proposes the design and implementation of low-- energy tokens for smart interaction with capacitive touch-- enabled devices by associating the token's identity with its contact, or touch. The proposed token's design features two key novel technical components: (1) a through--touch--sensor low--energy communication method for token identi...

Communication between pedestrians' mobile devices and vehicles can play a vital role in improving traffic safety. Enabling such communication is challenging in areas where pedestrian density is high, since transmissions from all pedestrians could lead to high channel load, co-channel interference, and degraded communication performance. To understa...

This paper explores the feasibility of tracking motion and activities of humans using visible light. Shadows created by casting visible light on humans and objects are sensed using sensors that are embedded along with the light sources. Existing Visible Light Sensing (VLS) techniques require either light sensors deployed on the floor or a person ca...

This paper reports on our methodology and experience from a multi-year effort to cross-validate a vehicular network experiment with four hundred Dedicated Short Range Communications IEEE 802.11p transmitters through ns-3 simulations. With most of these transmitters in communication range, this represents an extremely dense wireless configuration th...

Driver distraction due to in-vehicle device use is an increasing concern and has led to national attention. We ask whether it is not more effective to channel the drivers' device and information system use into safer periods, rather than attempt a complete prohibition of mobile device use. This paper aims to start the discussion by examining the fe...

VibKeyboard could accurately determine the location of a keystroke on extended surface areas leveraging a single vibration sensor. Unlike capacitive sensing, it does not require conductive materials and compared to audio sensing it is more robust to acoustic noise. In VibKeyboard, the received vibration signals are determined by the location of the...

This work explores vibration-based sensing to determine the location of a touch on extended surface areas as well as identify the object touching the surface leveraging a single sensor. It supports a broad array of applications through either passive or active sensing using only a single sensor. In the passive sensing, the received vibration signal...

Platypus is the first system to localize and identify people by remotely and passively sensing changes in their body electric potential which occur naturally during walking. While it uses three or more electric potential sensors with a maximum range of 2 m, as a tag-free system it does not require the user to carry any special hardware. We describe...

Channel congestion is one of the major challenges for IEEE 802.11p-based vehicular networks. Unless controlled, congestion increases with vehicle density, leading to high packet loss and degraded safety application performance. We study two classes of congestion control algorithms, i.e., reactive state-based and linear adaptive. In this paper, the...

Given the increasing popularity of wearable devices, this paper explores the potential to use wearables for steering and driver tracking. Such capability would enable novel classes of mobile safety applications without relying on information or sensors in the vehicle. In particular, we study how wrist-mounted inertial sensors, such as those in smar...

We exploit human color metamers to send light-modulated messages less visible to the human eye, but recoverable by cameras. These messages are a key component to camera-display messaging, such as handheld smartphones capturing information from electronic signage. Each color pixel in the display image is modified by a particular color gradient vecto...

Precise positioning of an automobile to within lane-level precision can enable better navigation and context-awareness. However, GPS by itself cannot provide such precision in obstructed urban environments. In this paper, we present a system called CARLOC for lane-level positioning of automobiles. CARLOC uses three key ideas in concert to improve p...

Precise positioning of an automobile to within lane-level precision can enable better navigation and context-awareness. However, GPS by itself cannot provide such precision in obstructed urban environments. In this paper, we present a system called CARLOC for lane-level positioning of automobiles. CARLOC uses three key ideas in concert to improve p...

We present a technique for creating multi-key conductive ink touch user interfaces that can be printed on paper in a single pass. While 3D printing and open-source electronics platforms have led to enormous creativity in creating smart objects, the means for user interaction with such objects are often limited and require remote interaction through...

This paper explores the limits of audio ranging on mobile devices in the context of a keystroke snooping scenario. Acoustic keystroke snooping is challenging because it requires distinguishing and labeling sounds generated by tens of keys in very close proximity. Existing work on acoustic keystroke recognition relies on training with labeled data,...

Given the penetration of mobile devices equipped with cameras, there has been increasing interest in enabling user interaction via visual codes. Simple examples like QR Codes abound. Since many codes like QR Codes are visually intrusive, various mechanisms have been explored to design visual codes that can be hidden inside regular images or videos,...

Channel congestion is one of the major challenges for IEEE 802.11p-based vehicular ad hoc networks. Facing the challenge, several algorithms have been proposed. Two good representatives are the decentralized Congestion Control (DCC) framework defined by ETSI and LIMERIC, a linear control algorithm. Both algorithms control message transmission rate...

Creating devices or systems that can leverage computationally rich environments to support day-to-day interactions between humans and computers is the goal of mobile computing and communication. The increasing use of mobile devices and their data-intensive apps has generated extensive opportunities to monitor and optimize real-world processes throu...

This paper explores the potential for wearable devices to identify driving activities and unsafe driving, without relying on information or sensors in the vehicle. In particular , we study how wrist-mounted inertial sensors such as those in smart watches and fitness trackers, can track steering wheel usage and inputs. Identifying steering wheel usa...

This video is a demonstration of the work discussed in our full paper available in the MobiSys'15 proceedings. The video illustrates a sensing technology for fine-grained location classification in an urban environment, for enhancing pedestrian safety. Our system seeks to detect the transitions from sidewalk locations to in-street locations, to ena...

Motivated by safety challenges resulting from distracted pedestrians, this paper presents a sensing technology for fine-grained location classification in an urban environment. It seeks to detect the transitions from sidewalk locations to in-street locations, to enable applications such as alerting texting pedestrians when they step into the street...

There has been a long line of wearable research in the mobile computing community. However, a new, easily accessible platform often functions as a research enabler and leads to a burst of research activity in the field. Will Google Glass and similar offerings from other vendors have this catalyst effect on mobile and wearable research?

We present a novel method for communicating between a camera and display by embedding and recovering hidden and dynamic information within a displayed image. A handheld camera pointed at the display can receive not only the display image, but also the underlying message. These active scenes are fundamentally different from traditional passive scene...

This paper utilizes smartphone sensing of vehicle dynamics to determine driver phone use, which can facilitate many traffic safety applications. Our system uses embedded sensors in smartphones, i.e., accelerometers and gyroscopes, to capture differences in centripetal acceleration due to vehicle dynamics. These differences combined with angular spe...

Cameras are ubiquitous and increasingly being used not just for capturing images but also for communicating information. For example, the pervasive QR codes can be viewed as communicating a short code to camera-equipped sensors. Such communications could be particularly attractive in pervasive camera based applications, where such camera communicat...