Fig 1 - uploaded by Jarek Krajewski
Content may be subject to copyright.
The average green channel on a face region during rigid head motions. In the first 250 frames the user is in a resting state and the fine pulsation of blood volume is visible. After 300 frames the user started to move his head and the pulse signal gets lost. (Color figure online)
Source publication
![Fig. 4. Correlation and Bland-Altman [2] plots of PPGI diffusion...](profile/Jarek-Krajewski/publication/319106408/figure/fig4/AS:590378801573888@1517768683438/Correlation-and-Bland-Altman-2-plots-of-PPGI-diffusion-process-estimated-heart-rate_Q320.jpg)
This work addresses the problem of estimating heart rate from face videos under real conditions using a model based on the recursive inference problem that leverages the local invariance of the heart rate. The proposed solution is based on the canonical state space representation of an Itō process and a Wiener velocity model. Empirical results yiel...
Contexts in source publication
Context 1
... most significant factors influencing the operational performance are the sensors spectral response, user movements like rigid head motions and facial expressions as well as fast varying illumination conditions. Figure 1 illustrates the disturbing influence of head motions on the raw PPGI blood volume pulse signal. During head motion the pixel intensity distribution varies significantly, much more than the expectation which can be attributed to blood pulsation. ...
Context 2
... natural conditions, illumination is not static as well as not uniform dis- tributed over facial skin regions. The intensity distribution of skin pixels varies significantly when an user starts to move the head as illustrated in Fig. 1. Assum- ing that the image source S produces random variables X with an associated probability density function p(x), a suitable feature representation for the skin blood perfusion, invariant with respect to the location and large magnitude dif- ferences of pixel intensities can be expressed as the average quantization error d of an ...
Context 3
... ICA performs worst and is not able to provide reliable heart rate information during head motion. Although SSR performs better it cannot compete against the robustness of the diffusion process. [35], the SSR [44], the diffusion process model and the reference finger pulseoximeter. These estimates are based upon the video illustrated in Fig. 1. ...
Citations
... However, many factors influence the accuracy of rPPGbased HR measurement. These include the subject's skin tone [10], head movement [10], rapid changes during recovery from exercise [11], variation in ambient illumination [12], and, when imaging the head, facial expression changes associated with talking, for example, during human-machine interaction [13]. To address these issues, many approaches have been proposed, the principal ones of which include signal decomposition, model-based methods, and data-driven methods. ...
... The local invariance of the HR model, developed by Pilz et al., converts rPPG signals to certain specific spaces and uses the local group invariance of HR to estimate its value. They have successively proposed the diffusion process method [13], the local group invariance method [20], and the spherical method [22] and investigated the effect of head movement and person-to-person interaction. ...
Remote photoplethysmography (rPPG) can be used for noncontact and continuous measurement of the heart rate (HR). Currently, the main factors affecting the accuracy and robustness of rPPG-based HR measurement methods are the subject’s skin tone, body movement, exercise recovery, and variable or inadequate illumination. In response to these challenges, this study is aimed at investigating a rPPG-based HR measurement method that is effective under a wide range of conditions by only using a webcam. We propose a new approach, which combines joint blind source separation (JBSS) and a projection process based on a skin reflection model, so as to eliminate the interference of background illumination and enhance the extraction of pulse rate information. Three datasets derived from subjects with different skin tones considering six environmental scenarios are used to validate the proposed method against three other state-of-the-art methods. The results show that the proposed method can provide more accurate and robust HR measurement for all three datasets and is therefore more applicable to a wide range of scenarios.
... The size of the observed skin/body region and the spatial resolution can be arbitrarily chosen, depending on the utilized camera lens and distance between camera and measuring object. To minimize motion artifacts, which can be created by the separation of the camera sensor from the skin, different algorithm have been developed in the last decade [77,78]. Different research groups has already shown that robust and unobtrusive estimation of the HR and the HRV is possible by the PPGI, also by using low cost cameras [79][80][81]. ...
The dynamics of a rhythm band observed first in the ear skin microcirculation of awake human subjects were scrutinized using a naturalistic study design and nonlinear frequency and phase analysis. Since this frequency band with its centre between 0.12 and 0.18 Hz was slower than the respiratory rhythm but faster than the 0.1 Hz sympathetic dominated rhythm, this rhythm band is referred to as intermediary rhythm. Using nonlinear frequency and phase detection methods pronounced differences between episodes exhibiting amplitude modulated intermediary rhythm activity and episodes void of it were identified. Varying coherence, the modulation of peak to peak distances and fluctuations of energy transfer (modulation of the dV/dt in the protosystolic and the diastolic phase of cardiac driven pulsations) and, most importantly, the objective documentation of phase jumps relate the intermediary rhythm to the principles of ‘synergetic self-organization’ as discussed by Haken. The emergence of the intermediary rhythm is suggested to originate in non-equilibrium phase transitions in the network of lower brainstem neurons and is further linked to parasympathetic neuronal effectors, e.g. parasympathetic innervations of facial skin microcirculation. Thus, the intermediary rhythm comprises oscillations of a dynamic equipoise allowing either deceleration to slower sympathetic rhythms or acceleration to faster vagal and respiratory related rhythms.
... Wang et al. [41] reported an orthogonal behavior of skin color and motion artifacts but introduced a static operator for feature transformation representing results on private data. A new stochastic model formulation was introduced by Pilz et al. [27] outperforming Wang et al. [42], however the proposed features based upon vector quantization seem to be a rather time consuming heuristic approach. All these important contributions share the problem that there exists no consensus about an unique benchmarking criterion as well as an agreement on suitable open data sets for fair comparison of algorithms. ...
... In the following we describe features invariant with respect to the action of the Lie group [12] of rigid transformations, the Special Euclidean group SE(3) and a stochastic frequency representation invariant with respect to the quasi-periodic nature and nonstationarity of heart rate. The model space is based upon the previous works of Särrkä [31] and Pilz et al. [27]. ...
... The solution is given by computing the Gaussian mixture approximation to the joint posterior distribution of the latent variables and states [6]. Pilz et al. [27] stated that slow varying drifts can be modeled by a Wiener process, whereby any kind of violation of the smoothness criterion yields to a Poisson process. We didn't consider the drift and jump case since the feature representation based upon local group invariance already accounts for such behavior. ...
We study the impact of prior knowledge about invariance for the task of heart rate estimation from face videos in the wild (e.g. in presence of disturbing factors like rigid head motion, talking, facial expressions and natural illumination conditions under different scenarios). We introduce features invariant with respect to the action of a differentiable local group of local transformations. As result, the energy of the blood volume signal is rearranged in vector space with a more concentrated distribution. The uncertainty in the feature distribution is incorporated with a model that leverages the local invariance of the heart rate. During experiments the method achieved strong estimation performance of heart rate from face videos in the wild. To demonstrate the potential of the approach it is compared against recent algorithms on data collected to study the impact of the mentioned nuisance attributes. To facilitate future comparisons, we made the code and data for reproducing the results publicly available.
Souce code: https://github.com/partofthestars/PPGI-Toolbox
For adequate skin perfusion rhythmicity assessment, sensor technology must be used, which fulfills basic demands such as unobtrusiveness and continuous monitoring with spatial resolution photoplethysmography (PPG) and its modern camera-based imaging variant (PPGI) ideally meet these requirements, and they are generally accepted in the field of noninvasive medical diagnostics. PPG can work in reflective or transmissive mode and detects blood volume changes in the vascular plexus within the region of transilluminated tissue. The PPG and PPGI signals comprise a complex of pulsatile wave formations (AC) associated with cardiac, respiratory, and different nervous system activities, which are superimposed to a non-pulsatile baseline (DC) due to optical damping of bloodless tissue. Although most of the PPG applications in medical diagnostics are today devoted to recording cardiac rhythmicity, the strength and future of PPG lie in detecting the distributed and, in some cases, highly autonomous skin perfusion dynamics below the frequencies of the “central oscillators”, namely the heartbeat and breathing. This chapter presents selected activities and results of the bilateral and interdisciplinary long-term cooperation between IIT Madras in Chennai and RWTH Aachen University in Aachen in this exciting research field of the dermal perfusion dynamics.
We investigate the regulation of human brain arousal in the central nervous system and its synchronization with the autonomic nervous system affecting the facial dynamics and its behavioral gestalt. A major focus is made on the sensing observable during natural human eye to eye communication. Although the inner state of the autopoietic system is deterministic, its outer facial behavioral component non-deterministic. Beside the introduction of general validity of the classical empirical interpretation of the vigilance continuum during open eyes, we show that the facial behavior can be used as suitable surrogate measurement for specific states of mind. As a consequence we predict brainwaves from face videos formulated as inverse problem of the underlying stochastic process. Finally, we discuss the impact and range of application field.
We study the vector space of visible wavelength intensities from face videos widely used as input features in Pho-toplethysmography Imaging (PPGI). Based upon theoretical principles of group invariance in the Euclidean space, we derive a change of the topology where the corresponding distance between successive measurements is defined as geodesic on a Riemannian manifold. This lower dimensional embedding of the sensor signal unifies the invariance properties with respect to translation of the features as discussed by several former approaches. The resulting operator acts implicitly on the feature space without requiring any kind of prior knowledge and parameter tuning. The resulting features time varying quasi-periodic shaping naturally occurs in form of the canonical state space representation according to the known diffusion process of blood volume changes. This reduces drastically computational complexity and consequently simplifies the implementation. Experiments from face videos on two public databases have shown a competitive estimation performances of heart rate and robustness in comparison with already available methods . Souce code: https://github.com/partofthestars/PPGI-Toolbox
Over the last few years, the contactless acquisition of cardiovascular parameters using cameras has gained immense attention. The technique provides an optical means to acquire cardiovascular information in a very convenient way. This review provides an overview on the technique’s background and current realizations. Besides giving detailed information on the most widespread application of the technique, namely the contactless acquisition of heart rate, we outline further concepts and we critically discuss the current state.
