Norimichi Tsumura

• Ph.D
• Professor (Associate) at Chiba University

Motivation is a primary driver of goal-directed behavior. Therefore, the development of cost-effective and easily applicable systems to objectively quantify motivational states is needed. To achieve our goal, this study investigated the feasibility of classifying high- and low-motivation states by machine learning based on a diversity of features o...

In this paper, we propose an algorithm for estimating respiratory state using near-infrared facial video images. Estimation of respiratory state is an important indicator for early detection of respiratory diseases. In particular, there is a demand for monitoring respiratory state during the night. One method of monitoring respiratory state is to u...

We investigated the influence of the Proteus effect on physical activity. We found that obese avatar recorded the lower height about the vertical jump than normal one.

We measured the color and shape of foot using the smartphone RGB camera and LiDAR sensors. As a result, we found that these measurements are accurate enough for practical use in nursing fields.

In this study, we explored non-contact pulse wave measurements from the extremities, particularly the soles, and examined their differences. Two experiments were conducted. First, we identified the optimal method for capturing pulse waves and discovered that the peak intensity of the green signal was the most effective. Then, we analyzed the tempor...

In this paper, we propose a noise-robust pulse wave estimation method from near-infrared face video images. Pulse wave estimation in a near-infrared environment is expected to be applied to non-contact monitoring in dark areas. The conventional method cannot consider noise when performing estimation. As a result, the accuracy of pulse wave estimati...

We propose a method of reproducing perceptual translucency in three-dimensional printing. In contrast to most conventional methods, which reproduce the physical properties of translucency, we focus on the perceptual aspects of translucency. Humans are known to rely on simple cues to perceive translucency, and we develop a method of reproducing thes...

In this study, we introduce a measurement method for the bidirectional reflectance distribution function (BRDF) for curved surfaces. Nearly all BRDF measurements assume that the target surface is planar. This is because if the object is non-planar, the reflection angle changes from position to position, making accurate measurement impossible. Howev...

Colored product textures correspond to particle size distributions. The microscopic images of colorants must be divided into regions to determine the particle size distribution. The conventional method used for this process involves manually dividing images into areas, which may be inefficient. In this paper, we have overcome this issue by developi...

In this paper, we propose a pipeline that reproduces human skin mockups using a UV printer by obtaining the spatial concentration map of pigments from an RGB image of human skin. The pigment concentration distributions were obtained by a separating method of skin pigment components with independent component analysis from the skin image. This metho...

Photoplethysmographic (PPG) pulses contain information about cardiovascular parameters. In particular, blood pressure can be estimated using PPG pulse decomposition analysis, which assumes that a PPG pulse is composed of the original heart ejection blood wave and its reflections in arterial branchings. Among pulse decomposition wave functions that...

During the course of treatment in plastic surgery and dermatology, especially aesthetic surgery, it is essential to regularly observe and record any appearance changes of the body. Physicians typically use an interview, photographs of the patient’s face before and after treatment, and anatomical data including size, position, and color. However, it...

Since emotions have an inordinate amount of influence on daily life behaviors such as perception, memory, and decision making, it has recently become necessary to consider user emotions in the field of human–computer interface (HCI). This is especially important for future HCI applications that are expected to work in concert with humans. In this p...

In this research, we propose a method using a spectroscopic camera to estimate the concentration of blood in different layers of skin tissue. For the demonstration that shows the possibility of application of our method, we conducted a stimulation experiment on 20 subjects involving hot or carbonated baths to promote blood circulation, and estimate...

In this paper, we proposed a method for matching the color and glossiness of an object between different displays by using tone mapping. Since displays have their own characteristics, such as maximum luminance and gamma characteristics, the color and glossiness of an object when displayed differs from one display to another. The color can be correc...

Analysis of concentration of components in skin tissue, such as melanin, blood, oxygen saturation, is important in medical or cosmetic fields. Some researchers have developed methods to estimate the components. However, almost all the research has not considered a depth dependency of blood concentration and oxygen saturation, which is actual charac...

In this study, we verified the effectiveness of Kampo medicine by evaluating the changes in the feature values of facial skin texture and microcirculation at two distinct tissue depths (subcutaneous 2 mm and 8 mm). A total of 80 patients who took the Kampo formula participated in this study, and the changes in the feature values of facial skin text...

We developed a system to improve the quality of telemedicine, and the test results obtained have been presented in this paper, along with the technical details of the system. The spread of COVID-19 has accelerated the need for telemedicine to effectively prevent infections. However, in traditional Japanese medicine (Kampo), where color is essential...

We propose a method for measuring capillary refill time based on skin color changes after releasing finger pressure on the forearm using an RGB camera. We also estimate the finger force from the fingernail color.

In this paper, we propose a non-invasive method for measuring pulse waves corresponding to heart rate (HR) by capturing the color change on the soles of rats’ feet using a high-speed RGB camera. Remote photoplethysmography (rPPG) with a camera has been used as a non-invasive biometric method. However, the rPPG method has been challenging to apply t...

Insufficient signal-to-noise ratio (SNR) limits the quality of cardiac pulses measured by multitap CMOS image sensors (CISs). We present an in-pixel temporal redundant samplings (TRS) method for enhancing the SNR through sampling the charges originated by one individual optical scene into the multiple storage taps in turn with the identical time wi...

The circumference of a limb is an important parameter in the follow-up of an edema. Recently, several methods of measuring the circumference on a limb using 3D cameras have been proposed. However, the 3D cameras used are expensive and difficult to implement in general medical facilities. In this study, we propose a circumference-measurement method...

The aim of this study was to propose a method to assess images of the tongue captured using a polarized light camera for diagnostic use in Kampo medicine. Glossy and non-glossy images of the tongue were captured simultaneously using a polarizing camera and a polarizing plate. Data augmentation was performed by modulating the color and gloss, result...

Noncontact camera-based methods for measuring the pulse rate have been introduced in recent years. However, in previous studies, pulse-rate measurement experiments have been conducted only for adults and/or sleeping infants, and not for infants, who are awake. In this study, we compare the principal noncontact camera-based methods for pulse-rate me...

In this article, the authors propose a method to estimate the ink layer layout for a three-dimensional (3D) printer. This enables 3D printed skin to be produced with the desired translucency, which they represent as line spread function (LSF). A deep neural network in an encoder–decoder model is used for the estimation. It was previously reported t...

We propose a layout estimation method for multi-layered ink using a measurement of the line spread function (LSF) and machine learning. The three-dimensional printing market for general consumers focuses on the reproduction of realistic appearance. In particular, for the reproduction of human skin, it is important to control translucency by adoptin...

People with autism spectrum disorder (ASD) exhibit atypicality in various domains of behavior. Previous psychophysiological studies have revealed an atypical pattern of autonomic nervous system (ANS) activation induced by psychosocial stimulation. Thus, it might be feasible to develop a novel assessment tool to evaluate the risk of ASD by measuring...

A 7.1- $\mu \text{m}$ -pitch lock-in pixel (LIP) CMOS image sensor (CIS) that is capable of simultaneously phasing with two different near-infrared-band (NIR-band) light pulses is developed for remote physiological measurement, particularly a heart rate variability (HRV) signal monitor. The proposed dual NIR-band lock-in operation is realized by a...

In this study, we propose a method to measure simultaneously the bidirectional reflectance distribution function (BRDF) and the radius of curvature of a curved surface. The surfaces of pre-molded products or artworks are not always flat. The measurement targets were a planar surface and convex surfaces with constant radiuses of curvature. We develo...

We propose an efficient framework with compatibility between normal printing and printing with special color inks in this paper. Special color inks can be used for printing to represent some particular colors and specific optical properties, which are difficult to express using only CMYK inks. Special color layers are required in addition to the ge...

In this paper, we numerically analyze the effect of light source distance on the appearance of gloss unevenness. It is important to measure gloss unevenness because it affects the texture and appearance of a product. We propose a more efficient technique for measuring gloss unevenness. The experimental results for the proposed apparatus in this stu...

We propose an efficient framework with compatibility between normal printing and printing with special color inks in this paper. Special color inks can be used for printing to represent some particular colors and specific optical properties, which are difficult to express using only CMYK inks. Special color layers are required in addition to the ge...

Artificial red blood cells (i.e. hemoglobin [Hb] vesicles [Hb‐Vs]) function effectively as photosensitizers in flashlamp‐pumped pulsed‐dye laser (PDL) treatment for port‐wine stains in animal models. Hb‐Vs deliver more Hb to the vicinity of the endothelial cells. Both Hb‐Vs and red blood cells absorb the laser energy and generate heat, supporting t...

In this paper, we propose a method for estimating stress levels using multibiological information obtained from red-green-blue (RGB) facial videos. The most popular method for estimating stress is by questionnaire, which is not highly objective or accurate. To estimate stress levels more accurately, it is desirable to establish the method which can...

We propose a simple and robust method to measure skin capillary refill time (CRT), which quantitatively assesses the quality of peripheral perfusion of a person, using an iPhone.

In this study, we propose the simultaneous measurement method of the bidirectional reflection distribution function (BRDF) and the radius of curvature by using pattern illumination. For nonplanar objects, the angle of reflection light changes according to the surface normal angle of curved object. Therefore, it is necessary to consider the effects...

In this paper, we propose a method to estimate ink layer layout used as an input for 3D printer. This method makes it possible to reproduce a 3D printed patch that gives a desired translucency, which is represented as Line Spread Function (LSF) in this study. Deep neural networks of encoder decoder model is used for the estimation. In a previous re...

In this research, we propose a method to assess images of the tongue captured using a polarized light camera for diagnostic use in Kampo Medicine. The polarized light camera is used to simultaneously capture glossy and non-glossy images of the tongue. Data augmentation was performed by modulating the color and gloss, through which the number of ima...

Regular observation and recording of the changes in body appearance are essential for the process of the treatment of plastic surgery and dermatology, especially aesthetic surgery. Usually, physicians treat patients with medical interviews, pictures of the patient's faces before and after their treatment, anatomical data that including size, locati...

In this paper, we propose a system that can measure the spatially-varying reflectance of real materials. Our system uses two handheld cameras, a small LED light, a turning table, and a chessboard with markers. The two cameras are used as a view and light cameras respectively to acquire incoming and outgoing light directions simultaneously, and the...

In this study, we conducted an experiment using a mobile augmented reality (AR) system that allows direct manipulation of a virtual object with the user's hand and that presents material appearance of the object. The system presents user-perspective images on the mobile display, which enables natural interaction with virtual objects. We compared th...

Tongue diagnosis is an important diagnostic method in Japanese-traditional (Kampo) medicine, and the tongue is considered as a mirror reflecting the internal state of the body. Imbalance of water, heat, bloodstream, and to diagnose blood stagnation, blood deficiency, yin deficiency and so on in Kampo medicine can be diagnosed with tongue diagnosis....

In this paper, we investigate the possibility of using image-based pulse transit time (iPTT) to estimate blood pressure using an RGB (red, green, blue)-camera video taken from the face only. It is generally known that there is a correlation between iPTT and blood pressure. iPTT refers to the difference between the times when a pulse wave reaches tw...

In this study, we verified whether or not feature values of the facial skin texture can be used for facial complexion diagnosis. A total of 10 patients with fatigue were recruited and categorized under two groups: acupuncture group and blank control group. In the acupuncture group, the contact needle technique was used, while in the blank control g...

In this paper, we propose a color reproduction method using color charts to improve the color quality of a telemedicine system. Owing to the spread of COVID-19, the need for telemedicine is rapidly increasing to prevent infections more effectively. However, in practices such as traditional Japanese (Kampo) medicine, where color is used as an import...

In this paper, we propose a method to measure multiple stress levels using pulse wave extracted from a facial video. Stress is a crucial factor which triggers mental illnesses. For the prevention, it is necessary to measure and handle with stress. Many researchers have attempted to measure stress using with a contact sensor. However, contact sensor...

In this paper, we propose a method to remove the influence of light on the face from display in imaging photoplethysmography (iPPG) to measure blood flow of the face when watching a video with an RGB camera. We propose a simple method which is used to correct the pixel values of the face by the pixel values of a standard white plate. The proposed m...

In this paper, we propose a method to visualize blood flow of palm in different muscle tense state using RGB high-speed camera. Recently, new modalities are needed to develop a more accurate system to non-contact multi-modal affect analysis. Then, we focus on muscle tense. The muscle tense is caused by stress. Hence, the muscle tense is one of the...

In this study, U-Net based deep convolutional networks are used to achieve the segmentation of particle regions in a microscopic image of colorants. The material appearance of products is greatly affected by the distribution of the particle size. From that fact, it is important to obtain the distribution of the particle size to design the material...

A method is proposed to estimate the concentration of pigments mixed in a painting, using the encoder–decoder model of neural networks. The model is trained to output a value that is the same as its input, and its middle output extracts a certain feature as compressed information about the input. In this instance, the input and output are spectral...

Biometric template protection techniques have been proposed to address security and privacy issues inherent to biometric-based authentication systems. However, it has been shown that the robustness of most of such techniques against reversibility and linkability attacks are overestimated. Thus, a thorough security analysis of recently proposed temp...

Gloss is evaluated based on not only the intensity of reflected light but also the sharpness of the light source image. In this paper, we discuss the line spread function of specular reflection (SR-LSF). We developed a measurement apparatus for the SR-LSF based on the visual inspection of gloss. The measurement results of six types of samples with...

Background: Some theorists have suggested that the ability of visual perspective-taking (VPT) constitutes a rudimentary process of social cognition, and as such, the ability of VPT in people with autism spectrum disorder (ASD) has been the focus of intensive research. Aim: The present study investigated whether adult males with ASD show signs of...

Improving drivers’ risk prediction ability can reduce the accident risk significantly. The existing accident awareness training systems show poor performance due to the lack of immersive sense. In this research, an immersive educational system is proposed for risk prediction training based on VR technology. The system provides a highly realistic dr...

The market size of online video advertising is expanding rapidly along with the spread of smartphones and social media. In this study, we estimate advertising effectiveness in the natural environment using online data collection and the remote measurement of webcam facial expressions and physiological responses. We collected 4, 108 videos of the fa...

In this paper, we propose a method of image conversion that changes unglossy images into glossy ones using deep photo style transfer. This style of transfer is expected to reproduce the desired image with a metallic appearance based on the technique used for texture transfer. A practical challenge was undertaken to create a gold, metallic image by...

The conventional skin color separation method suffers from the severe limitation that the input skin image must have no shading component and must have substantial variation in the pigment density distribution, simultaneously. We therefore propose an improved method, which uses two input skin images instead of one. In this improved method, one inpu...

In recent years, the problem of declining birthrate has occurred, and how to increase the working-age population has been discussed as a problem to be tackled immediately. The rise in the unmarried rate also contributes to a decline in the birth rate. To solve these problems, you need a way to choose a partner appropriately. In this study, we inves...

In this paper, we proposed a useful calculation method to perform the perception-based selection of lighting position that emphasizes the material appearance of CG object. The proposed method is based on the conventional viewpoint entropy which is used to find an appropriate viewing angle. To find an appropriate shot for material appearance, we fir...

In this study, we propose a method to detect wetness on the surface of human skin and skin phantoms using an RGB camera. Recent research on affect analysis has addressed the non-contact multi-modal analysis of affect aimed at such applications as automated questionnaires. New modalities are needed to develop a more accurate system for analyzing aff...

In this research, we proposed a method to estimate the concentrations of melanin and blood in different layers of skin tissue. Furthermore, we stimulated skin with a warm bath and a carbon dioxide bath and obtained spectral data by a multispectral camera six times during 18 minutes. Based on the captured image, we estimated the blood concentrations...

In this paper, we propose a method to estimate the concentration of pigments mixed in a painting, using the encoder-decoder model of neural networks. Encoder-decoder model is trained to output value which is same as input and its middle output extracts a certain feature as compressed information of the input. In this instance, the input and the out...

3D reconstruction is used for inspection of industrial products. The demand for measuring 3D shapes is increased. There are many methods for 3D reconstruction using RGB images. However, it is difficult to reconstruct 3D shape using RGB images with gloss. In this paper, we use the deep neural network to remove the gloss from the image group captured...

In this paper, we propose a layout estimation method for multi-layered ink by using PSF measurement and machine learning. This estimation can bring various capabilities of color reproduction for the newfangled 3D printer that can apply multi-layered inkjet color. Especially, the control of translucency is useful for the reproduction of skin color t...

In this paper, we perform multi-resolution analysis by using Wavelet transform for the components of surface reflection on faces in order to acquire features of wrinkles and fine asperities on the face. Also, by applying principal component analysis to the acquired trend of the wrinkles and the fine asperities, we analyze the relationship between t...

In this paper, we investigated the correlation between blood pressure (BP) and image-based pulse transit time (iPTT) acquired from only face using RGB camera. In general, the value of iPTT can be calculated from a transition time at peaks of hemoglobin amount which is extracted from RGB values of the camera. The transition time of peaks is obtained...

In this paper, we examined the physical meaning of the widely used gloss evaluation. Gloss is one of the important qualities for materials. A visual test of the sharpness of a specular reflection image, i.e. gloss, on the surface of materials is often performed to estimate the gloss of materials. If the specular reflection image is sharp, we estima...

In this paper, we propose a method for measuring the reflectance of anisotropic materials using a simple apparatus consisting of two handheld cameras, a small LED light source, a turning table and a chessboard with markers. The system is configured to obtain the different incoming and outgoing light directions, and the brightness of pixels on the s...

Appearance is observed as the results of association among various physical factors. For example, the color appearance is well known. We have an ability to estimate the primary color of object through an adaptation, even if only the light source color is changed. It is called as color constancy which is remarkable ability of human eyes. Likewise, h...

Non-contact (remote) physiological monitor systems become popular for developing the next generation internet-of-things (IoT) products aimed to social and human security applications. CMOS image sensors (CIS) act as a very important role as the front-end signal receiver of the monitor systems. Recently, we reported on a time-resolved (TR) near-infr...

In this study, we analyzed color changed by wetness in various areas on palm to clarify the influence of highlights. To improve the accuracy of emotion estimation, it is necessary to suggest other modalities combined with the conventional modalities. Therefore, we focused on emotional sweating, which is reliable modality of contact methods but not...

We propose a method of estimating a bivariate BRDF from a small number of sampled data using compressed sensing. This method aims to estimate the reflectance of various materials by using the representation space that keeps local information when restored by compressed sensing. We conducted simulated measurements using randomly sampled data and dat...

The change in the edema condition is visualized considering the three-dimensional shape. Continuous treatment and observation are indispensable for patients with edema. The measurement and evaluation of the three-dimensional shape of the leg are thus important in evaluating edema of the leg. Such an evaluation can confirm the therapeutic effect and...