Tiago H. Falk

• Professor
• Professor (Associate) at National Institute of Scientific Research

Introduction This study aimed to investigate the psychological and physiological impacts of audio-visual (AV) and audio-visual-olfactory (AVO) stimuli within an immersive virtual nature environment. Methods Twenty-two nurses from the mental health in-patient ward of a Canadian hospital participated in the study. Each participant chose one of the t...

Provides society information that may include news, reviews or technical notes that should be of interest to practitioners and researchers.

Speech is known to carry health-related attributes, which has emerged as a novel venue for remote and long-term health monitoring. However, existing models are usually tailored for a specific type of disease, and have been shown to lack generalizability across datasets. Furthermore, concerns have been raised recently towards the leakage of speaker...

On the centenary of the first human EEG recording, more than 500 experts reflect on the impact that this discovery has had on our understanding of the brain and behaviour. We document their priorities and call for collective action focusing on validity, democratization and responsibility to realize the potential of EEG in science and society over t...

We present a one-year-long multi-sensor dataset collected from honey bee colonies (Apis mellifera) with rich phenotypic measurements. Data were collected non-stop from April 2020 to April 2021 from 53 hives located at two apiaries in Québec, Canada. The sensor data included audio features, temperature, and relative humidity. The phenotypic measurem...

Speech is known to carry health-related attributes, which has emerged as a novel venue for remote and long-term health monitoring. However, existing models are usually tailored for a specific type of disease, and have been shown to lack generalizability across datasets. Furthermore, concerns have been raised recently towards the leakage of speaker...

In this paper, we present a multimodal dataset obtained from a honey bee colony in Montr\'eal, Quebec, Canada, spanning the years of 2021 to 2022. This apiary comprised 10 beehives, with microphones recording more than 2000 hours of high quality raw audio, and also sensors capturing temperature, and humidity. Periodic hive inspections involved moni...

Deep neural network (DNN) classifiers are potent instruments that can be used in various security-sensitive applications. Nonetheless, they are vulnerable to certain attacks that impede or distort their learning process. For example, backdoor attacks involve polluting the DNN learning set with a few samples from one or more source classes, which ar...

With advances seen in deep learning, voice-based applications are burgeoning, ranging from personal assistants, affective computing, to remote disease diagnostics. As the voice contains both linguistic and para-linguistic information (e.g., vocal pitch, intonation, speech rate, loudness), there is growing interest in voice anonymization to preserve...

This survey focuses on a crucial virtual reality (VR) issue that has been reported to affect roughly 40% of VR users – cybersickness. Cybersickness is similar to motion sickness but occurs with electronic screens or VR displays instead of actual movement. Cybersickness can refer to a cluster of symptoms, including nausea, eye strain, vertigo, and s...

We developed and studied a hardware-based neuromorphic wave computer using optical nonlinearities. Here, we showcase the low-power consumption, robustness, and scalable complexity of soliton-based spectral broadening as a novel neuromorphic approach.

Brain–computer interface (BCI) technology has emerged as an influential communication tool with extensive applications across numerous fields, including entertainment, marketing, mental state monitoring, and particularly medical neurorehabilitation. Despite its immense potential, the reliability of BCI systems is challenged by the intricacies of da...

Recent advances in self-supervised learning have allowed automatic speech recognition (ASR) systems to achieve state-of-the-art (SOTA) word error rates (WER) while requiring only a fraction of the labeled data needed by its predecessors. Notwithstanding, while such models achieve SOTA results in matched train/test scenarios, their performance degra...

Multisensory virtual reality (VR) experiences, which include haptic and/or olfactory feedback, are growing in popularity. Notwithstanding, an understanding of their impact on perceived quality of experience (QoE), and on several QoE factors, such as sense of presence, immersion, realism, engagement, emotions, and cybersickness, is still limited. He...

The performance limitations of traditional computer architectures have led to the rise of brain‐inspired hardware, with optical solutions gaining popularity due to the energy efficiency, high speed, and scalability of linear operations. However, the use of optics to emulate the synaptic activity of neurons has remained a challenge since the integra...

Introduction: Immersive virtual reality (VR) applications are burgeoning within healthcare as they promote high levels of engagement. Notwithstanding, existing solutions only stimulate two of our five senses (audio and visual), thus may not be optimal in the sense of promoting immersion and of “being present”. In this paper, we explore the benefits...

We have all experienced the sense of time slowing down when we are bored or speeding up when we are focused, engaged, or excited about a task. In virtual reality (VR), perception of time can be a key aspect related to flow, immersion, engagement, and ultimately, to overall quality of experience. While several studies have explored changes in time p...

Photoplethysmography (PPG) is used to measure blood volume changes in the microvascular bed of tissue. Information about these changes along time can be used for estimation of various physiological parameters, such as heart rate variability, arterial stiffness, and blood pressure, to name a few. As a result, PPG has become a popular biological moda...

Large self-supervised pre-trained speech models have achieved remarkable success across various speech-processing tasks. The self-supervised training of these models leads to universal speech representations that can be used for different downstream tasks, ranging from automatic speech recognition (ASR) to speaker identification. Recently, Whisper,...

Unsupervised speech models are becoming ubiquitous in the speech and machine learning communities. Upstream models are responsible for learning meaningful representations from raw audio. Later, these representations serve as input to downstream models to solve a number of tasks, such as keyword spotting or emotion recognition. As edge speech applic...

With advances seen in deep learning, voice-based applications are burgeoning, ranging from personal assistants, affective computing, to remote disease diagnostics. As the voice contains both linguistic and paralinguistic information (e.g., vocal pitch, intonation, speech rate, loudness), there is growing interest in voice anonymization to preserve...

Brain-computer interfaces (BCI) have been developed to allow users to communicate with the external world by translating brain activity into control signals. Motor imagery (MI) has been a popular paradigm in BCI control where the user imagines movements of e.g., their left and right limbs and classifiers are then trained to detect such intent direc...

Self-supervised speech pre-training enables deep neural network models to capture meaningful and disentangled factors from raw waveform signals. The learned universal speech representations can then be used across numerous downstream tasks. These representations, however, are sensitive to distribution shifts caused by environmental factors, such as...

Biomarkers based on resting-state electroencephalography (EEG) signals have emerged as a promising tool in the study of Alzheimer’s disease (AD). Recently, a state-of-the-art biomarker was found based on visual inspection of power modulation spectrograms where three “patches” or regions from the modulation spectrogram were proposed and used for AD...

The coronavirus disease 2019 (COVID-19) pandemic has drastically impacted life around the globe. As life returns to pre-pandemic routines, COVID-19 testing has become a key component, assuring that travellers and citizens are free from the disease. Conventional tests can be expensive, time-consuming (results can take up to 48h), and require laborat...

Heart rate variability (HRV) has been a useful tool for understanding human behavior. HRV features, derived from the inter-beat interval (RR) time series, reflect the autonomic nervous system processes of the body and have shown correlates with various mental processes. These processes include mental fatigue, workload, and anxiety, to name a few. D...

p>Speech COVID-19 detection systems have gained popularity as they represent an easy-to-use and low-cost solution that is well suited for at-home long-term monitoring of patients with persistent symptoms. Recently, however, the limited generalization capability of existing deep neural network based systems to unseen datasets has been raised as a se...

p>Speech COVID-19 detection systems have gained popularity as they represent an easy-to-use and low-cost solution that is well suited for at-home long-term monitoring of patients with persistent symptoms. Recently, however, the limited generalization capability of existing deep neural network based systems to unseen datasets has been raised as a se...

The vital role of honeybees in pollination and their high rate of mortality in the last decade have raised concern among beekeepers and researchers alike. As such, robust and remote sensing of beehives has emerged as a potential tool to help monitor the health of honeybees. Over the last decade, several monitoring systems have been proposed, includ...

Self-supervised speech representation learning aims to extract meaningful factors from the speech signal that can later be used across different downstream tasks, such as speech and/or emotion recognition. Existing models, such as HuBERT, however, can be fairly large thus may not be suitable for edge speech applications. Moreover, realistic applica...

p>Cough is an important symptom of numerous respiratory diseases, including COVID-19. While different cough phases (i.e., inhalation, compression, and expulsion) have been shown to be related to different pathological origins, existing cough-based COVID-19 detection systems rely on the entire cough recording, thus such phase-related characteristics...

p>Cough is an important symptom of numerous respiratory diseases, including COVID-19. While different cough phases (i.e., inhalation, compression, and expulsion) have been shown to be related to different pathological origins, existing cough-based COVID-19 detection systems rely on the entire cough recording, thus such phase-related characteristics...

With the advent of standalone virtual reality (VR) headsets, multisensory VR experiences are emerging with the hopes of better simulating real-world experiences. For example, innovations in haptic suits and scent diffusion devices are burgeoning. While stimulating multiple senses re-orientates the perceived quality of experience (QoE) by increasing...

Stress and anxiety are increasingly present in society, contributing to many chronic diseases and decreasing quality of life. Non-pharmacological therapies to relieve these symptoms and promote relaxation have been developed, including immersing oneself in nature (so-called ‘forest bathing’). Access to nature, however, is not available to many arou...

Measuring a gamer’s behaviour and perceived gaming experience in real-time can be crucial not only to assess game usability, but to also adjust the game play and content in real-time to maximize the experience per user. For this purpose, affective and physiological monitoring tools (e.g., wearables) have been used to monitor human influential facto...

Automatic speech emotion recognition (SER) has gained popularity over the last decade and numerous Challenges have emerged. While the latest Challenges have shown that deep neural networks achieve the best results, existing input features are still a bottleneck and cause severe performance degradation in realistic “in-the-wild” scenarios. In this p...

Assessment of mental workload in real-world conditions is key to ensuring the performance of workers executing tasks that demand sustained attention. Previous literature has employed electroencephalography (EEG) to this end despite having observed that EEG correlates of mental workload vary across subjects and physical strain, thus making it diffic...

Neurofeedback has been suggested as a potential complementary therapy to different psychiatric disorders. Of interest for this approach is the prediction of individual performance and outcomes. In this study, we applied functional connectivity-based modeling using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) modali...

Recent technological advances have allowed for virtual reality applications to burgeon. With virtual reality (VR), so-called human influential factors play a crucial role in the final perceived immersive media experience (IMEx). While two individuals can use the same VR headset, play the same game in the same location, and have the same goals, the...

To date, several methods have been explored for the challenging task of cross-language speech emotion recognition, including the bag-of-words (BoW) methodology for feature processing, domain adaptation for feature distribution “normalization”, and data augmentation to make machine learning algorithms more robust across testing conditions. Their com...

Virtual reality applications are on the rise and touching numerous domains, including healthcare, training, and gaming, to name a few. Existing experiences, however, are not fully immersive, as only two senses (audio-visual) are stimulated. To overcome this limitation, olfactory and haptic devices are emerging, thus making multisensory immersive ex...

Wearable devices are burgeoning, and applications across numerous verticals are emerging, including human performance monitoring, at-home patient monitoring, and health tracking, to name a few. Off-the-shelf wearables have been developed with focus on portability, usability, and low-cost. As such, when deployed in highly ecological settings, wearab...

Virtual reality (VR) applications, especially those where the user is untethered to a computer, are becoming more prevalent as new hardware is developed, computational power and artificial intelligence algorithms are available, and wireless communication networks are becoming more reliable, fast, and providing higher reliability. In fact, recent pr...

Virtual reality (VR)-mediated rehabilitation is emerging as a useful tool for stroke survivors to recover motor function. Recent studies are showing that VR coupled with physiological computing (i.e., real-time measurement and analysis of different behavioral and psychophysiological signals) and feedback can lead to 1) more engaged and motivated pa...

Bees play an important role in agriculture and ecology, and their pollination efficiency is essential to the economic profitability of farms. The drastic decrease in bee populations witnessed over the last decade has attracted great attention to automated remote beehive monitoring research, with beehive acoustics analysis emerging as a prominent fi...

Time delay neural networks (TDNN) have become ubiquitous for voice biometrics and language recognition tasks relying on utterance-level speaker- or language-dependent representations. In this paper, we discuss directions to improve upon the conventional TDNN architecture to render it more generally applicable. More specifically, we explore the util...

The prevalence of stress and anxiety has increased dramatically in recent decades, especially with the global COVID-19 pandemic. In parallel, effective ways of objectively assessing and quantifying these conditions have continued to be explored. Affective computing is one such technique that has gained popularity recently, using physiological signa...

It is known that human influential factors (HIFs, e.g., sense of presence/immersion; attention, stress, and engagement levels; fun factors) play a crucial role in the gamer’s perceived immersive media experience [1]. To this end, recent research has explored the use of affective brain-/body-computer interfaces to monitor such factors [2, 3]. Typica...

Background Alzheimer’s disease (AD) is a progressive neurodegenerative disease accounting for 60–80∖% of dementia cases worldwide[2]. Early diagnosis can decrease the severity of the disorder in addition to improving the quality of life of patients. Biomarkers based on electroencephalography (EEG) have emerged as a promising tool in the study of AD...

Unobtrusive monitoring of driver mental states has been regarded as an important element in improving the safety of existing transportation systems. While many solutions exist relying on camera-based systems for e.g., drowsiness detection, these can be sensitive to varying lighting conditions and to driver facial accessories, such as eye/sunglasses...

Chronic obstructive pulmonary disease (COPD) is one of the leading causes of human mortality worldwide. Traditionally, estimating COPD severity has been done in controlled clinical conditions using cough sounds, respiration, and heart rate variability, with the latter reporting insights on the autonomic dysfunction caused by the disease. Advancemen...

Single-shot 2D optical imaging of transient scenes is indispensable for numerous areas of study. Among existing techniques, compressed optical-streaking ultrahigh-speed photography (COSUP) uses a cost-efficient design to endow ultrahigh frame rates with off-the-shelf CCD and CMOS cameras. Thus far, COSUP’s application scope is limited by the long p...

Prediction of mental states, such as stress and anxiety, can be important in situations where reduced job performance due to increased mental strain can lead to critical situations (e.g., front-line healthcare workers and first responders). While recent advances in biomedical wearable sensor technologies have allowed for collection of multiple phys...

While several biomarkers have been developed for the detection of Alzheimer's disease (AD), not many are available for the prediction of disease severity, particularly for patients in the mild stages of AD. In this paper, we explore the multimodal prediction of Mini-Mental State Examination (MMSE) scores using resting-state electroencephalography (...

Smart healthcare is a framework that utilizes technologies such as wearable devices, the Internet of Medical Things (IoMT), sophisticated machine learning algorithms, and wireless communication technology to seamlessly access health records, link individuals, resources, and organizations, and then effectively handle and react to health environment...

Dementia describes a set of symptoms that occur in neurodegenerative disorders and that is characterized by gradual loss of cognitive and behavioral functions. Recently, non-invasive neurofeedback training has been explored as a potential complementary treatment for patients suffering from dementia or mild cognitive impairment. Here we systematical...

Hospital workers are known to work long hours in a highly stressful environment. The COVID-19 pandemic has increased this burden multi-fold. Pre-COVID statistics already showed that one in every three nurses reported burnout, thus affecting patient satisfaction and the quality of their provided service. Real-time monitoring of burnout, and other un...

Affective neurofeedback training allows for the self-regulation of the putative circuits of emotion regulation. This approach has recently been studied as a possible additional treatment for psychiatric disorders, presenting positive effects in symptoms and behaviors. After neurofeedback training, a critical aspect is the transference of the learne...

Intra-speaker variability, caused by emotional speech, is a real threat to the performance of speaker recognition systems. In fact, as human beings, we are constantly changing our emotional state. While many efforts have been made to increase automatic speaker verification (ASV) robustness towards channel effects or spoofing attacks, only a handful...

Recently, due to the emergence of mobile electroencephalography (EEG) devices, assessment of mental workload in highly ecological settings has gained popularity. In such settings, however, motion and other common artifacts have been shown to severely hamper signal quality and to degrade mental workload assessment performance. Here, we show that cla...

Dementia describes a set of symptoms that occur in neurodegenerative disorders and that is characterized by gradual loss of cognitive and behavioral functions. Recently, non-invasive neurofeedback training has been explored as a potential complementary treatment for patients suffering from dementia or mild cognitive impairment. Here we systematical...

Spoofing attacks have been acknowledged as a serious threat to automatic speaker verification (ASV) systems. In this paper, we are specifically concerned with replay attack scenarios. As a countermeasure to the problem, we propose a front-end based on the blind estimation of the channel response magnitude and as a back-end a residual neural network...

Measuring saccadic eye movements when wearing a virtual reality (VR) head-mounted display (HMD) has recently gained a lot of attention, as it allows for enriched user experiences. This has led to an increase in devices showcasing camera-based eye tracking capabilities. Such devices, however, can be orders of magnitude more expensive than convention...

With the burgeoning of wearable devices and passive body/brain-computer interfaces (B/BCIs), automated stress monitoring in everyday settings has gained significant attention recently, with applications ranging from serious games to clinical monitoring. With mobile users, however, challenges arise due to other overlapping (and potentially confoundi...

Assessment of mental workload is crucial for applications that require sustained attention and where conditions such as mental fatigue and drowsiness must be avoided. Previous work that attempted to devise objective methods to model mental workload were mainly based on neurological or physiological data collected when the participants performed tas...