Visar Berisha’s research while affiliated with Arizona State University and other places

Purpose Phonetic forced alignment has a multitude of applications in automated analysis of speech, particularly in studying nonstandard speech such as children's speech. Manual alignment is tedious but serves as the gold standard for clinical-grade alignment. Current tools do not support direct training on manual alignments. Thus, a trainable speaker adaptive phonetic forced alignment system, Wav2TextGrid, was developed for children's speech. The source code for the method is publicly available along with a graphical user interface at https://github.com/pkadambi/Wav2TextGrid. Method We propose a trainable, speaker-adaptive, neural forced aligner developed using a corpus of 42 neurotypical children from 3 to 6 years of age. Evaluation on both child speech and on the TIMIT corpus was performed to demonstrate aligner performance across age and dialectal variations. Results The trainable alignment tool markedly improved accuracy over baseline for several alignment quality metrics, for all phoneme categories. Accuracy for plosives and affricates in children's speech improved more than 40% over baseline. Performance matched existing methods using approximately 13 min of labeled data, while approximately 45–60 min of labeled alignments yielded significant improvement. Conclusion The Wav2TextGrid tool allows alternate alignment workflows where the forced alignments, via training, are directly tailored to match clinical-grade, manually provided alignments. Supplemental Material https://doi.org/10.23641/asha.28593971

Objectives/Goals: Aspiration causes or aggravates lung diseases. While bedside swallow evaluations are not sensitive/specific, gold standard tests for aspiration are invasive, uncomfortable, expose patients to radiation, and are resource intensive. We propose the development and validation of an AI model that analyzes voice to noninvasively predict aspiration. Methods/Study Population: Retrospectively recorded [i] phonations from 163 unique ENT patients were analyzed for acoustic features including jitter, shimmer, harmonic to noise ratio (HNR), etc. Patients were classified into three groups: aspirators (Penetration-Aspiration Scale, PAS 6–8), probable (PAS 3–5), and non-aspirators (PAS 1–2) based on video fluoroscopic swallow (VFSS) findings. Multivariate analysis evaluated patient demographics, history of head and neck surgery, radiation, neurological illness, obstructive sleep apnea, esophageal disease, body mass index, and vocal cord dysfunction. Supervised machine learning using five folds cross-validated neural additive network modelling (NAM) was performed on the phonations of aspirator versus non-aspirators. The model was then validated using an independent, external database. Results/Anticipated Results: Aspirators were found to have quantifiably worse quality of sound with higher jitter and shimmer but lower harmonics noise ratio. NAM modeling classified aspirators and non-aspirators as distinct groups (aspirator NAM risk score 0.528+0.2478 (mean + std) vs. non-aspirator (control) risk score of 0.252+0.241 (mean + std); p Discussion/Significance of Impact: We report the use of voice as a novel, noninvasive biomarker to detect aspiration risk using machine learning techniques. This tool has the potential to be used for the safe and early detection of aspiration in a variety of clinical settings including intensive care units, wards, outpatient clinics, and remote monitoring.

Millimeter Wave (mmWave) radar has emerged as a promising modality for speech sensing, offering advantages over traditional microphones. Prior works have demonstrated that radar captures motion signals related to vocal vibrations, but there is a gap in the understanding of the analytical connection between radar-measured vibrations and acoustic speech signals. We establish a mathematical framework linking radar-captured neck vibrations to speech acoustics. We derive an analytical relationship between neck surface displacements and speech. We use data from 66 human participants, and statistical spectral distance analysis to empirically assess the model. Our results show that the radar-measured signal aligns more closely with our model filtered vibration signal derived from speech than with raw speech itself. These findings provide a foundation for improved radar-based speech processing for applications in speech enhancement, coding, surveillance, and authentication.

Existing methods for analyzing linguistic content from picture descriptions for assessment of cognitive-linguistic impairment often overlook the participant's visual narrative path, which typically requires eye tracking to assess. Spatio-semantic graphs are a useful tool for analyzing this narrative path from transcripts alone, however they are limited by the need for manual tagging of content information units (CIUs). In this paper, we propose an automated approach for estimation of spatio-semantic graphs (via automated extraction of CIUs) from the Cookie Theft picture commonly used in cognitive-linguistic analyses. The method enables the automatic characterization of the visual semantic path during picture description. Experiments demonstrate that the automatic spatio-semantic graphs effectively differentiate between cognitively impaired and unimpaired speakers. Statistical analyses reveal that the features derived by the automated method produce comparable results to the manual method, with even greater group differences between clinical groups of interest. These results highlight the potential of the automated approach for extracting spatio-semantic features in developing clinical speech models for cognitive impairment assessment.

Purpose: This commentary introduces how artificial intelligence (AI) can be leveraged to advance cross-language intelligibility assessment of dysarthric speech. Method: We propose a dual-component framework consisting of a universal module that generates language-independent speech representations and a language-specific intelligibility model that incorporates linguistic nuances. Additionally, we identify key barriers to cross-language intelligibility assessment, including data scarcity, annotation complexity, and limited linguistic insights, and present AI-driven solutions to overcome these challenges. Conclusion: Advances in AI offer transformative opportunities to enhance cross-language intelligibility assessment for dysarthric speech by balancing scalability across languages and adaptability by languages.

Hypothesis testing is a statistical inference approach used to determine whether data supports a specific hypothesis. An important type is the two-sample test, which evaluates whether two sets of data points are from identical distributions. This test is widely used, such as by clinical researchers comparing treatment effectiveness. This tutorial explores two-sample testing in a context where an analyst has many features from two samples, but determining the sample membership (or labels) of these features is costly. In machine learning, a similar scenario is studied in active learning. This tutorial extends active learning concepts to two-sample testing within this \textit{label-costly} setting while maintaining statistical validity and high testing power. Additionally, the tutorial discusses practical applications of these label-efficient two-sample tests.