Adam Lammert’s research while affiliated with College of the Holy Cross and other places

Reverse Correlation (RC) is an established method for reconstructing auditory representations, that has recently emerged as a tool for characterizing the sounds experienced by tinnitus patients. Toward further optimizing RC for auditory research, the present work investigated the influence of subject instructions on characterization quality of tinnitus-like sounds. A validation study was conducted in which eighteen normal-hearing subjects were randomly assigned one of three candidate instruction sets, each inspired by the RC literature. Results show a significant effect of instruction set on characterization quality, and reveal that instructing subjects to detect a hidden signal in the RC stimuli resulted the best reconstruction.

italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Goal: We validate a recent reverse correlation approach to tinnitus characterization by applying it to individuals with clinically-diagnosed tinnitus. Methods: Two tinnitus patients assessed the subjective similarity of their non-tonal tinnitus percepts and random auditory stimuli. Regression of the responses onto the stimuli yielded reconstructions which were evaluated qualitatively by playing back resynthesized waveforms to the subjects and quantitatively by response prediction analysis. Results: Subject 1 preferred their resynthesis to white noise; subject 2 did not. Response prediction balanced accuracies were significantly higher than chance across subjects: subject 1: 0.5963, subject 2: 0.6922. Conclusion: Reverse correlation can provide the foundation for reconstructing accurate representations of complex, non-tonal tinnitus in clinically diagnosed subjects. Further refinements may yield highly similar waveforms to individualized tinnitus percepts.

Uncovering cognitive representations is an elusive goal that is increasingly pursued using the reverse correlation method, wherein human subjects make judgments about ambiguous stimuli. Employing reverse correlation often entails collecting thousands of stimulus-response pairs, which severely limits the breadth of studies that are feasible using the method. Current techniques to improve efficiency bias the outcome. Here we show that this methodological barrier can be diminished using compressive sensing, an advanced signal processing technique designed to improve sampling efficiency. Simulations are performed to demonstrate that compressive sensing can improve the accuracy of reconstructed cognitive representations and dramatically reduce the required number of stimulus-response pairs. Additionally, compressive sensing is used on human subject data from a previous reverse correlation study, demonstrating a dramatic improvement in reconstruction quality. This work concludes by outlining the potential of compressive sensing to improve representation reconstruction throughout the fields of psychology, neuroscience, and beyond. Supplementary Information The online version contains supplementary material available at 10.3758/s13428-023-02281-4.

Goal We validate a recent reverse correlation approach to tinnitus characterization by applying it to individuals with clinically-diagnosed tinnitus. Methods Two tinnitus patients assessed the subjective similarity of their non-tonal tinnitus percepts and random auditory stimuli. Regression of the responses onto the stimuli yielded reconstructions which were evaluated qualitatively by playing back resynthesized waveforms to the subjects and quantitatively by response prediction analysis. Results Subject 1 preferred their resynthesis to white noise; subject 2 did not. Response prediction balanced accuracies were significantly higher than chance across subjects: subject 1: 0.5963, subject 2: 0.6922. Conclusion Reverse correlation can provide the foundation for reconstructing accurate representations of complex, non-tonal tinnitus in clinically diagnosed subjects. Further refinements may yield highly similar waveforms to individualized tinnitus percepts. Impact Statement Characterization of tinnitus sounds can help clarify the heterogeneous nature of the condition and link etiology to subtypes and treatments.

High-fidelity sensor data gathered at high sampling rates is required by Human Context Recognition (HCR) to accurately predict the users’ current context. This approach consumes a significant amount of energy and increases the likelihood of uninstallation. Prior work addressed energy scarcity by reducing smartphone sensor sampling rates, which in turn reduces HCR accuracy. Up-sampling or domain adaptation techniques were then introduced to maximize HCR accuracy from sensor data with low-sampling rates. These solutions, however, are only applicable if high sampling rates of data from all context classes can be gathered from the target population, which is not realistic in a real-world application. Alternatively, the learning of HCR on multiple datasets with different sampling rates and labels is proposed in this study. We address the problem as a dataset shift problem to improve the learning of the low-sampling rate HCR dataset using information augmented from a high-sampling dataset, collected under various setups (labels, mobile devices, and population). To reduce the discrepancy of feature distribution between sampling rates and dataset domains, we propose Maximum Mean Discrepancy-based Multi-Task Learning (MMD-MTL), which innovatively introduces MMD into MTL to address the covariance shift between sampling rates within the same dataset and the dataset shift between multiple HCR datasets. In rigorous evaluation, using an external HCR dataset that gathered sensor data at 50Hz from various sets of populations, devices, and HCR labels, MMD-MTL was found to significantly increase the HCR accuracy of a crowdsourced dataset that was sampled at a rate of 5Hz up to 74%.

italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Goal: This study validates an approach to characterizing the sounds experienced by tinnitus patients via reverse correlation, with potential for characterizing a wider range of sounds than currently possible. Methods: Ten normal-hearing subjects assessed the subjective similarity of random auditory stimuli and target tinnitus-like sounds (“buzzing” and “roaring”). Reconstructions of the targets were obtained by regressing subject responses on the stimuli, and were compared for accuracy to the frequency spectra of the targets using Pearson's r . Results: Reconstruction accuracy was significantly higher than chance across subjects: buzzing: $0.52 \pm 0.27$ (mean $\pm$ s.d.), $t(9) = 5.766$ , $p < 0.001$ ; roaring: $0.62 \pm 0.23$ , $t(9) = 5.76$ , $p < 0.001$ ; combined: $0.57 \pm 0.25$ , $t(19) = 7.542$ , $p < 0.001$ . Conclusion: Reverse correlation can accurately reconstruct non-tonal tinnitus-like sounds in normal-hearing subjects, indicating its potential for characterizing the sounds experienced by patients with non-tonal tinnitus.

Modern smartphones are equipped with powerful audio hardware and processors, allowing them to acquire and perform on-device speech processing at high sampling rates. However, energy consumption remains a concern, especially for resource-intensive DNNs. Prior mobile speech processing reduced computational complexity by compacting the model or reducing input dimensions via hyperparameter tuning, which reduced accuracy or required more training iterations. This paper proposes gradient descent for optimizing energy-efficient speech recording format (length and sampling rate). The goal is to reduce the input size, which reduces data collection and inference energy. For a backward pass, a masking function with non-zero derivatives (Gaussian, Hann, and Hamming) is used as a windowing function and a lowpass filter. An energy-efficient penalty is introduced to incentivize the reduction of the input size. The proposed masking outperformed baselines by 8.7% in speaker recognition and traumatic brain injury detection using 49% shorter duration, sampled at a lower frequency.

The types of Activities of Daily Living (ADL) a person performs or avoids, and underlying patterns can provide insights into physical and mental health, making passive ADL recognition from smartphone sensor data important. However, as people perform ADLs unequally in real life, ADL datasets collected in the wild can be extremely imbalanced, which presents a challenge to Machine Learning (ML) ADL classification. Prior solutions to mitigating imbalance, such as oversampling and instance weighting, reduce but do not completely eliminate the problem. We instead propose ADL-GAN, which utilizes translation Generative Adversarial Networks (GANs), to synthesize smartphone motion and audio sensor data to improve ADL classification performance. ADL-GANs augment the minority ADL of subject A by translating real samples from either 1) other ADLs where subject A has adequate data in Context-transfer ADL-GAN or 2) other subjects with adequate ADL data in Subject-transfer ADL-GAN . ADL-GANs utilize multi-domain and contrastive loss functions to perform many-to-many translations between ADL classes and subjects, respectively. Subject-transfer ADL-GAN outperformed baselines and improved balanced accuracy (BA) on an in-the-wild ADL dataset by 27.9 %, while context-transfer ADL-GAN performed best on a scripted dataset, improving the BA of baselines by 9.58 %. The augmented samples from ADL-GANs were shown to be more realistic and diverse than conditional GAN.

Goal This study validates a method for characterizing the sounds experienced by tinnitus patients with potential for characterizing a wider range of sounds than currently possible. The approach is based on reverse correlation, an established behavioral method widely used in psychophysics for unconstrained characterization of internal perceptual representations. Methods Ten normal-hearing subjects participated in a reverse correlation experiment, where they assessed the subjective similarity of random auditory stimuli and target tinnitus-like sounds (“buzzing” and “roaring”). Reconstructions of the targets were obtained by regressing subject responses on the stimuli, and were compared for accuracy to the frequency spectra of the targets using Pearson’s r . Results Reconstruction accuracy was significantly higher than chance across subjects: buzzing ( M = 0.53, SD = 0.27): t (9) = 5.766, p < 0.001; roaring ( M = 0.57, SD = 0.30): t (9) = 5.76, p < 0.001. Conclusion Reverse correlation can accurately reconstruct non-tonal tinnitus-like sounds in normal-hearing subjects, indicating its potential for characterizing the sounds experienced by patients with non-tonal tinnitus. Impact Statement Characterization of tinnitus can inform tinnitus treatment by allowing for individualized sound therapies, leading to better outcomes for patients suffering from the cognitive and psychological effects of tinnitus.