Lawrence O. Hall’s research while affiliated with University of South Florida and other places

Traditional deep learning-based methods for classifying cellular features in microscopy images require time- and labor-intensive processes for training models. Among the current limitations are major time commitments from domain experts for accurate ground truth preparation; and the need for a large amount of input image data. We previously proposed a solution that overcomes these challenges using OpenAI's GPT-4(V) model on a pilot dataset (Iba-1 immuno-stained tissue sections from 11 mouse brains). Results on the pilot dataset were equivalent in accuracy and with a substantial improvement in throughput efficiency compared to the baseline using a traditional Convolutional Neural Net (CNN)-based approach. The present study builds upon this framework using a second unique and substantially larger dataset of microscopy images. Our current approach uses a newer and faster model, GPT-4o, along with improved prompts. It was evaluated on a microscopy image dataset captured at low (10x) magnification from cresyl-violet-stained sections through the cerebellum of a total of 18 mouse brains (9 Lurcher mice, 9 wild-type controls). We used our approach to classify these images either as a control group or Lurcher mutant. Using 6 mice in the prompt set the results were correct classification for 11 out of the 12 mice (92%) with 96% higher efficiency, reduced image requirements, and lower demands on time and effort of domain experts compared to the baseline method (snapshot ensemble of CNN models). These results confirm that our approach is effective across multiple datasets from different brain regions and magnifications, with minimal overhead.

Stereology methods are the current best practice for state-of-the-art quantification of stained biostructures in tissue sections. Over the past six decades, stereological applications to bioscience research have added knowledge about neurostructural changes in normal aging, age-related neurodegeneration (Alzheimer’s, Parkinson’s, ALS), oncology, toxic exposures, learning and memory behavior, a range of mental illnesses, and drug discovery to name a few. In current practice, however, all computer-assisted stereology systems require time-consuming manual data collection by well-trained experts, which is prone to inter-rater error due to the variation in training, experience, motivation, and fatigue. Recent work shows these limitations can be overcome by novel developments in artificial intelligence (AI)-based deep learning (DL) as shown in other domains for solving similar problems. In this chapter, we review basic stereology concepts for avoiding bias, tissue processing, and tissue sampling for quantifying total cell number in tissue sections and show how DL approaches can automatically collect stereology data with comparable accuracy but superior precision (reproducibility) and throughput efficiency as manual stereology methods. Finally, we identify the major challenges to automation using DL approaches and review novel applications to the quantification of stained biostructures using unbiased methods.

Advances in high-resolution microscopy and cell staining techniques allow for quantifying well-stained cells and other microstructures in anatomically defined regions of interest (ROIs) in ex vivo brains from humans and animal models of human disease. The current best practice is the optical disector, an unbiased stereology method for counting individual cells or other tissue deposits while manually focusing a thin focal plane through a known z-axis volume (optical disector). By eliminating all known sources of methodological bias, the optical disector method ensures that the ex vivo count of the total number of neurons (∑NCells) converges on true in vivo values. A current limitation with all computer-assisted methods is the time, labor, and effort for counting (clicking) on a hundred or more cells as the counting process repeats at 100–200 x-y locations through each ROI. Faster data collection methods include semiquantitative approaches for sampling and counting 2D neuron profiles (∑NProf), though these methods can include unknown and unknowable amounts of systematic error (bias). Here we compare accuracy, reproducibility, and efficiency of counting total number of NeuN-immunostained neurons using the manual optical disector (gold standard) versus three techniques for counts of NeuN profiles. All counts of ∑NCells and ∑NProf were done to the same sampling intensity in the same high-power (100×) stacks of z-axis images (i.e., disector stacks) collected in a systematic-random manner through the entire neocortex (NCTX) in six mouse brains. The findings showed no statistical differences between ∑NCells by the gold standard and ∑NProf counts using three methods (fully manual, semiautomatic, fully automatic). Reproducibility (inter-rater error) by two similarly trained data collectors ranged from 0% for fully automatic counts of ∑NProf to ~5% for ∑NCells counts by gold standard with average data collection times between <1 min per case to ~12 min per case, respectively. Notably, all three methods for counting ∑NProf required a significant amount of unsupervised time, i.e., from 30 to 90 min per case, for preprocessing image stacks prior to data collection. Estimates of ∑NCells by the manual optical disector required no preprocessing. In summary, the manual optical disector remains the best practice due to the avoidance of all known sources of methodological bias, low inter-rater error, and moderate-to-high throughput. These results provide a baseline for comparisons with other methods for quantifying stained cells in tissue sections, including novel deep learning approaches for automatic counts of well-stained cells and other microstructures in the brain.

The progress of incorporating deep learning in the field of medical image interpretation has been greatly hindered due to the tremendous cost and time associated with generating ground truth for supervised machine learning, alongside concerns about the inconsistent quality of images acquired. Active learning offers a potential solution to these problems of expanding dataset ground truth by algorithmically choosing the most informative samples for ground truth labeling. Still, this effort incurs the costs of human labeling, which needs minimization. Furthermore, automatic labeling approaches employing active learning often exhibit overfitting tendencies while selecting samples closely aligned with the training set distribution and excluding out-of-distribution samples, which could potentially improve the model’s effectiveness. We propose that the majority of out-of-distribution instances can be attributed to inconsistent cross images. Since the FDA approved the first whole-slide image system for medical diagnosis in 2017, whole-slide images have provided enriched critical information to advance the field of automated histopathology. Here, we exemplify the benefits of a novel deep learning strategy that utilizes high-resolution whole-slide microscopic images. We quantitatively assess and visually highlight the inconsistencies within the whole-slide image dataset employed in this study. Accordingly, we introduce a deep learning-based preprocessing algorithm designed to normalize unknown samples to the training set distribution, effectively mitigating the overfitting issue. Consequently, our approach significantly increases the amount of automatic region-of-interest ground truth labeling on high-resolution whole-slide images using active deep learning. We accept 92% of the automatic labels generated for our unlabeled data cohort, expanding the labeled dataset by 845%. Additionally, we demonstrate expert time savings of 96% relative to manual expert ground-truth labeling.

Microglial cells mediate diverse homeostatic, inflammatory, and immune processes during normal development and in response to cytotoxic challenges. During these functional activities, microglial cells undergo distinct numerical and morphological changes in different tissue volumes in both rodent and human brains. However, it remains unclear how these cytostructural changes in microglia correlate with region-specific neurochemical functions. To better understand these relationships, neuroscientists need accurate, reproducible, and efficient methods for quantifying microglial cell number and morphologies in histological sections. To address this deficit, we developed a novel deep learning (DL)-based classification, stereology approach that links the appearance of Iba1 immunostained microglial cells at low magnification (20×) with the total number of cells in the same brain region based on unbiased stereology counts as ground truth. Once DL models are trained, total microglial cell numbers in specific regions of interest can be estimated and treatment groups predicted in a high-throughput manner (<1 min) using only low-power images from test cases, without the need for time and labor-intensive stereology counts or morphology ratings in test cases. Results for this DL-based automatic stereology approach on two datasets (total 39 mouse brains) showed >90% accuracy, 100% percent repeatability (Test-Retest) and 60× greater efficiency than manual stereology (<1 min vs. ∼ 60 min) using the same tissue sections. Ongoing and future work includes use of this DL-based approach to establish clear neurodegeneration profiles in age-related human neurological diseases and related animal models.

Large language models (LLMs) have shown progress and promise in diverse applications ranging from the medical field to chat bots. Developing LLMs requires a large corpus of data and significant computation resources to achieve efficient learning. Foundation models (in particular LLMs) serve as the basis for fine-tuning on a new corpus of data. Since the original foundation models contain a very large number of parameters, fine-tuning them can be quite challenging. Development of the low-rank adaption technique (LoRA) for fine-tuning, and the quantized version of LoRA, also known as QLoRA, allows for fine-tuning of LLMs on a new smaller corpus of data. This paper focuses on the repeatability of fine-tuning four LLMs using QLoRA. We have fine-tuned them for seven trials each under the same hardware and software settings. We also validated our study for the repeatability (stability) issue by fine-tuning LLMs on two public datasets. For each trial, each LLM was fine-tuned on a subset of the dataset and tested on a holdout test set. Fine-tuning and inference were done on a single GPU. Our study shows that fine-tuning of LLMs with the QLoRA method is not repeatable (not stable), such that different fine-tuned runs result in different performance on the holdout test set.

The primary benefit of stereology methods is quantification of well-stained biological objects in tissue sections with the ability to adjust sampling intensity to achieve desired levels of precision. The advent of hand-crafted algorithms and artificial intelligence-based deep learning (DL) provides an opportunity for more standardized collection of stereology data with enhanced efficiency and higher reproducibility compared to state-of-the-art manual stereology. We contrasted and compared the performance of four manual, semi-automatic, and fully automatic approaches for generating data for total number of Neu-N immunostained neurons in neocortex (NCTX) in the mouse brain. The gold standard for these studies was manual counts using the state-of-the-art optical fractionator method on 3-D reconstructed serial z-axis image stacks through a known tissue volume (disector stacks). To allow for direct methodological comparisons on the same images, disector stacks were automatically converted into extended depth of field (EDF) images in which all neurons in the disector stack were imaged at each cell’s maximal plane of focus. Total number of Neu-N neurons on the same EDF images were counted by a fully automatic hand-crafted method [automatic segmentation algorithm (ASA)] and a semi-automatic method [ASA counts manually corrected for false positives and negatives]. All comparison counts were done using unbiased frames and counting rules with total counts of NeuN-immunostained neurons by the optical fractionator method. The results were comparable across methods with wide variations in throughput efficiency and inter-rater agreement. These results are discussed with respect to applications to experimental studies of brain aging, neuroinflammation and neurodegenerative disease.