Murat Sariyar’s research while affiliated with Bern University of Applied Sciences and other places

GNU Health, an open-source clinical information system, offers a comprehensive solution for managing health records, hospital information, and laboratory data. Despite its robust functionality and cost-effective nature, GNU Health remains underutilized in the European healthcare context. This paper explores the potential benefits of implementing GNU Health in European healthcare systems, emphasizing its capacity for customization, integration, and scalability. We also examine the barriers to its widespread adoption, including regulatory challenges, interoperability issues, and resistance to change from established proprietary systems. Through one case study and expert interviews, we provide insights into why these obstacles can hardly be overcome.

Healthcare systems worldwide face escalating costs and demographic changes, necessitating effective evaluation tools to understand their underlying challenges. Switzerland’s high-quality yet costly healthcare system underscores the need for robust assessment methods. Existing international rankings often lack transparency and comparability, highlighting the value of structured frameworks like the Health System Performance Assessment (HSPA) by the World Health Organization (WHO). This framework evaluates healthcare systems across multiple dimensions including governance, resource generation, financing, and service delivery. This paper aims to integrate Swiss healthcare indicators from the Swiss Health Observatory (Obsan) into the HSPA framework, addressing the central research question: How can these indicators be mapped to the HSPA framework, and what insights does this integration provide? Our methodology includes selecting and categorizing Obsan indicators, manually mapping them to HSPA sub-functions, and validating these mappings using word embeddings and cosine similarity. An R Shiny application was developed for interactive visualization. Results demonstrate accurate indicator assignment, enabling intuitive visualization and enhancing data structuring.

Natural Language Processing (NLP) has shown promise in fields like radiology for converting unstructured into structured data, but acquiring suitable datasets poses several challenges, including privacy concerns. Specifically, we aim to utilize Large Language Models (LLMs) to extract medical information from dialogues between ambulance staff and patients to populate emergency protocol forms. However, we currently lack dialogues with known content that can serve as a gold standard for an evaluation. We designed a pipeline using the quantized LLM “Zephyr-7b-beta” for initial dialogue generation, followed by refinement and translation using OpenAI’s GPT-4 Turbo. The MIMIC-IV database provided relevant medical data. The evaluation involved accuracy assessment via Retrieval-Augmented Generation (RAG) and sentiment analysis using multilingual models. Initial results showed a high accuracy of 94% with “Zephyr-7b-beta,” slightly decreasing to 87% after refinement with GPT-4 Turbo. Sentiment analysis indicated a qualitative shift towards more positive sentiment post-refinement. These findings highlight the potential and challenges of using LLMs for generating synthetic medical dialogues, informing future NLP system development in healthcare.

Pharmacogenetics (PGx) explores the influence of genetic variability on drug efficacy and tolerability. Synthetic Data Generation (SDG) has emerged as a promising alternative to the labor-intensive process of collecting real-world PGx data, which is required for high-qualitative prediction models. This study investigates the performance of two Generative Adversarial Network (GAN) models, CTGAN and CTAB-GAN+, in generating synthetic PGx data. The benchmarking is based on utility metrics (Hellinger distance and Random Forest accuracy) and ϵ-identifiability. Results demonstrate that synthetic data generated by CTAB-GAN+ can surpass the original dataset in terms of utility. For instance, CTAB-GAN+ achieves higher Random Forest accuracy compared to the original data, indicating better predictive performance. These improvements suggest that synthetic data not only capture the essential patterns of the original data but also enhance model generalization and prediction capabilities, providing a more robust training ground for machine learning models. Consequently, SDG offers a promising solution to address data scarcity and imbalance in pharmacogenetic research.

BACKGROUND Recent advancements in Generative Adversarial Networks (GANs) and sophisticated language models have significantly impacted the synthesis and augmentation of medical data. These technologies facilitate the creation of high-quality, realistic datasets essential for enhancing machine learning (ML) applications in healthcare. GANs, through their adversarial framework, and Large Language Models (LLM), with their advanced Natural Language Processing (NLP) capabilities, offer innovative solutions for generating synthetic data that mirrors real-world medical information. This is particularly valuable in scenarios constrained by data privacy and availability. However, challenges persist in accurately capturing complex associations within medical datasets. Misrepresentation of these can lead to synthetic data that poorly reflects the real-world data variability and relationships, impacting model performance in clinical applications. OBJECTIVE This study aims to evaluate the effectiveness of Synthetic Data Generation (SDG) methods in replicating the correlation structures of real medical data and assess their performance in downstream tasks using Random Forests (RF). We compare two SDG approaches, CTGAN and the Tabula-Framework, with a focus on their ability to maintain accurate data correlations and their implications for model accuracy and variable importance. METHODS We assess synthetic data generation methods using real-world and simulated datasets. Simulated data involve ten Gaussian variables with different correlation structures, generated via Cholesky decomposition to create binary target variables. Real-world datasets include Body Performance (BP) with 13,393 samples for fitness classification, Wisconsin Breast Cancer (BC) with 569 samples for tumor diagnosis, and Diabetes (DB) with 768 samples for diabetes prediction. Data quality is evaluated through the Euclidean Distance (L² norm) between original and synthetic correlation matrices and through downstream classification tasks using Random Forests (RF) and computing F₁ scores. Variable importance (VIMP) measures, i.e., Gini impurity and permutation-based methods, are employed for assessing the mechanism behind the RF results. For each model and epoch combination, 100 samples are drawn, conducting outlier analysis to ensure robust performance evaluation. RESULTS In smaller datasets (samples = 1000), synthetic data utility remains stable under both high and moderate correlations, with moderate correlations occasionally enhancing utility. However, as correlation complexity increases, particularly with stronger correlations across multiple features, models struggle, reflected by higher L^2 values for the correlation matrix distance. CTGAN improves with more training epochs but requires significant tuning to handle complex patterns, while LLMs show promise with larger datasets despite their computational demands. Real-world data mirrors these findings, with LLMs outperforming in scenarios with intricate dependency structures. VIMP score analysis underscores the importance of aligning model complexity with data correlation structures. CONCLUSIONS Our findings emphasize that correlation complexity, not strength, is the key challenge in synthetic data generation. While CTGAN and LLMs show varying success based on dataset size and complexity, careful tuning and model selection are essential. Further research should focus on optimizing training protocols, exploring simpler neural network architectures, and expanding simulations to better handle nonlinear and high-order interactions in complex datasets.

Synthetic tabular health data plays a crucial role in healthcare research, addressing privacy regulations and the scarcity of publicly available datasets. This is essential for diagnostic and treatment advancements. Among the most promising models are transformer-based Large Language Models (LLMs) and Generative Adversarial Networks (GANs). In this paper, we compare LLM models of the Pythia LLM Scaling Suite with varying model sizes ranging from 14M to 1B, against a reference GAN model (CTGAN). The generated synthetic data are used to train random forest estimators for classification tasks to make predictions on the real-world data. Our findings indicate that as the number of parameters increases, LLM models outperform the reference GAN model. Even the smallest 14M parameter models perform comparably to GANs. Moreover, we observe a positive correlation between the size of the training dataset and model performance. We discuss implications, challenges, and considerations for the real-world usage of LLM models for synthetic tabular data generation.

Biomedical decision support systems play a crucial role in modern healthcare by assisting clinicians in making informed decisions. Events, such as physiological changes or drug reactions, are integral components of these systems, influencing patient outcomes and treatment strategies. However, effectively modeling events within these systems presents significant challenges due to the complexity and dynamic nature of medical data. Especially the differentiation between events and processes as well as the nature of events is often unclear. This paper explores approaches to modeling events in biomedical decision support systems, considering factors such as ontology-based representation. By addressing these challenges, we strive to provide the means for enhancing the functionality and interpretability of biomedical decision support systems concerning events.

Coding according to the International Classification of Diseases (ICD)-10 and its clinical modifications (CM) is inherently complex and expensive. Natural Language Processing (NLP) assists by simplifying the analysis of unstructured data from electronic health records, thereby facilitating diagnosis coding. This study investigates the suitability of transformer models for ICD-10 classification, considering both encoder and encoder-decoder architectures. The analysis is performed on clinical discharge summaries from the Medical Information Mart for Intensive Care (MIMIC)-IV dataset, which contains an extensive collection of electronic health records. Pre-trained models such as BioBERT, ClinicalBERT, ClinicalLongformer, and ClinicalBigBird are adapted for the coding task, incorporating specific preprocessing techniques to enhance performance. The findings indicate that increasing context length improves accuracy, and that the difference in accuracy between encoder and encoder-decoder models is negligible.

BACKGROUND : In healthcare settings, especially in high-pressure environments like Emergency situations, the ability to document and communicate patient information rapidly and accurately is crucial. Traditional methods for manual documentation are often time-consuming and prone to errors, which can adversely affect patient outcomes. To address these challenges, there is growing interest in integrating advanced technologies, especially Large Language Models (LLMs), into medical communication systems. However, deploying LLMs in clinical environments presents unique challenges, including the need to ensure the accuracy of medical content and to mitigate the risk of generating irrelevant or misleading information. OBJECTIVE This paper aims to address these challenges by developing a Natural Language Processing (NLP) pipeline for the extraction of text from German rescue services treatment dialogues. The objectives are twofold: (1) to generate realistic, medically relevant dialogues where the ground truth is known, and (2) to accurately extract essential information from these dialogues to populate emergency protocols. METHODS This study utilizes the MIMIC-IV-ED dataset, a de-identified, publicly available resource, to generate synthetic dialogue data for emergency department scenarios. By selecting and anonymizing data from 100 patients, we created a baseline for generating realistic dialogues and evaluating an NLP pipeline. We applied the Post Randomization Method (PRAM) for non-mechanical data perturbation, ensuring patient privacy and data utility. Dialogue generation was conducted in two stages: initial generation using the "Zephyr-7b-beta" model, followed by refinement and translation into German using GPT-4 Turbo. A Retrieval-Augmented Generation (RAG) approach was developed for extracting relevant information from these dialogues, involving chunking, embedding, and dynamic prompt templates. The model's performance was evaluated through manual review and sentiment analysis, ensuring that the generated dialogues maintained clinical relevance and emotional accuracy. RESULTS The data generation pipeline produced 100 dialogues, with initial English dialogues averaging 2,000 tokens and German dialogues 4,000 tokens. Manual evaluation identified certain redundancies and formal language in the German dialogues. Sentiment analysis revealed a reduction in negative sentiment from 67% to 59% and an increase in positive sentiment from 27% to 38%, which may negatively impact text extraction, as positive sentiments may not align well with identifying critical topics such as suicidal thoughts. The RAG-based extraction system achieved high precision and recall in both nominal and numerical features in the initial dialogues, with F1-scores ranging from 86.21% to 100%. However, performance declined in the refined dialogues, with notable drops in precision, particularly for "Diagnosis" (60.82%) and "Pain Score" (57.61%). CONCLUSIONS The results of the study underscore the system's robust capabilities in processing structured data efficiently, demonstrating its strength in managing well-defined, quantitative information. However, the findings also reveal limitations in the system’s ability to handle nuanced clinical language, particularly when it comes to non-English and non-Chinese languages.