Project

PAMBAYESIAN (PAtient Managed decision-support using Bayesian networks)

Goal: PAMBAYESIAN is a 3-year EPSRC funded project to develop a new generation of intelligent medical decision support systems based on Bayesian networks. The project focuses on home-based and wearable real-time monitoring systems for chronic conditions including rheumatoid arthritis, diabetes in pregnancy and atrial fibrillation. The project has the potential to improve the well-being of millions of people.

Updates
0 new
19
Recommendations
0 new
4
Followers
0 new
27
Reads
0 new
869

Project log

Scott McLachlan
added a research item
Information visualisation creates visual representations that more easily convey meaningful patterns and trends hidden within large and otherwise abstract datasets. Despite potential benefits for understanding and communicating health data, information visualisation in medicine is underdeveloped. This is especially true in midwifery, where no qualitative research exists regarding the impact of different graphs on clinicians’ and patients’ understanding. This position paper is part of ongoing work investigating this gap and its potential impact. This work reviews a collection of literature from within the midwifery domain. We found almost two-thirds do not use data visualisation approaches to present knowledge realised from data, and those that did were generally restricted to basic bar charts and line graphs. Without effective information visualisation midwives will continue to be constrained by the challenge of trying to see what datasets.
Norman Elliott Fenton
added a research item
AN UPDATED VERSION OF THIS ARTICLE IS AVAILABLE: http://dx.doi.org/10.13140/RG.2.2.19703.75680 A recent peer reviewed meta-analysis evaluating ivermectin (Bryant et al, 2021) concluded that this antiparasitic drug is a cheap and effective treatment for reducing Covid-19 deaths. These conclusions were in stark contrast to those of a later study (Roman et al, 2021). Although (Roman et al, 2021) applied the same classical statistical approach to meta-analysis, and produced similar results based on a subset of the same trials data used by (Bryant et al), they claimed there was insufficient quality of evidence to support the conclusion Ivermectin was effective. This paper applies a Bayesian approach, to a subset of the same trial data, to test several causal hypotheses linking Covid-19 severity and ivermectin to mortality and produce an alternative analysis to the classical approach. Applying diverse alternative analysis methods which reach the same conclusions should increase overall confidence in the result. We show that there is overwhelming evidence to support a causal link between ivermectin, Covid-19 severity and mortality, and: i) for severe Covid-19 there is a 90.7% probability the risk ratio favours ivermectin; ii) for mild/moderate Covid-19 there is an 84.1% probability the risk ratio favours ivermectin. Also, from the Bayesian meta-analysis for patients with severe Covid-19, the mean probability of death without ivermectin treatment is 22.9%, whilst with the application of ivermectin treatment it is 11.7%. The paper also highlights advantages of using Bayesian methods over classical statistical methods for meta-analysis. creative commons license 2
Scott McLachlan
added a research item
Clinically trained reviewers have undertaken a detailed analysis of a sample of the early deaths reported in VAERS (250 out of the 1644 deaths recorded up to April 2021). The focus is on the extent to which the reports enable us to understand whether the vaccine genuinely caused or contributed to the deaths. Contrary to claims that most of these reports are made by lay-people and are hence clinically unreliable, we identified health service employees as the reporter in at least 67%. The sample contains only people vaccinated early in the programme, and hence is made up primarily of those who are elderly or with significant health conditions. Despite this, there were only 14% of the cases for which a vaccine reaction could be ruled out as a contributing factor in their death.
Norman Elliott Fenton
added a research item
This is about the widely publicised claim that "1 in 3 people with Covid-19 have no symptoms".
Norman Elliott Fenton
added 2 research items
Contradictory conclusions have been made about whether unarmed blacks are more likely to be shot by police than unarmed whites using the same data. The problem is that, by relying only on data of 'police encounters', there is the possibility that genuine bias can be hidden. We provide a causal Bayesian network model to explain this bias, which is called collider bias or Berkson's paradox, and show how the different conclusions arise from the same model and data. We also show that causal Bayesian networks provide the ideal formalism for considering alternative hypotheses and explanations of bias.
There has been great concern in the UK that people from the BAME (Black And Minority Ethnic) community have a far higher risk of dying from Covid19 than those of other ethnicities. However, the overall fatalities data from the Government's ONS (Office of National Statistics) most recent report on deaths by religion shows that Jews (very few of whom are classified as BAME) have a much higher risk than those of religions (Hindu, Sikh, Muslim) with predominantly BAME people. This apparently contradictory result is, according to the ONS statistical analysis, implicitly explained by age as the report claims that, when 'adjusted for age' Muslims have the highest fatality risk. However, the report fails to provide the raw data to support this. There are many factors other than just age that must be incorporated into any analysis of the observed data before making definitive conclusions about risk based on religion/ethnicity. We propose the need for a causal model for this. If we discount unknown genetic factors, then religion and ethnicity have NO impact at all on a person's Covid19 death risk once we know their age, underlying medical conditions, work/living conditions, and extent of social distancing.
Norman Elliott Fenton
added a research item
When analysing Covid-19 death rates by ethnicity in the USA it has been shown that, although the aggregated death rate for whites is higher than for blacks, in each main age subcategory the death rate for blacks is higher than for whites. This apparent statistical anomaly is an example of Simpson's paradox. While the paradox reveals blacks are more at risk than whites, this is only because age is a much more important risk factor than ethnicity. Hence, any public policy with respect to Covid-19 that prioritises ethnicity over age is misguided. We have found similar evidence of Simpson's paradox in UK Covid-19 death rates but the widely publicised conclusions about the risk to the Black and Minority Ethnic (BAME) population are misleading. In particular, the conclusion in the recent UK Office of National Statistics (ONS) report, that blacks are more than four times more likely to die from Covid-19 than whites, may create an unjustified level of fear and anxiety among the BAME community. It is misleading for three reasons: 1) It appears to rely on old 2011 census data about the population proportions rather than on more recent estimates; 2) It is appears to be based on an 'age standardized' measure of risk that is very different from that used by the World Health Organisation (WHO); and 3) It focuses on relative rather than absolute measures of risk. Hence, we believe the ONS conclusions may be misleading from a risk assessment perspective and may serve as a poor guide to public policy.
Norman Elliott Fenton
added a research item
Concerns about the practicality and effectiveness of using Contact Tracing Apps (CTA) to reduce the spread of COVID19 have been well documented and, in the UK, led to the abandonment of the NHS CTA shortly after its release in May 2020. One of the key non-technical obstacles to widespread adoption of CTA has been concerns about privacy. We present a causal probabilistic model (a Bayesian network) that provides the basis for a practical CTA solution that does not compromise privacy. Users of the model can provide as much or little personal information as they wish about relevant risk factors, symptoms, and recent social interactions. The model then provides them feedback about the likelihood of the presence of asymptotic, mild or severe COVID19 (past, present and projected). When the model is embedded in a smartphone app, it can be used to detect new outbreaks in a monitored population and identify outbreak locations as early as possible. For this purpose, the only data needed to be centrally collected is the probability the user has COVID19 and the GPS location.
Norman Elliott Fenton
added a research item
Widely reported statistics on Covid-19 across the globe fail to take account of both the uncertainty of the data and possible explanations for this uncertainty. In this paper we use a Bayesian Network (BN) model to estimate the Covid-19 infection prevalence rate (IPR) and infection fatality rate (IFR) for different countries and regions, where relevant data are available. This combines multiple sources of data in a single model. The results show that Chelsea Mass. USA and Gangelt Germany have relatively higher infection prevalence rates (IPR) than Santa Clara USA, Kobe, Japan and England and Wales. In all cases the infection prevalence is significantly higher than what has been widely reported, with much higher community infection rates in all locations. For Santa Clara and Chelsea, both in the USA, the most likely IFR values are 0.3-0.4%. Kobe, Japan is very unusual in comparison with the others with values an order of magnitude less than the others at, 0.001%. The IFR for Spain is centred around 1%. England and Wales lie between Spain and the USA/German values with an IFR around 0.8%. There remains some uncertainty around these estimates but an IFR greater than 1% looks remote for all regions/countries. We use a Bayesian technique called "virtual evidence" to test the sensitivity of the IFR to two significant sources of uncertainty: survey quality and uncertainty about Covid-19 death counts. In response the adjusted estimates for IFR are most likely to be in the range 0.3%-0.5%.
Norman Elliott Fenton
added a research item
An important recent preprint by Griffith et al highlights how ‘collider bias’ in studies of COVID19 undermines our understanding of the disease risk and severity. This is typically caused by the data being restricted to people who have undergone COVID19 testing, among whom healthcare workers are overrepresented. For example, collider bias caused by smokers being underrepresented in the dataset may (at least partly) explain empirical results that suggest smoking reduces the risk of COVID19. We extend the work of Griffith et al making more explicit use of graphical causal models to interpret observed data. We show that their smoking example can be clarified and improved using Bayesian network models with realistic data and assumptions. We show that there is an even more fundamental problem for risk factors like ‘stress’ which, unlike smoking, is more rather than less prevalent among healthcare workers; in this case, because of a combination of collider bias from the biased dataset and the fact that ‘healthcare worker’ is a confounding variable, it is likely that studies will wrongly conclude that stress reduces rather than increases the risk of COVID19. Indeed, the same large study showing the lower incidence of COVID among smokers also showed a lower incidence for people with hypertension, and in theory it could also lead to bizarre conclusions like “being in close contact with COVID19 people” reduces the risk of COVID19. To avoid such potentially erroneous conclusions, any analysis of observational data must take account of the underlying causal structure including colliders and confounders. If analysts fail to do this explicitly then any conclusions they make about the effect of specific risk factors on COVID19 are likely to be flawed.
Scott McLachlan
added a research item
Many digital solutions mainly involving Bluetooth technology are being proposed for Contact Tracing Apps (CTA) to reduce the spread of COVID-19. Concerns have been raised regarding privacy, consent, uptake required in a given population, and the degree to which use of CTAs can impact individual behaviours. The introduction of a new CTA alone will not contain COVID-19. The best-case scenario for uptake requires between 90 and 95% of the entire population for containment. This does not factor in any loss due to people dropping out or device incompatibility or that only 79% of the population own a smartphone, with less than 40% in the over-65 age group. Hence, the best-case scenario is beyond that which could conceivably be achieved. We propose to build on some of the digital solutions already under development, with the addition of a Bayesian network model that predicts likelihood for infection supplemented by traditional symptom and contact tracing. When combined with freely available COVID-19 testing with results in 24 hours or less, an effective communication strategy and social distancing, this solution can have a very beneficial effect on containing the spread of this pandemic.
Norman Elliott Fenton
added a research item
A person is labelled as having COVID-19 infection either from a positive PCR-based diagnostic test, or by a health professional's assessment of the clinical picture in a process described by some as symptom screening. There is considerable fragility in the resulting data as both of these methods are susceptible to human biases in judgment and decision-making. In this article we show the value of a casual representation that maps out the relations between observed and inferred evidence of contamination, in order to expose what is lacking and what is needed to reduce the uncertainty in classifying an individual as infected with COVID-19.
Norman Elliott Fenton
added an update
Blog post and link to paper saying how we are looking at the COVID-19 data + some recommendations about testing strategies:
 
Norman Elliott Fenton
added a research item
Computational models that need to incorporate domain knowledge for realistic solutions to problems often lead to technologies that get transferred to developing countries. The support for managing the knowledge incorporated into these technologies is important for customisation to suit local conditions. This work investigates this problem through the challenge of incorporating food and nutrition therapy guidelines (FNTG) into ICT-based solution models for the meal planning problem (MPP) for HIV/AIDS patients in developing countries. An experiment is undertaken to demonstrate the limitations of existing approaches and framework is investigated for manageable incorporation of knowledge into solution models. The paper contributes a clear understanding of, and directions for addressing, the problem of support for managing the knowledge incorporated into solutions models to support customisability of technologies. The significance of this contribution is that solutions will allow resulting technologies to be customised for use in different global contexts.
Norman Elliott Fenton
added a research item
Problem: Learning health systems (LHS) are an underexplored concept. How LHS will operate in clinical practice is not well understood. This paper investigates the relationships between LHS, clinical care process specifications (CCPS) and the established levels of medical practice to enable LHS integration into daily healthcare practice. Methods: Concept analysis and thematic analysis were used to develop an LHS characterisation. Pathway theory was used to create a framework by relating LHS, CCPS, health information systems and the levels of medical practice. A case study approach evaluates the framework in an established health informatics project. Results: Five concepts were identified and used to define the LHS learning cycle. A framework was developed with five pathways, each having three levels of practice specificity spanning population to precision medicine. The framework was evaluated through application to case studies not previously understood to be LHS. Discussion Clinicians show limited understanding of LHS, increasing resistance and limiting adoption and integration into care routine. Evaluation of the presented framework demonstrates that its use enables: (1) correct analysis and characterisation of LHS; (2) alignment and integration into the healthcare conceptual setting; (3) identification of the degree and level of patient application; and (4) impact on the overall healthcare system. Conclusion: This paper contributes a theoretical framework for analysis, characterisation and use of LHS. The framework allows clinicians and informaticians to correctly identify, characterise and integrate LHS within their daily routine. The overall contribution improves understanding, practice and evaluation of the LHS application in healthcare.
Norman Elliott Fenton
added 3 research items
Objective: Learning Health Systems (LHS) are one of the major computing advances in healthcare. This paper presents an investigation into the barriers, benefits and facilitating factors for LHS in order to create a basis for successful implementation and adoption of LHS. Method: First, the ITPOSMO-BBF framework was developed based on the established ITPOSMO framework, extending it for analysing barriers, benefits and facilitators. Second, the new framework was applied to LHS. Results: No prior research has systematically analysed barriers and facilitators for LHS. However, by applying ITPOSMO-BBF, we found that LHS shares similar barriers and facilitators with Electronic Health Records (EHR); in particular, most facilitator effort in implementing EHR and LHS goes towards barriers categorised as human factors, even though they were seen to carry fewer benefits. Barriers whose resolution would bring significant benefits in safety, quality and health outcomes, remain. Discussion: LHS envisage constant generation of new clinical knowledge and practice based on the central role of collections of EHR. Once LHS are constructed and operational, they trigger new data streams into the EHR. So LHS and EHR have a symbiotic relationship. The implementation and adoption of EHRs has proved challenging and there are many lessons for LHS arising from these challenges. Conclusion: Successful adoption of LHS should take account of the framework proposed in this paper, especially with respect to its focus on removing barriers that have the most impact.
Getting access to real medical data for research is notoriously difficult. Even when data exist they are usually incomplete and subject to restrictions due to confidentiality and privacy. Synthetic data (SD) are best replacements for real data but must be verifiably realistic. There is little or no investigation into systematically achieving realism in SD. This work investigates this problem, and contributes the ATEN framework, which incorporates three component approaches: (1) THOTH for synthetic data generation (SDG); (2) RA for characterising realism is SD, and (3) HORUS for validating realism in SD. The framework is found promising after its use in generating the realistic synthetic EHR (RS-EHR) for labour and birth. This framework is significant in guaranteeing realism in SDG projects. Future efforts focus on further validation of ATEN in a controlled multi-stream SDG process.
Norman Elliott Fenton
added a research item
Objective Gestational diabetes is the most common metabolic disorder of pregnancy, and it is important that well-written clinical practice guidelines (CPGs) are used to optimise healthcare delivery and improve patient outcomes. The aim of the study was to assess the methodological quality of hospital-based CPGs on the identification and management of gestational diabetes. Design We conducted an assessment of local clinical guidelines in English for gestational diabetes using the Appraisal of Guidelines for Research and Evaluation (AGREE II) to assess and validate methodological quality. Data sources and eligibility criteria We sought a representative selection of local CPGs accessible by the internet. Criteria for inclusion were (1) identified as a guideline, (2) written in English, (3) produced by or for the hospital in a Western country, (4) included diagnostic criteria and recommendations concerning gestational diabetes, (5) grounded on evidence-based medicine and (6) accessible over the internet. No more than two CPGs were selected from any single country. Results Of the 56 CPGs identified, 7 were evaluated in detail by five reviewers using the standard AGREE II instrument. Interrater variance was calculated, with strong agreement observed for those protocols considered by reviewers as the highest and lowest scoring based on the instrument. CPG results for each of the six AGREE II domains are presented categorically using a 5-point Likert scale. Only one CPG scored above average in five or more of the domains. Overall scores ranged from 91.6 (the strongest) to 50 (the weakest). Significant variation existed in the methodological quality of CPGs, even though they followed the guideline of an advising body. Specifically, appropriate identification of the evidence relied on to inform clinical decision making in CPGs was poor, as was evidence of user involvement in the development of the guideline, resource implications, documentation of competing interests of the guideline development group and evidence of external review. Conclusions The limitations described are important considerations for updating current and new CPGs.
Norman Elliott Fenton
added a research item
Introduction Learning health systems (LHS) are one of the major computing advances in health care. However, no prior research has systematically analysed barriers and facilitators for LHS. This paper presents an investigation into the barriers, benefits, and facilitating factors for LHS in order to create a basis for their successful implementation and adoption. Methods First, the ITPOSMO‐BBF framework was developed based on the established ITPOSMO (information, technology, processes, objectives, staffing, management, and other factors) framework, extending it for analysing barriers, benefits, and facilitators. Second, the new framework was applied to LHS. Results We found that LHS shares similar barriers and facilitators with electronic health records (EHR); in particular, most facilitator effort in implementing EHR and LHS goes towards barriers categorised as human factors, even though they were seen to carry fewer benefits. Barriers whose resolution would bring significant benefits in safety, quality, and health outcomes remain. LHS envisage constant generation of new clinical knowledge and practice based on the central role of collections of EHR. Once LHS are constructed and operational, they trigger new data streams into the EHR. So LHS and EHR have a symbiotic relationship. The implementation and adoption of EHRs have proved and continues to prove challenging, and there are many lessons for LHS arising from these challenges. Conclusions Successful adoption of LHS should take account of the framework proposed in this paper, especially with respect to its focus on removing barriers that have the most impact.
Norman Elliott Fenton
added a research item
This short note demonstrates how a causal Bayesian network model-and its extension to an influence diagram-can improve decision-making about treatment options A and B. Using observational study data it appears that the use of option B should be terminated as it is far less cost effective (taking account of cost and quality of life measures). However, the data 'hides' the impact of the drugs for different levels of seriousness of the condition. When this information is added Drug B is found to be much more cost-effective for a specific class of patients. We show how the influence diagram extension (implemented in AgenaRisk) automatically computes the optimal drug option for different classes of patients.
Norman Elliott Fenton
added a research item
Uses a simple example of comparing the efficacy of two alternative drugs for the same condition to illustrate the power of Bayesian networks.
Norman Elliott Fenton
added a research item
Summarises Norman Fenton's research on 'smart data'
Norman Elliott Fenton
added 2 research items
In secondary uses of data, access to real data is problematic due to data being non-existent, incomplete, or avoiding privacy and confidentiality breaches. Synthetic data (SD) are best replacements for real data but must be verifiably realistic. There is little or no investigation into systematically achieving realism in SD. This work investigates this problem, and contributes the ATEN framework, which incorporates three component approaches: (1) THOTH for synthetic data generation (SDG); (2) RA for characterising realism is SD, and (3) HORUS for validating realism in SD. The framework is found promising after its use in generating the realistic synthetic EHR (RS-EHR) for labour and birth. This framework is significant in guaranteeing realism in SDG projects. Future efforts focus on further validation of ATEN in a controlled multi-stream SDG process.
The clever algorithms needed to process big data cannot (and will never) solve most of the critical risk analysis problems that we face. The problems are especially acute where we must assess and manage risk in areas where there is little or no direct historical data to draw upon; where relevant data are difficult to identify or are novel; or causal mechanisms or human intentions remain hidden. Such risks include terrorist attacks, ecological disasters and failures of novel systems and marketplaces. Here, the tendency has been to rely on the intuition of ‘experts’ for decision-making. However, there is an effective and proven alternative: the smart data approach that combines expert judgment (including understanding of underlying causal mechanisms) with relevant data. In particular Bayesian Networks (BNs) provide workable models for combining human and artificial sources of intelligence even when big data approaches to risk assessment are not possible.
Norman Elliott Fenton
added 2 research items
The learning health system (LHS) is one in which progress in science, informatics and care culture converges to continuously create new knowledge as a natural by-product of care processes. While LHS was first described over a decade ago, much of the recent published work that should fall within the domain of LHS fails to claim or be identified as such. This observation was confirmed through a review of papers published at the recent 2017 IEEE International Conference on Health Informatics (ICHI 2017), where no single LHS solution had been so identified. The authors lacked awareness that their work represented an LHS, or of any discrete classification for their work within the LHS domain. We believe this lack of awareness inhibits continued LHS research and prevents formation of a critical mass of researchers within the domain. Efforts to produce a framework and classification structure to enable confident identification of work with the LHS domain are urgently needed to address this pressing research community challenge.
Background There are many proposed benefits of using learning health systems (LHS), including improved patient outcomes. There has been little adoption of LHS in practice due to challenges and barriers that limit adoption of new data-driven technologies in healthcare. We have identified a more fundamental explanation: the majority of developments in LHS are not identified as LHS. The absence of a unifying namespace and framework brings a lack of consistency in how LHS are identified and classified. As a result, the LHS 'community' is fragmented, with groups working on similar systems being not aware of each other's work. This leads to duplication and the lack of a critical mass of researchers necessary to address barriers to adoption. Objective To find a way to support easy identification and classification of research works within the domain of LHS. Method A qualitative meta-narrative study focusing on works that self-identified as LHS was used for two purposes. First, to find existing standard definitions and frameworks using these to create a new unifying framework. Second, seeking whether it was possible to classify those LHS solutions within the new framework. Results The study found that with apparently limited awareness, all current LHS works fall within nine primary archetypes. These findings were used to developa unifying framework for LHS to classify works as LHS, and reduce diversity and fragmentation within the domain. Conclusion Our finding brings clarification where there has been limited aware- ness for LHS among researchers. We believe our framework is simple and may help researchers to classify works in the LHS domain. This framework may enable realisation of the critical mass necessary to bring more substantial collaboration and funding to LHS. Ongoing research will seek to establish the framework’s effect on the LHS domain.
Scott McLachlan
added a research item
The learning health system (LHS) is one in which progress in science, informatics and care culture converges to continuously create new knowledge as a natural by-product of care processes. While LHS was first described over a decade ago, much of the recent published work that should fall within the domain of LHS fails to claim or be identified as such. This observation was confirmed through a review of papers published at the recent 2017 IEEE International Conference on Health Informatics (ICHI 2017), where no single LHS solution had been so identified. The authors lacked awareness that their work represented an LHS, or of any discrete classification for their work within the LHS domain. We believe this lack of awareness inhibits continued LHS research and prevents formation of a critical mass of researchers within the domain. Efforts to produce a framework and classification structure to enable confident identification of work with the LHS domain are urgently needed to address this pressing research community challenge.
Norman Elliott Fenton
added a research item
Bayesian networks help us model and understand the many variables that inform our decision‐making processes. Anthony C. Constantinou and Norman Fenton explain how they work, how they are built and the pitfalls to avoid along the way Bayesian networks help us model and understand the many variables that inform our decision‐making processes. Anthony C. Constantinou and Norman Fenton explain how they work, how they are built and the pitfalls to avoid along the way.
Norman Elliott Fenton
added an update
Norman Elliott Fenton
added a research item
In decision theory models, Expected Value of Partial Perfect Information (EVPPI) is an important analysis technique that is used to identify the value of acquiring further information on individual variables. EVPPI can be used to prioritize the parts of a model that should be improved or identify the parts where acquiring additional data or expert knowledge is most beneficial. Calculating EVPPI of continuous variables is challenging, and several sampling and approximation techniques have been proposed. This paper proposes a novel approach for calculating EVPPI in Hybrid Influence Diagram (HID) models (these are Influence Diagrams (IDs) containing both discrete and continuous nodes). The proposed approach transforms the HID into a Hybrid Bayesian Network (HBN) and makes use of the Dynamic Discretization (DD) and the Junction Tree (JT) algorithms to calculate the EVPPI. This is an approximate solution (no feasible exact solution is possible generally for HIDs) but we demonstrate it accurately calculates the EVPPI values. Moreover, unlike the previously proposed simulation-based EVPPI methods, our approach eliminates the requirement of manually determining the sample size and assessing convergence. Hence, it can be used by decision-makers who do not have deep understanding of programming languages and sampling techniques. We compare our approach to the previously proposed techniques based on two case studies.
Norman Elliott Fenton
added an update
Project website is now live: https://pambayesian.org/
 
Norman Elliott Fenton
added a project goal
PAMBAYESIAN is a 3-year EPSRC funded project to develop a new generation of intelligent medical decision support systems based on Bayesian networks. The project focuses on home-based and wearable real-time monitoring systems for chronic conditions including rheumatoid arthritis, diabetes in pregnancy and atrial fibrillation. The project has the potential to improve the well-being of millions of people.