Celia Brown

• Doctor of Education
• Professor (Full) at University of Warwick

Objectives Standardisation of medical examinations involves minimising assessor stereotyping and bias for a fair process. This study aimed to determine whether being a non-white candidate affected scoring by simulated patients, compared with a white candidate, at three different performance grades in the same history-taking station. Design Single-...

Purpose: To compare student performance, examiner perceptions and cost of GPT-assisted (generative pretrained transformer-assisted) clinical and professional skills assessment (CPSAs) items against items created using standard methods. Methods: We conducted a prospective, controlled, double-blinded comparison of CPSA items developed using GPT-as...

Purpose: Delivering fair and reliable summative assessments in medical education assumes examiner decision making is devoid of bias. We investigated whether candidate racial appearances influenced examiner ratings in undergraduate clinical exams. Methods: We used an internet-based design. Examiners watched a randomised set of six videos of three...

Background Community Health Workers (CHWs) play an essential role in linking communities to facility-based healthcare. However, CHW programmes have often been hampered by low levels of staff motivation, and new tools aimed at improving staff motivation and work environment are needed. One such intervention is the “Learning from Excellence” (LfE) pr...

Purpose Previous models identify knowledge and attitudes that influence prescribing behaviour. The present study focuses on antibiotic prescribing for urinary tract infections (UTIs) to describe levels of health care professionals' knowledge and attitude factors in this area and how those levels are assessed. Methods A systematic search was conduc...

Objectives To better characterise the Awarding Gap (AG) between black, Asian and other minority ethnic (BAME) and white students in UK undergraduate medical education by examining how it affects eight minority ethnicity subgroups (Bangladeshi, black, Chinese, Indian, Pakistani, mixed, other Asian background and other ethnic background) and whether...

Objectives The present study aimed to identify factors related to knowledge and attitudes that influence primary care clinicians' decision‐making when diagnosing, treating, and managing suspected urinary tract infections (UTIs) in women. Understanding the factors that influence clinicians' decision‐making is important for maximising health outcomes...

Objectives: To identify knowledge and attitude factors influencing primary care clinician decision-making in diagnosing, managing, and treating urinary tract infections. Design: A qualitative think-aloud study. Methods: Semi-structured qualitative interviews were conducted with primary care clinicians in England over Microsoft Teams. Interviews wer...

Background: Standard setting for clinical examinations typically uses the borderline regression method to set the pass mark. An assumption made in using this method is that there are equal intervals between global ratings (GR) (e.g. Fail, Borderline Pass, Clear Pass, Good and Excellent). However, this assumption has never been tested in the medica...

Background We investigated whether question format and access to the correct answers affect the pass mark set by standard-setters on written examinations. Methods Trained educators used the Angoff method to standard set two 50-item tests with identical vignettes, one in a single best answer question (SBAQ) format (with five answer options) and the...

Most undergraduate written examinations use multiple-choice questions, such as single best answer questions (SBAQs) to assess medical knowledge. In recent years, a strong evidence base has emerged for the use of very short answer questions (VSAQs). VSAQs have been shown to be an acceptable, reliable, discriminatory, and cost-effective assessment to...

Background: We have previously shown that clinical examiners' scoring is not negatively impacted when a candidate has a tattoo, unnatural hair colour, or a regional accent. We investigated whether these physical attributes in exam candidates impact patient scoring. Methods: Simulated/real patients were randomly assigned to watch five videos of s...

Background Urinary tract infections (UTIs) are the second most common condition (after upper respiratory tract infections) for which adults receive antibiotics, and this prevalence may contribute to antibiotic resistance. Knowledge and attitudes have been identified as potential determinants of antibiotic prescribing behaviour among healthcare prof...

Background Few studies exist on the tools for assessing quality-of-care of community health worker (CHW) who provide comprehensive care, and for available tools, evidence on the utility is scanty. We aimed to assess the utility components of a previously-reported quality-of-care assessment tool developed for summative assessment in South Africa. M...

Background Weinvestigated whether question formatand access to the correct answers affect the pass mark set by standard-setters on written examinations. Methods Trained educatorsused the Angoff method to standard set two 50-item tests with identical vignettes, one in a single best answer question (SBAQ) format (with five answer options) and the oth...

Background: Student performance in examinations reflects on both teaching and student learning. Very short answer questions require students to provide a self-generated response to a question of between one and five words, which removes the cueing effects of single best answer format examinations while still enabling efficient machine marking. The...

Background The University Clinical Aptitude Test (UCAT) is an admissions assessment used by a consortium of universities across the UK, Australia, and New Zealand, to aid the selection of applicants to medical and dental degree programmes. The UCAT aims to measure the mental aptitude and professional behaviours required to become successful doctors...

Background Community health worker (CHW) programmes are a valuable component of primary care in resource-poor settings. The evidence supporting their effectiveness generally shows improvements in disease-specific outcomes relative to the absence of a CHW programme. In this study, we evaluated expanding an existing HIV and tuberculosis (TB) disease-...

Purpose: Disciplinary action imposed on physicians indicates their fitness to practice medicine is impaired and patient safety is potentially at risk. This national retrospective cohort study sought to examine whether there was an association between academic attainment or performance on a situational judgment test (SJT) in medical school and the...

Background Single-best answer questions (SBAQs) are common but are susceptible to cueing. Very short answer questions (VSAQs) could be an alternative, and we sought to determine if students’ cognitive processes varied across question types and whether students with different performance levels used different methods for answering questions. Method...

Background Interventions using positive psychology (PP), which build on positive qualities of healthcare personnel and institutions, could potentially enhance organisational performance in healthcare. The aim of this systematic review was to identify if PP interventions have an impact on organisational performance of healthcare personnel, and if so...

Objectives This paper reports a mixed methods evaluation of a new pathway to improve clinical outcomes for older people with fractures treated at a hospital Trust in the West Midlands, UK. The paper focuses specifically on the context surrounding the translation of the new pathway into practice and the way that external and internal factors influen...

Background and Aims The allocation of medical school graduates to Foundation Schools (post‐qualification training, organized at regional level) in the United Kingdom uses a ranking process that takes into account educational performance at medical school and performance on a situational judgment test (SJT). We aimed to compare the performance of Un...

Background/Aims The ‘Learning from Excellence’ (LfE) programme aims to provide a means to identify, appreciate, study and learn from episodes of excellence in frontline healthcare. The aim of this study was to explore the impact of LfE on organisational performance in NHS trusts in the United Kingdom (UK), how this impact is achieved and which cont...

Recruitment to General Practice (GP) is currently low in many countries. Here we focus on two binary choices for junior doctors: first, whether to apply to GP; second, whether to accept a GP training place if offered. Previous attitudinal studies have indicated factors claimed to affect recruitment. The current study goes further by quantifying the...

Background Although most health care is high quality, many patients and members of staff can recall episodes of a lack of empathy, respect or effective communication from health-care staff. In extreme form, this contributes to high-profile organisational failures. Reflective learning is a universally promoted technique for stimulating insight, cons...

Background Patient and staff experiences provide important insights into care quality, but health systems have difficulty using these data to improve care. Little attention has been paid to understanding how patient experience feedback can act as a prompt to reflection in practice in the clinical setting. Objective We aimed to identify the ways in...

This paper outlines the common statistical methods used to analyze four types of observational study: ecological studies, cross-sectional studies, case–control studies and cohort studies. Three statistical methods are considered in detail: correlation coefficients, t-tests for the difference between two means from independent samples and odds ratio...

Background: Choosing who should be recruited as a community health worker (CHW) is an important task, for their future performance partly depends on their ability to learn the required knowledge and skills, and their personal attributes. Developing a fair and effective selection process for CHWs is a challenging task, and reports of attempts to do...

Public health and service delivery programmes, interventions and policies (collectively, ‘programmes’) are typically developed and implemented for the primary purpose of effecting change rather than generating knowledge. Nonetheless, evaluations of these programmes may produce valuable learning that helps determine effectiveness and costs as well a...

Objective To develop a tool for use by non-clinical fieldworkers for assessing the quality of care delivered by community health workers providing comprehensive care in households in low- and middle-income countries. Design We determined the content of the tool using multiple sources of information, including interactions with district managers, n...

Objectives The study aimed to compare candidate performance between traditional best-of-five single-best-answer (SBA) questions and very-short-answer (VSA) questions, in which candidates must generate their own answers of between one and five words. The primary objective was to determine if the mean positive cue rate for SBAs exceeded the null hypo...

Introduction Patient and staff experiences are strongly influenced by attitudes and behaviours, and provide important insights into care quality. Patient and staff feedback could be used more effectively to enhance behaviours and improve care through systematic integration with techniques for reflective learning. We aim to develop a reflective lear...

Background: Most patients with CKD are managed in the community. Whether nurse-led CKD management programs improve outcomes in patients with CKD in primary care is unclear. Methods: To assess the effect of such a program on the rate of renal function decline in patients with CKD (stages 3-5) in primary care in the United Kingdom, we conducted a...

Objectives To assess the prevalence of mild-to-moderate distress in patients with end-stage renal disease (ESRD) and determine the association between distress and patient characteristics. Design Cross-sectional survey using emotion thermometer and distress thermometer problem list. Setting Renal units in four hospital Trusts in the West Midlands...

Recruitment and selection are critical components of human resource management. They influence both the quantity and quality of the healthcare workforce. In this article, we use two different examples of primary care workers, General Practitioners in the UK and Community Health Workers in low- and middle- income countries, to illustrate how recruit...

Background: Low and middle income countries (LMICs) face severe resource limitations but the highest burden of disease. There is a growing evidence base on effective and cost-effective interventions for these diseases. However, questions remain about the most cost-effective method of delivery for these interventions. We aimed to review the scope,...

The use of donor human breast milk instead of formula reduces the risk of necrotising enterocolitis in preterm infants when their mother's own milk is insufficient. Use of donor milk is limited by the cost of establishing a milk bank and a lack of donors, but the optimal rationing of limited donor milk is unclear. This paper uses an economic model...

Background: Community Health Workers (CHWs) provide basic health screening and advice to members of their own communities. Although CHWs are trained, no CHW programmes have used a formal method to identify the level of achievement on post-training assessments that distinguishes “safe” from “unsafe”. Objectives: The aim of this study was to use Ebel...

Background: Community Health Workers (CHWs) provide basic health screening and advice to members of their own communities. Although CHWs are trained, no CHW programmes have used a formal method to identify the level of achievement on post-training assessments that distinguishes "safe" from "unsafe". Objectives: The aim of this study was to use Ebe...

Background Ensuring that selection processes for Community Health Workers (CHWs) are effective is important due to the scale and scope of modern CHW programmes. However they are relatively understudied. While community involvement in selection should never be eliminated entirely, there are other complementary methods that could be used to help iden...

Background Community Health Workers (CHWs) have a crucial role in improving health in their communities and their role is being expanded in many parts of the world. However, the effectiveness of CHWs is limited by poor training and the education of CHWs has received little scientific attention. Methods Our study was carried out in two districts of...

CHW workshop costs. Costs of the workshops in an Excel file. (XLSX)

Knowledge questionnaires. List of HAST and WSRHR assessment questions for knowledge test. (DOCX)

Confidence questionnaires. List of HAST and WSRHR rating questions to measure confidence in advising clients. (DOCX)

Satisfaction questionnaires. List of HAST and WSRHR rating questions to measure satisfaction. (DOCX)

Background: Since 2012, The World Health Organization and UNICEF have advocated for community health workers (CHWs) to be trained in Integrated Community Case Management (iCCM) of common childhood illnesses, such as pneumonia. Despite the effectiveness of iCCM, CHWs face many barriers to accessing training. This pilot study compares traditional tr...

Introduction This protocol concerns the implementation and evaluation of an intervention designed to realign the existing cadre of community health workers (CHWs) in Neno district, Malawi to better support the care needs of the clients they serve. The proposed intervention is a ‘Household Model’ where CHWs will be reassigned to households, rather t...

Background There is currently a shortage of qualified GPs in the UK and not all of the training posts available each year are filled. Changing the way in which GP trainees are selected could help increase the training post fill rate and the number of new entrants to the GP Register. The aim of this study was to model the impact of changing the sele...

Objectives: We aimed to create a library of logic models for interventions to reduce diagnostic error. This library can be used by those developing, implementing, or evaluating an intervention to improve patient care, to understand what needs to happen, and in what order, if the intervention is to be effective. Methods: To create the library, we...

Introduction Community health worker (CHW) programmes have low costs per person served and are central to achieving universal healthcare. However, their total cost is high and the target of one million CHWs for sub-Saharan Africa by 2015 was not met. We consider the affordability of rural CHW programmes by estimating total programme costs relative...

Background: Necrotizing enterocolitis (NEC) is a costly gastrointestinal disorder that mainly affects preterm and low-birth-weight infants and can lead to considerable morbidity and mortality. Mother's own milk is protective against NEC but is not always available. In such cases, donor human milk has also been shown to be protective (although to a...

Purpose: The aim of this study was to identify the dose-response relationship between measures of training load (TL) and changes in aerobic fitness in academy rugby union players. Methods: Training data from ten academy rugby union players was collected during a six-week in-season period. Participants completed a lactate threshold (LT) test whic...

Objectives Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common ‘one from five’ single‐best‐answer (multiple‐choice) items used in graduation‐level applied knowledge examinations and explores potential reasons for any differences. Methods A repeated cross‐secti...

Aim(s): Newly graduated doctors write a large proportion of prescriptions in UK hospitals but recent studies have shown that they frequently make prescribing errors. The Prescribing Safety Assessment (PSA) has been developed as an assessment of competence in relation to prescribing and supervising the use of medicines. This report describes the de...

Objectives: Given the absence of a common passing standard for students at UK medical schools, this paper compares independently set standards for common 'one from five' single-best-answer (multiple-choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. Methods: A repeated cross-...

\textbf{Objectives}$: Given the absence of a common passing standard for students at UK medical schools, this paper compares independently -set standards for common “1 from 5” single-best answer (multiple choice) items used in graduation-level applied knowledge examinations and explores potential reasons for any differences. $\textbf{Methods}$: A r...

Training load measures that demonstrate a strong dose-response relationship will provide coaches with a greater understanding of how players may respond to a given training session (Manzi et al., 2009). Therefore, the aim of this study is to identify the dose-response relationship between training load measures and changes in aerobic fitness in aca...

Background As a result of difficulties related to their illness, diagnosis and treatment, patients with end-stage renal disease experience significant emotional and psychological problems, which untreated can have considerable negative impact on their health and wellbeing. Despite evidence that patients desire improved support, management of their...

Whilst systematic reviews, meta-analyses and other forms of synthesis are considered amongst the most valuable forms of research evidence, their limited impact on educational policy and practice has been criticised. In this article, we analyse why systematic reviews do not benefit users of evidence more consistently and suggest how review teams can...

Whilst systematic reviews, meta-analyses and other forms of synthesis are considered amongst the most valuable forms of research evidence, their limited impact on educational policy and practice has been criticised. In this article, we analyse why systematic reviews do not benefit users of evidence more consistently and suggest how review teams can...

Randomized controlled trials (RCTs) are generally acknowledged by methodologists to be the gold-standard method in evaluation research. This chapter focuses on the use of RCTs in evaluating social policy interventions. It highlights the key characteristics of well-designed RCTs and demonstrates how biases and limitations in the method can be avoide...

Women remain under-represented in leadership positions in both clinical medicine and medical education, despite a rapid increase in the proportion of women in the medical profession. This chapter explores potential reasons for this under-representation and how it can be ameliorated, drawing on a range of international literatures, theories and prac...

Introduction: Widening participation in Medicine is a key policy priority as it helps promote a diverse and representative workforce and improves patient care. The selection process employed can influence the socio-economic composition of the student cohort and this study therefore evaluated whether Multiple Mini Interview (MMI) performance was in...

The economic evaluation of medical products and services is increasingly prioritised by healthcare decision makers and plays a key role in informing funding allocation decisions. It is well known that there are a number of methodological difficulties in the health technology assessment of medical devices, particularly in the provision of efficacy e...

The headroom method offers medical device developers a simple way to integrate health economics into the decision of whether or not to develop a medical device. By estimating the maximum reimbursable price (MRP) for a new device idea, and comparing this reimbursement opportunity with a developer’s expected costs, the method offers a way to ensure d...

Given evidence of differences between UK medical schools' curricula and assessments, and their graduates' performance in Royal college examinations, this retrospective cohort study analyses the effect of medical school on the incidence of General Medical Council fitness to practise sanctions.

This article examines the fairness, effectiveness (validity and reliability) and acceptability of the 2009 selection processes for 10 hospital specialties, using data from a nationwide evaluation of 33 selection processes across the UK.

Background: Tertiary reviews (review of reviews) in medical education tend to focus on doctors, continuing education and professional practice. This paper provides a tertiary review of all areas of medical education at all levels. An in-depth focus on prescribing is included. Methods and results: A systematic search using the keywords ‘medical educ...

A new 'Cohen' approach to standard setting was recently described where the pass mark is calculated as 60% of the score of the student at the 95th percentile, after correcting for guessing. This article considers how two potential criticisms of the Cohen method can be addressed and proposes a modified version, with the assumptions tested using loca...

Effective use of the laryngeal mask airway (LMA) requires learning proper insertion technique in normal patients undergoing routine surgical procedures. However, there is a move towards simulation training for learning practical clinical skills, such as LMA placement. The evidence linking different amounts of mannequin simulation training to the un...

To describe the application of the stepped wedge cluster randomized controlled trial (CRCT) design. Systematic review. We searched Medline, Embase, PsycINFO, HMIC, CINAHL, Cochrane Library, Web of Knowledge, and Current Controlled Trials Register for articles published up to January 2010. Stepped wedge CRCTs from all fields of research were include...

To explore the prior beliefs regarding the effectiveness of percutaneous vesico-amniotic shunting in congenital lower urinary tract obstruction among different clinical specialist groups involved in caring for this condition. Questionnaire survey of experts in the field based on principles of Bayesian elicitation of prior beliefs. The sample was 59...

The effect of many cost effective policy and service interventions cannot be detected at the level of the patient. This new framework could help improve the design (especially choice of primary end point) and interpretation of evaluative studies

At the concept stage, many uncertainties surround the commercial viability of a new medical device. These include the ultimate functionality of the device, the cost of producing it and whether, and at what price, it can be sold to a health-care provider (HCP). Simple assessments of value can be made by estimating such unknowns, but the levels of un...

Background: In September 2007, the UK government announced a £57.5m programme of `deep cleaning' for every NHS hospital in England. The programme was met with some scepticism and this paper provides an outline economic evaluation of the programme. Methods: We use information on costs of the programme, the opportunity cost of closing wards for clean...

This is the first of a four-part series of articles examining the epistemology of patient safety research. Parts 2 and 3 will describe different study designs and methods of measuring outcomes in the evaluation of patient safety interventions, before Part 4 suggests that "one size does not fit all". Part 1 sets the scene by defining patient safety...

This is the second in a four-part series of articles detailing the epistemology of patient safety research. This article concentrates on issues of study design. It first considers the range of designs that may be used in the evaluation of patient safety interventions, highlighting the circumstances in which each is appropriate. The paper then provi...

This article builds on the previous two articles in this series, which focused on an evaluation framework and study designs for patient safety research. The current article focuses on what to measure as evidence of safety and how these measurements can be undertaken. It considers four different end points, highlighting their methodological advantag...

This is the final article in the series on the epistemology of patient safety research, and considers the selection of study design and end points during the planning of an evaluation. The key message of this series is that "one size does not fit all": the nature of the evaluation will depend on logistical and pragmatic constraints, a priori assess...

Tests of cognitive ability are probably the best method at present The selection of the doctors of tomorrow is a subject of constant interest because it raises questions about ensuring equity, predicting human behaviour, and defining the characteristics of a good doctor. In the United Kingdom, it costs about £200 000 (€260 000; $400 000) to train...

Celia Brown and Rrichard Lilford describe the recommendations of an MRC sponsored network to improve the quality of research into patient safety.