Avery A. J., Savelyich B. S., Sheikh A., et al. Identifying and establishing consensus on the most important safety features of GP computer systems: e-Delphi study

Primary Care Information Services (PRIMIS), Division of Primary Care, University of Nottingham, The Medical School Queen's Medical Centre, Nottingham, UK.
The Journal of Innovations in Health Informatics 02/2005; 13(1):3-12. DOI: 10.14236/jhi.v13i1.575
Source: PubMed


Our objective was to identify and establish consensus on the most important safety features of GP computer systems, with a particular emphasis on medicines management. We used a two-round electronic Delphi survey, completed by a 21-member multidisciplinary expert panel, all from the UK. The main outcome measure was percentage agreement of the panel members on the importance of the presence of a number of different safety features (presented as clinical statements) on GP computer systems. We found 90% or greater agreement on the importance of 32 (58%) statements. These statements, indicating issues considered to be of considerable importance (rated as important or very important), related to: computerised alerts; the need to avoid spurious alerts; making it difficult to override critical alerts; having audit trails of such overrides; support for safe repeat prescribing; effective computer-user interface; importance of call and recall management; and the need to be able to run safety reports. The high level of agreement among the expert panel members indicates clear themes and priorities that need to be addressed in any further improvement of safety features in primary care computing systems.

Download full-text


Available from: Aziz Sheikh,
  • Source
    • "The toolkit of safety measures identified in this study to date addresses all of these issues. Many are underpinned by the need for accurate and reliable health informatics [25] including electronic health records in general practice and across the primary-secondary interface, good coordination between primary and secondary care and effective multi-professional teams. The use of computerized provider order entry, medication reconciliation and clinicians working with clinical pharmacists to reduce adverse drug events have, for example, been emphasised as patient safety strategies that could be adopted in the US now [21]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background The majority of patient contacts occur in general practice but general practice patient safety has been poorly described and under-researched to date compared to hospital settings. Our objective was to produce a set of patient safety tools and indicators that can be used in general practices in any healthcare setting and develop a ‘toolkit’ of feasible patient safety measures for general practices in England. Methods A RAND/UCLA Appropriateness Method exercise was conducted with a panel of international experts in general practice patient safety. Statements were developed from an extensive systematic literature review of patient safety in general practice. We used standard RAND/UCLA Appropriateness Method rating methods to identify necessary items for assessing patient safety in general practice, framed in terms of the Structure-Process-Outcome taxonomy. Items were included in the toolkit if they received an overall panel median score of ≥7 with agreement (no more than two panel members rating the statement outside a 3-point distribution around the median). Results Of 205 identified statements, the panel rated 101 as necessary for assessing the safety of general practices. Of these 101 statements, 73 covered structures or organisational issues, 22 addressed processes and 6 focused on outcomes. Conclusions We developed and tested tools that can lead to interventions to improve safety outcomes in general practice. This paper reports the first attempt to systematically develop a patient safety toolkit for general practice, which has the potential to improve safety, cost effectiveness and patient experience, in any healthcare system.
    BMC Family Practice 06/2014; 15(1):110. DOI:10.1186/1471-2296-15-110 · 1.67 Impact Factor
  • Source
    • "Previous alert fatigue studies have focused on clinical decision support systems (CDS) and computerized provider order entry (CPOE) systems which aim to improve efficiency and quality of care. These studies have investigated how CDS and CPOE systems alert providers to potential adverse medication events and interactions [3,4], the frequency and factors associated with overriding CPOE alerts [2,5], recommended design approaches for mitigating adverse drug events and alert fatigue [6,7], and unintended consequences of these systems in contributing to information and mis-information overload, role confusion, excessive errors and alert fatigue [8]. In addition to the clinical system messages, HCPs receive public health alerts and advisories, clinical guidelines and updates, and training notifications from professional associations, agencies and organizations [9] and through a system of national, state and local public health communication channels [10]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Health care providers play a significant role in large scale health emergency planning, detection, response, recovery and communication with the public. The effectiveness of health care providers in emergency preparedness and response roles depends, in part, on public health agencies communicating information in a way that maximizes the likelihood that the message is delivered, received, deemed credible and, when appropriate, acted on. However, during an emergency, health care providers can become inundated with alerts and advisories through numerous national, state, local and professional communication channels. We conducted an alert fatigue study as a sub-study of a larger randomized controlled trial which aimed to identify the most effective methods of communicating public health messages between public health agencies and providers. We report an analysis of the effects of public health message volume/frequency on recall of specific message content and effect of rate of message communications on health care provider alert fatigue. Health care providers enrolled in the larger study (n=528) were randomized to receive public health messages via email, fax, short message service (SMS or cell phone text messaging) or to a control group that did not receive messages. For 12 months, study messages based on real events of public health significance were sent quarterly with follow-up telephone interviews regarding message receipt and topic recall conducted 5-10 days after the message delivery date. During a pandemic when numerous messages are sent, alert fatigue may impact ability to recall whether a specific message has been received due to the "noise" created by the higher number of messages. To determine the impact of "noise" when study messages were sent, we compared health care provider recall of the study message topic to the number of local public health messages sent to health care providers. We calculated the mean number of messages that each provider received from local public health during the time period around each study message and provider recall of study message content. We found that recall rates were inversely proportional to the mean number of messages received per week: Every increase of one local public health message per week resulted in a statistically significant 41.2% decrease (p < 0.01), 95% CI [0.39, .87] in the odds of recalling the content of the study message. To our knowledge, this is the first study to document the effects of alert fatigue on health care providers' recall of information. Our results suggest that information delivered too frequently and/or repetitively through numerous communication channels may have a negative effect on the ability of health care providers to effectively recall emergency information. Keeping health care providers and other first-line responders informed during an emergency is critical. Better coordination between organizations disseminating alerts, advisories and other messages may improve the ability of health care providers to recall public health emergency messages, potentially impacting effective response to public health emergency messages.
    BMC Health Services Research 08/2013; 13(1):295. DOI:10.1186/1472-6963-13-295 · 1.71 Impact Factor
  • Source
    • "In order to establish professional consensus on the most relevant criteria of anaphylaxis management plans, we used an adaptation of the Delphi technique. This is a method of reaching consensus on a particular research question and has been widely used in healthcare research.22,23 It involves circulating a set of statements, assumptions, solutions or options to be anonymously scored by participants, thereby minimizing the risk of actual or perceived peer pressure influencing participants' responses. "
    [Show abstract] [Hide abstract]
    ABSTRACT: There is no international consensus on the components of anaphylaxis management plans and responsibility for their design and delivery is contested. We set out to establish consensus among relevant specialist and generalist clinicians on this issue to inform future randomized controlled trials. A two-round electronic Delphi study completed by a 25-person, multidisciplinary expert panel. Participants scored the importance of a range of statements on anaphylaxis management, identified from a systematic review of the literature, on a five-point scale ranging from 'very important' to 'irrelevant'. Consensus was defined a priori as being achieved if 80% or more of panel members rated a statement as 'important' or 'very important' after Round 2. Primary and secondary care and academic settings in the UK and Ireland. Twenty-five medical, nursing and allied health professionals. Consensus on the key components of anaphylaxis management plans. The response rate was 84% (n = 21) for Round 1 and 96% (n = 24) for Round 2. The key components of emergency care on which consensus was achieved included: awareness of trigger factors (100%); recognition and emergency management of reactions of different severity (100%); and clear information on adrenaline (epinephrine) use (100%). Consensus on longer-term management issues included: clear written guidelines on anaphylaxis management (96%); annual review of plans (87%); and plans that were tailored to individual needs (82%). This national consensus-building exercise generated widespread agreement that emergency plans need to be simple, clear and generic, making them easy to implement in a crisis. In contrast, long-term plans need to be negotiated between patient/carers and professionals, and tailored to individual needs. The effectiveness of this expert-agreed long-term plan now needs to be evaluated rigorously.
    10/2010; 1(5):42. DOI:10.1258/shorts.2010.010060
Show more