No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Modeling implementation context in telemedicine: work domain analysis of Endoscopic Retrograde Cholangiopancreatography (Preprint)
Abstract
Background
A telemedicine service enabling remote surgical consultation had shown promising results. When the service was to be scaled up, it was unclear how contextual variations among different clinical sites could affect the clinical outcomes and implementation of the service. It is generally recognized that contextual factors and work system complexities affect the implementation and outcomes of telemedicine. However, it is methodologically challenging to account for context in complex health care settings. We conducted a work domain analysis (WDA), an engineering method for modeling and analyzing complex work environments, to investigate and represent contextual influences when a telemedicine service was to be scaled up to multiple hospitals.
Objective
We wanted to systematically characterize the implementation contexts at the clinics participating in the scale-up process. Conducting a WDA would allow us to identify, in a systematic manner, the functional constraints that shape clinical work at the implementation sites and set the sites apart. The findings could then be valuable for informed implementation and assessment of the telemedicine service.
Methods
We conducted observations and semistructured interviews with a variety of stakeholders. Thematic analysis was guided by concepts derived from the WDA framework. We identified objects, functions, priorities, and values that shape clinical procedures. An iterative “discovery and modeling” approach allowed us to first focus on one clinic and then readjust the scope as our understanding of the work systems deepened.
Results
We characterized three sets of constraints (ie, facets) in the domain: the treatment facet, administrative facet (providing resources for procedures), and development facet (training, quality improvement, and research). The constraints included medical equipment affecting treatment options; administrative processes affecting access to staff and facilities; values and priorities affecting assessments during endoscopic retrograde cholangiopancreatography; and resources for conducting the procedure.
Conclusions
The surgical work system is embedded in multiple sets of constraints that can be modeled as facets of the system. We found variations between the implementation sites that might interact negatively with the telemedicine service. However, there may be enough motivation and resources to overcome these initial disruptions given that values and priorities are shared across the sites. Contrasting the development facets at different sites highlighted the differences in resources for training and research. In some cases, this could indicate a risk that organizational demands for efficiency and effectiveness might be prioritized over the long-term outcomes provided by the telemedicine service, or a reduced willingness or ability to accept a service that is not yet fully developed or adapted. WDA proved effective in representing and analyzing these complex clinical contexts in the face of technological change. The models serve as examples of how to analyze and represent a complex sociotechnical context during telemedicine design, implementation, and assessment.
Background:
Home care is facing increasing demand due to an aging population. Several challenges have been identified in the provision of home care, such as the need for support and tailoring support to individual needs. Goal-oriented interventions, such as reablement, may provide a solution to some of these challenges. The reablement approach targets adaptation to disease and relearning of everyday life skills and has been found to improve health-related quality of life while reducing service use.
Objective:
The objective of this study is to characterize home care system variables (elements) and their relationships (connections) relevant to home care staff workload, home care user needs and satisfaction, and the reablement approach. This is to examine the effects of improvement and interventions, such as the person-centered reablement approach, on the delivery of home care services, workload, work-related stress, home care user experience, and other organizational factors. The main focus was on Swedish home care and tax-funded universal welfare systems.
Methods:
The study used a mixed methods approach where a causal loop diagram was developed grounded in participatory methods with academic health care science research experts in nursing, occupational therapy, aging, and the reablement approach. The approach was supplemented with theoretical models and the scientific literature. The developed model was verified by the same group of experts and empirical evidence. Finally, the model was analyzed qualitatively and through simulation methods.
Results:
The final causal loop diagram included elements and connections across the categories: stress, home care staff, home care user, organization, social support network of the home care user, and societal level. The model was able to qualitatively describe observed intervention outcomes from the literature. The analysis suggested elements to target for improvement and the potential impact of relevant studied interventions. For example, the elements "workload" and "distress" were important determinants of home care staff health, provision, and quality of care.
Conclusions:
The developed model may be of value for informing hypothesis formulation, study design, and discourse within the context of improvement in home care. Further work will include a broader group of stakeholders to reduce the risk of bias. Translation into a quantitative model will be explored.
Background
Teleguidance, a promising telemedicine service for intraoperative surgical consultation, was planned to scale up at a major academic hospital in partnership with 5 other hospitals. If the service was adopted and used over time, it was expected to provide educational benefits and improve clinical outcomes during endoscopic retrograde cholangiopancreatography (ERCP), which is a technically advanced procedure for biliary and pancreatic disease. However, it is known that seemingly successful innovations can play out differently in new settings, which might cause variability in clinical outcomes. In addition, few telemedicine services survive long enough to deliver system-level outcomes, the causes of which are not well understood.
Objective
We were interested in factors related to usability and user experience of the telemedicine service, which might affect adoption. Therefore, we investigated perceptions and responses to the use and anticipated use of a system. Technology acceptance, a construct referring to how users perceive a technology’s usefulness, is commonly considered to indicate whether a new technology will actually be used in a real-life setting. Satisfaction measures were used to investigate whether user expectations and needs have been met through the use of technology. In this study, we asked surgeons to rate the perceived usefulness of teleguidance, and their satisfaction with the telemedicine service in direct conjunction with real-time use during clinical procedures.
Methods
We designed domain-specific measures for perceived usefulness and satisfaction, based on performance and outcome measures for the clinical procedure. Surgeons were asked to rate their user experience with the telemedicine service in direct conjunction with real-time use during clinical procedures.
Results
In total, 142 remote intraoperative consultations were conducted during ERCP procedures at 5 hospitals. The demand for teleguidance was more pronounced in cases with higher complexity. Operating surgeons rated teleguidance to have contributed to performance and outcomes to a moderate or large extent in 111 of 140 (79.3%) cases. Specific examples were that teleguidance was rated as having contributed to intervention success and avoiding a repeated ERCP in 23 cases, avoiding 3 PTC, and 11 referrals, and in 11 cases, combinations of these outcomes. Preprocedure beliefs about the usefulness of teleguidance were generally lower than postprocedure satisfaction ratings. The usefulness of teleguidance was mainly experienced through practical advice from the consulting specialist (119/140, 85%) and support with assessment and decision-making (122/140, 87%).
Conclusions
Users’ satisfaction with teleguidance surpassed their initial expectations, mainly through contribution to nontechnical aspects of performance, and through help with general assessment. Teleguidance shows the potential to improve performance and outcomes during ERCP. However, it takes hands-on experience for practitioners to understand how the new telemedicine service contributes to performance and outcomes.
Many promising telemedicine innovations fail to be accepted and used over time, and there are longstanding questions about how to best evaluate telemedicine services and other health information technologies. In response to these challenges, there is a growing interest in how to take the sociotechnical complexity of health care into account during design, implementation, and evaluation. This paper discusses the methodological implications of this complexity and how the sociotechnical context holds the key to understanding the effects and outcomes of telemedicine. Examples from a work domain analysis of a surgical setting, where a telemedicine service for remote surgical consultation was to be introduced, are used to show how abstracted functional modeling can provide a structured and rigorous means to analyze and represent the implementation context in complex health care settings.
Background
Telemedicine innovations are rarely adopted into routine health care, the reasons for which are not well understood. Teleguidance, a promising service for remote surgical guidance during endoscopic retrograde cholangiopancreatography (ERCP) was due to be scaled up, but there were concerns that user attitudes might influence adoption.
Objective
Our objective was to gain a deeper understanding of ERCP practitioners’ attitudes toward teleguidance. These findings could inform the implementation process and future evaluations.
Methods
We conducted semistructured interviews with ERCP staff about challenges during work and beliefs about teleguidance. Theoretical constructs from the technology acceptance model (TAM) guided the thematic analysis. Our findings became input to a 16-item questionnaire, investigating surgeons’ beliefs about teleguidance’s contribution to performance and factors that might interact with implementation.
Results
Results from 20 interviews with ERCP staff from 5 hospitals were used to adapt a TAM questionnaire, exchanging the standard “Ease of Use” items for “Compatibility and Implementation Climate.” In total, 23 ERCP specialists from 15 ERCP clinics responded to the questionnaire: 9 novices (<500 ERCP procedures) and 14 experts (>500 ERCP procedures). The average agreement ratings for usefulness items were 64% (~9/14) among experts and 75% (~7/9) among novices. The average agreement ratings for compatibility items were somewhat lower (experts 64% [~9/14], novices 69% [~6/9]). The averages have been calculated from the sum of several items and therefore, they only approximate the actual values. While 11 of the 14 experts (79%) and 8 of the 9 novices (89%) agreed that teleguidance could improve overall quality and patient safety during ERCP procedures, only 8 of the 14 experts (57%) and 6 of the 9 novices (67%) agreed that teleguidance would not create new patient safety risks. Only 5 of the 14 experts (36%) and 3 of the 9 novices (33%) were convinced that video and image transmission would function well. Similarly, only 6 of the 14 experts (43%) and 6 of the 9 novices (67%) agreed that administration would work smoothly. There were no statistically significant differences between the experts and novices on any of the 16 items (P<.05).
Conclusions
Both novices and experts in ERCP procedures had concerns that teleguidance might disrupt existing work practices. However, novices were generally more positive toward teleguidance than experts, especially with regard to the possibility of developing technical skills and work practices. While newly trained specialists were the main target for teleguidance, the experts were also intended users. As experts are more likely to be key decision makers, their attitudes may have a greater relative impact on adoption. We present suggestions to address these concerns. We conclude that using the TAM as a conceptual framework can support user-centered inquiry into telemedicine design and implementation by connecting qualitative findings to well-known analytical themes.
Background
The need for better methods for evaluation in health research has been widely recognised. The ‘complexity turn’ has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research—currently denigrated as poor evidence—is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear.
Main body
Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports.
Conclusion
Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.
Introduction
Improving the quality of healthcare has proven to be a challenging task despite longstanding efforts. Approaches to improvements that consider the strong influence of local context as well as stakeholders’ differing views on the situation are warranted. Soft systems methodology (SSM) includes contextual and multi-perspectival features. However, the way SSM has been applied and the outcomes of using SSM to stimulate productive change in healthcare have not been sufficiently investigated.
Aim
This scoping review aimed to examine and map the use and outcomes of SSM in healthcare settings.
Method
The review was based on Arksey and O’Malley’s framework. We searched six academic databases to January 2019 for peer-reviewed journal articles in English. We also reviewed reference lists of included citations. Articles were included if they were empirical studies focused on the application of SSM in a healthcare setting. Two reviewers conducted the abstract review and one reviewer conducted the full-text review and extracted data on study characteristics, ways of applying SSM and the outcomes of SSM initiatives. Study quality was assessed using Hawker’s Quality Assessment Tool.
Result
A total of 49 studies were included in the final review. SSM had been used in a range of healthcare settings and for a variety of problem situations. The results revealed an inconsistent use of SSM including departing from Checkland’s original vision, applying different tools and involving stakeholders idiosyncratically. The quality of included studies varied and reporting of how SSM had been applied was sometimes inadequate. SSM had most often been used to understand a problem situation and to suggest potential improvements to the situation but to a lesser extent to implement and evaluate these improvements.
Conclusion
SSM is flexible and applicable to a range of problem situations in healthcare settings. However, better reporting of how SSM has been applied as well as evaluation of different types of outcomes, including implementation and intervention outcomes, is needed in order to appreciate more fully the utility and contribution of SSM in healthcare.
Background:
Despite making great strides in improving the treatment of diseases, the minimization of unintended harm by medication therapy continues to be a major hurdle facing the health care system. Medication error and prescription of potentially inappropriate medications (PIMs) represent a prevalent source of harm to patients and are associated with increased rates of adverse events, hospitalizations, and increased health care costs. Attempts to improve medication management systems in primary care have had mixed results. Implementation of new interventions is difficult because of complex contextual factors within the health care system. Abstraction hierarchy (AH), the first step in cognitive work analysis (CWA), is used by human factors practitioners to describe complex sociotechnical systems. Although initially intended for the nuclear power domain and interface design, AH has been used successfully to aid the redesign of numerous health care systems such as the design of decision support tools, mobile patient monitoring apps, and a telephone triage system.
Objective:
This paper aims to refine our understanding of the primary care office in relation to a patient's medication through the development of an AH. Emphasis was placed on the elements related to medication safety to provide guidance for the design of a safer medication management system in primary care.
Methods:
The AH development was guided by the methodology used by seminal CWA literature. It was initially developed by 2 authors and later fine-tuned by an expert panel of clinicians, social scientists, and a human factors engineer. It was subsequently refined until an agreement was reached. A means-ends analysis was performed and described for the nodes of interest. The model represents the primary care office space through functional purposes, values and priorities, function-related purposes, object-related processes, and physical objects.
Results:
This model depicts the medication management system at various levels of abstraction. The resulting components must be balanced and coordinated to provide medical treatment with limited health care resources. Understanding the physical and informational constraints on activities that occur in a primary care office depicted in the AH defines areas in which medication safety can be improved.
Conclusions:
Numerous means-ends relationships were identified and analyzed. These can be further evaluated depending on the specific needs of the user. Recommendations for optimizing a medication management system in a primary care facility were made. Individual practices can use AH for clinical redesign to improve prescribing and deprescribing practices.
Objective
To gain a deeper understanding of the information requirements of clinicians conducting neonatal resuscitation in the first 10 min after birth.Background
During the resuscitation of a newborn infant in the first minutes after birth, clinicians must monitor crucial physiological adjustments that are relatively unobservable, unpredictable, and highly variable. Clinicians’ access to information regarding the physiological status of the infant is also crucial to determining which interventions are most appropriate. To design displays to support clinicians during newborn resuscitation, we must first carefully consider the information requirements.Methods
We conducted a work domain analysis (WDA) for the neonatal transition in the first 10 min after birth. We split the work domain into two ‘subdomains’; the physiology of the neonatal transition, and the clinical resources supporting the neonatal transition. A WDA can reveal information requirements that are not yet supported by resources.ResultsThe physiological WDA acted as a conceptual tool to model the exact processes and functions that clinicians must monitor and potentially support during the neonatal transition. Importantly, the clinical resources WDA revealed several capabilities and limitations of the physical objects in the work domain—ultimately revealing which physiological functions currently have no existing sensor to provide clinicians with information regarding their status.Conclusion
We propose two potential approaches to improving the clinician’s information environment: (1) developing new sensors for the information we lack, and (2) employing principles of ecological interface design to present currently available information to the clinician in a more effective way.
Background and study aims The aims of this study was to document the clinical and training relevance of endoscopic retrograde cholangiopancreaticography (ERCP) teleguidance (as a clinical model for applied telemedicine) with health economic modeling methodologies.
Methods Probabilities and consequences of complications after ERCP performed by either a novice-trainee or supported through teleguidance (TM) by an expert formed the basis of the health economic model.
Results The main clinical and economic outcomes originated from the base case scenario representing a low-volume center. In the cohort the patient age was 62 years, 58 % were females, the expert was doing ≥ 250 ERCPs per year and 50 for the novice-trainee. The expert knowledge transferred was set to 50 % and the average complexity grade to 1.98. Given a willingness to pay threshold of 56,180 USD/ quality-adjusted life years (QALY), the probability of cost-effectiveness of TM assistance was 98.9 %. The probability of a QALY gain for patients having an ERCP, to which was added TM, was 91.6 %. Adding TM saved on an average 111.2 USD (95 % CI 959 to 1021 SEK) per patient, and remained cost-effective basically insensitive to the level of willingness to pay.
Conclusion Teleguidance during an ERCP procedure has the potential to be the prefered option in many low- to medium-volume hospitals. The main mechanisms behind these effects are positive impact on several adverse patient outcomes, QALY increase, and decreased costs. TM should be considered for integration into future teaching curriculums in advanced upper gastrointestinal endoscopy.
Keynote Conference of the Cognitive Science Society
Full treatment is chapter 12 of Joint Cognitive Systems: Patterns in Cognitive Systems Engineering. see
https://www.researchgate.net/publication/284173735_Laws_that_Govern_JCSs_at_Work
Background
The relevance of context in implementation science is reflected in the numerous theories, frameworks, models and taxonomies that have been proposed to analyse determinants of implementation (in this paper referred to as determinant frameworks). This scoping review aimed to investigate and map how determinant frameworks used in implementation science were developed, what terms are used for contextual determinants for implementation, how the context is conceptualized, and which context dimensions that can be discerned.
Methods
A scoping review was conducted. MEDLINE and EMBASE were searched from inception to October 2017, and supplemented with implementation science text books and known published overviews. Publications in English that described a determinant framework (theory, model, taxonomy or checklist), of which context was one determinant, were eligible. Screening and inclusion were done in duplicate. Extracted data were analysed to address the study aims. A qualitative content analysis with an inductive approach was carried out concerning the development and core context dimensions of the frameworks. The review is reported according to the PRISMA guidelines.
Results
The database searches yielded a total of 1113 publications, of which 67 were considered potentially relevant based on the predetermined eligibility criteria, and retrieved in full text. Seventeen unique determinant frameworks were identified and included. Most were developed based on the literature and/or the developers’ implementation experiences. Six of the frameworks explicitly referred to “context”, but only four frameworks provided a specific definition of the concept. Instead, context was defined indirectly by description of various categories and sub-categories that together made up the context. Twelve context dimensions were identified, pertaining to different aggregation levels. The most widely addressed context dimensions were organizational support, financial resources, social relations and support, and leadership.
Conclusions
The findings suggest variation with regard to how the frameworks were developed and considerable inconsistency in terms used for contextual determinants, how context is conceptualized, and which contextual determinants are accounted for in frameworks used in implementation science. Common context dimensions were identified, which can facilitate research that incorporates a theory of context, i.e. assumptions about how different dimensions may influence each other and affect implementation outcomes. A thoughtful application of the concept and a more consistent terminology would enhance transparency, simplify communication among researchers, and facilitate comparison across studies.
Electronic supplementary material
The online version of this article (10.1186/s12913-019-4015-3) contains supplementary material, which is available to authorized users.
Technologies are often viewed as the route to better, safer and more efficient care, but technology projects rarely deliver all the anticipated benefits. This is usually because they are too complex - and because the complexity is suboptimally handled. This article summarises a new framework to improve the success of technology projects: the nonadoption, abandonment, scale-up, spread and sustainability (NASSS) framework. The framework is based on a narrative systematic review and empirical work, and addresses the different domains in technology projects and how different aspects of complexity may be handled in each of them.
Complexity is much talked about but sub-optimally studied in health services research. Although the significance of the complex system as an analytic lens is increasingly recognised, many researchers are still using methods that assume a closed system in which predictive studies in general, and controlled experiments in particular, are possible and preferred. We argue that in open systems characterised by dynamically changing inter-relationships and tensions, conventional research designs predicated on linearity and predictability must be augmented by the study of how we can best deal with uncertainty, unpredictability and emergent causality. Accordingly, the study of complexity in health services and systems requires new standards of research quality, namely (for example) rich theorising, generative learning, and pragmatic adaptation to changing contexts. This framing of complexity-informed health services research provides a backdrop for a new collection of empirical studies. Each of the initial five papers in this collection illustrates, in different ways, the value of theoretically grounded, methodologically pluralistic, flexible and adaptive study designs. We propose an agenda for future research and invite researchers to contribute to this on-going series.
Background:
Implementation science has a core aim - to get evidence into practice. Early in the evidence-based medicine movement, this task was construed in linear terms, wherein the knowledge pipeline moved from evidence created in the laboratory through to clinical trials and, finally, via new tests, drugs, equipment, or procedures, into clinical practice. We now know that this straight-line thinking was naïve at best, and little more than an idealization, with multiple fractures appearing in the pipeline.
Discussion:
The knowledge pipeline derives from a mechanistic and linear approach to science, which, while delivering huge advances in medicine over the last two centuries, is limited in its application to complex social systems such as healthcare. Instead, complexity science, a theoretical approach to understanding interconnections among agents and how they give rise to emergent, dynamic, systems-level behaviors, represents an increasingly useful conceptual framework for change. Herein, we discuss what implementation science can learn from complexity science, and tease out some of the properties of healthcare systems that enable or constrain the goals we have for better, more effective, more evidence-based care. Two Australian examples, one largely top-down, predicated on applying new standards across the country, and the other largely bottom-up, adopting medical emergency teams in over 200 hospitals, provide empirical support for a complexity-informed approach to implementation. The key lessons are that change can be stimulated in many ways, but a triggering mechanism is needed, such as legislation or widespread stakeholder agreement; that feedback loops are crucial to continue change momentum; that extended sweeps of time are involved, typically much longer than believed at the outset; and that taking a systems-informed, complexity approach, having regard for existing networks and socio-technical characteristics, is beneficial.
Conclusion:
Construing healthcare as a complex adaptive system implies that getting evidence into routine practice through a step-by-step model is not feasible. Complexity science forces us to consider the dynamic properties of systems and the varying characteristics that are deeply enmeshed in social practices, whilst indicating that multiple forces, variables, and influences must be factored into any change process, and that unpredictability and uncertainty are normal properties of multi-part, intricate systems.
Background
There is much interest in virtual consultations using video technology. Randomized controlled trials have shown video consultations to be acceptable, safe, and effective in selected conditions and circumstances. However, this model has rarely been mainstreamed and sustained in real-world settings.
Objective
The study sought to (1) define good practice and inform implementation of video outpatient consultations and (2) generate transferable knowledge about challenges to scaling up and routinizing this service model.
Methods
A multilevel, mixed-method study of Skype video consultations (micro level) was embedded in an organizational case study (meso level), taking account of national context and wider influences (macro level). The study followed the introduction of video outpatient consultations in three clinical services (diabetes, diabetes antenatal, and cancer surgery) in a National Health Service trust (covering three hospitals) in London, United Kingdom. Data sources included 36 national-level stakeholders (exploratory and semistructured interviews), longitudinal organizational ethnography (300 hours of observations; 24 staff interviews), 30 videotaped remote consultations, 17 audiotaped face-to-face consultations, and national and local documents. Qualitative data, analyzed using sociotechnical change theories, addressed staff and patient experience and organizational and system drivers. Quantitative data, analyzed via descriptive statistics, included uptake of video consultations by staff and patients and microcategorization of different kinds of talk (using the Roter interaction analysis system).
Results
When clinical, technical, and practical preconditions were met, video consultations appeared safe and were popular with some patients and staff. Compared with face-to-face consultations for similar conditions, video consultations were very slightly shorter, patients did slightly more talking, and both parties sometimes needed to make explicit things that typically remained implicit in a traditional encounter. Video consultations appeared to work better when the clinician and patient already knew and trusted each other. Some clinicians used Skype adaptively to respond to patient requests for ad hoc encounters in a way that appeared to strengthen supported self-management. The reality of establishing video outpatient services in a busy and financially stretched acute hospital setting proved more complex and time-consuming than originally anticipated. By the end of this study, between 2% and 22% of consultations were being undertaken remotely by participating clinicians. In the remainder, clinicians chose not to participate, or video consultations were considered impractical, technically unachievable, or clinically inadvisable. Technical challenges were typically minor but potentially prohibitive.
Conclusions
Video outpatient consultations appear safe, effective, and convenient for patients in situations where participating clinicians judge them clinically appropriate, but such situations are a fraction of the overall clinic workload. As with other technological innovations, some clinicians will adopt readily, whereas others will need incentives and support. There are complex challenges to embedding video consultation services within routine practice in organizations that are hesitant to change, especially in times of austerity.
Background
Many promising technological innovations in health and social care are characterized by nonadoption or abandonment by individuals or by failed attempts to scale up locally, spread distantly, or sustain the innovation long term at the organization or system level.
Objective
Our objective was to produce an evidence-based, theory-informed, and pragmatic framework to help predict and evaluate the success of a technology-supported health or social care program.
Methods
The study had 2 parallel components: (1) secondary research (hermeneutic systematic review) to identify key domains, and (2) empirical case studies of technology implementation to explore, test, and refine these domains. We studied 6 technology-supported programs—video outpatient consultations, global positioning system tracking for cognitive impairment, pendant alarm services, remote biomarker monitoring for heart failure, care organizing software, and integrated case management via data sharing—using longitudinal ethnography and action research for up to 3 years across more than 20 organizations. Data were collected at micro level (individual technology users), meso level (organizational processes and systems), and macro level (national policy and wider context). Analysis and synthesis was aided by sociotechnically informed theories of individual, organizational, and system change. The draft framework was shared with colleagues who were introducing or evaluating other technology-supported health or care programs and refined in response to feedback.
Results
The literature review identified 28 previous technology implementation frameworks, of which 14 had taken a dynamic systems approach (including 2 integrative reviews of previous work). Our empirical dataset consisted of over 400 hours of ethnographic observation, 165 semistructured interviews, and 200 documents. The final nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework included questions in 7 domains: the condition or illness, the technology, the value proposition, the adopter system (comprising professional staff, patient, and lay caregivers), the organization(s), the wider (institutional and societal) context, and the interaction and mutual adaptation between all these domains over time. Our empirical case studies raised a variety of challenges across all 7 domains, each classified as simple (straightforward, predictable, few components), complicated (multiple interacting components or issues), or complex (dynamic, unpredictable, not easily disaggregated into constituent components). Programs characterized by complicatedness proved difficult but not impossible to implement. Those characterized by complexity in multiple NASSS domains rarely, if ever, became mainstreamed. The framework showed promise when applied (both prospectively and retrospectively) to other programs.
Conclusions
Subject to further empirical testing, NASSS could be applied across a range of technological innovations in health and social care. It has several potential uses: (1) to inform the design of a new technology; (2) to identify technological solutions that (perhaps despite policy or industry enthusiasm) have a limited chance of achieving large-scale, sustained adoption; (3) to plan the implementation, scale-up, or rollout of a technology program; and (4) to explain and learn from program failures.
Background:
Health care is a complex sociotechnical system. Patient treatment is evolving and needs to incorporate the use of technology and new patient-centered treatment paradigms. Cognitive work analysis (CWA) is an effective framework for understanding complex systems, and work domain analysis (WDA) is useful for understanding complex ecologies. Although previous applications of CWA have described patient treatment, due to their scope of work patients were previously characterized as biomedical machines, rather than patient actors involved in their own care.
Objective:
An abstraction hierarchy that characterizes patients as beings with complex social values and priorities is needed. This can help better understand treatment in a modern approach to care. The purpose of this study was to perform a WDA to represent the treatment of patients with medical records.
Methods:
The methods to develop this model included the analysis of written texts and collaboration with subject matter experts. Our WDA represents the ecology through its functional purposes, abstract functions, generalized functions, physical functions, and physical forms.
Results:
Compared with other work domain models, this model is able to articulate the nuanced balance between medical treatment, patient education, and limited health care resources. Concepts in the analysis were similar to the modeling choices of other WDAs but combined them in as a comprehensive, systematic, and contextual overview. The model is helpful to understand user competencies and needs. Future models could be developed to model the patient's domain and enable the exploration of the shared decision-making (SDM) paradigm.
Conclusion:
Our work domain model links treatment goals, decision-making constraints, and task workflows. This model can be used by system developers who would like to use ecological interface design (EID) to improve systems. Our hierarchy is the first in a future set that could explore new treatment paradigms. Future hierarchies could model the patient as a controller and could be useful for mobile app development.
Background
Enthusiasts for telehealth extol its potential for supporting heart failure management. But randomised trials have been slow to recruit and produced conflicting findings; real-world roll-out has been slow. We sought to inform policy by making sense of a complex literature on heart failure and its remote management.
Methods
Through database searching and citation tracking, we identified 7 systematic reviews of systematic reviews, 32 systematic reviews (including 17 meta-analyses and 8 qualitative reviews); six mega-trials and over 60 additional relevant empirical studies and commentaries. We synthesised these using Boell’s hermeneutic methodology for systematic review, which emphasises the quest for understanding.
Results
Heart failure is a complex and serious condition with frequent co-morbidity and diverse manifestations including severe tiredness. Patients are often frightened, bewildered, socially isolated and variably able to self-manage. Remote monitoring technologies are many and varied; they create new forms of knowledge and new possibilities for care but require fundamental changes to clinical roles and service models and place substantial burdens on patients, carers and staff. The policy innovation of remote biomarker monitoring enabling timely adjustment of medication, mediated by “activated” patients, is based on a modernist vision of efficient, rational, technology-mediated and guideline-driven (“cold”) care. It contrasts with relationship-based (“warm”) care valued by some clinicians and by patients who are older, sicker and less technically savvy. Limited uptake of telehealth can be analysed in terms of key tensions: between tidy, “textbook” heart failure and the reality of multiple comorbidities; between basic and intensive telehealth; between activated, well-supported patients and vulnerable, unsupported ones; between “cold” and “warm” telehealth; and between fixed and agile care programmes.
Conclusion
The limited adoption of telehealth for heart failure has complex clinical, professional and institutional causes, which are unlikely to be elucidated by adding more randomised trials of technology-on versus technology-off to an already-crowded literature. An alternative approach is proposed, based on naturalistic study designs, application of social and organisational theory, and co-design of new service models based on socio-technical principles. Conventional systematic reviews (whose goal is synthesising data) can be usefully supplemented by hermeneutic reviews (whose goal is deepening understanding).
Electronic supplementary material
The online version of this article (doi:10.1186/s12872-017-0594-2) contains supplementary material, which is available to authorized users.
There is increasing demand for a systems approach within national healthcare guidelines to provide a systematic and sustainable framework for improvements in patient safety. Supported by this is the growing body of evidence within Human Factors/Ergonomics (HFE) healthcare literature for the inclusion of this approach in health service design, provision and evaluation. This paper considers the current interpretation of this within UK healthcare systems and the dichotomy which exists in the challenge to implement a systems approach. Three case studies, from primary and secondary care, present a systems approach, offering a novel perspective of primary care and blood sampling. These provide practical illustrations of how HFE methods have been used in collaboration with healthcare staff to understand the system for the purpose of professional education, design and safety of clinical activities. The paper concludes with the challenge for implementation and proposes five roles for systems HFE to support patient safety.
Practitioner Summary
Healthcare is classified as a complex and dynamic system within this paper and as such HFE systems methods are presented as desirable to understand the system, to develop HFE tools, to deliver education and integrate HFE within healthcare systems.
Background
The effectiveness of complex interventions, as well as their success in reaching relevant populations, is critically influenced by their implementation in a given context. Current conceptual frameworks often fail to address context and implementation in an integrated way and, where addressed, they tend to focus on organisational context and are mostly concerned with specific health fields. Our objective was to develop a framework to facilitate the structured and comprehensive conceptualisation and assessment of context and implementation of complex interventions. Methods
The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative manner and underwent extensive application. An initial framework based on a scoping review was tested in rapid assessments, revealing inconsistencies with respect to the underlying concepts. Thus, pragmatic utility concept analysis was undertaken to advance the concepts of context and implementation. Based on these findings, the framework was revised and applied in several systematic reviews, one health technology assessment (HTA) and one applicability assessment of very different complex interventions. Lessons learnt from these applications and from peer review were incorporated, resulting in the CICI framework. ResultsThe CICI framework comprises three dimensions—context, implementation and setting—which interact with one another and with the intervention dimension. Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise. The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level. Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments. Conclusions
The CICI framework addresses and graphically presents context, implementation and setting in an integrated way. It aims at simplifying and structuring complexity in order to advance our understanding of whether and how interventions work. The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders.
Telehealth literature has grown over the years to be a substantial body of work. However, research still highlights problems in extracting value from telehealth activities. In this paper, we analyse the telehealth literature over the period 2000–2015. The methodology focuses on paradoxes or surprises in the findings. We explain the lack of an operating model for telehealth as a debilitating feature of its history. We also note that many of the barriers cited in the literature have not changed over the period. We call for a focused research agenda on telehealth that provides clear evidence for some of the benefits such as cost savings and patient benefits. If telehealth activities are to progress, fundamental issues need to be addressed both in research and practice. Copyright
Background
Context is a problem in research on health behaviour change, knowledge translation, practice implementation and health improvement. This is because many intervention and evaluation designs seek to eliminate contextual confounders, when these represent the normal conditions into which interventions must be integrated if they are to be workable in practice. DiscussionWe present an ecological model of the ways that participants in implementation and health improvement processes interact with contexts. The paper addresses the problem of context as it affects processes of implementation, scaling up and diffusion of interventions. We extend our earlier work to develop Normalisation Process Theory and show how these processes involve interactions between mechanisms of resource mobilisation, collective action and negotiations with context. These mechanisms are adaptive. They contribute to self-organisation in complex adaptive systems. Conclusion
Implementation includes the translational efforts that take healthcare interventions beyond the closed systems of evaluation studies into the open systems of ‘real world’ contexts. The outcome of these processes depends on interactions and negotiations between their participants and contexts. In these negotiations, the plasticity of intervention components, the degree of participants’ discretion over resource mobilisation and actors’ contributions, and the elasticity of contexts, all play important parts. Understanding these processes in terms of feedback loops, adaptive mechanisms and the practical compromises that stem from them enables us to see the mechanisms specified by NPT as core elements of self-organisation in complex systems.
This article outlines the contribution that ethnography could make to process evaluations for trials of complex health-behaviour interventions. Process evaluations are increasingly used to examine how health-behaviour interventions operate to produce outcomes and often employ qualitative methods to do this. Ethnography shares commonalities with the qualitative methods currently used in health-behaviour evaluations but has a distinctive approach over and above these methods. It is an overlooked methodology in trials of complex health-behaviour interventions that has much to contribute to the understanding of how interventions work. These benefits are discussed here with respect to three strengths of ethnographic methodology: (1) producing valid data, (2) understanding data within social contexts, and (3) building theory productively. The limitations of ethnography within the context of process evaluations are also discussed.
This chapter discusses the cognitive systems engineering. To build a cognitive description of a problem solving world, it is necessary to understand how representations of the world interact with different cognitive demands imposed by the application world in question and with characteristics of the cognitive agents, both for existing and prospective changes in the world. Building a cognitive description is part of a problem driven approach to the application of computational power. In tool-driven approaches, knowledge acquisition focuses on describing domain knowledge in terms of the syntax of computational mechanisms, that is, the language of implementation is used as a cognitive language. Semantic questions are displaced either to whoever selects the computational mechanisms or to the domain expert who enters knowledge. The alternative is to provide an umbrella structure of domain semantics that organizes and makes explicit what particular pieces of knowledge mean about problem solving in the domain. Acquiring and using domain semantics is essential to be capable of avoiding potential errors and specifying performance boundaries when building intelligent machines.
Improving patient safety, within the context of a complex system, forms one of the key challenges in health care today. Cognitive work analysis (CWA) is one way to analyze complex systems, and although it has been applied to health care for 20 years, little is known about its effectiveness or future research needs. This article presents a review of CWA studies in health care, addressing questions of use, usefulness, challenges, and opportunities. Results of the review make clear that the research agenda is largely confined to acute care. Of the 39 articles reviewed, 28 relate to this setting. There appears to be a growing interest in medical informatics, error investigation, and decision support. Conversely, work in physiological monitoring has slowed, associated with the uncertainties of modeling “biological” systems. Studies related to “organic” social systems are similarly challenged, although there is a recognition that important opportunities exist, such as studying work flow processes between teams. Other opportunities relate to new methods to enhance CWA; new technologies, such as auditory displays; and new applications, such as requests for proposals and incident investigation. Ultimately, the capacity to foster an understanding into the deep structures of a system may prove to be the greatest contribution of CWA to health care today.
Unlabelled:
Medication safety and errors are a major concern in care homes. In addition to the identification of incidents, there is a need for a comprehensive system description to avoid the danger of introducing interventions that have unintended consequences and are therefore unsustainable. The aim of this study was to explore the impact and uniqueness of Work Domain Analysis (WDA) to facilitate an in-depth understanding of medication safety problems within the care home system and identify the potential benefits of WDA to design safety interventions to improve medication safety. A comprehensive, systematic and contextual overview of the care home medication system was developed for the first time. The novel use of the abstraction hierarchy (AH) to analyse medication errors revealed the value of the AH to guide a comprehensive analysis of errors and generate system improvement recommendations that took into account the contextual information of the wider system.
Practitioner summary:
It is widely acknowledged that a systems approach is necessary to improve medication safety. This study used a cognitive engineering method, Work Domain Analysis, to map the care home medication system and analyse medication errors. A macro-level view of the system was developed and this has provided a knowledge base for future interventions.
Implementation science has progressed towards increased use of theoretical approaches to provide better understanding and explanation of how and why implementation succeeds or fails. The aim of this article is to propose a taxonomy that distinguishes between different categories of theories, models and frameworks in implementation science, to facilitate appropriate selection and application of relevant approaches in implementation research and practice and to foster cross-disciplinary dialogue among implementation researchers.
Theoretical approaches used in implementation science have three overarching aims: describing and/or guiding the process of translating research into practice (process models); understanding and/or explaining what influences implementation outcomes (determinant frameworks, classic theories, implementation theories); and evaluating implementation (evaluation frameworks). This article proposes five categories of theoretical approaches to achieve three overarching aims. These categories are not always recognized as separate types of approaches in the literature. While there is overlap between some of the theories, models and frameworks, awareness of the differences is important to facilitate the selection of relevant approaches. Most determinant frameworks provide limited "how-to" support for carrying out implementation endeavours since the determinants usually are too generic to provide sufficient detail for guiding an implementation process. And while the relevance of addressing barriers and enablers to translating research into practice is mentioned in many process models, these models do not identify or systematically structure specific determinants associated with implementation success. Furthermore, process models recognize a temporal sequence of implementation endeavours, whereas determinant frameworks do not explicitly take a process perspective of implementation.
Telehealthcare involves the use of information and communication technologies to deliver healthcare at a distance and to support patient self-management through remote monitoring and personalised feedback. It is timely to scrutinise the evidence regarding the benefits, risks and costs of telehealthcare.Two reviewers searched for relevant systematic reviews published from January 1997 to November 2011 in: The Cochrane Library, MEDLINE, EMBASE, LILACS, IndMed and PakMed. Reviewers undertook independent quality assessment of studies using the Critical Appraisal Skills Programme (CASP) tool for systematic reviews. 1,782 review articles were identified, from which 80 systematic reviews were selected for inclusion. These covered a range of telehealthcare models involving both synchronous (live) and asynchronous (store-and-forward) interactions between provider and patients. Many studies showed no differences in outcomes between telehealthcare and usual care. Several reviews highlighted the large number of short-term (
We explored the use of Work Domain Analysis (WDA) in the context of maritime tactical picture compilation. In particular, we were interested in extracting requirements for collaboration within a naval task group. This work presents a novel use of WDA, as this analysis method has not previously been used specifically to examine collaborative work processes. The WDA identified unique domain views for various team positions and collaborative boundary objects shared within the team. This use of WDA in a group context provides new requirements for the design of collaboration tools.
UNSTRUCTURED
Many promising telemedicine innovations fail to be accepted and used over time, and there are longstanding questions about how to best evaluate telemedicine services and other health information technologies. In response to these challenges, there is a growing interest in how to take the sociotechnical complexity of health care into account during design, implementation, and evaluation. This paper discusses the methodological implications of this complexity and how the sociotechnical context holds the key to understanding the effects and outcomes of telemedicine. Examples from a work domain analysis of a surgical setting, where a telemedicine service for remote surgical consultation was to be introduced, are used to show how abstracted functional modeling can provide a structured and rigorous means to analyze and represent the implementation context in complex health care settings.
Neuro-critical care is a complex environment where clinicians have to deal with a great variety and amount of data on a daily basis. The different types of medical equipment and software often pose challenges to clinicians as they are separate systems and lack long data storage which is necessary to trend patient health over time. An observational study has been conducted, followed by the first step of a Cognitive Work Analysis approach to understand and visualize key components of this field in more depth. This so-called Work Domain Analysis shows findings for relations between the different abstract levels of neuro-critical care monitoring and treatment, as well as the nervous system and cognition. The abstract representation shapes the basis of designing an Ecological Interface.
Background:
Projects to implement health care and social care innovations involving technologies are typically ambitious and complex. Many projects fail. Greenhalgh et al's nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework was developed to analyze the varied outcomes of such projects.
Objective:
We sought to extend the NASSS framework to produce practical tools for understanding, guiding, monitoring, and researching technology projects in health care or social care settings.
Methods:
Building on NASSS and a complexity assessment tool (CAT), the NASSS-CAT tools were developed (in various formats) in seven co-design workshops involving 50 stakeholders (industry executives, technical designers, policymakers, managers, clinicians, and patients). Using action research, they were and are being tested prospectively on a sample of case studies selected for variety in conditions, technologies, settings, scope and scale, policy context, and project goals.
Results:
The co-design process resulted in four tools, available as free downloads. NASSS-CAT SHORT is a taster to introduce the instrument and gauge interest. NASSS-CAT LONG is intended to support reflection, due diligence, and preliminary planning. It maps complexity through stakeholder discussion across six domains, using free-text open questions (designed to generate a rich narrative and surface uncertainties and interdependencies) and a closed-question checklist; this version includes an action planning section. NASSS-CAT PROJECT is a 35-item instrument for monitoring how subjective complexity in a technology implementation project changes over time. NASSS-CAT INTERVIEW is a set of prompts for conducting semistructured research or evaluation interviews. Preliminary data from empirical case studies suggest that the NASSS-CAT tools can potentially identify, but cannot always help reconcile, contradictions and conflicts that block projects' progress.
Conclusions:
The NASSS-CAT tools are a useful addition to existing implementation tools and frameworks. Further support of the implementation projects is ongoing. We are currently producing digital versions of the tools, and plan (subject to further funding) to establish an online community of practice for people interested in using and improving the tools, and hold workshops for building cross-project collaborations.
International registered report identifier (irrid):
DERR1-10.2196/16861.
Despite the complexity of sociotechnical systems such as health care, the critical embedded element—human beings—is often minimized or ignored during the design phases of improvement efforts. The human is frequently considered only after an implementation has not gone well, while the relationships of workers to one another within the work system are rarely acknowledged. This stance is based on outdated assumptions that workers perform in “unsystematic and irrational ways” and that designers are “clumsy or even stupid” (Vicente 1999, p. xi). Cognitive work analysis (CWA) examines existing work systems and develops recommendations for designing for effective human involvement in the origination and nature of work. CWA views individuals embedded within a work system as “actors” rather than “users,” thus including the human as an influential member of each work system (Vicente 1999). Rasmussen and Vicente describe CWA as a method for truly understanding the influence of the work system and the relationship of its components (i.e., tools, technology, tasks, organization, people, environment, training, etc.) on work performance. Vicente et al. charge that researchers have failed to acknowledge that the system actually shapes worker behaviors, and as such, departures from expected behavior are generally the result of workers being clever, rational, and, more importantly, competent in their current work setting. These features of the “human factor” are imperative for the continuation and optimization of work in any complex system and, as such, must be designed for and optimized within their work system.
The core of the workshop would have to be an example application that called for cognitive systems engineering. We considered several different possibilities. One option was to design a kitchen. Another option was to redesign a global positioning system device for helping rental car customers navigate in unfamiliar cities. We initially selected the kitchen design exercise as one that would allow participants to immerse themselves in the design problem without any prior need for familiarization with the problem domain. We anticipated that workshop participants would be able to link kitchen design issues to the workshop exercises that were to be introduced through the remainder of the day.
Objectives: This article aims to search for the way patient care information systems can be most fruitfully put to work in health care.
Methods: Given the calls for improved health care quality, there is a need for IT’s coordination capacities in health care. Yet IT’s track record in this area is less than many expected. Moreover, many argue that the nature of health care work sets natural limits to the possibilities of IT to revolutionize this work. Starting with an analysis of the paper record, this article explores the way IT and professional work can be interrelated synergistically.
Results: Two principles are discussed: 1) The key to a fruitful operation of IT in health care work lies in the unraveling of the care process, and the redistribution of tasks between professionals and the IT application. 2) Professionals should be given the skills and resources to adapt the IT application’s demands to the needs of their work practices.
Conclusion: IT can bring true process support to health care when taking the two principles discussed here into account.
Healthcare organizations(HCOs), accreditation organizations (AOs) and national competent authorities (NCAs) have been using risk classification for many years to design and implement management strategies for devices within their inventory or jurisdiction. The initial risk models were focused on severity of harm with little consideration for the probability of failure. ISO 14971 gave equal attention to probability and severity and has been effective for manufacturers (OEMs) to manage risks associated with the devices they produce and market. However, because of the different scopes from those applicable to OEMs, HCOs, AOs, and NCAs cannot use ISO 14971 without significant adaptations. Furthermore, these organizations also need to use actual postmarket device failure rates and causes collected from their inventory or domain instead of premarket estimates made by OEMs to make their risk assessment pertinent to their reality. To assist HCOs, AOs, and NCAs, a collective risk management model is proposed in this article. First, severity is expanded from individual degree of hazard by combining it with the number of patients potentially affected. Next, the Swiss-cheese model proposed by Reason is used to break up the probability of harm as a combination of probabilities resulting from the postmarket activities of distinct stakeholders (patients, users, healthcare institutions, companies, etc.). Finally, failures caused by individuals are segregated from “latent conditions” created by organizational failures. Applying this new model to 24 common device categories found in HCOs shows significantly different results from those obtained from the risk classification methods currently used, as well as from ISO 14971. Thus it is believed that this collective risk model will allow HCOs, AOs and NCAs to better determine where they should focus their limited resources to achieve the highest possible degree of safety and best care for their patients.
The purposive use of theory is a foundational component of research, which underpins the design, methodology, measures, interventions, and interpretation of the research project. This should be considered from the time the nascent idea of the research is born, until the final interpretation of results and write up of the discussion. Several theories relevant to telemedicine are described, discussed, and linked to typical research questions in the field.
Research that fulsomely characterizes context improves our understanding of the processes of implementation and the effectiveness of interventions to improve the health of populations and reduce health inequalities. Context could be a key aspect to understanding what population health interventions best address underlying conditions that contribute to systematic differences in health status at the population level. Here, we present a ‘snapshot’ of representations of context in population health intervention research published recently in some influential journals in the field of public health. In general, we found that context was treated as a ‘black box’, or as something that needs to be ‘controlled for’. Context also was used to help explain intervention ‘failure’. There were few in-depth descriptions of the hypothesized pathways or mechanisms through which context and intervention (in combination or separately) influence population health. Recognizing that research on population health interventions can be less straightforward than research conducted under more ‘controlled’ (i.e. de-contextualized) conditions, we suggest that there is a great need for new theoretical and methodological work in this area.
Background:
Telemedicine (TM) is the use of telecommunication systems to deliver health care at a distance. It has the potential to improve patient health outcomes, access to health care and reduce healthcare costs. As TM applications continue to evolve it is important to understand the impact TM might have on patients, healthcare professionals and the organisation of care.
Objectives:
To assess the effectiveness, acceptability and costs of interactive TM as an alternative to, or in addition to, usual care (i.e. face-to-face care, or telephone consultation).
Search methods:
We searched the Effective Practice and Organisation of Care (EPOC) Group's specialised register, CENTRAL, MEDLINE, EMBASE, five other databases and two trials registers to June 2013, together with reference checking, citation searching, handsearching and contact with study authors to identify additional studies.
Selection criteria:
We considered randomised controlled trials of interactive TM that involved direct patient-provider interaction and was delivered in addition to, or substituting for, usual care compared with usual care alone, to participants with any clinical condition. We excluded telephone only interventions and wholly automatic self-management TM interventions.
Data collection and analysis:
For each condition, we pooled outcome data that were sufficiently homogenous using fixed effect meta-analysis. We reported risk ratios (RR) and 95% confidence intervals (CI) for dichotomous outcomes, and mean differences (MD) for continuous outcomes.
Main results:
We included 93 eligible trials (N = 22,047 participants), which evaluated the effectiveness of interactive TM delivered in addition to (32% of studies), as an alternative to (57% of studies), or partly substituted for usual care (11%) as compared to usual care alone.The included studies recruited patients with the following clinical conditions: cardiovascular disease (36), diabetes (21), respiratory conditions (9), mental health or substance abuse conditions (7), conditions requiring a specialist consultation (6), co morbidities (3), urogenital conditions (3), neurological injuries and conditions (2), gastrointestinal conditions (2), neonatal conditions requiring specialist care (2), solid organ transplantation (1), and cancer (1).Telemedicine provided remote monitoring (55 studies), or real-time video-conferencing (38 studies), which was used either alone or in combination. The main TM function varied depending on clinical condition, but fell typically into one of the following six categories, with some overlap: i) monitoring of a chronic condition to detect early signs of deterioration and prompt treatment and advice, (41); ii) provision of treatment or rehabilitation (12), for example the delivery of cognitive behavioural therapy, or incontinence training; iii) education and advice for self-management (23), for example nurses delivering education to patients with diabetes or providing support to parents of very low birth weight infants or to patients with home parenteral nutrition; iv) specialist consultations for diagnosis and treatment decisions (8), v) real-time assessment of clinical status, for example post-operative assessment after minor operation or follow-up after solid organ transplantation (8) vi), screening, for angina (1).The type of data transmitted by the patient, the frequency of data transfer, (e.g. telephone, e-mail, SMS) and frequency of interactions between patient and healthcare provider varied across studies, as did the type of healthcare provider/s and healthcare system involved in delivering the intervention.We found no difference between groups for all-cause mortality for patients with heart failure (16 studies; N = 5239; RR:0.89, 95% CI 0.76 to 1.03, P = 0.12; I(2) = 44%) (moderate to high certainty of evidence) at a median of six months follow-up. Admissions to hospital (11 studies; N = 4529) ranged from a decrease of 64% to an increase of 60% at median eight months follow-up (moderate certainty of evidence). We found some evidence of improved quality of life (five studies; N = 482; MD:-4.39, 95% CI -7.94 to -0.83; P < 0.02; I(2) = 0%) (moderate certainty of evidence) for those allocated to TM as compared with usual care at a median three months follow-up. In studies recruiting participants with diabetes (16 studies; N = 2768) we found lower glycated haemoglobin (HbA1c %) levels in those allocated to TM than in controls (MD -0.31, 95% CI -0.37 to -0.24; P < 0.00001; I(2)= 42%, P = 0.04) (high certainty of evidence) at a median of nine months follow-up. We found some evidence for a decrease in LDL (four studies, N = 1692; MD -12.45, 95% CI -14.23 to -10.68; P < 0.00001; I(2 =) 0%) (moderate certainty of evidence), and blood pressure (four studies, N = 1770: MD: SBP:-4.33, 95% CI -5.30 to -3.35, P < 0.00001; I(2) = 17%; DBP: -2.75 95% CI -3.28 to -2.22, P < 0.00001; I(2) = 45% (moderate certainty evidence), in TM as compared with usual care.Seven studies that recruited participants with different mental health and substance abuse problems, reported no differences in the effect of therapy delivered over video-conferencing, as compared to face-to-face delivery. Findings from the other studies were inconsistent; there was some evidence that monitoring via TM improved blood pressure control in participants with hypertension, and a few studies reported improved symptom scores for those with a respiratory condition. Studies recruiting participants requiring mental health services and those requiring specialist consultation for a dermatological condition reported no differences between groups.
Authors' conclusions:
The findings in our review indicate that the use of TM in the management of heart failure appears to lead to similar health outcomes as face-to-face or telephone delivery of care; there is evidence that TM can improve the control of blood glucose in those with diabetes. The cost to a health service, and acceptability by patients and healthcare professionals, is not clear due to limited data reported for these outcomes. The effectiveness of TM may depend on a number of different factors, including those related to the study population e.g. the severity of the condition and the disease trajectory of the participants, the function of the intervention e.g., if it is used for monitoring a chronic condition, or to provide access to diagnostic services, as well as the healthcare provider and healthcare system involved in delivering the intervention.
Four approaches to interface design are considered: technology centered, user centered, control centered, and use centered (ecological). Each perspective provides unique insights into pieces of the interface design problem. However, it is argued that the ecological or use centered approach provides a more comprehensive framework within which the other three perspectives can play important supporting roles. This approach goes beyond issues of information requirements to address meaning as an emergent property of a dynamic work ecology.
Work analysis techniques are the primary methods for designers to obtain knowledge for good interfaces. The majority of current techniques are either task-centered or system- or work domain-centered. In previous work (Miller and Vicente, 1998) we maintained that each technique focuses on different aspects of the design problem and has complementary strengths and weaknesses, thus mandating a unification for completeness. Here, we compare the results of two analyses of the same work environment, one using a work domain-centered technique (Rasmussen's (1985) Abstraction Decomposition Space or Abstraction Hierarchy) and the other using a task-centered technique (Hierarchical Task Analysis—Shepherd, 1989). We compare the requirements produced by each analytic technique and demonstrate their complementary nature. We argue for examining a work domain from both perspectives and discuss interface concepts that would satisfy both analyses. We argue that these interfaces would provide better user support than one designed from either analytic technique alone.
Cognitive work analysis (CWA) is a framework of methods for analysing complex sociotechnical systems. However, the translation from the outputs of CWA to design is not straightforward. Sociotechnical systems theory provides values and principles for the design of sociotechnical systems which may offer a theoretically consistent basis for a design approach for use with CWA. This article explores the extent to which CWA and sociotechnical systems theory offer complementary perspectives and presents an abstraction hierarchy (AH), based on a review of literature, that describes an 'optimal' CWA and sociotechnical systems theory design system. The optimal AH is used to assess the extent to which current CWA-based design practices, uncovered through a survey of CWA practitioners, aligns with sociotechnical systems theory. Recommendations for a design approach that would support the integration of CWA and sociotechnical systems theory design values and principles are also derived.
The term cognitive task analysis (CTA) has been appearing in the human factors literature with increasing frequency. Others have used the term cognitive work analysis (CWA). Is there a difference? Do either of these methods differ from traditional task analysis (TA)? If so, what advantages can CTA/CWA provide human factors engineers? To address these issues, the history of work analysis methods and the evolution of work are reviewed. Work method analyses of the 19th century were suited to manual labor. As job demands progressed beyond the physical, traditional TA was introduced to provide a broader perspective. CTA has since been introduced to increase the emphasis on cognitive task demands. However, CTA, like TA, is incapable of dealing with unanticipated task demands. CWA has been introduced to deal with complex systems whose demands include unanticipated events. The initial evidence available indicates that CWA can be applied to industry-scale problems, leading to innovative designs.
In this paper, we examine an approach to representing the impact of technological change on work practice. Taking Cognitive Work Analysis as a point of departure, we discuss an approach with which one can analyze work domains and work activities and then use these analyses as a ‘field’ upon which to trace the impact of new technology. The relation between activity and work structure is explored. Then, a case study of the introduction of an electronic anesthesia record is examined. Associated changes to work structures and activities are represented and interpreted. Finally, we discuss the implications of the work and future directions.
Many current global health opportunities have less to do with new biomedical knowledge than with the coordination and delivery of care. While basic research remains vital, the growing cancer epidemic in countries of low and middle income warrants urgent action - focusing on both research and service delivery innovation. Mobile technology can reduce costs, improve access to health services, and strengthen health systems to meet the interrelated challenges of cancer and other noncommunicable diseases. Experience has shown that even very poor and remote communities that only have basic primary health care can benefit from mobile health (or 'mHealth') interventions. We argue that cancer researchers and practitioners have an opportunity to leverage mHealth technologies that have successfully targeted other health conditions, rather than reinventing these tools. We call for particular attention to human centred design approaches for adapting existing technologies to suit distinctive aspects of cancer care and to align delivery with local context - and we make a number of recommendations for integrating mHealth delivery research with the work of designers, engineers and implementers in large-scale delivery programmes.
Cognitive work analysis (CWA) as an analytical approach for examining complex sociotechnical systems has shown success in modelling the work of single operators. The CWA approach incorporates social and team interactions, but a more explicit analysis of team aspects can reveal more information for systems design. In this paper, Team CWA is explored to understand teamwork within a birthing unit at a hospital. Team CWA models are derived from theories and models of teamworkand leverage the existing CWA approaches to analyse team interactions. Team CWA is explained and contrasted with prior approaches to CWA. Team CWA does not replace CWA, but supplements traditional CWA to more easily reveal team information. As a result, Team CWA may be a useful approach to enhance CWA in complex environments where effective teamwork is required.
Practitioner Summary: This paper looks at ways of analysing cognitive work in healthcare teams. Team Cognitive Work Analysis, when used to supplement traditional Cognitive Work Analysis, revealed more team information than traditional Cognitive Work Analysis. Team Cognitive Work Analysis should be considered when studying teams
Complex socio-technical systems are notoriously difficult to evaluate. The relationships between measurable processes and high-level descriptions of performance are neither clear nor well understood. Consequently, it is extremely difficult to relate changes made to physical components to overall system performance. Using the relationships captured in work domain analysis, this paper introduces a new approach for evaluating the impact of system changes. An event and actor-independent model is used to describe the given domain at a number of levels of abstraction. The relationship between these descriptions forms the basis of this approach. A case study of a recently upgraded UK land military headquarters is used to demonstrate the approach. As the presented case study will show, a clear link can be made between system changes that result in an increase in overall system performance, as well as changes that result in a degradation. It is contended that the described approach adds important structure to the way that evaluations of complex socio-technical systems are considered, creating a traceable link between changes made at a component level to high-order descriptions of domain purpose.