Strategies for More Effective Monitoring and Evaluation Systems in HIV Programmatic Scale-Up in Resource-Limited Settings: Implications for Health Systems Strengthening

Article (PDF Available)inJAIDS Journal of Acquired Immune Deficiency Syndromes 52 Suppl 1(Suppl 1):S58-62 · November 2009with107 Reads
DOI: 10.1097/QAI.0b013e3181bbcc45 · Source: PubMed
Program monitoring and evaluation (M&E) has the potential to be a cornerstone of health systems strengthening and of evidence-informed implementation and scale-up of HIV-related services in resource-limited settings. We discuss common challenges to M&E systems used in the rapid scale-up of HIV services as well as innovations that may have relevance to systems used to monitor, evaluate, and inform health systems strengthening. These include (1) Web-based applications with decentralized data entry and real-time access to summary reporting; (2) timely feedback of information to site and district staff; (3) site-level integration of traditionally siloed program area indicators; (4) longitudinal tracking of program and site characteristics; (5) geographic information systems; and (6) use of routinely collected aggregate data for epidemiologic analysis and operations research. Although conventionally used in the context of vertical programs, these approaches can form a foundation on which data relevant to other health services and systems can be layered, including prevention services, primary care, maternal-child health, and chronic disease management. Guiding principles for sustainable national M&E systems include country-led development and ownership, support for national programs and policies, interoperability, and employment of an open-source approach to software development.


    • "Quality assurance, including defining quality, setting standards, and developing performance indicators, is a critical step towards identifying and addressing quality deficits. Implementing quality assurance remains an important challenge for HIV programs, particularly those in resource-limited settings, where human resource constraints and weaker health systems can pose formidable barriers to data collection and interpretation [3]. Quality improvement is a second key component of quality management, and building site-level and program-level capacity to identify quality barriers and to rapidly test interventions is a high priority in many countries. "
    [Show abstract] [Hide abstract] ABSTRACT: The centrality of quality as a strategy to achieve impact within the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) has been widely recognized. However, monitoring program quality remains a challenge for many HIV programs, particularly those in resource-limited settings, where human resource constraints and weaker health systems can pose formidable barriers to data collection and interpretation. We describe the practicalities of monitoring quality at scale within a very large multicountry PEPFAR-funded program, based largely at health facilities. The key elements include the following: supporting national programs and strategies; developing a conceptual framework and programmatic model to define quality and guide the provision of high-quality services; attending to program context, as well as program outcomes; leveraging existing and routinely collected data whenever possible; developing additional indicators for judicious use in targeted, in-depth assessments; providing hands-on support for data collection and use at the facility, sub-national, and national levels; utilizing web-based databases for data entry, analysis, and dissemination; and multidisciplinary support from a large team of clinical and strategic information advisors.
    Full-text · Article · Jul 2015
    • "In response, the World Health Organization (WHO) public-health approach advocates a simplified strategy for M&E that relies primarily on aggregated, facilitylevel data [1,3,15]. While these aggregated data represent a critical resource161718 , they lack the detail and specificity of patient-level data and are therefore limited. In particular, investigations of associations for patient-level outcomes based on aggregated data may suffer from ecological bias [19,20] and, in the worst-case scenario, the ecological fallacy where conclusions based on aggregated data are different than those that would have been drawn had a patient-level analysis been performed [21]. "
    [Show abstract] [Hide abstract] ABSTRACT: In resource-limited settings, monitoring and evaluation (M&E) of antiretroviral treatment (ART) programs often relies on aggregated facility-level data. Such data are limited, however, because of the potential for ecological bias, although collecting detailed patient-level data is often prohibitively expensive. To resolve this dilemma, we propose the use of the two-phase design. Specifically, when the outcome of interest is binary, the two-phase design provides a framework within which researchers can resolve ecological bias through the collection of patient-level data on a sub-sample of individuals while making use of the routinely collected aggregated data to obtain potentially substantial efficiency gains. Between 2005-2007, the Malawian Ministry of Health conducted a one-time cross-sectional survey of 82,887 patients registered at 189 ART clinics. Using these patient data, an aggregated dataset is constructed to mimic the type of data that it routinely available. A hypothetical study of risk factors for patient outcomes at 6 months post-registration is considered. Analyses are conducted based on: (i) complete patient-level data; (ii) aggregated data; (iii) a hypothetical case-control study; (iv) a hypothetical two-phase study stratified on clinic type; and, (v) a hypothetical two-phase study stratified on clinic type and registration year. A simulation study is conducted to compare statistical power to detect an interaction between clinic type and year of registration across the designs. Analyses and conclusions based solely on aggregated data may suffer from ecological bias. Collecting and analyzing patient data using either a case-control or two-phase design resolves ecological bias to provide valid conclusions. To detect the interaction between clinic type and year of registration, the case-control design would require a prohibitively large sample size. In contrast, a two-phase design that stratifies on clinic and year of registration achieves greater than 85% power with as few as 1,000 patient samples. Two-phase designs have the potential to augment current M&E efforts in resource-limited settings by providing a framework for the collection and analysis of patient data. The design is cost-efficient in the sense that it often requires far fewer patients to be sampled when compared to standard designs.
    Full-text · Article · Apr 2015
    • "Observational studies relate site and programme characteristics to improved patient and programme outcomes [13,15], including higher CD4' cell count at the time of ART initia- tion [16,17] and improved ART medication adherence [18], service utilization [19] and loss to follow-up in HIV care [20]. Such analyses are an important part of the larger agenda of rigorous evaluation and implementation science in any largescale service or prevention programme [13,21,22] . Understanding programme characteristics, their evolution over time and how these characteristics may influence patient outcomes such as retention in care, immunodeficiency at ART initiation and response to ART strengthens the process of constructing and implementing more effective HIV care and treatment programmes. "
    [Show abstract] [Hide abstract] ABSTRACT: HIV care and treatment programmes worldwide are transforming as they push to deliver universal access to essential prevention, care and treatment services to persons living with HIV and their communities. The characteristics and capacity of these HIV programmes affect patient outcomes and quality of care. Despite the importance of ensuring optimal outcomes, few studies have addressed the capacity of HIV programmes to deliver comprehensive care. We sought to describe such capacity in HIV programmes in seven regions worldwide. Staff from 128 sites in 41 countries participating in the International epidemiologic Databases to Evaluate AIDS completed a site survey from 2009 to 2010, including sites in the Asia-Pacific region (n=20), Latin America and the Caribbean (n=7), North America (n=7), Central Africa (n=12), East Africa (n=51), Southern Africa (n=16) and West Africa (n=15). We computed a measure of the comprehensiveness of care based on seven World Health Organization-recommended essential HIV services. Most sites reported serving urban (61%; region range (rr): 33-100%) and both adult and paediatric populations (77%; rr: 29-96%). Only 45% of HIV clinics that reported treating children had paediatricians on staff. As for the seven essential services, survey respondents reported that CD4+ cell count testing was available to all but one site, while tuberculosis (TB) screening and community outreach services were available in 80 and 72%, respectively. The remaining four essential services - nutritional support (82%), combination antiretroviral therapy adherence support (88%), prevention of mother-to-child transmission (PMTCT) (94%) and other prevention and clinical management services (97%) - were uniformly available. Approximately half (46%) of sites reported offering all seven services. Newer sites and sites in settings with low rankings on the UN Human Development Index (HDI), especially those in the President's Emergency Plan for AIDS Relief focus countries, tended to offer a more comprehensive array of essential services. HIV care programme characteristics and comprehensiveness varied according to the number of years the site had been in operation and the HDI of the site setting, with more recently established clinics in low-HDI settings reporting a more comprehensive array of available services. Survey respondents frequently identified contact tracing of patients, patient outreach, nutritional counselling, onsite viral load testing, universal TB screening and the provision of isoniazid preventive therapy as unavailable services. This study serves as a baseline for on-going monitoring of the evolution of care delivery over time and lays the groundwork for evaluating HIV treatment outcomes in relation to site capacity for comprehensive care.
    Full-text · Article · Dec 2014
Show more