Strategies for More Effective Monitoring and Evaluation Systems in HIV Programmatic Scale-Up in Resource-Limited Settings: Implications for Health Systems Strengthening

International Center for AIDS Care and Treatment Programs, Columbia University Mailman School of Public Health, New York, NY, USA.
JAIDS Journal of Acquired Immune Deficiency Syndromes (Impact Factor: 4.56). 11/2009; 52 Suppl 1(Suppl 1):S58-62. DOI: 10.1097/QAI.0b013e3181bbcc45
Source: PubMed


Program monitoring and evaluation (M&E) has the potential to be a cornerstone of health systems strengthening and of evidence-informed implementation and scale-up of HIV-related services in resource-limited settings. We discuss common challenges to M&E systems used in the rapid scale-up of HIV services as well as innovations that may have relevance to systems used to monitor, evaluate, and inform health systems strengthening. These include (1) Web-based applications with decentralized data entry and real-time access to summary reporting; (2) timely feedback of information to site and district staff; (3) site-level integration of traditionally siloed program area indicators; (4) longitudinal tracking of program and site characteristics; (5) geographic information systems; and (6) use of routinely collected aggregate data for epidemiologic analysis and operations research. Although conventionally used in the context of vertical programs, these approaches can form a foundation on which data relevant to other health services and systems can be layered, including prevention services, primary care, maternal-child health, and chronic disease management. Guiding principles for sustainable national M&E systems include country-led development and ownership, support for national programs and policies, interoperability, and employment of an open-source approach to software development.

Download full-text


Available from: Denis Nash,
  • Source
    • "Application of information technology may greatly facilitate data management activities and improve the quality of data (6, 28–30), and hospitals in Kenya are investing in this technology. They are often doing this in the absence of local, professional technical support; however, this investment largely focuses on administrative management, notably improving revenue collection. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Background Hospital management information systems (HMIS) is a key component of national health information systems (HIS), and actions required of hospital management to support information generation in Kenya are articulated in specific policy documents. We conducted an evaluation of core functions of data generation and reporting within hospitals in Kenya to facilitate interpretation of national reports and to provide guidance on key areas requiring improvement to support data use in decision making. Design The survey was a cross-sectional, cluster sample study conducted in 22 hospitals in Kenya. The statistical analysis was descriptive with adjustment for clustering. Results Most of the HMIS departments complied with formal guidance to develop departmental plans. However, only a few (3/22) had carried out a data quality audit in the 12 months prior to the survey. On average 3% (range 1–8%) of the total hospital income was allocated to the HMIS departments. About half of the records officer positions were filled and about half (13/22) of hospitals had implemented some form of electronic health record largely focused on improving patient billing and not linked to the district HIS. Completeness of manual patient registers varied, being 90% (95% CI 80.1–99.3%), 75.8% (95% CI 68.7–82.8%), and 58% (95% CI 50.4–65.1%) in maternal child health clinic, maternity, and pediatric wards, respectively. Vital events notification rates were low with 25.7, 42.6, and 71.3% of neonatal deaths, infant deaths, and live births recorded, respectively. Routine hospital reports suggested slight over-reporting of live births and under-reporting of fresh stillbirths and neonatal deaths. Conclusions Study findings indicate that the HMIS does not deliver quality data. Significant constraints exist in data quality assurance, supervisory support, data infrastructure in respect to information and communications technology application, human resources, financial resources, and integration.
    Global Health Action 07/2014; 7:24859. DOI:10.3402/gha.v7.24859 · 1.93 Impact Factor
  • Source
    • "Substantial external investment, together with the requirement for service evolution to adapt to changing needs, including the constant shift to earlier ART initiation [1, 5, 6], makes outcome monitoring and reporting particularly important. However, there is growing concern at the inability of many high-burden countries to report on the outcomes of patients who have been in care for various durations, or even the number of patients in care at a particular point in time [4, 7–10]. In many instances, countries can only report on the number of patients ever initiated on ART. "
    [Show abstract] [Hide abstract]
    ABSTRACT: The provision of antiretroviral therapy (ART) in low and middle-income countries is a chronic disease intervention of unprecedented magnitude and is the dominant health systems challenge for high-burden countries, many of which rank among the poorest in the world. Substantial external investment, together with the requirement for service evolution to adapt to changing needs, including the constant shift to earlier ART initiation, makes outcome monitoring and reporting particularly important. However, there is growing concern at the inability of many high-burden countries to report on the outcomes of patients who have been in care for various durations, or even the number of patients in care at a particular point in time. In many instances, countries can only report on the number of patients ever started on ART. Despite paper register systems coming under increasing strain, the evolution from paper directly to complex electronic medical record solutions is not viable in many contexts. Implementing a bridging solution, such as a simple offline electronic version of the paper register, can be a pragmatic alternative. This paper describes and recommends a three-tiered monitoring approach in low- and middle-income countries based on the experience implementing such a system in the Western Cape province of South Africa. A three-tier approach allows Ministries of Health to strategically implement one of the tiers in each facility offering ART services. Each tier produces the same nationally required monthly enrolment and quarterly cohort reports so that outputs from the three tiers can be aggregated into a single database at any level of the health system. The choice of tier is based on context and resources at the time of implementation. As resources and infrastructure improve, more facilities will transition to the next highest and more technologically sophisticated tier. Implementing a three-tier monitoring system at country level for pre-antiretroviral wellness, ART, tuberculosis and mother and child health services can be an efficient approach to ensuring system-wide harmonization and accurate monitoring of services, including long term retention in care, during the scale-up of electronic monitoring solutions.
    Journal of the International AIDS Society 04/2014; 17(1):18908. DOI:10.7448/IAS.17.1.18908 · 5.09 Impact Factor
  • Source
    • "Resource intensive investments that can improve data quality include strengthening electronic health information systems, investing in tertiary education programs, and harmonizing data collection systems. Computerized point-of-care health information systems, particularly web-based systems, have the potential to dramatically reduce the data collection burden by automating data aggregation and reporting [25]. These systems would also allow for real-time access to data [25,26]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Ensuring that data collected through national health information systems are of sufficient quality for meaningful interpretation is a challenge in many resource-limited countries. An assessment was conducted to identify strengths and weaknesses of the health data management and reporting systems that capture and transfer routine monitoring and evaluation (M&E) data in Botswana. This was a descriptive, qualitative assessment. In-depth interviews were conducted at the national (n = 27), district (n = 31), and facility/community (n = 71) levels to assess i) M&E structures, functions, and capabilities; ii) indicator definitions and reporting guidelines; iii) data collection forms and tools; iv) data management processes; and v) links with the national reporting system. A framework analysis was conducted using ATLAS.ti v6.1. Health programs generally had standardized data collection and reporting tools and defined personnel for M&E responsibilities at the national and district levels. Best practices unique to individual health programs were identified and included a variety of relatively low-resource initiatives such as attention to staffing patterns, making health data more accessible for evidence-based decision-making, developing a single source of information related to indicator definitions, data collection tools, and management processes, and utilization of supportive supervision visits to districts and facilities. Weakness included limited ownership of M&E-related duties within facilities, a lack of tertiary training programs to build M&E skills, few standard practices related to confidentiality and document storage, limited dissemination of indicator definitions, and limited functionality of electronic data management systems. Addressing fundamental M&E system issues, further standardization of M&E practices, and increasing health services management responsiveness to time-sensitive information are critical to sustain progress related to health service delivery in Botswana. In addition to high-resource initiatives, such as investments in electronic medical record systems and tertiary training programs, there are a variety of low-resource initiatives, such as regular data quality checks, that can strengthen national health information systems. Applying best practices that are effective within one health program to data management and reporting systems of other programs is a practical approach for strengthening health informatics and improving data quality.
    Health Research Policy and Systems 01/2014; 12(1):7. DOI:10.1186/1478-4505-12-7 · 1.86 Impact Factor
Show more