Advertisement

The use of data in resource limited settings to improve quality of care

Open AccessPublished:February 03, 2021DOI:https://doi.org/10.1016/j.siny.2021.101204

      Abstract

      Quality improvement is driven by benchmarking between and within institutions over time and the collaborative improvement efforts that stem from these comparisons. Benchmarking requires systematic collection and use of standardized data. Low- and middle-income countries (LMIC) have great potential for improvements in newborn outcomes but serious obstacles to data collection, analysis, and implementation of robust improvement methodologies exist. We review the importance of data collection, internationally recommended neonatal metrics, selected methods of data collection, and reporting. The transformation from data collection to data use is illustrated by several select data system examples from LMIC. Key features include aims and measures important to neonatal team members, co-development with local providers, immediate access to data for review, and multidisciplinary team involvement. The future of neonatal care, use of data, and the trajectory to reach global neonatal improvement targets in resource-limited settings will be dependent on initiatives led by LMIC clinicians and experts.

      Keywords

      Abbreviations

      DHIS
      district health information system
      ENAP
      Every Newborn Action Plan
      ENN
      Ethiopian Neonatal Network
      HCW
      health care workers
      HMIS
      health management information systems
      LMIC
      low- or middle-income country
      PPIP
      Perinatal Problem Identification Programme
      SPA
      Service Performance Assessment
      QI
      quality improvement
      UNICEF
      United Nations Children's Fund
      VON
      Vermont Oxford Network
      WHO
      World Health Organization

      1. Introduction

      Benchmarking comparisons between and within institutions over time and the collaborative quality improvement efforts that stem from these comparisons drive much of the impact of quality improvement (QI). Benchmarking requires systematic collection and use of standardized data. Although low- and middle-income countries (LMICs) have great potential for improvements in newborn outcomes using QI, there are serious obstacles to data collection, analysis, and implementation of robust improvement methodologies [
      ∗Institute of Medicine (U.S.)
      Committee on quality of health care in America. Crossing the quality chasm.
      ].
      We aim to review the importance of this data collection, the internationally recommended neonatal metrics, selected methods of data collection, reporting of metrics to enable their use for improvement, several LMIC case examples, and opportunities for further investment.

      2. Importance of data

      In 2018, 5.3 million children under the age of 5 years died globally; 2.5 million, almost half of these deaths, occurred in the first 28 days of life [
      • Hug L.
      • Sharrow D.
      • You D.
      • Hereward M.
      • Zhang Y.
      Levels and trends in child mortality report 2019.
      ]. The vast majority of neonatal deaths occur in LMICs, largely due to preventable causes [
      • Bhutta Z.A.
      • Das J.K.
      • Bahl R.
      • Lawn J.E.
      • Salam R.A.
      • Paul V.K.
      • et al.
      Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost?.
      ].
      In many countries neonatal mortality rates have stagnated, despite an increase in births occurring in health institutions with skilled attendants [
      • Dickson K.E.
      • Kinney M.V.
      • Moxon S.G.
      • Ashton J.
      • Zaka N.
      • Simen-Kapeu A.
      • et al.
      Scaling up quality care for mothers and newborns around the time of birth: an overview of methods and analyses of intervention-specific bottlenecks and solutions.
      ,
      • Requejo J.H.
      • Bryce J.
      • Barros A.J.D.
      • Berman P.
      • Bhutta Z.
      • Chopra M.
      • et al.
      Countdown to 2015 and beyond: fulfilling the health agenda for women and children.
      ]. This “know-do gap” relates to the difference between providers’ knowledge of appropriate care and the care they actually provide in a clinical encounter [
      • Gage A.D.
      • Kruk M.E.
      • Girma T.
      • Lemango E.T.
      The know-do gap in sick child care in Ethiopia.
      ]. Quantifying this gap, understanding the context, and tracking progress to close the gap requires data. With the global focus on improving neonatal mortality, improving the quality of care is a priority [
      • ∗Austin A.
      • Langer A.
      • Salam R.A.
      • Lassi Z.S.
      • Das J.K.
      • Bhutta Z.A.
      Approaches to improve the quality of maternal and newborn health care: an overview of the evidence.
      ]. Without data, health systems are powerless to improve the care they provide, learn from their own practice, collaborate effectively with other centers, or advocate successfully for the resources they need [
      • World Health Organization
      Monitoring the building blocks of health systems: a handbook of indicators and their management strategies.
      ,
      • AbouZahr C.
      • Boerma T.
      Health information systems: the foundations of public health.
      ]. In LMICs, where data are scarce, there is frequently a large disconnect between providers on the ground and the political bodies directing funds [
      • Leonard E.
      • de Kock I.
      • Bam W.
      Barriers and facilitators to implementing evidence-based health innovations in low- and middle-income countries: a systematic literature review.
      ]. Quality data can support clinicians and clinical managers to prioritize certain actions.
      Health systems in LMICs that develop and invest in reliable data collection processes can collaborate across centers and countries to improve care [
      • World Health Organization
      Monitoring the building blocks of health systems: a handbook of indicators and their management strategies.
      ,
      Health Metrics Network, World Health Organization
      Framework and standards for country health information systems.
      ]. Incorporating QI into a facility or health system requires a change in culture, empowering individuals and teams to improve systems with the goal of improving care [
      • Vander Schaaf E.B.
      • Cornett A.C.
      • Randolph G.D.
      Improving culture, one quality improvement project at a time.
      ]. The ability to accurately collect and use data allows centers to improve together by comparing inputs, processes, and outcomes using the same harmonized data items and definitions. This collaborative improvement supports cross-cultural and international cooperation and empowers health care workers (HCW) around the globe to learn from the care they provide [
      • World Health Organization
      Monitoring the building blocks of health systems: a handbook of indicators and their management strategies.
      ]. When data are used for the purposes of implementation research, the goals may expand beyond improving care to strengthening health service delivery, informing policy design and implementation, and empowering families and communities [
      • Theobald S.
      • Brandes N.
      • Gyapong M.
      • El-Saharty S.
      • Proctor E.
      • Diaz T.
      • et al.
      Implementation research: new imperatives and opportunities in global health.
      ].

      3. Internationally recommended neonatal metrics

      The international community is focused on improving care and inequities in health outcomes for infants and their families, by targeting the Sustainable Development Goals to reduce the global neonatal mortality rate to 12 or fewer per 1000 live births [
      United Nations Department of Economic and Social Affairs
      Sustainable development goals n.d.
      ]. To elucidate information about quality of care and system performance surrounding the time of birth, input and process metrics that relate to the mortality targets and high-impact interventions associated with reduced perinatal mortality rates are recommended. Inputs into the health care system reflect necessary resources, and process metrics reflect the actions undertaken to improve outcomes [
      • ∗Donabedian A.
      Evaluating the quality of medical care.
      ]. In a resource-limited setting, tracking inputs and processes can identify key drivers that might require modifications to impact outcomes of interest, for example clinical skills, adherence to guidelines, or environmental factors. Fig. 1 illustrates the relationship of input, process, and outcome measures for admission hypothermia.
      Fig. 1
      Fig. 1Relationship of input and process measures to the outcome of admission hypothermia.
      An important consideration for comparing any metric is defining and quantifying the denominator. When calculating national perinatal mortality data, home births, undocumented births, stillbirths, and early neonatal deaths introduce considerable error and biases. It is estimated that half of the world's babies do not receive a birth certificate, including most stillbirths and neonatal deaths [
      • ∗Lawn J.E.
      • Blencowe H.
      • Oza S.
      • You D.
      • Lee A.C.C.
      • Waiswa P.
      • et al.
      Every newborn: progress, priorities, and potential beyond survival.
      ]. In recognition of the importance of a robust vital registration system, many LMICs have strategies to register pregnancies and determine birth outcomes [
      • World Health Organization
      Making every baby count: audit and review of stillbirths and neonatal deaths.
      ].
      The Every Newborn Action Plan (ENAP), led jointly by the United Nations Children's Fund (UNICEF) and the World Health Organization (WHO), was launched in 2014 with the aim of supporting countries in ending preventable newborn death [
      • ∗World Health Organization
      UNICEF. Every Newborn
      An action plan to end preventable deaths.
      ]. A key feature of the ENAP is the development of objective indicators of quality to identify gaps. The ENAP includes ten core indicators and additional indicators (Table 1). Some of the indicators are not currently tracked globally, whereas others lack consistent definitions, may be difficult to operationalize and measure accurately and reliably in the LMIC setting, or require additional testing [
      • Mason E.
      • McDougall L.
      • Lawn J.E.
      • Gupta A.
      • Claeson M.
      • Pillay Y.
      • et al.
      From evidence to action to deliver a healthy start for the next generation.
      ,
      • Moxon∗ S.G.
      • Ruysen H.
      • Kerber K.J.
      • Amouzou A.
      • Fournier S.
      • Grove J.
      • et al.
      Count every newborn; a measurement improvement roadmap for coverage data.
      ]. An area of continued research focus remains consideration of facility-level metrics targeting morbidities among preterm infants (such as necrotizing enterocolitis, intraventricular hemorrhage, or retinopathy of prematurity) and diagnoses among all infants (such as sepsis and hypoxic ischemic encephalopathy) that require laboratory, radiologic, or pathologic confirmation for standardized and comparative purposes. Additionally, neonatal teams may want to stratify mortality rates in their facilities, such as by birth weight category and inborn status, to better evaluate system performance as it pertains to high risk groups. Stagnant mortality rates in higher risk groups (such as outborn babies or extremely low birth weight infants) may, for example, obfuscate small improvements in the lower risk neonatal population.
      Table 1Every Newborn Action Plan (ENAP) core and additional indicators.
      Core ENAP IndicatorsAdditional Indicators
      Impact1) Maternal mortality ratio

      2) Stillbirth rate
      Not currently routinely tracked at global level.


      3) Neonatal mortality rate
      Intrapartum stillbirth rate
      Not currently routinely tracked at global level.


      Low birth weight rate

      Preterm birth rate
      Not currently routinely tracked at global level.


      Small for gestational age
      Not currently routinely tracked at global level.


      Neonatal morbidity rates
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.


      Disability after neonatal conditions
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.
      Coverage: Care for all Mothers and Newborns4) Skilled attendant at birth

      5) Early postnatal care for mothers and babies

      6) Early initiation of breastfeeding
      Antenatal care

      Exclusive breastfeeding up to six months
      Coverage: Care for Newborns at Risk or with Complications7) Antenatal corticosteroid use
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.


      8) Neonatal resuscitation
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.


      9) Kangaroo mother care
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.


      10) Treatment of severe infections
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.
      Caesarean section rate

      Chlorhexidine cord cleansing
      Not currently routinely tracked at global level.
      ,
      Indicator requiring additional testing to inform consistent measurement.
      Indicators to be disaggregated to assess equity such as urban/rural, regional, wealth quintile and education.
      Reproduced with permission from “Improving Neonatal Care, A Global Perspective” Danielle Ehret, Jacquelyn Patterson, Carl Bose, Clinics in Perinatology 2017, adapted from “Count every newborn; a measurement improvement roadmap for coverage data” Moxon, S.G., Ruysen, H., Kerber, K.J. et al. BMC Pregnancy Childbirth 15, S8 (2015). 20 https://doi.org/10.1186/1471-2393-15-S2-S8
      a Not currently routinely tracked at global level.
      b Indicator requiring additional testing to inform consistent measurement.
      The WHO also published quality standards for maternal and newborn health [
      • World Health Organization
      Standards for improving quality of maternal and newborn care in health facilities.
      ] and standards for improving the quality of care for small and sick newborns in health facilities [
      • ∗World Health Organization
      Standards for improving the quality of care for small and sick newborns in health facilities.
      ] (Table 2). These published standards are remarkable and bold. They combine a declaration of rights and a constitution of better care with identified high impact interventions and defined standardized outcomes. In addition to components of direct clinical care and data focused on the provision of care, the standards' scope includes QI and families’ experience of the care.
      Table 2WHO Standards for improving the quality of care for small and sick newborns in health facilities.
      StandardScopeNumber of Quality Statements
      1Evidence based practice42
      2Actionable information systems3
      3Functioning referral systems6
      4Effective communication and meaningful participation6
      5Respect, protection and fulfillment of newborn rights and preservation of dignity6
      6Emotional, psychosocial and developmental support5
      7Competent, motivated, empathetic multi-disciplinary human resources4
      8Essential physical resources for small and sick newborns6
      Adapted from Standards for improving the quality of care for small and sick newborns, WHO 2020.
      Human resources are nearly always limited and in high demand in LMIC settings. To that end, any data proposed for collection must be useful and important to the health care team, feasible to acquire, and valid. Data sets that are incomplete or have questionable definitions have little utility and may cause harm by consuming limited resources and by giving inaccurate representations. Most neonatal units have registers or logbooks developed by Ministries of Health to guide data capture for health management information systems (HMIS) and inform national metrics. Although these data items are captured at the patient level and are aggregated to the level of the facility and nationally, their quality can be variable and therefore limited for meaningful application in local QI initiatives [
      • Nshimyiryo A.
      • Kirk C.M.
      • Sauer S.M.
      • Ntawuyirusha E.
      • Muhire A.
      • Sayinzoga F.
      • et al.
      Health management information system (HMIS) data verification: a case study in four districts in Rwanda.
      ,
      • Ouedraogo M.
      • Kurji J.
      • Abebe L.
      • Labonté R.
      • Morankar S.
      • Bedru K.H.
      • et al.
      A quality assessment of health management information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia.
      ]. However, as routine data entry in these logbooks is an integral aspect of neonatal unit workflows, the value of additional or duplicative data collection must be carefully weighed against initiatives to improve the quality and use of existing data. Alternatively, initiatives to collect and improve patient level data that directly input to aggregate existing data systems such as HMIS, without duplication of data collection effort, remain an area for continued investment and innovation.

      4. Methods of data collection

      The methods and frequency for data collection should reflect the level of population or health service under consideration, the measures, and the available resources. This section will focus on data collected at the level of health facilities, with examples of intermittent and continuous data collection for input, process, and outcome measures.

      4.1 Input measures

      A periodic audit or observation tool could be used to assess availability of inputs to the health system. The Newborn Services Rapid Health Facility Assessment is a newborn-specific audit and health worker interview developed by the Interagency Newborn Indicators Technical Working Group to identify potential gaps in policy or implementation [
      Newborn Indicators Technical Working Group
      Newborn services rapid health facility assessment.
      ]. Availability of inputs can also be assessed on a continuous basis. For example, time intervals between stock outages can be measured over time with the goal of increasing the time between one stock outage and the next.

      4.2 Process measures

      Several innovative tools have been developed to assist an improvement team in establishing local baseline data and identify opportunities for improvement. The Service Performance Assessment (SPA) modules include observation protocols, patient exit interviews, and health worker interviews with the output of both input and process metrics at the facility level [
      • Moxon∗ S.G.
      • Ruysen H.
      • Kerber K.J.
      • Amouzou A.
      • Fournier S.
      • Grove J.
      • et al.
      Count every newborn; a measurement improvement roadmap for coverage data.
      ]. Process measures can also be assessed on a continuous basis, either included in an existing newborn register, a separate neonatal database, or a project-focused data collection tool in the context of a QI initiative. An example of continuous data collection of a process measure is recording whether a temperature is measured and its value within an hour of admission to a neonatal unit.

      4.3 Outcome measures

      A database that collects standardized items on all patients or a select high-risk group of patients such as very low birth weight infants may include multiple relevant clinical outcome metrics. As the denominator of this type of database is clearly defined, the incidence of these outcomes and its trend over time can be compared. Alternatively, informative data may be appreciated from intermittent audits. An example of an intermittent audit or sampling strategy for the outcome of admission hypothermia would be to ascertain the proportion of infants with an admission temperature less than 36.5° Celsius during a defined time interval. When repeated serially over time, an improvement team can track measures without the burden of continuous data collection.

      5. Data reporting

      The primary benefit of collecting quality measures is to understand performance over time, at the facility level and compared to other facilities or groups such as countries or regions. Benchmarking helps facilities appreciate if their rates of processes and outcomes are similar to or different from their peers.
      To turn data into action for improvement, reports generated from quality measures should be easy for clinicians to understand and to share with others. Data that are not reported should be avoided. Clinicians should share results with colleagues and leaders at their facilities and specialty groups. The specific mechanism for reporting — such as on a data dashboard, website, an app, a poster, or a printed report — will depend on the local context. Teams should consider the easiest way to share and understand data and encourage the use of the report by minimizing its complexity.
      Reporting should be timely and represent the current (or nearly current) reality. However, time intervals will differ based on the measure, purpose of the data collection, and necessity for inclusion of benchmarking within a single facility over time or compared to other facilities or groups. For example, a local quality improvement project to reduce admission hypothermia may require local weekly or real time data reporting as teams implement small tests of change. These data would be appropriately displayed in a run chart [
      • Perla R.J.
      • Provost L.P.
      • Murray S.K.
      The run chart: a simple analytical tool for learning from variation in healthcare processes.
      ], but special technology is not needed. The run chart can be drawn on paper, updated by hand, and hung in the unit so that everyone understands the team's progress. Alternatively, data visualization can be achieved digitally. Some data, particularly rare outcomes, are better suited to annual reporting because accumulating cases takes time and benchmarking more frequently would not be accurate.
      Reporting should offer context, with reminders about the populations that are represented and how measures are defined. In some settings, it may be important to remind data collectors and users why the data are being collected in the first place. Data collected for the purpose of quality improvement needs to be honest and reflect the actual status of care delivery, not an idealized version of it. Over time, data use often improves data quality [
      • Bing M.
      • Abel R.L.
      • Pendergrass P.
      • Sabharwal K.
      • McCauley C.
      Data used to improve quality of health care.
      ,
      • Bensley R.P.
      • Beck A.W.
      Using the vascular quality initiative to improve quality of care and patient outcomes for vascular surgery patients.
      ].

      6. From data collection to data use; examples from four LMIC projects

      The transformation from data collection to data use is illustrated by several data system examples from LMICs below. In each case neonatal team members identified aims and measures important to them. Local providers helped develop the data systems. Data were immediately available for review and reflection, and multidisciplinary teams were involved in the improvement efforts.

      6.1 The Safety Cross – a local, low cost, low tech, data driven QI project with real-time reporting that engages families and multi-disciplinary team members

      Late onset neonatal sepsis, a nosocomial infection, is a devastating condition with a high morbidity and mortality. It incurs high costs and is largely preventable. Clinical leadership at Groote Schuur Hospital, a public hospital in Cape Town, South Africa, identified infection prevention and control as a priority intervention to improve outcomes and reduce costs. The team instituted a bundle of care with protocols, education, and a culture of assisting everyone in the unit to practice good hand hygiene. A clinician on the unit records all cases of late onset sepsis as soon as a blood culture is flagged positive. Cases are immediately displayed pictorially on a Safety Cross.
      A Safety Cross is a diagram in the shape of a cross with 31 blocks, each one to represent a day of the month (Fig. 2). On every day, the block is completed to display whether there were zero cases of infection (green card), one case of infection (red card), or more than one case of infection (striped red card). The Safety Cross is displayed prominently and the process well understood by everyone in the unit, including parents and non-clinicians. Copies of previous months’ outcomes are also available to observe trends.
      Fig. 2
      Fig. 2The safety cross at groote schuur hospital neonatal unit, cape town.
      The Safety Cross is the face of the infection control bundle, which reduced infections from approximately ten infections per month to only one or two per month over a three-year period [
      • Raban M.S.
      • Bamford C.
      • Joolay Y.
      • Harrison M.C.
      Impact of an educational intervention and clinical performance dashboard on neonatal bloodstream infections.
      ]. As a data and QI system the project has many strengths. It is a low cost and low technology initiative. The data are collected and displayed in real time, allowing early identification of outbreaks and immediate action. The data are displayed publicly and available to parents who play an important role in infection prevention and are key stakeholders in the quality of the unit. Whilst the Safety Cross was used for an infection prevention and control initiative in this instance, this type of reporting and display can be used for monitoring any adverse outcomes, for examples cases of hypoxic ischaemic encephalopathy or critical stock outages.

      6.2 NeoTree: a digital data capture and QI system for frontline HCWs

      NeoTree is a digital system aiming to improve quality of care and neonatal survival through tablet-based data capture, clinical decision support, embedded behavioral science driven QI, and feedback of data to dashboards and national aggregate data systems. It has been and continues to be co-developed with HCWs in Bangladesh (2014–2015), Malawi (since 2016), and Zimbabwe (since 2018). It reports data on newborns and stillbirths. Clinical decision and management support is tailored upon implementation to the human and staffing resources available in each facility.
      HCWs, typically nursing staff, use the app at the bedside to admit and discharge each neonate. As they complete an admission, they receive prompts to respond appropriately to the data they have entered and manage patients according to evidence-based guidelines. Co-development of data capture on admission, embedded education in basic newborn care, and clinical decision support was completed in Zomba, Malawi where these functions were found to be highly useable, feasible, and acceptable to HCWs following implementation on the newborn care ward [
      • Crehan C.
      • Kesler E.
      • Nambiar B.
      • Dube Q.
      • Lufesi N.
      • Giaccone M.
      • et al.
      The NeoTree application: developing an integrated mHealth solution to improve quality of newborn care and survival in a district hospital in Malawi.
      ,
      • Crehan C.
      • Kesler E.
      • Chikomoni I.A.
      • Sun K.
      • Dube Q.
      • Lakhanpaul M.
      • et al.
      Admissions to a low-resource neonatal unit in Malawi using a mobile app: digital Perinatal Outcome Audit.
      ].
      In November 2018, NeoTree was further co-developed and implemented as a pilot QI tool in Sally Mugabe Hospital in Harare, Zimbabwe, replacing all paper admission and discharge forms. In addition to discharge data capture and linkage with admission data, a NeoLab function was also developed, allowing online feedback of results from laboratory to the neonatal ward. As a result of the NeoLab function, median time for the reporting of blood culture results improved from 6 days to 3 days. At the same time, the percentage of infants discharged on oral antibiotics fell from 97% to 0% [
      • Chimhini G.
      • Chimhuya S.
      • Madzudzo L.
      • Heys M.
      • Crehan C.
      • Robertson V.
      • et al.
      Auditing use of antibiotics in Zimbabwean neonates.
      ]. This decrease was made possible through easy data capture, frequent audits, and feedback to all HCWs. Expanded co-development has focused on linkage to national data systems in Zimbabwe, bilateral data sharing with the national electronic health care records system, and expansion to a second Zimbabwean hospital, Chinhoyi Provincial Hospital. Challenges have included change in HCW capacity due to job action, communicating the purpose of technology to families in order to maximise acceptability, communicating across clinical and technology teams to translate user needs to software code and architecture, procurement of blood cultures with which to validate the sepsis diagnostic algorithm, hardware management for local data dashboards, and balancing the sometimes competing requirements of national digital systems with local clinical quality improvement. Future steps are continued co-development and implementation evaluation of non-emergency clinical decision support, data dashboard feedback loops in Zimbabwe, and linkage to aggregate national data systems (DHISv2). Once functionalities have been fully developed (anticipated Spring 2022), a cluster stepped wedge trial is planned.

      6.3 Perinatal Problem Identification Programme (PPIP): a national programme of amalgamating perinatal death audit findings

      In South Africa, neonatal deaths are reported relatively reliably by the national health department through the district health information system (DHIS). However, the DHIS gives little additional information about these deaths, such as causes. This gap is filled by the Perinatal Problem Identification Programme (PPIP), a comprehensive data collection system developed by the University of Pretoria for stillbirths and neonatal deaths [].
      Initially developed as a single-facility based audit tool, PPIP has expanded to all 52 districts in South Africa; 77% of the DHIS total reported deaths are now captured by PPIP, largely from the government sector [
      • Rhoda N.
      • Velaphi S.
      • Gebhardt G.S.
      • Kauchali S.
      • Barron P.
      Reducing neonatal deaths in South Africa: progress and challenges.
      ,
      • Rhoda N.R.
      • Greenfield D.
      • Muller M.
      • Prinsloo R.
      • Pattinson R.C.
      • Kauchali S.
      • et al.
      Experiences with perinatal death reviews in South Africa--the Perinatal Problem Identification Programme: scaling up from programme to province to country.
      ]. The project is being introduced in other countries such as Zambia, Uganda, and Lesotho. PPIP data has been incorporated into national planning, identifying interventions such as facility improvement, equipment availability, staffing and staff training. Triennial “Saving Babies” reports are published [
      National Perinatal MorbidityMortality Committee
      Saving babies: triennial report on perinatal mortality in South Africa.
      ].
      The backbone of PPIP remains the perinatal audit of deaths at regular compulsory morbidity and mortality meetings where the responsible doctor or nurse codes and enters the data in real-time following the confidential inquiry. This system allows for important and accurate data at the facility level, but as deaths that occur at home or outside the health facility are rarely included or registered by parents, a significant data and health service delivery gap results nationally when data is aggregated from all participating sites.
      The causes of stillbirths and neonatal deaths are recorded with early neonatal deaths receiving both a primary obstetric cause and final neonatal cause. Each is further categorized as an avoidable or unavoidable death. The avoidable deaths are grouped into patient, medical, personnel or administrative-related deaths and coded accordingly on the data collection form. Avoidable factors linked to each death are identified and discussed (Table 3). These pages are batched and sent monthly to the district PPIP coordinator who enters the data onto the electronic system. There are built-in data validity checks.
      Table 3Top ten probably avoidable healthcare-system-related factors and numbers of deaths, 2014–2015 (PPIP) (n = 3105).
      Modifiable factorn
      Inadequate facilities/equipment in neonatal unit/nursery617
      Nosocomial infection423
      Fetal distress not detected intrapartum; fetus monitored417
      Delay in referring patient for secondary/tertiary treatment337
      No accessible neonatal ICU bed with ventilator306
      Neonatal care: management plan inadequate288
      Neonatal care: inadequate monitoring254
      Insufficient nurses on duty to manage the patient adequately167
      Lack of transport – home to institution148
      Lack of transport – institution to institution148
      Adapted from Rhoda N, Velaphi S, Gebhardt G, Kauchali S, Barron P. Reducing neonatal deaths in South Africa: Progress and challenges. South African Medical Journal. 2018; 108(3):9–16.
      The PPIP system is not without challenges. Every unit needs leaders to champion the process and promote a no-blame culture to ensure transparency. Not all districts have good quality data with lesser resourced areas having both more neonatal deaths and less capacity to submit data.

      6.4 The Global Neonatal Database: a multi-institution neonatal network developed with and for units in low income settings

      Vermont Oxford Network (VON), a non-profit, voluntary collaboration dedicated to improving the quality, safety, and value of neonatal intensive care, has over 1,300 member neonatal units in 32 countries worldwide, predominantly in high income settings. The annual cost and administrative human resource inputs required to participate fully in VON membership are a barrier to many neonatal units in LMIC settings. In South Africa, better financed private neonatal units partnered with government-funded units offering expertise, financial, and administrative support which has allowed many public government units to join VON and participate in the database for the high risk neonatal intensive care unit population of very low birth weight infants as well as QI education. Some South African units have been able to utilize the database to improve outcomes and produce novel valuable research [
      • Gibbs L.
      • Tooke L.
      • Harrison M.C.
      Short-term outcomes of inborn v. outborn very-low-birth-weight neonates (<1 500 g) in the neonatal nursery at Groote Schuur Hospital, Cape Town, South Africa.
      ,
      • Riemer L.J.
      • Le Roux S.M.
      • Harrison M.C.
      • Tooke L.
      Short-term outcomes of HIV-exposed and HIV-unexposed preterm, very low birthweight neonates: a longitudinal, hospital-based study.
      ,
      • Van Wyk L.
      • Tooke L.
      • Dippenaar R.
      • Rhoda N.
      • Lloyd L.
      • Holgate S.
      • et al.
      Optimal ventilation and surfactant therapy in very-low-birth-weight infants in resource-restricted regions.
      ]. The partnership has been mutually beneficial as neonatologists in the public sector have been able to assist with both data interpretation and improvement in the private hospital groups.
      For Ethiopia, a different approach was developed. After more than a decade of partnering in Ethiopia on capacity-building and system-strengthening initiatives, at the request of Ethiopian colleagues, VON developed the patient-level Global Neonatal Database. Given the three main causes of neonatal mortality — intrapartum-related/birth asphyxia, infection, and complications of prematurity [
      • United Nations
      The millennium development goals Report 2015.
      ] — all neonates admitted to the neonatal intensive care unit are included in the database. Recognizing the human resource constraints and appropriate priority of patient care over data entry, VON and Ethiopian partners developed a one-page data collection form with indicators of highest yield for describing the neonatal population and care practices appropriate for benchmarking and QI in a low-income country neonatal unit. Data are collected in a custom data entry platform hosted at Vermont Oxford Network utilizing the REDCap mobile app with offline data collection capabilities due to challenges with internet reliability [
      • Harris P.A.
      • Taylor R.
      • Thielke R.
      • Payne J.
      • Gonzalez N.
      • Conde J.G.
      Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support.
      ,
      • Harris P.A.
      • Taylor R.
      • Minor B.L.
      • Elliott V.
      • Fernandez M.
      • O'Neal L.
      • et al.
      The REDCap consortium: building an international community of software platform partners.
      ].
      The VON Global Neonatal Database was pilot tested at one hospital in Addis Ababa in 2017 with acceptance and enthusiasm. In partnership with the Ethiopian Pediatrics Society and with the support of the Ethiopian Federal Ministry of Health, the Ethiopian Neonatal Network (ENN) was established in 2018 as twenty public neonatal units adopted the VON Global Neonatal Database. All teams have real-time access to their hospitals’ database electronically and receive annual reports with benchmarking of their hospital metrics to the aggregated ENN rates. Key elements for initial success included in-kind support of ENN activities by VON, establishment of neonatal QI teams with a lead physician and nurse at each hospital, serial face-to-face meetings of all ENN teams to foster networking, collaboration and development of a neonatal community, parallel introduction of the neonatal database with foundational and facilitated QI education, and small-group collaborative quality improvement exercises for teams with similar aims. Key elements for continued success include continued mentored QI education and coaching, strong partnerships with national leadership, willingness to adapt and desire to incorporate ENN activities into national neonatal strategic planning, and facilitated professional development of Ethiopian ENN leaders. Challenges moving forward include scalability of QI education and coaching, incorporation of research funding into plans for sustainability, and a continued search for a more ideal data entry platform that minimizes duplicative efforts while syncing with national HMIS.

      7. Conclusion

      Whilst the path to collecting facility-level data in an LMIC setting can seem daunting, much progress has been made, and future projects can build on these successes and lessons learned. The ENAP and the WHO have suggested what data needs to be collected at a national and global level; more work is needed at the facility level. Most importantly, any data collection project must be tailored to the local situation and mindful of resources, processes, and outcomes. LMICs are not a homogenous entity, and their data requirements and administrative resources differ.
      Data collection is a means to an end: to use that data to improve quality of care and reduce preventable neonatal morbidities and mortality. Any data capture initiative must therefore be integrated into a continuous QI process with ease of use for local HCWs. As with any intervention in medicine, implementing a data system has potential benefits but also very real risks. The consequences of using precious human resources for data entry must be recognized if that is coming at the expense of patient care in a neonatal unit or if the effort results in incomplete or unreliable data that discourages successful small improvement efforts. Facilities should prioritize capturing basic data such as mortality and birth weights accurately and completely before investing resources and effort in more complex projects. If a facility has obvious and fundamental deficiencies in core areas such as resuscitation, thermal control, infection prevention and control, it is important to address and evaluate these problems before progressing to less urgent aspects of care.
      Electronic health records and multi-facility networks are already being used with good effect in resource-limited settings and offer tantalising insights into the potential of databases for QI in LMICs. However, they are still mostly being used within research settings or with outside support. Whether they can be sustainably rolled out at large scale remains to be seen. Relevant data items and outcomes, such as sepsis, respiratory distress syndrome, and hypoxic ischemic encephalopathy need standard definitions appropriate to local resources. Establishing these definitions to facilitate benchmarking and coordinated improvement efforts in LMICs is a priority.
      The future of neonatal care and the use of data in resource-limited settings will be bright if planning and development includes or is led by LMIC clinicians and experts. The Sustainable Development Goals are unlikely to be reached without this recognition and investment.

      References

        • ∗Institute of Medicine (U.S.)
        Committee on quality of health care in America. Crossing the quality chasm.
        National Academies Press, Washington, D.C.2011
        • Hug L.
        • Sharrow D.
        • You D.
        • Hereward M.
        • Zhang Y.
        Levels and trends in child mortality report 2019.
        United Nations Children’s Fund, New York, NY2019
        • Bhutta Z.A.
        • Das J.K.
        • Bahl R.
        • Lawn J.E.
        • Salam R.A.
        • Paul V.K.
        • et al.
        Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost?.
        Lancet Lond Engl. 2014; 384: 347-370https://doi.org/10.1016/S0140-6736(14)60792-3
        • Dickson K.E.
        • Kinney M.V.
        • Moxon S.G.
        • Ashton J.
        • Zaka N.
        • Simen-Kapeu A.
        • et al.
        Scaling up quality care for mothers and newborns around the time of birth: an overview of methods and analyses of intervention-specific bottlenecks and solutions.
        BMC Pregnancy Childbirth. 2015; 15https://doi.org/10.1186/1471-2393-15-S2-S1
        • Requejo J.H.
        • Bryce J.
        • Barros A.J.D.
        • Berman P.
        • Bhutta Z.
        • Chopra M.
        • et al.
        Countdown to 2015 and beyond: fulfilling the health agenda for women and children.
        Lancet Lond Engl. 2015; 385: 466-476https://doi.org/10.1016/S0140-6736(14)60925-9
        • Gage A.D.
        • Kruk M.E.
        • Girma T.
        • Lemango E.T.
        The know-do gap in sick child care in Ethiopia.
        PloS One. 2018; 13e0208898https://doi.org/10.1371/journal.pone.0208898
        • ∗Austin A.
        • Langer A.
        • Salam R.A.
        • Lassi Z.S.
        • Das J.K.
        • Bhutta Z.A.
        Approaches to improve the quality of maternal and newborn health care: an overview of the evidence.
        Reprod Health. 2014; 11: S1https://doi.org/10.1186/1742-4755-11-S2-S1
        • World Health Organization
        Monitoring the building blocks of health systems: a handbook of indicators and their management strategies.
        World Health Organization, Geneva, Switzerland2010
        • AbouZahr C.
        • Boerma T.
        Health information systems: the foundations of public health.
        Bull World Health Organ. 2005; 83 (S0042-96862005000800010): 578-583
        • Leonard E.
        • de Kock I.
        • Bam W.
        Barriers and facilitators to implementing evidence-based health innovations in low- and middle-income countries: a systematic literature review.
        Eval Progr Plann. 2020; 82: 101832https://doi.org/10.1016/j.evalprogplan.2020.101832
        • Health Metrics Network, World Health Organization
        Framework and standards for country health information systems.
        second ed. World Health Organization, Geneva, Switzerland2012
        • Vander Schaaf E.B.
        • Cornett A.C.
        • Randolph G.D.
        Improving culture, one quality improvement project at a time.
        J Public Health Manag Pract JPHMP. 2018; 24: 57-62https://doi.org/10.1097/PHH.0000000000000554
        • Theobald S.
        • Brandes N.
        • Gyapong M.
        • El-Saharty S.
        • Proctor E.
        • Diaz T.
        • et al.
        Implementation research: new imperatives and opportunities in global health.
        Lancet Lond Engl. 2018; 392: 2214-2228https://doi.org/10.1016/S0140-6736(18)32205-0
        • United Nations Department of Economic and Social Affairs
        Sustainable development goals n.d.
        https://sdgs.un.org/goals
        Date accessed: November 12, 2020
        • ∗Donabedian A.
        Evaluating the quality of medical care.
        Milbank Q. 2005; 83: 691-729https://doi.org/10.1111/j.1468-0009.2005.00397.x
        • ∗Lawn J.E.
        • Blencowe H.
        • Oza S.
        • You D.
        • Lee A.C.C.
        • Waiswa P.
        • et al.
        Every newborn: progress, priorities, and potential beyond survival.
        Lancet Lond Engl. 2014; 384: 189-205https://doi.org/10.1016/S0140-6736(14)60496-7
        • World Health Organization
        Making every baby count: audit and review of stillbirths and neonatal deaths.
        World Health Organization, Geneva, Switzerland2016
        • ∗World Health Organization
        • UNICEF. Every Newborn
        An action plan to end preventable deaths.
        World Health Organization, Geneva, Switzerland2014
        • Mason E.
        • McDougall L.
        • Lawn J.E.
        • Gupta A.
        • Claeson M.
        • Pillay Y.
        • et al.
        From evidence to action to deliver a healthy start for the next generation.
        Lancet Lond Engl. 2014; 384: 455-467https://doi.org/10.1016/S0140-6736(14)60750-9
        • Moxon∗ S.G.
        • Ruysen H.
        • Kerber K.J.
        • Amouzou A.
        • Fournier S.
        • Grove J.
        • et al.
        Count every newborn; a measurement improvement roadmap for coverage data.
        BMC Pregnancy Childbirth. 2015; 15: S8https://doi.org/10.1186/1471-2393-15-S2-S8
        • World Health Organization
        Standards for improving quality of maternal and newborn care in health facilities.
        World Health Organization, Geneva, Switzerland2016
        • ∗World Health Organization
        Standards for improving the quality of care for small and sick newborns in health facilities.
        World Health Organization, Geneva, Switzerland2020
        • Nshimyiryo A.
        • Kirk C.M.
        • Sauer S.M.
        • Ntawuyirusha E.
        • Muhire A.
        • Sayinzoga F.
        • et al.
        Health management information system (HMIS) data verification: a case study in four districts in Rwanda.
        PloS One. 2020; 15e0235823https://doi.org/10.1371/journal.pone.0235823
        • Ouedraogo M.
        • Kurji J.
        • Abebe L.
        • Labonté R.
        • Morankar S.
        • Bedru K.H.
        • et al.
        A quality assessment of health management information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia.
        PloS One. 2019; 14e0213600https://doi.org/10.1371/journal.pone.0213600
        • Newborn Indicators Technical Working Group
        Newborn services rapid health facility assessment.
        Save the Children, London, United Kingdom2012
        • Perla R.J.
        • Provost L.P.
        • Murray S.K.
        The run chart: a simple analytical tool for learning from variation in healthcare processes.
        BMJ Qual Saf. 2011; 20: 46-51https://doi.org/10.1136/bmjqs.2009.037895
        • Bing M.
        • Abel R.L.
        • Pendergrass P.
        • Sabharwal K.
        • McCauley C.
        Data used to improve quality of health care.
        Tex Med. 2000; 96: 75-79
        • Bensley R.P.
        • Beck A.W.
        Using the vascular quality initiative to improve quality of care and patient outcomes for vascular surgery patients.
        Semin Vasc Surg. 2015; 28: 97-102https://doi.org/10.1053/j.semvascsurg.2015.09.003
        • Raban M.S.
        • Bamford C.
        • Joolay Y.
        • Harrison M.C.
        Impact of an educational intervention and clinical performance dashboard on neonatal bloodstream infections.
        South Afr Med J Suid-Afr Tydskr Vir Geneeskd. 2015; 105: 564-566https://doi.org/10.7196/SAMJnew.7764
        • Crehan C.
        • Kesler E.
        • Nambiar B.
        • Dube Q.
        • Lufesi N.
        • Giaccone M.
        • et al.
        The NeoTree application: developing an integrated mHealth solution to improve quality of newborn care and survival in a district hospital in Malawi.
        BMJ Glob Health. 2019; 4e000860https://doi.org/10.1136/bmjgh-2018-000860
        • Crehan C.
        • Kesler E.
        • Chikomoni I.A.
        • Sun K.
        • Dube Q.
        • Lakhanpaul M.
        • et al.
        Admissions to a low-resource neonatal unit in Malawi using a mobile app: digital Perinatal Outcome Audit.
        JMIR MHealth UHealth. 2020; 8e16485https://doi.org/10.2196/16485
        • Chimhini G.
        • Chimhuya S.
        • Madzudzo L.
        • Heys M.
        • Crehan C.
        • Robertson V.
        • et al.
        Auditing use of antibiotics in Zimbabwean neonates.
        Infect Prev Pract. 2020; 2: 100046https://doi.org/10.1016/j.infpip.2020.100046
      1. The perinatal problem identification programme n.d.
        • Rhoda N.
        • Velaphi S.
        • Gebhardt G.S.
        • Kauchali S.
        • Barron P.
        Reducing neonatal deaths in South Africa: progress and challenges.
        S Afr Med J. 2018; 108: s9-16https://doi.org/10.7196/SAMJ.2017.v108i3b.12804
        • Rhoda N.R.
        • Greenfield D.
        • Muller M.
        • Prinsloo R.
        • Pattinson R.C.
        • Kauchali S.
        • et al.
        Experiences with perinatal death reviews in South Africa--the Perinatal Problem Identification Programme: scaling up from programme to province to country.
        BJOG Int J Obstet Gynaecol. 2014; 121: 160-166https://doi.org/10.1111/1471-0528.12997
        • National Perinatal Morbidity
        • Mortality Committee
        Saving babies: triennial report on perinatal mortality in South Africa.
        2014-2016 ([n.d])
        • Gibbs L.
        • Tooke L.
        • Harrison M.C.
        Short-term outcomes of inborn v. outborn very-low-birth-weight neonates (<1 500 g) in the neonatal nursery at Groote Schuur Hospital, Cape Town, South Africa.
        South Afr Med J Suid-Afr Tydskr Vir Geneeskd. 2017; 107 (900–3)https://doi.org/10.7196/SAMJ.2017.v107i10.12463
        • Riemer L.J.
        • Le Roux S.M.
        • Harrison M.C.
        • Tooke L.
        Short-term outcomes of HIV-exposed and HIV-unexposed preterm, very low birthweight neonates: a longitudinal, hospital-based study.
        J Perinatol Off J Calif Perinat Assoc. 2020; 40: 445-455https://doi.org/10.1038/s41372-019-0541-4
        • Van Wyk L.
        • Tooke L.
        • Dippenaar R.
        • Rhoda N.
        • Lloyd L.
        • Holgate S.
        • et al.
        Optimal ventilation and surfactant therapy in very-low-birth-weight infants in resource-restricted regions.
        Neonatology. 2020; 117: 217-224https://doi.org/10.1159/000506987
        • United Nations
        The millennium development goals Report 2015.
        United Nations, New York, NY2017
        • Harris P.A.
        • Taylor R.
        • Thielke R.
        • Payne J.
        • Gonzalez N.
        • Conde J.G.
        Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inf. 2009; 42: 377-381https://doi.org/10.1016/j.jbi.2008.08.010
        • Harris P.A.
        • Taylor R.
        • Minor B.L.
        • Elliott V.
        • Fernandez M.
        • O'Neal L.
        • et al.
        The REDCap consortium: building an international community of software platform partners.
        J Biomed Inf. 2019; 95: 103208https://doi.org/10.1016/j.jbi.2019.103208