Identifying Major Health-System Challenges in Developing Countries Using PERs: Equity is the Elephant in the Room

ABSTRACT Despite an unprecedented increase in official development assistance to health in the last 25 years, there is no systematic way to assess dominant patterns in health-system challenges and opportunities in developing countries. Developing a new global instrument for and by donors and development partners would be resource-intensive and cumbersome. In this article, we demonstrate that Public Expenditure Reviews (PERs) can be used to reveal such patterns. PERs are analytical reports financed and conducted by the World Bank that have been used for years to identify and prioritize country-specific health sector reform needs. In order to extend their use beyond the country level, a reading instrument is developed in the form of a questionnaire to systematically identify the different themes addressed in each PER. All PERs published over a period of ten years are reviewed for health sector content. A new database is created with data on 70 PERs, spanning 61 countries. Analysis of the data reveals dominant themes globally, patterns across development levels, and some regional variations. Our main finding is that issues related to equity strongly dominate and are relevant across all regions and income groups. In addition, the article highlights the usefulness of PERs beyond providing country-specific information. Without losing the country-focus and flexibility of PERs, thoughtful and minor investments in how Health PERs are conducted can create a relatively cheap and strongly operational instrument for building global knowledge bases on health sector needs and challenges.


Introduction
A 2016 commentary in this Journal 1 raised a concern that heavy donor investments in expensive and hard to conduct National Health Accounts (NHA) in low-and middleincome countries may have displaced attention from cheaper and more policy friendly Public Expenditure Reviews (PERs). 1 This article does not oppose NHA and PERs that can be seen as competing for resources but are also complementary; rather, it draws attention to the fact that existing investments in PERs can, in fact, be exploited further to support knowledge investment in the global health community. NHA, because they are meant to generate a full accounting of health financing and expenditure, including from private sources, require expensive data collection in addition to what is typically produced by governments; but the additional data produce limited value to policy makers. PERs, on the other hand, focus mainly on public spending that directly reflects public policies and choices; they rely on a small subset of NHA data that are systematically collected yearly, following international standards. Data used in PERs are generally reliable, since budget and public expenditure accounts are typically subject to external review in developing countries. Consequently, PERs do not depend on heavy data collection efforts, rest on good quality data, are timely, repeatable, and policy friendly. Finally, it is worth noting that PERs for the health sector and other sectors continue to be used by the World Bank and other agencies making the analysis from the 10 years covered in this article highly relevant moving forward.
This paper uses a systematic review of ten years' worth of publicly available PERs to highlight their usefulness beyond providing information at the individual country level and to reveal dominant health system challenges that are common across large groups of countries. Health system challenges are classified into broad categories related to funding, institutions, organization of the health-sector and health-care provision. Note that the focus here is on health system challenges, not on the specific private or public health issues that underlie the need to address systemic issues. In addition to identifying major health system challenges across countries, the article addresses how future actions around PERs can improve future global knowledge. First, the global themes identified in the analysis can serve as targeted knowledge agenda for academic institutions and development agencies. Global themes require a global effort and this can be used to harmonize efforts in the development community. Second, while PERs are primarily designed to serve country-specific needs, it is conceivable that with some minor efforts to create a menu of topics and to refine the reading instrument, they can increasingly be used to look at global trends and regional patterns. This would represent a far cheaper and more operationally sustainable approach to building health system knowledge than relying on one-time crosscountry surveys or a new standalone instrument.
The article is organized as follows: the Background section includes a short review of the data available at the international level-to provide evidence of the knowledge gap faced by the international community in terms of understanding dominant health-system challenges in developing countries-and gives a summary description of what PERs are intended to cover. The next section describes how we used PERs to provide information about patterns of health system challenges across developing countries and describes the resulting database. The section on Findings reports on dominant themes at the global level-relevant across different types of developing countries-followed by differences across regions and levels of income. The concluding section summarizes results and proposes a way to increase the potential for using country-specific PERs as an instrument for comparative health systems analysis for low and middle-income countries.

Review of Existing Data: More Data Produced Globally but Still No Systematic Way to Identify Priorities for Developing Countries
Efforts to systematically understand cross-country health systems challenges and reforms globally are complicated by the fact that tools are lacking to assess needs globally. What is now available, thanks in large part to the World Health Organization (WHO) and a few other agencies, is an improved set of databases focused on national averages for a range of health outcomes (e.g., life expectancy at birth, infant and child mortality), health expenditures (Global Health Expenditure Database), and a range of health sector inputs (e.g., health care providers and facilities). 2 Other database efforts of note that extend beyond the more basic inputs and outputs are by the Institute of Health Metrics and Evaluation (IHME) and the World Bank Group. IHME has built on the WHO databases in a few ways, including estimation of some difficult to measure outcome variables such as maternal mortality, and has extended the data on burden of disease. 3 Twenty years ago, the World Bank started making available national and subnational-level data by wealth quintile to address issues of wealth-based inequalities in basic health outcomes and outputs. 4 The data described above have proven very useful to test hypotheses, monitor progress and identify cross-country heterogeneity in health performance in select areas of interest. Such data, however, cannot be used to identify the type of analyses most needed. a In particular, they cannot take on the very challenging problem of looking at and comparing health sectors from a systems perspective with a focus on what reforms may be needed. The Organization for Economic Cooperation and Development (OECD) and WHO's European Observatory have produced some data that can be used for that purpose but the efforts focus on upper income countries and some upper middle-income countries, mostly in Europe; they are not easily reproducible in most developing countries due to resource and data constraints as well as greater heterogeneity across countries. 5,6 Beyond that, and particularly for lower income countries, there is currently no systematic way to review patterns or global themes and challenges to health systems. If some attempts were made to identify major challenges, they relied on one-time specific surveys or brainstorming approaches and spanned a limited set of countries or a specific region. 7,8

Typical Purpose and Content of PERs
The PER (sometimes called a Public Expenditure and Institutional Review-PEIR) is a multi-sectoral core diagnostic tool utilized by the World Bank to help client countries explore policy and budget choices, review the extent to which policy decisions reflect stated priorities, and more broadly, quantify the extent to which past policies and budget choices (including how they are put together and implemented) produce impact on efficiency, equity, and other measures of effectiveness. Importantly for this paper, the PER usually explores more nuanced policy questions within sectors. In the health sector, for example, it can reveal how allocations to the sector prioritize national programs and disease targets, geographic distribution relative to needs, levels of service delivery (primary, secondary and tertiary), and/or population groups. Because it is evidenced-based and focused on budget expenditures, the PER will also reveal decisions that policy makers implement regardless of what was noted in public statements and across a broad range of policies. Although it is not usually designed to capture the reasons why allocations are not aligned with policies, it can measure the extent to which they do or do not align. Overall, the PER presents a reality check and can point to areas where hard questions need to be asked.
As multi-sectoral instruments that are country focused, PERs typically cover a limited set of sectors as well as macro-fiscal challenges facing a country. In addition to the selection of specific sectors, the focus within sectors is flexible and reflects country-specific needs at the time. The PER focus within sectors will thus vary in terms of specific objective, depth of analysis, choice of analytical tools, and time-period covered. Depending on perceived needs, PERs may focus on efficiency (allocative and/or technical goals), cost containment (as many did during the recent global economic downturn), equity in public resource allocations, and the process of budgeting or budget transfers (typical of a PEIR). The depth of analysis and choice of analytical tools are typically driven by the availability of data and information systems. Finally, the time period analyzed is influenced by changes in public policies and political conditions and depends on constraints related to data availability, especially when more than one year is analyzed.
The systematic review of health sector coverage in PERs does not attempt to address all possible PER objectives; it focuses on topics that have potential to be relevant across different countries. In particular, no data are gathered on coverage related to gaps between stated policies and budget allocations, or on country-specific institutional constraints in budgeting and implementation. Instead, it focuses on the role of PERs as instruments to measure the importance of different policy challenges and on the tools used to analyze and offer policy advice.

Using PERs as a Tool to Identify Patterns across Groups of Countries
The main advantage of using PERs as a tool to identify major challenges in developing countries is that topics covered are country-specific but the format and methods used are mostly homogeneous across countries. Indeed, while principal topics to be treated in PERs are chosen in close discussions with national governments, the PER is a well-established diagnostic tool that is used across a large range of countries and must follow specific guidelines; in particular, all analysis contained must be based on hard evidence. Two central questions could be addressed by compiling information on country characteristics and health content of PERs: (i) are there dominant themes that may require global attention? we will indeed find that some themes are consistent across regions and development levels despite the extreme heterogeneity of country health systems in terms of resource availability, epidemiological and demographic profiles, as well as different capacity and governance profiles-, and (ii) can we detect patterns in health system performance/challenges that are different for different types of countries (e.g., regional clusters, levels of per capita income)?
It is important to stress that PERs are meant to serve a specific purpose for each country, namely understanding the country's constraints and challenges in the health sector and to provide advice for ways of improvement. This matters because topics proposed for analysis are likely influenced by what global issues are most "fashionable" at the time the PER is developed; in practice, however, after analyzing the evidence, the themes that end up being reported in published PERs are fully driven by the country-specific situation. We argue in this article that existing PERs can therefore be exploited to provide comparative analysis across countries, and that, with minimal investment in revising reading instruments and harmonizing the PER tool, it is possible to build an effective global database of health system challenges moving forward. The new PER database created for this article extends the value of the PERs by allowing the exploration of themes across countries. Since this new objective was not considered when the PERs reviewed were developed, findings are to be approached with some humility.

The Reading Instrument (Questionnaire)
In order to create a database suitable for analysis, it was important to combine a quantitative look with a qualitative dimension. This was done by creating an intelligent reading instrument that counted specific references to health sector challenges. The instrument was also designed to allow some measure of the intensity of references (e.g., the number of times and the different ways a policy challenge or recommendation was made).
The reading instrument was developed independently, prior to identifying and reviewing PERs. Indeed, it was important that the actual content of PERs not influence what topics were considered. The instrument was designed to be as complete as possible, keeping in mind themes potentially treated in an expenditure review. It was prepared by a senior health economist with indepth knowledge of health systems, and a clear understanding of the variety of themes that could be addressed using expenditure data to describe and evaluate health sectors and to formulate policy recommendations. During the systematic reading of PERs, if a topic appeared typically missing in the questionnaire, it was included in the closest category and a note was made in case it was found recurrent in other PERs. A few subcategories (e.g., personnel deployment) were added as they were found in several PERs but could not be placed in any other preestablished category. Table 1 lists the themes included in the questionnaire and describes the type of content included in each subcategory. The reader may consult the full reading instrument to better understand what was included in each category; it is available as supplementary material.
To ensure consistency and thoughtful reporting, a single senior researcher completed all the questionnaires. The filled-out questionnaires include both qualitative and quantitative information. The information could be used to assess the extent to which efficiency and equity questions are addressed and what types of policy recommendations are stressed. The questionnaires were also used to identify the most-cited links from the health sector to the general economy. All quantitative information was recorded in the database described below.

The Database
The PER database used in this paper was created by the authors in 2012/2013 as technical support for the Human Development Network's "Health and Economy" project of the World Bank. It was developed by (i) quickly reviewing all PERs publicly released over a 10-year period (details below), (ii) identifying the PERs where the health sector received significant attention, and (iii) systematically reviewing those, using the reading instrument described above. Publicly-released documents and reports of the World Bank are searchable and downloadable through their website. 9,b All 136 full documents classified as PERs that were available in the system at the time and with publication dates posterior to 2000, were screened for health content. Of the 136 PERs 70 were found to have enough health sector content to allow responding to the health questionnaire. These 70 PERs serve as a base for the analysis that follows; they span 61 countries and include four subnational states. Table 2 lists the 70 PERs by region and level of economic development.

Global Themes (Themes of Interest in a Substantial Majority of Developing Countries)
Despite the country-specific nature and focus of PERs and the extreme heterogeneity of health systems, the review of the 70 documents revealed that four general themes were addressed in analytical findings and/or policy recommendations in over three-fourth of the PERs. The top four themes (clockwise from top in Figure 1) were (i) the need to improve equity of health sector spending and target resources and services to Human resource issues (labor share, categories of personnel, deployment, etc) and, to a lesser extent, methods of procuring and pricing drugs, largely dominated the "other: health sector" category.
The full reading instrument is annexed as supplementary material. Source: Authors.
vulnerable populations (89% of PERs); (ii) the need to strengthen primary care services and outreach in the health sector (84% of PERs); (iii) the need to improve governance and deal more effectively with corruption in the health sector (83% of PERs); and (iv) the need to address institutional weaknesses, including fragmentation, in the health sector (77% of the PERs)-the different categories are described in Table 1.
As presented below, there are other themes and topics that vary by different country clustering, but these four themes are almost universal in nature and may reflect blind spots for health systems in low-and middle-income countries. It is clear from the above that equity and poverty targeting is the prominent health-system challenge for developing countries overall. Not only issues of equity are cited most often, but the second most cited topic concerns primary health care and outreach, themes that directly relate to the need to better serve poor populations. Weaknesses in primary care and outreach services are likely to negatively impact the poor more than the better-off making investments in strengthening such services not only a move to improving the more cost effective services but also to strengthening targeting of spending (level of care targeting) to services that are more likely to be accessed by the poor.
This finding-equity/poverty focused issues dominate all health-sector policy challenges in developing countries-is important as it can provide a useful target to galvanize global action. The same finding came across all geographic clustering and across the different levels of development (Figures 2 and 3). On the one hand, this is expected, given the overwhelming evidence on health outcome and health system output inequality in the last 20 years. 4,10-14 On the other hand, and despite increasing evidence on what works in addressing inequality, 15-20 the global health community and particularly the World Bank that produces these PERs, has mainly focused on efficiency issues, such as ensuring spending on cost-effective services. c,21-24 A strong message from this finding is the need at the global level to increase and sustain efforts for tracking and tackling inequalities in the health sector in support of low-and middle-income countries across the globe.
The other two dominant topics-governance/corruption and institutional coordination/fragmentation-also have something in common as they both focus on institutional factors and capacity and relate to the basic capacities of Ministries of Health to perform their functions. It should be noted that, while both topics appear in large numbers of PERs (about 80 percent of PERs include either or both), they are more dominant at lower income levels (e.g., governance/corruption issues, are addressed in 91% of low-income country PERs) which happens to be where most donor and development agencies focus their energy. It is hard, with the available data, to tease out the extent to which global development   Table 1  attention to these countries is driven by these weaknesses or is part of the cause. Regardless of the directionality of cause and effect, it might be worth exploring, in future research, the extent to which the health sector in developing countries has more challenges with corruption and fragmentation than other sectors. It should be noted that a number of issues that are prominent in the global health literature do not appear prominently in the PERs reviewed. This is the case for issues of drug procurement and pricing included in the D6 category (health provision-others). But the absence of discussion of some other topics in our results is also explained by the fact that the reading instrument was not designed to identify them separately from broader issues. For example, behavioral issues such as policies to reduce smoking were included in A2 (sources of health finance) and C2 (primary healthcare policy and outreach); access and population/public health issues were in C2, C1 (organization of health services) and C3 (equity/targeting); gender issues were included but not identified separately in C3. The reading instrument could be adjusted in future analysis to highlight specific concerns separately.

Patterns and Variations across Regions and Income Categories
As expected, many notable challenges to country health systems exhibited variations and patterns across regions and levels of national income. Some of these patterns are presented in this section. We start with differences across national income categories, followed by differences across the regional clusters most represented in the database. Figure 4 plots the list of challenges requiring action in the same order as Figure 1 but the new percentages are calculated by income level with differences between lines capturing variations in the three income clusters of low-income countries (LIC), lower-middle-income countries (LMIC) and upper-middle-income countries (UMIC).

Clustering by Levels of National Income (All Regions Included)
Looking at the three sets of challenges that relate to health financing, the low-versus middle-income differences are pronounced in an expected way. Challenges with the level of public spending on health follow a clear gradient against country level of income. Ninety percent of PERs for LICs found level of public financing to health to be too low and recommended a stronger budgetary commitment to the sector. While still high, it is 20 percentage points lower in LMICs; it drops to only 30% in UMICs. Given what is known about health spending, including from public sources, increasing with levels of national income, this finding makes a lot of sense. 25  Note: The axis measures the percentage of PERs with content related to a given health system challenge; the different categories of challenges are described in Table 1. Income categories as defined in Table 2.
The other two financing challenges (also in Figure 4) move in the opposite direction to level of spending and exhibit a stronger split between LICs versus middleincome countries (MICs = LMICs + UMICs). In particular, insurance and/or pooling issues came up in only 25% of PERs for LICs against 75% and 67% for LMICs and UMICs, respectively. Given the high out-of-pocket spending in LICs, one would expect pooling/insurance solutions to be important to LICs. One possible explanation that LICs do not focus their attention on such issues is that insurance mechanisms for health are complex and LICs face superior challenges in terms of capacity and governance. This may also reflect stronger interest in pursuing insurance systems in MICs.
Another clear pattern relates to the organization of the health sector, especially around levels of care. Challenges around how the health sector is organized and the linkages and balance across levels of care as well an inpatient and outpatient care are highlighted more in UMIC PERs (83%) than in LMICs (65%) and LICs (53%). As we see later when comparing regions, the main driver may not be the level of economic development in this case; given the dominance of this issue in ECA countries, which are largely UMICs, the difference may be regional.
The reverse order appears to be the case for the identification of governance/corruption challenges across PERs and income levels. The highest percentage of challenges in this category are LICs (90.63%) followed by LMICs (80%) and UMICs (72%). Similarly, the same pattern emerges around institutional coordination and fragmentation issues (81%, 80%, and 67% respectively for LICs, LMICs, and UMICs). Both of these categories of challenges are among the most prevalent, but deteriorate with level of national income.

Clustering by Regions
The total number of PERs with strong health sector content allows for three substantial regional clustering: 24 are in sub-Saharan Africa (AFR), 21 in Europe and Central Asia (ECA), and 14 in Latin America and the Caribbean (LAC). Although the database includes countries in the three other regions (South Asia, Middle East and North Africa, and East Asia and the Pacific) and these countries are included in the analysis above, there were not enough PERs in these regions to make meaningful inference.
When looking at differences across regions it is important to consider that levels of incomes are not equally represented in each region. Sub-Saharan Africa is dominantly LICs while ECA and LAC are dominantly MICs. When comparing AFR, ECA, and LAC regions ( Figure 5) some salient differences are indeed consistent with differences in levels of income noted above. In particular, consistent with the fact that challenges related to the level of financing tend to be much more relevant to LICs than MICs, it is not surprising that 92% of PERs in the AFR region note this challenge while only around 49% of PERs in ECA and 57% in LAC do. Also  Table 1. Source: Authors.
consistent with the earlier findings is the emphasis on pooling and insurance in the dominantly MIC regions, 71% in ECA and 79% in LAC, compared to only 25% in AFR. What comes out as unexpected when looking at regional clustering, however, is that the region that stands out most is ECA, not AFR as is often the case. As Figure 5 shows, six challenges are more often cited in ECA than other regions. The largest differences concern the HC facility-level efficiency category (ECA 90%, LAC 50% and AFR 42%), hospital policy (ECA 71%, LAC 35% and AFR 25%), and the need for investments and modernization (ECA 67%, LAC 29% and AFR 41%). A broad finding, therefore, is that regional differences exist regardless of the levels of national income. This has implications, discussed briefly in the next section, as to ways of setting up regional learning and South-South knowledge management systems.
When comparing the AFR region to non-African LICs ( Figure 6), we find that regional variations are much smaller than variations across level of income (presented in Figure 4). Across a number of dimensions, the variations between Africa and other LICs are minimal. There are, however, a few stand-out differences. The challenges with primary health care are high on the list in Africa (75%) but less than they are in LICs outside of Africa where they are universally noted (100%)-the difference remains when we exclude African MICs (79% cite PHC as a challenge). The biggest gap concerns challenges related to provider payment methods noted only in 50% of PERs in AFR against 83% in LICs outside of AFR. The direction of the gap is reversed for challenges related to sources of financing (67% in AFR versus 33% in other LICs). The other substantial gap concerns topics related to the quality of health care, more prevalent in PERs of non-African LICs that in AFR PERs.

Conclusions and a Way Forward
As the global health community explores areas for funding research and knowledge exchange, the analysis presented in this article offers important and useful information on the topics that require most attention as well as on ways to organize knowledge sharing. Despite the heterogeneity in health systems around the world and the flexibility in how PERs are applied and focused by the World Bank, clear areas of focus emerge.
Our systematic review of the PERs published in 2002-2011 revealed that issues related to equity-as described in Table 1-strongly dominated all other challenges to health systems. Topics in the equity category were of concern for almost all 61 countries covered and cutting across regions and levels of national income. This is in contrast with the dominance of efficacy-focused research (e.g., cost-effectiveness) in the global health literature and research agendas. The second global theme in the PERs, challenges in primary care and outreach, is Note: The axis measures the percentage of PERs with content related to a given health system challenge; the different categories are described in Table 1. Source: Authors.
also intimately linked, in a focused way, to poverty and inequality. The analysis also highlighted two other topics/ challenges relevant to most developing countries across all regions, namely governance/corruption and institutional challenges and fragmentation, challenges that also require research and knowledge sharing focus by global health partners. The review also identified patterns across national income and geography that can further inform knowledge investments and potentially identifying South-South platforms for exchange of knowledge experience. There are a number of ways that these variations and patterns can be used. The most obvious approach is to use the gradient by national income to help identify future challenges to be faced by countries currently at lower levels of income. This is not new given the use of high-income country research to clarify options in lower levels of development, but this can be further nuanced by looking at the patterns across UMICs, LMICs, and LICs. A second obvious approach is regional clustering where health systems share structures and governance systems and consequently face similar challenges. Regional agencies can be used to explore regional challenges and successes. A final approach, rarely used, would be to identify specific challenges faced by health systems across national income and geography and structure South-South knowledge platforms for sharing experiences (research and implementation). Finally, the analysis revealed that PERs, despite being country-focused and flexible analytical instruments, have the potential to be used to systematically assess important challenges faced by health systems in low-and middle-income countries. In a way, this exercise is a proof of concept for the use of PERs beyond their original intended purpose. A useful thought exercise, and a strong recommendation is then to identify ways future PERs can be strengthened through limited and targeted standardization to make this new use of the instrument even more effective. The reading instrument should be refined to include important categories that were not identified separately in this review. Given the importance of public health measures, a standalone Population and Public-Health Services category would be desirable; the questionnaire could also be refined and adapted to provide information on subcategories of potential global interest and to identify specific challenges of global importance, in particular those related to expenditures on Common Goods for Health d or specific issues within the equity realm such as gender. 28 In fact, it would be useful to have periodic discussions among global experts (WHO, The World Bank, and other agencies) of the different categories/ topics that one would expect to see in the review questionnaires. Guidelines could also be developed to promote the use of resources that will help create a menu of topics and increase comparability. In fact some efforts have already been extended, particularly in the last decade, to provide new standardized tools for PER data and analysis and more systematically perform some typical analyses. e Central to this endeavor is the need to achieve a balance between flexibility and harmonization. While a menu of topics and some level of standardization is desirable, it remains important for our purpose that countries continue to choose the topics that are most relevant to their situation and not try to systematically cover all topics suggested; they could choose in a menu of relevant health PER topics what best suits their needs, and possibly add topics that are of specific concern to them. Indeed, standardizing coverage would introduce bias in the identification of major global or regional challenges. Increased standardization of methods and tools (rather than content), however, would facilitate analysis across countries, without introducing such bias. Notes a. It is often the case that the quality of the data obtained in specific areas of study will determine what themes are analyzed, but the issues studied may not be the most pressing. b. The World Bank Documents and Reports retrieval site allows searches by criteria including keywords, topic, and type of document. PERs are classified in "Economic and Sector Work". The site is continuously updated and old documents may be added as they are digitalized so searches at different times may yield different numbers of documents available. As of October 5, 2020, there were 631 PER-type documents (some may be different volumes of the same PER) dating back to 1985 [search output: https://documents.worldbank.org/en/publica tion/documents-reports/documentlist?docty_key= 904591&srt=docdt&order=desc] c. Issues related to poverty and inequality have received increasing attention in the last 15 years, including part of the 10-year period covered in this review; the focus on efficiency, however has continued to be strong (with the underlying belief that there is no efficiency/equity trade off in health. 14 d. Common Goods for Health are defined as a cluster of feasible interventions exhibiting two fundamental characteristics: (i) market failures due to their public good nature or the large health externalities they generate; and, (ii) strong potential impact on human life. They are typically not provided or severely underprovided by the private sector. 28 e. For example, the World Bank has invested in fostering consistency and transparency in budget and expenditure data with the Boost effort that started in 2010-see https://www.org/en/programs/boost-portal.