The effects of market-based reforms on access to electricity in developing countries: a systematic review

Market-based reforms have been promoted over the past decades to improve the performance of the power sector. This systematic review assesses the effect of market-based reforms in developing countries on intermediate outcomes like technical efficiency and the resulting impacts on electricity access. Using a pool of 70 well-designed qualitative and quantitative studies, the review synthesizes impacts of private sector involvement, privatisation, liberalisation, and regulation. This mixed-methods approach detects only few andmostly weak effect patterns for reform types sufficiently evaluated in the primary literature. The qualitative synthesis further distils factors that likely contribute to successful electricity sector reforms as tentative guidance for coherent policy delivery. ARTICLE HISTORY Received 14 May 2019 Accepted 23 May 2019


Introduction
Lack of access to basic electricity services is still widespread in many developing countries (World Bank 2017). Overcoming this deficiency occupies a central place in today's global development agenda as evidenced by the Sustainable Development Goal 7 on affordable and clean energy for all. IEA (International Energy Agency) (2017) estimates that providing electricity for all by 2030 would require annual investment of $52 billion per year, more than twice the level mobilised under current and planned policies. Beyond additional infrastructure investments, strategies are required to improve the performance of the power sector across all stages of electricity provision, namely generation, transmission, and distribution. Sources of underperformance include excess transmission and distribution losses, overstaffing costs, bill collection failure, and underpricing (Bacon 2018), or, more generally, financial obstacles, lack of human capital and deficient technical capacity. The past literature often linked these problems to the state-owned and vertically integrated nature of utilities. A seemingly obvious (though often contested) remedy was the introduction of market-based reforms as spearheaded by the two pioneering countries, Chile and the United Kingdom, from the late 1970s on. These reforms generally encompass private sector involvement, privatisation of market players, liberalisation of electricity markets, or regulatory interventions such as changes in the pricing design. Today, the large majority of developing countries adopted such reforms, although selectively and by far not to the degree observed in the industrialized world. Reforms often stagnated at an intermediate stage or were partly even reversed (Hall and Nguyen 2017;Foster et al. 2017;Urpelainen and Yang 2019).
This systematic review brings together the accumulated evidence on the effectiveness of market-based electricity sector reforms in developing countries. In light of the abovementioned ambition of making electricity universally accessible, electricity access is the main outcome against which effectiveness is measured. At the same time, electricity access is indirectly affected by a number of intermediate outcomes such as electricity system efficiency, which are thoroughly assessed as well. The systematic review synthesizes both quantitative and qualitative evidence on this complex topic and extracts the main lessons. The analysis is based on a pool of 70 welldesigned qualitative and quantitative studies. The quantitative studies are assessed using standardized effect sizes and, depending on the homogeneity of included evidence, pooled effect sizes using meta-analysis and meta-regressions. Qualitative studies are analysed based on an iterative logic model approach, with a particular focus on mechanisms behind impacts observed in quantitative studies. Related to the macro-level nature of electricity sector reforms, the quantitative evidence mostly consists of panel studies, which is unusual for a systematic review. Among others, this review therefore relies on newly developed effect size standardization approaches.
Through its systematic assessment and synthesis of findings, this review particularly extends the work by Jamasb, Nepal, and Tilmisina (2017) and Bacon (2018) who narratively summarize the linkage between power sector reforms, economic and technical efficiency, and poverty reduction. 1 Contrary to the original planning laid out in the protocol of the review (Bensch et al. 2015), cost-effectiveness of the assessed market-based reforms is not covered in this review. This is simply due to the fact that no data on reform costs could be retrieved from the included studies, a challenge commonly observed with systematic reviews (Masset et al. 2018).
After developing a theoretical framework in the section on the Theory of change, I describe the systematic search and selection process (Study search and selection) before a brief summary of the included studies follows in the section on Study search and selection results. The Evidence synthesis presents the results, which are discussed and contextualized in the Conclusion section.

Theory of change
The focus of this systematic review is delineated in Figure 1, the logic model of the intervention: market-based reform interventions in the electricity sector in developing countries and their effect on the outcome (increased) access to electricity among households or other customer types. More specifically, the following potentially non-mutually exclusive intervention types are considered, some of which may be individual projects or elements of larger energy policy reform measures: • Privatisation, which refers to a change in utility ownership from governmental to private actors. • Liberalisation, i.e. the opening of the electricity market to competition. Previously vertically integrated energy markets may undergo unbundling of network services from other business fields as a prerequisite for common access to the network. Another main example is the introduction of a wholesale electricity market, which is the trading platform where competing generators offer their electricity output to retailers at the distribution stage. Finally, there are other competition-enhancing policies such as the free choice of suppliers. • Private sector involvement, where tasks previously handled exclusively by public utilities are now also offered by private entities. Particularly common in the electricity sector is private investment in generation, also referred to as Independent Power Producers (IPPs). • Regulation, which accompanies the abovementioned interventions to guarantee a level playing field, including for example changes in the electricity tariff design. • Decentralisation, where formerly centralized decision power is dispersed over various hierarchical, administrative levels.
The potential pathways and transmission channels between reform interventions and outcome are rather implicitly found in the literature. Zhang, Parker, and Kirkpatrick (2008) is one of the few studies that explicitly looks at electricity sector reforms in the context of improving access to electricity. One of the authors' hypotheses, for example, states that competition in the sector will lead to higher labour productivity and higher capacity utilisation. These hypotheses are included in a neutral, non-judgemental manner in the logic model in Figure 1. The model is generically sketched and will be enriched later on in this article based on the qualitative literature. The model also hints to the strong techno-economic and political complexities of the electricity sector: Intervention effects are likely to depend on the specific implementation mechanism that is applied in the reform, such as power purchase agreements. In addition, different types of moderators potentially codetermine effects; they include contextual factors such as the existing energy mix or the institutional capacities of regulatory authorities. Furthermore, the causal chain linking interventions and access to electricity implicitly depends on three overarching assumptions: First, more market-based structures and procedures on the supply side help to increase resource efficiency. Second, these efficiency gains are reinvested in the system by increasing electrification efforts. And third, potentially adverse effects of the interventions are mitigatede.g. the dampening effect of profit-orientation on investments in socially desirable expansions of supply. Against this background, net effects of reform measures are hard to predict.
Ultimate welfare outcomes presented at the bottom of Figure 1 are not the subject of this review, also given that their accomplishment again depends on a variety of demand and supply-side factors, first of all the actual use of electricity but also other factors like targeting, grid reliability, 2 and the productive use of electricity.

Study search and selection
Overall, this review was conducted according to the Campbell Collaboration Review Methods Guidance (Campbell Collaboration 2014) and accounting for the 3ie systematic review methods appraisal check (3ie (International Initiative for Impact Evaluation) 2011). The following summarizes these methodological procedures; further details can be found in Bensch et al. (2016). 3

Inclusion criteria
This mixed-methods review synthesizes quantitative and qualitative evidence, for which the following types of study designs and methods of analysis were eligible: As quantitative evidence, generally accepted higher-quality experimental and quasi-experimental causal inference designs were considered, ranging from experiments over panel data methods to interrupted time series designs. Simple before-after comparisons were allowed as long as any kind of statistical control was applied. By contrast, quantitative approaches based on 'non-factual' evidence were excluded, i.e. simulation or modelled benchmarking results from Computable General Equilibrium (CGE) designs, Data Envelope Analysis (DEA) or Stochastic Frontier Models.
As qualitative evidence, any approach based on factual evidence was eligible. Among others, this comprised institutional analyses and case studies. Institutional analyses generally analyse institutional actors, structures, and processes, or more generally factors and mechanisms influencing success and failure, in the form of (not necessarily counterfactual) assessments based on deep contextual knowledge. In line with Rehfuess et al. (2014), case studies are defined as assessments that rely on at least one source of empirical information and report information on the analysis. The qualitative evidence studies obviously had to discuss mechanisms behind observed reform effects as the core aspects for which they were included in this review. Furthermore, the studies must have focused on previous or ongoing electricity market reforms in developing countries and be of higher quality according to critical appraisal (see also the next sub-section Study search and selection methods and data extraction).
Irrespective of the quantitative or qualitative nature of the evidence, further inclusion criteria applied: With regards to eligible participants and settings, only studies on low and middle income countries (LMIC) were included in accordance with the main objective of this review. The World Bank LMIC classification (as of June 2013) was slightly adapted by excluding the former Soviet Union, former Yugoslavia and Turkey. These countries have enjoyed virtually full electrification coverage for several decades and insights from their electrification process were considered to contribute little to the learning process on how to reach universal access in today's developing countries.
The interventions listed in the section Theory of change were considered for this review. Generally, the categorization of interventions followed the definitions of the original authors. Taking the example of an estimation in a quantitative evidence study which uses a dummy indicating whether an independent regulatory agency is in place, this estimation is considered to assess the intervention type regulation. No restriction was put on outcome measures used in the estimations in the quantitative evidence literature.
Finally, eligibility extended to studies published or reported within the period January 1980 to March 2018. Studies published before this window were considered to contribute little to the learning process sought in this review. Studies published in any language were eligible, regardless of their publication type.

Study search and selection methods and data extraction
A range of search methods were applied to ensure the identification of published and not-yetpublished studies. The screening involved eight international electronic databases and relevant websites (see Appendix A), the consultation of various researchers and key experts, bibliographic backreferencing (reviewing references of included studies) as well as citation tracking (reviewing references in which the included study has been cited). The studies identified in this primary search subsequently underwent a three-stage screening process based on their title, their abstract, and their full text, respectively. In the first and second stage, three junior reviewers and one senior reviewer singlescreened titles and abstracts. Based on that, another team of three junior reviewers supervised by another senior team member, the author of this review, determined the studies to be finally included in the third stage. Reviewers were generally rather over-inclusive and relied on full texts in case of doubts. Any further uncertainties and discrepancies were resolved by discussion, further review of the respective studies and, where necessary, consultations with the senior team member.
These internal quality assurance procedures were also followed in data extraction, which was done by the team that already determined the studies to be finally included, including the author of this review. The extracted information involved information regarding the internal and external validity of studies, mostly based on a risk of bias assessment for quantitative evidence studies as proposed by Waddington and Hombrados (2012) and a quality assessment of qualitative evidence using the guidance provided in the Critical Appraisal Skills Programme checklist (CASP (Critical Appraisal Skills Programme) 2006) and in IOB (Policy and Operations Evaluation Department of the Dutch Ministry of Foreign Affairs) (2009), see also Skalidou and Oya (2018).

Search results and identification
The systematic study search identified 15,177 records that were subject to the three screening and eligibility stages described above. Together with 29 studies identified through back referencing and citation tracking, the first stage of title screening yielded 3,556 potentially relevant studies. Among these potentially relevant studies, 113 qualified as impact evaluations and 395 as qualitative evidence studies, the remainder being descriptions of interventions or in fact not meeting the inclusion criteria. After the third screening stage, 27 quantitative and 43 qualitative evidence studies were selected for analysis as they met all requirements. The fact that only one back-referencing study was finally included as quantitative evidence study underpins the comprehensiveness of the systematic search strategy, at least for the quantitative evidence. The search result flow diagram which includes more detailed reasons for exclusion of studies can be taken from Appendix B. Table 1 presents main characteristics of the 27 included quantitative evidence articles (study-level information can be taken from Appendix B). Since countries in Latin America were the first in the developing world to undergo market-based reforms in the electricity sector, 4 fairly rich data on the continent is available, allowing as well cross-country panel analyses. The literature on Asia tends to focus on individual countries, namely China, India and Pakistan. Sub-Saharan Africa lacks any dedicated study, which can be explained by the fact that countries on the continent introduced reforms later and more slowly throughout the last decades (Foster et al. 2017). Sub-Saharan Africa is only covered in cross-regional studies, which mostly do not allow for a disaggregation of effect sizes by continents. The same holds for Eastern Europe, which is largely due to the exclusion criteria adopted in this review. This sample composition suggests that review findings shed particular light on Latin America and, to a lesser extent, Asia, with unclear transferability to Sub-Saharan African countries.

Brief characterization of included quantitative studies
The units of observation in these studies are quite diverse and depend largely on data availability. For cross-regional comparison, studies use country-level data, whereas regional studies within Latin America and Asia can also rely on data ranging from the country level to the household level including utilities, power plants and sub-country regions such as provinces. To compile their data, study authors mostly merged various secondary sources including general and electricity-or infrastructure-related databases. In certain cases, authors requested particular information directly from firms or regulatory offices (e.g. Guasch, Foster, and Andres 2006;Estache and Rossi 2005). Table 1 furthermore makes clear that the techniques used in the articles are dominated by fixed or random effects panel estimation methods, which differ in the way they model unobserved heterogeneity (for details refer to Panda 2002, 144ff). In virtually all cases, multiple intervention types were assessed indicating more profoundthough not necessarily completeelectricity sector reforms. Cubbin and Stern (2006), for example, assess privatisation (via two dummies on minority and majority privatization), competition-enhancing policies (using a dummy for the legal right to generate electricity for resale), and regulation (among others relying on an independent regulator dummy). Privatisation and regulation are the two most assessed types of interventions. It is also not surprising that privatisation in Latin America is the most common region-intervention combination given that the continent accounted for 55 percent of total privatisation revenues in the developing world in the 1990s (Chong et al. 2004). In contrast, no primary studies could be found that address decentralisation as the fifth reform intervention type introduced in the section Theory of change.
The studies mostly assess intermediate outcomes.
As an example, roughly one third of the articles discuss electricity prices and tariffs as an outcome (further examples of specific outcome variables are listed in Appendix C). To a large extent, the studies cover relatively long time periods through an average of 12 rounds of (typically annual) data. The number of countries covered ranges between 9 and 19 for the Latin American cross-country analyses and between 22 and 181 for the global cross-regional studies. The vast majority have been published in peer-reviewed journals after 2000. Only one quantitative study, Alcázar, Nakasone, and Torero (2007), makes a distinction between rural and urban areas by focussing on rural Peru. Note: The category "Type or Sub-type of Reform Intervention" allows for combined interventions with multiple reform types being assessed in a single study. If a study did not differentiate between individual intervention types and instead looked at a reform as an aggregate, this is referred to as "Composite Reform". Difference-in-Differences analyses either rely on two waves of data or otherwise pool data from different waves into before and after without using fixed effects. ‡ Generalized Estimating Equations (GEE) are an extension of generalized linear models to the analysis of longitudinal data first proposed in Liang and Zeger (1986) and Zeger and Liang (1986).
An additional risk of bias analysis concluded that all included studies exhibit medium risk of bias. Thus, a differentiation by this characteristic in the results section is not feasible. Relatedly, an assessment of potential reporting bias using standard tools (see Sterne et al. 2011) yielded mixed results, which in part even hint towards the existence of a negative publication bias: published articles present smaller outcome coefficients compared to unpublished papers, which may reflect improvements in the methodological approaches made during peer-review revision processes or incentives to produce better data and use better methods in first instance that makes publication with a peer-reviewed journal more likely.

Brief characterization of included qualitative studies
All included qualitative studies have been published after 2000 and relatively earlier than the quantitative evidence studies. Given also the adopted quality criteria, basically all 43 studies are published in peer-reviewed journals or books (see the list of references). Similar to the regional and thematic focus of the quantitative studies, Latin American countries and privatisation received the most attention in qualitative evidence studies. In contrast to the quantitative studies, the qualitative literature usually does not restrict itself to a certain unit or method of analysis. Relatedly, on average more than three types of interventions are explicitly studied in the articles and more studies can be found on Africa. Karekezi and Kimani (2002), for example, review the challenges of power sector reforms in eastern and southern Africa with particular reference to privatisation, regulation and unbundling.

Synthesis of quantitative evidence
The following presentation of the quantitative evidence results is structured along the four intervention types, privatisation, liberalisation, private sector involvement, and regulation. Outcomes are expressed in terms of their standardized mean difference (SMD), i.e. as the ratio of the change in the outcome attributed to the intervention to the standard deviation of the outcome. Standardization of outcome measures proved to be necessary before synthesis, as studies measured outcomes in different ways using different methods. Due to data availability issues, the pooled post-treatment standard deviation as the generally preferred standard deviation could not be used. Instead, I relied on a set of alternative standard deviations and newly developed standard deviation approximations outlined in Appendix D.1. This worked out for all but one study (Vagliasindi and Besant-Jones 2013), which thus completely left the sample of studies used for the quantitative analysis in this review.
I separately synthesized SMDs that were of the same construct (or 'pool'), each representing a combination of one of the four abovementioned intervention types with one of the following six outcome categories: (i) efficiency, (ii) labour force, (iii) supply and investment, (iv) quality, (v) tariffs and costs, completed by the outcome type (vi) household welfare. Separate pooled estimates were calculated for each of these 4 × 6 pools. All effect sizes were computed in a way that positive effect sizes represent increases in the respective outcome type category. To give an example, inefficiency outcomes, such as occurrence of outages or transmission and distribution losses, were inverted to be used as efficiency measures.
In a meta-analysis, the unit of analysis is the study. To maintain the assumption of statistical independence in the data, it is important that for each pool and independent study 5 only one effect size is retrieved (Borenstein et al. 2009b). In the presence of multiple dependent effect sizes within a studywhich was the case in all but one studythese estimates thus have to be hierarchized or combined. The set of successively applied priority criteria is outlined in Appendix D.2. 6

Effects of privatisation
The estimated effect sizes are synthesized in the form of forest plots using inverse-variance random effects meta-analysis (Borenstein et al. 2009a) as long as at least three studies contribute to the respective combination of intervention and outcome type. Related to privatisation impacts, this was the case for the three outcome categories, Efficiency, Supply and Investment as well as Tariffs and Costs. For the measures of Efficiency presented in Figure 2, I abstain from computing an overall pooled effect estimate. This is due to the fact that the most pronounced estimate, the one by Sen and Jamasb (2012), seems to reflect flaws in the underlying outcome data on state-level transmission losses: as noted by the authors, prior to the reform, the State Electricity Boards in India would often include transmission losses in agricultural consumption data to hide the true levels of losses. As a consequence, reform measures tended to reveal previously hidden information and caused an artefactual increase in loss figures (i.e. reduction in efficiency). Studies based on disaggregated Note: On the left side of the graph, study references are noted together with the respective sample and unit of analysis in parentheses. Studies are ordered according to the unit of analysis, from the macro (country) down to the micro (household) level. Cross-country samples are written in capital letters. LA stands for Latin America and hh for households. For an explanation of the I-squared statistic, refer to footnote 7. The forest plot shows point estimates and 95% confidence intervals as well as pooled estimates in the shape of a diamond. See the text for the reasons why results have partly not been pooled. utility and plant data tend to find weakly positive effects on efficiency. Yet, even the more pronounced (though less precisely estimated) improvement observed by Panda (2002) merely translates to an increase in 2.3 percentage points in plant availability from a baseline level of around 70 percent. As also noted by Zhang, Parker, and Kirkpatrick (2005), who use capacity utilisation as outcome measure, it can thus be concluded that privatisation on its own seems insufficient to significantly affect efficiency performance.
As for Efficiency, Guasch, Foster, and Andres (2006) find stronger positive effects on the disaggregated utility level for their Supply and Investment outcome, the number of household connections. I also abstain from calculating an overall pooled effect estimate for Supply and Investment, since both measure and unit of analysis differs considerably between the Guasch, Foster, and Andres (2006) study and the other studies, which all assess national electricity generation capacity per capita.
Tariffs and Costs is the third outcome category for which at least three privatisation impact estimates could be retrieved (even though the two outcomes Tariffs and Costs may each contribute less than three estimates as in Figure 2(c)). These estimates provide indications on whether privatisation reform effects trickle down to the micro level in the form of changes in tariffs or at least electricity generation costs. Authors use residential and average tariffs as outcomes across all levels except for Khan (2014) who uses unit costs of production at power plant level. Tariffs turn out to be more rigid than unit costs and show an overall pooled effect size that is significantly negative, though close to neutral (see the diamond at the bottom of the tariff subgroup; SMD = −0.07, 95% confidence interval = −0.12 -−0.02, I-squared = 0%, 7 4 observations). This is also interesting to observe as reforms were sometimes taken as an opportunity to introduce cost-reflective pricing in the short term. Prices were raised to lower negative price-cost margins (or increase weakly positive price-cost margins) and thus to strengthen the financial sustainability of the electricity sector.
Results for the outcome types studied by less than three articles (here: Labour Force, Quality, and Household Welfare) can be found for all intervention types in Appendix E. They are complemented by results disaggregated by main outcomes, such as transmission and distribution losses and the number of employees of the outcome type Efficiency. These results substantiate that the impacts of privatisation are moderate with indications for impacts on supply and investment.

Effects of liberalisation
A limited number of estimations included in the results synthesis addresses liberalisation reform activities, mostly looking at Efficiency outcomes. 8 The five observations shown in the forest plot in Figure 3 cover a relatively wide range of units of analysis and outcome measures. Malik et al. (2015) contribute two estimates on operating heat rate as well as plant availability using two different samples, one finding a weakly positive and the other a weakly negative impact of liberalisation on efficiency. Panda (2002) uses data from India as well, but reaches a different conclusion in that he finds a distinct increase in plant availability of 13.7 percentage points, again from a baseline level of around 70 percent. Having a closer look at the studies, this may be explained by the different time frames. In fact, Malik et al. (2015) find heterogeneous effects across dates of reform: Plants that underwent reform longer time ago observe increases in efficiency, those that have been reformed more recently significant decreases. The authors Figure 3. Impact of liberalisation on efficiency. Note: EGU refers to electricity-generating unit. Additionally, refer to the notes to Figure 2. find indications that these decreases are likely due to short-term and idiosyncratic local factors. 9 To conclude, all studies tend to find efficiency increases after liberalisation activities.
In addition, it has to be noted that intervention sub-types are more diverse than for the other interventions. They include unbundling as the most discussed intervention sub-type, the introduction of a wholesale electricity market, and other competition-enhancing policies such as the legal right to generate electricity for resale (Cubbin and Stern 2006). 10 Estimates, however, do not seem to differ substantially across these sub-types. It is not possible to further discern these results.

Effects of private sector involvement
The forest plot for the impact of private sector involvement on Efficiency indicates a weakly positive effect (Figure 4(a)). The effect sizes for all studies but one are positive, though mostly borderline insignificant as indicated by the 95 percent confidence interval. In addition, the authors of the study that finds a negative effect, Koo et al. (2012), note that an interaction term of regulation and private sector involvement, something which is not included in the estimation considered in this study, is positive. This implies that the effect of private sector involvement on efficiency at least improves as the level of government regulation increases (Koo et al. 2012).
Among the private sector investment studies that include estimations on Supply and Investment (see Figure 4(b)), Pargal (2003) assesses whether real private investment is affected in the first place and, not too surprisingly, finds a positive effect. While Balza, Jiménez, and Mercado Díaz (2013) and Andres, Guasch, and Lopez Azumendi (2009) both assess residential electricity access, Balza, Jiménez, and Mercado Díaz (2013) interestingly find a larger effect on country level than Andres, Guasch, and Lopez Azumendi (2009) on utility level. Balza et al.'s result would imply that an increase of 1 percent in cumulative private investment is statistically significantly associated with a 0.11 percent increase in access to electricity services. Yet, it is likely that the estimations by Andres, Guasch, and Lopez Azumendi (2009), which account for utility-specific time trends, are more accurate.
The forest plot in Figure 4(c) has some resemblance to the previous ones for Efficiency and Supply and Investment, mirrored vertically due to the reverse implied impact direction of Tariffs and Costs: Costs in the first place seem to be negatively affected (Du, Mao, and Shi 2009), but -for various conflicting reasons as already mentioned under privatisation -the overall pooled effect for Tariffs is insignificant and merely weakly negative (SMD = −0.02, 95% confidence interval = −0.08-0.04, I-squared = 39%, 4 observations). Again, the utility-level analysis of Andres, Guasch, and Lopez Azumendi (2009) finds basically no effect as opposed to the negative estimate in Balza, Jiménez, and Mercado Díaz (2013) based on country data. Compared to the impact of privatisation on Tariffs and Costs, there is more heterogeneity between studies on private sector investment as indicated by the higher I-square statistic shown in the forest plot. While the variety of samples and units of analysis is similar in both studies, this heterogeneity may alternatively be due to other methodological particularities of the studies or a larger diversity in the implementation of concrete private sector involvement measures.
The outcome types assessed by fewer studies paint a similar picture in that effects tend to be weak but consistently showing into directions generally considered as improvements: higher quality, household welfare, and electricity access on the one hand and lower transmission and distribution losses, prices and workforce on the other (see Appendix E).

Effects of regulation
Regulation is generally understood by the primary study authors as the existence of a regulatory body that usually goes along with the presence of an electricity (or energy) regulatory law. While most authors create dummy variables out of the information on the respective regulatory framework, Balza, Jiménez, and Mercado Díaz (2013), Cubbin and Stern (2006) and Zhang, Parker, and Kirkpatrick (2008) use additive indices with four dimensions including a sub-index on whether the regulator is an autonomous agency or the sector ministry. 11 Since regulation is inevitably linked with any of the other reform interventions, concerns about how well one can isolate the impact of an individual reform intervention particularly apply for this intervention type.
Examining the results for the different outcome types in Figure 5, an unclear picture for Efficiency emerges. There is only one study that finds a positive impact on efficiency (Andres, Guasch, and Lopez Azumendi 2009), with Zhang, Parker, and Kirkpatrick (2008) basically finding no effect at all. Nagayama (2010) finds the largest decrease in efficiency (via increases in transmission and distribution losses) among the included studies and explains this by expansion periods of the electricity sector: a higher share of the generated electricity got lost because reform countries simultaneously extended transmission and distribution grids to more remote areas. The negative sign thus not necessarily reflects inefficiencies but rather the inability to control for electricity network sizes in the analysis, a fact that may also plague the other intervention types. More fundamentally, regulation may not have yielded efficiency gains since regulation often seemed to have favoured cost pass-through by utilities leaving little incentive to cut inefficiencies. It is also worth translating these standardized figures into economic terms. This makes clear that Balza, Jiménez, and Mercado Díaz (2013), for example, merely find an increase in losses of on average 0.28 percent.
Supply and Investment outcomes are the same as presented for private sector involvement, with Cubbin and Stern (2006) and Zhang, Parker, and Kirkpatrick (2005) additionally assessing electricity generation capacity per capita. Overall, the literature provides less precise clues for this intervention-outcome combination than for private sector involvement. Results for Tariffs and Costs are even less conclusive. The three studies that examine the impact of regulation on electricity prices shown in Figure 5(c) come to quite distinct conclusions reflected in a very wide diamond representing the pooled effect estimate. Balza, Jiménez, and Mercado Díaz (2013) find significant reductions in end-user prices, while Nagayama (2010) finds clear increases in prices. The more focused analysis by Andres, Guasch, and Lopez Azumendi (2009) on utilities in Latin America gives an idea for why this heterogeneity can be observed, substantiating a claim made earlier in this review: they find a 14 percent increase in residential tariffs under the presence of a regulatory agency, while industrial tariffs showed a 5 percent reduction and the cost-recovery ratio rose significantly by 13 percent. Cost recovery and the overall tariff structure are thus important co-determinants of impact directions related to reform endeavours including regulatory interventions.

Impact synthesis across intervention types
To conclude the quantitative synthesis, I take a look at three aspects across the intervention types discussed in the previous sub-sections: first, I show results of studies that analyse electricity sector reforms as one aggregate, composite concept. Second, I depict a summarizing forest plot for residential electricity access, the outcome of particular interest in this review. Thirdly, I run a meta-regression across the four individual intervention types, thus combining meta-analytical tools with the regression approach to assess the magnitude of effect sizes of potential influencing factors in a multivariate manner.

Electricity sector reforms as a composite intervention type
Regrettably, there are in total merely four effect estimates on outcome type level for studies looking at composite market-based reforms (Table 2). These are retrieved from the three studies Nagayama (2009), Yu andPollitt (2009), andErdogdu (2011b). A fourth study, the one by Urpelainen, Yang, and Liu (2018), also uses an additive index of power sector reforms, but their coefficients are not directly comparable to those of the other studies and therefore not shown in the table. Though one mostly observes small positive impacts (including price reductions), the largest coefficient in the table is negative: Erdogdu (2011b) finds considerable increases in transmission and distribution losses, which may add up to 10 percent depending on the extent of reforms undertaken.
Interestingly, the author's main reasoning for efficiency reductions relates to reduced economies of scope once plant location decisions by unbundled power producers do not take potential transmission losses anymore into account. Thus, he argues that liberalisation has negative effects on efficiency, which not only contradicts the indications of positive effects found in this review. It also contradicts Urpelainen, Yang, and Liu (2018), who use a larger sample and an arguably more robust method and find that non-OECD countries experience sizable improvements not only in terms of Efficiency (transmission and distribution losses) but also in Supply (electricity generation capacity). The little existing literature, hence, only allows for intermediate conclusions in that there are more indications for positive reform impacts, though the aggregation across reform components into a single index likely falls short of capturing the particularities and interdependencies of individual reform steps. Figure 6 brings together the results on residential electricity access across the intervention types, privatisation, private sector involvement, and regulation. All have been retrieved from Latin American countries and may thus not inform about the situation on other continents. Since results partly come from the same study, I abstain from calculating pooled effect sizes across the intervention types. Still, the pooled effect sizes for the individual intervention types suggest that all are non-negative.

Meta-regression across intervention types
The pooling of all effect size estimates from the previous sub-sections yields the results presented in Table 3. Since the presented estimations are pooled across outcomes and thus partly violate the independence of observations, results should be read carefully. Having said that, the results suggest impacts are weak, most pronounced for privatisation and least pronounced for regulation, to show little overall difference between Latin America and Asia, and to have the strongest effect on supply indicators. It would have been desirable to further look at interaction terms, in particular between intervention types and outcomes, but this clearly overstrains the degrees of freedom determined by the sample size of these regressions.

Synthesis of qualitative evidence
The following synthesis seeks to condense the academic debate about mechanisms behind electricity sector reform successes and failures in a stylized manner, mainly in two key tables. The synthesis is based on an iterative logic model approach focused on the identification of themes that allow enriching the hypothesized theory of change and its assumptions. It is thereby intended to shed light on the cause-effect relationship between the reforms and their impacts identified in the previous section. For this I rely on qualitative evidence, since the quantitative evidence literature is largely silent about mechanisms beyond a selective reasoning for significant coefficients as presented in the previous section. In order to understand these mechanisms it is crucial to understand the drivers and barriers of electricity sector reform in the first place. Reforms are highly endogenous in the sense that driving factors of reforms at the same time co-determine outcomes. This holds true for many of the drivers listed in Table 4. The table differentiates between national electricity sector drivers, national policy drivers and external drivers. This difference is also critical in order to be able to anticipate in how far reform measures may be interpreted as being dictated from certain internal or external circumstances or entities. In this regard, scepticism by main stakeholders like employee unions Figure 6. Impact on residential electricity access. Note: On the left side of the graph, study references are noted together with the respective sample and unit of analysis in parentheses. Studies are ordered according to the unit of analysis, from the macro (country) down to the micro (household) level. Cross-country samples are written in capital letters. LA stands for Latin America and hh for households. For an explanation of the I-squared statistic, refer to footnote 7. The forest plot shows point estimates and 95% confidence intervals as well as pooled estimates in the shape of a diamond. See the text for the reasons why results have partly not been pooled.
has been identified as one main barrier to early reforms in Asian countries such as Thailand, Indonesia and Sri Lanka (Bhattacharyya 2007;Nagayama and Kashiwagi 2007). Other main barriers to the introduction of reforms identified in the literature are not specific to the power sector either, most notably political instability (see, for example, Bhattacharyya 2007;Nair 2008;Srivastava and Kathuria 2014;Dornan 2014).
Obstacles and shortcomings have also been identified in the implementation of market-based electricity sector reforms. Most notably, the literature highlights that reforms were too much conceived as a 'one-size-fits-all' textbook approach as laid out in the power sector transition indicators of the European Bank for Reconstruction and Development, for example (Kennedy 1999). The adopted approach typically failed to take account of the vastly different circumstances prevailing in developing countries from those in industrialized countries. For example, countries with small electricity systems, such as many small African countries, lacked the necessary degree of competition. As a consequence, liberalisation interventions and private sector involvement like IPPs partly led to low-competition oligopolistic market structures (Nepal and Jamasb 2012;Hall and Nguyen 2017;McCulloch, Ward, and Sindou 2017). Note: t-values in parentheses. EGU refers to electricity-generating unit and hh to household. Ref. is the reference case in the estimation. Regressions are weighted by the inverse of the number of observations that came from the same study over and above the inverse variance weighting typically used in meta-regressions.
Reforms additionally proved much harder to be adopted than originally believed because they failed to take account of the underlying political constraints facing decision makers (McCulloch, Ward, and Sindou 2017; Lee and Usman 2018). Among others, this led to reforms being only selectively adopted according to ease of implementation, sometimes packaged and sequenced in ways unrelated to the original logic (Foster et al. 2017). Chatterjee (2018), for example, finds for India that high electoral volatility makes electricity sector reforms less likely. Under these circumstances, electricity is used as a 'political sop' (Chatterjee 2018, 129) which conflicts with the experience that electricity sector reforms often bring tariff rises for key constituencies in the short term and deliver visible benefits only in the longer term.
While the qualitative evidence literature finds it harder to chart concrete pathways that successfully navigate these contextual complexities, some stylized-fact lessons can be drawn on the mechanisms linking reforms and their impacts. In general, the literature came to some consensus on what is conducive to successful electricity sector reforms: (i) a commercial approach, (ii) competitive arrangements, (iii) cost-reflective pricing, and (iv) independent, empowered and efficient regulation. Table 5 exposes these electricity policy drivers of reform impacts in a matrix that relates the intervention types and outcome types assessed in this review. All four drivers are more extensively discussed in Appendix F. In how far the outcomes are effectively affected in a positive way is essentially attributed to how serious, timely and complementarily the mechanisms are used by reform actors and in how far negative side effects are mitigated, including how well they are embedded within wider mutually reinforcing economic reforms.

Conclusion
This systematic review examined the effects of different market-based reforms in developing countries and the mechanisms that help explain them. Table 6 summarizes the synthesis of results of the primary quantitative evidence studies included in this review in the spirit of an evidence matrix suggested by White (2018). All outcome types listed in the table are intermediate indicators, since too few studies embrace the whole results chain from market-based reform activities to electricity access. The overall message transpiring from this table is that there is not sufficient evidence to make robust statements about the aptness of market-based reforms as a means to foster electricity access effort in developing countries. There are merely weak indications that ownership (i.e. privatisation) plays less of a role than other market-based interventions and that    Note: CI = 95% confidence interval; n = number of studies assessing the respective intervention-outcome combination; SMD = standardized mean difference; No synthesis for intervention-outcome combinations with less than three studies. regulation can show mixed results depending on how it is designed as part of a broader reform agenda. Among outcomes, supply and investment indicators are the only ones that coherently present positive, though weak, impacts. These findings are broadly in line with results obtained in previous narrative reviews in that electricity sector reforms are no panacea on their own. To make bolder statements, a systematic review requires clearer indications emerging from the primary studies, which is not the case for the question of this review. Despite a careful separate pooling by intervention and outcome types, the quantitative synthesis has been plagued by substantive heterogeneity among primary studies in terms of study designs, units of analyses and applied outcome variables. The whole set of meta-analytical instruments could thus in many cases not be applied. In addition, the piecemeal and isolated assessment of the different reform components likely does not do justice to the interdependencies between them. The evidence base neither allowed embarking on relevant participant sub-group analyses, for example in order to assess claims as made in Argentina that reforms failed to provide equitable benefits to poorer segments of the society (Haselip, Dyner, and Cherni 2005).
Technological and political shifts towards low-carbon, intermittent renewable energy but also towards decentralized electricity provision and greater regional electricity market integration will change the face of the electricity sector. These new elements create scope for additional market-based mechanisms, e.g. the translation of bilateral cross-country trading agreements into competitive market arrangements. More generally, the question on the suitability of market-based reforms for improving the performance of the electricity remains relevant in the future, since the dominant structure and challenges of electricity systems will likely persist despite the transformations.
Research can improve the evidence base on this question, first, through better and more comprehensive data. The dataset published by Urpelainen and Yang (2019) is a laudable effort in this regard. Second, a consistent use of best practise in panel estimations, including appropriate clustering of standard errors, will contribute to higher validity of results. Instrumental variables may represent a workaround for remaining endogeneity problems, but possibilities for their application are very limited in the given context. A more promising avenue seems to be a greater consideration of mixed methods (cf. Anderson et al. 2013). Thereby, not only solid evidence can be generated but, even more importantly, this evidence is then linked to the underlying mechanisms of reform successes and failures. This is of particular relevance in the present case, as market-based electricity sector reforms take place at the intersection of the technological, economic and political space. Hence, various techno-economic and political economy matters need to be taken care of and come into play in order to reach desired outcomes.
In the present case, the synthesis of the qualitative evidence illustrated that four factors linking the individual interventions and outcomes at the heart of the underlying theory of change are critical to increase the likelihood of positive effects of electricity sector reforms: a commercial approach, competitive arrangements, cost-reflective pricing, as well as independent, empowered and efficient regulation. This should give tentative guidance for coherent policy delivery while still leaving room for context-specific adjustments, experimentation, and combination with technical interventions that more directly increase electricity access or supply efficiency. Notes 1. Furthermore, there is a systematic review on private sector involvement in the delivery of water, telecommunication and electricity services and its impact on access and quality of service in developing countries (John et al. 2015). This study uses unstandardized t-statistics rather than effect sizes. Other relevant publications tend to be relatively old and, more importantly, do not pursue a systematic approach of searching and synthesizing findings: Albouy and Nadifi (1999), Andres, Guasch, and Lopez Azumendi (2009) and Jamasb et al. (2005). 2. This review was intended to more specifically use reliable access to electricity as outcome, i.e. a guaranteed sufficient level of service quality of the electricity access, e.g. measured in terms of households suffering from different levels of blackouts and/or brownouts. Accounting for reliability in this review, however, proved impossible, since such information could not be retrieved from the original studies. 3. The review approach adopted in this article does not differ from the one presented in Bensch et al. (2016), which is a 3ie Systematic Review report. The primary substantive change is that studies had previously been considered that were published before July 2015, whereas this article covers the timespan until March 2018. 4. See the cross-country timelines presented in Nagayama and Kashiwagi (2007) and Foster et al. (2017). 5. Independent studies have to be understood as independent measurements based on non-overlapping samples. It may thus be the case, that multiple original studies based on the same data and outcomes represent a single independent study. At the same time, a single original study may include multiple substudies using independent samples. In a few cases, authors based their studies on similar data, notably Andres, Guasch, and Lopez Azumendi (2009) and Guasch, Foster, and Andres (2006), Zhang, Parker, and Kirkpatrick (2008) and Zhang, Parker, and Kirkpatrick (2005), as well as Nagayama and Kashiwagi (2007); Nagayama (2009);2010). The studies, however, typically assess different intervention and outcome types and use sufficiently distinct data sets. Dependency between different results is therefore not an issue. 6. Since these priority choices potentially involve some degree of arbitrariness, 'synthetic effect sizes' (for details, see Appendix D.3) were as well assessed in a sensitivity analysis. The latter yielded results that are qualitatively very similar to those of the main analysis. Hence, they are not shown in the following. 7. The I-squared test statistic is a relative measure of statistical heterogeneity. It represents the percentage of total variation across studies that is due to unexplained heterogeneity rather than chance. A value higher than 50 percent hints to substantive heterogeneity (Higgins and Green 2011). 8. Note that this also has to do with the exclusion of four papers from the analysis of liberalisation, namely Erdogdu (2011a), Nagayama and Kashiwagi (2007); Nagayama (2010), and Sen and Jamasb (2012). These articles included selected sub-types of the intervention type liberalisation (e.g. wholesale electricity market and unbundling in Erdogdu 2011a) that, in sum, do not necessarily reflect liberalisation as a whole. See also PCa11 in Appendix D. 9. On the one hand, availability of electricity-generating units seemed to have gone down because of increased restoration and maintenance shortly after reform. On the other hand, plant efficiency decreases (expressed as increasing operating heat rates) may have been triggered by shocks to the quality of coal in terms of ash and moisture content in two major states of India. 10. Further competition-enhancing policies studied are the free choice of supplier (Erdogdu 2011a), the introduction of retail competition (Nagayama 2007(Nagayama , 2010, open access to network (Sen and Jamasb 2012) and the introduction of a wholesale market or the permission for generators to compete in concluding supply contracts with distributors or large users (Zhang, Parker, and Kirkpatrick 2005). 11. Note that a few papers have been excluded from the analysis in this section, namely Erdogdu (2011a), Nagayama (2007), and Sen and Jamasb (2012), for the same reasons as described for liberalisation under footnote 8: these articles included selected sub-types of the intervention type regulation that, in sum, do not necessarily reflect the likely effect of the intervention type as a whole.

Acknowledgments
I would like to thank the Canadian Department of Foreign Affairs, Trade and Development (DFATD, formerly CIDA) for funding this study and the International Initiative for Impact Evaluation (3ie) in the person of Hugh Waddington and Phil Davies. Their very thoughtful and constructive comments and suggestions on the 3ie Systematic Review, which laid the foundation for this article, are highly appreciated. I would also like to warmly acknowledge the advisory group of the 3ie Systematic Review, including Eva Rehfuess, Antonio Estache, Michael Grimm, and Subhrendu Pattanayak. Similarly, my thanks go to two external reviewers for useful comments and suggestions. The article very much relied on the support to previous versions of this review by Nadine Kneppel (NK), Jörg Langbein (JL), Anicet Munyehirwe (AM), Jörg Peters (JP), and Maximiliane Sievert (MS). JP, MS, and AM contributed to the initial study scope and design. MS also contributed to the study protocol and developed the search strategy. MS, NK, and JL joined the study search. JL supported the quantitative effectiveness synthesis, NK the qualitative synthesis of mechanisms. Furthermore, I also thank Kyra Eusemann, Sophie Wannemacher, Friederike Blönningen, Ann-Kristin Reitmann, Lara Roetzel, Avery Maloney and Sebastian Mertesacker for valuable research assistance, notably in the stages of study search and data extraction, and Jonathan Stöterau for helpful methodological discussions. I am as well thankful for helpful feedback by participants of the third meeting of the Sustainable Energy Transitions Initiative (SETI) at Duke University.

Disclosure statement
No potential conflict of interest was reported by the author.

Notes on contributor
Gunther Bensch is a researcher in the Department of Environment and Resources at RWI Leibniz Institute for Economic Research. His fields of research include environment and energy economics and policy, with a special emphasis on the empirical evaluation of development policies such as improved cookstoves and infrastructure interventions. He holds a PhD in Economics from University of Bochum.