![ORCID Icon]({"type":"image","src":"/templates/jsp/_style2/_tandf/images/orcid.svg"} "ORCID Icon"){kind=small}
https://orcid.org/0000-0002-9451-9174
ABSTRACT
ABSTRACT
To achieve its ambitious climate targets, the European Union (EU) must adopt new policies, increase the impact of existing policies and/or remove dysfunctional ones. The EU has developed an elaborate system to monitor national policy mixes in order to support these challenging requirements. Data that member states have reported to the EU over the last ten years reveal that the average expected per-policy-instrument emission reduction has declined, while national policy mixes have remained generally stable over time. This is strikingly discordant with the EU’s ambitious commitment to become carbon neutral by 2050 (‘net zero’).
According to the European Environment Agency (EEA), the European Union (EU) as a whole is not on track to achieve its 2030 climate and energy targets, let alone the long-term target of net-zero emissions by 2050 (EEA 2019a). This projected shortfall means that the EU and its member states will have to significantly upgrade their policy efforts in order to deliver on their policy promises. Doing so has become especially urgent in light of the EU’s desire for international climate leadership (Oberthür and Dupont 2021; Parker et al. 2017; Ruiz-Campillo 2020) through the Commission’s European Green Deal (European Commission 2019) and the associated EU Climate Law (European Commission 2020).
In light of these commitments, there is a pressing need to understand the evolution of climate policies. New insights may emerge from looking beyond the mere reporting of national greenhouse gas emissions and exploring climate policy changes reported by the member states over the ten-year period between 2009-2019. An assessment of policy developments over a period of ten years is a solid basis for analysing public policy change (Jenkins-Smith et al. 2014) and allows us to answer our two research questions: What information do member states provide on their climate policies in the context of the EU Monitoring Mechanism? What do these monitoring activities reveal about the long-term evolution of climate policy mixes in the EU?
There are in principle three ways in which the EU and its member states could strengthen their climate policy efforts: namely, by (1) adding new policies to the extant stock; (2) strengthening existing policies in order to increase their impact; or (3) removing dysfunctional policies (policy dismantling). Combinations of these three approaches are also imaginable. With regard to options one and three, important questions then become: What sectors do the new/removed policies address (for example, the energy sector, or transport and/or agriculture?); and what types of policy instruments are added/removed (that is, are they, for example, economic, regulatory or information-based?). Previous research suggests that certain policy instrument types (for example, those linked to public expenditure and those providing clear economic incentives) tend to be more impactful than others (Auld et al. 2014). Furthermore, policies may interact with one another and/or generate feedback effects that extend their impact beyond what was originally assumed. Such effects necessitate a broader view that goes beyond the study of individual instruments to understand different mixes of instruments (Edmondson et al. 2019; Flanagan et al. 2011; Jordan and Moore 2020).
EU member states may seek to improve policy stringency, for example, by tightening emission limits or directly increasing the carbon price. Alternatively, they could shift their policy efforts to sectors where there is still low-hanging fruit, that is, where more limited policy efforts can generate more substantial emission reductions. Similarly, member states may change their policy instruments if some types are perceived to be more effective than others (for example, more regulatory, instead of information-based, instruments, or the other way around). Many existing instruments – notably, the EU Emissions Trading Directive – have hitherto focused on the energy sector, where the limited number of producers has made it comparatively straightforward to effect change, such as shifting to renewable energy. In sum, there are multiple ways in which member states could, in principle, adjust their climate policy mixes.
Before digging deeper, it is important to be clear about what a climate change policy is. One common way to define ‘policy’ is to break it down into different elements – including policy objectives, policy instruments and the calibration or setting of specific policy instruments (Hall 1993). However, other aspects can also be included, such as broader policy programmes and even overarching policy paradigms. The interminable debates about the nature of ‘policy’ have made it difficult to reach an academic consensus on defining, operationalising and measuring policy change (see Howlett and Cashore 2009). In this article, we therefore explore the potential value of adopting a very different approach, that is, analysing climate policy based on what countries themselves report. While earlier research has frequently relied on a range of sources and methods, in particular expert judgements, for identifying and analysing policy change (for example, Knill et al. 2012), jurisdictions have themselves also stepped up efforts to catalogue and report on their climate policy instrument choices. For example, many recent climate laws include policy monitoring/reporting obligations. It is important to draw these efforts into the debate about the policy drivers of deep decarbonisation.
Crucially, the systems that countries and administrations have established to track policies produce ‘administrative’ rather than ‘scientific’ data. According to Peter Elias (2014, 47):
[a]dministrative data are defined as data which derive from the operation of administrative systems, typically by public sector agencies. […] While such data are not designed for research purposes they often have significant research value, especially when linked to other datasets or to user-generated surveys.
The existing academic debate on administrative data typically focuses on data collection at the level of individuals. In this article, we explore the potential of extending the concept to include data collected and published by public agencies with regard to public climate policy efforts (see Connelly et al. 2016).1
In order to analyse the EU’s climate policy as implemented by the member states, we explore what member states report against a backdrop of the literatures on policy change, on policy mixes (see Flanagan et al. 2011) and on sectoral policies. Assessing changes in policy mixes has been identified as a key area for future research (Capano and Howlett 2020; Ossenbrink et al. 2019), and this article responds to this call. Against this background, we present our empirical material, namely the data emerging from the EU Monitoring Mechanism,2 a long-standing EU-level instrument to collect and track climate policy data supplied by the member states (Hyvarinen 1999; Schoenefeld and Rayner 2019). To put it simply, this Mechanism involves the European Commission collecting a wide variety of information on individual climate policy instruments from the member states, which together generates a bigger picture of the changing climate policy mix across the EU. It includes both policies of exclusively national origin and policies that the EU member states put in place to implement EU requirements (such as on renewables or emissions trading).
Having introduced the general context and background, the remainder of this article proceeds as follows. The next section discusses policy mixes, as well as the sectoral focus of (climate) policy, from a conceptual perspective. The following section contains the methodology, including a more detailed description of the Monitoring Mechanism. We especially draw on the ex-ante, per-policy predictions that member states submit to the EU with regards to greenhouse gas emissions reductions as a measure of expected policy impact. These ex-ante projections are analytically important because they reflect the member states’ expectations of how far their policies will fulfil jointly agreed targets. For the European Commission, these projections matter because they help it to judge the adequacy/feasibility of the EU’s overall emission targets. The article then presents the results, followed by conclusions on what can be learnt from past climate policy efforts and new avenues for research and policy.
Understanding policy change: policy mixes and sectoral foci
What constitutes policy and policy change is an ongoing debate with high relevance for climate change, where massive policy change is likely necessary to reach the stated EU targets. For many scholars, a key approach is Peter Hall’s (1993) orders of policy change. First-order policy change may happen at the level of the settings of policy instruments, such as for example when the maximum greenhouse emissions (per unit of distance) or the minimum mileage (per unit of fuel) for new cars are regulated. Second-order change, in turn, emerges when policy-makers change the policy instruments themselves, such as a shift from taxation to emissions trading (the former being a system where a fixed tax is levied on each emitted ton of greenhouse gases; the latter implying an overall cap, within which tradable permits to pollute are issued to polluters). Finally, third-order change becomes apparent when the overall policy goals change (Hall 1993). The EU’s adoption of a binding net-zero target may be described as a third-order change.
This article focuses on the second order of policy change, namely, potential changes in the quantity and quality of policy instruments identifiable in administrative data. While sustained attention has been paid to the measurement and the determinants of policy change (Capano and Howlett 2009; Knill et al. 2012), especially cross-cutting policy fields such as climate change have stimulated a strong interest in policy change at the aggregate level, namely across many different instrument types (Schaffrin et al. 2015; Schulze 2021). One way to do so involves exploring different “policy mixes” (Capano and Howlett 2020), that is, the overall pattern of climate policy instruments adopted in different jurisdictions.
Policy mixes
Single policy instruments rarely suffice to address complex and wide-ranging societal and/or ecological issues such as climate change, poverty or lack of adequate health care. This understanding has stimulated vibrant research efforts centring on policy mixes, or on “complex arrangements of multiple goals and means which, in many cases, have developed incrementally over many years” (Kern and Howlett 2009, 395). Understanding policy mixes is important to enable productive policy design. It is also essential to avoid potentially unhelpful instances of “policy accumulation” (Adam et al. 2019). For example, one strand of work has focused on the coherence and consistency of policy mixes by looking at the characteristics of the individual policy instruments contained therein (Howlett and Rayner 2007). Another line of research involves assessing which instrument types have been applied, be they more traditional regulation (command-and-control), economic or information-based instruments, meaning policies that inform citizens or business actors about desirable actions or behaviour (Wurzel et al. 2019). Other efforts have unpacked individual policy instrument characteristics, notably their intensity and technology-specificity, to assess policy change (Schmidt and Sewerin 2019). In spite of this significant progress, numerous gaps remain in the literature (see Capano and Howlett 2020). This article identifies two new aspects, namely, the member states’ own conceptions of their policy mixes via the administrative data they supply to the European Commission, as well as what we (do not) know about how these conceptions have changed over time.
Such efforts may be understood as concerted political and administrative interventions to present one’s own policy mix in a given policy area (see Schoenefeld 2021). Because governments have limited resources and specific interests, they tend to carefully choose the policy areas where they publicly characterise their policy mixes (Xun et al. 2017). The monitoring may thus be understood as potentially significant political signalling and an effort to demonstrate active engagement in an issue such as lowering emissions in a particular sector. As a first step, this article focuses on how each member state presents its climate policy mix. Having done so, a second step involves examining the extent to which their characterisation of this policy mix has changed over time.
Sectoral focus
Traditionally, particular policies have been assigned to substantial policy sectors, such as the energy, transport, waste or agriculture sectors. Climate policy scholars have long argued that different sectors may require different policy approaches (Meckling and Chung 2009). It is also clear that the number of policies applied in each sector varies in the EU (Dauwe et al. 2019). The long-term, net-zero carbon emissions target by 2050 requires substantial emissions reductions in all policy sectors, that is, a clear, innovative step change, because no sector will be left with nothing to do (see Hartley and Torfing 2016). In other words, the increasing importance of the quintessentially cross-cutting field of climate policy has brought to the fore new questions about the sectors in which countries should target their policies. The ambitious net-zero targets force countries to examine possibilities for emission reductions also in sectors that have been difficult to address so far. Therefore, it is arguably becoming increasingly important for the member states and other policy-makers to assess the sectoral balance of their climate policies, in order to identify the past, current and potential future foci of policy effort and – in a broader sense – the social, political and technical implications of these approaches. The European Green Deal also advances this line of thinking.
Methodology
Empirically, this article focuses on member states’ efforts to collect and present data on the climate policies that have been or will be put in place to achieve collectively agreed EU climate change targets. From the perspective of policy mix studies, this represents a bottom-up approach, meaning that we begin with an “impact domain” (here, broadly, the area of climate change policy) and consider what policies have been put in place (and reported) in this domain by the EU member states (see Ossenbrink et al. 2019). In doing so, it is important to recall that climate policy in the EU represents a complex set of shared (that is, EU) and exclusively national competencies. In areas such as greenhouse gas emissions targets, renewables policy or emissions trading, the EU institutions and member states share competence in deciding on policy targets and the means to achieve them. In other areas, such as the determination of national energy mixes, member states retain exclusive competence to decide, although they are subject to certain limitations, such as those emerging from the EU-level decisions on renewable energy (see Wettestad et al. 2012).
The EU Monitoring Mechanism has existed since 1992 (Hyvarinen 1999), but its efforts to monitor climate policies were strengthened significantly in 2009 (Hildén et al. 2014). It exists alongside other reporting requirements, such as in the areas of renewable energy and energy efficiency legislation (Schoenefeld and Rayner 2019), and now forms part of the Energy Union Governance reporting that brings together different monitoring streams (Schoenefeld and Jordan 2020). In future, it will become part of the Climate Law (COM(2020) 80 final). Since the EU member states must report on their climate policies and measures every other year, a unique longitudinal dataset exists comprising six data iterations over a period of ten years (2009, 2011, 2013, 2015, 2017, 2019).3 Using these data thus comes with various advantages, including the large number of countries in the database (28+), as well as the substantial time period over which data are now available.
However, since the data analysed here emerged from official climate policy monitoring and reporting by the member states, it is important to understand how the production process impacts on data quality, reliability and validity (Connelly et al. 2016; Hand 2018). In fact, administrative data collection invariably incorporates political judgments, because data may alter power relations among actors, expose successes or failures, but also drive improvement (see Hughes et al. 2019; Schoenefeld 2021). Earlier research efforts have demonstrated that the Monitoring Mechanism involves a range of multi-level institutions, interactions and path-dependent dynamics (Schoenefeld et al. 2019) and has generated data whose plausibility and quality should be examined (Hildén et al. 2014; Schoenefeld et al. 2018).
In analysing the quantitative outputs of the Monitoring Mechanism, this article focuses on four important aspects of policy instruments. First, there is the reporting year (see above). Second, we draw on the information provided on the policy sector, coded using the EEA’s definitions (energy supply, energy consumption, transport, industrial processes, agriculture, waste, land use, land-use change and forestry [LULUCF], and/or cross-cutting) (see EEA 2019b; Wurzel et al. 2019). Third, we examine the instrument type, that is, whether the member states together with the EEA classified a given instrument as an economic, fiscal, voluntary, regulatory, information, education, research, planning and/or “other” instrument (EEA 2019b). Fourth, we consider whether the Monitoring Mechanism provides quantitative, ex-ante (forward-looking) projections of expected emissions reductions (expressed in megatonnes of carbon dioxide [CO2] equivalent per year) for a given reported instrument. We used the quantitative projections of expected per-policy emissions reductions for the year 2020, because these are available in all reporting rounds.
In order to assess the expected impact of the overall policy mix across EU member states, we calculated the average of expected emissions reductions attributed to each climate policy, that is, by dividing the grand total of projected greenhouse gas emissions reductions by the number of climate policies that the member states reported.4 In doing so, it is important to recall that our calculations provide a rough indicator of the expected, not actual, policy impact. Under the effort sharing decision, the member states have different reduction targets. In the past, some members states even had negative targets, meaning that they were allowed to increase their emissions (Haug and Jordan 2010).
To further investigate the diversity of climate policy instruments reported by the EU member states, we use a standard diversity measure, that is, the inverse Hirschman-Herfindahl Index (IHHI) (Boydstun et al. 2014; Lubell et al. 2020). The index builds on the shares of the different instrument types and can be interpreted as indicating the heterogeneity in the overall climate policy mix, that is, in terms of the deployed instruments. Specifically, it measures the probability that two randomly picked instruments in each reporting period are of different types (see also Schmidt and Sewerin 2019). The IHHI ranges from 0 to 1, with higher values indicating a higher instrument diversity. For instance, if the number of instruments (within a sector) is equally distributed between ten instrument types, the IHHI is (1-10*0.12) = 0.9, that is, the likelihood that two randomly picked instruments are of different types is 90 per cent (Lubell et al. 2020, 228).5
Results
Number of climate policies
Our analysis reveals an overall increase in the number of policies reported by the member states (see also Schoenefeld et al. 2019), from 1,273 (in the year 2009) to 1,397 (2011), 1,526 (2013), 1,323 (2015), 1,582 (2017) and to 2,016 (2019).6 However, while before 2015, there was a steady growth of reported climate policies with projected emissions reductions, between 2015 and 2019, their number has by and large remained unchanged, fluctuating around 500 policies (Figure 1). This means that the member states are reporting more and more climate policies whose expected impact in terms of emissions reductions goes unreported.
Published online:
06 September 2021Figure 1. Number of climate policy instruments reported by all EU member states (2009-2019)
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rspe20/2021/rspe20.v056.i03/03932729.2021.1956827/20210920/images/medium/rspe_a_1956827_f0001_b.gif"})
Figure 1. Number of climate policy instruments reported by all EU member states (2009-2019)
Note: An earlier version of this figure was published with a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/) in Schoenefeld et al. (2019) – that version does not include the new 2019 data. Based on data from the EEA Monitoring Mechanism/PaMs database, http://pam.apps.eea.europa.eu (last accessed 9 July 2021).Projected policy impact
Figure 2 reveals that there was a period of relative stability until 2015 in terms of the level of projected reductions per policy. Since then, the average projected emissions reductions per policy have declined, mainly because the member states have introduced additional instruments without estimating their projected emissions reductions (see above). However, the average per-policy reductions for policies with projected reductions also reveal a decrease of almost a third from 2009 until 2013. Member states also report that they expect fewer reductions, on average, from their policies – at least beginning with the 2017 and 2019 reporting rounds. Specifically, the implementation of the new Monitoring Mechanism Regulation (MMR) in 2013 (which had first visible effects in the 2015 reporting round) resulted in a substantial initial increase in emissions reductions reported for the policy instruments with quantified projections. The data emerging from 2017 and 2019, however, suggest that member states expect fewer per-instrument emission reductions. This observation may in part reflect adjustments to the new reporting system (post-2013) rules (Schoenefeld and Jordan 2020).
Published online:
06 September 2021Figure 2. Projected emissions reductions per instrument (2009-2019)
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rspe20/2021/rspe20.v056.i03/03932729.2021.1956827/20210920/images/medium/rspe_a_1956827_f0002_b.gif"})
Figure 2. Projected emissions reductions per instrument (2009-2019)
Source: Compilation by the authors, based on data from the EEA Monitoring Mechanism/PaMs database, http://pam.apps.eea.europa.eu (last accessed 9 July 2021).Sectoral focus
In order to understand the sectoral policy effort in the EU, Figure 3 shows how the number of reported climate policy instruments is spread across individual sectors, giving a first impression of where member states have focused their efforts in the past and, in particular, how this has (not) changed over time.7 Indeed, the sectoral breakdown reveals that the relative policy activity devoted to each sector has been rather stable, indicating considerable path dependencies in climate policy development. In each reporting round, nearly half of the deployed instruments were categorised as dealing with either energy production or energy consumption. In terms of the number of policy instruments, the two energy sectors and the transport sector are by far the most addressed sectors, containing about two-thirds of all instruments reported across the EU member states. We also observe a high level of stability (rather than growth) of policies relating to industrial processes, cross-cutting policies and ‘other’ sectors. There are, however, subtle changes. Agriculture, as well as LULUCF, have attracted an increasing number of policies over the period studied, suggesting greater efforts in two previously neglected areas of climate policy.
Published online:
06 September 2021Figure 3. Number of climate policy instruments by sector (2009-2019)
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rspe20/2021/rspe20.v056.i03/03932729.2021.1956827/20210920/images/medium/rspe_a_1956827_f0003_b.gif"})
Figure 3. Number of climate policy instruments by sector (2009-2019)
Source: Compilation by the authors, based on data from the EEA, Monitoring Mechanism/PaMs database, http://pam.apps.eea.europa.eu (last accessed 9 July 2021).Instrument types
Figure 4 describes the number of reported instruments according to their qualitative types.8 Most notably, in aggregate terms, the distribution of different instrument types appears fairly stable. In other words, while the number of instruments in the EU has increased over time, the relative distribution of instrument types has remained by and large constant.
Published online:
06 September 2021Figure 4. Number of climate policy instruments by type (2009-2019)
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rspe20/2021/rspe20.v056.i03/03932729.2021.1956827/20210920/images/medium/rspe_a_1956827_f0004_b.gif"})
Figure 4. Number of climate policy instruments by type (2009-2019)
Source: Compilation by the authors, based on data from the EEA Monitoring Mechanism/PaMs database, http://pam.apps.eea.europa.eu (last accessed 9 July 2021).Figure 5 demonstrates that the diversity of reported instruments for each sector in the EU has also remained relatively stable since 2009. The overall diversity across member states decreased somewhat from .82 to .78 on the IHHI. In other words, the overall probability that two randomly picked instruments in the member states’ climate policy mix are of a different type decreased by four percentage points. This decrease in diversity can, for instance, be witnessed in Germany, which has reported an increasing share of economic instruments in several sectors over the last two reporting rounds. This includes, among others, the introduction of tendering and other incentive-based programmes to enhance efficiency in the energy sector, but also in agriculture and transport, such as higher highway tolls for heavy-duty vehicles and subsidies for electric mobility. By contrast, the number of fiscal (tax-based) measures has been declining in Germany, contributing to an overall decrease in the diversity of its climate policy mix. While the overall decrease in instrument diversity is rather modest, there appears to be a tendency towards regulatory instruments in sectors such as industrial processes and waste.
Published online:
06 September 2021Figure 5. Instrument diversity by sector (2009-2019, IHHI Index)
![]({"type":"image","src":"/na101/home/literatum/publisher/tandf/journals/content/rspe20/2021/rspe20.v056.i03/03932729.2021.1956827/20210920/images/medium/rspe_a_1956827_f0005_b.gif"})
Figure 5. Instrument diversity by sector (2009-2019, IHHI Index)
Note: Please note that the y-axis does not start at “0” for better readability. Source: Compilation by the authors, based on data from the EEA Monitoring Mechanism/PaMs database, http://pam.apps.eea.europa.eu (last accessed 9 July 2021).Conclusion
Climate change policy must become considerably more effective in order to achieve the long-term targets that the EU has set. In particular, the EU-wide net-zero target for 2050 implies that public policies should ensure that significant emission reductions are achieved in all sectors (in other words, no sector can continue with business as usual), although reduction targets are likely to differ given sectoral differences and differences between member states in obligations to reduce emissions. The research presented here has revealed something new about how member states’ monitoring and reporting of national climate policy mixes have (not) changed in the past. To do so, we drew on little-studied data that the member states routinely provide to the European Commission via the Monitoring Mechanism/Energy Union Governance Regulation. In principle, the Monitoring Mechanism should offer insights into how the member states monitor and report on their own policies, and how they describe the impact of their own actions. However, research on the Monitoring Mechanism data remains scarce, which is surprising, given that it is the EU’s main instrument to track its policies in what is a signature policy area. Our results thus offer an original analysis of climate policy efforts across member states, revealing how their sectoral focus and instrument mix have reportedly (not) changed.9
Considering the ten-year period between 2009 and 2019, we found that the member states reportedly added more and more climate policies over time, reaching more than 2,000 individual policies and measures at the national level by 2019 (many of which have, of course, been implemented in response to EU-level policies). At the same time, however, the number of reported climate policies with projected emissions reductions has not increased, and the amount of projected emissions reductions per policy instrument has decreased. This seems to indicate that the member states are adding more climate policies to the overall mix rather than increasing the expected impact of existing instruments. However, it may also partly reflect a gradual readjustment of the projections as the target year is coming closer. Estimates made in 2009 for a reduction to be achieved in 2020 may have been scaled down in light of insights from later reporting rounds.10 The sectoral distribution of the policies has maintained a strong focus on energy and transport, but there have been some subtle changes; for example, a growing number of instruments relate to land use and agriculture. At the same time, the relative distribution of qualitative instrument types has remained stable between 2009 and 2019, once again demonstrating little change. Overall, these findings are strikingly at odds with the EU’s ambitious commitment to reach climate neutrality by 2050 (‘net zero’).
New EU-wide targets may stimulate fresh climate policy development, especially if it becomes clear that existing efforts are insufficient. Recall that significant policy development and target-setting happened at the EU level in 2008-2009 (with the negotiation of the 2020 package) and since 2014-2018 with the negotiation of the 2030 package, as well as with respect to the net-zero target (Jordan and Rayner 2010; Knodt et al. 2020). Given that it takes time for EU-level targets to trigger member states’ policymaking, there may have been a policy instrument diversity peak in some sectors in 2011. Thereafter, diversity declined, but following the new target-setting in the late 2010s, there is a hint of an increase in diversity in 2019 and a clear increase in the number of policies (see Figure 1). Future research should establish whether these (emerging) trends are sustained and whether they result from a growing focus on the most impactful policies and policy mixes. The Fit for 55 package published by the Commission in July 2021 provides a starting point for such work (European Parliament 2021). Furthermore, the underlying drivers behind changes in diversity demand scholarly attention. Changes in perceived climate policy diversity might reflect a search for more effective policy mixes, especially in hard-to-mitigate sectors such as agriculture and LULUCF, or in sectors where new policies are needed to effect rapid changes such as the large-scale transition to renewable energy.
New climate policy development may, however, also have limits: in aggregate terms, member states claim to be using similar instrument types in more or less equal proportions in the various climate policy sub-sectors across the reporting years. This suggests significant path dependency in the overall trajectory of EU climate policy. The radical change that some suggest is required to reach the EU’s long-term targets could nevertheless appear as major qualitative adjustments within existing policy types: for example, economic or regulatory instruments that dismantle the existing fossil economy (see also Hurka and Steinebach 2020; Kivimaa and Kern 2016). This evidence of path dependency chimes with the insights of other scholars who have examined the EU’s 2030 framework (Kulovesi and Oberthür 2020). The results are also in line with previous work on policy mixes, which has highlighted path dependencies and a tendency to layer new policies on top of existing ones rather than attempt fundamental reforms (Ossenbrink et al. 2019).
The big challenge is that the step change implied by the Climate Law cannot easily be detected in what national officials have reported thus far.11 Ideally, the standardisation of monitoring should advance to allow for distinguishing between changes in monitoring practice and underlying policy change. Currently, this distinction is difficult to make, and this greatly limits the insights one can gain from administrative data at EU level. In individual member states, governments and administrators may have a sense of policy effectiveness based on multiple sources of information, but they are likely to know less about policies in other countries.
Developing further the monitoring of climate policy mixes and their changes may be one piece in the puzzle of achieving a greater understanding of policy effectiveness, because it provides knowledge about the means by which member states seek to achieve the headline targets and how effective these means are expected to be. The potential contribution of policy monitoring and evaluation to successful policy development depends to a significant degree on data quality and insights that these activities generate (Stake and Schwandt 2006). While monitoring and reporting quality ultimately require agreement on social values, that is, what is deemed important and desirable (Dahler-Larsen 2019; Fischer 2006), the presently low level of quantification makes it difficult to estimate the effects of many existing member state climate policies, pointing to a key area where improvements in monitoring could ultimately lead to greater policy impact. Furthermore, greater standardisation will be necessary to permit reliable comparisons of total projected emissions reductions across different instruments and member states. There is also a need to achieve greater standardisation in the narrative descriptions of the policies so that these narratives support and explain the quantitative projections. The current classification into instruments is not sufficient when it comes to identifying and understanding diversions from an incremental path. In sum, from the perspective of effective use of monitoring data – including in policy evaluations – greater streamlining and standardisation in monitoring would be a desirable course of action.
Before concluding, it is important to be aware of some potential limitations in our analysis. The rules underpinning the Monitoring Mechanism have changed over time. The absence of a coherent set of procedures, in turn, means that implementation varies in the member states, driven by changing governments and personnel in the national ministries/institutes, creating additional challenges in dealing with data (in)consistency and interpretation (Hildén et al. 2014; Schoenefeld et al. 2018; Schoenefeld et al. 2019). Using the Monitoring Mechanism to understand how EU member states characterise their policy mixes is thus a worthy endeavour, but some caveats apply (Hand 2018).
The political nature of monitoring data has also been a focus of scholars working on the politics of ex-ante policy assessment (Paul 2020; Turnpenny et al. 2009; see also Schoenefeld 2021). It is, after all, policy-makers in the ministries who have de facto the final say on what their respective member state ultimately reports to the EU (Hill and Varone 2016, 245-8). Given that member states use partly different sets of methods and assumptions to model their projections, these should not be compared without due attention to the prevailing national context. Ongoing endeavours have sought to develop more standardised ways of assessing the impact of technology and potentially policies on reducing emissions (EU Science Hub 2019). However, the distinct advantage of the Monitoring Mechanism is that it produces regular data, so trends have become discernible; in addition, quality assurance processes have been introduced to improve its outputs (Schoenefeld et al. 2018). In other words, Monitoring Mechanism data permit unique insights into the ways in which member states themselves understand and characterise their climate policy mixes. An important future task, therefore, involves comparing these administrative data to other sources of information such as those provided by academics in order to further unpack their usability, validity and plausibility (see Graeff and Baur 2020; Kapteyn and Ypma 2007). Compared with the category of ‘regular’ scientific data, which usually derive from experiments or observation and are thus – from the perspective of a scientist – ‘made’, the defining characteristic of administrative data is that they are ‘found’, meaning that they were originally collected for administrative as opposed to research purposes (Connelly et al. 2016). For social scientists, the Monitoring Mechanism data lie on a continuum between scientific and administrative data.
To conclude, it has been noted that climate policy monitoring in the EU is changing, with the insertion of “harder elements”, such as more transparency and linking with effort sharing legislation and/or via linkages with broader policy assessment processes (Schoenefeld and Jordan 2020). Monitoring has evolved, potentially contributing to greater policy effectiveness. Graeme Auld et al. (2014), for example, suggest that policies that involve mandatory monitoring tend to be more effective than those for which it is voluntary. However, one should also recognise that monitoring may not be suitable or necessary in every case. This article provides a firmer foundation to continually assess the EU’s evolving climate policy mix, with a view to changing the nature of monitoring practices and contributing new insights into how to promote more effective policymaking in the EU (Bürgin 2021), commensurate with the unprecedented scale of the deep decarbonisation challenge.
Acknowledgments
The authors are grateful to the editors of the journal and the Special Issue for their constructive comments on earlier drafts of this article. Anonymous peer reviewers helped strengthen the text. The authors also thank those who participated in the UACES Research Network on ‘The Role of Europe in Global Challenges: Climate Change and Sustainable Development’ for their insightful remarks. The COST Action INOGOV (IS1309) originally sparked the authors’ interest in the Monitoring Mechanism and funded Jonas Schoenefeld to spend time at the Finnish Environment Institute (SYKE). Andrew Jordan’s contribution was also supported by the ESRC Centre for Climate Change and Social Transformations (CAST) (ES/S012257/1) and the ERC (Advanced Grant DeepDCarb – 882601). Mikael Hildén’s contribution was part of the SET-project (grant STN 314325) and STN Carbon Neutral and Resource Efficient Finland (grant STN 314350). Finally, the authors are very grateful to several EEA Staff Members and Tom Dauwe (European Topic Centre on Climate Change Mitigation and Energy, ETC/CME) for helping them to understand the data reported in this paper.
Notes
1 The EEA regularly publishes the climate policy data from the Monitoring Mechanism/Energy Union Governance Regulation online in the Policies and Measures (PaMs) database. The database is available at: http://pam.apps.eea.europa.eu (last accessed 10 May 2021).
2 Since 2018, the Monitoring Mechanism Regulation (No 525/2013) has been included in the Energy Union Governance Regulation (2018/1999), but the underlying monitoring remains intact.
3 Earlier climate policy data are not available.
4 Projected emissions may include a range of projected external influences on policy effectiveness. We assume that the member states’ modelling has taken these into account as far as possible.
5 The IHHI is equivalent to the Gini-Simpson Index. Economists have typically referred to the former to measure market concentration, while ecologists have used the latter to measure biodiversity in ecosystems. Alternative measures such as Shannon’s H Information Entropy are more precise at high and low levels of diversity and particularly suitable if the number of items (in our case, instrument types) varies (Boydstun et al. 2014). We use the IHHI because it is easily accessible and because the number of available instrument types does not vary in our data. In our case, calculating Shannon’s H produced very similar results (available from the corresponding author on request).
6 For 2017 and 2019, these numbers differ somewhat from those in Dauwe et al. (2019), because the country sample differs and the analysis presented here includes reporting bundles.
7 Please note that the member states were able to assign more than one sector to any one policy.
8 Please note that the member states were able to assign more than one instrument type to any one policy.
9 Earlier monitoring data on climate policies are not available.
10 While the overall number of policies has increased in the EU, this does not preclude the possibility of dismantling for individual policies and member states. It is in principle possible that member states removed some policies while adding others. Detecting such changes would require further analysis beyond the scope of this article.
11 Regulation (EU) 2018/1999 of the European Parliament and of the Council of 11 December 2018 on the Governance of the Energy Union and Climate Action.