Policy adaptability in practice

Abstract Designing public policies to effectively address comingled economic, social and environmental issues is a fundamental challenge facing sustainable development policy-makers in the twenty-first century. Raising the stakes is the added challenge of doing so in today’s complex, dynamic and uncertain conditions. Policies that cannot perform under such conditions run the risk of not achieving their intended purpose and hindering the ability of individuals, communities and businesses to cope with and adapt to change. To explore the principles of adaptive policies, a four-year empirical investigation was launched in Canada and India to extract practical insights from complex adaptive systems literature and to study the characteristics of policies that have been effective under changing socio-economic and environmental conditions. Seven core principles for creating adaptive policies were identified and a practical policy analysis tool was developed to help policy-makers translate the principles into tangible recommendations. This paper presents the results of applications of the ADAPTool (Adaptive Design and Assessment Policy Tool) by four provincial governments in Canada on policies aimed at supporting climate change adaptation efforts. Lessons learned from the applications are discussed.


Introduction
The World Economic Forum's 2015 Global Risk Report aptly described the challenge faced by policy-makers in the early throws of the twenty-first century as follows: "Ongoing political, economic, social, environmental and technological developments are challenging many of our underlying assumptions. Across every sector of society, decision-makers are struggling to cope with heightened complexity and uncertainty resulting from the world's highly interconnected nature and the increasing speed of change (WEF 2015)." Indeed the global risk landscape is complex and dynamic, including water crises, fiscal crises, interstate conflict, energy price shocks, failure of climate change adaptation, unemployment, and the list goes on. Creating policies and programs to address any one of today's issues is like trying to hit a moving target that keeps changing its shape. And policy-making is now even more complex and for good reason. In 2015 all member states of the United Nations adopted the 2030 Agenda for Sustainable Development and its suite of 17 global sustainable development goals (SDGs) and 169 targets. The global SDGs are meant to be tailored to national and local circumstances and are described as " … integrated and indivisible and balance the three dimensions of sustainable development: the economic, social and environmental (United Nations 2015)." Given the demands of the global risk landscape and the new 2030 Agenda, public policies will need to take into account not only their influence on a comprehensive range of issues, but they must also be adaptive to an ever-changing and uncertain development context. Some of the first discussion of adaptability in policy-making appeared in the early 1900s in which Dewey (1927) proposed that "policies be treated as experiments, with the aim of promoting continuous learning and adaptation in response to experience over time (in Busenberg 2001)." This line of thinking was popularized almost a century later when Lee (1993) introduced the "adaptive policy" terminology to the literature in his account of ecosystem management in the Columbia River Basin in the U.S. Pacific northwest, building on the adaptive management concepts of Holling (1978). Notions of adaptability are increasingly seen as essential elements for realizing progress on critical issues. For example, the 2012 Global Environment Outlook report of the United Nations stated: "Recognizing that humanity is encroaching on critical planetary boundaries, new modes of adaptive governance are needed to initiate transition management and achieve internationally agreed goals and targets (UNEP 2012)." From 2005 to 2009 the Canadian-based International Institute for Sustainable Development (IISD) and India-based The Energy and Resources Institute (TERI), with funding from the International Development Research Centre, conducted a fouryear empirical study to inductively derive principles for adaptive policy-making under conditions of complexity and uncertainty. The research effort focused on public policies for water and agriculture resource management in the context of climate change in Canada and India.
The conceptual and theoretical aspects of the research were published in book and journal form in 2009 and 2010 and outlined seven core principles for adaptive policymaking (Swanson and Bhadwal 2009;Swanson et al. 2010). These seven principles were then distilled into a series of training courses for policy-makers that were delivered at provincial, national and international meetings in Canada, India and across Southeast Asia. While this evolution from concept to guidance was an important step, it was clear from interaction with policy-makers around the world that more was needed to translate the principles into practical policy advice. This realization led to the creation of the Adaptive Design and Assessment Policy Tool, or ADAPTool for short.
The ADAPTool is a web-enabled, Excel-based tool that guides policy partners through a structured, participatory process that compares existing policies and programs to the principles of adaptive policies . This paper provides an overview of the seven principles for adaptive policy-making and presents the results of ADAPTool applications in four Canadian provinces, with a focus on insights and lessons learned from policy-makers through the process of using the tool to improve policies and programmes that support climate change adaptation efforts.

Concepts and principles for adaptive policy-making
The theoretical and conceptual foundations for creating adaptive policies were published previously in 2009 (Swanson and Bhadwal 2009) and 2010 (Swanson et al. 2010) and were based on a comprehensive review of academic and professional insights on how to intervene effectively in complex adaptive systems as experienced by a range of sectors, including healthcare (Glouberman et al. 2003), information technology (Axelrod and Cohen 2000), international development (Rihani 2002), water resources (Moench et al. 2003), agriculture (Scoones 2004), and transportation (Walker and Marchau 2003).
Additionally, hundreds of interviews were conducted with farmers and water resource managers in Canada and India to identify policies that have helped people and communities adapt to historic weather-related shocks and stresses. From this field, a dozen policy case studies were selected in Canada and India as a means for identifying what makes a policy adaptive to changing socio-economic and environmental conditions. These insights and examples illuminated seven core principles that policymakers have uses to create better policies in complex, dynamic, and uncertain conditions, including (Swanson and Bhadwal 2009;Swanson et al. 2010;Tyler et al. 2014): Integrated and forward-looking analysis. By identifying key factors that affect policy performance and creating scenarios for how these factors might evolve in the future, policies and programs can be made robust to a range of anticipated conditions. Such an approach uses indicators for monitoring and triggering policy adjustments when needed. Multi-stakeholder deliberation. Multi-stakeholder deliberation is a collective and collaborative effort to examine an issue from different points of view as part of a decision-making process. Deliberative processes strengthen policy and program design by building recognition of common values, shared commitment and emerging issues, and by providing a comprehensive understanding of causal relationships. Automatic policy adjustment. Automatic adjustment mechanisms can speed up the response to conditions that are more or less anticipated. They can be used in complicated policy environments by separating the various issues into units (both qualitative and quantitative) in which the understanding of the system is high, allowing for fine-tuning and making adjustments that help reduce risks and maintain performance. Automatic adjustments can be both fully and semi-automatic. Enabling self-organization and social networking. Ensuring that policies do not undermine existing social capital; creating forums that enable social networking; facilitate the sharing of good practices; and removing barriers to local self-organization, all strengthen the ability of stakeholders to respond to unanticipated events through innovation. Decentralization of decision making. In governance terms, the principle of "subsidiarity" means decentralizing decision making to the lowest effective and accountable unit of governance. The adaptive advantages are better opportunities for feedback and information sharing to ensure that decision-makers are aware of unexpected problems and effects of proposed interventions, as well as the nature of different interests. Promoting variation. Given the complexity of most policy settings, implementing a variety of policies to address the same issue increases the likelihood of achieving desired outcomes. A common risk-management approach is developing diverse responses, which make it easier to perform efficiently in the face of unanticipated conditions. Yet another approach is to use policy tools to facilitate variation by removing barriers to alternative solutions and providing information to support the exploring of options. Formal policy review and continuous learning. Even when a policy or program is performing well, regular review and well-designed pilots administered throughout the life of the policy/program to test assumptions related to performance can help address emerging issues and trigger value-added policy adjustments.

Methodology and engagement process
The ADAPTool was originally developed in 2011 in a collaboration with the Saskatchewan Water Security Agency and Manitoba Agriculture, Food and Rural Initiatives to examine public policy support in the context of climate change adaptation. In 2013 the tool was further applied and refined in collaboration with the provinces of British Columbia (B.C. Ministry of Agriculture 2013a, 2013b), Manitoba (Echeverria et al. 2013a(Echeverria et al. , 2013b, Saskatchewan (Roy 2013 andRoy andZubrycki 2013) and Nova Scotia (Bizikova and Vodicka 2013) with funding from Natural Resources Canada's Adaptation Platform.
Through the ADAPTool applications, provincial agencies sought to better understand the potential for the selected policies to support climate change adaptation efforts and for the policies to be adaptable themselves to the environmental and socio-economic shocks and stresses that may accompany climate change. In addition, the analysis was to help "mainstream" awareness and consideration of climate adaptation in policy processes.
The web-enabled and Microsoft Excel-based ADAPTool was used to examine policies and programs in relation to a defined stressor (weather variability and climate change in this application). The ADAPTtool examined the selected policies in two ways: 1. Assessed the ability of policies to support the adaptation actions of stakeholders in response to a stressor; and 2. Assessed the adaptability of the policies themselves to the stressor.
Using the tool, provincial government program staff were led through a series of fifteen questions born out of the seven principles for creating adaptive policies (see Box 1). This standardized list of questions allowed the project team to work with a number of provincial policy-makers to follow a common methodology. The methodology for applying the tool in all provinces was similar, with variations to suit the different contexts (Table 1). In all cases, the ADAPTool was introduced to relevant provincial government staff through a one-day training session. This introduction emphasized that policies or programs were not being evaluated against program objectives, but instead were being analyzed for their adaptability to a defined external stressor (weather variability and climate change, in this case).
Scope of analysis worksheet: 1) What is the geographic scope of the analysis (e.g. watershed, municipality, region, province/state)? 2) What is the stressor of concern (e.g. climate change impacts such as drought and excess moisture)? 3) What are the policies/programs to be assessed? Adaptation Analysis Worksheets: 4) What are the main sectors relevant to the policy(ies)? 5) In what ways is the sector vulnerable to the stressor? 6) What adaptation actions might be necessary to address the vulnerabilities? 7) Are the identified adaptation actions supported by the policies/programs? Adaptive capacity analysis worksheet: 8) Is the policy itself vulnerable to the stressor identified? 9) Does the policy enhance the capacity of actors within each sector to adapt? 10) Were foresight methods and multi-stakeholder deliberation used in the scoping and design of the policy? 11) Were foresight methods and multi-stakeholder deliberation used in the implementation of the policy? 12) Does the policy enable self-organization among affected stakeholders? 13) Is the policy sufficiently decentralized to the lowest and most accountable unit of governance with appropriate resources and capacity? 14) Are a variety/mix of policy instrument types used? 15) Does the policy have a regular formal review process in place that can detect emerging issues? Synthesis worksheet An aggregate ranking was provided for the overall suite of policies/programs, as well as for each individual policy/program.
After the training, provincial program staff worked with project personnel to implement the analysis. In each case, a different set of policies or programs was analyzed, according to provincial priorities. Provincial staff contributed to the analysis which was led by the project team in each case. The initial stages of the analysis dealt with selecting focus sectors of the economy and then identifying each sector's vulnerability to weather variability and climate change. This stage was completed through in-person interviews and/or a series of phone conferences and emails to identify key vulnerabilities and adaptations (including 35 staff in British Columbia, 20 staff in Manitoba, 10 staff in Nova Scotia and 20 staff in Saskatchewan). At this point, the outcomes of prior climate change vulnerability assessments were included in the analyses to inform the nature of the stressors, vulnerabilities and adaptation needs.
Through the ADAPTool applications, 27 policies were analyzed, including 20 discrete policies and seven broadly-defined goals from Saskatchewan's 25 Year Water Security Plan. The policies covered diverse economic sectors, including agriculture, forestry, water, and parks and protected areas management (Table 2). Within each of the policy sectors, specific target sub-sectors were separately identified for purposes of climate vulnerability analysis (e.g. the agricultural policy sector was subdivided into different industry sub-groups such as grains and oilseed producers, field crops, dairy, cattle, orchard and fruit tree crops). The specific climate stressors chosen, against which to assess the policies, were drought, heavy rainfall and flooding.
Once the adaptation needs were identified and entered in the ADAPTool, the provincial government and/or project team analysts began assessing the policies based on the questions outlined in Box 1. The project team helped to clarify the questions, give details about the scoring and provide examples from already-assessed policies. In Manitoba and British Columbia, the assessment was led by the provincial policy-makers who consulted with the project team and provided the outcomes for review. In British Columbia the project team provided support to enable Ministry of Agriculture staff to lead the analysis, including technical guidance, training, interview design and facilitating some of the interviews, as well as collection and synthesis of data. In Saskatchewan and Nova Scotia, the policy assessment began with a project team member who then consulted the provincial policy-makers about scoring and the key characteristics of the analyzed policies. Once the ADAPTool process was completed, the project team developed a report for each of the four provinces, which were provided to the provincial policy-makers for review. In British Columbia, where follow-up actions were planned, there was an extensive review process that included all program managers in iterative reviews of summary results, draft reports and final documents prior to briefings for the Ministry of Agriculture's senior executives. Once approved, these reports, along with the completed Excel spreadsheets and the Table 1. An overview of application approaches across the four provinces (Bizikova et al. 2014 outcomes of a series of interviews, formed the basis of a synthesis report (Bizikova et al. 2014). The scoring method varied across the fifteen questions (Box 1), but generally followed a three or four-point scale. For example, Question #7 used a four-point scale as follows: score 2 if the adaptation action is directly supported by policy; 1 if indirectly supported; 0 if not supported; À1 if the policy potentially hinders the ability to implement the adaptation action. Alternately, Question #12 was scored based on the policy's ability to contribute to the sharing and copying of best practices and lessons learned, scoring 2 if the policy provides direct support, 1 if partial support, and 0 if no support. A more detailed description of the scoring methodology is provided by Tyler et al. (2014).
Scoring was typically done by program staff familiar with the details of the policy or through interviews with program staff by members of the analysis team familiar with the intent and process of the ADAPTool. Data collection was seen as a learning process. Those already familiar with the policies they administered were offered a different perspective through the lens of adaptive policies. While everybody likes to feel that their program scored "well," there is no need for all programs to be adaptable. A low adaptability score may be perfectly reasonable for a narrowly targeted program of limited duration. And a high score may increase expectations for performance under stress. Therefore, respondents were discouraged from strategic responses intended to make their program "look better." Data collection involved iteration and discussion. This required staff time investment, but it was also a key benefit and a source of broader organizational learning.

Application results
The results of the tool applications can be summarized according to planned adaptability versus autonomous or adaptability. Planned adaptability refers to the ability of policies to address anticipated issues. Autonomous adaptability in the context of this paper refers to the ability of a policy to adapt to unanticipated conditions. A visual summary of policy adaptability scores across all policies examined is shown in Figure 1. The figure shows the scores of each policy along two axes: planned adaptability (Questions #4 through #11 and Question #15) and autonomous adaptability (Questions #10-15). Policies that scored in the green zone are considered adaptive to both anticipated and unanticipated conditions, while those in yellow are considered less adaptable and those in red considered not adaptive. Of the 27 policies/programs assessed, six scored in the green zone (four from British Columbia and two from Manitoba), two policies scored in the red zone (one each from Nova Scotia and Saskatchewan) while the rest scored in the intermediary yellow zone.

Results related to planned adaptability
Planned adaptability is facilitated via direct support to stakeholder adaptation actions, building the adaptive capacity of stakeholders, using foresight approaches and through multi-stakeholder deliberation. ADAPTool questions #4 through #10 and #15 addressed these aspects. The results are summarized below.

Support to specific stakeholder adaptation actions (Questions #4 through #7)
A key question of decision-makers and policy-makers is to what degree can a suite of policies potentially support the adaptive actions needed by stakeholders on the front lines? Across all policy applications, 168 stakeholder vulnerabilities to weather variability and climate change were identified across the 23 economic sub-sectors (Table 3). To address these vulnerabilities, 468 possible adaptation actions were identified as important by the government personnel involved in the analysis.
Of all the adaptation actions identified, more than half were considered not relevant to the policies examined (i.e. not at all within the scope of original policy intent). In other words, for this set of actions, it made no sense to ask whether the policy supported or hindered adaptation needs. This was not surprising considering the diverse nature of the sectors and adaptation actions identified, and the relatively small sample of policies. Of the adaptation actions that were considered within the mandate of the suite of policies examined, between 10 and 20 percent of these actions were directly supported by at least one policy (Figure 2). The types of adaptation actions most directly supported by the suite of policies were those related to economic policy, strategy, and monitoring. Adaptation actions that were least supported by the policies examined were related to research, awareness raising, and providing information. For most of the part, however, the identified adaptation actions were at least indirectly supported by the suite of policies examined.  Vulnerability of policy to stressor (Question #8) With regard to a policy's vulnerability to the stressor (weather variability and climate change in these applications), the results revealed that 30 percent of the policies examined were considered directly vulnerable to the stressor, while approximately 30 percent were not (with 40 percent being marginally vulnerable). Of the directly vulnerable policies, the majority were related to the water and forestry sectors.

Supporting adaptive capacity of stakeholders (Question #9)
Question #9 asked if the policy enhances the capacity of actors to adapt to the stressor. In doing so, the ADAPTool uses the determinants of adaptive capacity as set forth by Smit and Pilifosova (2001) to examine if a policy provides access to: relevant financial resources; technology; information and skills; infrastructure; institutions and networks; and all of the preceding in an equitable manner. For the most part the analyzed policies provided stakeholders with relevant information and skills and access to information and networks. The lowest level of support given by the policies was for access to relevant technology and infrastructure.

Use of foresight and deliberation in policy design (Questions #10 and #11)
Through questions #10 and #11 the ADAPTool examined the use of foresight methods and multi-stakeholder deliberation in both policy design and implementation (the British Columbia interviews were completed before the foresight question was added).
In the analyzed policies, stakeholders deliberation and consultations and/or foresight methods were used in close to 60 percent of the policies during the policy design stage (Figure 3). For the recently developed policies, formal multi-stakeholder consultations were often guided by municipal/provincial regulation on stakeholder consultations. Most of the analyzed policies undertook major formal consultation efforts, both through face-to-face consultations and by providing opportunities for stakeholders to comment and provide input through designated websites.

Results related to autonomous adaptability
Autonomous adaptability, or the ability to adapt to unanticipated conditions can be facilitated through engagement with key stakeholders during the implementation of the policy, enabling self-organization, decentralized decision-making, promoting variation, and conducting formal policy review.
Use of foresight and deliberation in policy design (Questions #10 and #11) In addition to facilitating planned adaptability, the use of foresight methods and multi-stakeholder deliberation also contributes to a policy's ability to respond to unanticipated events through the insights provided by the analysis of stakeholder input. During the implementation stage, the ADAPTool results revealed that approximately 20 percent of the policies examined used both multi-stakeholder deliberation and foresight analysis to some degree. As would be expected, this is less than seen during the policy design stage. However, over 50 percent of the policies reported a partial use of one of the methods during policy implementation.

Enabling self-organization (Question #12)
Across the analyzed policies, high levels of direct and partial support were provided for enabling self-organization and social networking (almost 90 percent). The policies supported social networks and self-organization in a variety of ways: extension services through field days and meetings-bringing ranchers together, discussing science, pilot application of suggested practices, supporting local agriculture advisory committees, and offering workshops in key areas such as new standards and guides.

Decentralized decision-making (Question #13)
The suite of policies examined supported decentralization to various degrees ( Figure 4). In terms of direct support, the decisions and actions outlined in the policies and programs were frequently devolved to regional and local offices. As an Figure 4. Results for policy support of decentralization of decision making (Bizikova et al. 2014).
example, some programs were implemented by forest or water resource managers in collaboration with farm operators and other specific stakeholder groups such as Indigenous peoples and rural municipalities. Finally, some policies used more centralized proceduresfor example, needing to obtain provincial approvals for decisions and signatures.

Promoting variation (Question #14)
Complex systems insight and policy experience demonstrate that utilizing a diversity of policy instruments to address the same issue increases the likelihood of achieving desired outcomes. Among the policies examined and listed in Table 2, there were 14 regulatory instruments, 17 expenditure instruments, 13 institutional instruments and 7 economic instruments (noting that several policies were categorized by more than one instrument type). This represented a good mixture of different policy instrument types.

Formal policy review and learning (Question #15)
Almost 40 percent of the policies examined used formal review procedures and 52 percent used partial review procedures. For example, the Manitoba Timber Quota Allocation policy is reviewed every five years. Partial support for review procedures means they include reviews of only some of the issues and outcomes of the policythe forest renewal charge rates in the Manitoba Forest Renewal Program are reviewed every year. Partial support also includes using only internal reviews involving small groups of experts and policy-makers and/or conducting a small, internal review and not publishing the results for broader deliberation and response.

Lessons learned from the application process
In order to gain specific insights about the process of using the ADAPTool, a series of semi-structured interviews were conducted with twelve provincial staff members across the four provinces who had been directly involved in the applications. These interviews explored key processes, challenges and lessons learned from the pilots. Insights from the interviews revealed that in general, the tool performed the analytical tasks expected from it, identifying the ways in which different policies were more or less adaptive to both anticipated and unanticipated stress. These analytical results were helpful to the analysts in two ways: (1) they showed provincial climate adaptation analysts some of the strengths and weaknesses in existing policies in relation to supporting climate adaptation in a particular sector; and (2) they provided insights to sectorial policy analysts and program managers about how and why their programs could be made more adaptable. Participants in the process reported gaining several insights about their programming that were likely to affect future management decisions. For example, some reported that they could see how specific modifications to their programming could make it more adaptable; others reported a greater appreciation for the drawbacks of inflexibility in current regulatory structures. The results of the analysis were validated by staff directly involved in the programming, demonstrating both that the results were consistent with staff experience, but also that they were readily understood and interpretable.
The experience also suggested that the ADAPTool worked predictably when applied to existing policies that had been implemented in practice. However, the tool required more careful interpretation when applied to policies in the design stage or to new policies, due mainly to the fact that many of the diagnostic questions in the tool assumed some period of implementation. When applied to cases in which new policies were already in a fully formed state the tool did not prove useful. This was because the attention of analysts was understandably focused on other policy launch issues, and the timing not conducive to responding to lessons from the adaptability assessment. Additionally, analysis of policy goals alone (as in Saskatchewan's Water Security Plan) proved unsatisfying and generated little insight for analysts. Our conclusion is that the ADAPTool is most appropriately applied either at the very early stages of policy design (using modified diagnostics), or after substantial policy implementation experience.
Another lesson from the applications was that the motivation for conducting the analysis may differ between the technical analysts and the policy managers. During application of the tool, the motivation for the analysis of adaptability to climate change came from concern about climate impacts and adaptation (the stressor in the tool applications). But program managers and policy makers in other sectors had other priorities and did not see the value of the adaptability assessment to that particular stressor. Therefore, it is important at the outset to ensure that there is clarity about the policy rationale for the assessment. In order to gain broad support for the time and effort required by staff, the best results were born out of situations where responding to the stressor (climate change, in this case) had already been articulated as an explicit cross-cutting policy goal at the provincial level.
Post-application interviews also suggested that the tool was a useful platform for learning and exchange between different kinds of analysts who otherwise might not interact frequently. The tool had two main functions, each of which corresponded to a slightly different target group of policy decision makers and analysts. The first function was to gauge the ability of existing policies or programs to support climate adaptation actions and adaptive capacities of the actors who are primary targets of the policy in question. This part of the analysis was most relevant to those involved in planning and mainstreaming climate adaptation. The second function of the tool was to assess the general adaptability of the policies or programs themselves, particularly to unanticipated changes. This part of the analysis was probably most useful to policy designers and program managers attempting to build the most effective programs. According to participants, connecting these two audiences in deliberation and discussion was a central benefit from application of the tool.
The tool also served an important capacity-building function, building awareness not only of the stressor and its potential impacts, but also of policy adaptability in general. While the analysis can be successfully undertaken with varying levels of staff input, the greatest benefits to policy analysis and design seemed to come from higher levels of staff engagement in the analysis itself. Beyond identifying candidate policies, data sources and reviewing results, engagement in the analysis was important to become familiar with the tool, how it works, and how the results should be interpreted. Staff engaged in the project were able to achieve this familiarity after a one-day training session, with the aid of limited coaching and problem-solving support. With this kind of familiarity, further applications of the tool become much simpler, recognition of adaptive policy features comes more easily, and general conclusions for policy design can be formalized and integrated into standard operating procedures with accumulated experience.

Conclusions
The ADAPTool applications in four Canadian provinces provided a productive platform for knowledge exchange between technical analysts and policy decision makers that might not otherwise have taken place. The assessments differentiated clearly between different policies and exposed specific factors that limit policy adaptability and provided new insights to program managers and policy decision makers.
The applications also provided capacity-building opportunities for key groups engaged in policy process such as for policy decision-makers and analysts to interact in a structured target-oriented manner. Given that climate change adaptation covers multiple sectors as well as requires mainstreaming into current policies and decisionmaking, the interactions between different sectoral policy-makers, program managers and analysts is critical. From the analyst's point of view, identifying candidate policies, data sources and reviewing results, and engaging in the analysis had important benefits for understanding the whole suite of policies which they may not have otherwise been familiar with beyond their specific focus. Connecting these two audiences in deliberation and discussion was a central benefit from application of the tool.

Disclosure statement
No potential conflict of interest was reported by the authors.