Reducing ambiguity to close the science-policy gap

Abstract Scientists often worry that their evidence is not used properly in the policy-making process. Their main response is to change the supply of evidence to reduce policymaker uncertainty. They should focus more on ambiguity, combining evidence and persuasion to help policymakers define the policy problem. To do so, they need to understand the policy process in which they engage. They cannot do so alone. Policy scholars can help, by articulating the practical value of policy theories. To help most effectively, they need to state clearly the “causal mechanisms” of the policy process. For example, what causes policymakers to pay attention to an issue informed partly by evidence, or what rules guide their behavior most strongly when weighing up evidence with other factors? In this paper, we show that policy theories have informed this debate, but often without making explicit statements of causality. We draw on the social science causal mechanisms field to improve such analysis and suggest use qualitative methods to clarify and measure causal mechanisms to benefit policy scholars and the wider policy analysis community. A focus on mechanisms can inform policy scholarship, science community engagement and on-the-ground policy work.

1. How can we make scientific evidence count?Develop a mechanismbased theory of policy processes and foster new analytical methods Fears about the diminishing role of experts and expertise seem to dominate plenary discussion at conferences on science advice to government (Gluckman 2011).Scientific articles and commentaries reinforce this picture.A large literature on the "barriers" to evidence impact on policy, and exemplar commentaries, blame decisionmakers for failing to make decisions based on the best evidence (Oliver, Lorenc, and Innvaer 2014;Sutherland and Wordley 2017).Few studies describe how the policy process works or how to engage effectively to improve the use of scientific evidence (Oliver, Lorenc, and Innvaer 2014).Therefore, the literature usually suggests that to bridge the science-policy gap we must reduce uncertainty by improving (a) the scientific literacy of politicians, and (b) the communication of scientific insights, such as by producing shorter reports with less jargon, to (c) increase the impact of scientific findings (Oliver, Lorenc, and Innvaer 2014;Cairney 2016).Meanwhile, within government departments, there has been a focus on increasing "analytical policy capacity" in order to enhance the possibility of policy success by improving the amount and type of information processed in public policy decision-making (Howlett 2009, 157).This evidence-based approach is also focused on addressing uncertainty.Howlett defines analytical policy as "the amount of basic research a government can conduct or access, its ability to apply statistical methods, applied research methods, and advanced modelling techniques to this data and employ analytical techniques such as environmental scanning, trends analysis, and forecasting methods in order to gauge broad public opinion and attitudes, as well as those of interest groups and other major policy players, and to anticipate future policy impacts" (p.162).Yet, for both scientists and policy analysts reducing or even understanding uncertainty is only half the battle.
Equally important is the need to minimize policy ambiguity: the lack of agreement on the causal nature of the problem and the most appropriate solution, determined by evidence and values.Policy problems and solutions can be interpreted and framed in multiple ways.People exercise power to persuade policymakers to adopt certain frames; such as to treat tobacco as an epidemic not an economic good, or to understand "fracking" as an economic opportunity rather than a potential public health risk (Cairney 2016).One can entertain multiple "images" of a problem, but to focus primarily on one image is to signal a dominant way to define and solve a problem (Baumgartner and Jones 2010).
Scientists often have less to say about ambiguity, for two reasons.First, many try to draw an artificial line between facts and values, or science and politics, and prefer to be the "honest broker" rather than "issue advocate" (Pielke 2007;Jasanoff 2008;Douglas 2009;Smith and Stewart 2017).Second, they do not understand the policy process.For example, it is common to discuss evidence use with reference to a mythical "policy cycle" (Oxman et al. 2009;Cairney 2016) or a mysterious "black box" containing irrational policymakers with low "political will," responsible for unpredictable and suboptimal outcomes (Biesbroek et al. 2015).
Policy analysts have also been largely silent on policy process and instead are either focused on highly technical methods (e.g.cost-benefit analysis and risk assessment) measuring or predicting policy outcomes or are more likely occupy the ranks of generalists who are more concerned with the day-to-day business of the departmental demands (Page and Jenkins 2005).Cairney and Weible (2017) lament that "policy analysis research is often conducted with an insufficient appreciation of basic science, and policy process research is often esoteric and detached from practice" (p.620).
Policy process theories already suggest ways to engage in policy making (Cairney et al. 2015).First, use persuasion strategies to influence policymakers who draw on "rational" and "irrational" ways to make decisions.Combine factual and emotional arguments, tell stories with a hero and moral, frame the implications of evidence with relation to the beliefs of key actors, and exploit temporary opportunities in which policymakers are receptive to new ideas (Kingdon 1986;True, Jones, and Baumgartner 1999;Jones and McBeth 2010;Weible, Heikkila, and Sabatier 2012;Cairney and Kwiatkowski 2017).Second, adapt to complex policymaking environments containing many actors at multiple levels and types of government, each with their own institutions, networks, and dominant ways of seeing the world, and influenced in different ways by socio-economic conditions (Cairney et al. 2015).Successful adaptation requires us to engage for long-term to understand the "rules of the game" and form relationships with key actors to earn their trust and understand how they think.
Yet, these recommendations are often vague and may be unhelpful to practitioners.Therefore, this article shows how we can improve understanding and effective engagement by identifying clearly the causal mechanisms of policy processes.Although the mechanisms concept has been widely employed in history, political science, and sociology, it has received at best scant attention in the policy field.This paper represents one of the first explicit attempts to discuss mechanisms in a policy process context.First, it explains the meaning of causal mechanisms and their role in explanation.Second, it focuses on mechanisms relating to the ways in which policy actors address ambiguity.Finally, it describes the payoffs to policy scholarship, science community engagement, and on-the-ground policy work.
2. Peering into the "Gray" box of the policy process Weible and Sabatier (2018) argues that "causal drivers" are at the heart of the scientific assumptions underlying policy theories.Given that many of these frameworks were intended as guides to hypothesis generation and empirical testing, there is surprisingly little granular reference to causality.Causation is often implied or sketched, and there are few explicit discussions of causal mechanisms (Yee 1996;Johns 2003;Steinberg 2007;Real-Dato 2009;Nowlin 2011;Kay and Baker 2015) .Bunge described such incomplete theorizing as a "gray box theory" where causality is assumed but the mechanisms are poorly described and hypotheses are often infrequently tested Morgan and Winship (2014) describe "mechanism sketches" in contrast to "mechanism schema" where the component parts or entities are known.This is the possibility that contributors to the literature using policy-process frameworks may engage in implicit mechanistic thinking even if they do not explicitly use mechanism language, particularly when operationalizing frameworks for empirical studies.Instead, we need to develop a "translucent" box that identifies the specific details about mechanisms or the cause and effect relationships, such as to explain the cause of policymaker attention to evidence on policy problems and their receptivity to evidence-informed policy solutions.When the nature of the mechanism is understood, inferences can be made and rigorous methods are applied.Identifying causal mechanisms helps us to explain how and why some evidence-advocacy strategies have the desired effect and others are resisted.

Practical lessons from a mechanisms approach
We argue that all scholars should be interested in causality, but also focus on practical steps.To do so requires a simple definition of causation, but few exist beyond the identification of actions or entities which produce a regular series of changes from a beginning state to an ending (McAdam et al. 2008).Such discussions of change usually "invoke some form of 'causal agent' that is assumed to have generated the relationship between the entities observed" and "are analytical constructs that provide hypothetical links between observable events" (Hedstr€ om and Swedberg 1998).In other words, we theorize the world to identify its key components and their relationship to policy change.A mechanisms approach is best suited to understand causality in within-case study research which dominates policy research.
Simply put, scholars interested in studying mechanisms are focused on the connection between cause and effects (Beach and Pedersen 2016).This can be a difficult task because mechanisms are often unobservable or hidden phenomena.They are sensitive to variations in context and triggering activators such as events, occurrences, or decisions that trigger or stimulate mechanisms (Pawson and Tilly 1997).Thus, assessing the logic of association helps us to make theory-drive causal inferences (Falleti and Lynch 2009).To identify mechanistic nature of causality requires to uncover empirically traceable processes which will uncover how X produces Y under specific conditions by describing "properties of the relationships among phenomena with the potential to recur, which helps explain why X causes Y" (Hall 2013, 21).
Context is crucial to this relationship.Initial conditions play a key role in determining how mechanisms are triggered.Identifying the context and the mechanism is important when formulating hypotheses.Key aspects of a "setting" influence when and how certain mechanisms are triggered and how they play out.Context is also critical because similar initial conditions may lead to dissimilar outcomes (multifinality) and outcomes can be reached from any number of different developmental paths (equifinality) (Biesbroek, Dupuis, and Wellstead 2017).Various scholars have adopted Pawson and Tilly's well-known "context-mechanism-outcome" (CMO) approach: namely the observed patterns of (un)intended outcomes can be explained by identifying the plausible causal set of mechanisms within the situational context of the process (Pawson and Tilly 1997) (Figure 1).Pawson and Tilly (1997) applied the CMO approach to explain the impact of policy interventions to explain larger social changes.We go one step further, to argue that we can use a CMO approach to make sense of the ways in which policy process theories explain specific types of policy change.
This understanding of causality permits the skeptical scientist to open up the black or gray boxes of policy-making.In doing so, they will find a diversity of causal mechanisms that affect and explain policy outcomes.At first glance, there are different broad mechanism types: structural (e.g.environment, institutions), cognitive (e.g.individual perceptions and ideas), and relational (e.g.network connections between people).Second, mechanisms can span between micro-level (individual) and macrolevel (structural) phenomena (Bunge 1997;Checkel 2006).Given the multi-level nature of policy making these mechanisms are particularly important.These are illustrated in Figure 2. "Situational" mechanisms occur when social structures or environmental phenomenon constrain individuals' action or shape and beliefs."Action-formation" mechanisms link individual micro-level activities or behavior to their actions.Transformational mechanisms are those in which individuals, through their actions and interactions, generate intended and unintended outcomes.
Third, mechanisms have temporal features which are often overlooked (Pierson 2003;Beach and Pedersen 2013).For example, some slow-moving causal processes result in a threshold event that result a sudden change.In the policy and social science, there are many examples of mechanisms that fit these broad categories.Identifying high-level and abstract mechanisms in policy theory still leaves the understanding of the causal factors within policy theory in a gray causal box (Sartori 1970;Falleti and Lynch 2008).
Drawing from Sartori's (1970) "ladder of abstraction," Falleti and Lynch (2008) argue that more compelling and measurable causal explanations can only be realized if mechanisms are disaggregated from a high level of abstraction, which they label as "processes," "mechanisms-as-type," and "mechanisms-as-example."They claim that "mechanisms-as-cause" and "mechanism-as-indicators" are critical to making a measurable causal claim and to describe how things happen.Alternatively, Machamer, Darden, and Craver (2000) argue that mechanisms are often nested hierarchies that contain "lower level entities, properties, and activities" that "produce higher level phenomena" (p.13).There are "the components that are accepted as relatively fundamental or taken to be unproblematic" as far as the observables in the data (Morgan and Winship 2014, 239).Machamer, Darden, and Craver (2000) borrow from molecular biology and find that mechanisms "bottom out in descriptions of the activities of macromolecules, smaller molecules, and ions" (p.14).Identifying mechanisms in this way helps us describe key dynamics such as the use of evidence in the policy process.By doing so, researchers and practitioners will able to understand broad causal issues that define problem definitions (e.g.poverty causes health inequalities).

POLICY DESIGN AND PRACTICE
Fortunately, policy scholarship has developed many mechanism-like concepts which are currently employed in policy process frameworks.For example, procedural rationality underpins the advocacy coalition framework (ACF).ACF scholars have championed the "devil shift" a psychological process whereby individuals exaggerate the negative motives, behavior, and influence of opponents (Weible and Sabatier 2018).Bounded rationality and incrementalism are central to the multiple streams approach.Policy layering and drift concepts are found in the policy feedback theory.The challenge will be to systematically identify and organize the full suite of policyoriented mechanisms.To do so will require the integration of policy analysis and the policy process by suggesting a mechanistic-based evidence policy analysis.

A new lens on analytical policy capacity: mechanistic-based evidence policy analysis
A mechanistic-based analysis style is critical because "when the procedural sides of a policy making or decision-making process have been thought through properly, it will greatly increase the likelihood of substantive problems being resolved" (Mayer, Daalen, and Bots 2013, 181).The policy research and analysis communities should be interested in testing policy mechanisms so as to measure policy-oriented causal inferences and explain outcomes in more practical terms.Typically, qualitative methods are associated with small N between-case and within-case studies, which are so prevalent in policy process research (Steinberg 2007).When the mechanisms are known, analysts can collect diagnostic evidence, theorize variables and empirical proxies, and test hypotheses which then provide a narrative explaining how a particular outcome or set of events came about (Kay and Baker 2015).
Rather than testing for probability, mechanistic policy analysts begin with considering necessary and sufficient conditions, which is derived from Boolean logic, as their criteria (Kay and Baker 2015).Necessity refers to the situation where a causal factor (X) is a necessary condition if the outcome (Y) occurs only if X exists whereas sufficiency refers to the situation in which a condition (X) itself can produce the outcome without the help of other conditions. 1 Process tracing is central to the empirical and theoretical development of mechanism research (Mahoney 2015).
Process tracing, which is more of a general approach than a specific method, can investigate understanding a simplistic change of events related to a single phenomenon, the convergence of a number of conditions, or complex interactions causal factors (Meyfroidt 2016;Trampusch and Palier 2016).This is often achieved by converting narratives into mechanistic explanations.Beach (2017) argues that analysts can develop minimalist or systematic understandings of mechanisms.A minimalist approach considers only diagnostic evidence.A systematic approach explicitly unpacks the causal process and delves into understanding the empirical fingerprint that the mechanism makes and unpacking it into its constituent parts.In part, this can be a function of the variety of information sources collected (archival documents, interviews, reports, memos) that are accumulated over a given period of time are referred to as causal process observations (Charbonneau et al. 2017).Beach and Pedersen (2013) identify three approaches to process tracing, namely: theory-testing, theory building, and explaining outcomes.Theory-testing process tracing is employed when a phenomenon X is causing outcome Y is known but the mechanism is not specified.Since mechanisms are portable concepts they can be applied by policy researchers to further elaborate the long-term nature of policy change (Falleti and Lynch 2009).Theory-building process-tracing is undertaken when the relationship between X and Y is detected but the researcher cannot identify the mechanism or when the outcome (Y) is known, but X is unknown.In both cases, the researcher develops a new mechanism.Beach and Pedersen (2013) describe Janis' (2013) development of "groupthink" mechanism as the cause of the Bay of Pigs fiasco.Theory-building requires considerably more time and effort than theory-testing.In explaining-outcome process tracing the outcome Y is known but X is unknown or the researcher is interested in fully explaining why Y happened.In each type of process tracing, the analyst will develop a causal mechanism.The second step involves operationalizing the mechanism based on "observable manifestation" from different types of evidence. 2From collecting such information, the inferential weight of the evidence and the hypotheses can be assessed using four well-known tests that apply set-theoretic analysis of event (straw-in-the-wind, hoop, smoking gun, and doubly decisive tests) (See Van Evera 1997;George and Bennett 2005).These tests examine necessary and/or sufficient conditions for inferring evidence from the hypotheses exist.The principles of certainty and uniqueness of the evidence reflect the necessary and sufficient conditions.The straw-in-the-wind test supports or weakens a hypothesis but does not exclude it.The smoking-gun test confirms the hypothesis but does not exclude other hypotheses.Hoop tests reject a hypothesis but do not influence other hypotheses and a double-decisive test confirms a single hypothesis and disconfirms other rival hypotheses.Finally, analysts can apply more formal probabilistic Bayesian perspectives that use set theory and necessary/sufficient conditions (Ulriksen and Dadalauri 2016).

What is a policy mechanism about and for?
It may sometimes be tempting to think that politics is dysfunctional; that the number of likes of a tweet holds sway over scientific expertise.Yet, there is a big difference between a policy process that does not work and a process we do not understand enough to influence with evidence.If we know more about causal mechanisms, we can tell the difference and act accordingly.In the case of ambiguity, an effective strategy is to influence the debate on what causes policy problems before people use evidence to try to solve them.In some cases, presenting more information and analysis that reduces uncertainty may even be counterproductive.We argue that the ambiguous nature of the policy-making process requires attention.There is a growing interest in the utility of policy theory in bridging the illusive scientific evidence-policy gap, and the role that evidence plays in an environment of growing ambiguity.Causal mechanisms have been overlooked because they are usually hidden but are sensitive to variations in context.A toolkit equipped with well-elaborated mechanisms is not only useful for precision and depth to understand the generative processes of existing theoretical models but is also valuable for empirical research and enhancing policy-making decisions (Tranow, Beckers, and Becker 2016).
Theories of the policy process are currently presented to the scientific community as a helpful heuristic technique that uncovers the bewildering world of politics.This is a positive step.Ironically, although causality is assumed, very little is actually understood about underlying mechanisms in these approaches thus relegating them to causal gray boxes.However, a significant pay off can be achieved by incorporating causality, inference, and hypothesis testing as part of this research agenda.
Policy scholars should take up the challenge and identify more specific mechanisms via process tracing in existing social science theories or develop new mechanisms.This, we argue may lead to improving, changing, or refuting the broad and sometimes vague assumptions about existing theories and frameworks.However, in order to fully understand the policy ambiguities associated with the policy process, it is necessary to go beyond theoretical discussion of mechanism and empirically identify causal process observations and make inferences.
By taking a new approach to policy science more seriously, this research should provide an enlightenment function for scientists by changing the focus of the debate and rearticulating the nature of policy problems (Weiss 1977).Rather than a one-way stream of data, research, findings, and advice that often goes unheeded by decisionmakers, the policy-science interface is "a series of spaces with multiple knowledge types, political interests and ongoing deliberation" (Evans et al. 2017).The policy process literature provides insights into such deliberations.However, by adopting a more rigorous approach, specifically an emphasis on the causal nature of policy-making, greater buy-in from skeptical scientists can be achieved.A mechanism-based approach also eliminates the longstanding divide between policy analysis and policy theory.By employing methods such as process tracing, a new avenue for analytical policy capacity can be realized.
Thus, the policy scholar, scientist, and public official can benefit from deeper understanding of causal mechanisms.We call on policy scientists to become the creators of policy-based mechanisms, scientists the consumers, and on-the-ground government officials to empirically employ them.By doing so, this may lead to what Dietz, Ostrom, and Stern (2003) refer to as "analytical deliberation" which provides for "improved information and the trust in it that is essential for information to be used effectively, builds social capital, and can allow deal with inevitable conflicts" (p.1910).

Notes
1. Researchers can also undertake counterfactual and convergence analysis (See Beach and Pedersen 2016).2. Beach and Pedersen refer to this evidence as account evidence (empirical material such as interviews, meetings, focus groups), trace evidence (were the existence provides proof), pattern evidence (statistical patterns), and sequence evidence (chronology of events).

Disclosure statement
No potential conflict of interest was reported by the authors.

Figure 2 .
Figure2."Bath tub" approach for identifying different levels of mechanisms.