Co-production and economics: insights from the constructive use of experimental games in adaptive resource management

ABSTRACT I envision new directions in the methodology of experimental games in the field of developmental, environmental and resource economics. Although there have been extensive discussions on experimental practices in recent years, following Guala's [(2005). The methodology of experimental economics. Cambridge University Press] pioneering work, the methodology has narrowly focused on issues related to the internal and external validity of experimental results, in particular the extrapolation of results for policy. I introduce co-production as a popular perspective in the recent methodological discussion on sustainability science, and then I illustrate how it works in the familiar context of game-theoretic studies of common pool resource management. I then distinguish various ways in which methodologists could engage in the normative appraisal of co-production using economics, ranging from conservative to radical approaches.


Introduction
I will use this occasion to introduce so-called co-production as a new scientific practice, and to discuss how economic methodology might normatively engage with it. Even those in the academic realm of philosophy who are relatively insulated from real-world topics will find it difficult not to notice, if not share, the rising concern over climate change and its lasting impact on humanity, which is already unfolding. This and other related topics are clustered as issues of sustainability. I notice this in my philosophy students' interest in sustainability. I and my fellow philosophers have programmatically argued elsewhere how the philosophy of science might engage with sustainability science in general , but considered from a more narrow perspective in terms of how economic methodology (or the philosophy of economics) as a field is responding to this zeitgeist, the answer is 'very little'. One observes an interest in evidence-based policy and the use of behavioral economics for policy (so-called green nudges), but all this is somewhat limited in scope. A strong trend that currently lacks attention is interest in co-production as a method of usable knowledge production, beyond the transfer or application of scientifically established knowledge to policy domains. This blind spot nevertheless offers economic methodologists a great opportunity to make a substantial contribution, because the literature on co-production tends to focus on other sciences and also to use scientific projects as a unit of analysis, leaving aside how economic theories and models participate in the construction of policies. In a sense, economics is a field with a strong tradition of co-production in that economic theory is frequently developed in tandem with policy developments. One aim of this paper is to promote this aspect and thereby create a space for fruitful methodological studies of economics-based co-production.
More specifically, despite the wide-ranging discussions on experimental practices since Guala's 2005 pioneering work, economic methodology has become more narrowly focused on methodological issues related to the internal and external validity of experimental results, notably their extrapolation for policy. Science and technology studies, in contrast, show a broader interest in the constructive role of experiments and models in policy-making (the so-called co-production of science and policy, or knowledge and governance), for instance, and in their coordinative and mediative roles of these as performative and boundary objects at the science-policy interfaces (Van Egmond & Zeiss, 2010). I propose that economic methodology should be broader in focus and should critically appraise the new ways in which science has an impact on policy. To be sure, there has been a major development in the literature on performativity, but it mainly concerns conceptual and epistemological questions such as what it means to say that economic theories perform. I further propose that economic methodology can widen its scope in this way. When I use the term critical appraisal I do not mean a sociological critique of economics that has performative power: it is rather an extension of the normative appraisal of policy effectiveness.
The paper is organized as follows. Section 2 reviews the recent literature on co-creation, drawing on the three-part distinction made by Miller and Wyborn (2020). Section 3 illustrates the practice of co-production using games as a motivating case for analysis. Section 4 discusses how economic methodology could normatively appraise co-production as a methodological practice. Section 5 concludes the paper.

What is co-production?
Co-production is increasingly gaining traction among researchers and policy makers as a new mode of doing science, coalescing into one category predecessors such as participatory action research, mode-2 knowledge production, transdisciplinary research, postnormal science, and civic science. Wyborn et al. (2019, p. 320) define co-production as 'processes that iteratively unite ways of knowing and actingincluding ideas, norms, practices, and discoursesleading to mutual reinforcement and reciprocal transformation of societal outcomes', or alternatively 'processes that iteratively bring together diverse groups and their ways of knowing and acting to create new knowledge and practices to transform societal outcomes' (p. 322). As is clear in these alternative formulations, the 'co-' (jointly) element of co-production refers to the plurality of products (the first definition) or of subjects (or both, the second definition). In other words, co-production refers to the fact that knowledge and public services, for example, are produced by different groups, typically non-academic actors (policy makers and stakeholders, etc.) in addition to scientists. This sense of co-production overlaps with a particular connotation of the term transdisciplinarity (including non-scientific agents) as contrasted to interdisciplinarity (involving multiple scientific disciplines) as a mode of knowledge production. However, it could also refer to product plurality, namely knowledge about a system (e.g. the economy of a country conceptualized through macroeconomics), and the way that system is managed (through interventions in items of macroeconomic national accounting). The second definition cited above mentions both, which seems to assume a natural association between subject plurality and outcome plurality. However, Jagannathan et al. (2020), who reviewed 21 co-production research projects conducted between 2016 and early 2019 in the field of climate adaptation, report a wide gap between theoretical expectations that co-production resulting in a radical transformative change in science-policy-society interaction on the one hand, and the actual reported outcomes on the other, there being more tangible and pragmatic outcomes on the project level. This finding implies that the two aspects of co-production may not necessarily be tightly linked. In fact, scholars in the field of science and technology studies highlight how deeply economics as a theory permeates social life, even though it is regarded as a body of expert knowledge produced for the most part by academic economists and government researchers without much stakeholder involvement in its creation. The (con)fusion about the notion of 'co-' in the literature on co-production is easier to understand if one recalls that the notion has developed in three distinct fields, namely public administration, science and technology studies (STS), and sustainability science (Miller & Wyborn, 2020). Among these, STS, which intersects with the philosophy of economics on the performativity of economic theories, models and experiments, is the most familiar to economic methodologists. In this context, co-production concerns multiple products, in particular scientific knowledge and social order. For example, on the most general level it is argued that the expansion of economic theory coincides with the expansion of the market economy in society. More specifically, the involvement of economists as consultants in the design and implementations of the FCC auction scheme and any other matching and market mechanisms is discussed in terms of questioning in what sense such an application of economics to policy is successful (Guala, 2007;Mirowski & Nik-Khah, 2007). In this context, the legitimacy of narrow epistemic analysis is challenged by the wider consideration of the involvement of power and politics in applying economic knowledge for policy-making purposes. Jagannathan et al. (2020) categorize the goals of co-production as 'scope 1' (incremental, projectbased) and 'scope 2' (ambitious, extended and radical transformation). Within this framework, coproduction with economics tends to be seen as an obstacle to achieving scope 2 goals, embodying some kind of system resilience in a negative sense. However, this is compatible with the recognition of economics as a science with the power to co-produce knowledge and social order, because coproduction in this sense is at work in both the maintenance and transformation of social order (Barnes, 1983).
In contrast, the traditions of co-production in public administration and sustainability science are much more closely aligned with the more tangible normative methodological concerns of economic methodologists. In essence, co-production is promoted in these traditions as a normative principle of effective knowledge production and governance. In other words, there is a normative claim to the effect that (i) knowledge is more valuable when it is co-produced by scientists and non-scientists than when it is created by scientists alone, and (ii) the applications of knowledge to policy is more successful when the mode of governance is co-produced with knowledge than when knowledge is produced in isolation and applied later. As far as I am aware, economic methodologists have said very little about co-production as a normative methodological principle. There have been lively debates on the problem of external validity or extrapolation, for example, with a focus on economic methods of different types (e.g. Khosrowi, 2019, in this journal), but they tend to be preoccupied with the questions of whether and how inferences made using one system can be successfully applied to the external system. Critics of the very notion of external validity (Reiss, 2019;Jiménez-Buedo 2011] also seem to be focused on evidential reasoning as a basis for policymaking.
My invitation to economic methodologists is thus to extend their study to practices of co-production in economics, and to evaluate them. This will require an expansion of the set of conceptual and theoretical tools they adopt, but it does not neccesitate abandoning the standard emphasis on normative methodology. In this spirit, below I briefly review a case in which experimental games are used in a co-productive way.

Constructive use of game-theoretic experiments
To bring home the idea and practice of co-production to economic methodologists, it would be useful to look at concrete co-production practices that are easily recognizable. In this section, thefore, I focus on co-production drawing broadly on game theory. Game theory is the main tool economists use to study the behavior of systems comprising multi-agents with varying interests, and it has been applied in various theoretical, experimental and simulation studies. Philosophers and methodologists are familiar with the approach, including experimental or behavioral game theory (Camerer, 2003), but their main interest has mostly been in epistemic questions such as how theoretical models, simulations or laboratory experiments afford reliable inductive causal inferences (internal validity), and how such inferences can be extrapolated to real target systems, of which researchers lack knowledge and control (external validity). Guala (2005) lay the foundation for this philosophical framework of internal and external validity in the methodology of experimental economics.
To strengthen the external validity of experimental games, there has been a tendency to conduct economic experiments in the field as field experiments. Although this trend relies to a significant extent on the idea of using randomized controlled trials as in social science , another strand of field experiments in economics extends laboratory methods to increase the relevance of the experimental knowledge to policy and practice. In particular, many field experiments concern conservation and resource management. Drawing on game theory for the most part, these experiments tend to be conducted in the global South, or in developing countries, and to involve real decision makers (such as fishermen, forest managers, and farmers) as experimental participants instead of undergraduate students as in the standard laboratory experiments in universities.
The underlying motivation for these field experiments is to produce causal knowledge that can be more easily extrapolated to non-experimental contexts of knowledge applications. In this context, Torres-Guevara and Schlüter (2016), for example, discuss the lack of correlation between the behavior of the same subjects in their field experiment and in a non-experimental setup, and the reasons behind it. In this line of experimental research, as well as in randomized field experiments, the production and use of knowledge are sharply separated and the role of researchers is confined to the former, in other words, to the production of evidence for policymaking.
However, this clear separation of evidence production and policymaking is being challenged outside the field of economics. In a review article in ecology, for example, Redpath et al. (2018) categorize three ways in which game theory is used in conservation and resource management involving conflicts: theoretical, experimental, and constructivist. Whereas the first two approaches are intended to facilitate traditional epistemic goals familiar to methodologists such as prediction, explanation and hypothesis testing, the last one aims specifically at the facilitation of conflict resolution by engaging stakeholders in the knowledge-production process. A representative approach in this category is called Companion Modelling (www.commod.org), in which stakeholders and researchers develop models and simulations in an interactive and iterative process, which serves as a mediating tool for better problem framing, collective learning, dialogue, and collective decision-making (Étienne, 2014).
With regard to the first of the two aspects of the 'co-' in co-production, participants are not merely studied, they join the researchers in creating knowledge: 'constructivism integrates players inside the gamebringing in their needs, desires, beliefs, and intentions, allowing their behavior in the game to represent differences in knowledge and values (Redpath et al., 2018, p. 418). This is expected to result in the better representation of human agency in the models. 'Representation' here has non-epistemic consequences such as stakeholder trust in the model through which management strategies are simulated. Another potential benefit is that new strategies for addressing particular management problems could be produced through the exploration of the system behavior: 'the players are given freedom to explore a range of possible outcomes in strategic situations such that they can reframe the problem and the game, and create new options not initially contemplated by the research team.' (ibid.) I will draw on one study as an illustration of the constructivist use of game theory. 1 Worrapimphong et al. (2010) applied the companion modeling approach in the context of fishery resource management in Don Hoi Lord, a coastal wetland in the upper gulf of Thailand, which is famous for the razor clam (Solen regularis). The context is typical of common pool management, which experimental economists have extensively studied both in the lab and in the field. The main goal of this study includes collective learning and the promotion of discussion among stakeholders, however. To this end, the researchers first built a biological state-distribution model of razor clams, which was plugged into a role-playing common pool resource management game (see Figure 1). The game was played by the fishermen and local government representatives, who then discussed the outcomes and came up with alternative scenarios such as the establishment and rotation of closed zones for harvesting and quota systems. These proposed scenarios were played out in an agent-based model, and the findings were further discussed among the stakeholders. The end result was the identification of lacking scientific information (such as factors affecting clam recruitment processes) as well as of coping management scenarios (such as the quota system, which is not dependent on the missing information about the recruitment process). 2

How should methodologists appraise co-production in economics?
The above case is an example of what Muniesa and Callon (2007) calls platform experiments. As such, it is not a straightforward extension of lab experiments to the field with the same epistemic goals in mind, i.e. robust causal inference by controlled variation. The main goal is rather to interact and to construct reality with the help of experiments as a socio-technological device, to use STS jargon. Thus it seems inappropriate to apply the methodological standards of lab or field economic experiments mechanically to constructive experiments of this type. Although the role-playing game was played without monetary incentives, it does not automatically mean that the players' behavior is an unreliable indicator of their field behavior. Given sufficient interests on the part of the players, such as the epistemic interest of fishermen in enhancing their understanding of the interaction between social-ecological systems, on which their livelihood depends, one would expect that the motivation to 'play for real' is even higher than in an abstract public-goods game in the lab with minor monetary payoffs. 3 How, then, should methodologists evaluate such co-productive use of experimental games? Approaches range from the relatively comfortable to the more uncomfortable. The former reflect the business-as-usual methodology, namely highlighting methodological innovations in a narrow technical sense, as in the consistent operationalization of 'the conceptual model of resource dynamics and fishermen activities' (i.e. the social-ecological coupled model) in computer-simulation and role-playing models. As the practitioners note (Worrapimphong et al., 2010(Worrapimphong et al., , p. 1336, these improvements in modeling pose numerous methodological challenges. Hence, methodologists can evaluate how (well) practitioners address these challenges, focusing on what kind of cognitive, conceptual, and computational issues arise, and how certain epistemic trade-offs are resolved in practice (e.g. MacLeod & Nagatsu, 2016). More specifically, given that economic approaches to conservation and conflict management are sometimes criticized for their lack of an explicit spatial dimension and unrealistic dynamic resource components, one can analyze how modelers address these problems by engaging stakeholders in the validation process.
I refer to this approach as business-as-usual because the evaluative focus is still on how co-production as a method contributes to the epistemic benefits: these include eliciting stakeholders' insights into the nature of conflicts to give a more realistic representation of interactions in socialecological systems. In principle, this means-end framework could be extended to assess whether and in what way co-production facilitates the achievement of wider policy goals by means of increasing trust in the scientific basis of management policy, sharing the framing of problems, and representation (in a political sense), and so on. This is to say that methodologists are able to assess not only how science produces reliable evidence for policy making, but also how it supports (or hinders) different policy goals in more or less direct ways. For example, practitioners of the constructive use of games refer to power inequality (both procedural and substantial) among the stakeholders (Barnaud et al., 2010), and the lack of institutional support for co-production from the higher organizational levels (Worrapimphong et al., 2010) as 'methodological' challenges. Methodologists might be reluctant to describe issues of institutional power as methodological, but this is how some practitioners conceive of methodology. At the very least, methodological analysis of decision-support models inevitably raises the question of trade-off between epistemic and ethical considerations (e.g. Vezér et al., 2018). Given that many economic models are used for decision-support purposes, it is somewhat surprising how little methodological work there has been on the ethical robustness of economic modeling, as opposed to epistemic robustness, for example.
The approach I refer to as uncomfortable addresses the difficult normative question of whether co-production should be considered a mere means to an end. Criticizing the tradition of optimization-based scientific resource management, Norton (2015) emphasizes the need to overcome the sharp separation of means (certain harvesting strategies of renewable resources) and ends (e.g. maximal sustainable yields or maximal economic yields). Specifically, he argues that the wicked nature of sustainability problemsthe plurality of values and worldviews, as well as the uncertainty inherent in the management of complex and dynamic social-ecological systemsdemands that managers shift their focus from the identification of best means to achieve a given goal, to the design of fair and adaptive processes of decision-making and deliberation. One could make the analogous observation that economic methodologists have traditionally made a sharp distinction between means (scientific methods such as modeling, statistics, experimentation, and simulation) and ends (epistemic goals such as explanation and prediction), narrowing their interests in how the former contributes to the latter, just as in the scientific resource-management paradigm. However, as the ends of science become more complex and heterogeneous, ranging from multiple epistemic goals to epistemic and non-epistemic policy goals, and further to multiple societal goals (e.g. there are 17 Sustainable Development Goals on the UN's Agenda 2030 for Sustainable Development, with 169 associated targets), methodologists may have to reconsider the bounds of normative methodology to include explicit discussion on the ends of economics.
I must confess that I do not have a clear vision of such a radical or what might be called political economic methodology. As an economic methodologist, however, I am concerned with the increasing gap between economics and the rest of society. On the one hand, some economists conclude that people often make irrational decisionsone insight from behavioral economics, among many othersand systematically misunderstand how the economy worksa recent 'discovery' in the studies of folk economics (Boyer & Petersen, 2018;Leiser & Shemesh, 2018). 4 On the other hand, disappointment with and mistrust in economics seem to be growing, particularly among young intellectual members of the public concerned with sustainability issues. Rethinking economics as a process and practice of co-production in both sensesco-production with non-economists and co-production of economic knowledge and the social-ecological ordermight be a good step toward addressing this gap between economists and non-economists, and helping make the most of economics as a science for society. If economists are too busy doing their science, economic methodologists should play a bigger role in this endeavor.

Conclusion
In this article, I have introduced co-production as a popular method/perspective in the recent methodological discussion on sustainability science, and I have illustrated how it works in the familiar context of game-theoretic studies of common resource pool management. I have further identified different ways in which economic methodologists could engage in the normative appraisal of coproduction, ranging from conservative to radical approaches. I warmly invite methodologists to join this research program, starting from the approach they find most comfortable.
I would like to conclude my discussion by explicitly addressing some of the questions posed by the editors of this special issue. Below I list the questions touched upon in this paper, to which I give my responses: (1) Should developments in other disciplines play a great role in economic methodology? Economic methodology can be enriched by engaging with the developments in science and technology studies in general. In particular, it is more useful to discuss the role of economics experiments in the context of science-policy interfaces and the co-production of knowledge (science) and governance (social order), rather than narrowly discussing epistemic problems of extrapolation. (2) What sort of relations should prevail between methodology and other disciplines? The role of science and technology studies, from the perspective of economic methodology, is to provide richer details of the institutional and social conditions attached to the use of science for social change, in addition to the epistemic conditions. In this sense, STS does not challenge the traditional role of normative methodology, but rather complements it. However, as I suggest in Section 4, the meaning of 'normative' might have to be extended to include the attainment of non-epistemic societal goals. (3) What developments in the practices of economists practices should methodologists know more about? As a starting point, those familiar with economic modeling, experimentation and simulation could investigate their constructive use in practice. Note that many of these studies are published in non-economics journals. This publication bias in itself may reflect a particular stance in economics that methodologists would like to examine. (4) What balance should there be between explaining economics and appraising it? My answer is that methodologists should do both equally well. I argue that it may be benefitial to expand the bounds of normative-methodological appraisal to increase the societal relevance of economic methodology. This could include rethinking normative assumptions in economics, such as individualism, liberalism and consumer sovereignty. However, this must be done without reducing the emphasis on explaining how these abstract principles figure in real methodological practices (and co-produced modes of governance). The solid description of theory and practice would, I believe, be a major source of comparative advantage among economic methodologists relative to critics of neoclassical economics-consumerism-capitalism-neoliberalism (often deemed to be the same thing) should they decide to join the normative debate on social progress and the role of economics in the science-policy-society nexus.