The social lab as a method for experimental engagement in participatory research

How does the Social Lab methodology support participatory research? This paper provides an evidence-based analysis of experiences of 19 implemented Social Labs applying experiential learning cycles on the question of how to induce Responsible Research and Innovation in the Horizon2020 research funding scheme of the European Commission and beyond. It looks at the potentials of Social Labs to allow participation in research and innovation addressing societal challenges and contrasts empirical results with the theoretical conceptualisation of a scienti ﬁ c Social Lab methodology. It discusses drivers and barriers of engagement, and provides evidence for the impacts of experimental engagement on participation in the context of the labs, substantiated by concrete examples from some of these labs.


Introduction
Ever since the term wicked problems (Rittel and Webber 1973) was coined, the understanding that Western societies are confronted with intractable societal challenges that cannot be solved by applying a classic linear techno-scientific approach has been growing. As Rittel and Webber (1973) note, the classical paradigms of science and engineering, with a focus on efficiency and optimising solutions, do not apply to the problems of open societal systems (idem). More recently, scholars have begun to realise that social challenges such as food sovereignty, energy security, and social inequalities point to persistent problems that are embedded in the system itself (Schuitmaker 2012). They are complex in the sense that multiple causes and consequences interrelate, and that the concomitant uncertainty cannot be reduced with producing more disciplinary knowledge about them. Furthermore, many different actors are often involved with such complex challenges (idem), every possible solution changes the perception of the problem and 'at times there is no agreement on values, facts and relevancy of facts' (Loorbach 2007). The complex nature of such challenges lies in the fact that they are emergent, with constant new information being generated leading to constantly adaptive behaviour of the actors in the system (Hassan 2014, 19-20;Timmermans et al. 2020). This means that they follow from interactions between the parts within a system and that any attempt to approach these challenges from a linear, optimisation-oriented point of view is doomed to fail.
One possible way to address such persistent open-ended societal problems has been sought in the realm of public participation. In the context of governance of complexity in communities, Wagenaar has written that participation is superior to representative arrangements in dealing with system complexity since it increases the diversity and interaction of a social system (Wagenaar 2015, 29). This contributes to the flow of experiential knowledge and enables actors in a system to 'produce, appreciate, and select productive intervention strategies and arrive at the coordination of problem solving and decision making ' (idem: 18). In parallel, in the context of scientific research and technological innovation, interest in public participation is growing. Here we note the move from the deficit model of public engagement towards approaching the public as a partner for dialogue and even upstream participatory engagement in research and innovation processes (Wilsdon and Willis 2004;Stilgoe, Lock and Wilsdon 2014).
A particular way to bring participation in addressing societal challenges and participation in research and innovation together is through participatory action research. According to Reason and Bradbury, participatory action research is a 'process concerned with developing practical knowing in the pursuit of worthwhile human purposes. It seeks to bring together action and reflection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the flourishing of individual persons and their communities' (Reason and Bradbury 2008, 4).
In governance the epistemic argument for participatory, deliberative democracy is that it increases system diversity and system interaction by contributing to the flow of experiential knowledge through the system. This enables the actors in the system to produce, appreciate, and select productive intervention strategies and arrive at the coordination of problem solving and decision making (Wagenaar 2007). Actors start for instance cross-sector collaborations and engage stakeholders when they experience that they cannot address wicked problems like climate change or a pandemic on their own (van Huijstee et al. 2007;Blok 2022). Participatory action research utilises a similar epistemology that offers engagement through the lens of democratic participation and collective action. Participatory action research correspondingly claims that knowledge is embedded in social relationships and is most effective when produced collaboratively through practice, which is the action component of the research. Although participation appears under a variety of namesranging from Cross Sector Partnerships (Bryson, Crosby, and Stone 2006) to Multi stakeholder Alliances (Lundy, Gottret, and Ashby 2005), and from stakeholder engagement (Freeman 1984) to Quadruple Helix Collaborations (Carayannis and Campbell 2010;Popa, Blok, and Renate Wesselink 2020)they roughly follow a cyclical process of collective knowledge production: observation, reflection, collaboration, implementation, re-observation and re-reflection, recollaboration, re-planning and re-implementation. It is a process to facilitate the flow of experiential knowledge, this time within the research process, whereby participants continually examine and re-examine the problem at hand with the ambition of improvement. It involves taking a vantage point 'away' from the specifics of a particular situation, as understood by the people within, to explore the potential of different perspectives that might assist improvement. It also encompasses taking a vantage point towards the 'here and now' from the different perspectives offering a critical grasp of the problem participants are confronted with in specific local situations. Thus, actors in the research process may produce, appreciate, and select intervention strategies and arrive at problem solving and decision making strategies by changing their own perspectives as well as the theories and practices of others whose perspectives may help to shape the conditions of solving a specific problem or situation (Kemmis and McTaggart 2005).
Participatory methodologies are highly flexible in terms of research procedures, and implementation depends on the specifics of local settings and people. Although it is expected that stakeholders should be involved 'right from the start', in order to identify problems and incorporate relevant interests and values, fundamental deviations in power, vision, goal, sector, and motive among multiple stakeholders limits the possibility to engage them (Blok and Lemmens 2015). What is more, participatory research unfolds in complex processes of relating, influencing action, interaction, identity, and meaning (Mosleh and Larsen 2020)ranging from a focus on consensus (Croteau and Hicks 2003) to constructive conflict (Cuppen 2012) and agonistic approaches (Blok 2019)and thus warrants constant reflection and learning.
Recently, there is a growing interest in more experimental approaches to participation in research and innovation (Lezaun, Marres, and Tironi 2017;Binder et al. 2015;Gross and Schulte-Römer 2019;Chilvers and Kearnes 2015;Laurent 2017). As persistent problems require openness, and thus present unclear structures and blurred boundaries, as well as a strong system dynamics, more experimental forms of stakeholder participation may deliver iterative and agile models of experimentation with different actions to address societal challenges. Such micro level experimentation, when applied to real life settings involving multiple stakeholders, being reflected upon and amended on-the-go in multiple iterative cycles by experimenters, may support solutions and prototypes also on a systemic level.
In this regard, on the basis of early conceptions of Social Labs as fora to collaboratively work on societal challenges (Hassan 2013) the Social Lab methodology was developed to operationalise experimental participatory research (Timmermans et al. 2020). A Social Lab approach brings together intentionally diverse teams of stakeholders and gives them time to experiment and to iterate solutions. The teams do not meet around a project but a shared problem (see below Social Lab methodology). Social Labs provide participants with a specific setting and formats to experiment with developing and discussing solutions to complex technological, societal and ecological challenges (Lezaun, Marres, and Tironi 2017).
While efforts to apply similar approaches can be found in transdisciplinary sustainability research (Schäpke et al. 2018), there is a dearth of insights into the application and usefulness of the approach for participatory action research related to R&I governance. This paper aims to fill this gap by answering the following research question: How can the different steps of the Social Lab process support participatory research for implementing Responsible Research and Innovation? While drawing on the theoretical and empirical insights of participatory research in general, this paper strives to draw on practical experiences gained by the facilitation of 19 Social Labs in the framework of a European Commission Horizon 2020 research programme funded project, NewHoRRIzon. In particular, this article investigates NewHoRRIzon as a case study of the ways the Social Lab methodology contributes to participatory research in experimenting with ideas for incorporating RRI at different levels and scales. The concept of RRI intends to bridge gaps among science, research and innovation communities, and society at large by fostering more inclusive, anticipatory, open and responsive research and innovation systems (Owen, Macnaghten, and Stilgoe 2012;Von Schomberg 2013;European Commission 2012). For this reason, this paper also builds on research in participation practices in the context of RRI (Blok 2019;Cuppen 2012).

Social Lab methodology
Social Labs (SL) provide research settings for experimenting with possible solutions in the real-life context of particular societal challenges where experts and stakeholders collectively work together to initiate actions focused on addressing these challenges (Hassan 2014). They provide a generic and open approach to bringing together different stakeholders for collaboration (Hassan 2013). Social Labs are not guided by predetermined project plans, but instead, they aim at proactive experimentation, testing and trying out possible strategies, approaches, and solutions at the micro level in order to draw lessons for the systemic level of the addressed societal challenge. Notably, these experiments differ from natural science experiments conducted in a closed laboratory. Since potential solutions are developed and tested in the social context the challenges arise, they can be typified as 'social experiments' (Timmermanns et al. 2020;Kieboom 2014). The Social Lab methodology allows researchers to study societal challenges with experts and stakeholders, while at the same time collaborating with them to develop and experiment with potential solutions. Social Labs as a participatory action research method bridge theory and practise with their real-world focus (Timmermanns et al. 2020) and offer an agile approach building on the theory of experiential learning for systematic cyclic exchange between conceptual abstractions and concrete experience (Moon 2004). By integrating existing frameworks of Social Labs (Hassan 2014) and action research, Timmermans and colleagues (2020) provided a theoretical conceptualisation of a scientific Social Lab methodology that is applicable in research contexts. According to the authors, Social Labs encompass six main characteristics: (1) Social Labs offer a space for experimentation with actions to address societal challenges.
(2) This experimentation involves social experiments in real-life settings.
(3) In these real-life settings, the Social Lab brings together various stakeholder groups to actively address the societal challenge. (4) Social Labs are interdisciplinary involving a wide range of expertise and backgrounds.
(5) Experimentation with actions on the micro level supports solutions and prototypes on a systemic level and aims to achieve systemic change. (6) Social Labs have an iterative, agile approach and involve learning cycles, allowing for the evolution of prototypes and solutions over time.
While the characteristics 1, 3, and 4 need to be steered already during the design and set-up phase of Social Labs, characteristics 2, 5 and 6 are reflected in the Social Lab process itself.
On this basis, we next describe how we have translated the Social Lab approach to the context of the NewHoRRIzon project.

The design and set-up of Social Labs in the NewHoRRIzon project
The NewHoRRIzon project, aiming at further integrating Responsible Research and Innovation (RRI) in the research and innovation systems on national and international levels, established 19 Social Labs. Each Lab was dedicated to a different section of Horizon 2020 (H2020), the Eighth European Framework Programme for Research and Innovation. H2020 encompassed different sections and programmes, like the Excellent Science Pillar, which included for instance programming of the ERCthe European Research Council, FET -Future and Emerging Technologies and the Marie Skłodowska-Curie Actions; as well as the Societal Challenges Pillar, funding programmes such as Health, Demographic Change and Wellbeing, Climate, Energy, Food, and a diversity of other areas. The Social Lab process spanned a period of more than 1.5 years from spring 2018 to fall 2019 (Spring 2020 in some cases). In this period, three face-to-face two-days workshops were organised in each SL. Additionally, two cross-sectional workshops brought together all management teams of the Social Labs (and additionally pilot hosts in the second one), allowing for cross-fertilisation and mutual learning. The social lab managing teams comprised a manager, a facilitator and an assistant. Social Lab managers were the main contact persons responsible for recruiting and communicating with participants, taking care of coordination, supporting Lab activities and logistics. Facilitators were the responsible moderators during the lab workshops, they found appropriate participatory formats and facilitated the dialogues between Social Lab participants. The assistants supported both the manager as well as facilitator in their roles.
Each Lab was designed in a way to allow adaptations to the local context and the needs and requirements of the Lab participants and managing team. A Social Lab manual ) as well as a generic cross-lab design for each workshop allowed a basic cross-lab standardisation and thus enabled comparability and a community of practice. Across these 19 labs, 314 of these lab participants actively took part. The participant composition represented different stakeholder groups, with academia being the biggest group (52%), followed by civil society organisations (CSOs 12%), 10% policy makers, 7% funding agencies, 7% industry, 3% from the education sector, and 9% being from another (other or unidentifiable stakeholder group). From a binary understanding, gender distribution was equal overall, although some specific Labs did not have balanced shares of women and men participating.
The scope of the NewHoRRIzon project was to integrate RRI and its political dimensions in the research and innovation systems. This complex challenge was addressed by Social Labs, which offered a setting for experimentation with RRI ideas and practices through so-called pilot actions (c.f. characteristic 1 of Social Lab approach). Pilot actions, in turn, were intended to enhance broader adoption of the RRI concept.
Thus, the Social Labs were seen as an instrument to apply participatory action research, addressing one of the key dimensions of the RRI concept, which is public engagement, as well as allowing for practical processes of 'doing' RRI at the same time. Reflecting the kind of changed relation of science and society envisioned by RRI, Social Lab stakeholders are invited to have a say and contribute to research and innovation, from agenda setting to active participation in the process. This mode of collaboration and co-creation enables a 'consistent, ongoing involvement of society, from beginning to end of the innovation process' (Sutcliffe 2011, 3).
In total, the Lab teams implemented 59 pilot actions (see Annex for a complete list). These actions had organically emerged from each Social Labreflecting the unique combination of stakeholders and their disciplinary and technological foci. Expressed through the pilot actions, Social Labs provided room for active participation in defining problems, collaboratively finding solutions and working on their implementation. The set of pilot actions was thus intended to responsively reflect the needs of the SL stakeholders in the respective field while also cohering to the common challenge of advancing RRI across various European research domains (c.f. characteristic 3 of Social Lab approach).

Material and methods
A reporting template was developed to collect data from the specific Social Lab workshops. Each Social Lab management team was asked to fill in the template within a week after completing a workshop. Further data collection about the Social Labs occurred during the two cross-sectional workshops, documented by capturing all outputs produced over the two days of each workshop. Data was generated through reflective activities designed to stimulate discussions on common challenges, approaches, solutions, and concerns connected to establishing and maintaining the Social Labs and pilot actions. In total, the empirical base of the research at hand comprises 59 reporting templates, i.e. three templates for each of the 19 Social Labs ( = 57) and the documentation of two cross-sectional workshops, (57 + 2).
Documents were analysed qualitatively (Mayring 2010;Krippendorff 2012) following a deductive and inductive coding approach and making use of the qualitative analysis software MAXQDA which support coding across multiple files. In the qualitative coding process, researchers revise the written material with predefined codes (i.e. deductive codes) and identify new topics of interest which are, however, not yet covered by the existing codes (i.e. inductive codes, allowing for the unexpected) (Flick 2014;Reichertz 2012). The deductive codes were based on the templates used for documenting the Social Lab workshops and processes regarding engaged stakeholder groups, employed methods, developed pilot actions and tapped into the specific methodological set-up of the Social Lab. For instance, 'Methods used in Social Lab' is a deductive code from the template. Inductive codes emerged during the coding process on the basis of the collected data. For example, 'workshop locations', or 'methods for supporting pilot actions after selection' evolved as inductive codes. To enhance intersubjectivity, three researchers coded all the material. The resulting coding tree allows an extraction of 'codings', i.e. text snippets that have been associated with a particular code. The final code system had 40 inductive and deductive codes encompassing 1,953 codings; of these 529 explicitly referred to the Social Lab methodology.

Results
The empirical results have been organised according to the qualitative codes and a synthesis of results is given in each of the sections. In order to provide more in-depth information and to illustrate key findings, real Lab examples are described in each section. Thus, we first present the empirical findings in a chronological order when setting up a Social Lab and afterwards synthesise them in light of the scientific lab concept and the six specific characteristics mentioned in the previous sections.

Workshop locations and venuesa space for experimentation
Characteristics of the chosen Social Lab locations are reported influence on the productivity of the workshop in terms of fruitful discussions and reflections, creative development of ideas, as well as concrete planning of new activities (characteristic 1). Lab managing teams put much effort in choosing the location and workshop venue. Many Social Lab managers described aiming at creating a warm and welcoming atmosphere by choosing bright rooms with many windows and in some cases, nice views. To create temporary settings for participatory experimentation (see main characteristics of Social Labs), also rooms and furniture had to be adapted. A flexible setting with chairs and tables to move around providing space for different workshop elements was mentioned as key several times. In some Labs, the managers created the opportunity to move workshop elements outdoors and leave buildings entirely. These labs appreciated pleasant environments not only for breaks, but also for actively continuing to work on the Social Lab process by having 'walkshops'guided discussions in groups while walking in parks, in the mountains or along the seaside (see Wickson, Strand, and Kjølberg 2015). All these applied formats were meant to support the teams to experiment creatively with their ideas and solutions.
Lab example: In the case of Social Labs on Research Infrastructures and Secure Clean and Efficient Energy, the team decided to move the workshops to secluded places (for instance an alpine inn or a sea side resort) to allow the group to retreat from daily businesses and to pay full attention to the workshop. This asked for additional travel efforts, which participants were willing to take into account. In contrast to previous workshops, held close to the managers' premises, all participants including the management team stayed under the same roof, all the time. This allowed for informal and ad hoc encounters besides the workshop, not only during meals, but also in short walks, or other activities in and around the house. The workshop rooms offered many features necessary for enabling the participants to feel welcome and willing to contribute with their time and inputs. Both the meeting rooms and the places to sit together such as a 'fireplace room' or 'sea view café' offered a different flair than the usual. Fresh air and silence at night did the rest, especially to those participants who arrived at the very last minute and out of breath. 'I could not have spent my time better' was the statement of one participant who had arrived all stressed out and was visibly more relaxed when he left after the workshop.
Most importantly, the locations chosen were quiet places close to nature, which were especially suited for activities outdoors. The chosen remote locations could be used even better as they offered much more space without disturbances and therefore allowed expanded activities tailored to the specific aspects of the issues to be discussed. In one session, for instance, the Social Lab process could be reflected physically by leading participants step-by-step through the Lab process by actually following a specific path through the venue grounds.

Recruitment processan inter-and multidisciplinary approach
The recruitment process was challenging, time intensive across all Labs, and included iterations. Stakeholders with heterogeneous backgrounds were personally invited to participate in the Lab to enable interdisciplinary engagement of a wide range of expertise (characteristic 4). Potential candidates were contacted through different channels based on detailed research on the Lab theme, potential stakeholders, and experts. Leveraging existing personal contacts and asking for further dissemination in research networks were vital strategies for successful recruitment. However, this snowball system caused some biases in the composition of Lab teams. As some lab managers reported, it proved difficult to convince people to participate given the overall duration of the Social Labs and the needed efforts to be dedicated. In these cases, participants contacted did not see any 'value' in participating, did not feel up to the task, or they did not feel having the needed expertise for the issue.
Lab examples: In the SwafS ('Science with and for Society'') lab, the management team considered it important not to invite via the personal networks of the project partners themselves in order to avoid meeting up with old friends and colleagues as the NewHoRRIzon project itself was funded under a SwafS call. In their discussions with the project officer at REA (Research Executive Agency), they agreed thus to send out a call for participation to current and closed SwafS projects. The REA officer mobilised his colleagues to send the call to their projects. This happened just three weeks before the workshop and it turned out to be very successful. The team believed that it made a huge difference for the success of this call that it was sent by REA. Registrations for the workshop increased, so that by the end, almost half of the workshop participants joined in this way. In consequence, the composition of the group was far less controllable in terms of various aspects of diversity. Nevertheless, the Lab achieved an unexpectedly high level of heterogeneity in the group, particularly with regard to expertise in certain RRI key dimensions, place of residence, and duration of professional practice (but less so in the gender dimension, with four male and 13 female participants). The lab management team also found it particularly fruitful for the dynamics and outcomes of the workshop that the participants had different levels of knowledge in and experience with science-society topics; some with a long-standing experience in the European framework programmes, others were newcomers and just recently engaged with SwafS and thus had a fresh view on the issue. These diverse experiences combined provided a whole new perspective on RRI, SwafS and what it means for the future, and led to three ambitious Social Lab pilot actions.
Methods for pilot idea developmentaddressing societal challenges Developing pilot actions was at the core of the Social Labs. Identifying real-life environments where different types of stakeholders could actively engage in the societal challenge (characteristic 3) was a driving success factor for Social Labs. Therefore, the first workshop of the Social Lab aimed at the formulation, selection, and organisation of pilot actions (characteristic 5). In addition, each Social Lab was allocated a budget to support the implementation of the pilot actions (€15,000 for each SL). Instructions in the Social Lab Manual as well as a cross-project generic design of the first workshop guided the way for pilot ideas generation in and across all Social Labs. Pilot actions addressed all dimensions of the RRI concept and happened in specific formats and ways chosen by the participants, such as workshops, trainings, discussions, case studies, dissemination activities, and events. Pilot actions led to different outputs such as different kinds of tools, documents, websites and best practice examples (See Table 1 in Annex). Whilst the pilot actions are an inspiring and interesting field of interest in its own right, the analysis of their nature and effectiveness unfortunately extends the scope of this paper, as the focus is put on the Social Lab process and its characteristics that led to these actions.
The suggested method to support the creation of pilot actions was through development of a marketplace of ideas that allowed all Social Lab members to share their own ideas first and then to discuss with others. After the marketplace was filled with ideas, the ideas were noted on flip charts and distributed in the room and then visited by all the team members. This so-called gallery walk was used to enable all 'market visitors' to form an opinion on the pilot ideas and at the same time to have the opportunity to talk to the owner of the pilot idea.
A common challenge faced by most Social Labs in encouraging the development of pilot ideas, was finding a balance between providing an open and creative process of idea generation and providing sufficient guidance on the scope of the potential pilot action. The stakeholders and perspectives present at the Social Lab workshop were critical in shaping the pilot actions. An unrepresented stakeholder group also implied a lack of corresponding pilot ideas.
Lab example: The Social Lab on Marie Skłodowska-Curie Actions emphasised the importance of, first, inviting a divergence of perspectives on pilot actions before, subsequently, converging on ideas and plans. Like in some Labs, participants arrived at pilot actions through World Café exercises. The goal for such brainstorming sessions was presented as coming up with actions to advance different aspects of RRI in their specific H2020 programming. Facilitators often encouraged participants to brainstorm pilot actions in the context of their own projects and organisations. This step was taken as a tactic to better ensure pilot action viability. In addition, facilitators also encouraged participants to take into consideration elements of the larger research and innovation system, for example, related to supporting incentives or barriers. Scaffolded in this manner, actions were intended to connect to the participants' organisational contexts and provide learning in broader R&I system settings.   Methods for pilot idea selectionexperimentation with action on micro level Selecting the most appropriate and doable pilot action idea was a participatory process. In some Social Labs, the pilot action selection was not completed in workshop 1. In these cases, the design and selection process carried over into the second, or even third workshop. Providing adequate time in these circumstances was important to support the formation of pilot-action teams and to further define actions on a micro level (characteristic 5). Additional time to specify actions on a micro level was helpful also for pilot actions selected in the first workshop, but, by the second workshop, were not completed. In such instances a lack of clear commitment, clear goals, or participant time or motivation, hampered action. In these cases, another selection process was implemented in the second workshop. In some Social Labs, pilot actions developed in workshop 1 sparked additional ideas for pilot actions, which participants could subsequently develop in workshop 3. After pilot ideas were developed, they were presented either by the participants themselves or by Social Lab facilitators, sometimes with visual representations such as flipcharts. The Social Labs would then apply workshop techniques such as voting with sticky dots for selecting pilot ideas, and in most Social Labs the ideas were selected collaboratively and transparently.
The criteria for the selection process varied between the different Social Labs. While some chose the pilots with the highest impact on research and innovation in their respective fields, others asked to vote for those to which they wanted to personally contribute or which were the most 'doable' ones. Some Social Labs used several rounds of voting.
In most cases, the selection processes was fairly smooth, as clear group favourites emerged during the process. As the feedback results at the end of Workshop 1 show, the participants expressed that they valued the selection process, the related discussions and the results. However, the selection process was sometimes not easy and required agile facilitation to steer the group and still support them to decide autonomously.
Although the selection process in the Social Labs was generally smooth, the SL management team observed that some good ideas were not pursued. In these cases, participants did not identify with pilot ideas, ownership or participation in subsequent development and implementation of the pilots. Additionally, some Social Labs reported that the selection process created tensions within the group. Sometimes the selection process took longer than expected, which required a flexible adjustment of the remaining programme. Therefore, the selection process was regarded as a key element of the entire workshop, which was a condition for moving forward. Once conceived as a series of activities supporting a range of RRI keys, pilot actions were selected by participants through a variety of sociometric methods (exercises for visualising relations and opinions). First, however, participants were often given a night to reflect on and digest project ideas. This quiet period was important to give participants time to discuss similar and exciting ideas, to consolidate around potential pilot action groups, and to identify ideas of common interest. Upon returning to the workshop the next day, participants were offered the opportunity to change or add to the brainstorm list before voting.
Lab examples: In the Social Lab connected to the European Research Council (ERC), ideas were spaced around the room on flipcharts and participants were invited to stand near preferred ideas. The Marie Skłodowska-Curie Actions (MSCA) Social Lab used a back casting exercise (Quist and Vergragt 2006) to turn future RRI visions into concrete implementable ideas in the present. In multiple rounds, participants were asked to come up with concrete steps to realise their visions until they created a shortlist which they were asked to aggregate into four pilots. On the recommendation of participants, the MSCA Lab also deliberated on a list of criteria that a suitable pilot action would have to fulfil, after which teams formed and plans were developed with the help of predesigned forms. In the Social Lab Future and Emerging Technologies (FET), participants were asked to vote according to three criteria in three rounds. First, on the pilot actions that they thought would make the biggest difference to R&I in their particular programme line; second, on the pilot actions that seemed most relevant to their project or organisation; third, on the pilot actions that they felt they would be most excited to get involved with. Participants were encouraged to use sticky-dots to represent votes, allocating three dots per voting round described above. Based on these voting rounds, a pilot action short-list was developed.

Methods for identifying pilot hostscoordinating the various types of stakeholders
Within the process of developing a Social Lab and planning pilot actions on a micro level with various types of stakeholders, the selection of the pilot hosts was crucial. According to the project's Social Lab Manual, pilot hosts, 'manage the implementation of a specific social experiment (pilot)', 'oversee[ing] the development of a 'prototype' intervention, tak [ing] it to the field and implement the experiment in the case (project, call, or program level) and take care of appropriate feedback to the team and the Social Lab in general' (Braun et al. 2020, 22). Thus, the pilot host has a major role in managing the process and coordinating the various types of stakeholders (characteristic 3). The role of a pilot host might well be demanding, and in addition to intellectual commitment to a pilot idea, it also requires emotional engagement, as well as time and resources to put the selected pilot idea in practice (which became particularly evident during the second cross-sectional workshop among the Social Lab community of practice).
Providing enough time to clarify roles, expectations, and available resources of potential pilot hosts was key to a successful pilot host selection in some Labs. While teams were formed to work on the pilot action development and the next steps to be implemented, not all Social Labs selected pilot hosts in the first workshop but left this designation open for later steps. Selecting pilot hosts was easy in those cases where the pilot ideas were clear and the commitment already established in the process of pilot action selection and development. However, not all Social Labs experienced this process smoothly, with some Labs reporting moments of frustration and unease once pilot action development came to the step of identifying hosts behind the pilot actions. In order to ease the burden of the host and the selecting process, some Social Lab managers offered all kinds of support, for instance offering to host a pilot action themselves, or resources to work on the pilot through the NewHoRRIzon project.
Lab examples: In the case of the FOOD and FET Social Labs, the Social Lab management team adopted flexible approaches to allow hosts to emerge naturally from the discussions and reflections on needs, expectations, motivations, and the further planning of pilot actions. This included a combination of several activities and several rounds of reflections.
Participants were asked to self-organise in smaller groups based on their personal prioritisation and then further plan the pilot actions collaboratively. Only at this point was each groupamong other topics associated with the potential pilot actions developmentasked to decide on a host and supporting member of the team to ensure pilot action implementation. In the FOOD Social Lab, several pilot action hosts and supporting teams emerged from the conversations. In contrast, in the FET Social Lab, one pilot action got a self-nominated pilot host, and the other pilot action got the host nominated by others.

Further pilot developmentimplementing social experiments in real-life settings
Most Social Labs laid the focus of the second workshop on further developments of the pilot activities, to improve them collaboratively and to reflect on the progress of the pilot in between the first and the second workshop. It proved to be important to end workshop 1 with a concrete plan on how to proceed with the pilots, otherwise the time in between workshops was not used to make any progress in the pilots. Participants benefited from leaving the face-to-face workshops with clear next steps in mind.
Building a team around a pilot was an important resource in the pilot development to enable division of tasks. The Lab facilitator was required to emphasise that it was not the sole responsibility of the pilot host, instead clear roles and tasks needed to be defined which ideally were part of the action planning. Sometimes redistribution of tasks was necessary to avoid overburdening team members. Some pilots also sought external support by bringing in experts from a specific field, by asking for feedback, and for dissemination purposes. Sometimes the exchange with external experts even resulted in collaborations with the pilot teams.
The Social Lab management team supported pilot development in many ways: in terms of offering modest incentives as external motivation (e.g. covering travel costs for the workshop and certain pilot action implementation costs) and of supporting internal motivation (e.g. ensuring alignment among pilot action, pilot team, and project objectives), with the aim to enhance the commitment of the pilot hosts and their team; in terms of preparing and structuring the face-to-face workshops and steering the group; and in terms of supporting exchange and communication between workshops. While it is beyond the scope of this paper to delve into the nature and content of pilot actions, it is notable that many of them applied participatory research methods: systematically researching, developing, testing, evaluating, and adjusting actions in collaboration with diverse stakeholders and thereby multiplying the participatory character of the Social Lab.
Lab example: In the case of the Social Lab on Secure Societies (SECURITY) there were originally, as a result of the first workshop, two realisable pilot ideas: The pilot idea 'Mesikäpp' (i.e. Estonian for bear) was developed by participants working in the higher education sector. The objective of the original pilot idea was to build a core architecture, a 'canvas' for implementing projects, which would include various political, tactical, operational and practical themes of security related business, knowledge production, software tools and hardware tools. The pilot team and the Social Lab managers agreed on the further development. This was followed by a number of meetings in which the idea was further discussed and developed into its final form. 'Capacity building of RRI in higher security education', being rather far away from the original starting points. The pilot created and tested on-line course material on RRI (Moodle) for a Police University College. In addition, an interactive game application demo 'Build your dream project' was designed for the students to reflect their learning about RRI and ethics through concrete and interactive project examples.
Face-to-face meetings between the Social Lab managers and pilot hosts were key for the progress of this SECURITY pilot. In the discussions, a shared understanding was created on the very concept of RRI, its meanings in the security field, and on the expectations towards the pilot. In these discussions Social Lab managers had a significant role in coaching and sparring with pilot teams so that the pilot would be specific and concrete enough and within the scope of RRI. Furthermore, there were also various more practical challenges, which would have led to problems or even to the cancellation of the pilot action without continuous support. For instance, one of the core challenges was that the Social Lab resources were very limited and mostly focused on supportive actions. This made it difficult for the pilot owners to devote active working time to the pilot development as it had to be allocated to projects with budgeted resources. Furthermore, the lack of project resources also increased the importance of sparring and coaching the Social Lab managers were giving to the pilot. Thus, the implementation of the pilot required a lot of personal motivation and devotion of time and creativity from the people involved. This was also one of the constant challenges for the Social Lab managers in a number of meetings: How to support and maintain motivation of the participants? A lot of time was devoted to discussing these challenges and trying to find ways to overcome them by linking the topic more closely with their everyday work and providing concrete help in pilot work.
The social lab processchallenges and gainsfostering a learning cycle For many of the participants the Social Lab was a learning experience with personal gains and they have, occasionally, been brought out of their comfort zones as well (characteristic 6, involvement of learning cycles, allowing the evolution of prototypes and solutions over time). The Lab process of thinking together was very much appreciated; the heterogeneous teams and the collaboration among the teams was valued very positively, although the fact that they brought in diverse backgrounds also complicated the process, sometimes resulting in conflicts on different ideas. Labs also created teams to combine forces and bundling cooperation. In many cases, Lab participants felt that they became part of a team, thus it can be stated that often, the Social Labs also generated a feeling of belonging and community.
One of the major challenges described in a few Social Labs was a lack of clarity with respect to the Social Lab methodology, its objectives, and its process. For the Social Lab management team, the level of standardisation across labs persisted as an issue of discussion. A certain level of standardisation was necessary in order to have comparable processes across all 19 Social Labs. It was, however, also important to set up each Social Lab as a place for experimentation, leaving room for bottom-up ideas to emerge and respond in a flexible manner. While the Lab process is conceived as an open process, guiding questions, visions, and social challenges should be kept in mind.
In relation to time constraints and resources, it became obvious that this was relevant in many different aspects: time and resources constraints on participants' side, time constraint in respect to the workshop duration, the timeframe and the respective deadlines for Social Lab phases, and time in terms of the duration of the overall Social Lab process and the need to commit over such a long period of time and to invest in the pilot activities.
Further challenges arose in the interaction with participants: from recruiting participants, to keeping them engaged and dealing with emotions and group dynamics in the workshops. Creating a common vision and uncovering challenges helped to induce agency on different levels. Although the feeling of agency could be supported in most cases and participants were passionate about the core ideas of the Social Lab, it was also observed that the momentum was sometimes lost, their efforts and initiatives stopped before reaching societal change, and before institutionalisation.
Lab Examples: Personal motivation and commitment of the participants in the Social Lab process was vital for the experimentation with pilot actions, however, sustaining this commitment was not an easy task. This turned out to be a challenge for several Social Labs, including the European Institute of Innovation and Technology (EIT) and Instruments of Horizon2020 (INST), which experienced several drop-outs along the way. Common reasons for dropping out were a lack of funding, time, or interest in the project, and the process then highly relied on the personal motivation and engagement of individual participants and their ability to convince their home organisations to put time into the effort. In the case of two pilot groups in INST, internal friction between group members was also a contributing factor most likely because one group member cared deeply about the pilot idea but did not find the needed support from their team, which caused frustration.
During this process, with the challenges of commitment and getting Social Lab participants to carry out their pilot ideas, the Lab managers experienced a dilemma. A dilemma between being primarily facilitative, offering inspiration but not too much guidance, and letting the Social Lab process unfold entirely on the participants' terms without too much intervention versus managing the process more closely: stepping in during workshops to make decisions when participants find it hard to reach a conclusion, setting clearer demands, and help actual planning and task distribution. From the perspective of the Lab managers, the motivation and drive needed to come from the participants themselves; they needed to have ownership of their activities and drive the change in their context. In such cases, some participants needed more management to get the experiment started and to stay on track.

Discussion
Returning to the main research question of this paper: how the Social Lab approach supports participatory research, we can now draw upon our results on the contributions of Social Lab workshop location and the applied methods for pilot activity selection; for identifying pilot hosts, and further pilot development; as well as findings in relation to the overall Lab process and experienced challenges. Reviewing these results, we propose the question can be answered as follows: Social Labs contribute to participatory research via experimental engagement. In this regard, Social Labs can be seen as experimental approach to participation in research and innovation that we encounter in the literature as they offer rooms, iterative loops and reflections throughout the lab process to experiment with ideas and concrete real-life actions (Lezaun, Marres, and Tironi 2017;Binder et al. 2015;Gross and Schulte-Römer 2019;Chilvers and Kearnes 2015;Laurent 2017). Since Social Labs operate in the same field the challenges arise, a first advantage of the Social Lab methodology is that it provides a clear operationalization and roadmap of experimental engagement at the micro level. The second advantage is that it is action driven, as pilot actions and their monitoring are key success factors in the Social Lab methodology.
Experimental engagement in the Social Lab process means that people come together regularly and, using inquiry and real-life experience, try to understand and shape the world in which they live, co-define a problem, and engage in experimentation around potential solutions or ideas that offer directions towards solutions. In most cases, the engagement process is guided by basic rules and minimal critical specifications, mostly by creating a safe space for the free expression of ideas and feelings about a problem situation, and resisting the urge to resolve it immediately. Experimental engagement is an explanatory, practical and dialogical means to interact, transform, and work towards change with others in real-life settings (Chevalier and Buckles 2019). Experiments are co-created, tested and reflected upon in cycles of mutual learning. The experimental engagement in the Social Labs follows the characteristics as argued by Timmermans et al. (2020), now substantiated with experiences from 19 Social Labs, resulting in empirical findings.
Our analysis illuminates that the first characteristic of Social Labs (criterion 1offer space for experimentation with actions to address societal challenges) represents a driver of success of Social Labs. Social Labs offer a space for experimentation and experiential learning with actions to address societal challenges: It was open (of course within the parameters of the project mission: to advance RRI) in terms of what was going to be developed. It was up to the participants to identify the gap between the current situations in the different program lines and the desired future and in that way defining the specific challenge to be addressed and to develop the means to address this challenge.
Regarding the second criterion, experimental engagement in the Labs encompassed social experiments in real-life settings, implying that the pilots resulted in concrete and tangible outputs that were implemented in real and concrete environments where challenges arise, each addressing different aspects of the identified challenge. Methods for developing pilot ideas, and further pilot development have been described in the result section. Labs offer a creative momentum of thinking beyond disciplinary boundaries.
Correspondingly, in these real-life settings, various types of stakeholders participated actively in addressing the societal challenges (per criterion 3). The pilots were the product of collaborative work and experimental engagement of stakeholders with various backgrounds who actively and jointly developed the pilot actions. While the pilot host, and often also the Social Lab management team, were core drivers of the pilot, the responsibility was shared among the Social Lab members. All of the 19 Social Labs were diverse in terms of their members, even though some stakeholder representatives were more difficult to attract, addressing criterion 4 (Social Labs are interdisciplinary involving a wide range of expertise and backgrounds).
Regarding social experimentation with action on the micro level (criterion 5), the pilots were designed as actions to be integrated into the daily activities of the Social Lab members and their ecosystems, resulting in small interventions that thus were positioned to have a longer term impact (Termeer and Dewulf 2019). Data from reflection and reporting templates suggest that pilot actions indeed had ripple effects (Van Huijstee, Mara Francken, and Leroy 2017) in the home institution of the Social Lab members, however to measure this impact lies beyond the scope of this paper (Cohen et al., under review).
Finally, regarding iterative and agile experimentation involving learning cycles and the evolution of prototypes and solutions over time (criterion 6) the series of three workshops in each of the Social Labs, as well as two cross sectional workshops provided a structure for reflection and mutual learning along the lab processes and the phases of development and implementation of pilot activities. Within one and a half years that most of the lab process took, there was enough time to find the pilot ideas and to let them develop over time.
At the same time, the research revealed several barriers that participants experience in their engagement in Social Labs that confirm the difficulty to implement participation in R&I practice (Novitzky et al. 2020). Our results confirm earlier research on barriers to participation. The reluctance to participate because of a deficiency of knowledge, an expected lack of added value, or the insufficiency of clear goals and commitments in the pilot actions is also found in the literature on participation in general. In this regard, the further theoretical development of the Social Lab methodology might benefit from a deeper engagement with related literature on cross-sector partnerships and quadruple helix collaborations, as they highlight the need of a common problem statement and common goal of participation (von Schomberg 2007) in which the participation is conceived as value adding processes (Popa, Blok, and Renate Wesselink 2020). At the same time, the literature on cross-sector partnerships and quadruple helix collaborations could benefit from the results presented in this paper, as they provide insights in progressive strategies like the marketplace of pilot actions, and enable to manage the dynamic process of the participation.

Conclusions
In this paper, it has been shown that Social Labs are appropriate means of participatory research to address societal challenges, if organisers pay attention to the following aspects and pre-requisites in implementation.
First of all, expectations on Lab participants and outcomes have to be made explicit right from the beginning. It needs to be clear to participants that they have a role in shaping their own pilot activity, in making it work for them and in their organisational setting. In the Labs, participants have to undergo the transition from passive participants to active owners of their activities and can thus become agents of change. Second, the work in the Labs is demanding and time consuming. To avoid disappointment or too many efforts at later stages of the process, the greatest possible transparency about the process in general and decision making should be sought.
The experiences in the Social Labs show that experimental engagement of multi-stakeholder teams in practical real life participatory experiments provides a great potential to come up with concrete solutions to complex social challenges. Furthermore, the experience of successful participation and the experimental implementation of real actions or the production of concrete outputs empower Lab participants for future activities as envisaged in participatory action research endeavours.
A potential limitation to this paper is that it predominantly focused on the methodological aspects of running Social Labs as a form of experimental participatory research, specifically providing insight into the methodological choices in implementing Social Labs in practice. Given their systemic ambitions, we believe it may be of interest to further explore and investigate how experimental participatory action research formats such as Social Labs may contribute to more structural and institutional changes (cp. Owen et al. 2021;Cohen et al., under review) in the research and innovation system so that it can better contribute to the tackling of complex social challenges in lasting and systemic ways.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This work was conducted in the realm of the NewHoRRIzon project, which has received funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under Grant number 741402.

Notes on contributors
Dr. ilse Marschalek is a Senior Researcher at the Centre for Social Innovation in Vienna (ZSI), Austria. She is a sociologist with several years of experience in national and international studies. Research fields are inter-and transdisciplinarity, participatory action research and involvement of non-scientific persons into research processes; science communication, participatory evaluation and responsible research and innovation. In the NewHoRRIzon project, she was lab manager, lab facilitator and coordinator of the cross lab learning process. Michael Bernstein is a scientist at the AIT Austrian Institute of Technology and Assistant Research Professor at Arizona State University. He applies descriptive and participatory social science methods to align research and innovation with long-term societal interests, like sustainability. He currently focuses on ethical assessment of new and emerging technologies for the ECfunded TechEthos project. In addition, he is developing foresight and strategic planning tools to support business innovation for social value (The Global KAITEKI Centre).

Robert
Braun is a Senior Researcher at the Institute for Advanced Studies, Vienna in the Research team on autonomous mobility within the Techno-Science and Societal Transformation.
He is specialist in Ontological politics and social epistemology, technology transitions (e.g. autonomous mobility), responsible research and innovation (RRI), [political] corporate social responsibility (pCSR), critical automobility studies, critical data studies. Robert studied philosophy of arts and history at the University of Budapest, in 2002 he did his PhD in philosophy. He is currently also teaching at Masaryk University, Brno where he has submitted his habilitation which is under review. Joshua Cohen is a PhD candidate at the University of Amsterdam Political Science Department/ AISSR. Ever since he studies philosophy and political science and he is interested in making science and innovation more public. He is particularly interested in using action research to contribute to institutional change towards public engagement. In his current work he focuses on improving the relationship between publics and institutions of research and innovation from a pragmatist perspective. In the NewHoRRIzon project, he is involved with action research in the MSCA Social Lab and the comparative narrative evaluation of all 19 Social Labs.
Dr. Margit Hofer is a senior researcher at the ZSI with extensive experience in research and coordination alongside EU projects. Her main research interests include the implication of innovations in education, workplaces and life in general. Over the past years, she has been actively involved in European research projects related to responsible research and innovation as well as maker movements where she focuses on qualitative and quantitative analysis as well as participative and co-designed activities.
Lisa M. Seebacher studied International Development Studies and Sociology at the University of Vienna and did their master's degree in Socio-Ecological Economics and Policy at the Vienna University of Economics and Business. Since 2017 they are working as social scientist at the Centre for Social Innovation (ZSI) in Vienna. Their research focus lies on studying inequalities from an intersectional feminist perspective, participatory research methodologies, pedagogy, as well as the degrowth perspective and movement.
Elisabeth Unterfrauner holds a master degree in Psychology and a postgradual degree in Clinical Psychology and Health Psychology. Furthermore, she gained her PhD in the interdisciplinary Lifelong Learning College. As an expert in learning psychology and qualitative and quantitative methods she was involved in various projects in these fields. Unterfrauner has been a lecturer at the University of Vienna for several years in the psychology department and works as senior researcher and project manager at the Centre for Social Innovation (ZSI) in the Department of Technology and Knowledge. Since 2021 she is head of the department 'Technology&Knowledge'. Her main research fields are evaluation, e-Inclusion, the Maker movement and Responsible Research and Innovation (RRI).
Stephanie Daimer is coordinator of the Business Unit Policy for Innovation and Transformation in the Competence Center Policy and Society and has worked at the Fraunhofer Institute for Systems and Innovation Research ISI since 2008. She holds a Master's degree in political science from the University of Passau (Germany) and received her doctoral degree in political science from the University of Mannheim (Germany) in 2008. From 2013 to 2019 she served as a member of the Executive Committee of the European Forum for Studies of Policies for Research and Innovation (Eu-SPRI-Forum). From 2014 to 2015 she was a member of the expert group Foresight on Key longterm Transformations of European systems: Research, Innovation and Higher Education (KT2050) of the European Commission.
Mika Nieminen is working currently as a team leader (Responsibility and Ethics in Innovations) in VTT, Technical Research Centre of Finland ltd. He has studied e.g. transformation of research and research organisations, intermediaries, spin-off companies, impacts of STI policy, innovation policy, resilience in organizations, and Responsible Research and Innovation (RRI) related questions. He has experience on demanding developmental and administrative tasks in the universities, consultancy work to the ministries and national positions of trust. He has published widely articles, monographs and edited volumes in above mentioned research areas.
Malene Vinther Christensen is an Administrative Officer at the Department of Political Science / Danish Centre for Studies in Research and Research Policy at Aarhus University. Her work is centred around Responsible Research and Innovation and how different organizations work to strengthen responsible practices. She is particularly interested in facilitating discussion and reflection about social responsibility and RRI initiatives and especially on how to engage non-academics in discussions about science and new technologies, which she has done so in several Horizon2020. She has also built an expertise in evaluating RRI initiatives via participatory evaluation designs. Her work has been published in leading journals including Science and Public Policy.
Raj Kumar Thapa is currently working as a research scientist at GenØk-Centre for biosafety in Tromsø, Norway. Research areas: Responsible Innovation, Innovation, Entrepreneurship, Social innovation and entrepreneurship, digital innovation and entrepreneurship.