Experimentation, learning, and dialogue: an RRI-inspired approach to dual-use of concern

ResponsibleResearchandInnovationispromotedbyresearchfundersandscienti ﬁ c communities as a way to place societal needs and values at the centre of research and innovation. In practice, however, legal compliance still tends to dominate the RRI agenda. In order to move beyond the dominance of legal compliance and address a broader societal agenda, this article argues that RRI requires: (1) a productive intertwining of research and practice; (2) the integration of anticipation, re ﬂ ection, engagement, and action (AREA) in a non-linear process; and (3) an experimental approach. Based on this framework, this article draws on our experience of developing and institutionalizing an RRI-inspired approach to address dual-use and misuse issues in the EU-funded Human Brain Project. Our experience suggests that the four dimensions of the AREA framework work better not as separate stages but rather being ﬂ exibly intertwined to enable experimentation, learning, and dialogue.


Introduction
Over the past decade, Responsible Research and Innovation (RRI) has been promoted by research funders and scientific communities as a way to put societal needs and values at the centre of research and innovation (Doezema et al. 2019;Frahm, Doezema, and Pfotenhauer 2021;Owen, von Schomberg, and Macnaghten 2021;Stilgoe, Owen, and Macnaghten 2013).However, experience shows that in practice, legal compliance still tends to dominate the RRI agenda (Stahl et al. 2021).This raises concerns about the implications of narrowing the scope of RRI and highlights a need to resituate it within a broader purpose that attends to the social, political, and economic context as well as implications of science and technology (Aicardi, Reinsborough, and Rose 2018;Salles and Farisco 2020;Stahl et al. 2021).This article examines the following questions: how can we mobilize RRI to address a broad range of societal questions and concerns beyond legal compliance?What does this entail, and what challenges and opportunities arise in the process?To address these questions, we examine our experience of developing an developments (Doezema et al. 2019;Fisher 2020;Owen, von Schomberg, and Macnaghten 2021).Even within the HBP, RRI is conceived of and implemented in multiple ways (Stahl et al. 2021).We maintain that the lack of specificity in defining RRI is in itself a strengthas a set of loosely held together orientations and practices, it allows for the fluidity needed to work across a range of scientific and technological domains while remaining coherent enough to attract and consolidate people and resources.It is important to retain the fluidity and multiplicity of RRI and challenge attempts to standardize and operationalize its concepts and practices via toolkits and indicators (Owen, von Schomberg, and Macnaghten 2021).In this section, we highlight the following closely related aspects of RRI that we consider to be key in moving beyond the dominance of legal compliance in order to address broader societal issues at the project level: (1) the importance of intertwining research and practice; (2) the integrated dimensions of anticipation, reflection, engagement and action (AREA); and (3) the need for a collective experimental approach.

Intertwining research and practice
The HBP Ethics & Society team's approach aligns with the view that the implementation of RRI in the neurosciences requires not just normative discussion but also conceptual analysis and a deep empirical knowledge of the research, clinical, cultural, and social contexts to be intervened in (Gardner and Williams 2015).Indeed, since its inception, the Ethics & Society team has used historical research and methodologies, ethnographic research, and in-depth interviews to engage with the HBP research communities while developing background reports for its joint publications.Empirical grounding and theoretical reflection have formed the basis of engagements with researchers within and outside the project to encourage reflexivity about the social, political, and economic contexts and implications of their research, and have been foundational to implementing a number of the recommendations that we later discuss.
The mechanisms proposed to intertwine the Ethics & Society team's research and practice have included: (1) developing joint research documents known as 'Opinions' on relevant issues, in collaboration with internal and external stakeholders; (2) developing concrete recommendations to the HBP governing bodies and others; and (3) following-up on these recommendations in collaboration with relevant stakeholders within and beyond the project.To facilitate RRI practices and implementation within the HBP, the Ethics & Society team developed and put into effect additional processes and structures such as the Ethics Rapporteur Programme and the Ethics Advisory Board (Aicardi, Reinsborough, and Rose 2018;Rainey et al. 2019;Stahl et al. 2019aStahl et al. , 2019bStahl et al. , 2021) ) that are particularly relevant to our work on dual-use (for more information, see the section on institutionalizing an RRI-inspired approach).

Integrating anticipation, reflection, engagement and action
The work of Stilgoe, Owen, and Macnaghten (2013) led to the development of the AREA framework for responsible innovation (anticipation, reflection, engagement, and action) 4 which was subsequently endorsed and adopted by the HBP's Ethics & Society team.While the European Commission's RRI framework centres on six keyspublic engagement, gender, open access, ethical issues, education, and governance (Owen, von Schomberg, and Macnaghten 2021)these represent isolated themes rather than a coherent approach.By contrast, the AREA framework offers a broader, integrated vision rooted in the social study of science and technology (Owen and Pansera 2019).The emphasis on the need for public engagement, researcher awareness, and a conceptual analysis of HBP research in the HBP's first proposal to the EC in 2012 'lent itself especially well, a posteriori, to the broad AREA framework for RRI proposed by the EPSRC in the UK, rather than mirroring the "six keys" of the EC framework' (Aicardi, Reinsborough, and Rose 2018, 27-28).The HBP was, therefore, one of the first largescale science and technology projects to formally adopt an RRI approach in the early stages of its design and development and the first to adopt the AREA framework to specifically address ethical and societal issues within international neuroscience research.
Moving from anticipation, reflection, and engagement to action -'closing the AREA loop'and ensuring that the recommendations made by interdisciplinary groups such as the HBP's Ethics & Society team influence the governance of the projects they are embedded in, is particularly challenging.Initially, the Ethics & Society team organized its activities to reflect the AREA framework: anticipation of the potential future impact of new knowledge and technologies; reflection on the conceptual, social, ethical, and regulatory implications of neuroscientific research and emerging neurotechnologies; engagement and dialogue with diverse stakeholders on issues of immediate relevance to the HBP; and action based on insights gained through anticipation, reflection and engagement.These research-based activities were carried out by four work packages: the Foresight Lab; Neuroethics and Philosophy; Engagement; and Ethics Support.As we demonstrate in further sections, Anticipation, Reflection, Engagement, and Action have not been separate processes but rather enmeshed in non-linear ways within the ongoing collaborations between the Ethics & Society team members, and the scientists and engineers in the HBP.The Ethics & Society sub-project is now distributed between a dedicated RRI work package and a number of philosophy, neuro-ethics and RRI-related tasks embedded across different HBP work packages.

Collective experimentation
Several authors have highlighted the importance of experimentation with and within RRI, emphasizing its open-ended and inventive nature as well as its productive potential to generate unexpected questions and insights (Delgado and Åm 2018;Stilgoe 2016;Stilgoe, Owen, and Macnaghten 2013).Ongoing experiments with and within RRI are particularly important due to the highly uncertain nature of research and innovation and the challenges involved in foreseeing their future developments and impacts, especially during the early stages of research.This is known as the 'dilemma of control', namely, at early stages of technology development it is difficult to predict its social consequences but by the time undesirable consequences are discovered, technology is already so deeply embedded in society that change is extremely difficult (Collingridge 1980).Sheila Jasanoff suggests the concept of 'technologies of humility' to describe the methods or institutionalized habits of thought that try to come to grips with the unknown, the uncertain, the ambiguous and the uncontrollable (Jasanoff 2003).According to Jasanoff, technologies of humility acknowledge the limits of prediction and control, the possibility of unforeseen consequences as well as the need for plural viewpoints and collective learning.Due to the high uncertainty of emerging technologies, it has been suggested that they need a 'tentative governance' that is more flexible and provides spaces for probing (Kuhlmann, Stegmaier, and Konrad 2019).In this context, collective experimentation (Stilgoe 2016) can be seen as an element of governance that provides space for the contestation and negotiation of emerging technologies early on.Experimentation goes hand in hand with approaching RRI through the lens of collective improvisation (Sauer and Bonelli 2020) and a mutual learning process that is reflective and dialogue-based (Egeland, Forsberg, and Maximova-Mentzoni 2019).
Closely related is the dialogical approach developed by some of the Ethics & Society team members (Stahl et al. 2019a).Building on discourse ethics and RRI, this approach suggests an open, inclusive, and responsive dialogue as a way to identify and address societal issues.It recognizes that in the case of novel technologies, these issues cannot be known a prioriongoing and continuous dialogue and reflection is needed.Thus, instead of a box-ticking mentality, a dialogical approach aims to open up and critically scrutinize the societal and ethical aspects of research.Common features of approaching RRI as a collective experimentation and improvisation, mutual learning and dialogue are openness, exploration, and flexibility which are crucial considering the uncertainty of research and technology.Moreover, an experimental, learning-based, and dialogical approach to RRI challenges assumptions about RRI as a standardized method, tool, or recipe (Egeland, Forsberg, and Maximova-Mentzoni 2019).
To sum up, we suggest that in order to address a broader societal agenda, RRI should not be perceived simply as a linear process where research and engagement are followed by action.These are all continuous, overlapping, and interlinked processes that resist the structure, formulism, and standardization of RRI according to a set of keys, indicators, and measurements.Thus, it is fruitful to conceive RRI and its AREA framework as sources of inspiration for developing and advancing novel approaches to addressing societal issues in research and innovation rather than recipes, checklists, and toolboxes to be followed.Here we agree with Owen and colleagues about the need to resist attempts to reify RRI as a set of disparate keys, including the institutional impulses (well-intentioned though these may be) driving such attempts in the guise of making RRI pragmatic, actionable and measurable.In part, we have argued instead the need to return to and regain some of the substance of the original visions made by RRI's early protagonists.(Owen, von Schomberg, and Macnaghten 2021, 227) We now turn to the development and institutionalization of an RRI-inspired approach to dual-use issues in the HBP where the above-mentioned aspects of experimentation, an integrated AREA framework, and the interlinking of research and practice have played a crucial role.

Developing an RRI-inspired approach to dual-use
While in the process of developing the Ethics & Society proposal for the HBP, members began working on a 'map' of ethical and societal issues raised by its proposed research (Stahl et al. 2019b).The map included a number of topics, such as clinical translation, mental and brain-related disorders, consciousness, community-building, data governance, research integrity, open science, artificial intelligence, and dual-use.Given the impossibility of working effectively on multiple topics simultaneously, a decision was made during the first phase of the HBP to focus on four or five main topics for joint research.Priority was given to those that, after extensive discussions within the HBP and with the Ethics Advisory Board, were selected as being most immediately relevant for HBP research.Dual-use and data protection were identified as highly relevant to HBP research while also addressing the EU Framework Programme's ethics issues (e.g.European Parliament and the Council of the EU 2013).Thus, the first Ethics & Society sub-project's Opinion on Data Protection and Privacy was followed by the second one on Dual-Use.
Dual-use and misuse are among the key societal and ethical issues raised by the neurosciences (Butorac, Lentzos, and Aicardi 2021;Giordano and Evers 2018;Global Neuroethics Summit Delegates et al. 2018;Ienca, Jotterand, and Elger 2018;Mahfoud et al. 2018 brain inspired neuro-and ICT technologies that are already in use or in advanced stages of development, for example, in warfighter 'enhancement', intelligence gathering, image analysis, threat detection, manipulation of emotional states, incapacitation of adversaries, and the development of autonomous or semi-autonomous weapons, or weaponized robots using artificial intelligence technologies and machine learning algorithms for target detection an elimination.(Aicardi et al. 2018a, 5-6) The concept of dual-use has many definitions and connotations which have changed over time and which vary across different fields (Aicardi et al. 2018a).Traditionally, research and technology have been considered to be dual-use when they have current or potential civilian and military applications (see, e.g.Molas-Gallart 1997).This definition of dualuse focusing on the civil-military dichotomy is still used in the EU Framework Programme (European Commission 2019).However, dual-use experts have recognized the limitations of the traditional civil-military dichotomy and highlighted the need to understand dual-use research and technology more broadly as having beneficial as well as harmful purposes (see e.g.Ienca, Jotterand, and Elger 2018;Kavouras and Charitidis 2019;Oltmann 2015;Ulnicane 2020) in a broad range of domains such as political, security, military, and intelligence (Giordano and Evers 2018;Mahfoud et al. 2018).
While in some countries, such as the US, brain research receives funding from defence agencies, 5 all research activities funded by the EU Framework Programmes (FP), including the HBP, must comply with the FP regulation and 'have an exclusive focus on civil applications' (European Parliament and the Council of the EU 2013).In the Ethics & Society team, this focus was seen as too limited to address concerns about emerging brain research, therefore < in 2015 we started to work on an Opinion to develop a broader, RRI-inspired approach to dual-use.RRI, and more specifically the three aspects we outline above (experimentation, integrated AREA framework, and intertwined research and practice), have played a key role in the conceptual advancement of dual-use as well as in the process of the development of an approach to tackling dual-use at the project level.
The HBP Ethics and Society approach to addressing dual-use is encapsulated in the title: 'Opinion on Responsible Dual-Use: Political, Security, Intelligence and Military Research of Concern in Neuroscience and Neurotechnology'.According to the Opinion, dual-use of concern refers to: neuroscience research and technological innovations, and brain inspired developments in information and communication technologies, for use in the political, security, intelligence and military domains, which are either directly of concern because of their potential for use in ways that threaten the peace, health, safety, security and well-being of citizens, or are undertaken without responsible regard to such potential uses.(Aicardi et al. 2018a, 5) The Opinion recognizes that the identification of research of concern is far from straightforward and will remain a matter of debate.It also points to the crucial role that RRI and the AREA framework play in enabling the identification of dual-use of concern, building capacity for debate, and engaging researchers and other stakeholders.In line with much RRI literature (see e.g.Stilgoe, Owen, and Macnaghten 2013), the Opinion understands responsibility to be based in collective and institutionally governed processes and practices, rather than as the responsibility of any individual researcher.While RRI has been mentioned in the context of dual-use before (e.g.Rychnovská 2016), 6 the Opinion places the RRI framework at the centre of the HBP approach to dual-use.
To advance this approach to dual-use, the Opinion ends with a number of recommendations for the HBP, EU policymakers, and other stakeholders.These recommendations include suggestions to establish processes to continuously scrutinize brain research from the perspective of dual-use of concern and to develop educational activities on dual-use issues.To follow-up on these recommendations, the HBP governing bodies decided to establish a project-wide working group (see below section on institutionalizing an RRI-inspired approach to dual-use).
Adapting the RRI framework to dual-use issues in the Opinion enables us to address dual-use issues at the project level via engagement with a broad range of stakeholders within and outside of the project.Moreover, it allows for the anticipation of and reflection on potential uses and implications at the early stages of research.In combining 'dual-use research of concern' and RRI, our work goes beyond existing approaches to dual-use which largely focus on macro-level policies and regulation of technologies.While those approaches sometimes recognize the need for ethics guidelines, awareness raising, and education, they tend to pay little if any attention to broader engagements with researchers and stakeholders at the laboratory and project level to the extent the HBP has.

Between anticipation, reflection, and engagement
The development of the dual-use opinion entailed undertaking a broad range of activitiesfrom an analysis of the concept of dual-use to the organization of workshops and webinars with HBP researchers, dual-use experts, and policymakers; to citizen consultation and educational activities.These research and engagement activities resulted in reports which analysed existing understandings of the notion of dual-use, critically examined relevant policies and regulations, identified dual-use issues in neuroscience and computing, and articulated HBP-relevant issues as well as expert and citizen views (Aicardi et al. 2018a).
The process of preparing the dual-use opinion was informed by the RRI and AREA frameworks.The range of activities undertaken illustrates how anticipation, reflection, engagement, and action can inform research and practice on a specific topic.These activities have been well received by dual-use experts who, for example, saw a webinar organized by the HBP as 'a first promising step in the direction of awareness-enhancing strategies' (Ienca, Jotterand, and Elger 2018, 273).
To illustrate the activities that shaped the approach developed in the Opinion, we provide insights into two dual-use workshops that contributed to the way the Opinion was developed, what case studies it focused on, and which recommendations were made.These workshops demonstrate how anticipation, reflection, engagement, and action were intertwined throughout the process of working on the Opinion and in engagement with the scientists and engineers.While bureaucratically these processes were divided along the AREA framework in order to allocate resources across different partner institutions within HBP Ethics & Society team, in practice, they were much more integrated, and necessarily so in order to develop a project-based understanding of dual-use and RRI issues.
While some scientists and engineers are eager to reflect on ethical and societal aspects of their work, it can be challenging to encourage others to do soparticularly when they are under increasing time pressures and when there aren't any institutionalized mechanisms that support and reward discussions of 'ethics and society' issues (Aicardi, Reinsborough, and Rose 2018).Reflection is an important dimension in RRI, and in the AREA framework specifically.RRI can be achieved in part through institutional reflexivity 'in which the value systems and theories that shape science, innovation and their governance are themselves scrutinised' (Stilgoe, Owen, andMacnaghten 2013, 1571).This 'role'the 'reflexivity inducer'has often been assigned to social scientists working within post-ELSI frameworks (Balmer et al. 2015).Embedding social scientists in the laboratory, and using ethnography as a method, can enable laboratory work to become more sensitive to its potential societal implications (Fisher 2007). 7Working in this spirit, the HBP Ethics & Society sub-project developed its Researcher Awareness programme to engage HBP researchers with the Ethics & Society team's research processnot just as the 'recipients' of our research results.While researcher awareness activities have covered a range of topics and mechanisms (see e.g.Aicardi et al. 2020), here we focus on two dual-use workshops that took place in 2017 and 2018 to illustrate in detail one particularly productive RRI approach that we developed.
Rather than instituting a project-wide researcher awareness programme, we identified laboratories whose research was more likely to raise dual-use of concern issues and worked closely with members of those labs to organize informative and exploratory workshops.The aim of the workshops was to inform the researchers about the research conducted by the Ethics & Society team on dual-use issues of concern, and to encourage reflection on the social, political, and economic implications of the research they were conducting and the technologies they were developing.We made a few methodological choices based on previous and ongoing ethnographic engagement with various labs participating in HBP research: (1) to work closely with PhD students and early career researchers since they are the members of the lab conducting the majority of the research work, (2) develop the content of the workshops in close collaboration with members of the labs so as to encourage ownership of the conversation, (3) limit the number of participants to around 20, and (4) ensure that the content of the discussion that took place during the workshop would be confidential, and would not be shared outside of the meeting through reports or publications.
The workshops followed a similar format.The first session included short presentations by the Ethics & Society team's members that gave more background to the draft Opinion which was pre-circulated to workshop participants.In the second and third sessions, we discussed scenarios which had been prepared in advance in collaboration with the lab members.While scenarios are a common methodological tool used in RRI (Aicardi, Reinsborough, and Rose 2018), it was actually the scientists we collaborated with who suggested we use science-fiction-like scenarios as a basis for our discussion.The scenarios were designed to be hypothetical so that they would not directly reference their own research.But we made sure to develop scenarios that were inspired by their research as well as historical knowledge and current events relating to developments in the life sciences and engineering.This made the scenarios 'strange' enough to feel distant from them to critically engage, but familiar enough to have a provocative reaction.This framing also allowed for the scenarios to be used as an opportunity to reflect on how past and current events can be used to understand and intervene in thinking about and planning for the implications of their research.These scenarios served as frameworks and stimuli for evaluating the possible consequences and the social and ethical implications the technologies and research may have.In this way, these workshops helped to build anticipation and reflection capacities that are necessary for RRI.
The workshop discussion sessions have been generative.They have led to changes in how involved teams report on the potential implications of their work in their Ethics Rapporteur self-assessment of ethical and societal issues, and have pushed the 'ethics and society' discussion beyond mandatory compliance and research ethics, towards a much broader discussion of how their research is both shaped by the political, social and economic contexts they work within, and how their research contributes to shaping these, in turn.Some of the topics discussed at these workshops also informed the work of the Dual-Use Working Groupfor example, the concern many researchers had about the tensions between developing open-access research infrastructure and security concerns about how that research would be used and by whom.The workshops also led to a list of recommendations for individual labs to formalize a yearly discussion with all members of the team to discuss ethical concerns people have, and what they can do about them, and to include a course or session on RRI in Masters and PhD curricula, and in early career researcher training.
Furthermore, we have worked closely with the HBP Education programme to provide training on dual-use and RRI.While laboratory-based workshops have been a successful format, they are very time-consuming to prepare for and conduct, and cannot be used to reach all of the 100+ participating labs in the HBP.The Ethics & Society team members have, therefore, offered lectures and workshops at student-run conferences, and have recorded online lectures that are available to all HBP researchers.We have found that without a deep empirical knowledge of the research in the HBP, it would have been impossible to identify what is relevant for researchers to learn in order to better reflect on the implications of their research.While some RRI work has focused on developing standardized toolkits, a toolkit cannot address the variety of interconnected issues that arise in interdisciplinary projects (Owen and Pansera 2019).In the HBP, we have developed general as well as more targeted approaches, working closely with laboratories on particular problems related to their work instead of directly applying general RRI and dual-use toolkits which would not be as relevant to the particular needs of HBP researchers.

Institutionalizing an RRI-inspired approach to dual-use
A key milestone in the institutionalization of a broader RRI-inspired approach to dualuse of concern in the HBP was the establishment of a project-wide Dual-Use Working Group (DUWG) in December 2018.This group was established by one of the main HBP governing bodies (the Directorate) which adopted the mandate for the DUWG that defines its purpose, composition, expected activities, and resources.The DUWG was established to develop and implement follow-up activities to the Dual-Use Opinion and to ensure that the HBP acts responsibly with regard to the potential dual-use issues of concern raised by and through its research and innovation activities.The DUWG brings together representatives from all parts of the project to scrutinize HBP activities in relation to potential dual-use of concern, to prepare for the EBRAINS infrastructure, and to plan and execute other necessary activities such as raising awareness and education.

Building on and expanding the HBP RRI infrastructure
The DUWG was established approximately half-way through the 10-year project, which has allowed it to draw on already established structures and processes for addressing ethical and societal issues in the HBP.One of these is the already mentioned Ethics Rapporteur Programme which is an important resource in helping us to identify and address ethical and societal issues arising from HBP research.Each sub-project (now work package) appoints one or more Ethics Rapporteur from among their researchers, engineers, or managers.Ethics Rapporteurs communicate regularly with representatives of the Ethics & Society team and with members of the independent Ethics Advisory Board to deepen our understanding of existing and emerging ethical and societal issues in their respective sub-projects.In particular, they are tasked with preparing annual reports ('one-pagers') where they outline issues which are then discussed during annual trilateral meetings that bring together Ethics Rapporteurs, sub-project leaders and managers with representatives from the Ethics & Society team and the Ethics Advisory Board.These meetings illustrate the dialogical approach introduced above (Stahl et al. 2019a).
Already in the Opinion and the DUWG mandate, the Ethics Rapporteurs have been assigned a key role in evaluating dual-use of concern issues in their respective work packages.The majority of DUWG members are the Ethics Rapporteurs.Building on the existing RRI infrastructure in the HBP, the evaluation of dual-use of concern has been incorporated in the annual process of the preparation of Ethics Rapporteurs onepagers and trilateral meeting discussions.
The DUWG has virtual meetings once every two months.These meetings are usually attended by around 15 participants and typically last for an hour.When needed, they are followed up by individual meetings to discuss issues related to specific work packages.In addition, to reach out to other stakeholders within and beyond the HBP, the group and its representatives organize sessions at the annual HBP summit which are open to all HBP members; workshops at the HBP student conferences; early career researcher events; as well as internal and external webinars and talks.
While being able to draw on existing RRI structures and processes in the HBP for addressing dual-use issues is beneficial, it also often requires challenging entrenched understandings and perceptions about ethical and societal aspects of science and technology, which still often focus on legal compliance.In conversations with researchers, it becomes obvious that their primary ethical concerns often focus on getting ethics approvals, which in their experience are becoming more complex and demanding, due to the requirements of the EU General Data Protection Directive.Moreover, in a large-scale EU project like the HBP, scientists and managers often associate ethics with mandatory regular reporting requirements, which again largely focus on legal compliance.In this context, an invitation to think about broader potential societal concerns is not a straightforward task.

Experimentation, learning, and dialogue
Accordingly, one of the first steps to start a conversation about broader potential concerns and misuses of research is a range of awareness-raising activities.This requires experimentation with various approaches to engage researchers to consider potential concerns and misuses of their own research early on, for example, discussing how RRI and the AREA framework's dimensions can help here.Whenever we run workshops at student and early career-researcher focused conferences in the field of brain science within and beyond the HBP, we often encounter participants who are highly interested in learning more about the societal and ethical aspects of their research.But they also tell us that they have had little if any training or education on these issues, and some of them also admit that they have never considered the potential misuse of their research.These training and awareness-raising activities aim to change the culture of research by sensitizing researchers to societal concerns and the potential misuse of their research.They are also intended to support and prepare researchers for one of the key tasks of the DUWG, namely, the identification of dual-use of concern and misuse issues in their work.
As mentioned earlier, an approach the DUWG has chosen to pursue the identification of dual-use of concern and misuse issues is to include these issues in the short self-assessment (one-pagers) of the ethical issues prepared by Ethics Rapporteurs for each work package.These one-pagers are internal documents intended for discussions and follow-up actions within the project.In the preparation of the dual-use of concern and misuse part of the one-pagers, the DUWG has collaboratively prepared a living document with key definitions and questions to support anticipation and reflection on these issues.Ethics Rapporteurs are invited to share their potential concerns at the DUWG meetings, allowing them to exchange their reflections, comment on each other's approaches, and learn from each other.It is intended that the DUWG serves as a safe space where any potential concerns can be shared for initial discussion and potential follow-up actions.
To identify dual-use issues in their work packages, Ethics Rapporteurs are encouraged to experiment with different methods of anticipation and reflection, and to afterwards consider the advantages, disadvantages, and results of these different methods.It is suggested that when preparing their one-pagers, Ethics Rapporteurs follow a deliberative approach of discussing any issues with their colleagues in their respective work packages.However, each rapporteur has their own approach for doing that.Some of them undertake a survey in their work package to identify ethical issues, others consult with colleagues at meetings or over email, while some might have to do most of the reflection themselves.This is followed up by discussion, for example, on whether surveys are a useful tool for facilitating reflection on dual-use of concern issues, which are sometimes misunderstood as the civil-military dichotomy.This can lead to predictable survey answers that there are no dual-use issues in the respective research.
As pointed out in the Opinion, the identification of potential dual-use of concern and misuse issues is far from straightforward.Moreover, going beyond legal compliance and immediate risk assessment often means that potential concerns and misuses are vaguer and less concrete, requiring expanded vocabulary and practices of anticipatory governance to address these more uncertain issues (Rychnovská 2021).
A lot of research in the HBP is in its early stages when it is still difficult to predict potential uses and related concerns.This can lead to some standard answers that any possibility of dual-use of concern is very remote and highly unlikely.This is in line with the findings of Isobel Butorac and colleagues that researchers find it difficult to predict long-term effects and risks, and to be what they perceive as 'fortune tellers' (Butorac, Lentzos, and Aicardi 2021).On the other hand, researchers also point out that anything can be misused, and it is difficult to avoid that.Hesitation to talk about potential dual-use of concern and misuse might also stem from concerns about how it could influence the future of their specific research.
In this context, it is crucial to explain and follow our approach that in emerging sciences and technologies such as brain research, ethical issues, concerns, and misuses often cannot be identified upfront and require continuous reflection and dialogue as research develops (Stahl et al. 2019a).Our dialogical approach also involves resisting requests for checklists and reminding researchers and managers that the identification of potential concerns and misuses is a dialogue and not a checklist.According to this dialogical approach, the DUWG is intended to serve as a network that supports and facilitates a process of mutual learning and experimentation.At the core of this network is the DUWG, but it is an open and collaborative formation that regularly interacts with others within and beyond the HBP including other HBP groups such as the Data Governance Working Group and HBP student initiatives as well as policymakers, ethics and dual-use experts, and social scientists interested in the governance of technologies.Opportunities to draw on this DUWG network for discussing and sharing any concerns and ways of addressing them are appreciated by HBP colleagues and students.Richard Owen and colleagues have suggested that community building and socializing might be some of RRI's most enduring legacies (Owen, von Schomberg, and Macnaghten 2021).The effects of community building and socializing are also strongly present in the DUWG work of bringing together researchers for experimentation and mutual learning.
To summarize, the institutionalization of an RRI-inspired approach to dual-use of concern in the second half of the HBP has benefited from RRI processes and structures established in the first half.However, this also came with some challenges.In addition to the need to challenge entrenched, narrow understandings of ethics, it was also necessary to consider that a systematic introduction of dual-use issues should not 'overload' existing structures and processes.Ethics Rapporteurs, who typically are researchers or managers and can only devote a limited share of their time to their Ethics Rapporteur activities, already have a lot of responsibilities.Similarly, their one-pagers and trilateral meetings already have a range of ethical issues on the agenda.Therefore, it was decided to take a 'light' approach, for example, to meet only once every two months rather than more often.This can also limit how deeply it is possible to engage with experimentation and reflection.However, institutionalizing dual-use of concern within an existing RRI infrastructure has allowed us to develop it further by expanding the range of issues discussed and the activities undertaken as well as experimenting with more ambitious approaches to address broader societal issues that encourage reflection on societal concerns and potential misuses that might emerge in a longer-term and various ways of addressing them.

Conclusions
In this article, we have explored our experience of developing and institutionalizing an RRI-inspired approach to dual-use that goes beyond legal compliance and that considers a broader range of societal concerns.We have argued that it is essential to maintain RRI as a flexible, fluid, and open concept and challenge attempts to standardize and instrumentalise it.We highlight the importance of three closely connected aspects of RRI: (1) a productive intertwining of research and practice; (2) the integration of anticipation, reflection, engagement, and action in a non-linear process; and (3) an experimental approach.Our experience suggests that the AREA framework works better when its four dimensions are used in a flexible and intertwined manner that enables experimentation, learning, and dialogue rather than when trying to artificially separate them into four distinct stages.
Developing and institutionalizing an RRI-inspired approach to dual-use has required extensive interdisciplinary research on topics such as the governance of technologies, neuro-ethics, and scientific collaboration drawing on Science and Technology Studies, philosophy, and other disciplines.This research has played an important role in anticipation, reflection, and education activities and has supported experimentation and mutual learning.Research and practice in our approach to RRI continue to mutually shape each other: ideas and insights from research influence practices, while practices shape future research questions and focus.Rather than reducing RRI to standardized practices, the combination of research and practice allows us to enrich and deepen the potential of RRI to tackle a broader set of societal issues.However, the success of RRI in addressing societal issues by increasing understanding and changing culture and practices remains difficult to measure (Stahl et al. 2021).
In our experience, RRI works best when it is neither conceived nor followed as a set of principles.While the HBP Ethics and Society sub-project has adopted the AREA framework, in practice our work does not so neatly fit into the separate 'boxes' of anticipation, reflection, engagement, and action.In fact, we suggest that the AREA framework should not be interpreted in a linear fashion but should rather demonstrate the messiness of research and innovation itself.Because of the fluidity and flexibility of the RRI framework, the HBP Ethics & Society sub-project has been able to interpret RRI in ways that work for the specific context of the HBP.Because RRI does not give priority to some principles or values over others (Salles and Farisco 2020), we have been able to work closely with researchers to identify particular areas of concern and reflect on possible solutions for addressing them.We have focused on particular concerns that arise across the intersection between neuroscience, computing, and medicine without the ease of relying on already existing dual-use or RRI toolkits.While this has made our reflection and practice more demanding, it has also been generative and productive.
Researchers who have studied RRI in other contexts have observed a reluctance to address dual-use issues because they are sometimes perceived as controversial (Frahm, Doezema, and Pfotenhauer 2021).The development of an RRI-inspired approach to dual-use in the HBP encountered similar reactions and questions about the relevance of the topic, which have also provided opportunities for discussions and awareness raising about dual-use of concern issues.This work on developing and institutionalizing an RRI-inspired approach to dual-use has attracted interest within and beyond the project.It has been highlighted as one of the scientific achievements of the HBP (Amunts 2020) and recognized by experts as an important step forward (Garden et al. 2019;Ienca, Jotterand, and Elger 2018;NIH 2019).However, it remains a work in progress that requires a continuous challenge to standardized understandings of RRI and dual-use in order to open space for a more reflective approach to potential concerns that arise from emerging research and technology.