Co-production for innovation: the urban living lab experience*

Abstract Urban Living Labs (ULLs) are public spaces where local authorities engage citizens to develop innovative urban services. Their strength and popularity stem from a methodology based on open innovation, experimentation, and citizen engagement. Although the ULL methodology is supposed to largely adopt a co-production approach, connections between the two have not yet been thoroughly investigated. The paper seeks to fill this gap by examining through a qualitative analysis three experiences of ULLs made in Amsterdam, Boston and Turin. Specifically, the paper aims to assess whether ULLs can be really conceptualised as a form of co-production and, if so, which elements characterised them as innovative in comparison to ‘mainstreaming’ co-production; Then it analyses benefits and drawbacks related to their implementation.


Introduction
The concept of co-production can be broadly defined as the involvement of individual citizens and/or groups of citizens in public service delivery (Verschuere, Brandsen, & Pestoff, 2012, p. 1086). Especially at the local level, where interactions between the local community and public authorities can be very close, co-production has been viewed as a practical solution to improve the quality and efficiency of services. Involving citizens in producing their own solutions, in fact, is expected to allow for producing outputs that are better tailored to their needs at considerable cost-savings. Additionally, citizen participation in service delivery would promote democracy and increase social capital (Ostrom, 1996).
The recent economic crisis has contributed to enlivening the debate on co-production by forcing reconsiderations of new forms of collaborations between users and private and public bodies. Organisations such as MindLab in Denmark and Nesta in the UK, 1 for instance, emphasised the need to implement co-production processes as the only viable solution to the growing complexity and wicked nature of issues tackled by public authorities (Bason, Mygind, & Sabroe, 2013;Christiansen & Bunt, 2012). Within this debate, Urban Living Labs (ULLs) represent a good example of methodology based on co-production and aimed at coping with policy challenges occurring at the local level (Bason, 2013;Bason et al., 2013;Boyle & Harris, 2009;Christiansen & Bunt, 2012;Coenen, van der Graaf, & Walravens, 2014;Eskelinen, Robles García, Lindy, Marsh, & Muente-Kunigami, 2015;Kulkki, 2014). ULLs are public spaces where local authorities engage citizens to develop innovative urban services (Eskelinen et al., 2015, p. 27). Their strength and popularity stem from an approach based on open innovation, experimentation and citizen participation in service design and production.
Although the ULL methodology is supposed to largely adopt a co-production approach, connections between the two have not yet been thoroughly investigated. This paper seeks to fill this gap by examining, through a qualitative analysis, three experiences of ULLs developed in Amsterdam, Boston and Turin.
Specifically, the paper aims to assess whether ULLs can be really conceptualised as a form of co-production and, if it so, which elements characterised them as innovative in comparison to 'mainstreaming' co-production; Then it analyses benefits and drawbacks related to their implementation.
The paper proceeds as follows. Section 2 proposes a definition of co-production based on recent developments in the literature and adopts it as conceptual framework to analyse ULLs experiences. Section 3 explains how ULLs operate and what characterised them. Section 4 describes the research methodology and illustrates three case studies of ULLs. Using the framework proposed in Section 2, Section 5 analyses how co-production empirically works in ULLs and identifies peculiarities and critical points. Conclusions summarise main findings and suggest directions for future research.

The concept of co-production: an operational definition
From the seminal work of Ostrom in the 1970s, co-production has been attracting scholarly attention for decades. Co-production has been extensively analysed in the literature on public administration due to its implications for administrative paradigms, organisation and duties. Co-production, in fact, challenges the Weberian idea of public administration, where service planning and delivery are monopolised by public officials and users are passive consumers (Pestoff, 2006). However, co-production also moves beyond the New Public Management approach, which neglects the role of citizens and places strong emphasis on the 'customisation' of services and performance assessment (Joshi & Moore, 2004). In both cases, the central and exclusive role played by bureaucrats and professionals is questioned, and user participation emerges as an opportunity to innovate and to improve policy outputs.
The latest developments in the theory of public administration tackle the question of co-production by arguing for a 'service-oriented' turn Denhardt, 2000, 2007). The New Public Service approach, in fact, emphasises the duty of public managers 'to serve and empower citizens as they manage public organisations and implement public policy' , to build a shared notion of public interest, and to enhance the collaboration among citizens and private and public actors (Denhardt & Denhardt, 2000, p. 549). Drawing on the same premises, the New Public Governance (NPG) approach places particular emphasis on co-production as an inherent characteristic of services. For NPG users, their experiences and knowledge are central in public service design and delivery. On the other side, policy-makers, particularly public managers, have the important duty of understanding user needs and bringing them into the policy process (Osborne, Radnor, & Nasi, 2012, p. 146).
In public policy terms, eventually, public managers can fruitfully adopt co-production as a policy tool to promote users' participation in service delivery. As suggested by Howlett, Kekez Koštro, and Poocharoen (2015), co-production can be conceived as a procedural policy tool, i.e. an instrument helping to accomplish a policy goal by altering actor behaviours (Howlett, 2011, pp. 22-24).
The growing body of academic literature examining co-production has led, nevertheless, to a conceptual stretching that generates confusion about what co-production really means (Brandsen & Honingh, 2016;Nabatchi, Sancino, & Sicilia, 2017). The idea of co-production, in fact, has been expanded to cover different types of 'co-activities' , such as co-design and co-evaluation, to be applied to various services, and to involve a wide range of participants.
To cope with the diverse uses and applications of the concept Nabatchi et al. recently proposed to define co-production 'as an umbrella concept that captures a wide variety of activities that can occur in any phase of the public service cycle and in which state actors and lay actors work together to produce benefits ' (2017, p. 4). 2 In their typology the authors distinguish two types of participants (Nabatchi et al., 2017, p. 4): (1) state actors, i.e. governmental or non-governmental actors who regularly produce the service; and (2) lay actors, who are members of the public and who are involved in co-producing the service as citizens (the members of a community), clients (the recipients of a service not paying for it), or customers (the recipient of a service who pay for it). Co-production can produce personal or social benefits (Nabatchi et al., 2017, p. 6). Namely, for Alford (2002) people can co-produce for material rewards, like monetary incentives or a particular benefit in return for the time and effort spent. But more often, people participate in co-production for non-material rewards, such as intrinsic rewards, like increasing self-esteem, solidary incentives, like the sense of group or taking enjoyment from participating or gaining social approval, and normative values, like the satisfaction to have contributed to a cause, to have promoted participation and democracy (Verschuere et al., 2012;Pestoff, 2012).
Co-production between state actors and lay actors can take place individually, in groups or collectively (Brudney & England, 1983), depending on whether the service is targeted at the individual level, at a specific group of population or to 'diverse members of a community' (Nabatchi et al., 2017, p. 6). Finally, lay actors can contribute to different phases of the service cycle, such as co-commissioning, when actors co-identify the priorities of a service, co-design, when actors co-create the service, co-delivery, when actor together implement the service, and co-assessment, when actors jointly evaluate the service.
Main effects of co-production are the improvement of the effectiveness and quality of service delivery, and the promotion of democracy and accountability (Verschuere et al., 2012). Co-production, in other terms, represents a useful policy tool because it should help to clarify user needs and related solutions. It also promotes user empowerment, social capital, and the provision of services in a more democratic and transparent way (Bovaird & Downe, 2008;Ostrom, 1996;Verschuere et al., 2012).
Finally, effective co-production also depends on the capacity of state actors to understand clients' needs, to communicate the value they are trying to achieve, to clarify goals, to define all the stages of the co-production process, and to adopt an organisational structure with sufficient autonomy and coordination capacity (Verschuere et al., 2012).
For the purposes of the present article, I will rely on the framework proposed by Nabatchi et al. to examine three experiences of ULLs in order to identify actors involved in the co-productive activity, the phases of the service cycle across which co-production is applied, the level at which it occurs, and main benefits generated by collaboration both for professionals and citizens. The definition of co-production provided by the authors, in fact, is broad enough to be applied to ULLs in order to analyse how collaboration between state actors and lay actors take place, whether ULLs represent an innovative form of co-production and which are their main strengths and weaknesses.

The ULL approach
The concept of the 'Living Lab' was first used in the early 1990s in a paper by Bajgier et al. to describe students' experimentation with a problem-solving approach in an inner-city neighbourhood in Philadelphia, called a 'living laboratory ' (1991, p. 701). The concept was further developed by William J. Mitchell from the MIT Media Lab and School of Architecture in 1995 to define an innovative research approach aimed at developing and testing new technologies and strategies to cope with complex social problems (Mitchell, 2003). 3 After that, Living Labs (LLs) experienced a certain amount of popularity among academics and practitioners as a new model of business innovation.
In the European Union ( ULLs represent a sort of evolution from the traditional LLs approach with which they share the basic characteristics. ULLs, in fact, 'have become a trend in cities all over the world. The term is used to refer to a wide variety of local experimental projects of a participatory nature. The aim is to develop, try out and test innovative urban solutions in a real-life context' (Steen & Van Bueren, 2017, p. 5). Empirical research highlighted some specific features shared by ULLs (Almirall, Lee, & Wareham, 2012;Baccarne, Schuurman, Mechant, & De Marez, 2014;Bakici, Almirall, Mezquita, & Wareham, 2013;Carstensen & Bason, 2012;Coenen et al., 2014;Mulder, 2012;Nesti, 2016). Namely, ULLs are characterised by three peculiarities. First, they are based on an organisational approach inspired by the 'quadruple helix' , i.e. collaboration among public authorities, firms, research organisations and people (Battaglia & Tremblay, 2011). ULLs are also labelled as PPPP, or public-private-people partnership (Molinari, 2011). Local authorities, in fact, are usually the promoters and creators of ULLs. Firms participate providing technologies, products and services that should be tested in the urban Labs. But citizens are the key players in laboratories, where they are engaged, usually on a voluntary basis, in the innovation process (see below). ULLs coordination can be entrusted directly to public administration but also to universities or to 3 See also http://livinglabs.mit.edu/ (accessed on 30 July 2017). 4 enoll is a Brussels-based non-profit organisation that now counts almost 300 lls among its members. 5 See nesti (2016) for a review.
non-profit organisations that manage them on behalf of local government also providing support and expertise.
The second feature is the adoption of a methodology based on experimentation of solutions for societal problems that are designed, prototyped, validated and refined with participants in a real life context (Pierson & Lievens, 2005;Westerlund & Leminen, 2011). More specifically, the ULL methodology usually includes co-design and co-creation by users together with professionals and producers of a new product or a new service; the exploration of potential new usages and emerging behaviours; the experimentation with a prototype in real settings with communities of users; and finally, the evaluation of the impact generated by the innovation (Pallot, Trousse, Senach, & Scapin, 2010;Paskaleva, Cooper, Linde, Peterson, & Götz, 2015;Westerlund & Leminen, 2011).
Third, at the core of ULLs lies the concept of open innovation. The basic idea is that knowledge is diffused within society and that new solutions to problems can come and should be collected from inside to outside an organisation (Chesbrough, 2003). Within ULLs, in particular, participants are stimulated to generate and to discuss ideas through the adoption of various techniques, such as brainstorming, focus groups, scenario building and other ethnographic tools (Nesti, 2016). Then participants' ideas are included in the original project and tested. Thus, open innovation is fostered by a continuous process of exchange of knowledge between actors and of learning-by-doing. Remarkably, unlike public innovation approaches that are mainly targeted at internal processes and rarely affect the society as a whole, ULLs represent an attempt to foster innovation from inside governments towards the community (Carstensen & Bason, 2012).
One last point that characterises several ULLs is the utilisation of ICTs. Collaboration with citizens often entails the testing of sensors or smart-grids or the development of apps. But ULLs frequently use technologies as part of their methodology (Nesti, 2016) like, for instance, open source software and the Web 2.0, 6 that allow users to generate their own content and to share it immediately with other users. ICTs, in fact, open up new opportunities for collaboration because they reduce the costs of connection and made interactions possible at any time (Meijer, 2012).
Notwithstanding ULLs share the basic characteristics illustrated above, literature often portrays them as a multifaceted phenomenon that covers a wide range of local experimentations (Nesti, 2016;Steen & Van Bueren, 2017). Thus, to specifically identify the characteristics of ULLs, three empirical cases of ULLs will be presented in the following paragraphs.

Case selection and research methods
To analyse how ULLs operate in practice, if they can be conceived as a form more or less innovative of co-production and their main benefits and drawbacks, three experiences have been selected as typical cases of ULLs and were analysed using a replication approach to multiple case studies (Yin, 2009, p. 56). The first case study is the Housing Innovation Lab 6 Web 2.0 is the network as platform, spanning all connected devices; Web 2.0 applications are those that make the most of the intrinsic advantages of that platform: delivering software as a continually updated service that gets better the more people use it, consuming and remixing data from multiple sources, including individual users, while providing their own data and services in a form that allows remixing by others, creating network effects through an 'architecture of participation' , and going beyond the page metaphor of Web 1.0 to deliver rich user experiences (o'Reilly, 2007, p. 17).
developed by the Department Urban Mechanics of the Mayor of Boston. The second is the Amsterdam Smart Citizen project created by the Foundation Waag Society in collaboration with the Amsterdam Economic Board 7 and the Amsterdam Smart City Initiative. 8 The third is the Turin Living Lab promoted by the Municipality of Turin. The empirical research is based on the qualitative analysis of data available on ULLs' websites. 9 Information related to Boston and Turin cases has been integrated with interviews with key informants. Three interviews were carried in Boston on June 2017: one with the Director of the Housing Innovation Lab and two members of the team, and two with the co-Chairs of the Mayor's Office of New Urban Mechanics. Two interviews were carried with the Chief Officer for Innovation and Economic Development of the Municipality of Turin and with the Head of the District Campidoglio on July 2017.
Data related to the Amsterdam case study have been gathered from three publications that illustrate and assess the Smart Citizen experience (Bozzon, Houtkamp, Kresin, de Sena, & de Weerdt, 2016;Henriquez, 2015;Van den Horn & Boonstra, 2014).

Housing innovation lab in Boston
The Laboratory was created in 2015 by the Mayor of the City of Boston as part of the strategy 'Housing A Changing City: Boston 2030' 10 in order to explore new affordable housing solutions for citizens. The Lab was initially managed by the Department of Neighbourhood Development and the Mayor's Office of New Urban Mechanics and was financed with a start up funding of $1.3 million over three years from the Bloomberg Philanthropies Innovation Team grant. Now the Lab has become a permanent office in the City's Department of Neighbourhood Development.
Between 2015 and 2017, the team collaborated with housing experts, community organisations, and Boston residents to gather and generate solutions to the problem of high cost housing in Boston. The Lab engaged in pilot projects 11 related to density, compact living, alternative housing models and home buying.
The approach followed by the Lab is based on exploring, experimenting and evaluating innovative housing models. In the first stage professionals collect information on a housing problem from residents, dialoguing with them and listening to their stories. This process can take place directly in the neighbourhood or during roadshows. Then professionals analyse residents' needs, explore new solutions and define the project. This draft is submitted to residents for further comments and it is redefined including their suggestions. In other cases professionals propose a solution and test it with residents in a real context in order to collect feedbacks on it and to elaborate recommendations for policy-makers. For instance, in the project 'Urban Housing Unit Roadshow' a compact apartment on wheels was located for two weeks in different areas around the city. Residents were asked to get into the unit, to check out and to talk with the team of the Housing Lab. Two questions were posed to participants in the roadshow: Who do they think would want to live in a space like that and what kinds of benefits, services, and infrastructure would need to exist around a unit of this size to make it liveable. The experimentation was successful since the Housing Lab staff collected hundreds of comment cards with helpful feedbacks. Information helped professionals to define what's needed for compact living to work in Boston and then recommendations were issued to the Municipality.
The first strength of the methodology adopted by the Lab is, according to our interviewed key-informants, the continuous process of connection with people, at the beginning to understand their needs and after to validate the project. The process of co-design with residents allows professionals to learn more about problems and to define and redefine better housing solutions. The second strength is the adoption of a collaborative approach with all the relevant stakeholders of the territory. In the exploratory stage, in fact, professionals work together with community groups, academic, non-profit and profit organisations. They also have regular meetings with the Planning Department and the Housing Department of the Municipality, and the Boston development agency, to get their input, advice and expertise.
Weaknesses, on the other side, are few. According to the Director of the Lab, one problem relates to how to collaborate with citizens in a more effective way, due to the huge amount of time required to co-design solutions in respect of the limited duration of projects. Professionals make strong efforts with citizens, in fact, to frame the problem, to elicit people expectations and to translate them into viable solutions. But solutions are usually 'prototypes' that may not necessarily lead to long-term policies, due to the experimental nature of the Lab.
A second problem pertained to the sustainability of the Lab or, in Director's words, 'its capacity' . The Housing Lab, in fact, was initially created as an experimental initiative itself and it has operated for two years as an innovation team within the Mayor's Office of New Urban Mechanics. But on 19 July 2017 the Mayor announced the Lab would become a permanent office in the City's Department of Neighbourhood Development. This process of institutionalisation is expected to secure more resources, the possibility to mainstream innovations within the Boston housing policy and to coordinate them with other initiatives.

Amsterdam smart citizen project
The project is an example of urban laboratory aimed at co-producing sensors with the 'Do-it-yourself ' (DIY) approach. 12 The use of toolkits for users innovation is diffused in the ICT sector as a way to solve the problem of how to match users needs with product solutions at low costs (the so called problem of 'sticky information transfer cost'). Through the DIY approach 'manufacturers actually abandon the attempt to understand user needs in detail in favor of transferring need-related aspects of product and service development to users' (Von Hippel & Katz, 2002, p. 1).
Smart Citizen is an initiative originally created by Fab Lab Barcelona 13 in 2012. It is an open online participatory platform that produces open data and indicators to monitor the urban environment. It consists of the 'Smart Citizen Kit' with sensors for measuring toxic gases like CO and NO 2 , air temperature, humidity, light intensity and noise pollution, an Arduino computer board for processing the data, a Wi-Fi module for sending the data to web portal, a mobile app and API for on-the-go access. 14 Smart Citizen uses ICTs and crowdsourcing to produce data for public administration in order to improve the monitoring of pollution.
In September 2013 the Waag Society invited a member of Fab Lab Barcelona to present the project Smart Citizen to Amsterdam administration. The basic idea was to improve the existing network of quality air measurement stations though the direct involvement of citizens and in an affordable way. But promoters would also experiment a new approach to the smart city based on civic engagement and learning. The administration agreed to participate in the project covering the expenses for the toolkits. 15 The Waag Society in collaboration with the Amsterdam Economic Board and the Amsterdam Smart City Platform launched the initiative through the journal Het Parool and started recruiting volunteers. On February 2014 the Waag Society and its partners selected 73 peoples, tested the upgrade version of the Smart Citizen Kit and opened a Help Desk. In March 2014 the project was launched: participants received the kits and some workshops were organised for the community aimed at creating awareness about pollution and at illustrating how to use the sensors kit. But these meetings were also seen as an opportunity to discuss and to co-design solutions for urban issues and to collaborate with researchers and public officials to co-create new environmental policies (Henriquez, 2015).
Between April and June 2014 participants gathered data, uploaded them on the dedicated website and shared them with the Public Heath Department of the Municipality of Amsterdam. All along the process citizens were supported and trained by expert from RIVM to TNO.
At the end of the experimentation, the project was evaluated by citizens, the Public Heath Department of the Municipality of Amsterdam and TNO (the Netherlands Organisation for Applied Scientific Research). Evaluation results were different. Participants mainly reported technical problems related sensor hardware and software, while they expressed a general satisfaction for the project. They claimed they have learnt a lot about climate issues and that they would be involved again in similar initiatives. A positive judgement was also expressed by the representative of TNO who noticed the high number of citizens engaged in the project while the representative of the Public Health Department complained about the fact that kits produced unreliable data (Henriquez, 2015, p. 25). In their final assessment organisers agreed on the need for more a sophisticated hardware and software, nevertheless the Waag Society decided to continue the project in cooperation with the Chief Technology  Citizen Kit experiences organisers launched the Amsterdam Citizen Lab 16 that is now part of the EU-funded project Make Sense. 17

Turin living lab
It represents the typical ULLs where companies use the city as a living environment to test their products before commercialisation. 18 This process entails small scale testing -usually in a limited city area -and then the implementation of the device across the entire city. More precisely, in the latter case, the municipality enables companies that are developing innovative solutions in various fields (such as energy, mobility, lightening, or urban planning) to test them in a specific district through pilot trials. Here, citizens are selected by a team of professionals or can be engaged on a voluntary basis to be involved in the testing phase.
The initiative 'Turin Living Lab' 19 was ideated in 2013 by the former Executive Councillor for the Environment and by the Chief Officer for Innovation and Economic Development of the Municipality of Turin with the aim of supporting innovation in local enterprises and associations. The main rationale was to give these entities the opportunity 'to promote, develop, test, and experiment technological initiatives and solutions' 20 in a specific District of the city of Turin, Campidoglio -a predominantly residential area, in a semi-central position, in the north-western quadrant of the City (District 4) -without costs or commitments for the Municipality and involving the local community. The Lab falls into the more general Smart City Strategy of the Municipality of Turin aimed at developing a model of urban development based on the promotion of environmental and social sustainability.
The Lab was officially launched in January 2016 with the opening of a call and firms were invited to participate in it by submitting their ideas there should be consistent with the smart strategy and with the general goals of the Municipality. A technical commission made of representatives from the Departments of the Municipality, the District and local multi-utilities selected 32 projects that have been implemented between 2016 and 2017. In some projects citizens simply use products made available in the neighbourhood, like e-bikes, bike locking systems, or apps for tourism. Other projects entail the test of products by the Municipality, like street vacuum cleaners, or control units to monitor polluters. Other projects, finally, involve citizens directly in co-producing services using a crowdsourcing approach, for instance to improve apps, to collect environmental data through low cost portable sensors that are uploaded to a central server to be consulted by the Municipality, to maintain historical public drinking fountains, to create a carpooling service, to implement a neighbourhood watching service via app. 21 The governance of the Turin Living Lab is entitled to the Department for Innovation and Economic Development of the Municipality of Turin that manage it in collaboration with the other Departments involved in the experimentation and with two public officers working in the District Campidoglio. According to interviewed key-informants, the implementation of the selected projects run quite smoothly and results were positive. The main benefit for the Municipality originated from the Living Lab was the experimentation of an innovative type of PPP that makes possible to support innovation with firms avoiding red tape associated with traditional public procurement models. Citizens actively participated in the projects and were satisfied. They represent the 5% of the population of the District, i.e. 1000 out of 20,000. But this result was indeed positively evaluated.
The principal problem emerged in the experimentation concerned the fact that few people complain when initiatives end even if the Municipality presented to the local community goals, projects, and timing of the Lab during several meeting, a dedicated website, and leaflet.
Public officers working in the District sometimes experienced difficulties in coordinating with the different Departments of the Municipality. The experimentation was also costly in terms of time spent in supporting the implementation of the projects. This caused some complaints by public officials due to their increased workload.

Co-producing in ULLs: variations on a theme
The three experiences described above point out that the approach to co-production adopted in ULLs share some of the characteristics illustrated in Section 2 but also that ULLs 'decline' co-production in their own way.
State actors involved in ULLs are mainly governmental. The Boston Lab is part of the Municipality administration and civil servants run it. The Amsterdam Lab is physically placed outside the Municipality and it is managed by a non-profit organisation in partnership with the Municipality. In Turin profit and non-profit organisations were responsible for the implementation of their projects but with prior approval of the Municipality that is also responsible for the whole coordination and management of the Lab. Lay actors of the three Labs were members of the local community -neighbourhood residents in Boston and Turin and citizens in Amsterdam -who were engaged on a voluntary basis. An interesting aspect, nevertheless, differentiates one of the three experiences. While in Boston and Turin participants were ordinary people without a specific demographic or social connotation, in the Amsterdam Lab the technical nature of the experimentation produced a self-selection of 'geeks' who already participated in the Internet of Things Lab. 22 Collaboration in the three Labs occurs in different phases of the service cycle: in the co-design stage in Boston, in co-designing, co-delivery ad co-assessment in Amsterdam, and in co-designing and co-delivery in Turin (see Table 1).
Remarkably, co-design took place through a face-to-face dialogue with professionals, such as in the case of the Housing Lab in Boston or in the Amsterdam Lab where citizens exchanged their views directly with professionals. But co-design can also occur without the direct support of professionals, like in the case of the development of open source software or apps. Moreover, due to the experimental nature of ULLs, co-delivery usually coincides with the process of prototyping and testing the product.
Collaboration between professionals and citizens in the assessment stage of the service cycle was only present in Amsterdam. Co-assessment proved to be very useful for 22 See http://iotlivinglab.com for more details.
professionals in order to reframe the Smart Citizen experience and to transform it into a EU-wide project.
Co-productive activities took place largely at a collective level. Nevertheless, elements of individual and group co-production are also present in the DIY sensing approach followed by the Amsterdam lab and in the work of the Boston Lab with specific targets of residents.
Concerning the main outcomes deriving from co-production in the three ULLs, professionals reported that collaboration with citizens allow them to define innovative and effective solutions to local problems (in Boston), to test products and services useful to improve the quality of life of citizens (in Turin), to promote environmental activism and to help citizens to develop an environmental consciousness (in Amsterdam). Co-production in ULLs, therefore, is supposed to create mainly social benefits that are enjoyable by the whole community.
A specific social value produced by innovation associated to ULLs relates to the production of knowledge. More precisely, the Amsterdam Smart Citizen Lab is inspired by the idea of 'illuminated city' elaborated by Van Timmeren, Pimentel, and Reynolds (2015) and by the principles of the 'citizen science' . In contrast with the business-led smart city, the illuminated city is a place where technologies are at the service of citizens and of local governments to solve collective problems and to promote democracy. The concept of citizen science refers, on the other side, to 'the active involvement of non-professional scientists in research' (LERU, 2016, p. 5). 23 Thus, professionals of the Amsterdam Lab were specifically interested in promoting -through co-production -the scientific empowerment of everyday people in order to enable them to participate in and to give their contribution to local environmental policies in a more informed way.
Beside social benefits, nevertheless, also personal non-material benefits emerged as relevant for citizens engaged in ULLs. Lay actors, in fact, participate in co-productive activities mainly to take enjoyment from participating and to improve the quality of local services and of their lives. In the case of Amsterdam participants were driven also by other material and intrinsic rewards, such as the possibility to gain immediately data about environmental pollution surrounding them, the interest in learning or in improving their technological skills, the curiosity for the methodology adopted but also the interest in getting in touch with other people with similar interests.
To sum up, co-production implemented in ULLs only partially fits with the model presented above. The principal point of departure from 'mainstream' of co-production relates the phases where the 'co' activity takes place. ULLs are prevalently focussed on co-design, i.e. on the inclusion of citizens' experiences, suggestions, and opinions in the creation or in the 23 See http://citizenscience.org for more details.

Boston ULL
Amsterdam ULL Turin ULL co-commissioning co-designing X X X co-delivery (X) X (X) co-assessment X individual co-production X X Group co-production X X collective co-production X X X re-definition of the attributes of a service or product. Co-delivery -a basic characteristic of mainstream co-production -in ULLs takes the form of testing and does not usually entail a join activity between state and lay actors. The empirical analysis revealed also that co-production in ULLs is marked by other specific features. First, ULLs produce not only services but also 'physical' products used to achieve specific policy goals. In the case of Amsterdam, for instance, citizens are called to materially create a sensor while in Turin they can develop and test ICTs services in logic of crowdsourcing. Remarkably, the application of new technologies and, mainly, of the Internet to public policy opens up new possibilities to alter actor behaviour, particularly in the environmental policy domain, where 'to have the greatest chance to slow and perhaps even reverse the slide toward calamitous climate change, we need to mobilise the widest possible public support for effective actions' (Patchen, 2006, p. 1). Involving citizens in co-designing and co-producing products and services in ULLs can effectively contribute to enhancing behavioural changes towards more sustainable habits.
The second peculiar aspect of the three ULLs is that they pursue public innovation. Through ULLs, in fact, Municipalities try to find new solutions to complex urban challenges but also to boost local economy (Bekkers, Edelenbos, & Steijn, 2011). Experimentation in ULLs helps local government to overcome two important barriers to innovation: excessive bureaucracy and risk-aversion (Sørensen & Torfing, 2011). On the one side, in fact, the experimental approach adopted by Labs allow professionals to pilot and testing projects on a smaller timeline so they can cut red tape and contribute 'to move things forward' . On the other side, professionals working in a laboratory and not in a Department can take the risk for possible failures and process it as an ordinary externality of innovation without loosing reputation.
Co-production in ULLs is crucial for public innovation because it contributes to reduce time and costs. Working with groups of citizens helps better focus on their needs, facilitates the extrapolation of 'tacit or sticky' knowledge from them (Osborne et al., 2012, p. 146), encourages process of learning through trials and errors, and allows to share the burden of innovation.
Co-production in ULLs also suffers of some limitations. A first problem concerns maintaining motivation to collaborate high among volunteers. Stable participation in co-productive activities is essentials in ULLs to achieve significant and effective results in the experimentation. In Amsterdam, professionals complained about the fact that someone quits the Lab without completing his or her work on sensors. Some participants, in fact, were very enthusiastic about the project at the beginning but when problems occurred during the experimentation with the hardware and software components they became demotivated.
A second problem concerns the governance of co-production. In general terms, the management of ULLs can be very challenging for public administration. Civil servants who coordinate Labs, in fact, must adopt a radically new mind-set because open innovation requires a flexible approach to problem-solving that is different from traditional project management where objectives and resources are supposed to be pre-determined. Interviews with key-informants, moreover, revealed that coordination among public agencies involved in ULLs could be a difficult task if they do not share the same working method and procedures. Thus, professionals must put great effort into bringing coherence to the process, and sometimes they lack sufficient time, resources, and/or experience to allocate towards it.
The last problem relates to the sustainability of ULLs. On the one side the question is whether is possible to institutionalise them or, put in other words, how to transform ULLs' pilot projects into continuous programmes of local innovation (Kantola et al., 2014). The literature suggests that ULLs are generally of a limited duration because they are often subsidised by a local politician (generally the mayor) or by a chief official who creates them to carry out specific projects and then shuts them down when that project is complete (Tonurist, Kattel, & Lember, 2015). This aspect has been confirmed by the empirical analysis. Both in Boston and Turin Labs were sponsored by local politicians (the mayor and an executive councillor, respectively) and they are still surviving after having accomplished their mandate thanks to political will. But the real challenge is how to mainstream the ULLs methodology into the ordinary policy-making. Unfortunately, the empirical analysis leaves this issue open. Only the Housing Lab in Boston, in fact, has been recently 'stabilised' within the Municipal administration but it is too early to assess the impact this choice would have on the more general housing policy.
On the other side, financial sustainability can be problematic, too. Due to their time-limited nature, ULLs usually have narrow budget. Since innovation can be costly and with long-term effects, local administration can be reluctant to finance a Lab for a long time. An alternative solution to public funding can be the search for external funds, like in the case of the Boston Municipality that subsidised the Housing Lab with a Foundation's grant. Or the Turin's choice to have a Lab at no cost for the administration. Other empirical research on ULLs, however, revealed that one striking characteristic of these experimentations is their high mortality rate (Nesti, 2016).

Conclusions
Co-production has been the focus of particular attention from scholars for decades. In recent years, co-production attracted the interest also of public authorities and non-profit organisations for its potential capacity to solve complex policy challenges through citizens' collaboration. At the local level the search for innovative solutions to local problems, especially in the environmental sector, has been encouraging municipalities to create urban laboratories where citizens are involved in experimenting new products or services with professionals through a co-production approach.
Drawing on the analysis of three experiences of ULLs the paper confirmed that the co-productive activity is an essential component of Labs. But it also revealed that this form of co-production has its own peculiarities, particularly the fact that it takes places mostly in the co-design stage of the service cycle while the co-delivery stage is represented by testing new service or products. These differences mainly stem from the experimental nature of ULLs that is base on a 'prototyping' approach to public innovation.
Remarkably, experimentation, open innovation, active civic engagement and the creation of physical products to support policy implementation would probably become distinctive elements of future processes of policy-making also as a consequence of the pervasive diffusion of ICTs, new media, and Internet applications.
A future challenge for academics, therefore, would be to take these changes into consideration in their reflections on the concept of coproduction. For policy-makers the challenge would be to guarantee an effective and sustainable management of ULLs and to improve the capacity of co-producing innovation with citizens.
The growing relevance of the topic, the presence of a still underdeveloped literature and empirical analysis on ULLs suggest that further research on co-production through ULLs, on its characteristics, benefits and limitations is needed.