CO/DEsign: building a shared dialogue around analysis within co-design

Abstract In times of rapid social, economic, environmental and technological change designers can play a valuable role by applying their creativity to catalyse innovative solutions to address complex problems. As they do so, it becomes apparent they need to ask fundamental questions about what they make, how they make it, and who for. The mindsets and postures of designers often go unnoticed and unacknowledged, but they profoundly influence what is identified as a problem and how it is framed and addressed. This paper draws upon a research project titled ‘CO/DEsign’, which explores the application of agile co-design methods in an endeavour to understand and identify the most appropriate approach for rigorous analysis. The ‘CO/DEsign’ project argues that, while it is important to draw upon other disciplines and borrow methods such as thematic analysis, further methods should be developed that better represent and support designers and their approaches.


Introduction
Living in and through transitional times calls for self-reflection and new ways of 'being' in the world. Our individual and collective mindsets represent the beliefs, values, assumptions, and expectations formed by our individual experiences, cultural norms, and the socio-economic and political paradigms to which we subscribe. Irwin, Kossoff and Tonkinwise (2015) believe new ways of designing need to be informed by knowledge outside design (science, philosophy, psychology, social science, anthropology, and the humanities etc.) in order to gain a deeper understanding of how to design for change and transition in complex systems. This paper will explore aspects of qualitative data coding in co-design, which this paper has titled CO/DEsign. Through this work the authors sought to identify the most appropriate design-led approaches for analysis that attend to addressing gaps in knowledge and practice. This builds upon the work of Krippendorff (2005) who argues that designers must be provided with new ways to substantiate the claims made for their design contribution. Through positing the research question, 'which research principals are required to support data analysis post co-design?' the paper draws upon a review of current perspectives and practice, including insights generated from a co-design workshop, alongside semi-structured interviews between the authors as co-design researchers. The findings will be discussed to extrapolate ways in which creative and collaborative design approaches can be used to stimulate productive dialogue around both the analysis, and broader research issues. This led us to identify six principles for design researchers to consider when conducting analysis. We then conclude by reflecting on the implications of these principles to highlight the significance of analysis within design-led approaches in strengthening communication, promoting creative action, and embedding collaborative ways of working.

Current Perspectives
Over the past six decades, the evolution in design research from a user-centred approach to codesigning is changing the role of the designer, the researcher and the person formerly known as the 'user'. As a result, a range of different design research methods, tools and artefacts to capture valuable insights have been developed, which places increasing demand for novel approaches to evaluate and analyse the end results. This scope of current perspectives does not provide an exhaustive overview, but identifies a limited scope of literature relating to the role of the specifics of analysis within design from a design researchers perspective to begin to consider how processes of analysis in design research can gain stronger validation.
Design research has largely been framed as research for design (clinical), research about design (basic), and research through design (applied) (Frankel & Racine, 2010), asserted through influential figures such as Sir Christopher Frayling, Ken Friedman, Richard Buchanan and Donald Schön, among others. Commenting on how design research knowledge moves into practice, Friedman argues that, 'at every stage, knowledge, experience and questions move in both directions… Practice tends to embody knowledge. Research tends to articulate knowledge' (Friedman, 2000: 23). The challenge for identifying modes of analysis in design research is in the sheer breadth of options that can be applied, which shows both a healthy growth in an advancing field and a lack of cohesive methodological theory. As a starting point for this paper, however, thematic analysis was identified as a widely cited method borrowed from the field of social science. Braun and Clarke (2006) define thematic analysis as: "A method for identifying, analysing and reporting patterns within data," (2006:79). However, it is important to acknowledge that thematic analysis is "not another qualitative method but a process that can be used with most, if not all, qualitative methods..." (Boyatzis, 1998:4). This method is widely used as it allows for researchers to select their theoretical framework. The thematic analysis process allows for a detailed description of data through four key stages, which involves: i) coding text, ii) developing descriptive categories, iii) generating analytical themes, and iv) demonstrating results (Braun and Clarke, ibid).
Following a review of key literature, the authors presented a discussion poster to capture insights from researchers across the authors' home institute around this issue. Within perpendicular axes, design and social science were positioned North to South, alongside intuition and rigour West to East. The audience was invited to cite references and a colour code was assigned to differentiate case studies, projects, papers, tools, research methods, frameworks and finally an open category for additional input. The captured data is documented within figure 1 (see below). The open category provided discussion on the role of objects and artefacts, and serendipity, to convey that often the most valuable insights are unexpected and can't be captured through a controlled process. The other categories referenced actor network theory (Latour, 2005), situational analysis (Clarke, 2005), participatory action research (Walter, 2009), and design things (Binder et al., 2011). The research around design thinking was also considered valuable and cited Brown (2008), Kimbell (2011) andNussbaum (2011).
The differentiation between rigorous and intuitive methods provoked rich discussion and was intentionally engineered to be provocative, drawing upon Krippendorff (2005), who argues that a healthy design discourse must examine itself and continuously expand its vocabulary. The captured data depicts some divided examples that bridge across design and social science. Essentially, the references more akin to participatory design research were plotted centrally and closer to the core, emphasising that design led research analysis aspires for both rigour and intuition and, while it initially borrows from social science, further work is required to evidence design-led approaches. The approach taken for this focused study was to engage with researchers from multiple disciplines working within the authors' home institute. This engagement used a mixed method approach of design-led activities, thematic analysis and semi-structured interviews (see figure 2) to develop an insight into how analysis of co-design approaches was being performed across sixty-eight Experience Labs (French, Teal & Raman, 2016) delivered up to the time of this paper.
Following the first objective to gather literary sources, the second objective was to deliver an Internal Lab, which was an exploratory design-led workshop to facilitate internal discussion on the processes of designing and delivering Experience Labs to analyse findings for a final, deliverable report. The Internal Lab involved ten participants overall -five designer/researchers from Experience Labs, and five design/researchers from the home institute more widely -engaging in three activities. The first activity split them into pairs, one from Experience Labs and one from the home institute, and asked them to fill out a profile card for each other on their background discipline(s) and what they loved or found a challenge in their disciplinary experience of research. The second activity kept them in their pairs and asked the home institute participant to interview the Experience Lab participant, with an A3 interrogation sheet, about a particular Lab they had delivered. On this interrogation sheet they captured the project title, how data was captured, what happened to data gathered, before reflecting on approaches they value, as well as any weaknesses or risks they see in the approach. This led into a final activity, labelled the data wall, where a long whiteboard sheet was split into three sections replicating the interrogation sheets: data capture, data analysis and a SWOT analysis matrix. The wall was used to facilitate discussion with the whole group on these topics, asking each pair to share the stories and insights they captured around the processes of each Experience Lab participant. Only the group discussion session was audio recorded for the purposes of analysis for the next objective.
The third objective was to perform thematic analysis separately on each of the different forms of data capture used during the Internal Lab: the design tools, audio transcript and data wall. Each data analysis was performed by the three lead authors to ensure individual accounts. The fourth and fifth objectives were for these separate accounts to undergo a form of triangulation, where each researcher was interviewed about their process of analysis, their key findings in this short analysis and any reflections regarding the process. The researchers then brought their findings together for a S4245 final comparison between each process of analysis, and any insights generated by bringing them together, towards an initial provocation of six principles of design analysis for wider discussion.

Internal Lab Outputs
There are two levels of findings presented from the above methodology: direct findings from engaging the research staff internally at Experience Labs, and extrapolated findings through processes of thematic analysis and reflective semi-structured interviews. The Internal Lab will be presented first according the three activities delivered.
The profile cards revealed all ten participants' diverse backgrounds, which had not been shared in this way before. These varied from textiles, engineering and artificial intelligence, to psychology, journalism and architectural heritage. Participants also expressed this multidisciplinarity as both an area of strength if it can be managed, and a weakness if team members do not have a shared frame of reference.
The interrogation sheets revealed varied methods of data capture, with strong reference to multimethod approaches, bespoke visual mapping, provocations based on existing examples, prototypes, storyboarding and role-play. There were also singular references to objects as abstract representations, preferred future projections, visual feedback and even ideation. Across approaches of data analysis there was strong reference to thematic analysis, the importance of transcriptions, affinity mapping, post-its to summarise data and even analysis written directly into reports. Wider references of interest included the use of feature documents (such as idea refinement), how visual tools are not used, and yet ensuring reports were image rich. Of particular interest are the summaries of strengths, weaknesses and risks involved in these processes, which are summarised below in table 1. The plenary session facilitated conversation through the 'data wall' by posing the question, 'how do we get from the post-it note to the final report?' The purpose and context of each project or design brief was highlighted as key and each design researcher invested a lot of time in developing bespoke design tools for each project. A SWOT analysis highlighted gaps in decision-making and, while the experiential skill of the designer can be perceived as a strength, it needs to be interdisciplinary. They saw opportunities for building multidisciplinarity into analysis but recommended triangulation be applied to avoid bias. Participants in the Lab wanted more scope to build on key points, such as reflecting on the analysis of artefacts, or were divided upon other points, i.e. the application of different methods. Overall, there were more insights on data capture than on data analysis, despite trying to frame conversation towards analysis, which showed the participants had more confidence around capture and agreed more work needed to happen to improve confidence in processes of analysis.

Design Tools
As presented in previous sections, there were two physical tools of data capture used within the Internal Lab: the profile cards, and the interrogation sheets. For this first line of analysis, the sections of both tools were summarised in an excel spreadsheet and then written onto post-its, and colourcoded according to their original source. This was a practical step of summarising what was on the tools as descriptive sub-categories, some of which were numbered where multiple examples occurred, so that they could be physically moved and clustered. The first process of clustering produced five core categories: building in authority, collaborative focussed, representing design contribution, serving the context, and practical activities. These were then each put through a second phase of clustering to produce sub-categories to break down each core category and substantiate them as initial themes in the data captured. Building in Authority broke down into 'proven methods of analysis' and the 'committal of resources'. Collaborative Focussed broke down into 'building consensus', 'facilitating critical evaluation', 'connecting disciplinary skill/expertise' and 'collaborative validity'. Representing Design Contribution broke down into 'connecting practice with context', 'purely qualitative', 'making the intangible tangible', 'live intuition' and 'articulating progression'. Serving the Context broke down into 'building meaning/purpose' and 'ways of expressing experience', while Practical Activities was simply redescribed as 'serving the process'.
In his interview reflecting on this process of analysis, the co-author expected that the data source would be the least in depth, however, due to the focussed nature of data capture, he was surprised by how he was able to see connections easily. The researcher's own experience of delivering Experience Labs also fed into this and was not seen as a problem, but a supporting part of the analysis. Participants had already gone some way to 'self-categorise' due to the need for concise answers, which made it easier to analyse. However, the more the author categorised, the more he reflected they became his own interpretations in his own language. Overall, the process felt like a validation that the tools asked the right questions.

Data Wall
For this second line of analysis, the data captured on the data wall was documented onto post-its, then clustered and grouped. Following this clustering, the respective co-author revisited the data wall to ensure nothing was missed and revisited the themes to draw out what key categories were there. The key themes that emerged from this analysis included: triangulation of different methods; the role of the artefact or tool; audio capture as a consistent method; bag of mixed methods; and adaptability to contexts. Of particular interest was the discussion around a 'bag of tools' to modify, alongside context being expressed as really key for generating bespoke tools. There was evidence of a lot of preparation and mindfulness for generating methods for data capture, however there was a disconnect in what the processes of analysis entail, as people will call their method affinity mapping or thematic analysis, but the levels of rigour could vary drastically.
The data wall was reflected as an authentic series of findings documented in the moment, in a live context, while it was emerging. Participants did use the data wall to refer back to a point to verify things and discuss issues, particularly as a more inclusive approach. In the data analysis, those connections did remain. However, there were topics that were unanswered or couldn't be resolved within the discussion. There were prescribed headings that potentially drew out themes in the Internal Lab, so it was focused in that key themes were pre-defined (data capture and analysis). The headlines were really clear to reflect back on, however, it could have been more detailed, possibly through using the audio recording. With more time in the plenary discussion, the author also reflected attempts to highlight the role of the designer, and how the plenary could have been more methodical to draw out prominent themes with participants, as a form of co-analysis.

Audio
For this third line of analysis, the audio was transcribed for the respective co-author to read through, needing to refer to the audio for clarification, and used an online coding software to arrive at six overall categories: tool use, analysis of data, differences in approach and understanding, differences in ideology, key items to address when moving forward and uncertainty. Co-design tools were by far the easiest for the group to talk about, with some consensus that tools were a good way to prompt participants to share their experiences and move a conversation forward. The transcript fell short of capturing valuable nuances as notes were often incomplete sentences or words that only made sense in the context -hence the real data capture happened in the audio. During the workshop it became evident that different members of staff had not just different backgrounds, and hence different frames of reference, but that they also had differing approaches to analysis, which were informed by their differing educational backgrounds and by differences in ideology. The group mixed and matched methods brought in by different members of staff, giving the group a rich source of methods, but with that also the uncertainty of application. The authors would need to address the overall findings from the Lab to do with the individual versus collective understanding, the use of quick learning from other fields, and the uncertainty that follows when finding inspiration, methods and tools in other disciplines.
Reflecting on this process of analysis, the transcripts themselves were somewhat obtuse and required the co-author's own knowledge of the situation and context to make sense. Hence the role of the researcher as insider became both an obstacle to objective coding, as well as a necessary interpreter. Her findings also supported Archer's point that empiricism may be more important in the stages when a theory is under test than in the stages when it is being formulated (Archer, 1995:7) as attempting to apply any form of rigorous analysis at this stage in the process where the inquiry was still being formulated, was difficult. In fact, the process was viewed as not a good method for picking up underlying issues: "An outside researcher may have been more clear-cut about these issues, but I was questioning my own judgement". Group conversation made it more difficult to analyse, as conversation could have happened more informally, rather than part of a research process. The process in the Lab itself could have given more time for informal conversation before relaying the issues on the wall. Perhaps focusing on one thing at a time, giving each subject more space for discussion. Finally, using coding software was not seen as particularly useful, compared to possibly spending more time with the participants and "asking them to provide categories […] as a sort of coanalysis."

Discussion
From the above process of firstly engaging researchers within Experience Labs, then performing parallel processes of analysis and reflections, this discussion section presents our triangulation of findings as six emergent principles for design researchers conducting analysis. These are not provided as definitive principles, but more provocations for further research and discussion. It is recognised that the research performed for this paper had very limited scope to one project's cohort of researchers, however, the depth of inquiry is argued to suffice for setting a direction by which processes of analysis in design research can begin to frame consistent approaches.

Design Analysis should be framed within the context of investigation
From engaging the Experience Lab researchers, there was clear consensus and consideration for the design context as part of designing methods of data capture. The confidence from participants came in discussing their skillset towards exploring and responding to emergent insights within the context through bespoke methods. However, such a process means findings come from various sources at different stages, meaning a singular articulation of where findings come from can be very difficult. This explains the attraction of thematic analysis, as it was presented as 'a process that can be used with most, if not all, qualitative methods...' (Boyatzis, 1998:4). From the process of analysis, the focused nature of the design tools and data wall, which built on our initial process of scoping perspectives, ensured the context of 'design analysis' was embedded in the responses captured, whereas the discussion audio lacked such explicit reference points. Just as Friedman stated 'practice tends to embody knowledge ' (2000), so design analysis can be validated by iteratively connecting findings back into the context, often through methods and tools carrying that embodied knowledge.

Validation of findings depends on rigorous forms of co-analysis
From the Lab engagement, the group were cautious in talking candidly about the differences in their processes of analysis. Sanders and Stappers (2008) talk about a jostle between disciplines and claims to methods and analysis as being a step on the way to a new form of designers. The finding from the Lab was not on this jostle on which is the best data capture or analysis, but on what is perhaps the next step, namely, how to be able to move forward harnessing the power of an interdisciplinary team, as well as navigating between different disciplines. This opens up an exciting opportunity space for new forms of analysis. Part of the anxiety felt in the group is argued to have stemmed from the cultures of each disciplinary background. Where traditional methods of analysis in social science and psychology call for rigour through multiple carefully managed steps, design processes don't naturally follow such prescriptive patterns, allowing for individual embodied knowledge to come to the fore. From our process of analysis and reflection, we had deliberately deconstructed our forms of data capture, which revealed their individual strengths and weaknesses. Our capacity to make sense of our findings called for the opportunity to have included the participants as part of co-analysis. For design more widely, we argue that validation requires participants or stakeholders in the context being including as part of making sense of any findings.

Design rigour is based on iteratively co-developing frames of reference
From the Lab engagement, and gathering of perspectives, there was a clear breadth of frames of reference for co-design research. This is seen as symptomatic of co-design engaging in increasingly complex contexts in increasingly complex ways. Just as co-design needs to be framed by its context, the approach to framing that context should be under constant scrutiny through evidence developed in ongoing design activity. From Frayling's (1993) research 'for', 'through' and 'of' design, to the proliferation of management and design tools of analysis in design thinking (Brown, 2008), to newly adapted frameworks from other disciplines; such frames need to be translated across disciplinary perspectives to produce a common language of understanding. Rigour should increase the acceptability of a piece of design research. From our process of analysis and reflection, we only completed one cycle of iteration, but began to see pathways to a more rigorous approach.
The following three principles are proposed on a less sure footing, but represent nuances from our research that were deemed, nevertheless, worthy of inclusion:

Design analysis needs to reflect the complexity of design activities
This, again, echoes the framing of research 'for', 'through' and 'of' design, but more specifically how specific methods and techniques produce different qualities of outputs. From the process of analysis, it was striking how much more effective the design tools were as a form of data capture than the transcript from our audio discussion. During the Internal Lab, it transpired that, although all participants designed tools for use in their design workshops with participants, many fell back on the audio recording of workshops for capturing data. The authors also 'fell back' on using audio as a means of data capture, but reflected that it best represented a resource to validate analysis, rather than define it. Depending on the contextual need for evidence, a narrative should be built on insights across multiple methods of data capture weighted by reflections on the quality of interaction.

Design analysis needs to acknowledge changes in the application of design activities
A common understanding of design research, particularly co-design processes, is that the intended purpose of design activities can shift according to how participants respond. This means designers as facilitators require an almost 'improvisational' skillset, able to engage a meaningful conversation with participants, as well as collaborators, in multiple ways (Brandt, Binder and Sanders, 2012). These changes in approach or application of methods are just as important to capture and acknowledge in design analysis as any directed response to design inquiry. Knowing why a design activity changed is crucial to being able to critically evaluate how important this impacts contextual understanding, processes of validation, frames of reference, or sheer design competence.

The ultimate validation of design analysis is its influence after the lifespan of the research
This principle could be applied to all aspects of research in many ways, but it is felt as particularly poignant in this paper. All the principles above have emphasised collaborative processes of analysis towards progressive modes of understanding. Research, after all, is about the articulation of knowledge (Friedman, 2000), while practice is about the embodiment of knowledge. Analysis is argued here to form the bridge between articulation and embodiment for co-design, by itself being a collaborative process.

Conclusion
From gathering broad perspectives to design analysis, to engaging a captive group of designer/researchers, to a focused process of analysis and reflection, this paper levered the collective insight of researchers within the Experience Labs project to propose initial principles and practices in design analysis. Each cycle refined the frames of reference -particularly around the strengths and weaknesses of analysis in design -towards the principles presented above, which will ultimately form questions for further refinement, both with the researchers at Experience Labs, and with wider audiences of design research.
This paper not only examined aspects of qualitative data coding, such as thematic analysis, in codesign but also sought to identify the most appropriate design-led approaches for analysis that attend to addressing gaps in knowledge and practice. This builds upon the work of Krippendorff (2006) who argues that designers must be provided with new ways to substantiate the claims made for their design contribution. We drew upon a contextual review of current perspectives, including insights generated from a series of design-led activities. The findings were discussed to extrapolate the ways in which creative and participatory design approaches were used within Experience Labs around both the analysis, and broader research issues. This allowed us to identify six design principles for design researchers to consider when conducting analysis. These principles are intended to acknowledge the collaborative and interdisciplinary nature of current and future design research, and how current practices of analysis need to adapt in order to keep design relevant as a leader of innovation. It is not enough now to simply acknowledge emerging landscapes of design (Sanders and Stappers, 2008), it is necessary now to know how to shape them.