Systems scenarios: a tool for facilitating the socio-technical design of work systems

Abstract The socio-technical systems approach to design is well documented. Recognising the benefits of this approach, organisations are increasingly trying to work with systems, rather than their component parts. However, few tools attempt to analyse the complexity inherent in such systems, in ways that generate useful, practical outputs. In this paper, we outline the ‘System Scenarios Tool’ (SST), which is a novel, applied methodology that can be used by designers, end-users, consultants or researchers to help design or re-design work systems. The paper introduces the SST using examples of its application, and describes the potential benefits of its use, before reflecting on its limitations. Finally, we discuss potential opportunities for the tool, and describe sets of circumstances in which it might be used. Practitioner Summary: The paper presents a novel, applied methodological tool, named the ‘Systems Scenarios Tool’. We believe this tool can be used as a point of reference by designers, end-users, consultants or researchers, to help design or re-design work systems. Included in the paper are two worked examples, demonstrating the tool’s application.

Since Trist and Bamforth first coined the term in 1951, the merits of applying socio-technical principles to the design of work systems have been well documented (e.g. Trist and Bamforth 1951;Cherns 1976Cherns , 1987Clegg 2000;Kleiner 2006;Eason and Waterson 2013), and increasingly organisations are trying to apply them in practice. In essence, the socio-technical approach argues that work systems delivering products or services, comprise a social system (e.g. the people, working practices and roles, culture and goals) as well as a technical system (e.g. made up of the physical infrastructure, tools and technologies); and that work systems can only be fully understood and improved if these parts are treated as interdependent elements. This is because changes to one part of the system can necessitate changes to another. There is a body of evidence to demonstrate that treating systems as separate units -the more typical approach -can lead to overemphasis of some parts of the system, at the expense of others (e.g. Clegg and Walsh 2004;Mumford 2006;Seiffert and Loch 2005;Symon and Clegg 1991). For instance, Clegg and Shepherd (2007) have shown how organisational change initiatives that are driven solely by technological innovation, but fail to consider the way that humans interact with these technologies, are less likely to succeed; whereas applying a socio-technical systems approach has been shown to lead to successful organisational change interventions (e.g. Atkinson et al. 2001;Axtell et al. 2001;McGowan et al. 2013). More recently, evidence of the merits of the STS approach have influenced the development of related fields such as macroergonomics and systems ergonomics (Hendrick 1991;Kleiner 2006).
A variety of socio-technical and macroergonomic frameworks are presented in the ergonomics literature (see Carayon 2006 for synthesis of these). In particular, drawing on the work of Leavitt (1965), who viewed organisations as comprising four key interacting variables -task, structure, technology and people (actors) -the socio-technical hexagon pictured in Figure 1, has been developed (see Clegg 2000;Davis et al. 2014). This contains six core components and provides a high-level framework for analysing and understanding complex systems. The hexagon uses lines to represent the dependencies that exist between the components of the socio-technical system, and reinforces the argument that variables must not be approached in isolation when enacting organisational change. It is this Unquestionably then, a range of 'user-centred' , 'scenarios-based' and/or 'socio-technical analysis' design tools do exist. However, our analysis of those tools with features most similar to the SST -synthesised in Table 1 -shows that none satisfactorily meets all of these requirements simultaneously.
Moreover, amongst the tools that do exist, there remain challenges (Salmon et al. 2016). For instance, work systems are complex ones, often dealing with 'wicked' problems, in which the problem itself is not always clear to stakeholders, let alone the solution (Rittel and Webber 1973;Camillus 2008). Wicked problems are those that are difficult or impossible to solve, often due to incomplete or contradictory information, the large numbers of people involved in them, large economic implications, or because they are interconnected with other problems. Such problems are inherently socio-technical (e.g. see Westbrook et al. 2007), because changes to one part of the system will result in changes to others whether or not they are anticipated or initiated, and whether or not such change is desired. Recognising the challenges inherent in such work systems, there remains a need for tools that provide a means of gaining awareness of, and managing such unanticipated system changes and ripple effects.
Some existing tools have been criticised for being 'too academic' , and impractical to implement in practice, often because they require specialist software, skills or training to implement, or because they require substantial financial investment that is beyond scope for many organisations (e.g. Etzioni, 2000, as cited in Holman et al. 2003, p;.337;Wastell 2011;Waterson et al. 2015). Indeed Salmon et al. (2016, pp. 10) note that: 'despite the critical role of the design process, few ergonomics methods are actually used by designers to design'. Accordingly, several authors have called for investment in tools that incorporate systemic thinking into real-world design processes, and which enable us to analyse, understand, design and/or re-design work systems (e.g. Baxter and Sommerville 2011;Crowder et al. 2012;Lockton, Harrison, and Stanton 2010).
Addressing the gap identified in Table 1, this paper presents one such tool that we have developed, named the 'System Scenarios Tool' (SST) which provides a means of applying socio-technical thinking to the design of applied work systems. Characterised primarily by a socio-technical and user-centred approach to scenarios planning, we believe that this particular method offers a range of benefits that are not simultaneously realised by existing tools. This paper therefore answers four key questions: (1) What is the SST? (2) What are the potential benefits of the SST?
(3) What are the potential limitations and difficulties of the SST? (4) Under what circumstances is the SST most useful? The paper is organised in four corresponding sections. framework that underpins the SST and informs the discussions (described in subsequent sections) that are fundamental to its implementation.
Although the socio-technical systems approach to design is well recognised and supported (e.g. Charnley, Lemon, and Evans 2011;Mumford 2006), it is also acknowledged that realising the approach in practice can be challenging (e.g. Unsworth, Dmitrieva, and Adriasola 2012;Baxter and Sommerville 2011). Recent reviews of the methods available for ergonomists, have demonstrated that a variety of tools do exist to enable the application of systemic thinking, to organisational work problems (e.g. Waterson et al. 2015;Salmon et al. 2016;Stanton et al. 2013). These methods differ widely in scope and in their strengths: • Some methods are designed to facilitate understanding of specific organisational problems (e.g. safety -see Rasmussen 1997;team work -Grote et al. 2000), and do so in a high level of detail. • Some methods are used effectively for retrospective analysis (e.g. Leveson 2004), whereas others focus predominantly on predictive or futuristic design (e.g. García-Mira et al. 2016). • A number of methods are deliberately comprehensive, but are consequently time-consuming, so do not suit more low-key systemic analysis (e.g. Kleiner 2006). • A number of methods take a user-centred design approach (Go and Carroll 2004) by considering needs of different stakeholders, though fewer tools actually involve participants from all stakeholder groups in the design process. • Some methods enable the mapping of tasks and processes (e.g. see Salmon et al. 2010); but focus less explicitly on system implementation issues. • A number of methods include the development of scenarios to facilitate futuristic innovation (e.g. Carroll and Rosson 2007;Grote et al. 2000). • Some methods closely apply a particular socio-technical or macroergonomics framework (e.g. Kleiner 2006;Rasmussen 1997), whereas others are guided more generally by socio-technical principles.  (3) Collect 'as is' data Having identified system boundaries and performance criteria, a more formal process of data collection is undertaken with stakeholders to help generate a detailed description of the 'as is' (existing) system, using Davis et al. 's (2014) socio-technical framework, along with its performance against selected criteria. Typically, this involves interviews with representatives from each stakeholder group. If interviews are not feasible, stage 3 of the process can be omitted, and the data can be collected during the workshop (see stage 4).
(4) Analyse the 'as is' Stakeholder representatives attend a workshop and the 'as is' description established in stages 2 and 3 is presented to them, using a series of structured templates (described in more detail later), reflecting the socio-technical framework. Participants check the accuracy of the description and any discrepancies are discussed and addressed. Stakeholders then consider the system's pros and cons, and rate the existing scenario against the previously agreed criteria.
(5) Consider the 'to be' The workshop attendees then work in mixed (in terms of skills, experience and background) sub-groups, to develop alternative ways of organising the work system (i.e. a set of 'to be' scenarios). These are developed using the same structured templates as before. Some groups may opt for incremental 'safe' changes, whilst others favour more 'radical' designs. Groups can be asked to develop 'to be' scenarios to maximise performance against a particular criterion or objective, e.g. quality. Groups are encouraged to be innovative, work through the template headings, and consider the implications of their choices for the rest of the system. Once complete, and still working in sub-groups, each scenario is scored against the same, agreed criteria. In the workshop, groups report back on their design solutions to a plenary session and these are critically reviewed.
(6) Make choices and agree action plan In plenary, the 'as is' and 'to be' scenarios are rated and ranked against each other, in terms of how well they meet (or are expected to meet) the system's performance criteria. The resulting templates enable stakeholders to calculate scenario 'scores' which help inform choices about which scenario(s) to develop and which to rule out. Finally, a plan of 'next steps' is agreed, although the nature and detail of this will depend on the purpose of the workshops and the outputs agreed in stage 1.
The SST can be applied in a range of circumstances. Two examples therefore follow, to illustrate the versatility and value in this approach. The first demonstrates the application of SST to strategic planning, where the desired

What is the SST?
We begin this section by considering what the tool is, before describing how it works and the kinds of outputs it can provide. We then offer two worked examples of its use to illustrate the diversity of the tool, based on our own application of the SST.
Central to the SST is the logic that all work systems are designed through a series of choices, which may have been consciously or unconsciously made. Given the interdependencies inherent in work systems, these choices matter, because a choice that is made about one part of the system (e.g. to set particular targets or goals, or to use a particular technology), will affect many other parts of the system (e.g. it may require new processes or job roles). The SST helps make explicit the choices that underpin a system, and in so doing, enables those choices and their consequences to be scrutinised, from the perspectives of the different stakeholders, allowing them (and not just those in charge of the system) to become its architects.
The SST takes the format of a workshop, or a series of workshops. Key stakeholders are brought together from a cross section of all stakeholder groups involved in the system, to work collaboratively through a set of staged discussions. The SST process is straightforward, and can be summarised in six broad stages: (1) Involve key stakeholders Key stakeholders are identified and invited to take part in the process. In some cases, stakeholder groups can be easily identified through brief scoping interviews, but for more complex systems it can help to undertake more formal stakeholder analysis, where individuals' interest in, and relevance to, the system, as well as their control over resources can be assessed (Brugha and Zsuzsa 2000;Lindenberg and Crosby 1981).
(2) Agree on the system parameters Preliminary discussions with stakeholder groups help clarify boundaries around the system under examination. At this point, objectives for the workshop are also established, including agreeing what the workshop's outputs will be. These may include -but are not limited to -comprehensive analysis of the current system, identification of alternative ways of working (the new 'scenarios'), recommendations for improvements to the system, and/or decisions and actions to generate improvements. Stakeholders then identify and agree on some criteria to evaluate the system's performance. Typically some examples are proposed as a starting point, e.g. (1) High quality outputs; (2) On-time delivery; (3) Meeting the needs of the consumer; (4) Coping with variations in demand; (5) Low overall cost. The group should debate these parameters, before coming to agreement. key stakeholders were identified using snowball sampling (Coleman 1958). Stakeholders included healthcare commissioners, senior managers, industrial suppliers and manufacturers, technical installers, patients, and frontline staff (e.g. General Practitioners, nurses and community care workers).
(2) Agree on the system parameters It was agreed during these initial scoping interviews, that the system under examination would be as described above (40 diabetic patients, using telehealth equipment to monitor their vital signs at home).
During these interviews participants were also asked to identify key criteria to measure the system's performance against: 'What would you identify as the five most important ways that we could judge whether we had been successful in improving the [telehealth service] system?' The following performance criteria were agreed: (1) Provides low overall cost; (2) Copes with variation in demand for telehealth; (3) Reduces number of hospital admissions; (4) Meets patient needs; (5) Enables wide use of telehealth.
Those commissioning the work within this site requested that the process should lead to recommendations for actions (short-, medium-and long-term) to improve the delivery of telehealth services for users at this site, whilst delivering some new, alternative models for future service delivery in the future.
(3) Collect 'as is' data Representatives from each stakeholder group were interviewed about the existing telehealth service, to help gather a detailed description of the 'as is' system, along with their views on the barriers and facilitators of the existing service. 1 (4) Analyse the 'as is' The data from stage 3 were collated and thematically analysed 2 to develop a systemic description of the existing system. Stakeholders were then invited to attend a workshop where the 'as is scenario' was presented (see the first two columns of Template 1- Table 2). The group was asked to review this scenario in plenary discussion, where it was considered and then agreed as a satisfactory representation of how the system currently operates.
The workshop participants were then asked, in subgroups, to: 'Consider the extent to which the current scenario overall (i.e. the "as is") rates against the criteria agreed earlier. ' (Table 3) The scenario was rated on a scale of 1-10, where 1 meant 'this criteria is not met at all' and 10 meant 'this criteria is met perfectly' . The maximum score across the five criteria is therefore 50 (i.e. 5 × 10). The agreed scores are presented in Template 2 (column 2), showing that the workshop attendees thought the 'as is' service was performing well at reducing hospital admissions, but poorly at coping with variations in demand for services. outcome is innovative discussion, to inform long-range planning decisions. The second example demonstrates the application of SST to an acute organisational problem, involving evaluation of an organisational structure. Consideration of the tool's utility is then considered based on these two examples.

Example 1 -designing the future for UK telehealth
The first example is based on a research and development project that we worked on which explored how telehealth can be effectively utilised in the National Health Service, in England. Telehealth typically refers to the delivery of health-related services and information via telecommunications technologies in the patient's home. This can range from technologies such as personal alarms and self-monitoring equipment such as blood pressure or glucose monitors, to sophisticated video-conferencing technologies, which enable a patient to speak with specialist professionals, without having to leave their home. The aims of telehealth deployments are typically to help people self-manage health conditions, to reduce the need for outpatient clinic visits and hospital admissions, and to help people live independently for as long as possible (Department of Health 2011).
We employed the SST to understand the existing system, and to design some alternative scenarios for future telehealth delivery, that would improve telehealth provision for this group, and lead to the mainstreaming of telehealth in the longer-term. Eight SST workshops were undertaken as part of this research programme, across 4 different sites. They yielded results that varied in their depth, with some groups more 'blue sky' than others in their suggested 'to be' innovations. For illustrative simplicity, the following example is therefore based on an abridged summary of the key results, to demonstrate how the SST was applied within one such site.
This telehealth system was implemented by an NHS outpatient diabetes clinic operating in the north of England. The clinic had purchased 40 pieces of telehealth equipment and these were deployed for home use by 40 long-term patients who were regular users of outpatient clinic services. The equipment enabled patients (and their carers) to monitor their blood pressure, glucose levels and heart rates at home, and to report the data electronically to the clinic. The new telehealth system operated in parallel with 'normal' outpatient clinic services.
The process was as follows: (1) Involve key stakeholders Scoping interviews were undertaken to understand the existing telehealth system in operation, and during these, Table 2. Template 1: A worked example considering the future of telehealth in a UK health setting, containing both 'as is' and 'to be' scenarios.

As is scenario
To be scenario 1 To be scenario 2 Boundary: (i.e. The organizational unit under investigation) An outpatient diabetes clinic, which has purchased and deployed 40 units of telehealth equipment to enable 40 long-term outpatients to monitor key health indicators at home The new telehealth system operates alongside existing clinic services An outpatient diabetes clinic, leasing units of telehealth equipment to enable higher numbers of outpatients to monitor key health indicators at home Telehealth is offered alongside existing clinic services, by a new, inhouse, telehealth team (comprising telehealth-trained nurses) The outpatient diabetes clinic withdraws from the telehealth market directly. The clinic recommends useful telehealth products to patients/carers, which can be purchased by patients/carers if desired (e.g. via Amazon, e-bay, directly from the supplier) clinicians collaborate with suppliers to keep up-to-date with products and ensure that recommended products use interoperable systems; but clinic services are themselves unchanged Vision: (i.e. global summary of how the organizational unit exists) The new telehealth system is intended as a trial, and runs alongside existing services The clinic owns the equipment Telehealth is upscaled and becomes part of everyday care The clinic leases equipment, as required individual consumers drive the development, use and acceptance of telehealth The product list is regularly updated to include new products, and reflect patient preferences The clinic does not have to keep redesigning its services in response to transient market trends Reason for vision: (i.e. Why the organizational unit exists like this) The clinic wishes to learn from the trial, with a view to wider deployments (longer term), seeking to reduce costs and improve care The clinic wishes to keep control of the services it provides Telehealth enhances existing service for 40 patients Processes (This refers to the work processes and working practices that are in use in the system. it includes the organisational structure and the ways in which the work is organised.) clinic nurses monitor the data, following up with patients when data falls outside established boundaries. nurses consult with doctors and consultants 'as and when' they see fit The processes are emergent and not standardised As before, but processes need to be formalised now that patient numbers have increased, to ensure consistent care recommended products need evaluating by clinicians and patients, to ensure compatibility with service delivery and that they meet user needs Processes will need continual review, to remedy unintended service implications (e.g. a product could lead to an unexpected spike in call-back requests) Technology (This refers to the technologies, tools and equipment used within the system, and can include both hardware and software.) The equipment comprises a single unit and allows patients to gather data on their blood pressure, glucose levels and heart rates same equipment as before, but patients now benefit from software/product upgrading, as it becomes available choice of a range of products to suit individual needs multiple technologies could create interoperability problems Infrastructure (This refers to the physical infrastructure of the system, for example including its buildings and the physical assets. But it can also include the financial infrastructure, such as the business model that is in use.) Patients need space to keep the equipment so that it is accessible for daily use The clinic needs facilities to transport the equipment Long-term financial provision must be in place, to enable the clinic to run these additional services no budget is needed for telehealth The clinic must create and maintain relationships, with suppliers to ensure they are aware of new product developments People (This refers to the people working in the system and also the key stakeholders (including customers) and includes their attitudes, behaviours, skills and competencies.) increased workload for clinicians who must cope with increased volumes of patient data, and respond to patients' needs Training is needed for patients (and their carers) in how to operate the system, the meaning of the data, and how and when to contact clinicians Training needed for nurses in how to deal with the data and how to deal with remote patients Workloads are unpredictable, because the number of telehealth patients could change suddenly staff numbers may need to change to reflect a sudden increase in demand for telehealth Training needed for clinicians in how to deal with increased volumes of data (resulting from the upscaled system). Training for patients/ cares/clinicians is also needed as and when technology upgrades occur impact on workload is unclear, but it is hoped it will be reduced by patients being better equipped to self-manage their conditions suppliers will train patients/carers in how to operate products, the meaning of the data, and how and when to contact clinical staff staff may need training to understand data outputs, if the format or measurements are unfamiliar Culture (This refers to the shared norms, beliefs and values that permeate the system. This can be local to the system and/ or shared more widely outside the system) The new system represents a shift in behaviours and attitudes for patients especially, giving them more responsibility for their own data and thereby their own care Telehealth relies on new norms being created (for staff and patients) because remote care is not the usual way of caring for patients The success of this scenario will depend on patients proactively embracing telehealth There is also a shift for clinical staff as patients take more control, some of whom become more demanding

System benefits (examples)
The existing outpatient services and the new telehealth service work in parallel many telehealth patients respond positively to the new service and take more control of their monitoring and health clinic and hospital visits are reduced and this has the potential to reduce outpatient waiting times All the benefits of the 'as is' , but in addition: Telehealth can now be offered to a greater number of people increasing patient uptake and usage will improve awareness of telehealth, which has the potential to help shift attitudes and create a culture more receptive to remote care Patients/carers can make individualised care choices that better suit their needs, thereby increasing usage and acceptance of telehealth some patients will be able to self-manage their condition, reducing demand for clinic services Any slips in patient care arising through system misalignment will be deemed unacceptable, and could damage patient and staff acceptance of the system same as the 'as is' , but also: The external partner relationship must be managed (ongoing) The clinic cannot cope with the increased demand for telehealth Telehealth products are not regulated by governing bodies in the same way that, for instance, new drugs are Patients/carers could reject telehealth altogether because it is not embedded in their care plan Health problems are missed, by patients/carers who are managing their own care, but are not medically trained (n = 30) members were then invited to attend the workshop, which took place over a half day. As before, the participants were then asked to collectively agree on criteria for evaluating the system's performance. The following were chosen: 1. Produces high quality outputs; 2. Delivers on-time; 3. Outputs meet the needs of customers; 4. Solutions require little re-work; 5. Group functions with low overall cost. Once these criteria had been agreed, the findings of the interviews (the 'as is') were presented back to the group. In this application, the structured templates that generated the 'as is' were visually presented (see Figure 2) to help participants see straightforwardly the systemic inter-relationships.
The analysis enabled identification of systemic incongruences. For instance, there was a lack of consensus about the appropriate goals of the group -should it be a body for decision-making, progress chasing, knowledge sharing, or priority setting? Or, a group with an evolving purpose? An incoherent vision for the purpose of the group, had led to a lack of clarity about how to evaluate its performance, and related to this, what the agenda for each meeting ought to look like. It emerged that the group perceived that they lacked authority to make change, in part because the group's nominal leader lacked seniority, and in part because the group had no budget attached to it to mobilise change. The SST process also made explicit the impact of cultural artefacts -for instance, it was noted that engineers usually aspired to innovate (create), so the group's mission to standardise (reduce) the range of tools used by the organisation, was at odds with that, and led to resistance amongst members.
With this analysis presented to all attendees, workshop participants reported feeling better informed about previously unidentified systemic interdependences, and rated the current system against the previously agreed performance criteria. Working in groups, participants then developed 'to be' scenarios to help improve the functionality of the group. Some groups focussed on new visions for the group. Other groups considered more incremental 'improvements' (e.g. the effect of inviting different stakeholders, or appointing a particular 'leader' to chair the group). In each case, the groups considered the impact of such design changes on the other stakeholders, parts of the system, and on the overall functionality of the group.
(5) Consider the 'to be' The third part of the workshop focused on developing new scenarios, aimed at improving the system's performance. This was undertaken in sub-groups (of 5-6 people), each comprising mixed skills, experience and backgrounds. In each case the new scenarios were developed using the same socio-technical templates as above.
Each sub-group generated at least 2 new scenarios and rated them against the same criteria as above. Each group presented their findings to a plenary session and this led to lively discussion.
A sample of new (to-be) scenarios are presented alongside the 'as is' , in Template 1.
(6) Make choices and agree action plan In the plenary session, the workshop selected 2 new scenarios (those described above) considered worthy of closer evaluation based on the evaluations undertaken in stage 5. Following more in depth consideration, they rated each again using the same criteria as above and the results are summarised in columns 3 and 4 of Template 2 (Table 3).

Example 2 -improving the effectiveness of a work team
The second example is intended to show how the SST can be applied to more acute, localised organisational problems. This application took place in a large manufacturing organisation, where key stakeholders were members of an inter-disciplinary 'working group' , comprised of sub-teams of engineers. Group members had a collective remit to improve process standardisation across the engineering disciplines, in order to reduce inefficiency and lower costs. The SST was deployed to help assess and improve the functionality of this group. The intention was not to replace the existing system, in favour of completely new 'to be' scenarios, but to analyse the 'as is' and more comprehensively consider the systemic implications of change, in order to identify a clearer vision for the future ('to be') within the existing system framework. The agreed outcomes of the workshop were 'some agreed ways to move forward, including recommendations for short-and medium-term actions' .
Prior to the workshop, a sample of group members, along with 3 additional 'stakeholders' (e.g. a finance representative) were interviewed (n = 17). All working group Table 3. Template 2. ratings of 'as is' and exemplar 'to be' scenarios.
Performance criteria (i.e. how well does it deliver … ?) 'As is' scenario 'To be' scenario 1 'To be' scenario 2 (1) Provides low overall cost 8 5 10 (2) copes with variation in demand for telehealth 1 6 10 (3) reduces number of hospital admissions 9 8 5 (4) meets patient needs 5 5 7 (5) Enables wide use of telehealth 3 9 6 Total score (out of 50): 26 33 38 the system, in order to improve appreciation of the systemic nature of core issues. It delivered tangible recommendations, and did so quickly, at a low cost, and with little organisational investment -for most stakeholders they were only required to attend a half day workshop. Over time a number of the actions were implemented. For instance, a further meeting was organised to cement the working group's vision, which was taken to higher authority for approval. A more formal leadership structure for the group was subsequently defined, and outputs relating explicitly to the work of this group were included in the annual objectives of the sub-group leaders, to encourage them to promote work group collaboration via their team members. Internal group processes were reviewed, and the organisation invested in further research to help them improve the effectiveness of their meetings, recognising that some of the issues emerging from the SST required further investment and exploration. In the first example, however, the remit for the SST was very different, and consequently, so were the outcomes. At one level, the SST was asked to deliver short-term site improvements. In particular, it led to Commissioners recognising the need to achieve 'pull' from frontline staff and patients; appointing some workshop attendees as 'Champions' of telehealth, recognising the influence of role models in shaping a culture of telehealth acceptance (see The MALT Study Consortium 2014). It also exposed some of the unintended systemic consequences of the current way of working. For instance, at one site it emerged that the intended benefit of reducing out-patient appointments Following these discussions, the group resumed to rate the alternative scenarios, and in plenary discussion agreed actions to move towards this.

Outcomes and evaluation
For these examples, the authors gathered outcome and evaluation data in several ways. For the telehealth work, 28 of the workshop attendees completed a short, openended evaluation questionnaire following their participation in the workshop, to help us better evaluate the usability of the SST, and participant experience. Feedback was also gathered at a telehealth dissemination event, hosted a year after the workshops, during which synthesised scenarios summarising the key SST findings were presented back to attendees. 2 years after the project end, evaluation interviews were undertaken with two subject matter experts working in public and private sector telehealth roles, to consider the extent to which they believed changes had occurred as a result of the SST workshops. For the second example, information about the functionality of the group was gathered informally over subsequent years, as we continued to work with the organisation.
Inevitably the outcomes of the SST will vary depending on the purpose for which the SST was used; so evaluation of the value and contribution of the SST must be measured against the extent to which it delivers what it sets out to achieve (Waterson et al. 2015). In Example 2, the SST was applied to help stakeholders holistically analyse event, we recorded preferences consistent with the previous scenario ratings, suggesting that the SST workshops helped to clarify consistent preferences for longer-term telehealth provision. The SST therefore helped stakeholders to explore and evaluate the longer term role that telehealth might play in UK health services, and consider the systemic implications related to different visions of the future. For telehealth, the SST never intended to realise all of these changes, as it was recognised from the outset that implementation would likely require policy and societal change. However, it formed a structured basis for the evaluation of new and novel future scenarios, enabling their competing advantages, disadvantages and risks to be transparently evaluated, alongside the systemic dependencies that each scenario revealed. In so doing, it helped to broaden the range of future scenarios that were being considered by stakeholders prior to this research.

What are the potential benefits of the SST?
Summarising the feedback from SST participants, and our own experience in working with this tool, this section outlines 5 inter-related benefits that we believe the SST offers to people engaged in work systems design.

Applies socio-technical systems thinking in practice
A primary benefit of the SST is that it encourages socio-technical thinking, the benefits of which have already been highlighted in this paper. In telehealth evaluation, asked what had been the most useful part of the SST experience one Commissioner commented: 'knowing now about socio-technical thinking will help me with future service provision … it has made me really think through different scenarios in terms of what they would look like' . Through the various templates, the SST encourages system designers to consider both the social and technical dimensions of a system, and the systemic implications that each choice has, so that no single part is over-emphasised or neglected.

Involves stakeholders in the process
A distinct, but related, advantage of the SST is that it is a tool that enables all stakeholders within a system to be involved in the process. The workshop evaluation data that we collected, revealed that participants consistently reported that hearing the experiences, challenges and views of other stakeholders had been both positive and through telehealth (the goal), was not being realised because clinicians were keeping the original appointment as well (process), in case a patient recorded their indicators incorrectly (culture and technology). Such modus operandi had typically led to increased workloads (people). It is well recognised that in complex work systems, such workarounds, improvisations and adaptations are associated with accidents and errors (e.g. Salmon et al. 2016;Dekker 2011;Clegg 2000). Socio-technical analysis of the 'as is' , through the SST enabled individual sites to recognise these dependencies and to address them.
The evaluation data we collected showed that there were also a number of broader, longer-term outcomes of the SST in this context. For instance: • It facilitated a conversation between different stakeholders. It was clear that many clinicians were unaware of what was technically possible, whilst technology providers were developing increasingly sophisticated new technologies that were not operationally viable, because they failed to recognise that implementation would require systemic change. Moreover, it helped make the values of different stakeholders visible, thus identifying points of conflict; and by helping the different stakeholders to recognise the points of conflict, it facilitate the process of change. As one participant noted: 'People don't evaluate telehealth using the same metrics, and often the metrics they do value are at odds with each other. The SST process helped us identify where our values pull apart, which helped us to identify what the dilemma is' . • In this example, the SST helped facilitate understanding of a wicked problem -'The SST is really useful where you're trying to understand what needs to be overcome. It's not problem solving, because it's not a problem or an answer to a single question' (workshop participant). In this example, the SST helped participants to identify and work through a variety of interconnected challenges. As another participant noted, 'it helped us work out what sort of scenario we want to be most like in the future … and in doing so, helped work out the direction of travel' . Analysing the extent to which the new proposed scenarios were likely to deliver the intended benefits enabled the groups to make informed decisions about which scenario to aim towards. • In addition, the SST led to some consensus on future preferences. Although the 8 SSTs generated a wide range of 'to be' scenarios, our analysis of these uncovered 4 overarching funding models for future direction (The MALT Study Consortium 2014). Presented to mixed stakeholder groups at the dissemination out before really thinking them through. ' (workshop participant). • Reduces some of the risks and errors associated with decision-making and change implementationsuch as overconfidence and tunnel vision -because it enables the organisation to examine multiple future scenarios (Meissner and Wulf 2013). • Provides a mechanism to help organisations rank scenarios, and consider the most beneficial solution to a problem. Even if the final solution is imperfect, the SST can provide a justification as to why this solution is the favoured option (or even why the original solution should be retained), through comparison with possible alternatives. As one participant put it: '[The SST] provided us with a useful insight into the risks for differing models and ways for operatingthe strengths and weaknesses of different futures is quite enlightening'.

Encourages innovation
The SST process encourages imagination and innovation. Sometimes, the possible alternatives to an existing scenario are not immediately obvious, and commonly organisations and/or the individual stakeholders come to an SST believing that the existing way of doing things is the 'only way' . In the telehealth example provided, participants found it initially very difficult to think of solutions outside existing norms (e.g. patients' self-monitoring data and remote care are not typical of UK health services). The SST process can encourage the development of more innovative solutions, particularly in situations where system parameters appear restrictive. The SST enabled stakeholders (including budget holders) to explore (without the risks of undertaking the changes in the real world), the service design implications of leasing (instead of purchasing) equipment, comparing the merits of each approach. Participants reported finding this useful -e.g. 'it's unearthed a range of knowns as well as new risks and issues; but in a "safe" environment'. Inevitably, some scenarios remain unviable, but a critical aspect of the SST process is that the range of scenarios is only narrowed once a thorough evaluation has taken place; no scenario is ruled out to start with. It is important that more detailed proposals are developed for several scenarios, even if instincts suggest they will not work.

Easy to use, versatile and low cost
The SST is easy to use -our experience is that users appreciate the structure and simplicity of the tool. The 'instructions' are simple, and can be run by individuals from a range of backgrounds; it does not require specialist knowledge of human factors or psychology. Above all, the tool requires facilitation surprising -'enlightening' , 'thought-provoking' and 'interesting' -and thus a clear strength of the SST. This is consistent with socio-technical systems theory which advocates that all of the stakeholders involved in a given system (e.g. end-users, managers, designers, human resource experts and clients) should be involved in the design, development and implementation processes associated with it (e.g. Clegg, Older Gray, and Waterson 2000;Clegg and Walsh 2004;Mumford 2006). The additional benefits of such involvement include: • Better understanding of the nuances of system design and operation (from multiple perspectives) (e.g. Mumford 1983); • Better designs and more effective systems (e.g. Clegg and Shepherd 2007); • Improved engagement and commitment (e.g. Tzortzopoulos et al. 2006).
Certainly, a number of methods that claim to utilise a 'user centred' approach, do not actively engage the users of the system. This is because some methods consider what users might like but without actually involving them (e.g. Kleiner 2006), whilst others engage a subset of users -for instance, by consulting end-users of a technology, but not wider stakeholder groups, whose roles and interests are also affected by the implementation of such technology (e.g. Grote et al. 2000).

Provides structure and organisation for discussion and decision-making
A third advantage of the SST, lies in the structure and organisation that it provides for facilitating discussion and decision-making. In particular, it: • Ensures that the major socio-technical design issues are discussed -e.g. will the cultural norms of the organisation help or hinder the success of a new scenario? • Helps prevent discussions from rambling and straying off topic. • Enables organisations to keep a rationale underpinning various design choices, which provides transparency in organisational decision-making and enables designers and stakeholders alike, to understand the logic behind, for instance, choices made in system design and improvement. • Identifies choices that were made inadvertently, or that have emerged over time, and then helps make these explicit. • Ensures that the existing and new systems are evaluated against the same criteria, enabling a balanced debate: 'We spent equal time evaluating each of the four scenarios -I would have ruled some of them Finally here, the tool is relatively low cost. The main costs concern the time needed to analyse and understand the existing work system, and the time invested by the workshop attendees (before, during and after the workshop).

What are the potential limitations and difficulties of the SST?
As with any tool or technique, there are limitations that need to be considered before deciding that it is appropriate to use the SST. Here, we reflect on these.

Getting the right people together
Since one of the core benefits of the SST is bringing key stakeholders together, its value is limited when the right people are not able to attend a workshop. We would agree that this coordination can be difficult, and in our experience three main problems can occur: (a) Representation In organisations staff are often busy and over-loaded. Getting their release to attend a workshop can be difficult, as is the coordination of multiple diaries. We note too that this does not just apply to managers -getting the release of front line staff (such as nurses in the earlier example) can also be problematic, in part because of the lack of 'spare capacity' .
Our experience is that attendance is less of a problem when there is buy-in from influential stakeholders at the outset. We recommend the following strategies: skills to ensure that the views of all the stakeholder groups are heard and included. The data produced by the SST is easy to work with, because the process through which information is yielded, enables system designers to organise this data into narratives that are easy to grasp and use.
The SST is also versatile in a number of ways: Unlike some of the other methods outlined previously, the structure of the SST is flexible. Templates can be elaborated in more detail to undertake additional focused design, for example, of particular job roles. For instance, in the telehealth example, the roles for each scenario could have been further developed and specified using the template shown in Table 4, to ensure that the role activities and implications associated with the new scenarios are made explicit. This can help identify job roles that become necessary or redundant through a particular scenario.
If required, the tool can be used to allocate individual tasks to the different individuals involved in the system; an example is given in Appendix 1.
The level of detail that the SST goes into is flexible in other ways too. For example, when using criteria against which each scenario will be evaluated, traditional techniques from the human factors domain can be used. Thus, for example, each criterion can be weighted to establish priorities amongst the criteria, and/ or the criteria can be compared against one another in a series of paired comparisons to establish a ranking of importance. Examples of each are given in Appendix 2. One major benefit of such prioritising is that it promotes a useful discussion amongst the stakeholders on priorities -making these explicit can be very helpful in system design.  (Armenakis, Harris, and Mossholder 1993), so can be useful for the sale or marketing of a product or service (Lam and Schaubroeck 2000). In the telehealth example above, feedback data from patients who attended, revealed that they had enjoyed participating in the process and felt 'listened to' in discussions about how the service could be improved, whilst staff members reported that it was 'useful to hear patient views too' (workshop participant), and broadened their awareness of the needs of other groups.
(c) Role of the workshop facilitator It is important to recognise the political role of the facilitator (Nadin, Waterson, and Parker 2001) who therefore needs to be seen as independent of any particular interest group. Where the facilitator is part of the management team or reports to another participant, this can compromise freedom of expression in the workshop. It is very important that terms of reference for the workshop are defined in advance and that participants feel able to contribute honestly. Where there is any concern about repercussions for such honesty we advocate that the 'as is' scenario is developed through pre-interviews, so that views can be anonymised prior to the workshop. It is also important that conflicts of interest are recognised and addressed explicitly before the workshop takes place to ensure that proposals for new scenarios are not simply developed because they are consistent with management policy, ideology or strategy. Ideally the facilitator should be completely independent of the system, and without a direct interest in it.

No guarantee of agreement or consensus
The running of an SST does not guarantee in itself that there will be agreement or consensus across stakeholders (e.g. in terms of priorities for change, or the rankings of scenarios). Whilst this can be a frustrating limitation when running an SST, it is unrealistic to expect that any tool can guarantee this. The SST is certainly better placed than some tools to deal with this dilemma because, as outlined in earlier sections, the tool provides a structured framework for discussions, and the templates provide a set of mechanisms for developing consensus (e.g. discussing objectives and priorities). This means that even where people do not agree, at least key issues can be made explicit, and the core issues debated.

Loss of momentum post-SST
A third key challenge for the SST is maintaining momentum after the event. Typically impetus and enthusiasm is generated prior to the SST, and during the workshop itself. However, the danger is that after the SST, momentum is lost and great ideas fail to turn into actions. In the telehealth • Gaining momentum before the event (e.g. by inviting them to be interviewed) can help generate stakeholder 'pull' . • Forewarning clients of the difficulties they will face in mobilising change if they do not involve stakeholders with organisational power; or working hard to ensure that stakeholders with decision-making authority are well represented at the meeting. • Engaging early on with those in authority (e.g. those responsible for strategy or who manage budgets) can encourage others who may not otherwise be interested in attending that the meeting will be strategically useful (for instance, connecting them to 'useful' others). This can also generate commitment and interest amongst stakeholders. • In the event of diary clashes, ask a stakeholder to invite a briefed nominee, or someone who shares the same role and/or challenges. Or, another technique is to collect data from such individuals in other ways (such as interviews), feeding this into the process at other stages. Although this is not ideal, it is better to have direct input from them at some stage of the process, than not at all.
Related to this is the issue of how many people can logistically be involved. Too many or too few representatives can limit the SST's usefulness. Where there is a legitimate need to gain representation, it is possible to repeat the SST process on multiple occasions, however, this can lead to data integration problems. If possible, a single SST should be held, and limited to a manageable number of participants (≤30 members, working in smaller sub-groups of 5-6 people).
(b) Exposing system weaknesses in front of other stakeholders Occasionally there is debate about whether it is appropriate to include particular stakeholders (e.g. 'customers' , hospital patients, or service users) in such workshops. It can be argued that conversations about service faults or problems should not take place in front of service users who may lose faith in a system that they rely on, or a brand they trust. This is a difficult tension. However, we would make the following points: Research from psychology shows that employees who feel involved, identify better with their organisations, and demonstrate both higher commitment to their organisation and greater willingness to accept organisational change (e.g. Vakola and Nikolaou 2005;Armenakis, Harris, and Mossholder 1993). Including all stakeholder groups creates a 'pull system' (Clegg and Walsh 2004) which is more likely to result in users increasing their commitment and interest in a service, thereby ensuring 'buy-in' . Moreover, where users of a system actively champion its service, they become far more of organisational systems is ongoing, so too should our understanding of socio-technical design be dynamic and open to challenge ' (pp. 173). To this end, we see a range of future opportunities for the SST which might include the development of new workspaces and the design of green buildings (e.g. where the success of green initiatives are dependent on people interacting with physical or technical systems in anticipated ways). Certainly, to date we have used the SST in a wide variety of organisations to support research, organisational development, and in consulting; on projects ranging in scope from improving sustainability, improving organisational resilience, enhancing knowledge sharing across organisations, and event management (specifically with a view to managing crowds).
In our experience the SST is particularly useful in the following circumstances: • Where you need to design (or re-design) a complete work system; • Where you need to design (or re-design) part of a system (e.g. introducing a new process or new roles), but need to consider their impact on the rest of the system; • Where you wish to make implicit 'choices' made about a system explicit; • Where you need to reduce the risks associated with being futuristic, by evaluating different design options for system change; • When you need to encourage 'blue sky' thinking, where radical solutions may be required to achieve a step-change in performance; • Where you need to work through the inter-related challenges of a wicked problem.
In addition, we believe there are a number of future development opportunities for the SST. We propose that it would be possible and useful to combine the use of the SST with tools such as computer modelling and simulation, to enable the testing of different alternative scenarios (see Hughes et al. 2012; for an introduction to these approaches). In the area of crowd management, for example, both the SST and simulation techniques have been used separately to this end (see Challenger and Clegg 2011;Robinson 2010a, 2010b). However, there is no reason why these two approaches could not be better integrated. This may serve to cross-validate solutions and competitively test different scenarios, reducing further the risk of implementing these in real life.
This paper is not advocating that the SST is a panacea. For instance, we would not argue that SST would be a suitable alternative to computer based simulations in matters of safety, where acute detail is especially important. Moreover, we make no claim that the SST is the only tool that can deliver utility in these circumstances. Our argument is that example, this could be seen at some of the sites engaged in the work, where it was clear during follow up, that little had changed as a direct result of the SST. In at least one instance, an attendee reported that although they had 'enjoyed' the workshop, it would not change their work because 'I'm not involved in planning' . Where the SST was most influential, and followed by financial commitment and/or substantive change, we observed two key differences. A working group with clear actions, goals and targets was formed before the workshop closed, with clear follow-up dates and deliverables. Another part of the solution was to ensure that those with influence and power are included in the SST to begin with, thereby reducing the need for subsequent negotiations and further redesign, for instance on grounds of cost or resource. In Example 2 those with authority were involved throughout, and their commitment to act on the SST outputs was clear from the outset. It is possible that the organisation's 'readiness to change' is a contributing factor in such instances (Armenakis, Harris, and Mossholder 1993). Waterson et al. (2015) have proposed a 7 criteria framework for evaluating the value of a tool of this kind, which considers the extent to which the tool:

Under what circumstances is the SST most useful?
(1) Examines aspects of work tasks; (2) Examines aspects of the work domain; (3) Represents individual, team and organisational concerns; (4) Examines aspects of the wider environment/ context; (5) The types of outcomes produced by the method; (6) Is useable and requires support to administer; (7) The robustness of the method in terms of validity and reliability.
We argue that through the cases provided in this paper it can be seen that the SST is able to deliver convincingly against 1-6, noting that given the variety in the types of problem and possible outcomes that it is used for, it may not be appropriate or possible to evaluate the SST against 7.
The SST is already being used to tackle an increasing range of problems and topics. For instance, it was initially developed by the authors to support product design systems (e.g. product engineering and IT system implementation), but is increasingly being applied to service design (e.g. the delivery of telehealth, and in designing effective supply chains). Davis et al. (2014) have argued that the application of socio-technical systems thinking needs to be broadened to new problems, so that it can contribute to new fields of organisational enquiry, arguing that: 'just as the design there are instances where it can inform and enhance the decision-making capacity of organisations beyond that of existing tools, or can provide a low-cost, high utility tool for organisational researchers and specialists in macroergonomics. The flexibility of the SST is a particular strength. It has a 'plug and play' quality, allowing the technique to dovetail with other methods, and make the most of available data in whatever form it takes. Combining the SST with other methods can create great value for both short-term work design solutions, as well as longer-range planning. For these reasons, the SST has a substantive contribution to make to the literature, and so must be added to the 'tool-box' of methods available to ergonomists to draw on.

Notes
1. Note that, to ensure ease of reading, some details of this worked example have been simplified or amended for illustrative purposes. 2. Stakeholder interviews were analysed using a priori template analysis (see King 2012) which focused on the 6 socio-technical nodes (Davis et al. 2014) as core 'themes' .