From the inside looking out: towards an ecosystem paradigm of third sector organizational performance measurement

ABSTRACT Effective organizational performance measurement supports the long-term viability of third sector organizations. According to extant literature, it has been driven by accountability, legitimacy and improvement. While some aspects of these are outward-facing, their focus seems to be intra-organizational. Our study shows evidence of a different approach to third sector organizational performance: qualitative data from the Scottish social care third sector suggest a paradigm shift on what drives and enables performance measurement in this sector. Moving away from an instrumental, compliance- and inward-focused approach, our findings point to an organizational performance measurement ecosystem marked by adaptation, sustainability strategies and inter-organizational collaborations.


Introduction
This paper explores a new paradigm of organizational performance measurement in the third sector, taking into account the inter-organizational connectivity and cooperation within and across the sector.Indeed, the third sector operates in an increasingly complex external environment characterized by multiple relationships with partners (LeRoux 2009; Macmillan and Paine 2021; Taylor and Taylor 2014) across public, private and non-profit sectors (Caperchione, Demirag, and Grossi 2017;Kattel and Mazzucato 2018;Kominis et al. 2021).This poses a need to understand how organizational performance measurement is defined and practised in each of these sectors.In the case of the third sector, the temptation has been to import theory and models from public sector and business research into the non-profit realm (Colbran et al. 2019;Greatbanks et al. 2010;Moura et al. 2020;Moxham 2014;Taylor and Taylor 2014).However, this is problematic as the third sector is markedly different in both substance and form (Boateng, Akamavi, and Ndoro 2016;Milbourne 2013; CONTACT Adina Dudau adina.dudau@glasgow.ac.ukMoxham 2009).Indeed, third sector organizations are underpinned by social missions that are secondary goals in other sectors (Corduneanu and Lebec 2020), they navigate the demands of uncertain funding structures (over which, unlike businesses, they rarely have agency), and are also public service contractors (Egdell and Dutton 2017).
The practice of contracting out public services has led to growing emphasis on competition, regulation, and performance frameworks (Bach-Mortensen and Montgomery 2019; Carmel and Harlock 2008;Milbourne and Cushman 2012).Within these, performance measurement methods merit particularly careful analysis (Colbran et al. 2021;Moura et al. 2020;Taylor and Taylor 2014).Specifically in social services, social care outsourcing needs to balance out cost efficiency and service quality and effectiveness (Bach-Mortensen and Barlow 2021).
Our study of social care third sector organizations (TSOs) in Scotland takes place at a time of change within a challenging policy and financial climate for the Scottish third sector (Law and Mooney 2012;Pearson and Watson 2018).In a commissioning environment characterized by austerity and budget cuts (Egdell and Dutton 2017), TSOs must negotiate the tension between their mission-driven purpose and the requirement to deliver commissioned quality-audited public services (Egdell and Dutton 2017).There is also another important wind of change coming from the proposed integration of health and social care into a National Care Service (National Care Service (Scotland) Bill 2022), with significant implications for the measuring of impact and performance in the regulated social care sector (Bach-Mortensen and Movsisyan 2021; Hendry et al. 2021;Pearson and Watson 2018).This makes our research particularly timely.
Effective third sector organizational performance measurement (TSOPM) provides stakeholders with critical information, from ensuring funders are receiving value for money to reassuring beneficiaries about the quality-of-service provision (Colbran et al. 2021;Taylor and Taylor 2014;Yang and Northcott 2019b).So, what determines effective TSOPM?The extant literature refers to three main drivers: accountability, legitimacy and improvement of efficiency and effectiveness (Davenport, Harris, and Morison 2010;Greiling 2010;Moxham 2014;Yang and Northcott 2019a).Yet these are not without their critics.First, while accountability processes should ensure that resources are allocated to worthwhile programmes, some organizations do not go beyond symbolic or ceremonial performance measurement, and reports of mismanagement of funds increase scepticism about third sector effectiveness (Carman 2010;Moxham 2014;Yang and Northcott 2019a).Second, some TSOs engage in performance measurement for the sole purpose of enhancing their legitimacy in the eyes of specific stakeholders (Greiling 2010;Moxham 2014;Thomson 2010) as part of the interaction with the environment and with stakeholders (Cairns et al. 2005;Coule 2015).Finally, the underlying assumption that measuring third sector performance leads to the improvement of efficiency and effectiveness is facing criticism in the light of evidence that this is not always or necessarily the case (Bach-Mortensen and Montgomery 2018; Lee 2020; Moxham 2014) as it is reported that neither funders nor the organizations they are funding are maximizing the benefits of gathering data for the purpose of learning and improvement (Taylor and Taylor 2014).Furthermore, meaningful tools to evaluate effectiveness are largely absent, with financial indicators dominating (Liket and Maas 2015) and reporting on efficiency lacking transparency (Hyndman and McConville 2016).
At a theoretical level, these three precursors of TSOPM take an inter-organizational, instrumental approach in an us versus them relationship: us within the third sector versus them beyond the sector.However, thinking has now evolved towards a more nuanced approach in many areas of public and not-for-profit research, which are starting to embrace stakeholder theory (Best et al. 2019;Chen et al. 2020), network theory (Hodgkinson et al. 2017) and systems ecosystem thinking (Dudau et al. 2023;Osborne et al. 2022;Petrescu 2019).These frameworks are yet to be applied to the study of performance measurement.Our paper contributes, in this respect, by revealing evidence of a paradigm shift in TSOPM: measuring organizational performance internally in relation to what peer organizations (rather than principals) do, in an outward-looking fashion and in close collaboration with peers, suggesting equality and interactivity present in ecosystems.
Our research aim has been to push forward our understanding of third sector organizational performance for their stakeholders.Our arguments are presented as follows: we start by laying out current perspectives on TSOPM, with a focus on its drivers.Then, we look at newer lenses through which to examine organizational performance -ecosystems theory -to counteract earlier dyadic approaches to TSOPM.We explain our methodology and our findings, identifying key features characterizing a new TSOPM paradigm.In the discussion section, we reflect on our findings in the light of the extant literature and conclude by highlighting the limitations of this study and opportunities for further research.

Third sector organizational performance measurement
The meanings associated with the concept of 'performance measurement' are contested and overlap partially with 'performance management' and 'quality management'.For our purposes a working definition of 'performance measurement' comes from Bititci et al. (2012), who describe it as a social phenomenon where behaviours (organizational and individual) are shaped by the values and perceptions of the individuals and the communities within which the individual [or the organisation] operates.(Bititci et al. 2012, 319) This is pertinent for the values-driven third sector operating in a complex multistakeholder environment, because it recognizes the social constructedness of the concept and it is inclusive of actors who are in a more horizontal relationship with TSOs and not just the government, donors and beneficiaries of TSOs' work.
Despite the complexity of relationships within the TS and between TSOs and various actors, much of the TSOPM research to date has assumed a dyadic relationship with governments, donors and foundations, as well as with the public in terms of sustaining public trust and confidence (Yang and Northcott 2019a).Within this body of research, we can observe a thematic clustering according to three overarching strands: accountability (to stakeholders, including funders, regulators and beneficiaries) (Chen, Dyball, and Harrison 2020;Cordery and Sinclair 2013;Moxham 2014); legitimacy (adopting management practices to ensure credibility in the eyes of influential stakeholders) (Beer and Micheli 2018;Mitchell and Berlan 2016), and improvement, how TSOs use performance data to improve performance (Carman 2010;Moxham 2010;Taylor and Taylor 2014).

Accountability
Accountability is associated with the need to give reasons for conduct to salient authorities (Jiao 2021) and it is recognized as a clear driver for measuring the performance of TSOs (Moxham 2014;Yang and Northcott 2018).TSOs are accountable to funders and regulators, to beneficiaries and to other TSOs (Cordery and Sinclair 2013;Liket and Maas 2015).TSOs respond to accountability demands by reporting performance measures to their users (Cordery and Sinclair 2013).In the face of increased accountability pressures, they look for ways to demonstrate their effectiveness (Lecy, Schmitz, and Swedlund 2012;Liket and Maas 2015;Wadongo and Abdel-Kader 2014), despite there being no accepted standard for measuring effectiveness (Colbran et al. 2019).Accountability has become a pressing concern for scholars and practitioners alike (Boateng et al.2016;Carman 2010;Colbran et al. 2021;Ebrahim 2010;Liket and Maas 2015;Yang and Northcott 2019b) because performance demands are often seen as a 'resource drain and distraction' (Carman and Fredericks 2008, 58).In the same vein, Christensen and Ebrahim (2006) lament the 'absurd' focus by funders on the measurable as a way of enhancing accountability.Despite increasing accountability focus, it seems challenging to find a common evaluation framework (Kendall and Knapp 2000;Kim et al. 2019) and the data collected in order to meet external accountability requirements often do not provide meaningful information for improving performance (Kim et al. 2019).However, the survival of TSOs is threatened if they do not abide by accountability expectations (Colbran et al. 2019).
Framed in this context, accountability can be perceived as making unreasonable demands on TSOs, which can lead to resistance to the concept of performance and thus present organizations with additional managerial challenges (Arvidson 2009).Scholars have argued for the need for new ways of expressing accountability and operationalizing it (Arvidson 2009;Moxham 2014), through structured informality, and active inquiry and by encouraging communication within and beyond organizations (Arvidson 2009).

Legitimacy
Legitimacy has been identified as another key driver of third sector performance measurement, alongside accountability (Chen et al. 2020;Dhanani and Connelly 2012;Herman and Renz 2008;MacIndoe and Barman 2012;Moxham 2014;Tucker 2010).Whilst all organizations have legitimacy concerns and are arguably under pressure to 'do good' as well as 'look good' (Dick and Coule 2020), research shows that TSOs adopt management techniques from the business sector to improve their reputation and to (re)gain legitimacy (Greiling 2010;Theuvsen 2004).Coule (2015) however, argues that the social mission of TSOS is in itself, a source of legitimacy in the eyes of its stakeholders.While the adoption of quality systems has been identified as a proactive step in anticipation of future pressures, taken to maintain credibility and legitimacy with funding bodies (Cairns et al. 2005;Greiling 2010), it is also argued that many TSOs do not go beyond the symbolic identification of performance goals and indicators (Thomson 2010).These pressures include pressure from funders and securing legitimacy with a variety of stakeholders, external and internal, including staff, board members, and national infrastructure bodies (Cairns et al. 2005).This may reflect part of a more complex reaction with an organization's interaction with its environment, and Cairns et al. (2005) find that organizations struggle to secure external legitimacy and credibility with their stakeholders and consider organizational adaptations to respond to these pressures.Whilst for some organizations, legitimacy is undoubtedly a key motivator in initially adopting performance and quality initiatives (Herman and Renz 2008;Tucker 2010), there is little to be gleaned from the research about the longer-term impact on organizational performance measurement of that institutionally-influenced decision making.

Improvement
Organizational improvement and learning represent the third key rationale for implementing performance measurement, as identified in the literature (Cairns et al. 2005;Liao et al. 2014;Moxham 2014).However, despite the potential use of performance data for improvement, there are few reports of this happening in practice (Carman 2010;Carman and Fredericks 2008;Ebrahim 2005;Lee 2020).For example, Carman (2010) reports that the strategy of performance measurement has 'failed to promote organizational learning within non-profit organizations' (Carman 2010, 270).Taylor and Taylor (2014) note that neither funders nor the organizations they are funding are maximizing the benefits of gathering data for the purpose of learning and improvement and that the gap between performance measurement and organizational improvement should be addressed through approaches that facilitate internal learning, rather than just the dominant accountability agenda.Moxham (2010) too, identifies a weak link between performance measurement and performance improvement in UK non-profit organizations.
The pressures for performance improvement come either from TSOs attempting to improve the service provided, or from funders working to ensure that organizations have the capacity to deliver services effectively (Cairns et al. 2005;LeRoux 2009).However, any assumption that measuring performance leads to improvement of efficiency and effectiveness appears to be misplaced.Hyndman and McConville (2016) posit that efficiency reporting lacks transparency and that measures of efficiency are rarely in place.Liket and Maas (2015) report that the 'elusive' concept of TSO effectiveness has left TSOs with a lack of meaningful tools to evaluate it, financial ratios remaining the main indicator for it.The need to develop performance measurement approaches that meet performance improvement aims is imperative and TSOs should be asking questions about the purpose of gathering data that do not support performance improvement (Kim et al. 2019).
Accountability, legitimacy and improvement underpin a dominant paradigm to TSOPM where TSOs measure and display their OPM to satisfy their principals (e.g.funders), who are often the ones dictating the performance proxies (Liket and Maas 2015) and at whom TSOs' legitimacy efforts are directed (Greiling 20102010).But is this dyadic and hierarchical approach fit for purpose in an inter-connected environment where organizations, and perhaps especially TSOs, work closely and form interdependencies with others, not only vertically, but also horizontally?Is the 'us' versus 'them' approach to organizational performance feasible in this environment?Stakeholder theory often offers a useful lens through which to speak to a wider array of stakeholders, but that comes with challenges we argue ecosystems theory overcomes and is therefore a more promising way forward in the debate over where TSOPM is heading.

Beyond 'us' versus 'them': the contribution of ecosystems theory
The relationship between organizations and stakeholders is often framed through stakeholder theory which posits that organizations are influenced by the preferences of the most salient stakeholders, when salience is a function of power, legitimacy, and urgency (Freeman 1984;Mitchell et al. 1997).However, stakeholder salience in the third sector is sensitive to managerial perspective (Shea and Hamilton 2015).
The concept of stakeholder salience is contested in complex organizational and inter-organizational configurations set up to tackle 'wicked problems' (Dentoni, Bitzer, and Schouten 2018) such as poverty or climate change (Cottafava and Corazza 2021).In such configurations, ideas and solutions may well come from the partners seen as least important.Beyond stakeholder salience, there is a lack of dynamism in stakeholder theory and stakeholder management which is a largely linear process.
Ecosystems theory may offer ways to overcome these challenges, and therefore offer useful lenses for interpreting and visualizing the contribution that stakeholders make to performance measurement in organizations which often work as part of networks on specific, ill-defined, and inherently complex social issues.Ecosystems have indeed seen a surge of interest from scholars of the public services (Dudau et al. 2023;Leite and Hodgkinson 2023;Osborne et al. 2021;Petrescu 2019), but also management (Granstrand and Holgersson 2020;Iansiti and Levien 2004;Teece 2014) and strategy scholars and practitioners (Adner 2016;Jacobides et al. 2018).That interest in ecosystems and in the intuitively appealing idea that the whole is more than the sum of its parts is not new.The term 'ecosystem' was coined in 1935 by Arthur Tansley and was welcomed by non-ecologists (Golley 1991).Jacobides et al. (2018Jacobides et al. ( , 2255) ) define strategic ecosystems as: . . .interacting organizations, enabled by modularity, not hierarchically managed, bound together by the nonredeployability of their collective investment elsewhere.
Later in that same text (Jacobides et al. 2018(Jacobides et al. , 2264) ) they elaborate, claiming that it involves: a set of actors with varying degrees of multilateral, nongeneric complementarities that are not fully hierarchically controlled.
The idea of multi-lateralism resonates with Adner's earlier text on ecosystems (Adner 2016), which mentions it as one of their defining characteristics, alongside alignment structure, set of partners and a focal value proposition.
As organizations tend to interact dynamically when inter-dependent in the provision of a service, ecosystems have also appealed to public service scholars who adopt Vargo and colleagues' definition of an ecosystem: relatively self-contained, self-adjusting system of resource-integrating actors connected by shared institutional arrangements and mutual value creation through service exchange (Vargo and Lusch 2016, 11) The definitions of Jacobides et al. ( 2018) and Vargo and Lusch (2016) suggest several characteristics: the members of ecosystems, although bound by some institutional arrangement, appear to be equal in each other's eyes, invest in a shared mission or goal, are dynamic through the adaptation of modular parts (e.g.organizations and inter-organizational sub-configurations), and appear to be in a state of relative equilibrium (or tend towards equilibrium), making the 'system' sustainable.
There seems to be a great deal of agency among the members of ecosystems (Jacobides et al. 2018), but if ecosystems are applied to the world of organizations, performance or quality issues are still salient.However, our intra-organizational mindset means we are more concerned with organizational performance than with that of the ecosystem.The closest we get to understanding organizational performance through the ecosystem lens is through the idea that performance is a 'shared fate' (Iansiti and Levien 2004), where the performance of individual organizations is closely related to that of their ecosystem (Jacobides et al. 2018).
Thinking around ecosystems has changed since Tansley (1935) (Pickett and Grove 2009).They no longer need to be persistent over long time periods to be sustainable; they may be tightly or loosely organized and may not need to be in equilibrium.Thinking about the governance of ecosystems and whether they are 'open' or 'closed' is also in a state of flux, as is our understanding of the role of ecosystem managers (Jacobides et al. 2018), or architects (Gulati et al. 2012).Organisations which may provide further stability to ecosystems by setting system-level goals and establish standards (Teece 2014) and other tools to motivate ecosystem members (Alexy et al. 2013).Arguably, these developments have taken place to accommodate an element only slightly touched upon by Tansley in his original text: humans as creators, enablers or disturbers of ecosystems, and with them, organizations.

Methodology
To address our exploratory research aim, a qualitative, interpretive approach was considered appropriate.This is in line with other studies in the field of TSOPM where the potential of qualitative research to shed light on real-life practices by exploring the 'how' and 'why' questions has been emphasized (Carnochan et al. 2014;Liao et al. 2014;Moxham and Boaden 2007;Tucker and Parker 2013).
The Scottish third sector environment has proven to be very fertile ground for research, as it offers a common social care regulatory and funding environment against which to interpret what is seen and heard during the research.The organizations approached in our study operate in the social care sector and are registered as charities with OSCR (Office of the Scottish Charity Regulator).We have classified them as medium, large, major or super-major service providers on the basis of their annual income (see Appendix).Capturing data from TSOs of different sizes ensures the representation of a variety of perspectives while also allowing for an exploration of commonalities between members of inter-organizational networks.Furthermore, our TSOs are part of a web of connections to other organizations, and researching the networks in which they operate enabled us to explore the means by which the norms of collaboration in relation to performance were taking place (MacIndoe and Barman 2012).Our sampling approach was purposive.We included registered TSOs with a minimum income of £1 million per annum, meaning they were categorized as large, major or super-major, providing social care services and regulated by the care regulator.As the focus of this study was on performance and quality, we judged smaller TSOs to be less likely to have dedicated resources to support formal performance and quality initiatives.
Our TSOs formed a multi-faceted yet homogenous group, varying not only in size, but also in type of service provision, including for example, disability and learning disability services, drug and alcohol dependency, mental health, services to children, and housing support.In total 23 individuals from 16 TSOs participated in interviews over the course of a year (see Appendix).Both semi-structured group and individual interviews were undertaken, alongside group observations which made it possible to triangulate data to ensure validity and reliability (Denzin and Lincoln 1998).Participants were all identified as experts in the field of TSOP, holding roles of Chief Executives, directors and quality and performance managers.It was anticipated that such roles would confer greater knowledge of performance requirements than dedicated service delivery staff.
Interviewees were contacted by email inviting them to take part in the research, briefly outlining its nature and purpose.Once they had agreed to participate, they received consent forms which gave further detail about the nature and purpose of the research.All interviews were carried out face to face and were digitally recorded and subsequently transcribed verbatim.Field notes were also taken from observations which accompanied the interview.Each interviewee was assured of anonymity and given the opportunity to withdraw at any time, thereby ensuring that participation was voluntary, as required by our research ethics committee.Interviews and observations were labelled for ease of reporting and to ensure anonymity.When identifying individual participants and groups within the presentation of the data in the subsequent findings, participants are given a number (P2, P3 and so on) and the group interviews are identified as either Group 1 or Group 2. Observational data is described as such.
The semi-structured interviews enabled interviewees to express their own views on organizational performance measurement, following three broad themes in which they were encouraged to explore the motivations and challenges presented by the use of performance measurement in their organizations: • identify the quality and performance measurement tools and approaches in the organization • reflect on key influences and challenges in adopting/maintaining those approaches • identify the key benefits of the approaches for key stakeholders Semi-structured interviews therefore enabled the interviewer to be guided by interviewees' perceptions of performance and quality measurement and followed a broadly inductive approach.The interview themes were informed by the issues raised in much of the key literature (Moxham 2009;Taylor and Taylor 2014), but also used follow-up and probing questions to clarify and dig deeper in some areas.Open-ended questions partly tailored to the interviewee and their organization secured further detail and explanation.
The focus groups were also comprised of quality and performance specialists who were also part of established, largely informal inter-organizational networks set up to support the development of knowledge and support around performance and quality issues pertinent to the third sector.Group members knew each other, having worked together and being linked in these partially overlapping networks (not unusual for organizations operating in the same regulatory environment in a small nation).In addition to the key themes raised above they were asked to discuss their perception of the importance of this professional network of performance and quality management professionals, its role in the third sector and the value the group has to the wider community of TSOs.Participants were highly motivated by their third sector distinctiveness and common approach to problem solving when it came to delivering on third sector performance.The table in the Appendix shows which participants took part in both the individual and the focus group interviews.Three field group meeting observations took place over the course of a year.In each network meeting there were between 23 and 30 participants from a wide range of social care TSOs in Scotland and the meetings were driven by the agenda of performance and quality issues in social care.They particularly focused on the ongoing regulatory changes necessitated by new care standards and the implications for TSOs measurement practices.Gathering data from these meetings through field notes supported the data collected from individual and group interviews.
The lead author conducted the interviews and focus groups, but both authors undertook data analysis (one on NVivo, the other manually), and met at regular intervals to discuss the codes and themes emerging from the analysis.Data analysis began in the early stages of data collection and followed the canons of thematic analysis (Boyatzis 1998;Gioia et al. 2012;Miles et al. 2013;Saldana 2013), summarized in Figure 1.
In total, over 50 first order codes were generated, and these were clustered into fewer, more abstract, themes (Miles et al. 2013) during the process of analysis, as depicted in Figure 1.

Findings: organisational performance measurement -from the inside looking out
While interviewees were not prompted to talk about the themes that occur in the literature of accountability, legitimacy, and improvement, and none of our interview questions were specifically around these, some expressed strong opposition to accountability and legitimacy as drivers for performance and quality concerns in TSOs: While it's reassuring, actually, that it is not so funder-driven as it used to be, that is always part of the story, and that is reality, they need the money . . .but what is quite reassuring is that a lot of organisations are focused on just 'we just want to do a really good job' . . . the process of getting funded organisations to report isn't just mechanistic accountability.(P10) The inference made by P10 (TSO director) in this case is that the reporting requirements imposed on them by funders leave little space to improve and do full justice to the work at hand.Indeed, the interview data suggest an underlying tension between a functional approach and accountability to a regulator: That's the thing with quality, it's trying to get something that is compliant and is useful.It is not easy bridging the gap, trying to create something people actually use.(P13) In this instance P13, a quality and performance specialist from a children and young people's TSO, was supportive of trying to find a common measurement framework to support both the funder and the TSO.But this is sometimes made difficult by the perception that the funder cannot fully appreciate the third sector approach, as P2 (quality and performance specialist from a TSO supporting people with learning disabilities) articulated: In this instance having measurement systems which are determined by funders' needs can lead to a perceived misrepresentation of the TSO's activity and achievements, which tend to be driven by softer, 'subtler' indicators than any measurement framework could encompass.Participants indicated that they were working at a deep level of understanding of the purpose and relevance of organizational performance measurement to the third sector.
In the children's area, you've got the regulation versus evaluation challenge (. ..)Are we talking about compliance with regulatory requirements, are we talking about delivering a good-quality service, are we talking about making a difference to people's lives, and are those three the same things?And on one level they are, and on one level they are not.(P10) By triangulating the debate here, this TSO director (P10) articulates an interesting set of challenges facing TSOPM: TSOs could be meeting the (implied minimum) standards required by the regulator (i.e.compliance), but this may fall short of enabling the TSO to pursue its mission.The TSOs seem willing to be part of the debate, keen to be transparent about their mission as well as meeting the upward accountability requirements of regulator and funder.
The perceived misrepresentation of TSOs in other sectors which might be using metrics to dig into the responsiveness and subtlety of TSO work with vulnerable clients has arguably led to a deep antipathy towards the need for certification and awards, which undermines legitimation as a driver for performance measurement: 'What do we want to be able to say to a funder, to an external, to a client, we are good at this because we do X, Z, Y? Or because . . .we've got a brownie badge?' (Group 1).This discussion among a networked group of quality and performance specialists in a range of TSOs suggests that quality of service provision held an overarching importance for the TSO, as opposed to any external recognition, and a tension between the two is evident, as a TSO Chief Executive reflects on here: it's not really about the badge or the plaque on the wall, for most of them, it's about 'we are improving as an organisation, let's celebrate that and let's get that feedback from the team of assessors that we can use to drive forward.(P12) Indeed, it seems that the antipathy towards accountability and legitimacy does not extend to improvement.The benefits of a focus on organizational improvement appear to be overtaking legitimacy as a dominant motivating force, refocusing attention on organizational purpose.

A performance measurement ecosystem?
Participants articulated ways in which organizational performance measurement was perceived in their respective organizations.A TSO Director explained that they considered performance measurement not only as the tools through which it is enacted, but also as the way an organization worked (e.g.culture): It is not just about evaluation skills, and IT systems and having the right measurement in place, it is also about how an evaluative culture is embedded in the way staff are managed, and inducted, and you know, how they think about reflective practice.(P10) By 'evaluative culture', an intra-organizational culture was inferred, one which supported learning and improvement based on performance information, and in which all stakeholders are actively involved.But beyond organizational boundaries, interviewees spoke of 'performance in all directions' (P10) to comprise all aspects of its 'ecosystem' (i.e. the broader system in which it operated): It's like performance in all directions . . . it is also the culture and the way in which the organisation works.And then there is a system . . .within an organisation and it is one of the great privileges . . .that you get into an organisation, and you get to see the 'system' as in 'ecosystem'. (P10) The concepts introduced by this participant show how performance is perceived as being synergetic and relational.This ecosystem is seen to be more than just a whole system of performance measurement tools, it is dynamic:'[quality and performance] is about, you know, the whole living and breathing thing . . .and I think there has certainly been a huge cultural change from . . .compliance to collaborations' (Group 1).
Indeed, performance appears to be seen as evolving, as an (evaluative) cultural change impacts on external organizations through inter-organizational work and networking.This pushes performance measurement beyond the realm of predictability, or what can be measured and controlled, and towards inter-relationships which are 'difficult to measure'.but things like 'how effective are our services in delivering to children and young people?' is significantly more of a challenge, and the danger is that you lose the numbers that matter the most because they are more difficult to measure.(P1) The Chief Executive interviewed here returns to the conundrum for TSOs in which 'easier' measures, typically associated with compliance, fail to support the delivery of the organizational mission.Participants further drew on the level of synergy required between people and the processes, which takes performance measurement further away from the realm of predictability, often inferred in quantifiable frameworks, and towards an art form: [performance in] social care is a bit of an art form, really . . .it's not a science!It's about people, isn't it?It's about interactions between people, so that is very hard to write down as a standard.(P2) This synergistic approach seems to entail an evolving, interactive and relational perception of performance measurement, which appears to be compatible with the perception of a 'performance ecosystem' which emerged in later interviews.These concepts were supported further by the evidence from interviews as well as from field observations.Indeed, the aggregate theme of an organizational performance measurement ecosystem appears to be characterized by several dimensions: adaptation, sustainability and collaboration (see Figure 1) which we illustrate next, as they emerged from our fieldwork.

Adaptation
The idea that TSOPM is adaptive came primarily from observations during group meetings.These took place over the course of a year while significant changes were taking place nationally in health and social care.It was therefore imperative that regulated organizations had an opportunity to understand how the forthcoming shifts in regulation, inspection and standards (as well as legislation) would impact on their organizations' performance, so they started to meet regularly and we observed some of these meetings.
In the group observation meetings there was evidence of learning and adaptation both from the TSOs and from the care regulator, as they engaged in consultation.Proposed changes planned by the regulator included a reduced number of inspections, with more reliance on individual organizations to assess service quality and an increased emphasis on self-assessment by the service provider.It was also evident that many of these changes were in fact driven by the TSOs themselves, such as a proposed change to the point scale used for inspections.Therefore, adaptation appeared to occur in both directions, which indicates a transformational way of working for the regulator.Adaptation within TSOs took place at both strategic and operational levels.
A participant in a group interview of quality and performance specialists (Group 2) explained how they had embedded aspects of a known, standard quality model into their bespoke system: EFQM?Well, I suppose we see it as a kind of model, I suppose we have taken, I think what we've taken from it primarily is RADAR . . .and sort of taken that and applied that within the organisation, the quality framework is, kind of, follows the RADAR process.(Group 2) This is a representative example of a TSO reporting how they make adaptations within their internal performance measurement approaches, modifying and tailoring existing models to fit their operational model and third sector mission, therefore ensuring a strategic and operational fit.EFQM is the European Foundation for Quality Management Model and RADAR (Results, Approaches, Deploy, Assess and Refine) stands for the logic underpinning the EFQM diagnostic tool for organizations.
Finally, a quality and performance specialist explained how they developed and subsequently presented their bespoke approaches to the government, showing a high level of confidence as a service provider interacting with a high-power stakeholder: We've done lots of things over the years, we've built our own outcomes framework tool, rather than using some of the government-based ones, we had an open day last year and some of the Scottish Government people came . . .we got some credit for that.(P13) If the above describes something that could be referred to as 'vertical' adaptations (topdown and bottom-up), the data also unravelled more 'horizontal' adjustments.
'That was another thing, as you start to talk to other organisations about policies and procedures, they are very similar, and you start to think 'why can we not just have one that we can share and just adapt slightly?Because we are all writing different things for the same purpose' (Group 2) There was evidence of how organizations had developed adaptive approaches through their inter-organizational interactions (TSO to TSO), which led to similar policies and procedures.
Participants' understanding of adaptability appeared to also involve taking the best from what was in existence and using it flexibly.A quality and performance specialist in a TSO supporting people with a disability mentioned having access to a 'toolbox' where approaches were assessed and deployed as appropriate: It is really encouraging other members of staff, right across the organisation to keep their eyes out and maybe pick up things that they think are just minor, somebody has used it somewhere in the past and it works very well with that, oh that's fine that's great, let's evaluate that, to see if it could work for some of our service users, because we don't want a one size fits all, because, it doesn 't!. (P6) This anticipatory response to seeking innovations for assessment and selection was not untypical amongst TSOs.Sourcing potential innovations for adapting and integrating was time-consuming and required a high level of organizational commitment at a strategic level.A quality and performance specialist for a social care provider described for example: 'We were quite ahead of the game a few years ago where we developed a whole outcomes framework for different care groups or different client groups and we kind of persevered with that for quite some time'.(P7) Their organization had found itself increasingly needing to meet what it considered to be onerous demands from the commissioning partners, whilst also ensuring that the development would work for the organization's needs, and that led to internal tensions.
Adaptability in building or developing approaches to performance measurement was taking place due to changes in the environment (e.g.legislation) or in new service developments or by gaining access to shared learning.This was done with sustainability (with new environment characteristics) in mind.

Sustainability
It was evident from our respondents that performance measurement had had a strategic role within their organizations for as long as they had worked there.This is to some extent inevitable, given the long timelines for social outcomes in much third sector work.As a quality and performance specialist reflected: When I joined the organisation, kind of found my feet-ish, (you never find your feet properly), but I was thinking, God this is a 10-15 year job!You know I am under no illusions, there are no short cuts, there are no quick fixes.(P2) This shows an emphasis on both a strategic and operational responsibility to engage with performance throughout the lifetime of the organization.Although organizations were making adaptations and improvements to the approaches they were developing or had in place, at no point did anyone doubt that performance measurement was integral to the organization's long-term existence or suggest that they might abandon their respective initiatives.During a group interview, a manager reported: That sort of strategic commitment [to performance] is there, it is not seen as a kind of thing which sits apart in an organisation like a sort of process that has to be gone through, it is recognised as the only way that the organisation is going to survive and I don't mean to be over-dramatic . . .but if we are not going to be competitive on cost, then there is no other way.(Group 2) The integration of performance and strategy was therefore perceived to be a nonnegotiable element in relation to organizational success.Sustainability was shown to exist not just in organizations' approach to developing performance measurement, but also concurrently in supporting networks.The network referred to has been in existence for over 20 years, showing a symbiosis between the intra-organizational and the inter-organizational: 'the [network] has always been really strong for a really long time . . . .There is that kind of mass of TSOs now that have these dialogues with each other, which is incredibly helpful' (P11).
The synergistic relationships between network members in a relatively stable group was seen as an underpinning factor which contributes to the network's success.
Despite an understanding that organizational performance was part of the longterm strategic direction for TSOs, the lack of long-term funding was considered a significant obstacle to developing sustainable performance measurement initiatives, as a Chief Executive reflected: It is sometimes easier to convince ourselves and we look at some piece of work and say that's really good, really ground-breaking, but I think . . .when you've been around a long time and you've seen a lot of initiatives and things and the real thing, I would argue, that most charities are trying to achieve is system change and just because you are doing something interesting and you get funding for 3 or 5 years doesn't necessarily change the system.(P1) While TSOs have a vision of contributing to wider societal improvement, they seem to recognize that the structures in place are hindering advances in those areas.Given these obstacles, interviewees inferred a need to become comfortable with uncertainty in their work.As a quality and performance specialist from a major social care provider reflected: It is a time of change, and we are reflecting on all our policies and procedures and what is going on, and I don't know the answer to where things are going, so it could be very different in a year's time.(P8) This lack of strategic certainty can be both destabilizing for ensuring core service delivery and disruptive for setting up approaches to measuring performance.Indeed, another quality and performance specialist recognized that, to ensure continuous improvement over the long term, there should be a consistent approach to quality: You see, I don't think you do something at the end of the day, or once a month, it is perpetual, and it is that understanding that quality is perpetual and every transaction, every conversation . . . in every action there should be something to do with being effective and efficient.(P2) TSOs therefore identified the need for an indelible integration of the core principles of measurement, despite system hindrances.A further challenge to building sustainable performance measurement systems was reflected in this comment from a Chief Executive who recognized that, at times, it was difficult to ensure that it was an issue for the whole organization, potentially due to a lack of skills or long-term vision: That's the bit that worries me, are we building a sustainable system here?Or are we taking some very good individuals and putting them together and they are doing good stuff but it's not actually impacting on the system?(P1) With the overriding impetus being broad long-term societal influence, we must ask the question, is the internal TSOPM integrated into the wider ecosystem?This brings us to the third theme we extracted from our data: collaboration.

Collaboration
Observations during the group meetings revealed ongoing networking and collaboration between the two different quality forums.TSOPM was perceived as an inherently collaborative activity, going beyond the organization and even beyond the networks of which they were members.A Chief Executive of a super-major organization reflected on how the networks were able to offer mutual support in the context of an environment which showed signs of institutional failings within both the regulation and commissioning of services.Participants reported that commissioners were attempting to drive down costs, with short-term funding contracts dominating the market.There was no evidence of this landscape being directly challenged, rather an acceptance of the benefits of TSOs working together in order to navigate the challenges of delivering performance measurement.
I think the advantage of networks within a closed and reasonably safe group is to say, okay, well if we collectively are serious about this how can we use each other to progress and all get better?(P1).
Despite working together in a competitive tendering environment, members of this group were operating as an informal knowledge exchange network.Working collaboratively was therefore identified as a strength for the third sector.A quality and performance specialist reflected: I think . . .you will have gathered from [partner organisation] that we do tend to network quite a lot, so we do share a lot of ideas between ourselves, and wegravitate towards people who we think come from the same place as us (P2).
Network members therefore reported sharing a mutuality of interests to meet their common goals.Interpersonal relationships, despite their intangibility, were a key factor in the success of these collaborations.Another quality and performance specialist in a disability charity supporting both adults and children explained that a key benefit of being a member of the network was that it supported her personal learning and development.That individual learning was subsequently being brought back to her organization.
When I first started Recognised for Excellence, submissions were really, really useful.And that was really, really helpful getting to know what people were putting in, how they were doing things, how they were designing things, looking at the systems that they use within the organisation, some of the talks that people will do after they've done one of the awards, it's really interesting, and it's how they're engaging with people and how they're collecting and analysing the results, has been quite interesting and quite enlightening.Because, as I say, I didn't come from a background where I had a lot of knowledge of the theory behind [TSOPM] and I think my view was a bit simplistic.(P5).
Interpersonal factors were therefore contributing to the knowledge deficit reported here.At an observation of a group meeting a comment was made by an attendee that 'the network stretches beyond the group meeting' (Group 1).Indeed, although there were typically four 'formal' meetings in a year, there was considerable ongoing networking taking place in the background, including mentoring, informal meetings, peer support and one-to-one support meetings.'They just see it as this is the way that we do it now' explained a development manager (P11).Collaboration was a normative way of working within the sector, used by the participants to support their individual and organizational learning and therefore contribute to the long-term future of their performance measurement initiatives.
'To be honest the third sector is right up there with the success stories to tell and to share and they are very open and willing to share that with others as well '. (P11) This suggests a level of attitudinal trust and cooperation amongst members who perceive the third sector to be more effective at delivering good-quality service.Overall then, there was a shared understanding between participants that, particularly in comparison with other sectors, the third sector valued its collaborative activity highly.In summary, our findings show elements emerging under the aggregate theme of OPM as an ecosystem by demonstrating adaptive approaches to developing TSOPM, adopting collaborative approaches by working together to support shared learning and development and advancing sustainable, long-term solutions to embedding TSOPM into their organizational lifecycles.

Discussion
While much of the literature reflects an instrumental and dyadic approach to performance measurement in TSOs (LeRoux 2009;LeRoux and Wright 2010;Moxham 2014), where TSOPM serves to communicate with funders or regulators, our empirical evidence indicates that the paradigm of performance measurement in the current third sector environment may be shifting towards a relational paradigm.In our study, organizational performance measurement appeared to be an integrated, collaborative, and evolving concept, not imposed on TSOs but, rather, organically expanded from a concern with the wider organizational network.Our analysis revealed that this process happened through adaptation, collaboration, and sustainability (see Figure 2), alluding to the characteristics of ecosystems in Jacobides et al. ( 2018) and Vargo and Lusch's (2016) definitions discussed earlier in this paper (see Table 1).
Figure 2 illustrates the process through which these mechanisms enable TSOP: through vertical and horizontal adaptation (to the regulator and to partner organizations), which is underpinned and supported by collaboration (e.g. through networks such as those in which the Scottish social care TSOs operate) and which takes place, and cements, over time.
In terms of adaptation, our data offered evidence that participants adapted experiences, learning and methods to embed approaches most appropriate to their organizations' internal systems.On the one hand, adaptation was prompted by changes in the external (policy) environment or new developments in models and frameworks.This was evidenced through TSOs' proactive and reactive alterations to accommodate changes in legislation and in the inspection regime, as well as using a range of standard quality models (even if they came from another sector), by tailoring them to suit the unique third sector culture.We have called this adaptation strategy 'vertical adaptation'.One the other hand, we also observed 'horizontal adaptation' between and across TSOs working in the same field, keeping an eye out for good practice, soliciting practice templates, innovating, and disseminating to peers (and ultimately to the regulator, suggesting that the two forms of adaptation may be circular -as shown in Figure 2).Our findings on adaptation resonate with those of Rees et al. (2022), who drew on adaptive organizational theories to improve our understanding of TSOs adapting to environmental norms to be 'strategically resisting those who judge their compliance' (Rees et al. 2022, 8).But we are adding nuance by exemplifying adaptation types and manifestations, as well as describing the bigger picture of a performance (TSOP) ecosystem to which they contribute.Our findings also put into perspective Choi and Woo's (2021) study of performance information use in the public sector in which they found coercive, rather than mimetic pressures to be persuasive drivers for public sector performance.Our study of TSOs reveals the opposite: we heard nothing at all about coercive pressures in the answers received to our open-ended interview  questions, and more about the mimetic dynamics, which we have labelled horizontal and vertical 'adaptations', driving TSOPM.The co-existence of vertical and horizontal adaptations of TSOPM brings to the fore the importance of collaboration, the second theme we derived from our study of mechanisms enabling TSOPM ecosystems.The development, understanding and implementation of collaboration supported TSOPM between stakeholders, both interorganizationally as well as intra-organizationally, and was a critical underpinning element of TSOPM.This complements the view, in an underdeveloped field, that the concepts of measurement and inter-organizational collaboration are mutually reinforcing, as TSOs showed commitment to navigating towards making a collective impact on communities and society (Yang 2021).The theme of collaboration may have emerged partly due to organizations in our study being known to each other and operating in the same field and within the same regulatory environment.This suggests that the context, e.g. the policy area in which TSOs operate (in our case, Scottish social care), may be the frame within which the TSOPM ecosystem develops through adaptation, collaboration, and sustainability.
Indeed, these adaptations and collaborations appeared to have taken place over time, which brings us to the final theme derived from our data: sustainability.At the heart of this theme lie the long-term outcomes of the work TSOs usually deliver (Moxham 2014), which means TSOPM takes place over a longer timescale than might be the case for other sectors.While TSO work outcomes are long-term, funding tends to be very much short-term, as we heard from interviewees, and that is consistent with a whole strand of literature (e.g.Moxham and Boaden 2007).This is a barrier to sustainability, and therefore impacts on the development of performance measurement.Indeed, TSOs have long been challenged to find a balance between their social and fiscal viability; strategic choices for sustainability exist and TSOs have been committed to navigating a route towards sustainable strategies (McDonald et al. 2015;Weerawardena et al. 2010).What we add to the body of work on TSO sustainability is the relational view of TSOPM sustainability; that is, the symbiosis between intra-organizational and inter-organizational work, which needs to be reflected in similar ties for TSOPM.
Our research suggests that rather than awaiting step-by-step guidance from the care regulator and from service commissioners, TSOs have risen to the challenge of addressing OPM by aligning their practices with those of members of their ecosystem in ways that both serve society and stay true to TSO language and culture.By viewing performance measurement as part of an ecosystem, they evidenced their adaptations through collaboration and confirmed their commitment to a long-term survival strategy, as well as shared long-term performance outcomes.While ecosystems theory was under-developed in organizational, management and public management scholarship when we started this research in 2018, Millar and Hall's (2013) findings that performance measurement in social enterprises is context-bound and ought to resist off-the-shelf instruments (such as a 'social return on investment') are similar to our findings on the TSOPM ecosystem.We take Millar and Hall's assertions forward by allowing the ecosystems framing to enable our observation of specific dynamics around collaboration and adaptation, seeking sustainability.
With this study, we contribute primarily to performance measurement theory and practice in the third sector by moving the conversation away from the more dyadic, us versus them approach to TSOPM (suggested by the focus on accountability, legitimacy and improvement), and towards adaptive, collaborative and sustainable practices through which performance measurement happens within organizational boundaries (from the inside) but reaching out to the rest of the ecosystem (looking out).Our findings also contribute to wider conversations on public service ecosystems (e.g.Dudau et al. 2019;Osborne et al. 2022, Leite andHodgkinson, 2023;Petrescu 2019).In Osborne et al.'s (2022) ecosystem framework, TSOs are ecosystem actors at a meso-level and, in different ecosystem configurations, can arguably contribute to 'macro-level' public service ecosystems, i.e. to social values, rules and norms.Our findings enhance the analytic generalizability (Yin 2013) of the ecosystem framework to the third sector and to the concept of organizational performance, which can open the door to new knowledge about organizational performance as a key aspect of public service management.

Conclusion
The aim of this study has been to push forward our understanding of third sector organizational performance measurement beyond dyadic approaches (organizations versus regulator).Our data from social care TSOs in Scotland suggest a paradigmatic shift: organizational performance measurement is seen as relational, considering how other members of the ecosystem measure performance, as well as their collective performance on the social issues on which they all focus.These organizations take an embedded, evolved, sustained approach to organizational performance measurement, away from compliance and towards a collaborative future.In a knowledge-deprived field, TSO practitioners are turning to informal inter-organizational networks to gain intra-organizational knowledge and more input is needed from infrastructure organizations to ensure they are formally supported to meet both funders' and regulators' performance requirements.
The study's limitation is that the sample of participating TSOs was of medium, large, major, or super-major organizations.These are all mature organizations with secure financial resources and constitute a small proportion of the overall sector.However, there was a good reason for selecting these organizations rather than a more diverse sample: they are all from the same sub-sector (social care), regulated by the Care Inspectorate, and operating widely in collaborative networks.Although this can be considered a strength of our research design, the experiences of our TSOs may not be comparable to those of smaller TSOs, TSOs operating in other sub-sectors, or those in different regulatory environments.
Future research in this area could explore the extent to which the aggregated theoretical construct of the ecosystem is valid for TSOs in other respects, in addition to performance measurement, as well as in policy areas other than social care.It could investigate whether the type of focal issue surrounding an inter-organizational ecosystem determines the dynamics, or even the very existence, of an ecosystem paradigm.The role of the keystone organization, of the ecosystem lead, could also be explored further to understand its salience in the sustainability of the ecosystem.Finally, the analytic generalizability of the ecosystem perspective can be extended in future studies of performance measurement in both public and non-profit contexts.
. . . in the current economic climate I think the qualitative things are becoming less important and less heard, but they are, because what people, what funders are wanting, and I'm not saying it's wrong, is . . . . the way funders are set up, they are not set up to appreciate . . .subtlety.(P2)

Figure 1 .
Figure 1.Aggregate theoretical dimensions and a sample of the quotes coded into these.

Figure 2 .
Figure 2. Performance measurement through adaptation, collaboration and sustainability.

Table 1 .
The three mechanisms enabling TSOP ecosystems.
Our findings of TSOPM Key ecosystem characteristics (Jacobides et al. 2018; Vargo and Lusch 2016) Adaptation dynamism through adaptation of modular parts Collaboration investment in a shared mission or goal Sustainability relative equilibrium making the system sustainable