Effectiveness of technology transfer in public research institutions in South Africa: A critical review of national indicators and implications for future measurement

South Africa’s poor economic performance in recent years has prompted calls by policymakers for structural transformation. To break the impasse, a stronger focus on innovation is one strategy the country has articulated through a new White Paper on Science, Technology and Innovation. Writing from this policy context, we adapt the revised contingent effectiveness model of technology transfer by including indigenous knowledge as a transfer object and emphasizes citizen needs and reparations as part of the demand environment, and appropriation as a transfer medium. Analytically, our use of this adaptation is to critically reflect on the typology of indicators produced in the first South African national survey of intellectual property and technology transfer at publicly financed research institutions. We find the dimension of least representation but presumably greatest significance in the typology is that of ‘public value’. Our contention is that the output-based and commercially-biased indicators of technology transfer activity, which predominate in the typology, are insufficient to inform decision-making on technology transfer policy in a context of profound national socio-economic challenges and deep historical legacies of indigenous knowledge misappropriation. Broader evaluation data would form a richer, more inclusive evidence-base to inform new investments, as well as ongoing policy assessment, at both institutional and national level.


Introduction
More than two decades after its democratic transition from colonial and apartheid rule, South Africa confronts at least four major socio-economic challengespoverty, unemployment, inequality and weak GDP growthwith persistent negative impacts on social cohesion and wellbeing (Soudien, Reddy, and Woolard 2018). Structural transformation has been posited as a route out of the impasse, with foreign-direct investment and innovation understood as among the key catalysts for growth (Department of Science and Technology 2019; Ramaphosa 2019). How effective is technology transfer at South African publicly financed research institutions, including higher education institutions (HEI) and science councils, in confronting these broad societal challenges? Under increasing pressure to demonstrate effectiveness of research policies and programmes to their constituencies, including taxpaying citizens, this question faces policymakers at both institutional and national levels. This paper contributes to the policymaker's analytical toolkit in two ways: first, it reflects on the South African contextual and legislative specificities in which the practice of technology transfer is carried out; second, it develops and critically assesses a typology of indicators used to assess technology transfer effectiveness. This paper ends with suggestions for formulation of a wider conception of effectiveness than currently used in South African research policy discussions.

South African specificities
Inequalities across society and within public research institutions Prior to the democratic era, which started with the first elections in 1994, the country's highly unequal socioeconomic class, gender and spatial structure was strongly associated with the racial categories spawned in earlier colonial discourses, and then intensively legislated and policed during apartheid. Much of government policy in the democratic era has been to undo and redress these intersectional and racially-encoded inequalities, for example, in the composition of business ownership through, predominantly, procurement, employment, and tax policy; as well as across the breadth of government departments, universities, and within civil society. Up until the time of writing in early 2020, however, the nature of South African inequality has remained largely untransformed, with many studies providing compelling evidence of stasis (for example, see Levy 2019 for qualitative data, and Stats SA 2019 for quantitative data).
Among the legacies of its colonial and apartheid history are, on the one hand, a set of well-resourced, research-intensive universities ('historically white and privileged universities') or technikons (now 'universities of technology'), located predominantly in the wealthy and populous Western Cape, KwaZulu-Natal, Free State, and Gauteng provinces; and, on the other hand, a set of less-resourced institutions ('historically black and disadvantaged universities') in the poorer, lesser populated provinces (Eastern Cape, Limpopo) (Strydom 2016). Many of the latter were established by the architects of apartheid to serve impoverished 'homelands' or 'Bantustans' with the implicit and explicit aim of producing a primarily cheap labour supply for the South African economy. Recent events within the universities sector, including the '#rhodesmustfall' and '#feesmustfall' movement (Habib 2019), have served to highlight the exclusionary nature of these privileged institutions, and the need for reform at all levels, both in terms of curriculum and leadership, as well as symbolically.
Legislation, policy goals and policy learning Nested within these institutions is a set of fairly new technology transfer organizations (TTO), a number of which have more elaborate track records within the largely historically white and privileged institutions, while others are less than a decade old (Mustapha et al. 2017). These institutions were established within the context of legislation that came into effect on 2 August 2010the Intellectual Property Rights from Publicly Financed Research and Development (Act No. 51, 2008) (IPR-PFRD) (The Government of the Republic of South Africa 2008). The IPR-PFRD Act has as its objective to: … make provision that intellectual property emanating from publicly financed research and development is identified, protected, utilised and commercialised for the benefit of the people of the Republic, whether it be for a social, economic, military or any other benefit.
The Act further specifies the requirements for realization of this objective. First, publicly financed research institutions are required to protect the IP stemming from their institutions and make it available to the public. They are to record and report on the benefits of their research to society and look for opportunities to commercialize the IP. Second, the 'people of the Republic' are to be granted preferential access to the benefits of publicly generated IP. In particular, the Act mentions small and black-owned businesses in this regard. Third, the Act specifies that the inventors of the IP generated must not only be acknowledged, but also rewarded. Furthermore, researchers may publish their findings after the evaluation of a disclosure. Fourth, and finally, according to the Act, the state may use the IP in the public good, if it deems this necessary.
From 2010 onwards, the (then) Department of Science and Technology began to put the provisions of the IPR-PFRD Act, No. 51 of 2008 into operation. An institutional structure, the National Intellectual Property Management Office (NIPMO), was located within the Department (Government of South Africa 2008).
In the late 1990s and early 2000s, several problems were identified in the technology transfer structures in South African publicly financed research institutions.
These challenges encompassed disparate practices in IP ownership, management and exploitation. A key document at the time, South Africa's National Research and Development Strategy ('R&D Strategy') (DST 2002), for example, highlighted the need to address 'inadequate intellectual property legislation and infrastructure' (Ibid.,p. 15), and an overarching goal of the Strategy was to protect South Africa's IP and indigenous knowledge, and conserve South Africa's unique biodiversity. Indeed, the protection of IP from publicly financed research was squarely placed in the context of the inadequate controls on protection of the unique biodiversity and indigenous knowledge, especially as it relates to the advances in biotechnology, and the innovation that research field has spawned.
A topical discussion point at the time of the development of the R&D Strategy concerned the trademarking of rooibos in the US in 1994. Rooibos is often consumed as a tea, has numerous health benefits, and is indigenous to the Cederberg mountains of South Africa. Its misappropriation by international enterprises in key export markets raised concern around the issue of protection of all knowledge that is indigenous to South Africa (Biénabe, Bramley, and Kirsten 2009). Twenty-five years after the event, a historic agreement between corporations producing and selling rooibos and the indigenous San and Under the rooibos agreement, the San and Khoi communities will receive 1.5% of the 'farm gate price'the price that agribusinesses pay for unprocessed rooibos (Aspalathus linearis), which is endemic to the Cederberg region, north of Cape Town. For 2019, the government considers that the compensation will amount to 12 million rand. (US$799,000). The San and Khoi communities will split the proceeds fifty-fifty. A third groupsmall-scale non-white rooibos farmers in the region who were disadvantaged under apartheidwill share in the Khoi portion.
The case of rooibos, and others, brought up the need for standardization and harmonization of these elements pertaining to technology transfer in the publicly funded research sector.

New white paper 2019
South Africa's science, technology and innovation policy is framed by a new White Paper (DST 2019). Its predecessor White Paper, published in 1996, and instituted after the democratic transition, established the policy basis for the current national system of innovation as that of addressing the dual goals of economic growth and social development. In contrast, the new White Paper foregrounds the role of innovation and inclusivity. It states: This White Paper, which is based on extensive review of the National System of Innovation (NSI), sets the longterm policy direction for the South African government to ensure a growing role for science, technology and innovation (STI) in a more prosperous and inclusive society. It focuses on using STI to accelerate inclusive economic growth, make the economy more competitive, and improve people's daily lives. It aims to help South Africa benefit from global developments such as rapid technological advancement and geopolitical and demographic shifts, as well as respond to the threats associated with some of these global trends. (DST 2019, x) To the extent that it addresses technology transfer specifically, the White Paper identifies the following aspects to be addressed in respect of the IPR-PFRD, including 'the expanded impact of Offices of Technology Transfer, enhanced commercialisation of intellectual property, support for openness, enhanced support for SMEs, and the appropriate structure and positioning of NIPMO in the NSI' (DST 2019, 36).
Critical assessment of the current indicator typology using the adapted 'revised contingent effectiveness model' The discussion so far, of both the context of South Africa in which scientific institutions are nested, and the current legislative and policy context, provides the backdrop against which the 'technology transfer system' has emerged. What evaluation approaches exist to assess this system?
Aspects This increased attention on technology transfer performance in South Africa has ignited new research, in some cases, and emboldened earlier research efforts. A longstanding and perhaps the leading commentator on university-industry linkages in South Africa, Kruss (for example, 2008Kruss (for example, , 2018; see also Kruss and Visser 2017) has led both the conceptual discussion and empirical evidence of South African specificities insofar as the measurement of innovation is concerned, though focusses less specifically on technology transfer as a discrete component of the broader innovation system. Bailey and Mouton (2005) have used and extended the work of Bozeman in relation to discussions on research utilization and uptake. Wolson (2007) identified constraints faced by TTOs in the context of the biotechnology sector. Kruger and Steyn (2019) put forward a conceptual framework to look at how innovation spaces at universities support entrepreneurs, with respect to what has become to be known as the fourth industrial revolution and in South Africa. Gumbi (2010) developed a review of performance standards for technology transfer offices, as viewed from its utility to the institutions to which the offices report and Nyatlo, Marcus, and Parsons (2015) have since then developed a framework for effective technology transfer offices. Finally, Alessandrini, Klose, and Pepper (2013) have argued that measuring the effectiveness of technology transfer in South Africa must consider the country's 'socio-economic and institutional specificities'; though they stop short of elaborating in detail what these are.
The work cited above tended to foreground outputbased performance indicators, with very little emphasis on assessment of the broader social or economic impacts of technology transfer. This reflects a significant gap within the South African literature, within which this paper situates its discussion and recommendations.

Revised contingent effectiveness model of technology transfer
The contingent effectiveness model of technology transfer ('the model') was developed in 2000 by Barry Bozeman (2000), then based at the Georgia Institute of Technology's School of Public Policy. The model was proposed by Bozeman as a way of organizing a proliferating body of literature on technology transfer arising from the activities of researchers and practitioners within universities and state research facilities in the United States (US). 2 The model recognizes, as Bozeman (2000) writes, 'that parties to technology transfer have multiple goals and effectiveness criteria,' and among the model's explicit purposes therefore was the elaboration of a more integrated understanding of effectiveness.
Unfortunately, many studies of technology transfer never make clear what is meant by effectiveness and seem simply to assume that we all hold some unspecified unitary concept of effectiveness […] This assumption is wrong, as we have shown with both statistical (sic) and case study (sic) evidence. (637) On the input side of the original model, as it were, Bozeman proposed five determinants of effectiveness, namely: characteristics of the transfer agent; characteristics of the transfer media; characteristics of the transfer object; demand environment; and characteristics of the transfer recipient. On the output side, it proposed six criteria against which effectiveness is typically but not uniformly assessed in the literature. These criteria were opportunity cost, scientific and technical human capital, political, economic development, market impact, and 'out-the-door' (Ibid). The revised model added a single additional criterion: public value. As Bozeman, Rimes, and Youtie (2015) write: The addition of the Public Value criterion arises from the recognition that transfer agents, particularly public sector transfer agents but others as well, are housed within agencies and organizations that are themselves in pursuit of broad public-interest goals. Thus, their endeavors are motivated, influenced, and directed by everchanging constellations of public values. (35) They continue: Importantly, the Public Value criterion counterbalances some of the emphasis on economic impacts of technology transfer. To this end, it is comparable to notions of responsible innovation, which take into consideration equity and inequality; sustainability, health and safety; and the improvement of quality of life through addressing societal needs or grand challenges. The expectation that African Journal of Science, Technology, Innovation and Development 865 invention and innovation will produce economic growth is not new; however, inclusion of the Public Value criterion in the Contingent Effectiveness Model acknowledges the fact that economic impacts are sometimes not the best measure of well-being.
To summarize, the revised thesis of Bozeman, Rimes, and Youtie (2015) is that researchers and evaluators of technology transfer understand effectiveness in a multitude of ways, but that public value is increasingly important as an effectiveness criterion. Bozeman and Sarewitz (2011) argue that without consideration of the public value impact of research, science policy makers will have little chance of achieving the desired social outcomes they seek.
Adapting the model for the South African context One of the gaps we identify in the revised model is the omission of Indigenous Knowledge as forming part of the suite of Transfer Objects. To the extent that Scientific Knowledge is included, as a legitimate form of knowledge, we suggest the same qualifying criteria for the inclusion of Indigenous Knowledge. The legitimacy of indigenous knowledge, in the South African context, is encoded in South Africa's Indigenous Knowledge Systems Policy of 2004 and, more recently, in a bill presented to the South African Parliament in 2017 by Minister of Science Naledi Pandor, entitled 'Protection, Promotion, Development and Management of Indigenous Knowledge Systems Bill'. Related to this is the vexed but legitimate issue of Reparations within the Demand Environment of the model, and the use of Appropriation as a strategy to obtain the Transfer Media. The example of rooibos described above provides the basis for inclusion of these additional components within the model.
The academic literature on Indigenous Knowledge Systems (IKS) is voluminous (a useful entry point in South Africa is Green 2007Green , 2008. This paper concerns itself with the integration of IKS with other (scientific) knowledge systems. Odora Hoppers (2002) discusses the integration of indigenous knowledge with knowledge systems, touching on many of the topics pertinent to the South African policy landscape at the time, stressing that 'development is crucial if [IKSs] are to be integrated with other knowledge systems'. More recently, Mistry and Berardi (2016) revisited the topic within the context of sustainable land use practices from local knowledge and technologies underlying IKS. The discussion was oriented towards the problems of environmental degradation and climate change. These narratives imply the importance of incorporating non-scientific knowledge such as those embedded in IKS in the framework for assessment of publicly funded research intellectual property protection. That is because knowledge generated from local or traditional sources are increasingly becoming the subject of study by publicly funded research institutes, as we have argued above.
Additionally, we articulate a concept of Citizen Needs in the Demand Environment, as forming one criticalbut of course not the onlyurgency for universities in South Africa and by extension their TTOs. Finally, a change we suggest to the model is to add bidirectionality between Transfer Recipient and Transfer Agent (Figure 1), reflecting a dialogical relationship between actors in innovation systems.

Review and clustering of current indicators
A survey has assessed the performance of this nascent public IP and technology transfer 'system' (Mustapha et al. 2017). 3 A subsequent opportunity for structured discussion was catalyzed in April 2017, when incumbent Minister of Science and Technology, Naledi Pandor, launched results of the study. These data were widely anticipated by research managers, lawyers, and policymakers as reflecting a 'baseline' of IP and technology transfer results seven years after the enactment of the IPR-PFRD Act. In her address, Pandor focussed on a selection of key indicators showcased in the research reporttechnologies managed, licenses executed, revenue generated, start-ups formed, as well as the skills levels of staff employed within technology transfer officesconcluding with modest praise and encouragement to all concerned: What these results show we are making progress, incremental but progress nonetheless, which is exciting. We are beginning to see enhancements to economic impact from public investment in R&Dand that's what we all really want for our country.
This section classifies each indicator in terms of the model, in order to develop an analysis of the clustering of indicators. To perform this review, we subjected each indicator to the evaluation question developed by Bozeman, Rimes, and Youtie (2015) ( Table 1, column 2) (Figure 2).
Deepening the critical assessment Multiple mandates of HEIs and science councils Weak GDP growth, increasing unemployment, persistent inequality and widespread poverty in both rural and urban areas are high on the national policy agenda in South Africa, as are the dire impacts of these socio-economic challenges on well-being and social cohesion. This article suggests that it is vital to maintain a sense of historical perspective as to what has precipitated this set of challenges, as well as to what has been the typical role of university and science council technology transfer practitioners and their institutional leaders in orienting themselves to these challenges. A critical discussion of the evaluation of the 'South African technology transfer system', to the extent it can be framed as such, is deficient without an understanding of the context within which key events have taken place. Nor can it be divorced from the greater legislative domain within which publicly financed research institutions are located, including the rights and responsibilities of the South African people, as enshrined in the Constitution.
Indeed, there are multiple and competing discourses about the role of these institutions, and their contributions within different economic and social spheres. Concepts such as the 'entrepreneurial university', 'neo-liberal university', 'decolonised university', or the 'university's third mission' have emerged, reflecting some uncertainty within academia about where a university's institutional purpose begins and ends. As institutions seen to be pivotal to South Africa's development trajectory, universities and science councils, which are targeted by the IPR Act, are compelled to 'balance multiple mandates' (Kruss et al. 2013(Kruss et al. , 2016. In this context a concept of 'public value' seems essential to the SA HEI and science council institutional context. Over and above the 'transference role' of TTOs in a purely economic sense, we view technology transfer as an element in the innovation system that can be considered to act at the national level. The naming of this process is perhaps unfortunate as it signifies the 'transfer' of technology from one party to another, whereas its function in our conception is rooted more in the mandate of what a university traditionally assumes, which is to generate new knowledge. In our conception, technology transfer is a process that has a role in the creation of new knowledge through an intermediary function. The process is located at the nexus of inventors within HEIs, HEIs themselves, and 'the people' as represented by government and industry (particularly black-owned, small businesses). The primary mediation role of technology transfer is between knowledge generators in publicly financed HEIs and research councils, and then these institutions and industry. The TTO performs this function acting within the regulatory framework provided by government, which sets the agenda through this role to some extent. Therefore, the perspective in this article is informed more directly by what the four requirements are from the Act (listed above), and what it requires from the four key actors engaged in these activities.

Indigenous knowledge protection and benefit-sharing
While there are notable exceptions, businesses that are black-owned in South Africa are, in 2020, primarily small formal businesses or small or micro businesses in the informal sector. It is often the case that what are effectively small informal businesses conduct services, either in the retail or health sectors that involve the use of indigenous knowledge and indigenous compounds derived from local flora or (sometimes) fauna. The combination of highly biodiverse and rare environment and the inequality prevalent in the society on all levels (asset, income, educational, health, etc.) has created a situation where people are put at risk to the predatory activities of business concerns both local and international. A case in point is that of rooibos mentioned above, while other cases from the popular media also worth noting include controversies over hoodia and moringa. Such scenarios are firmly within the realm of science and innovation policy as it relates to knowledge production (whether indigenous or not) and its protection and utilization.

Critical learning from the US experience
To the extent that the recent work of Bozeman, Rimes, and Youtie (2015) reflects the 'US experience', how can South Africa learn from the US experience, when it comes to effectiveness of technology transfer in publicly funded research institutions? A first learning opportunity between the US and South African situation is rooted in policy: South Africa's IPR Act adopts similar principles to the exploitation of publicly financed IP as its equivalent Bayh-Dole Act or Patent and Trademark Law Amendments Act (Pub. L. 96-517, December 12, 1980). The Bayh-Dole Act was developed in response to economic challenges in the USa period of high inflation and unemployment (stagflation). 4 South Africa passed the IPR-PFRD Act in 2010 only. In comparison with other more-developed countries, the state of technology transfer at publicly financed HEIs and science councils lags. If one uses the establishment of the Bayh-Dole Act as a watershed event that provided the legislation to underpin policy aimed at promoting technology transfer of research outputs at such      institutions, then South Africa is, in effect, 28 years behind the US in this regard. A second potential learning opportunity is that the US provides a number of excellent examples of innovation districts, such as the Boston Square Mile, or indeed Silicon Valley, that have successfully exploited linkages between universities and public laboratories, private investors (equity, venture capital), and inventors. The point we wish to stress here is not that South Africa should aim to emulate the US experience. Rather, we are curious as to what critical learning can be derived from the convergence of actors, including and especially venture capital, in the US that gave rise to rapid digital economy growth through technology firms, which have demonstrated both widespread economic and social impacts.
Third, several South African HEIs and science councils had been active in the management of the IP stemming from their institutions decades before the government established legislation and supporting structures to enable and promote technology transfer. In fact, prior to 2008, seven South African HEIs and four research councils were active in Patent Cooperation Treaty (PCT) patent applications (Sibanda 2009). Arguably, the success of their international counterparts in achieving generous incomes streams, including in the US context, has put the practice of technology transfer higher on the agenda of South African HEIs in the last few decades. The other perceived benefit of technology transfer at universities is the sharing of new knowledge encapsulated in the form of innovations, which relates closer to the traditionally accepted primary role of the university sector in society.

New sources of investment
In 2019, with support from technology transfer practitioners, a new University Technology Fund was established through the South African SME Funda publicprivate partnership established under the auspices of SA President Cyril Ramaphosa's officeas part of an effort to stimulate the country's lagging economy (Rajgopaul 2019). The genesis of the Fund, according to SME Fund CEO, Ketso Gordhan, who announced its launch at the SA Innovation Summit 2019, was a call from leading technology transfer practitioners to ramp up investment to more actively exploit opportunities emanating from university R&D. While it is too early to assess the Fund's impact, its formation points to both new sources  Table 1. Table 2: Determinants of effectiveness in the contingency effectiveness model (Bozeman 2000, 637) Dimensions Description of determinant Transfer Agent The institution or organization seeking to transfer the technology, such as government agency, university, company, etc., including characteristics, culture, personnel.

Transfer Medium
The vehicle, formal or informal by which the technology is transferred.

Transfer Object
The content and form of what is transferred, the transfer entity.

Transfer Recipient
The organization or institution receiving the transfer object. Demand Environment Factors (market and non-market) pertaining to the need for the transferred object.
African Journal of Science, Technology, Innovation and Development 873 of public support for technology transfer but also needed synergy between state and private venture capital capabilities to manage the return on investment.
Public value largely missing from measurement Key issues for measurement arise from a mapping of indicators to the revised contingent effectiveness model. Tables 1 and 2 taken together reflect a consensus within measurement practitioners and policymakers toward a focus on scientific & technical human capital, followed by out-the-door, and then indicators of opportunity cost. Of 38 indicators reported, only two indicators, we suggest, fit the Bozeman model for Public Value. Of 57 non-reported indicators, five align to a concept of Public Value.
Future national surveys of IP and technology transfer in publicly financed research institutions present new potential opportunities for refining the specific criteria against which effectiveness might be measured in order to aid actors, such as the NIPMO, the new Department of Science and Innovation (DSI), and others; that is, to allocate resources efficiently and to report national progress in line with the new White Paper on Science, Technology and Innovation. What should such surveys aim to measure?
What other indicators could be included under the heading of Public Value? At a glance, it could be useful to have a breakdown of the types of technology transfer activities in relation to social and economic challengeseducation and health, for example, are two of the country's most pressing challengesor in relation to the cultural and creative industries. Participatory action research would be useful and appropriate to articulate in more detail the basis of a Public Value indicator framework, and its data collection requirements.

Conclusion
Science, technology and innovation (STI)including and especially government STI policies and programmesare positioned in South African national policy discourse as among the key strategies to transform the country from its present impasse. Present within this discourse is a compelling idea: that if the country can harness its public R&D investments through technology transfer, then it can create wealth and, through the generation of licensable IP, improve on its technology balance of payments. This article points to gaps in the measurement and evaluation of the South African IP&TT system, and contributes to a strengthening of the measurement discussion at policy level, and, through its analysis, also raises a set of vital evaluation questions for University Technology Funds' designers, backers, and implementers.
Given the South African context, it becomes important to talk about the monitoring and evaluation of the technology transfer system as it pertains to national innovation strategies. Previous work has considered the monitoring and evaluation of the technology transfer system as it pertains to the public institutions involved. This paper considers question from the larger scale, placed in the context of the policy goals of the South African government. It draws theoretical developments on monitoring and evaluating technology transfer in the US, and critically discusses national level indicators collected in South Africa. Whereas previous studies have considered only the evaluation of the science system separate from the broader society, this paper intends to forefront the societal aspect as a fundamental element of the evaluation framework. Furthermore, it argues for additional indicators to fill measurement gaps.
We conclude that indicators need to be much more closely guided by country specificities. In the case of South Africa, indicators need better alignment with the goals and monitoring and evaluation strategies of the country's new White Paper on Science, Technology and Innovation.

Notes
1. Available at https://link.springer.com/journal/10961/44/5. 2. Bozeman's model which reflects research on nearly 4-5 decades of recent US experience of protecting IP and transferring technology generated by its public universities and laboratories. 3. The White Paper refers to a monitoring, evaluation and learning framework for the national system of innovation, in which it reflects components for measurement, including technology transfer (DST 2019, 28). 4. Lucier (2019) gives a historical overview of science and industry interaction in the USA.