Desperately seeking innovation nirvana: Australia’s cooperative research centres

Abstract The Cooperative Research Centre (CRC) program has been one of the flagship programs of Australia’s National Innovation System (NSI) for 30 years. But compared to other members of the Organisation for Economic Co-operation and Development (OECD), statistics indicate that Australia has persistent low levels of business expenditure on research and development (BERD), while World Intellectual Property Office (WIPO) reports highlight poor diffusion-of-innovation characteristics. In response to reports of this ilk, and in line with the generally evolving nature of NSIs, the CRC program, and the Australian NSI system, has developed and matured. In a wide-ranging longitudinal review of the program’s policy documentation, we discover that a number of changes have affected the language used within the program, the targeted organizations, and, crucially, the duration of the funding periods. We conclude that, within the confines of the nation’s historical and geographic context, the CRC program exhibits similar characteristics to NSIs elsewhere. In addition, there may be scope for Australia to split the CRC program into separate streams, and that even further support for integration into the Asia-Pacific markets is likely to be beneficial to the diffusion of Australian innovation.


Context setting
Australia's economic history since white settlement, which has generally been associated with agricultural and resource exploitation, has mainly tended to adopt and utilize innovation developed in other countries, rather than the development of homegrown, paradigm-changing innovation. This has meant that, with a few notable exceptions such as the Cochlear ear implant (Cochlear 2018) or wireless networking (CSIRO 2013), Australia has tended to adapt overseas technology to Australian conditions. For the last thirty years, successive Australian governments have realized the limitations of economic dependency on natural resources and agriculture, and have identified the need to diversify, especially in an age of disruptive technologies and services. Successive Australian governments, across the political spectrum, have therefore promoted innovation as being crucial to the nation's future wealth and continued high standard of living, and as a result have developed policy mechanisms to encourage it.
However, despite significant efforts through the innovation mechanism of university-industry collaborative (UIC) research, the country's ability to diffuse innovation to the rest of the globe remains mediocre in comparison to other Organisation for Economic Co-operation and Development (OECD) nations (Australian Government 2011;Miles 2015). The Australian Government's 2015 National Innovation and Science Agenda (Australian Government 2015b) attempted to address this problem, by encouraging universities and industry to strengthen their ties to drive innovation, thereby enhancing the nation's prosperity. In 2016, this was extended to an international context through the Global Innovation Strategy, with a modest budget allocated, in particular, to "build[ing] strong research and business connections within the Asia-Pacific region" (Australian Government 2016b, 6).
An important instrument with respect to meeting these government policy objectives is the Cooperative Research Centre (CRC) program. The program was originally intended to enhance the nation's research capacity by sponsoring collaborative research that would otherwise not have taken place owing, in part, to the "geographically and institutionally" dispersed nature of the national research network in Australia (Australian Government 1998, 8;Slatyer 1994, 147). Although governments across the political spectrum may disagree on the detail of the CRC program, there remains a relatively uncritical acceptance that the overall program is worth continuing. This viewpoint reflects O' Leary and Bingham's (2009, 6) criticism that the hype around collaboration is "often celebratory and only rarely cautious". Indeed, Dickinson and Sullivan (2014, 163) contend that "evidence of collaborative performance remains uncertain at best", while Sullivan et al. (2013) maintain that the evidence base for promoting and relying on UIC research as a national innovation driver remains debatable. In particular, the evidence suggests that, despite the investment of billions of dollars (Australian Government 2016a) over more than two decades, the CRC program may not have achieved the desired state of deep integration between academic and industry.
Despite the foregoing issues, the CRC program's founding principles nonetheless remain relevant in the twenty-first century. Australia continues to face intractable health epidemics such as obesity and diabetes (Colagiuri 2017), together with climate change and sustainability challenges. However, the impact of the nation's innovation on the global economy is limited (WIPO 2017), with Australia's business expenditure on research and development (BERD) remaining at the lower end of the OECD, and declining (OECD 2018). Despite the CRC program and similar flagship programs such as Australian Research Council (ARC) Linkages and the National Health & Medical Research Council (NHMRC), Australia's rankings in diffusing innovation on the global stage have slipped as much as 40 places in the last decade.
Although reports focussing on the CRC program, and the wider national innovation system, abound, this discussion is the first major examination of the CRC program since Garrett-Jones, Turpin, and Diment (2013) treatize, which suggested that the program's future was at the crossroads. Our overview of the program is particularly timely since arguably the biggest shift in the focus of the CRC program occurred in 2015, with the introduction of smaller, time-limited projects managed by industry, known as CRC Projects (CRC-Ps). In light of the continuing sub-optimal performance of the Australian NSI in terms of BERD statistics and global diffusion rankings, we provide here a longitudinal appraisal of the CRC program since its inception. We examine how and why the program has changed over time, and reflect on whether the original or even the current, intent of the program can be realised to the extent that policymakers desire. In particular, we look at how the CRC program has evolved, and determine whether the changes support the recent broader policy statements aimed at increasing Australia's innovation capacity. In undertaking this examination, we use international data to position the Australian CRC program on a global basis, rather than just examining it from an introspective national perspective.

Theoretical approach
The CRC program's raison d'être, as an integral part of Australia's NSI, embraces the notion of bringing researchers and practitioners together. A variety of theoretical approaches is available to analyse the way in which innovation operates within a nationaland even internationalcontext. These include an emphasis on: the technological aspects of innovation (Hekkert et al. 2007); the sectoral characteristics within which innovation occurs (Malerba 2002); the regional nature of innovation (Cooke, Uranga, and Etxebarria 1997); and the systemic environment within which innovation is nurtured and operationalized (Lundvall 2010;Nelson 2003). Increasingly, however, scholars and policymakers have tended to view UIC innovation through the lens of a "National System of Innovation" (NSI). This concept is generally attributed to Chris Freeman's seminal work in the late 1980s (Freeman 1987), although Freeman's own work was informed by earlier unpublished work of Lundvall (Edquist 2005, 3). A widely accepted model of a national system of innovation emerges from the OECD, shown in Figure 1 below. The complex interaction between various government and private actors, along with enabling infrastructures such as the communication, education and regulatory systems, combine to determine "innovation capacity" and ultimately "growth, jobs, and competitiveness" (OECD 1999, 23). The National Science Board (2012, 4) makes the point that the "relationship between R&D and innovation is highly complex", thus requiring a deep understanding of the interplay within the national innovation ecosystem, and also how this ecosystem interacts with and complements the economic conditions within a nation.
For neoclassical economists, addressing market failure is the underlying motivation for governments to enter the innovation policy space in order to restore competitive equilibrium. This perspective advocates government intervention wherever market performance produces a sub-optimal outcome (Keech, Munger, and Simon 2012, 9). When innovation, which from a Schumpeterian perspective is regarded as beneficial for economic growth (Mazzucato and Perez 2014, 4), fails to be developed sufficiently by the private sector through paradigm-changing innovation or through incremental improvements to existing products and services, the state steps in and implements policy initiatives. However, the neoclassical approach tends to favor a static costbenefit analysis of innovation policy, with an end-goal of some sort of innovation equilibrium in the free market. According to critics of this approach, the very creativity and uncertainty-embracing nature of the innovation endeavour can be stifled (Mazzucato 2017, 31;Foxon 2015). Indeed, a growing chorus of evolutionary economists view innovation policy as a dynamic process, with a biological-like complexity (Edquist 2005, 224), and constant feedback loops which interact together in a complex system (Hekkert et al. 2007). According to this conceptualization, technological innovation, market forces, societal changes, government objectives, and geo-political forces constantly shape the way in which policy is developed, and the direction and imperatives that guide it (Nelson 2017).
The Australian NIS, of which the CRC program constitutes an important element, allows the Australian Government to subsidize applied research that would not otherwise take place on account of excessive uncertainty and risk (De Jong, Kalvet, and Vanhaverbeke 2010;Love and Roper 2015). Auerswald and Branscomb (2003) identified five stages of innovation. These stages proceed from (background) research to concept invention, thence to early-stage technology development (ESTD), product development, and finally production and marketing. Venture capital generally only becomes interested around the ESTD stage when the uncertainty of the concept has become a manageable risk. In light of this, governments, through a variety of mechanisms such as R&D tax incentives, or even outright grants, seek to remove uncertainty associated with innovation, and lower the risk threshold for the development of new products, processes or services.

The study context: national innovation systems and innovation diffusion
It is important to differentiate here between the use of the word diffusion in the context of policy, and in the context of innovation. The former relates to the way in which policies are created and enacted to support particular governmental initiatives (in this discussion "innovation policy"), and the way in which the word diffusion is used when tracking the innovation performance of countries.
In the case of policy diffusion, Shipan and Volden (2008) differentiate between four mechanisms by which policy initiatives are diffused internationally. These are where governments adopt innovation policy mechanisms for a variety of reasons, including (i) where they learn from both the theoretical literature and through information exchange between governments and through organisations such as the OECD; (ii) through economic competition at a national level, where one nation seeks to perform better than another; (iii) through imitation; when one nation simply imitates what is happening in other areas; and, finally, (iv) coercive measures, where donor funding from organisations such as the International Monetary Fund (IMF) link funding to policy developments (Shipan and Volden 2008, 843). Milewicz et al. (2018) add "issue linkage" in free trade agreements to this list, this being where nontrade conditions may be imposed on the participating countries. That said, Izs ak, Markianidou, and Rado sevi c (2013, 50-51) caution that "it is not possible to define "optimal policy mixes'" and that a policy mechanism that works well in one location may not be successful in another. Indeed, the complexity of the way in which innovation systems interact with local conditions precludes a one-size-fits-all approach.
Innovation diffusion, by way of contrast, relates to the degree to which innovation generates export income and provides economic benefit to the country in which the innovation occurred. This is an important measure from both an economic and reputational perspective, and one of which Australian policymakers are acutely aware. Recent policy announcements in Australia such as the National Innovation and Science Agenda (Australian Government 2015b) and the National Science Statement (Australian Government 2017) have consistently decried the national innovation-diffusion-deficit and have pointed to the sub-optimal interaction between universities and industry.
Since 2007, the annual Global Innovation Index (GII) has tracked the number of factors contributing to a nation's innovation potential. Australia, ranking 25th globally in 2017 in the production of intellectual capital, underperforms with respect to diffusing that knowledge into innovation systems, with the nation ranking 100th (out of 128 nations), down from 63rd in 2013, as depicted in Figure 2. Despite the significant expenditure aimed at promoting a culture of R&D and innovation (Australian Government 2016a), the ability to transfer knowledge into practical applications that generates global trade income remains consistently below par (Lundvall 2016). With Australia's declining innovation diffusion performance, it follows that, if CRCs are intended as vehicles to produce knowledge that is applied by industry, and thus lead to innovation with international application, this element of the NSI is operating at lower than desired effectiveness. This is clearly indicated by the recent drop in world innovation diffusion rankings, as well as the government's own admission that "Australians are renowned for their smart ideas, but we often fail to … turn them into commercial realities" and that "Australia's rate of collaboration between research and industry sectors is the lowest in the OECD" (Australian Government 2015b).

Cooperative research centres: an examination of influences and trajectories
Throughout the 1980s, Australia's industrial base declined markedly as import tariffs were lowered, the Australian dollar was floated and consequently devalued, and new technologies were embraced. Changes at this time to the Australian motor vehicle industry were particularly emblematic of the era, with a reduction in the number of car industry jobs causing national angst. In response, early NSI initiatives such as the Information Industries Strategy (IIS), National Industry Extension Service (NIES), and the Research and Development program (Stewart 1990) were enacted. A tax incentive scheme and a grants scheme, which constituted an attempt to complement the Commonwealth Scientific and Industrial Research Organisation (CSIRO), were also created, with these schemes representing an early effort to bring together researchers and end-users (Stewart 1990).
Against the backdrop of a nation with a "low level of science and technology expenditure, a high level of government involvement in financing and undertaking research, a low level of private sector research and development and exceptionally high dependence on foreign technology" (Gregory 1993, 324), the CRC program began in 1991 as a crucial plank of the emerging NSI. This was at the suggestion of Chief Scientist Ralph Slatyer, who identified a number of deficiencies in the national research system, including a lack of research integration, relatively small and ineffectual research teams, and a need to connect researchers and end-users more closely (Slatyer 1994). At this juncture, the emphasis was on research, and there was no reference to a NIS.
In the beginning, Slatyer emphasized the longer-term nature of research that would manifest in self-sustaining centers, and research that included both technological and social outputs. Here, industry and academia were intended to enter into a symbiotic partnership, with an emphasis on tackling broader societal and economic problems, notably the declining competitiveness of Australian industry (Australian Government 1990). Perhaps the most succinct description of the rationale for CRCs comes from a report by Mercer and Stocker (1998, III): The CRC Programme's overall objective is to strengthen long-term collaboration between research organisations, and between these organisations and the users of research, in order to obtain greater benefits from Australia's investment in R&D. 1 Since the program's inception, reviews examining its efficacy have occurred at the behest of various government ministers, a parliamentary secretary, departments responsible for science, industry and education, and the CRC Association of Australia. 2 In general, each review has reported largely positive findings, with recommendations being mainly of an administrative nature. Some of these reports were a genuine attempt to assess and improve the capacity of CRCs to deliver high-quality research outcomes, including commercialisation. Others, such as the report by Miles (2015), were commissioned by an incoming government seeking to apply its own ideological approach which favoured the private sector.
The various reports, intended to examine a variety of different elements, can be grouped into five themes:  surveyed) and, secondly, the introduction, fine-tuning, and enforcement of performance measures and Key Performance Indicators (KPIs). The application process has also received attention, with nearly half the reviewed reports discussing if it should be a single or a two-stage process. The program has been the subject of often conflicting recommendations in various other government reports. One such document was the "public support for science and innovation" report (Australian Government 2007), which advocated maintaining the CRC program, with greater emphasis on its original social mandate, along with a complimentary, "more nimble" program more akin to the CRC-Projects recommended in the later Miles report. This contrasts with the report by the National Commission of Audit (2014), which suggested abolishing the program altogether, owing to a perceived duplication of services with organisations such as CSIRO and the Defence Science and Technology Organisation (DSTO). It was recommended that CRC funding should be redirected to the Australian Research Council, which also supports shorter-term industry-university collaborations through the ARC Linkage scheme.
The 2015 CRC report, commissioned by the Minister for Industry and Science included the direction to "consider whether it (i.e. the CRC program) is the most appropriate vehicle to support business and researchers to work together to develop and transition to Australia's industries of the future" (Miles 2015, 2). The report thus focussed on "increased jobs, exports, productivity, integration into global supply chains, new technologies, products or services, increased revenues and intellectual property outputs such as patents" (p. 10), picking up a number of elements in the OECD NSI model.

The study and methods employed
Against a backdrop of collaboration, innovation, and policy literature, a documentary review of the three sets of inter-related literature was undertaken. We used fourteen CRC guidelines for applicants covering the years 1990-2015 as a proxy for the way in which government policy becomes enacted. In addition, 21 government-appointed reviews were also examined, with these either specifically reviewing the CRC program or looking more generally into the Australian NSI. To ascertain the international standing of Australia with respect to innovation, ten years of GII reports were examined, together with multi-year OECD data pertaining to BERD. A variety of other Australian and international reports and policy documents were also inspected. Most of these were located by searching the Internet, OECD, and Australian government websites, with others sourced through the Australian National Library or Australian university libraries. Latent manual coding was used to identify themes and differences between the documents on a chronological basis. In cases where numerical counts were undertaken, these were undertaken using the search functions in Microsoft Word and Adobe Reader, and were tabulated in Excel to generate graphical representation. Three questions guided the study. First, how has the political and governance milieu within which the CRC program operates impacted the program? Second, by exploring the guidelines and reports, what changes are evident within the program? Finally, how has the CRC program, an important element of the Australian NSI, evolved over time?

Research outcomes
In the following section, we examine in detail areas of significant change within the CRC program. The program has moved away from what might be termed "basic" research with the aim of developing, "frontier" or "polyvalent" knowledge, "which is at the same time theoretical and practical" (Ranga and Etzkowitz 2013, 243) to a more short-term, outcome-focussed orientation, emphasizing the immediate applicability of knowledge to a commercial context. The changes underpinning this include: changes in the language of collaboration (e.g. from cooperate to collaborate); changes to the length of the funding period over time; the nature of leadership within the research center; whether there was a target sector such as the small-to-medium-enterprises (SME) sector; and if the concept of "social development" and "public good" were embedded into the guidelines, or if there is a greater emphasis on economic or social imperatives. 4 CRC Application Guidelines provide the basis for the program's administration. The first set of Guidelines, attributed to the program's architect, Ralph Slatyer, set out clearly the conceptual and policy basis underpinning CRCs in an extended introduction. In effect, there have been several "eras" within the program, as shown in Table 2 above. In some cases, transitions between eras reflect the nation's political landscape; in other cases, it relates to the program's maturation, in response to review recommendations, illustrative of what Mazzucato (2016, 146) describes as "ad hoc theoretical understandings and policy advice" of developing national systems of innovation. The seven areas listed in the left column of Table 2 were chosen as they represent the major shifts in how the program is administered. In the discussion below, "sector" and "public good" are dealt with together, as the outcome focus has moved from societal intractable problems to industry-centric ones.
As the Australian NSI has evolved, so too have the Guidelines documents changed over the years in both prescriptive and descriptive content. Earlier editions were very general; for example, with respect to how the CRC would be managed, or who would lead it. The level of prescriptive detail has fluctuated, with the greatest level of detail in the 1990s Guidelines. From 2000, the guidelines became more succinct and less granular or prescriptive, including referring applicants to the Australian Stock Exchange's (ASX) good governance principles, rather than spelling these out. In 2015, for example, the guidelines direct, with respect to governance, that applicants simply "must have governance arrangements that are suitable to deliver the proposed results" (Australian Government 2015a, 9).

Cooperate or collaborate?
Although collaboration appears central to Australia's innovation agenda, the literature suggests that the concept lacks a clear definition (Himmelman 2002;Salvato, Reuer, and Battigalli 2017). Collaboration could be broadly described as two or more people or organizations acting together, working toward a common objective (Gray 1989;Mattessich and Monsey 1992). Multiple terms are used for collaboration, including networking, coordinating, cooperating, collaborating, and boundary spanning. Although these terms are frequently used in an undifferentiated way, there are distinct sets of behaviors associated with each (Keast et al. 2004;Wilson 2012). In particular, Thomson and Perry (2006, 23) observe that "cooperation and collaboration differ in terms of their depth of interaction, integration, commitment, and complexity, with cooperation falling at the low end of the continuum and collaboration at the high end".
Based on the word stems and including nouns, adjectives and verbs, Figure 3 above reflects a count of the terms "cooperation" and "collaboration". As an example of how this change from cooperation to collaboration is operationaliszed, between the 1996 and 1998 versions of the guidelines, the "selection criteria" section changed from the heading "cooperative arrangements" to "collaborative arrangements" (Australian Government 1996Government , 32, 1998. The change in nomenclature could  1990 1991 1992 1994 1996 1998 2000 2006 2008 2009 2010 2012 2013 2015 Instances of "collaborat+" in CRC guidelines Collaborat+ Cooperat+ Figure 3. Instances of collaboration and cooperation in the Guidelines.
suggest that the program's administrators have developed a better understanding of the theoretical basis of the collaborative environment. Yet the current refocussing from long-term research in favour of shorter-term CRC-P ventures, which are more akin to principal-agent relationships, could also suggest that the current Australian government has, in some isomorphic fashion, embraced the fad of collaboration.
While the nomenclature change may prima facie appear insignificant, it is far from it. Collaboration has emerged as the default setting for governments worldwide. Yet collaboration has been criticised as simply being a "fad", a "cult" or an "obsession" (O'Flynn 2009, 112), while the "efficiency, effectiveness and responsiveness of collaboration is very contested" (Sullivan et al. 2013, 126).
Collaboration is a messy and contradictory process, often with unintended outcomes (Thomson and Perry 2006), and simply labelling an activity "collaboration" does not make it so (Innes and Booher 2010). In particular, genuine collaboration is characterized by dense, interdependent reciprocal connections, and frequent communication (Keast and Mandell 2014), so marriages of convenience merely for the sake of gaining government funding are unlikely to exhibit these characteristics (Peacock 2017). Moreover, collaborations without a solid relational foundation are problematic, with long-term unintended consequences including contentious relationships, which can ultimately lead to a view that collaboration or working together in any form is disadvantageous and risky. If the mechanism of UIC research is to remain a key component of the Australian NSI, there is arguably scope for a more nuanced understanding of the nature of genuine collaboration in policy development.

Period of funding
Closely aligned to how collaboration is perceived and understood is the duration of the CRC grant periods. The original vision for CRCs included funding that transcended election cycles, lasted for around seven years, and allowed researchers to "pursue long-term programs of research" (Australian Government 1990, 5). This extended operational timeframe enabled research to be undertaken in a programmatic manner on both the difficult "wicked problems" of the day, or even on "blue sky" basic research that might generate spin-offs. Slatyer makes the point in the introduction to the first guidelines that, just because the program is focussed on improving the competitiveness of Australian industry, there should not be "an emphasis on short-term near-market research at the expense of long-term strategic research" (Australian Government 1990, 2). Thus, the intention seems to be to provide enough time for genuine collaborative partnerships to form, that would produce paradigmchanging research, with researchers building upon nascent discoveries and working towards significant breakthroughs.
From the earliest days, it was made clear that funding durations were limited, and that more permanent research entities would ideally emerge after formal funding relationships had ceased (Slatyer in Australian Government 1990, 2): However, it is important to note that it is not an objective of the CRC Program to establish permanent research institutions supported indefinitely using program funds. Program funding is for a maximum of seven years. It is expected that participants will come to recognise the benefits of collaboration and user involvement in long-term research and will continue to collaborate when CRC funding ceases.
Although this goal was a significant element at the commencement of the program, it is no longer present in the policy literature. Indeed, the move towards shorter funding rounds with organizations funded in subsequent rounds of grants appears to be in direct contrast with Slatyer's original vision for this element of the NSI. The original group of centers was funded for between five and seven years, averaging a little under six and a half years. For the first nearly decade and a half of the program, the broad description of "long-term" was used. In 2007, the description changed to "medium to long-term", which manifested in a range of funding periods between two and nine years, with an average of just over six.
The 2015 Myers report recommended a significant change to CRC duration, with the program being split into two parts. One category is a long-term maximum of ten years, still called CRCs, with a second, arguably more transaction-oriented category, denominated as CRC-Projects (CRC-P), which run for a maximum of three years. In the future, it is expected that around 30 CRCs will operate, compared to 70 CRC-Ps, all reinforcing the growing emphasis on short-term outcomes. 5 However, research suggests that genuine collaboration takes up to three years to develop (Keast et al. 2004), other funding bodies acknowledge that five years is a more realistic time period (Casey 2000), while Mazzucato and Perez (2014) contend that it takes 15-20 years for major innovations to develop. By reducing the time-critical element of relationship building, program administrators run the risk of depleting one of the core components of this research paradigm: that of genuine collaboration, unless, of course, it is simply assumed that such relationships already existed -which is not always the case.

CRC leadership
In a likely reflection of its university-centric origins, the earlier guidelines emphasized the research credentials of the CRC's management team. In contrast, the latest version stipulates that the centre's leadership must originate in industry. In the examples shown above in Table 3, the early emphasis was on the research management credentials of the director, who would act as the CEO. In the 2000 guidelines, the emphasis remained on research credentials, but emphasized the management of research. The latest version places general management expertise front and center, and does not refer specifically to a research management background.
The first set of guidelines do not describe the CRC's management structure, but this has since become a key element. For example, the 1991For example, the -1994For example, the and 1998 Guidelines specify that there is no preferred management structure, whereas the 1996 Table 3. Comparison of 1991Comparison of , 2000Comparison of , and 2015Comparison of management statements. 1991Comparison of -1998 2000 2015 … it is the Committee's preferred position that each Center nominates a Director. The Director should be an experienced and highly regarded researcher with appropriate management skills, who will play a pivotal role in the research and educational programs of the Center, and will act as its chief executive.
Ideally, the CEO should be an experienced and highly regarded research manager, who will lead and manage the activities of the CRC.
The CEO must have experience in project management, business management, commercialization management (including Intellectual Property management) and relevant sector and technology experience.
and 2000-2015 versions mandate a specific structure. Where mentioned, the Guidelines direct that there should be a "balance of scientific, management and business skills" across the management team (1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006). The 2008-2013 Guidelines add the elements of project management and commercialization as key attributes of the management team. As a result of the Miles (2015) report, an industry partner must now lead and manage each CRC or CRC-P, with an emphasis on commercialisation of outputs. A practical implication is that, while many university managers may be experienced in managing research-specific projects, and are thus familiar with the nebulous nature of such endeavours, an industry manager may be less suited to the inherent dynamism, complexity, serendipitous outcomes, and ambiguity of the collaborative environment (Noble, Charles, andKeast 2018). Bradley (1993) points out that research is not an input/output exercise, but rather relies upon an exegetical mindset that valorises emerging realities rather than expecting a defined return on investment. While the CRC program is important to the overall economic vitality of Australia, national systems of innovation must contend with not only the positive outcomes of universityindustry research discoveries, but also the many research paths that lead to a dead end (Mazzucato and Semieniuk 2017, 43). An emphasis on industry management experience potentially exposes CRCs to early termination on account of unrealized (and unrealistic) expectations, or that unreasonable pressure could be applied to the research team to "produce results".

Target sector, social development, and public good
Between the CRC program's inception and the 1998 Guidelines, there was no mention of a particular target sector, except for an emphasis on the intractable problems of the time and industry in general. In contrast, the 2000 to 2013 guidelines encouraged the engagement of SMEs in CRC activity and, in the 2015 guidelines, this was strengthened to mandatory. Just how SMEs are to be included in CRC projects is not explained: the Guidelines simply say that "projects should benefit SMEs and increase their capacity to grow and adapt in changing markets" (Australian Government 2015a, 6). This trend towards the SME sector is indicative of the evolution of NSIs in Australia and elsewhere around the world. Countries increasingly incorporate lower value funding mechanisms, such as innovation vouchers, into their innovation ecosystems, with the European Union being an exemplar of this (Flanagan, Uyarra, and Laranja 2011). As an active member of the OECD, and thus exposed to the practices of other advanced countries, Australia's own policy evolution follows this trend towards supporting innovation that "wells up from below" (Bush 1945, 107) from the small business sector.
Although intractable societal problems have always been an element of the Australian national system of innovation, the degree of focus on these research areas has waxed and waned over the life of the CRC program. The first three CRC funding rounds focussed on dealing with environmental, social and health challenges, through the application of "basic" research (Miles 2015). The concept of "social development" is present in the guidelines in the funding period 1990-2000, is absent from the 2006 guidelines, and is then reinstated in 2008-2013. The O'Kane (2008) report notes that the left-leaning government that commissioned the report made it explicit that the concept of "public good" be reinstated in the Guidelines. The phrase "environmental and/or social benefits" was therefore included in the purpose statement, alongside the continuing emphasis on economic benefits (Australian Government 2008Government , 2009aGovernment , 2009bGovernment , 2010Government , 2012Government , 2013. The "social development" concept was absent in the 2006 and 2015 versions, which coincided with the incumbency of conservative governments. While it is not the intent of this study to make political judgments, this is an example of how the CRC program, although enjoying bilateral support across the Australian political system, is nonetheless subject to the vagaries of political ideology. The way in which NISs operate within a localized and politicized context means that politics sometimes "gets in the way of rational analysis and therefore good policymaking" (Flanagan, Uyarra, and Laranja 2011, 27).
A further issue complicating the desire for a public good component for the CRC program is what has been termed the "managerialism" of government bureaucracies (Davis and Rhodes 2000, 76), inherent in New Public Management (NPM) practices, with foci such as outcomes and measurement, accountability for performance, performance measurement, and improved accounting methods, among other private sector management characteristics (Gruening 2001). These are all characteristics that economists would attribute to a neo-classical market-failure perspective, rather than an evolutionary perspective (Schmidt 2018). Although NPM is arguably no longer the dominant administration paradigm, it nonetheless influences how programs such as CRCs operate, alongside classical public administration and new public governance, albeit in a hybrid model (Dickinson and Sullivan 2014). Under these conditions, it may be difficult to demonstrate financial ROI for public good projects through, for instance, the establishment of a social enterprise, potentially excluding these types of applications from successfully arguing for funding.

Concluding remarks
National innovation systems are embedded within a socio-technical context, and the Australian NSI is no exception. The CRC program reflects the evolutionary nature that typifies NSI programs throughout the world. The changes to the CRC, such as (i) the quiet demise of the goal of CRCs spawning independent high-class research centres, (ii) an increasing focus on the SME sector, and (iii) a transfer of management from the university sector to private industry, reflect not only a trial-and-error evolution of the program (Mazzucato 2016), but also the local political, structural, and even geographic context within which the program operates (OECD 2010).
That the CRC program has changed over time is abundantly clear. Most changes have been incremental, more akin to Lindblom's (1959) "muddling through", with manageable and low-risk, incremental change rather than paradigmatic disruption. However, the 2015 program modification, which introduced the CRC-P shorter-term version of the program, was perhaps the most significant single change in the program's history. This element of the program has the potential to bring early and visible "wins" that promotes the "social contract for science" (Auerswald and Branscomb 2003, 228) within the nation's populace, demonstrates public accountability (Noble, Charles, and Keast 2018), and, most importantly, may generate SMEdirected innovation that is capable of being diffused to the rest of the world. However, if cautionary remarks about the length of time it takes to generate truly ground-breaking innovations are well founded, the shorter-term program may yield not so much paradigm-changing research, but rather incremental changes that improve the productivity of firmswhich is effectively business-as-usual for Australia. Rather, the CRC-P program, or one of a similar nature, may actually be more useful as a proof-of-concept vehicle that allows innovators to push their ideas beyond the uncertainty stage to where either the larger CRC program may be a better research vehicle (which has occurred in at least two situations of which the authors are aware), or where venture capital and early-stage angel investors may become more interested in the concept due to uncertainty elimination. Either way, there may be a place and time for such a program, whether it continues to sit under the CRC program or is spun off (in a policy sense) as a separate program.
The CRC program exemplifies innovation policy emerging from "experimentation and trial and error" (Mazzucato 2016, 147), and thus there are likely to be further modifications, driven both by lessons drawn from the applicability of emerging NSI theory to the Australian context, as well as the political milieu of the national context. Australia appears to have a mature and well-developed NSI. Yet the nation suffers from a seemingly insurmountable geographic barrier when it comes to the size of the domestic market. In contrast to the United States and Europe, Australia does not have access to large, integrated markets, and thus would appear to be at a disadvantage from the perspective of absorptive capacitythe ready market that can allow a nation to capitalize on nascent discoveries (Soete, Verspagen, and Weel 2010). Nevertheless, there is perhaps potential to build on the Australian Government's 2016 Global Innovation Strategy, which advocated increasing research and business connections in the Asia-Pacific region (Australian Government 2016b, 6). The AUD40 million over four years allocated to this strategy is a relatively small part of the AUD10 billion 2016-17 NSI budget, but a much more significant investment may be required in order to fully capitalize on the nation's NSI on a global scale.

Disclosure statement
No potential conflict of interest was reported by the authors. Notes 1. The above quote uses the term "users of research", with this concept having a selfdeclared "broad" definition in the early CRC documentation: "an industry sector, a private company, or a Government department or instrumentality" (Australian Government 1991, p. 8). The Guidelines now refer to "industry entity", defined as "an entity where the majority of its revenue is not derived from any government, capable of deploying research outputs in a commercial context" (Australian Government 2015a, p. 24). Thus, potential research users could come from the private or the not-for-profit sectors, but not from any wholly-government-funded agencies. 2. These include the report by Myer (1995); the review by Mercer (1998); the "Chance to Change" report by the then Chief Scientist, which included a number of recommendations for CRCs (Batterham 2000); the "Measuring CRC Outcomes" report (BioAccent and Capital Hill Consulting 2002); an evaluation of the CRC program by Howard Partners (2003); an Economic Impact Study by the Allen Consulting Group