Public support to innovation
- a multi-method approach to evaluate RTOs the case of Rise
In various policy documents, it is stated that the role of the research and technology organisations (RTOs) is to act as a bridge between academic and business-oriented research, i.e. that activities are pursued in the interstices of academic research and new ways of creating value in business enterprises. In this report, Growth Analysis proposes an impact assessment system to assist Rise in steering its activities towards this position.
In recent years, the Swedish Government has strengthened and renewed the Swedish industrial RTOs. In just a few years, state support for the RTOs has risen from around SEK 200 million in 2009 to close to SEK 500 million in 2013. This state support devolves to Rise AB, which consists of a network of Swedish industrial RTOs. These sort into four groups: SP, Swedish ICT, Swerea and Innventia. With increased funding comes a greater requirement for auditing of the RTOs' work, and how the state’s contributions in the form of so-called strategic competence funds are spent.
In its appropriations letter for 2013, Growth Analysis was commissioned to design an impact assessment system for Rise AB.
Growth Analysis draws attention to three areas in the impact assessment system of particular importance for describing and measuring the activities of the RTOs:
- Choosing projects
- Monitoring projects
- Tracking and measuring the impact of projects
It is important that the use of state aid in the industrial RTOs accords with the Government’s expectations as expressed in the government guidelines for the coming fiscal year. Growth Analysis's survey shows however that the RTOs’ strategies for the use of state funds are set out before the government guidelines for the coming fiscal year are made available. Growth Analysis therefore recommends a common strategy process for all of Rise's activities to support the approach indicated by the Government. To accomplish this, there needs to be a better monitoring system than the present one, and an alternative model for the allocation of the strategic competence funds that corresponds to the Government’s intentions.
Growth Analysis’s survey of how Rise’s activities are currently performed shows that within each group of RTOs are IT-based systems that control and document the activities. The problem is that each of the four RTO groupings has its own such system. This means that they have no insight into each other’s systems. Moreover, Rise's holding company has no insight into any of the individual groupings' systems. This situation inhibits the RTOs’ efforts to improve traceability and the state’s ability to measure the impacts of the public-sector initiatives.
To make Rise’s activities transparent requires a common monitoring system for the whole enterprise. The holding company currently has little insight into the activities of the RTO groupings. Growth Analysis therefore proposes a common monitoring system, or a so-called project administration system, for all four groups of RTOs and the holding company combined. The aim of the system is to describe the projects which the RTOs undertake and to enable monitoring after project completion.
Tracking and measuring the impact of projects
Activities within the Rise Group are complex and consequently a single method cannot be used to measure the economic impacts of the state contributions. Growth Analysis therefore recommends that the impacts be measured by applying a range of proven assessment methods. One type of method is required to describe open research where the results are published freely in the form of articles that are available to all businesses. Closed, commissioned research requires other methods, since the results are often confidential and will accordingly not be disseminated in the same way. There follows a description of the four methods which, in combination, are capable of evaluating how well the RTOs’ activities strengthen the renewal of Swedish industry.
Growth Analysis proposes an assessment system comprising the following methods:
- Description of citations, patents and IP rights
- Follow-up survey of the RTOs’ customers (customer satisfaction survey)
- Quasi-control group survey
- Cost-benefit analysis
Allocating strategic competence funds
Under the present so-called “noms” model, state funding is allocated on the basis of sales. No distinction is made between the different types of activities that generate the sales. This means that all kinds of revenue sources justify receiving state funding, whether income is derived from research activities or pure technology consulting. The principle should be that the allocation of state funding encourages funds to be used for economically significant activities in accordance with the government’s intentions. Growth Analysis therefore discusses two different proposals for an enhanced allocation model.
Growth Analysis suggests that Rise should change its existing model, where the different types of revenue sources are treated equally, to a model which highlights open and commissioned research rather than “technical services”.