Evaluation of the program Robot Lift
The program's design, effects and conditions for impact evaluation
In this report, we study the effects of the support program Robotlyftet. One purpose is to create a deeper understanding of the program’s design and effects, thereby contributing knowledge for more well-targeted policy measures. Another, more overarching purpose is to provide knowledge on the conditions required for impact evaluations and how these conditions can be improved. Impact evaluations provide important insights for the development, reassessment, and streamlining of policy.
Growth Analysis has previously raised the issue of a lack of knowledge base for policy development within industrial policy, and how the conditions can be improved. [1] In this report, we can further concretize and illustrate this by using a specific support program as our starting point.
Robotlyftet ran from 2018 to 2021 and aimed to strengthen the competitiveness of small and medium-sized manufacturing companies by promoting automation and robotization. The program had a total budget of SEK 68 million and consisted of knowledge-enhancing activities, free feasibility studies, and financial support through automation vouchers.
Based on a literature review, we assess that the design and program theory of Robotlyftet were well suited to meet the specific challenges faced by small and medium-sized manufacturing companies in order to increase investment in, and facilitate the implementation of, new automation technology. The literature review also shows that automation technology can contribute to increased productivity, improved product quality, greater resource efficiency, higher profitability, and safer work processes.
Robotlyftet's contribution to increased competitiveness has been assessed by other actors as successful and appropriate. This study analyzes the effects on the participating companies' development through a counterfactual impact assessment, which has not been done previously. Previous evaluations of Robotlyftet consist of descriptive analyses and the companies' own statements, which cannot be equated with effects. For this, causality needs to be established—not just correlation—so that the results can be reliably linked to the intervention.
We find that companies that received a so-called automation voucher had higher growth in turnover and productivity after one year compared to a control group of companies that did not participate in the program. Furthermore, we find no significant differences in development between small and medium-sized companies with automation vouchers – despite the fact that small companies generally received a larger subsidy and had a higher self-reported investment rate. However, the impact assessment is subject to uncertainty due to, among other things, incomplete data and difficulties in identifying reliable control groups.
Growth Analysis generally sees a need for more support programs to be designed with greater consideration for creating better conditions for conducting counterfactual impact evaluations. When implementing authorities report back on support programs, Growth Analysis considers it important that it is made clear and transparent whether the results are based on causal effects or on other forms of evaluation such as correlations, self-assessments, or process assessments. This is crucial in order to build up long-term knowledge about the effects of various support measures and, ultimately, to ensure that government resources are used effectively.
The Swedish Parliament, government inquiries, and the Swedish National Audit Office (2020) have also emphasized that economic policy should be evaluated for impact to a greater extent. However, it is not enough to simply conduct impact evaluations; they must also be reliable. In its evaluation Effektutvärderingar av näringspolitiken (2020), the Swedish National Audit Office assesses that the majority of the various authorities' effect evaluations are not of sufficiently high quality to be reliable. They therefore recommend that authorities and the government make greater use of Growth Analysis for impact assessments of industrial policy, and that the government report the results of well-conducted impact assessments to the Riksdag to a greater extent.
In the report, we identify several areas where there is potential to improve the conditions for conducting counterfactual impact evaluations within industrial policy. Our proposals include a more proactive and appropriate implementation of new support programs and improved access to data and documentation, which facilitates the identification or creation of necessary control groups. In the specific case of Robotlyftet, more detailed information about the companies' actual investments and why some companies participated and others did not would have strengthened the analysis.
[1] The prerequisites for conducting effective impact assessments in growth and industrial policy are well known and have been discussed in several of Growth Analysis' reports, see for example Förslag till förbättrade förutsättningar för en evidensbaserad näringspolitik (2023a), Tillväxt- och Näringspolitikens samlade effekter (2024b), samt Effektutvärdering för politikutveckling – Styrning för bättre förutsättningar (2024d).
