Growth Analysis (The Swedish Agency for Growth Policy Analysis) was commissioned by the Swedish government to make an ex ante evaluation of the operational programmes for the European Regional Development Fund (ERDF). According to the regulatory framework for the structural funds, an ex ante evaluation is to be made for each individual programme. With eight regional programmes and one national programme, this results in nine ex ante evaluations.

The ex ante evaluation consists of two parts. The first part is to provide comments on drafts of the programmes during the programme writing process to the authorities respon­sible for the preparation of the programmes. The overall aim of this part is to try to im­prove the quality of the programmes. The second part of the task is to make a final assess­ment of the operational programmes.

The present report constitutes Growth Analysis’ report on the commission and contains the agency’s assessment of each individual programme and a description of how the work was done. This summary summarises our assessment of the nine ERDF programmes.

The ex ante evaluation

An important part of the ex ante evaluation is to provide comments on draft programmes during the writing process. Growth Analysis has provided comments on these programmes on three occasions for the regional programmes and on two occasions for the national pro­gramme. Growth Analysis visited at an early stage all the authorities responsible for the preparation of the programmes to describe the content of the ex ante evaluation and to initiate a dialogue.

Within the framework of the task, Growth Analysis also interacted with the Government Offices of Sweden (Ministry of Enterprise, Energy and Communications) and the Swedish Agency for Economic and Regional Growth (the managing authority). Growth Analysis has also collaborated with the actors who made the ex ante evaluation of the European Social Fund (ESF) and the European Agricultural Fund for Rural Development (EAFRD) and those responsible for the Strategic Environmental Assessment (SEA).

The programme writing process

An important requirement for the programme writing process is that important documents and guidelines are available and that time and resources have been allocated for analysis, the writing process and establishing political support. The programme writing process, however, lacked clear guidelines, information was insufficient and the timetable was tight. The programmes would have been completed earlier if the conditions had been different.

The tight timetable also had an impact on Growth Analysis’ ex ante evaluation to the ex­tent that the draft programmes reviewed were incomplete. Growth Analysis was unable to provide comments regarding target values and (in many cases) the selection of indicators.

Summary of the ex ante evaluation

The programmes were evaluated on the basis of a series of questions divided into six areas. The questions are based on the European Commission’s guidelines for ex ante evaluations as well as Growth Analysis’ mission statement (European Commission, 2013a, 2013b, the Ministry of Enterprise, Energy and Communications, 2013).

The assessment examines whether the programmes have taken into account the ex ante evaluation questions to a sufficient extent. This does not preclude the possibility of there being room for improvement.

(a)  Overall assessment

The programmes are considered to have taken into account the questions that the ex ante evaluation examined. The selections of priority axes are justified and in line with both the EU’s objectives and country- and region-specific priorities. Overall, the programmes are judged to have an adequate intervention logic and that there is an internal coherence from analysis to the selection of priority axes and supported actions. Some programmes, how­ever, need to be supplemented with regard to the selection of indicators and target values.

(b) The analysis

The analysis of the programme areas is in general considered to be relevant and based on reliable data and statistics and experience from previous programme periods. In most cases, however, there is room for improvement when it comes to summarising and clari­fying the challenges of the programme area.

The importance of the regional analysis ought to be emphasised and made clear in the work with the programmes. The fact that the programmes are explicitly directed by the EU's common objectives, as well as country and region specific priorities, implies that the importance and usefulness of the analysis can sometimes be questioned. Placing the analy­sis in an annex and not in the main text of a programme also gives an indication of the status of the analysis.

(c)  Priority axes

The selection of priority axes is assessed to generally conform to the EU’s common objec­tives, country specific recommendations and the challenges facing the programme areas. The programmes also point out the areas of cooperation with other EU structural funds and programmes.

(d)  Sustainable growth/horizontal principles

The operational programmes contain common sections of text regarding the horizontal principles. These state that the horizontal principles in all programmes will be taken into account through (i) the selection criteria, (ii) support to project applicants, (iii) training of project participants, and (iv) monitoring and evaluation. This approach entails taking the horizontal principles into account in the implementation of the programmes.

All the programmes have also taken the horizontal principles into account in both the design of the programme (e.g. that the analysis includes gender statistics) and the imple­mentation of the programmes.

(e)  Target values and the performance framework

For some programmes the target values are not realistic, and some programmes needs to more clearly justify their selection of target values in order for them to be considered rea­listic.

In the cases where the target values are considered realistic, the milestones in the perfor­mance framework are also considered realistic.

(f)   Selection of indicators

The indicators used are either output or result indicators.

Output indicators are selected from a list of indicators provided by the Swedish Agency for Economic and Regional Growth and consisting primarily of the EU’s common indi­cators.

The operational programmes include selections of output indicators which for the most part are considered relevant, but they sometimes do not provide a sufficiently good repre­sentation of the intended actions to be supported. Some of the programmes ought also to be supplemented with one or more indicators that monitor the intended outcome of the measures. Since the ERDF has a clear aim to contribute to increased employment, this ought to be reflected in the selection of indicators.

Result indicators are selected from a list provided by the Swedish Agency for Economic and Regional Growth. The use of result indicators is new for the 2014-2020 programme period and it is the responsibility of each individual member state to develop appropriate indicators. In Sweden, the Swedish Agency for Economic and Regional Growth developed the result indicators in dialogue with those responsible for drawing up the programmes, in which Growth Analysis and a number of other authorities acted as the reference group.

Growth Analysis considers that in many cases the use of result indicators will not provide an indication of whether or not the supported actions have had any impact. This assessment is not based on the programmes’ selections of indicators, but because the result indicators are to measure changes at the level of the programme area.

Disregarding this condition, the result indicators selected are in general considered to be relevant and to provide a good representation of the specific objectives. The combination of the selected output and result indicators is in general deemed to be logical and able to contribute to clarifying the programmes’ intervention logic.

(g)  Intervention logic and internal coherence

All programmes are assessed to have an adequate intervention logic. However, the inter­vention logic and the internal coherence could in many cases be improved by, for example, more clearly describing the programmes’ challenges and the actions to be supported. Well described programmes in terms of the intervention logic and the internal coherence will subsequently make it easier for future project participants to focus on programme objec­tives.

(h)  Added value and complementarity (the national programme)

The use of a national ERDF programme is new for the 2014–2020 programme period. The national programme differs from the regional programmes in that it operates at the national level and is thus intended to add value that is not possible to generate in the regional pro­grammes.

The national programme is expected to add value to the work of the regional programmes by providing support and implementing measures that cannot or ought not to be limited to a regional perspective.

Ex ante evaluation of the monitoring system and administrative capacity

According to the guidelines for the ex ante evaluation, the monitoring system and admin­istrative capacity are also to be assessed. The evaluation is based on a questionnaire sent to the managing authority (the Swedish Agency for Economic and Regional Growth). The questionnaire was based on the Commission’s guidelines as well as past experience (Euro­pean Commission, 2013a, Tillväxtanalys, 2013 and Tillväxtverket, 2011).

The managing authority is planning to, and has already, implement a number of changes compared to the previous programme period. The managing authority has showed that they are aware of the deficiencies experienced in the previous period and that they are planning to mitigate these. Therefore, it is the assessment of Growth Analysis that the managing authority has the capacity to be responsible for the administration and the monitoring sys­tem during the 2014–2020 programme period.

Growth Analysis assessment is that in many cases the use of result indicators will not be usable as an indication of whether or not the ERDF’s efforts have had any impact. The result indicators should therefore be used with caution, and should only be used alongside the output indicators. Growth Analysis is therefore of the opinion that the result indicator system needs to be monitored and evaluated.