Learning from evaluation: report for Scottish Enterprise

The evaluation involved a review of approximately 60 evaluations of programmes commissioned by Scottish Enterprise. It aimed to look for the application of learning beyond the original project and how that can be used to improve Scottish Enterprise Network performance. Looks at reviews under the 12 themes of a ‘Smart, successful Scotland’: greater entrepreneurial dynamism and creativity; more e-business; increased commercialisation of research and innovation; global success in key sectors; digital connectivity; increased involvement in global markets; Scotland to be a globally attractive location; more people choosing to live and work in Scotland; improving the operation of the Scottish labour market; the best start for all our young people; narrowing the gap in unemployment; and improved demand for high quality in-work training.

In reviewing the evaluations, learning summaries and executive summaries were used where available. For each theme, the review identified: a list of projects evaluated; market failures that the projects addressed; strategic lessons; future focus – what to think about in the future; and information gaps.

The review finds that there have been improvements in the Network’s use of evaluation evidence, but it reports a number of areas where further work is required: the identification of market failure; appraisal processes; an appropriate measurement framework; comparison with related work; appropriate target setting; and capture of information about the processes as distinct from the output.

The report suggests that a number of steps could be taken to significantly improve the productivity of evaluation processes at three levels: within individual projects, by improving the quantitative aspects of the evaluation process to ensure that they are consistent; between comparable projects there is scope for ‘serial learning’; and across Network activity - as the Network seeks to enhance the benefits it delivers through more complex and sophisticated programmes, learning that takes a ‘big picture’ view has greater value. To improve evaluation processes there is a need to understand how value is added in the evaluation processes and enhance those parts of the process, and also to understand where value is lost and ‘fix the leakages’. Areas to be addressed include: processes and systems; skills in designing, commissioning and managing evaluation projects; and behaviours and culture issues.