Learning review of strategic evaluations: report for Scottish Enterprise

Scottish Enterprise (SE) Network evaluation reports were commissioned for specific reasons connected with an individual project or programme intervention. The evaluation aimed to extract recent evidence of strategic learning and innovation from SE evaluation reports, in order to inform Network strategic thinking.

The methodology consisted of an examination of recent evidence of strategic learning and innovation from 100 evaluation reports.

Suggests that the SE Network evaluation processes offer important opportunities to learn from past experience as well as providing performance and financial accountability. Presents examples of evaluations that have contributed to SE strategic thinking. Observes that the evaluation processes do not build 'serial' knowledge. Many of the evaluations reviewed devote little time to addressing the market failure that the programme was designed to address and do not comment on whether real impact has been achieved. Finds that the Network evaluation library system does not appear to work and a large numbers of evaluations are never collated centrally. Notes the degree of fragmentation and duplication of evaluation work. Suggests there is currently no-one in the SE organisation with a remit to extract knowledge in ways that can be re-circulated to add value. Includes appendices of nine strategic issue summaries of the new key result areas (KRAs).

SE should consider: identifying the optimum balance in evaluation processes; clarify why evaluations are actually being conducted; questioning whether evaluation is of strategic or operational importance; setting up processes to ensure the learning is captured and recycled across the Network; more standardised quantitative processes and presentation within evaluations; designing qualitative evaluation processes; briefing, and possibly training, its consultant suppliers; maintaining the knowledge resource; and creating new attitudes and practice to evaluation.