Step-by-step introductory guide to planning an evaluation
About the author
Dr Lesley Paterson is a strategic and creative leader, practitioner and evaluator with over 20 years’ experience in public and community engagement and knowledge exchange (KE). Has led a whole variety of engagement programmes and projects: including PI for the £1M+ Wellcome-funded grants scheme Enriching Engagement (University of Oxford) to nurture high-quality engaged research; project lead for the complex, multi-partner project Curiosity Carnival (500+ researchers; 150+ activities; 7 venues; engaged 10,000+), in addition to facilitating participatory and dialogue engagement work.
Lesley is an Honorary Research Associate, Science & Technology Studies, University College London; UK Evaluation Society member; REF 2021 Impact Assessor; KE Concordat Evaluator; a #ResearchRevolutionary and was previously the University of Oxford’s Head of Public Engagement with Research; her first public engagement role was working for a sheep (called Dolly).
She is the Director of the Engagement Associates: a consultancy specialising in Engagement; Exchange & Evaluation.
Twitter: @lesleyapaterson
Linked-In: linkedin.com/in/lesley-paterson
Introduction
E-V-A-L-U-A-T-I-O-N. It’s a term that makes some people shudder, but evaluation is hardwired into all of us – to understand what works and what does not and learn from our successes and challenges and what did not work well or at all.
Me – I love evaluation – you learn stuff. You’ll only put your tongue onto the metal casing of an old-school frozen ice cube tray once. As you learn from that experience that it gets stuck there; it’s very uncomfortable and worrying and made you cry* a lot [yes, this may be a true story, Lesley, age 4, Linwood, Paisley, Scotland, or *‘greetin’ as I would have said then in colloquial Glaswegian].
Something that I am always keen to shout out from the rooftops is that it’s not a judgement from ‘on high’ or a stick to beat yourself up with. It is a process, slightly more formal than the tongue-on-ice-cube-tray scenario, by which:
- to collect evidence and enable critical reflection to understand the dynamics and effects of the project.
- learn from the experience to understand what works and does not work (and most importantly – why?) and what can be improved.
- explore progress, or otherwise, towards the project’s objectives.
- gain indicators, evidence and data to better understand and articulate the outcomes and impacts i.e., what difference has been made as a result of the engagement work; to whom and why?
What’s more, evaluation findings have so much more value than for writing a report for your funder, although that too is important apparently 8-), and the data, findings and evidence can be used:
- to share, discuss and reflect upon with your project team*, partners* or pubic/community participants* and so continue the engagement and relationship-building after the project has completed [*who can, understandably, sometimes feel a little ‘dropped’ once an in-depth or longer-term project has finished, if there is no further contact]
- as part of a research publication or other output [also see in step 7 on ethics re: publishing evaluation findings]
- to create case studies to share, summarise and raise the profile of your work and that of your team
- to support future funding applications or business cases (for example, for engagement staff roles or project support) to demonstrate the genuine potential for additional beneficial outcomes and impacts, if additional funding and resources are invested.
Planning an Evaluation
The following guide maps out the key steps in planning, conducting and reporting on a well-thought through, effective and feasible evaluation. Do not be tempted to select your evaluation tool/ method, without going through these steps first. That’s an order! :)
Ideally, evaluation planning should take place at the beginning of planning the engagement project or activity, however, even if it’s already started or been running for many years, then it is still most definitely worthwhile doing.
There are a number of key steps to planning an evaluation that are summarised in this downloadable template that guides you, step by step, through planning your evaluation, with additional guidance and top tips for each step provided in the following sections.
Part 1
- Step 1: Objectives
- Step 2: Formative or Summative Evaluation
- Step 3: Evaluation Questions
- Step 4: Outputs and Outcomes
Part 2
- Step 5: Resources
- Step 6: Evaluation Methodology & Tools
- Step 7: Values and Data Protection
- Step 8: Conduct your evaluation
Part 3
- Step 9: Synthesise and visualise the data
- Step 10: Reporting; sharing and acting on the results
- Step 11: And relax…….
Evaluation Tools
Some handy tools you can consider whilst planning your evaluation