Impact Case Studies

Common features in high-scoring ICS:

The highest scoring impact case studies were characterised by the below, in addition to significant impact:

• Meeting all eligibility criteria.
• An impact summary that followed the structure and stated the same impact claims as detailed in Section 4.
• Identifying the relevant research findings concisely and clearly in Section 2.
• A clear and compelling narrative explicitly linking the research findings and to the claimed impact.
• The beneficiaries of the impact were clearly described.
• Impact claims were contextualised, to make it easier for the panellist to evaluate the impact outlined.
• The impact claims focussed on a smaller number of coherent and well-evidenced impacts.
• Verifiable evidence (qualitative or quantitative) to support the credible impact claims was provided within the text of the case study and cross-referenced to Sections 2 and 5.
• The reach of the impact beyond the immediate beneficiaries to a much broader audience was demonstrated (where appropriate).

Common features in low-scoring ICS:

Any of the below could result in an ICS scoring lower than expected:

• They didn’t explicitly link the research findings to the impact; this was especially likely to be an issue where the case study focused on a single researcher.
• They mistook esteem or dissemination for impact.
• They didn’t clearly demonstrate the use of research findings as a tool in activism or by third parties (for example, NGOs) to create impact.
• They lacked a variety of robust evidence.
• They evidenced the activity rather than the impact from the activities.
• They used quotes from testimonials without appropriate context.
• They described lots of smaller impacts which made the link to the research findings hard to assess.
• They didn’t provide sufficient context to determine the significance of the change or the extent of the reach.
• They contained relatively superficial impacts.

Underpinning research in high-scoring ICS:

• The underpinning research was aligned with the impact claimed.

• The relevant minimum 2* quality research findings were concisely and clearly described.

• For complex multi-partner research programmes, the submitting unit’s contribution to the research was clearly explained.

• Co-produced research and interdisciplinary research underpinned strong case studies; the most successful were able to delineate and describe the specific contribution of the research produced by the submitting unit’s researchers.

• Excellent examples limited themselves to the six key output references requested and did not further subdivide categories.

Underpinning research in low-scoring ICS:

• The authors focused too much on explaining the research topic or process.

• The ICS was imprecise about which findings of the research were relevant to the impact claimed.

• The underpinning research for impact did not meet the eligibility criteria, either for quality, timeframe, or institutional contribution.

• The ICS drew on the research of a disparate group of researchers, claiming multiple strands of different activity and impacts.

• The contributions of the submitting unit were not clearly articulated vis-à-vis those of other partners and stakeholders, making it difficult to judge who had been responsible for the underpinning research.

• Co-production was not articulated clearly and appeared tokenistic.

Impact narrative in high-scoring ICS:

• The relationship between underpinning research findings and significant impact on defined beneficiaries is explicitly described and backed up by credible evidence in the narrative about the reach and significance of the claimed impact.
• The impact from the research of the submitted unit is clearly distinguished from that of other teams or stakeholders where relevant.
• They demonstrated (rather than just asserted) a clear pathway from the underpinning research to the impact.
• They took care to identify their audiences/beneficiaries, and made judicious selection of strong impacts, including only traceable and realistic claims with convincing demonstration of both reach and significance.
• They effectively pinpointed the nature, scale, and beneficiaries of the impact, describing not just what change, effect, or benefit had occurred (and for whom), but the extent and depth of that change for those affected.
• They showed how research findings had been taken on by users in ways that shifted practices and/or the agenda beyond the parameters originally established by the research itself.
• The strongest case studies presented concise and clear narratives and avoided the use of jargon or highly specialised language.
• They distinguished between impact and the activities that led to impact.
• They were very specific about the nature of the impact claimed.
• They did not ‘over claim’ by providing persuasive quantitative data and benchmarks, where possible, and convincing qualitative data when providing evidence of feedback from stakeholders.
• They made effective use of testimonial evidence through direct quotation.
• The best-case studies provided clear evidence of and familiarity with the underpinning research and the impact to which it led.

Impact narrative in low-scoring ICS:

• The impacts described in Section 4 of the template did not match the title of the impact case study and/or the impact summary.
• They failed to tell clearly the story of how the research findings led to the impact.
• The impact was not clearly or adequately described (only what is in the template can be scored).
• The presentation of a range of impacts detracted from the coherence of the case study.
• They presented activities with the potential to generate impact, but failed to describe or evidence the resulting change convincingly.
• Quantitative evidence was presented without any benchmarking or comparative data.
• Generalisations, or lack of specificity, for example when describing engagement with organisations or impacts.
• Some case studies clearly outlined the significance of the impact but failed adequately to demonstrate its reach.
• Statistics on public engagement and dissemination activities were not always accompanied by explanations of, or evidence for, the effects of those activities in delivering impact, related to the research findings described.
• They did not sufficiently distinguish between pathways to impact and the impact itself.
• They contained extraneous information asserting several types of impact across many places.
• There was a lack of clarity in identifying the beneficiaries of the impact.
• They employed a complex writing style, full of jargon and acronyms.
• They claimed impact for events not taking place within the specified assessment period or included 'predicted' future impact.
• They didn't get the division of lists and narratives within Sections 2 and 3 correct.
• They didn't provide context, or comparators, for evidence provided from social media.

Evidence in high-scoring ICS

• The impact was supported by explicit evidence.
• Provided specific quantitative evidence (properly contextualised with benchmarking) of the reach and significance of the impact in the assessment period.
• Evidence demonstrated how they have fed into a broad range of sectors.
• Supporting evidence was carefully collated and appropriately cross-referenced within the appropriate sections.
• Identified a clear evidentiary link between the underpinning research and the resulting impact.
• Provided a range of sources to corroborate their claims, including, for example, audience surveys, feedback from focus groups, visitor numbers, viewing figures, professional testimony, reports, media coverage, and commentary.
• Testimonials were properly integrated into the narrative and were used to explain and articulate the change experienced, not just to make positive endorsements.

Evidence in low-scoring ICS

• Provided unclear evidence, or a lack of detail for the impact being claimed, e.g. not including quantitative evidence (with appropriate context and benchmarks) alongside qualitative evidence.
• Quantitative evidence was generally less used, with greater reliance on vague testimonials.
• Overly reliant on testimonials, or relying too heavily on a selected quotation(s) from individual users to justify ambitious claims.
• For impact from public engagement, they provided evidence only in the form of footfall, visitor numbers, website hits, and audience numbers without demonstrating any resulting change.