Monitoring & Evaluation – Six Months In
Coventry City of Culture Trust, in partnership with the University of Warwick and Coventry University and strongly supported by Coventry City Council made a clear commitment to the monitoring and evaluation of Coventry UK City of Culture 2021. This was established during the bidding period prior to the city winning the title. The evaluation programme has also been established in the context of a broader UK policy and organisational environment that has seen a step-change in expectations around evaluation, impact reporting and learning.
In late 2019, the Trust and partners devised a Story/Theory of Change detailing four impact areas of change sought and fifteen measurable outcomes to progress towards this impact. These outcomes were derived from the public consultations leading to Coventry’s successful bid. Since the Trust published the Performance Measurement & Evaluation Strategy (PM&E) in January 2020, the world in which we live has moved into a very different place, the effects of the global pandemic, societal awareness of the Black Lives Matter movement and key challenges over the safety of women have all driven and pushing towards societal change. However, the impacts sought outlined in the Strategy have remained the same. The Story/Theory of Change created a blueprint for City of Culture to make change to the citizens of Coventry and partner organisations to producer targeted work.
The role of the evaluators had been set to evaluate the impact, legacy and learning of a very substantial scale and range of culturally inspired activities. However, the reality has been that the timing, scale, range, and nature of those activities has been a shifting kaleidoscope as the Trust and its programming teams have responded to the immense challenges of the pandemic. In turn, the monitoring and evaluation programme has sought to adapt.
A key example of this would be the signature event Coventry Moves in June 2021. In evaluation terms this moved from 'mass footfall and experiences in the city centre' to 'numerous continuously moving cultural displays throughout the city seeking engagement but not crowds', last-minute route changes and a major shift to online coverage including a 'coming together online key moment' required an extension of online data capture methodologies and metrics.
At the core of the PM&E Strategy has been the social ambition to understand not just the numbers of events, revenue and participants but to dig deeper to understand the patterns of who, where, when and how residents have differentially benefited and crucially why particular cultural interventions have impacted or not. The evaluation of this ambition is enabled by being able to geocode data to postcodes and subsequently neighbourhoods. This allows evaluators to see at a neighbourhood level what difference City of Culture is having hyper locally and geo-demographic representations of engagement and participation across the city down to neighbourhood level including local perceptions of quality and sentiments towards the City of Culture programme and its impacts.
To fully capture and tell the story of City of Culture, the evaluation team has also been busy commissioning a number of pieces of work which are now in full flow. A number of Case Studies has been commissioned from researchers from across Warwick and Coventry, these focus on civic pride, wellbeing, the environment, policing, and cultural leadership and citizenship. As mentioned, these studies are now underway, and results will be available around June 2022. Further to this the Trust has also commissioned an externally contracted Economic Impact Assessment which will run until 2024. The initial baseline report for this has highlighted that the UK City of Culture title has already to date seen approximately £172.6 million investment into the city secured which in turn has led to in excess of £500 million of regeneration activity. Wanting to really demonstrate not just the economic impact but the social impact, the Trust has also commissioned a number of Social Return On Investment studies into projects from across the City of Culture Programme.
Now that we are six months in since the commencement of Coventry’s year in the spotlight, we are beginning to see early positive indications that City of Culture is having a positive impact on the lives of communities and citizens in the city. At this stage it is too early to say what this impact is or how longitudinal it will be, but with six months to go early signs look positive. One challenge/tension with the monitoring and evaluation of cultural mega-events is the understanding of impact and the immediacy required for that impact to be demonstrated to feed into key planning for activity post the point of intervention. The evaluators are currently at the stage of seeing the indications from short term impact as demonstrated in the timeline below.
The true picture and extent of long term impact will not be known for some time but thanks to significant planning, the systems and methods are in place for this to be captured and demonstrated.
But in the six months since Coventry City of Culture kicked off on Saturday 15 May while Covid-19 restrictions were still in place there has been activity across every neighbourhood in Coventry. Just over 141,000 tickets have been issued for live events taking place in the first six months, with an estimated further 52,000 attending un-ticketed, free events. 260,000 people have also engaged with City of Culture events online due to an increased focus on live streaming. More than 1,100 City Hosts have been fully trained and deployed, racking up over 12,000 volunteering hours and in the first six months, the Trust has awarded £4.5 million to projects led by local artists and organisations. With six months to go it will be interesting to see how the short term impacts turn into long term impact for the city and its citizens.
Monitoring and Data Manager, Coventry City of culture Trust
Mark is responsible for the capture of monitoring data for the evaluation of UK City of Culture 2021 which is being undertaken by the University of Warwick and Coventry University.