Without statistics, we couldn’t know whether one drug is more effective than another one, predict the weather or use our phones.
The archives story began with a conversation between Professor Jim Smith (pictured below, with Martine) and various people from the Alan Turing Institute and the British Library. Jim, Manuele Leonelli and I had developed a decision-support system that used a Bayesian model and could, in theory, give people the information they needed to act in ways guided by statistically calculated probability and levels of likely risk. Suddenly, Jim was talking to people who could benefit from this in practice.
And as it turns out, digital archiving is far more exciting than I realised. It offers some obvious benefits over physical record keeping, but comes with all kinds of other dangers, from file corruption to tape failure and obsolescence. What’s less appreciated is that digital archiving is still a pretty new field, despite the sheer mass of digital information being recorded today.
The professionals broadly knew what was likely to keep archives safe and sound, but they didn’t have enough data to quantify the risks or advantages of particular systems – and when you need funding to improve your archiving, you want hard, cold stats to back up your business case.
That’s where we came in. Our DiAGRAM tool allows archivists to investigate potential mitigations to digital preservation risks in their own circumstances and make an evidence-based case for the resources they require.
There’s plenty of detail on the tool online, but it essentially lets you look at combinations of different steps – like sending staff on training courses or copying data onto new digital tapes – and compare cost-benefit analyses. I ran the project from the Warwick side, helped David Underdown at the National Archive to recruit a full-time member of staff who collaborated with our post-doctoral researcher.
It was a brilliant experience, but not without its challenges. At times we didn’t have any data, meaning we had to use a technique called structured expert elicitation; essentially getting specialists’ lowest and highest plausible estimates for, say, the lifespan of a digital device. But it was a joy to learn so much about digital archiving and the role statistics can play.
Looking back, I’m so proud of my work with the National Archive because it has helped them secure greater funding to keep doing their excellent, vital work.
But I’m also proud of DiAGRAM; it’s now being used in universities and archives to train the next generation of archivists.
Find out more about the department and the research and teaching taking place.
Explore the research taking place at Warwick and the researchers working to better the world.
Making an impact
Tackling issues across the Sciences, Arts, and Social Sciences. Discover how we're changing lives through our research.