Can AI be developed in a safe way? Meet the Warwick alumna contributing to the challenge

Artificial intelligence is rarely out of the headlines. This week Microsoft founder Bill Gates declared the development of AI “as fundamental as the creation of the microprocessor, the personal computer, the Internet, and the mobile phone.”

You may not be using ChatGPT to help with exam answers, or creating art with Midjourney – but you are very likely to be already interacting with aspects of AI in your everyday life – maybe through the algorithms that select which show to recommend you view next on your streaming media; or which of your friends’ posts you will see on social media; or through applying to a firm which uses AI to assist in hiring decisions. Or perhaps you have had a medical diagnosis backed by AI-assisted computer screening.

Like Bill Gates, many researchers think we are on the horizon of an AI revolution. As well as benefits, there are challenges – are there unethical uses of AI? A machine intelligence could predict whether a person is likely to commit a crime before they do so, tempting authorities to imprison them in advance. Among others, the systems can unintentionally learn to replicate unhelpful stereotypes and bias.

One of the researchers working on ways to ensure the next stage of AI development is helpful, not harmful, is Warwick Economics alumna Charlotte Siegmann (PPE 2021).

Taking recent concerns over inappropriate responses by ChatGPT to users as a starting point, Charlotte explains the challenges:

“ChatGPT was not unexpected - similar base models have existed for a few years now.

“‘Sydney’ was likely a poorly fine-tuned model that never posed a real danger to anyone. However, something very important can be gleaned from observing Sydney and other models: over the next 1, 3, or 10 years, models will become increasingly capable but remain likely unsafe.

“This is because we still do not fully understand current models, and we have not solved the bigger problem of how to ensure that AIs do not do things we do not want them to do. This problem is challenging for several reasons:

• The complexity and difficulty of interpretability.
• AIs can learn incorrect goals unnoticed by humans during the training process - this is called Goal Misgeneralization.
• Humans can’t fully specify what they find desirable - this is known as Reward Misspecification,

“There are many open questions in AI safety and AI governance that researchers and policymakers need to address. What is happening within big models? How can we guide them to elicit latent knowledge that they don’t necessarily reveal through simply prompting? How can we avoid deceptive models?

“Similarly, in AI governance, we need to understand how this technology will develop, how to evaluate its safety, how to incentivize labs to invest in safety and how to mitigate disrupting effects on the epistemic environment, job market or national security.”

Charlotte is about to take up a PhD position at MIT focusing on the safety of transformative AI systems, as well as governance, working together with scholars in both fields. But she has already contributed to the public debate, as co-author (with Markus Anderljung) of The Brussels Effect and Artificial Intelligence: How EU regulation will impact the global AI market.

The report takes a deep dive into the EU’s ambition to set the global standard on AI regulation, following its success in setting the global benchmark for data protection with the GDPR, and explores whether such a “Brussels Effect” is likely.

Charlotte and Markus argue that the EU’s proposed regulations are especially significant in offering the first and most influential operationalisation of what it means to develop and deploy trustworthy or human-centred AI.

“If the EU’s plans are likely to see significant global diffusion, ensuring the regulations are well-designed becomes a matter of global importance,” Charlotte explains.

Reflecting on her time as an undergraduate and how her interest in AI research was sparked, Charlotte says:

“I started thinking about transformative AI in my first year of University and enjoyed discussing the issues with fellow Warwick students. I also attended a summer school on the topic. The economics perspective came later - I first became interested in microtheory in the Game theory course with Costas Cavounidis in my second year.

“During the first wave of the Covid pandemic, I interned with the Future of Life Institute. Together with a policy expert at the institute, I worked on a consultation response to the EU AI Whitepaper, a roadmap laying out the EU’s response to AI technology. Back then, the work on transformative AI or even human-level AI was more speculative - ChatGPT or GPT-4 did not yet exist.

“Since graduating I have been working as an economics predoc at the Global Priorities Institute at the University of Oxford. The Institute combines a philosophy and economic research group and we focus on research that can inform prioritisation efforts of actors wanting to do the most good.

“I did research on the longtermism paradigm, ways of influencing the far future from existential risk reduction to population growth. As a predoc, I organised research workshops, collaborated with others on research projects and shared my research with colleagues both within and outside the institute, among others in Canada and Japan.

“It’s very exciting to work at a young research institute at which we support each other’s work and a lot of collaboration is happening.”

We wish Charlotte every success as she begins her PhD research later this year.