Current Research

I am a PhD student in the Mathematics for Real-World Systems CDT at the University of Warwick. My research is in the field of Artificial Intelligence, with a particular interest in Deep Reinforcement Learning; I am exploring new methods for learning in environments with states that are represented as graphs.

Most recently I have been investigating how Reinforcement Learning can be used to generate heuristics for Combinatorial Optimisation and Decision Problems. I have explored the application of Reinforcement Learning in several such domains and have created and implemented algorithms for the Graph Colouring and Graph k-Colouring problems. My results demonstrate that heuristics learned in this way have the ability to generalise effectively to new, previously unseen, problem parameterisations.

Education

• 2019-present: PhD at University of Warwick, researching Reinforcement Learning in graph-based environments
• 2018-2019: MSc in Mathematics for Real-World Systems, University of Warwick (Distinction)
• 2011-2012: PGCE, Institute of Education
• 2004-2008: MSci in Mathematics, Imperial College (First Class Honours)

Teaching

• 2022-2023: Artificial Intelligence, University of Warwick
• 2020-2021: Interdisciplinary Approaches to Machine Learning, University of Warwick
• 2018-2020: Quantitative Analysis for Management, Warwick Business School
• 2014-2016: Head of Year, King's College London Mathematics School (years 12-13)
• 2011-2014: Teacher of Mathematics, St. Marylebone School (years 7-13)

Talks & Presentations

• 2023
  • LION17 conference
    • Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks (video available hereLink opens in a new window)
  • Statistics, Probability, Analysis and Applied Mathematics Seminar
    • Colouring Graphs with Reinforcement Learning (video available here)Link opens in a new window
• 2022
  • Warwick Effective Altruism Society
    • The AI Revolution: The end of life as we know it?
  • Alan Turing Institute Networking Event
    
    • Using Reinforcement Learning to learn a heuristic for the Graph Colouring problem
  • Mathematics for Real World Systems Annual Conference
    
    • Colouring: Not as Therapeutic as Advertised
• 2020
  • Machine Learning, Reinforcement Learning and Bayesian Optimisation Reading Group
    • Reinforcement Learning algorithms
    • Policy Gradient methods
    • Variational Auto-Encoders
• 2019
  • Machine Learning, Reinforcement Learning and Bayesian Optimisation Reading Group
    • Loss functions in ML
    • Optimization Functions in ML
  • University of Warwick Data Science Reading Group
    • Deriving Policy Gradient methods

Other contributions

• Publication: Watkins, G, Montana, G, Branke, J. Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks. arXiv preprint.Link opens in a new window

This paper was accepted to LION17 and will be published in the conference proceedings. The GitHub repo, containing code and datasets, can be accessed here.

• Alan Turing Institute Reinforcement Learning Study Group

In 2021 I was selected to lead a team in one of the ATI's Data Study Groups. In collaboration with the Defence Science and Technology Laboratory (Dstl) we explored whether Reinforcement Learning can learn policies that are able to adapt to changes in the environment. The final report can be found hereLink opens in a new window.

• Master's project: Exploring Memory in Deep Reinforcement Learning for Partially Observable Tasks

Used a pursuer-evader tasks to explore the capacity of RL agents to remember previous states, and demonstrated that Recurrent Neural Networks can improve performance in POMDPs. See report hereLink opens in a new window and GitHub repo hereLink opens in a new window.

• Creator of Warwick University's Machine Learning, RL and Bayesian Optimisation Reading Group

Organised (and frequently delivered) talks that covered key ideas from each of the fields. The meetings gave participants the opportunity to learn, discuss recent research and collaborate on projects.

Information about the reading group can be found here: https://warwick.ac.uk/fac/sci/mathsys/news/readinggroups/machinelearningrg/Link opens in a new window

• Contributor to PyTorch Geometric and NetworkX libraries

Awards & Achievements

• Best presentation - Mathsys Annual Conference, 2022
  
• Top of cohort - Mathematics for Real World Systems Master's Programme, 2018-19
• Terminal finalist - Prize-winner in Europe-wide competition in which participants code algorithms for an AI game

 

---