Advancing Deep Reinforcement Learning Methodology

Much reinforcement research in the past has focused on robotics and physical control systems. However there are several emerging applications of RL where the actions are decisions within a computational process:

1. When working with big data we wish to automate decisions about which parts of a data to process.

2. When working with compute-intensive simulations, we need to make best use of the runs available; i.e. experimental design.

3. When building an automated decision system from data, we need to decide which computations or algorithms to run; i.e. automating data science.

Multi-Agent Deep Reinforcement Learning

WMG-funded Ph.D. Research.

Despite the great growth of reinforcement learning in the last decade, most of its successes have been in single agent domains, where behaviour of other agents is not so relevant. There are lots of application areas where the interaction between multiple agents, which can cooperate or compete, is critical. When applied to multi-agent domains, traditional RL approaches suffer from several problems (e.g. non-stationary environments). It is important to develop new methods for scaling the RL to those environments and for creating AI technologies which are able to interact with both each other and humans.

In this project we will develop novel methods for deep multi-agent reinforcement learning in the context of manufacturing applications such as robotics and computer vision.

Optimisation of Artificial Intelligence Algorithms

Artificial Intelligence (AI) algorithms based on large neural networks currently provide state-of-the-art performance in several computer vision tasks, including real-time object recognition and semantic segmentation. Such recent advances in deep learning motivate the use of deep learning in sensing applications such as autonomous driving. However, the excessive computational and energy consumption requirements remain an important impediment for the deployment on constrained embedded devices. A recently explored solution space lies in compressing - approximating or simplifying - deep neural networks in some manner before use on the device. We will explore general-purpose techniques for compressing any type of very large and deep neural network – including fully-connected, convolutional, recurrent neural networks, as well as their combinations, and obtaining a global view of parameter redundancies. A number of methodologies will be explored, including reinforcement learning.

Real-Time Automated Triage and Prioritisation of Chest X-Rays

Funded by a Wellcome Trust Innovator Award. This project builds on our ChestAI deep reinforcement learning prototype system and aims:

(i) to improve the predictive performance, achieving >98% sensitivity & specificity for all criticality levels;

(ii) to develop production-ready software enabling seamless integration with existing PACS (Picture Archiving and Communication System) and x-ray devices globally;

(iii) to complete a case study demonstrating feasibility of software integration with PACS systems and providing measurable evidence of clinical benefits through a ‘live-access’ validation study.