A new state of the art £3 million supercomputer, funded by The Engineering and Physical Sciences Research Council (EPSRC), delivered by HPC Midlands+ and supported by Midlands Innovation will provide researchers with a higher capability system to run large scale calculations. Named Sulis, it is the first machine of its kind in the UK to offer a focus on enhanced ensemble computing workflows, addressing an unmet need in the research and innovation sector.
Based at University of Warwick, the new HPC system was provided by high performance computing, storage, cloud and AI integrator, OCF. Sulis is the largest facility of its kind in the Midlands. It replaces the existing HPC Midlands+ cluster Athena in Loughborough and complements other Tier 2 HPC systems which primarily focus on providing a resource for parallel computing.
The HPC consortium, consisting of the Universities of Aston, Birmingham, Coventry, Leicester, Loughborough, Nottingham, Queen Mary University of London and Warwick, will provide engineering and physical sciences’ researchers with access to the new large-scale facility. By running thousands of simulations concurrently in high-throughput mode, time to solution is dramatically reduced allowing users to increase the scope and ambition of corresponding research.
Professor David Quigley at the University of Warwick and Project Lead at HPC Midlands+ explains: “Sulis is an HPC service developed with a strong steer from the HPC Midlands+ user community. Many of our research activities are based on relatively modest simulations that we need to repeat many thousands of times with different inputs. This helps us make predictions based on large samples of data, understand how sensitive our models are to their inputs, and then quantify the robustness of our predictions.”
Sulis will support alternative and complementary ways of exploiting parallelism, specifically high throughput computing. Working through ensembles of calculations needed for some calculation workflows could easily take decades on a single multi-core PC, or many months with university level facilities. However, Sulis will allow researchers to complete workflows such as this in less than a week, and be reactive to availability of new input data.
Dr James Kermode, Director of the Warwick Centre for Predictive Modelling (WCPM) commented: “Making useful predictions with computer models increasingly relies on quantifying the uncertainty in model outputs due to our imprecise knowledge of the form and parameters of the model. One way this can be done is by running ensembles of simulations to assess parametric and epistemic uncertainty. Activities in the Warwick Centre for Predictive Modelling and the EPSRC Centre for Modelling of Heterogenous Systems will benefit enormously from the HPC Midlands+ Sulis Tier 2 facility, which has been purpose-built for the ensemble computing workflows needed for uncertainty quantification.”
Ensemble computing is also used to generate, sample or process large datasets, often for subsequent use as inputs to train modern machine learning algorithms. For this reason, Sulis includes a high-capacity multi-petabyte data storage component, exploiting modern solid-state storage technologies to reduce bottlenecks arising from reading and writing of data. It also includes a large number of graphics processing units (GPUs) - accelerator devices which are themselves now ubiquitous for machine-learning applications.
Andrew Dean, Sales Director at OCF said: “The Midlands is now at the fore front of sustainable, world-class research thanks to the new state-of-the-art HPC platform. We are delighted to have helped develop Sulis which will undoubtedly future-proof the competitiveness of UK researchers in the region, by solving the challenges of computing at ever larger scales and in turn supporting ground breaking projects in scientific research.”
Sulis comprises of 25,728 AMD EPYC computing cores and 90 NVIDIA A100 GPUs equivalent to 1.8 double-precision PFLOPS of computing power, consisting of a number of compute nodes, GPU nodes, High Memory nodes, connected via Mellanox ConnectX-6 HDR100 Infiniband. The system also has 2PB of IBM Spectrum Scale filesystem, provided by both SSDs and hard disk.
Dr Andrew J. Morris of the University of Birmingham’s School of Materials who led Birmingham’s role in the bid said: “This is great news. We have needed a dedicated high-throughput supercomputer in the UK for some time. Just like with weather forecasting, where you run the prediction hundreds of times to find the percentage chance of rain, the same approach is required for lots of other scientific problems. We are delighted that, as part of the HPC Midlands+ consortium we can deliver this exciting new supercomputer to EPSRC-funded researchers throughout the UK.“
Jordan Dorell, a PhD student at the University of Birmingham who will use the new facility said: “Rather than synthesising thousands of possible new battery materials in the lab, which is slow, difficult and expensive, my research project involves predicting thousands of new candidate materials for batteries and testing them first within a computer. Sulis will allow me to increase the number of materials I try and find the best answers quicker.”
Professor Jonathan Hirst, RAEng Chair in Emerging Technologies, University of Nottingham said: “The HPC Midlands Plus tier 2 facility for high-throughput and ensemble computing workflows will significantly augment our research using machine learning and molecular simulation to make drug discovery more sustainable.”
Dr Dmitry Nerukh from Mathematics Department, University of Aston leads a research group explains: “We perform unique large scale molecular dynamics simulations of entire virus at atomistic resolution that includes all constituents of the virus in solution: the protein shell, the DNA/RNA genome, as well as all water molecules, ions and other small molecules present in the virus’ environment; this is only possible using state-of-the-art supercomputers like Sulis.”
According to Professor David Webb, who leads the Aston Institute for Urban Technologies and the Environment: “High performance computing is the backbone to much of the research in our institute, supporting highly complex simulations as well as the application of machine learning techniques. Participation in Sulis ensures that our research can be competitive with the best in the world.”
Professor Sergei Turitsyn, Head of the Aston Institute of Photonic Technologies said: “Machine learning techniques are becoming increasingly important in photonics. For example, they can compensate for the degradation of data-carrying optical signals transmitted over long-haul communication fibre links and provide new tools for the generation and measurement of laser radiation. Access to Sulis will enhance our research capabilities in this field.”
5 November 2021