Our teams from Warwick DCS have won both the 1st and 2nd place at the Zero Cost NAS Competition held in conjunction with the AutoML'22 conference.

Over the recent year, Neural Architecture Search (NAS) has attracted a lot of attention. While being able to automate the discovery of better performing neural architectures than hand-crafted ones, it comes at a great price, requiring thousands of GPU hours to perform the search. The Zero Cost NAS competition challenges the participants to design efficient proxies for NAS, using negligible computational resources to evaluate neural architectures.

In collaboration with the AutoCAML team at Samsung AI Cambridge (led by Dr. Hongkai Wen), our research students, Lichuan Xiang and Youyang Sha, proposed new zero-cost NAS metrics that exploit the compressibility of neural networks. Our metrics are extremely efficient to run (reducing search cost from weeks/days to minutes), and achieves impressive results across multiple search spaces and datasets. In the competition, our teams won both the 1st and 2nd places (using different scoring functions), and the performance gap with the 3rd winning team is almost 2x. Checkout our poster here.

This year marks the 25th anniversary of the operation of the UK’s XMaS (X-ray Materials Scattering) user facility at the European Synchrotron Research Facility (ESRF) in Grenoble. It has been directed for all that time from Warwick and Liverpool University Physics Departments and has provided hundreds of UK scientists (and many from further afield) with the opportunity to do leading research in a truly world-leading international centre.

XMaS came about when the first of the world’s ultra-bright synchrotrons was being designed and built in Grenoble, France in the early nineties. The ESRF’s bending magnets were originally designed to simply steer the electron beam around the synchrotron ring between the newly developed insertion devices. It was soon realised that they were a potent source of synchrotron radiation which could exploited as new beamlines if funded by national groups. The UK took advantage, with Malcolm Cooper, here at Warwick, and Bill Stirling, first at Keele and then Liverpool, asked to devise a plan and to bid for EPSRC funds. Needless to say the first back-of-the-envelope designs were, with hindsight, rather naive and embarrassingly under-costed but detailed design work by our small project team generated a viable blueprint, which has since stood the test of time.

When it opened for users in the autumn of 1997 it was never, in our wildest dreams, envisaged that it might be still operational 25 years later. Of course XMaS has undergone a continuous programme of improvement and upgrades over the years with including developing sophisticated sample environments and advances in x-ray metrology. A major refurbishment was necessitated by the recent comprehensive upgrade of the ESRF and as a consequence we now have what is virtually a new beamline and fit for purpose for many years to come.

Initially the science case for the beamline was devoted for the study of magnetic materials, very much in vogue in the 1990s. In fact the acronym XMaS stood for X-ray Magnetic Scattering but the facility has since evolved to encompass a broader materials programme (polymers, liquid crystals, catalysts, etc.) using a variety of techniques (spectroscopies, wide and small angle scattering etc.). The facility has four permanent staff and two postdocs on site, who carry out their own research as well as help the visiting research groups run a very diverse range of experiments. Following the retirement of Malcolm Cooper in 2010, XMaS has been led from Warwick by Tom Hase with admin support from Sarah Jarratt.

A team of researchers led by Professor Ioannis Nezis has identified the molecular and cellular mechanisms that regulate selective autophagy in the fruit fly Drosophila melanogaster.

Press Release (31 May 2022)

• Safety Pool™, led by WMG at the University of Warwick and Deepen AI, launches public access to the over 250,000 scenarios via a credit system
• Organisations can now earn credits for their scenario contributions to the database
• Credits can be used to access the entire set of scenarios in the database
• UK’s Vehicle Certification Agency is evaluating the use of Safety PoolTM as part of future type approvals of Automated Vehicles

Leading experts at WMG at the University of Warwick and Deepen AI, have launched credit-based access to the largest public store of scenarios for testing automated vehicles. The Safety Pool™ Scenario Database, features over 250,000 different scenarios. WMG and Deepen AI are also working with regulators internationally to help evaluate the safety of Automated Vehicles using the Safety PoolTM Scenario Database.

Safety remains one of the biggest challenges around self-driving vehicles. Driven by Safety PoolTM ’s vision of safety of self-driving vehicles being pre-competitive, the Safety PoolTM Scenario Database has created an ecosystem to enable sharing of test scenarios amongst organisations worldwide.

The database provides a large range of scenarios in different operational design domains (ODDs i.e., operating conditions) that can be leveraged by governments, industry, and academia alike to test and benchmark Automated Driving Systems (ADSs). Bolstered by scenarios generated by novel scenario generation methods: 1) knowledge-based and 2) data-based, a methodology also mentioned by the EU’s Regulatory Act on Automated Driving Systems, scenarios in Safety PoolTM are focused on uncovering failures in automated vehicles as they capture those edge-case scenarios.

Under the credit system, users are rewarded with credits for submitting scenarios to the database. Contributions are scored based on uniqueness of the scenarios and their validity, and corresponding credits are awarded to the organisation. These credits can be redeemed to gain access to more scenarios. This system encourages users to contribute to growing the database and making more scenarios available to the self-driving vehicle community.

Dr Siddartha Khastgir, Head of Verification & Validation, at WMG, University of Warwick, said: “With the public launch of the Safety Pool™ Scenario Database, we are democratizing critical data that was not easily available in the ecosystem. By aligning with international standards and working closely with regulators and developers in the UK and internationally, we are driven by the mission of making safety of automated vehicles pre-competitive.”

Jamie McFaden, Head of Automated Vehicle Technologies Group, Vehicle Certification Agency said: “The Vehicle Certification Agency, the UK’s Type Approval Authority, recognises the importance of ODD based testing and as a result is evaluating the use of Safety PoolTM Scenario Database as part of the Type Approval process for Automated Driving Systems.”

Coherent with our mission to align with international standards, today, each scenario has been launched in ASAM OpenScenario 1.1 and ASAM OpenDRIVE 1.6 formats. This provides further interoperability between stakeholders using ASAM OpenX standards for their simulation-based testing of automated vehicles.

The scenarios also cover a diverse set of ODD attributes and manoeuvre types such as urban environments, highways, and under varied environmental conditions where vehicles perform different manoeuvres such as cut-ins, overtaking etc. Use cases supported include Automated Lane Keeping Systems, low-speed shuttle, urban level 4, highway ADAS etc. Scenarios can also be efficiently searched using the ODD and Behaviour tags as per ASAM OpenLabel standard.

With Safety Pool™, industry, academic, industry and government experts come together to create the standards that will make the operation of automated vehicles safe everywhere. This builds an environment which enables the uptake of automated and autonomous technology in road vehicles.

Mohammad Musa, CEO, Deepen AI, said:

“By contributing and working within the Safety Pool™ framework, commercial stakeholders across AV fleet operators, OEMs, Tier 1 suppliers and all other Tier 2+ suppliers can massively accelerate verification & validation as well as share insights & edge cases that are very hard for a single stakeholder to collect by themselves.”

Eric Barbier, Head of Safety, Wayve said:

“Safety is core to the development of automated vehicles and everything we do at Wayve. As an industry, it is essential that we collaborate around a common ecosystem for safety assurance. Wayve is excited by the opportunity unlocked by the Safety Pool™ initiative and we look forward to leveraging its extensive database of curated scenarios.”

Since the launch of this pioneering project in March 2021, WMG at the University of Warwick and Deepen AI have collaborated with stakeholders around the world. To date, over 450 organisations have enrolled in the Safety PoolTM Scenario Database. The next step in WMG’s activity is working closely with regulators in the UK and internationally on enabling a safety assurance for automated vehicles through the Safety PoolTM Scenario Database.

Dr Maria J. Alonso, Lead, Automotive in the Software-Driven Era Initiative, World Economic Forum said:

“Collaboration among stakeholders is key to ensure that autonomous vehicles offer the highest possible levels of safety. The Safety PoolTM Scenario Database, with its over 250,000 scenarios and its collaborative approach, provides a platform to contribute to safer mobility on our roads.”

The development of the Safety Pool™ Scenario Database was funded by UK’s Centre for Connected & Autonomous Vehicles (CCAV), Innovate UK and Zenzic funded Midlands Future Mobility project led by WMG, University of Warwick. Furthermore, the WMG centre for High Value Manufacturing Catapult has further supported the continuous enhancements of the database with its support to the Verification & Validation team at WMG.

Notes to editors:

University of Warwick press office contact: Simmie Korotane | Media Relations | Press & Media Relations | University of Warwick | Email: Simmie.korotane@warwick.ac.uk 

Useful Links:

Link to Safety PoolTM videos: https://youtu.be/edxw_VhzAYA

Link to Safety PoolTM website: https://www.safetypool.ai/

Link to Safety PoolTM Knowledge base: https://docs.safetypooldb.ai/ 

See internal vacanciesLink opens in a new window for details of how to apply.