Verification and Validation Projects
Safe Autonomy Research Projects
Projects
Read some of our projects below. If you have any questions or want to work with us, please email wmgbusiness@warwick.ac.uk
Cross-domain Safety Assurance for Automated Transport Systems
The Safe Autonomy Research Group proposed a ground-breaking safety assurance framework that has the potential to be applied across automated transport modes.
In 2022-2023, WMG brought together 38 UK and international organisations in industry, academia, government, and regulation to develop this common safety assurance framework. Nearly 300 key stakeholders across the transport domains of land, sea, and marine, attend and contribute to the development of this report.
The report encourages government policy to tackle similar challenges all three domains face to realise the safe introduction of automated transport systems in a joined-up manner.
The report highlighted that while there are differences between the safety assurance processes of autonomous ships, aircraft, and vehicles, there are also large elements of crossover. This can be leveraged by governments, developers, and manufacturers by aligning safety artefacts across the different types of transport, allowing for greater safety and consumer acceptance.
Please click here to read our report.
Please click here to read about our journey on Cross-domain Safety Assurance.
DROIDS project
DROIDS stands for Digital Road Operator Information and Data Strategy. This project focuses on providing the National Road Authorities (NRAs) with increased knowledge and support to reap optimal benefits from digitalisation as they evolve to become digital road operators operating the physical, operational, and digital road infrastructures. Topics explored included the identification of the data requirements for digital twin applications and how to incorporate the element of trust within a system.
Reliable In-Vehicle Perception and Decision-Making in Complex Environmental Conditions (EVENTS)
Funder: European Union
Partners: I-SENSE Research Group, of the Institute of Communication and Computer Systems (ICCS), Institute of Measurement, Control and Microtechnology at Ulm University (UULM), SEAbility, Tecnalia, Perciv AI, HITACHI France, APTIV, HITACHI UK, STELLANTIS, TU Delft
WMG are heavily involved in the vehicle system hazard analysis & risk assessment, self-assessment for perception and localisation systems and Standardisation & international liaison activities of this project. Other partners involved are ICCS, CRF ,UULM ,SEAB ,TECN, TUD ,HITACHI, APTIV & PERCIV.
More information can be found here: Material Hub Archives - Events Project (events-project.eu)
GB Autonomous Vehicle Approval Scheme
Funder: Vehicle Certification Agency
The “GB Autonomous Vehicle Approval Scheme” represents a pivotal initiative driven by the Vehicle Certification Agency (VCA) to enhance the safety and security of Automated Driving Systems (ADS). As the landscape of autonomous mobility evolves, ensuring robust certification processes becomes paramount. This project builds upon the groundwork laid by the Department for Transport’s International Vehicle Standards (IVS) Division, specifically focusing on Work Package 3 (WP3): Scenario Generation, Selection, and Coverage.
The rapid evolution of Automated Driving Systems (ADS) demands robust safety assurance. Rooted in assessing the Operational Design Domain (ODD), the proposed methodology offers a generic framework applicable to ALKS and other ADS types. The project focuses on dual aspects: auditing management systems to ensure credible processes and assessing system performance. Additionally, it lays the groundwork for upskilling VCA engineers, enabling effective application of the ODD assurance framework. Beyond the UK, this initiative contributes to international ADS safety standards.
The project involves three work items. The first two focus on Management System Auditing and Performance Assessment. They involve comprehensive research and the creation of training materials for the Vehicle Certification Agency (VCA) engineers. The training will provide foundational knowledge needed before any Type Approval specific training, which would be part of a potential second phase of the project. The third work item will demonstrate how our flexible and scalable Operational Design Domain (ODD) based assurance framework can be applied to various use cases. This will also provide foundational training material on how the framework can be implemented.
A key component of our project is the development of scenarios, which are virtual ‘what if’ situations that the ADS might face. These scenarios are stored in WMG’s online Safety PoolTM Scenario Database, which currently hosts over a quarter million scenarios.
Our team, consists of experts from various fields including ADS verification and validation, project management, scenario generation, scenario simulation, systems and safety engineering, software architecture, and human factors. The project aligns with the high-level milestones requested by the Department for Transport (DfT) and VCA. The outcomes of this project will not only provide great value for VCA but also contribute to the safety assurance of ADS internationally.
i4Driving
Funder: European Union
Partners: Panteia BV, University of Naples Federico II, Aimsun, Delft University of Technology, Swedish National Road and Transport Research Institute, Automotive Technology Centre of Galicia, ZF Friedrichshafen AG, SwissRE, RDW, Technical University of Munich, University of Aschaffenburg, National Research Council, DENSO
The i4Driving project represents a pioneering initiative aimed at revolutionising the safety assessment of automated driving systems (ADS) within Cooperative, Connected, and Automated Mobility (CCAM) solutions. By establishing a robust human road safety baseline for virtual assessments, i4Driving seeks to address the challenges posed by the integration of automated vehicles into heterogeneous traffic environments. At its core, i4Driving embodies two central concepts: the development of a multi-level, modular simulation library integrating existing and novel human driving behaviour models, and the introduction of an innovative methodology to address the significant uncertainty inherent in human behaviours and use case circumstances. This collaborative endeavour draws upon expertise across various disciplines, including traffic engineering, human factors, data science, and computer science.
The i4Driving methodology employs an agnostic approach to modelling, utilising a combination of data mining and advanced cross-disciplinary techniques to guide the development of the simulation library. Through iterative cycles of model development, calibration, and validation, the project aims to strike a balance between model complexity and error, ensuring the plausibility and credibility of the developed models. A key innovation of the i4Driving project lies in augmenting existing human driver models with a 4D cognitive layer, drawing from diverse disciplines such as Human Factors, Road Safety, Traffic Flow Theory, Psychology, Artificial Intelligence, Statistics, Mathematics, and Social Science and Humanities. Sensitivity auditing and techniques for global uncertainty and sensitivity analyses drive the comparison of different modelling assumptions and formulations.
The project's methodology encompasses both inner and outer model development processes, utilising existing datasets and experimental data collected from driving simulator facilities within the consortium. Techniques for standardising experiments and encoding the heterogeneity of driving behaviours further enhance the credibility and applicability of the developed models. Through interdisciplinary collaboration and innovative methodologies, i4Driving aims to accelerate the adoption of CCAM technologies while ensuring the safety and reliability of automated driving systems, paving the way for a safer and more efficient future of transportation.
Midlands Future Mobility
Funder: Jointly funded by the Centre for Connected and Autonomous Vehicles and the industrial partners
Partners: Amey Consulting, AVL, Costain, Coventry University, Immense, MIRA, National Highways, Transport for West Midlands, Vodafone, Wireless Infrastructure Group
Over 7000 people die on our roads each year with countless more injured. Transport is now the number one emitter of carbon. Thousands of people rely on public transport every day and those users of public transport are disproportionately represented by lower social economic groups.
The advent of new technologies offers the potential to address these challenges, but with this comes new challenges such as assuring safety, providing an enabling standards and regulation environment and ensuring that users both trust and want these new technologies.
Midlands Future Mobility (MFM) is an initiative that brings together technology, people and policy to enable the development, deployment and scalability of transport technology.
Our vision is to achieve zero road-incidents, net-zero emissions and inclusive transport in the West Midlands.
Our mission is to create knowledge, methodologies and tools for every stakeholder in transport to enable this vision.
The West Midlands is at the heart of the UK’s Automotive industry, manufacturing industry, transport industry and has a leading local authority, Transport for West Midlands, in transport innovation. Lead by WMG, who bring world leading research innovation, our partners span research, connectivity, infrastructure, local authorities, road operators and vehicle testing. Together we have created a backbone of enabling technology across the West Midlands – including enabling testing infrastructure in campus and road environments, safety assurance methodologies for CAM, a range of connectivity solutions (including Small Cells, 5G, DSRC-units), research and lab capability, data, and modular testing vehicles. Through active projects and use cases we are addressing the challenges of integration to the existing mobility system, setting a blueprint for the UK’s future mobility path.
The UK’s world-leading testing facilities
Midlands Future Mobility is part of CAM Testbed UK, six core facilities within a three-hour drive offering a comprehensive set of capabilities for the testing and development of connected and automated mobility (CAM) technologies. Funded by the Centre for Connected and Autonomous Vehicles and coordinated by Zenzic, CAM Testbed UK is able to offer interoperability to customers that is unrivalled worldwide.
To learn more about us, please visit https://midlandsfuturemobility.co.uk/.
PAVE United Kingdom
Founders: WMG at The University of Warwick, the Department for Business and Trade, the Department for Transport, the Centre for Connected and Autonomous Vehicles (CCAV), and Transport for West Midlands.
Partners for Automated Vehicle Education United Kingdom (PAVE UK) is a nongovernmental and non-profit membership organisation created on behalf of the UK’s Centre for Connected & Autonomous Vehicles (CCAV) by WMG at the University of Warwick.
The PAVE UK initiative aims to build public confidence in self-driving technology through a programme of education and engagement, supporting the UK Government’s ambition to safely deploy self-driving vehicles on the road and its aim to make the UK the leader in artificial intelligence (AI).
PAVE UK is the country’s first non-governmental organisation that advocates for and delivers public education and engagement programmes on automated vehicles. Our vision is to enable public trust and acceptance of self-driving technology through accurate and inclusive awareness and education programmes.
Our missions are:
Learn: Using scientific rigour, we identify and understand different personas and their behaviours in our society. Identify suitable types of content and communication mechanisms based on scientific literature.
Communicate: Physical and digital educational content will be customised for different groups in society to gradually build up their understanding of selfdriving technology, from a generic introduction to its benefits and communicating safety.
Engage: Organise interactive and hands-on activities for the public to engage with self-driving vehicle technology to help build societal trust and acceptance in this new transport innovation.
Click here to read our brochure.
Safety Pool™ Scenario Database
Safety Pool™ Scenario Database is the world’s largest public library, with more than 270,000 scenarios for Automated Driving Systems (ADS) testing, validation, and certification. Powered by WMG, University of Warwick and Deepen AI, Safety Pool™ envisions that the safety of every ADS can be transparently tested, validated, and certified through common processes and infrastructures shared across industry, researchers, academia, and policymakers internationally. Serving the needs of multiple user “personas”, Safety Pool™ provides an independent database of scenarios, driven by the philosophy that safety of automated driving should be pre-competitive.
With over 150 organisations signed up, Safety Pool™ will be used by regulators in the UK for type approval of Automated Driving Systems in the UK.. It also leads the research and fosters the development of common description languages, standards, and mediums that allow for a meaningful exchange of technical information and artefacts across different stakeholders worldwide.
To learn more about Safety Pool™, please visit https://www.safetypool.ai/
Watch our monthly webinar recording on our Youtube channel: https://www.youtube.com/@safetypool227
Scenario, coverage, & Safe AI
Funder: DENSO
This project is contracted by DENSO to undertake multi-phased research activities on 1) STPA-based scenario generation; 2) ODD-based scenario generation; 3) coverage analysis development; and 4) safe AI.
Based on the system architecture, STPA analysis was carried out using domain experts’ knowledge to explore potential hazards within the system. Furthermore, scenarios were also created using the system’s ODD specification and combined with the system’s behaviour competency. Such created scenarios are hosted within the Safety PoolTM scenario database for testing purposes. A multi-pillar-based coverage analysis technique was also developed as part of the project to evaluate the completeness of the scenario-based testing process. In the last phase, the AI safety topic was explored using the perception module of a system and varying detail levels of the input data.
Scenario development for Automated Driving Systems (ADS) safety validation in Canada
Funder: Transport Canada
The Canadian driving environment is characterised by diverse landscapes, varying weather conditions, and a mix of urban, suburban, and rural roads. While the field of Automated Driving Systems (ADS) represents a paradigm shift in transportation, promising increased safety, efficiency, and convenience, the complexities inherent in these environments introduce novel challenges. This project focuses on developing a meticulously tailored scenario-based testing framework that reflects the diverse and distinctive Canadian driving context.
Scenarios, which encapsulate the scenery and environmental conditions and behaviour of road actors serve as the cornerstone for evaluating the functional correctness and safety of ADS. This project delves into the intricacies of scenario-based testing, scenario creation, and their application in the Canadian context. The work produces a catalogue of essential scenarios for testing in Canada. These scenarios, developed using our innovative data-based and knowledge-based scenario creation methods, form a comprehensive library that is available on the online Safety Pool™ scenario database. This library is not merely a collection of scenarios; it is a carefully curated resource that takes into account data and knowledge from various sources.
The development of this library involved a meticulous process of gathering and analysing data from diverse sources. This includes accident data from Canada’s national collision database, knowledge from consumer testing, such as EuroNCAP, and other standards, such as ALKS and LSAD. Furthermore, the library incorporates scenarios created to target testing against Canadian rules of the road, ensuring that the scenarios are as realistic and comprehensive as possible.
This project, therefore, provides a robust foundation for the safety assurance of ADS in Canada. It offers valuable insights and resources for researchers, car manufacturers, and other stakeholders in the field of autonomous driving. We also provide guidance to build and extend the scenario libraries, ensuring that our work continues to contribute to the safety assurance of ADSs in Canada and beyond.
SUNRISE
Funder: European Union
Partners: TNO, Siemens, ERTICO, TOYOTA, VEDECOM, Renault, Continental, Vicomtech, CVC, IDIADA, Chalmers, RISE, bast, IKA, Deepen, Infineon, AVL, Virtual Vehicle, Universita Di Trento, ICCS
SUNRISE stands for Safety Assurance Framework for Connected and Automated Mobility Systems. The main goal of this project is to develop a harmonised and scalable CCAM Safety Assurance Framework that fulfils the needs of different automotive stakeholders. CCAM stands for Cooperative, Connected and Automated Mobility. This safety assurance framework plays a key role in the project. As shown in the figure below, different stakeholders each have their own specific interest in this safety assurance framework. For example, developers of CCAM systems, but also the entities that certify these systems or that perform research in this field, and even citizens, might have their own specific interest in this safety assurance framework.
At a high level, there are three main components of the safety assurance framework: audit, in-service monitoring and reporting (or ISMR), and performance assurance. Performance assurance focuses on ensuring that the system can perform as intended within its defined Operational Design Domain (ODD), which can be visualised as a set of operating conditions within which the system is designed to operate safely. The audit focuses on ensuring the development process, the tools used, and the management system are adequate. The ISMR focuses on capturing and recording any additional information during the system's deployment for consideration in the system’s future design. All components are part of the safety assurance framework; however, in SUNRISE, we focus on performance assurance using a scenario-based approach and incorporate a virtual environment. Please note that we also have the input layer to the safety assurance framework, which includes the requirements containing ODD and behaviour, as well as other external requirements and test objectives.
The performance assurance workflow comprises three main blocks: scenario, execution, and analysis.
- Scenario Block: This involves creating, formatting, and storing scenarios. Scenarios can be created using knowledge-based (expert insights) or data-based (e.g., UK STATS 19 dataset) approaches. Scenarios are formatted for different stakeholders using common formats like the ASAM OpenSCENARIO/OpenDRIVE and BSI Flex 1889 and stored in a database such as the Safety PoolTM Scenario database.
- Execution Block: Concrete scenarios are obtained and ready for execution, combined with test objectives, and allocated to various testing environments. The framework supports virtual, simulated, and real-world testing environments.
- Analyse Block: Includes coverage analysis and test evaluation. Coverage analysis assesses scenario coverage from multiple perspectives and explores the scenario parameter spaces iteratively to identify failures. Test evaluation checks if individual tests meet criteria like speed limits and collision avoidance. The combined analysis results determine the system's overall safety assurance.
Ready to work with WMG?
Register your interest in our verification and validation research and start the conversation with us.