Verification and Validation Projects

Projects

Read some of our projects below. If you have any questions or want to work with us, please email wmgbusiness@warwick.ac.uk

Cross-Domain Safety Assurance for Automated Transport Systems

Cross-domain Safety Assurance for Automated Transport System Report Launch

The Verification and Validation team proposed a ground-breaking safety assurance framework that has the potential to be applied across automated transport modes.

In 2022-2023, WMG brought together 38 UK and international organisations in industry, academia, government, and regulation to develop this common safety assurance framework. Nearly 300 key stakeholders across the transport domains of land, sea, and marine, attend and contribute to the development of this report.

The report encourages government policy to tackle similar challenges all three domains face to realise the safe introduction of automated transport systems in a joined-up manner.

The report highlighted that while there are differences between the safety assurance processes of autonomous ships, aircraft, and vehicles, there are also large elements of crossover. This can be leveraged by governments, developers, and manufacturers by aligning safety artefacts across the different types of transport, allowing for greater safety and consumer acceptance.

Please click here to read our report.

DROIDS

DROIDS project

Funder: Conference of European Directors of Roads

DROIDS stands for Digital Road Operator Information and Data Strategy. This project focuses on providing the National Road Authorities (NRAs) with increased knowledge and support to reap optimal benefits from digitalisation as they evolve to become digital road operators operating the physical, operational, and digital road infrastructures. Topics explored included the identification of the data requirements for digital twin applications and how to incorporate the element of trust within a system.

EVENTS

Funder: European Union

Driving is a challenging task. In our everyday lives as drivers, we face complex situations that we need to handle safely and efficiently. The same is valid for Connected and Automated Vehicles (CAVs), which also need to handle these situations, to a certain extent, depending on their automation level and their target Operational Design Domain (ODD, terminology used by SAE J3016). The higher the automation level is, the higher the (user) expectations for the system to cope with these situations are. In the context of this project, these complex situations where the normal operation of the CAV is close to being disrupted (e.g. when the ODD limit is reached due to traffic changes, harsh weather or light conditions, unstructured roads, imperfect data, sensor or communication failures, etc.), are called "events."

Events are creating challenges for CAVs that should be overcome to enable safe and reliable automated driving in such cases. Within our scope and to cover a wide range of scenarios, these events are clustered under three main use cases (shown in Fig. 1):

• Interaction with VRUs (vulnerable road users) in Complex Urban Environments

• Non-standard and Unstructured Road Conditions

• Low Visibility and Adverse Weather Conditions

The vision of EVENTS is to create a robust and resilient perception and decision-making system for CAVs to manage various types of “events” on the horizon. EVENTS will allow overcoming the current CAV ODD limitations due to the dynamically changing road environment (VRUs, obstacles) and/or imperfect data (e.g., sensor and communication failures) while ensuring continuous and safe operation. In EVENTS, if the system or some of the subsystems cannot perform with the expected quality and reliability, an improved minimum risk manoeuvre is triggered.

I4DRIVING

i4Driving

Funder: UK Research and Innovation

The i4Driving project represents a pioneering initiative aimed at revolutionising the safety assessment of automated driving systems (ADS) within Cooperative, Connected, and Automated Mobility (CCAM) solutions. By establishing a robust human road safety baseline for virtual assessments, i4Driving seeks to address the challenges posed by the integration of automated vehicles into heterogeneous traffic environments. At its core, i4Driving embodies two central concepts: the development of a multi-level, modular simulation library integrating existing and novel human driving behaviour models, and the introduction of an innovative methodology to address the significant uncertainty inherent in human behaviours and use case circumstances. This collaborative endeavour draws upon expertise across various disciplines, including traffic engineering, human factors, data science, and computer science.

The i4Driving methodology employs an agnostic approach to modelling, utilising a combination of data mining and advanced cross-disciplinary techniques to guide the development of the simulation library. Through iterative cycles of model development, calibration, and validation, the project aims to strike a balance between model complexity and error, ensuring the plausibility and credibility of the developed models. A key innovation of the i4Driving project lies in augmenting existing human driver models with a 4D cognitive layer, drawing from diverse disciplines such as Human Factors, Road Safety, Traffic Flow Theory, Psychology, Artificial Intelligence, Statistics, Mathematics, and Social Science and Humanities. Sensitivity auditing and techniques for global uncertainty and sensitivity analyses drive the comparison of different modelling assumptions and formulations.

The project's methodology encompasses both inner and outer model development processes, utilising existing datasets and experimental data collected from driving simulator facilities within the consortium. Techniques for standardising experiments and encoding the heterogeneity of driving behaviours further enhance the credibility and applicability of the developed models. Through interdisciplinary collaboration and innovative methodologies, i4Driving aims to accelerate the adoption of CCAM technologies while ensuring the safety and reliability of automated driving systems, paving the way for a safer and more efficient future of transportation.

Midlands Future Mobility

Over 7000 people die on our roads each year with countless more injured. Transport is now the number one emitter of carbon. Thousands of people rely on public transport every day and those users of public transport are disproportionately represented by lower social economic groups.

The advent of new technologies offers the potential to address these challenges, but with this comes new challenges such as assuring safety, providing an enabling standards and regulation environment and ensuring that users both trust and want these new technologies.

Midlands Future Mobility (MFM) is an initiative that brings together technology, people and policy to enable the development, deployment and scalability of transport technology.

Our vision is to achieve zero road-incidents, net-zero emissions and inclusive transport in the West Midlands.

Our mission is to create knowledge, methodologies and tools for every stakeholder in transport to enable this vision.

To learn more about us, please visit https://midlandsfuturemobility.co.uk/.

PAVE United Kingdom

Founders: WMG at The University of Warwick, the Department for Business and Trade, the Department for Transport, the Centre for Connected and Autonomous Vehicles (CCAV), and Transport for West Midlands.

Partners for Automated Vehicle Education United Kingdom (PAVE UK) is a nongovernmental and non-profit membership organisation created on behalf of the UK’s Centre for Connected & Autonomous Vehicles (CCAV) by WMG at the University of Warwick.

The PAVE UK initiative aims to build public confidence in self-driving technology through a programme of education and engagement, supporting the UK Government’s ambition to safely deploy self-driving vehicles on the road and its aim to make the UK the leader in artificial intelligence (AI).

PAVE UK is the country’s first non-governmental organisation that advocates for and delivers public education and engagement programmes on automated vehicles. Our vision is to enable public trust and acceptance of self-driving technology through accurate and inclusive awareness and education programmes.

Our missions are:

Learn: Using scientific rigour, we identify and understand different personas and their behaviours in our society. Identify suitable types of content and communication mechanisms based on scientific literature.

Communicate: Physical and digital educational content will be customised for different groups in society to gradually build up their understanding of selfdriving technology, from a generic introduction to its benefits and communicating safety.

Engage: Organise interactive and hands-on activities for the public to engage with self-driving vehicle technology to help build societal trust and acceptance in this new transport innovation.

Click here to read our brochure.

Safety Pool™ Scenario Database

Safety Pool™ Scenario Database is the world’s largest public library, with more than 270,000 scenarios for Automated Driving Systems (ADS) testing, validation, and certification. Powered by WMG, University of Warwick and Deepen AI, Safety Pool™ envisions that the safety of every ADS can be transparently tested, validated, and certified through common processes and infrastructures shared across industry, researchers, academia, and policymakers internationally. Serving the needs of multiple user “personas”, Safety Pool™ provides an independent database of scenarios, driven by the philosophy that safety of automated driving should be pre-competitive.

With over 150 organisations signed up, Safety Pool™ will be used by regulators in the UK for type approval of Automated Driving Systems in the UK.. It also leads the research and fosters the development of common description languages, standards, and mediums that allow for a meaningful exchange of technical information and artefacts across different stakeholders worldwide.

To learn more about Safety Pool™, please visit https://www.safetypool.ai/

Watch our monthly webinar recording on our Youtube channel: https://www.youtube.com/@safetypool227

Scenario, coverage, & Safe AI

Funder: DENSO

This project is contracted by DENSO to undertake multi-phased research activities on 1) STPA-based scenario generation; 2) ODD-based scenario generation; 3) coverage analysis development; and 4) safe AI.

Based on the system architecture, STPA analysis was carried out using domain experts’ knowledge to explore potential hazards within the system. Furthermore, scenarios were also created using the system’s ODD specification and combined with the system’s behaviour competency. Such created scenarios are hosted within the Safety PoolTM scenario database for testing purposes. A multi-pillar-based coverage analysis technique was also developed as part of the project to evaluate the completeness of the scenario-based testing process. In the last phase, the AI safety topic was explored using the perception module of a system and varying detail levels of the input data.

SUNRISE

Funder: UK Research and Innovation

SUNRISE is a EU funded project aims at creating a harmonised safety assurance framework (SAF). The SAF includes scenario database utilising scenarios generated using knowledge-based or data-based approach and represented in open & standardised format. Furthermore the SAF include the creation of concrete scenarios for execution within a test environment, and consequently the analysis of the execution results to explore the failure mode of the underlying system under test.

VCA TYPE APPROVAL

Vehicle Certification Agency Type Approval

Funder: Vehicle Certification Agency

This project, led by WMG at the University of Warwick, UK, is a pioneering initiative aimed at enhancing the safety of Automated Driving Systems (ADS). The focus is on developing a robust testing framework that can effectively evaluate these complex systems.

The project is structured into three work packages. The first two packages focus on Management System Auditing and Performance Assessment. They involve comprehensive research and the creation of training materials for Vehicle Certification Agency (VCA) engineers. The training will provide the foundational knowledge needed before any Type-Approval-Specific training, which would be part of a potential second phase of the project. The third work package will demonstrate how our flexible and scalable Operational Design Domain (ODD)-based assurance framework can be applied to various use cases. This package will also provide foundational training material on how the framework can be implemented.

A key component of our project is the development of scenarios, which are virtual ‘what if’ situations that the ADS might face. These scenarios are stored in WMG’s online Safety PoolTM Scenario Database, which currently hosts over a quarter million scenarios.

Our team consists of experts from various fields, including ADS verification and validation, project management, scenario generation, scenario simulation, systems and safety engineering, software architecture, and human factors.

The project aligns with the high-level milestones requested by the Department for Transport (DfT) and VCA. The outcomes of this project will not only provide great value for VCA but also contribute to the safety assurance of ADS internationally.

Warwick Manufacturing Group

Verification and Validation Projects