Test Environment

Safe Autonomy

Overview

To perform the execution of the scenarios, we need an execution environment, which can range all the way from pure virtual to real-world public roads.

The virtual environment enjoys scalability and controllability, whereas the real world offers the exact realism one would encounter during the actual usage of the system. In the Safe Autonomy Research Group, we focus on answering: 1) how to incorporate different levels of environment for safety assurance; 2) which environment to use based on given objectives; 3) how to consolidate safety evidence across environments; and 4) how to validate the credibility of virtual environments.

Through our research and development outcomes, we have also led the development of several simulation-related international standardisation initiatives. Our goal is to become the go-to place (both nationally and internationally) for simulation-based testing of CAVs.

Research

Our 3xD Simulator is one of the world's most adaptable and advanced driving simulators.

It has the capability to replicate complex driving scenarios, changing lighting conditions, communications interference or unexpected events, all in a safe and repeatable environment.

Test environment projects

Learn more about our key research projects on test environment.

DriveSafeAI

Funding value: £1.8 millions
Partner: Wayve

Project overview:

The project DriveSafeAI is part of CCAV’s Commercialising CAM Supply Chain Competition (CCAMSC). The Commercialising CAM programme is funded by the Centre for Connected and Automated Vehicles, a joint unit between the Department for Business and Trade (DBT) and the Department for Transport (DfT), and delivered in partnership with Innovate UK and Zenzic.

The project brings together Wayve’s expertise in developing end-to-end machine learning for self-driving with WMG’s world-leading expertise in verifying and validating the safety of self-driving technologies. The project will develop a set of safety methods, tools, and datasets for self-driving vehicles and create the evidence to underpin future AI regulation and policy. An Independent Advisory Committee will provide feedback on the approach.

Project Outcomes & Impacts:

The project aims to create a set of safety methods, tools, and datasets to ensure the safe deployment of AI software used in self-driving technology, paving the way for the commercial deployment of AV technology.

This includes the evidence base for an AI safety assurance framework that is scalable to different Operational Design Domains (ODDs). Pioneering the novel OASISS (ODD-based AI Safety In Self-Driving Systems) approach, these methods can be easily adapted to new ODDs, ensuring straightforward verification and validation of AI-based software across different regions and deployment geographies. The innovations will benefit the entire self-driving industry and apply to all system architectures that use AI, either in discrete components or fully end-to-end.

Impact and Next Steps:

DriveSafeAI’s outputs will also help to shape emerging regulatory and standardisation frameworks at national and international levels by working closely with regulators and the UK Government through the CAVPASS programme to develop best practice and guidance on AI system safety validation.

Visit DriveSafeAI microsite

SUNRISE

Funder: European Union
Partners: TNO, Siemens, ERTICO, TOYOTA, VEDECOM, Renault, Continental, Vicomtech, CVC, IDIADA, Chalmers, RISE, bast, IKA, Deepen, Infineon, AVL, Virtual Vehicle, Universita Di Trento, ICCS

Project Context and Challenge:

SUNRISE stands for Safety Assurance Framework for Connected and Automated Mobility Systems. The main goal of this project is to develop a harmonised and scalable CCAM Safety Assurance Framework that fulfils the needs of different automotive stakeholders. CCAM stands for Cooperative, Connected and Automated Mobility. This safety assurance framework plays a key role in the project. As shown in the figure below, different stakeholders each have their own specific interest in this safety assurance framework. For example, developers of CCAM systems, but also the entities that certify these systems or that perform research in this field, and even citizens, might have their own specific interest in this safety assurance framework.

At a high level, there are three main components of the safety assurance framework: audit, in-service monitoring and reporting (or ISMR), and performance assurance. Performance assurance focuses on ensuring that the system can perform as intended within its defined Operational Design Domain (ODD), which can be visualised as a set of operating conditions within which the system is designed to operate safely. The audit focuses on ensuring the development process, the tools used, and the management system are adequate. The ISMR focuses on capturing and recording any additional information during the system's deployment for consideration in the system’s future design. All components are part of the safety assurance framework; however, in SUNRISE, we focus on performance assurance using a scenario-based approach and incorporate a virtual environment. Please note that we also have the input layer to the safety assurance framework, which includes the requirements containing ODD and behaviour, as well as other external requirements and test objectives.

The performance assurance workflow comprises three main blocks: scenario, execution, and analysis.

Scenario Block: This involves creating, formatting, and storing scenarios. Scenarios can be created using knowledge-based (expert insights) or data-based (e.g., UK STATS 19 dataset) approaches. Scenarios are formatted for different stakeholders using common formats like the ASAM OpenSCENARIO/OpenDRIVE and BSI Flex 1889 and stored in a database such as the Safety Pool^TMScenario database.
Execution Block: Concrete scenarios are obtained and ready for execution, combined with test objectives, and allocated to various testing environments. The framework supports virtual, simulated, and real-world testing environments.
Analyse Block: Includes coverage analysis and test evaluation. Coverage analysis assesses scenario coverage from multiple perspectives and explores the scenario parameter spaces iteratively to identify failures. Test evaluation checks if individual tests meet criteria like speed limits and collision avoidance. The combined analysis results determine the system's overall safety assurance.

SYNERGIES

SYNERGIES: Real and synthetic scenarios generated for the development, training, virtual testing and validation of CCAM systems

Funding value: €18,636,667.60, funded by EU Horizon (£1,064,591to WMG)

Partners: BMW (DE), IDIADA, Renault (FR), Stellantis (FR), Toyota Motor Europe (BE), AVL (AT), DLR (DE), TNO (NL), University of Genoa (IT), VICOMTech (ES), Virtual Vehicle Research (AT), VUFO Dresden (DE), and University of Warwick (UK),

Project Context and Challenge:

Connected, Cooperative, and Automated Mobility (CCAM) faces major challenges due to fragmented and non-interoperable scenario databases, lengthy development cycles, and regulatory uncertainty. Current validation methods are inefficient, hindering the safe deployment of automated driving systems. SYNERGIES addresses these challenges by building on previous EU initiatives (HEADSTART and SUNRISE) to deliver a harmonised, federated European Scenario Dataspace, enabling scenario-based testing, safety assurance, and data-driven innovation in CCAM systems.

Objectives:

SYNERGIES aims to accelerate the development, training, and validation of CCAM systems through an interoperable, European-wide Scenario Dataspace and Marketplace. The project will:

1.Federate existing and new scenario databases into a unified, dynamic platform ensuring scalable data access.

2.Extend the range of real and synthetic scenarios, including rural, urban, and edge-case environments.

3.Develop AI-based tools for semi-automatic data processing, annotation, and scenario extraction.

4.Establish governance frameworks and business models to ensure sustainability, security, and standardisation.

5.Facilitate broad stakeholder engagement, from industry to regulators and researchers, to promote adoption and interoperability across Europe.

The overall goal is to provide a trusted, user-centred, and open infrastructure supporting continuous validation and regulatory compliance for CCAM technologies.

Approach and Innovation:

SYNERGIES adopts a federated, AI-driven, and iterative approach to build a scalable Scenario Dataspace. It integrates lessons from previous EU projects (SUNRISE, Hi-Drive, FAME) and leverages existing scenario sources (Safety Pool™, ADScene, StreetWise, L3Pilot). The project introduces a Federated Layer Architecture to connect distributed databases securely while ensuring ownership sovereignty and interoperability. AI and generative methods will enhance scenario diversity, covering complex, rare, and adverse conditions. Governance models and open APIs will facilitate cross-sector collaboration. The platform’s Marketplace will provide stakeholders with tools for data processing, scenario generation, and testing. By combining semantic technologies, trustworthy AI, and standardisation input (ISO, ASAM, UNECE), SYNERGIES creates an integrated framework for scenario-based validation and certification of CCAM systems.

Impact and Next Steps:

SYNERGIES will accelerate Europe’s transition toward safe, efficient, and sustainable automated mobility by reducing testing costs, shortening development cycles, and enhancing safety assurance. The project will reinforce EU leadership in CCAM technologies, contributing to international standards and UNECE regulations. Its long-term legacy includes a continuously updated European Scenario Dataspace, an active stakeholder network, and a marketplace for data-driven innovation. By fostering interoperability, trust, and data sovereignty, SYNERGIES will lay the foundation for a harmonised European CCAM validation ecosystem and future large-scale deployments.

Scenario development for Automated Driving Systems (ADS) safety validation in Canada

Funder: Transport Canada

The Canadian driving environment is characterised by diverse landscapes, varying weather conditions, and a mix of urban, suburban, and rural roads. While the field of Automated Driving Systems (ADS) represents a paradigm shift in transportation, promising increased safety, efficiency, and convenience, the complexities inherent in these environments introduce novel challenges. This project focuses on developing a meticulously tailored scenario-based testing framework that reflects the diverse and distinctive Canadian driving context.

Scenarios, which encapsulate the scenery and environmental conditions and behaviour of road actors serve as the cornerstone for evaluating the functional correctness and safety of ADS. This project delves into the intricacies of scenario-based testing, scenario creation, and their application in the Canadian context. The work produces a catalogue of essential scenarios for testing in Canada. These scenarios, developed using our innovative data-based and knowledge-based scenario creation methods, form a comprehensive library that is available on the online Safety Pool™ scenario database. This library is not merely a collection of scenarios; it is a carefully curated resource that takes into account data and knowledge from various sources.

The development of this library involved a meticulous process of gathering and analysing data from diverse sources. This includes accident data from Canada’s national collision database, knowledge from consumer testing, such as EuroNCAP, and other standards, such as ALKS and LSAD. Furthermore, the library incorporates scenarios created to target testing against Canadian rules of the road, ensuring that the scenarios are as realistic and comprehensive as possible.

This project, therefore, provides a robust foundation for the safety assurance of ADS in Canada. It offers valuable insights and resources for researchers, car manufacturers, and other stakeholders in the field of autonomous driving. We also provide guidance to build and extend the scenario libraries, ensuring that our work continues to contribute to the safety assurance of ADSs in Canada and beyond.

CERTAIN

Funding value:€1.08 million
Partners:Applus IDIADA Automotive Technology SA, IDIADA UK Ltd, IDIADA Fahrzeugtechnik GmbH, Universitat Politècnica de Catalunya, BASt, CERTH, DAF Trucks N.V., Emotion3D, ERTICO – ITS Europe, I2CAT Foundation, ICCS, Infineon Technologies AG, Jaguar Land Rover Ltd, Karlsruhe Institute of Technology, University of Leeds, University of Warwick, TNO, Eindhoven University of Technology, Vicomtech, Valeo, ZF Friedrichshafen AG, SWOV, Technische Hochschule Ingolstadt, MAP Transport and Mobility, VAIVE Logistics SL, RWTH Aachen University, SIE-NL.

Project Context and Challenge:
Ensuring the safety, resilience, and societal acceptance of Connected, Cooperative and Automated Mobility (CCAM) systems is one of Europe’s most pressing transport challenges. Traditional validation methods—based on static testing and fixed certification cycles—cannot cope with the dynamic nature of AI-driven systems that evolve through software updates and learning processes. CERTAIN addresses this challenge by developing a continuous, human-centric safety assurance framework for all levels of automation, supporting the safe, trusted, and adaptive deployment of CCAM technologies.

Objectives:
CERTAIN aims to establish a resilient, continuous, and human-centred Safety Assurance Framework (SAF) for CCAM, extending existing methods developed under projects such as HEADSTART and SUNRISE. Its overarching goal is to embed safety, trust, acceptance, and comfort principles into the assurance process, ensuring that ADS and cooperative systems remain safe throughout their lifecycle. Specifically, the project will validate eleven representative sub-use cases spanning SAE Levels 2–4 and multiple environments—urban, highway, and logistics—by combining virtual, proving-ground, and real-world testing. The SAF will integrate trustworthy AI validation, human-machine interaction (HMI) assessment, and in-service monitoring to ensure continuous feedback and improvement. CERTAIN will also deliver harmonised procedures and tools aligned with the UNECE NATM pillars—Audit, Performance Assurance, and ISMR—and will provide guidelines and handbooks that support future regulatory frameworks, consumer assessment programmes, and standardisation efforts across the EU mobility ecosystem.

Approach and Innovation:
CERTAIN adopts a multi-pillar assurance approach consistent with the UNECE New Assessment/Test Method (NATM), building upon the methodological foundation established by SUNRISE. It extends the SAF into a lifecycle-oriented framework where assurance activities continue beyond type-approval through in-service monitoring. The approach integrates scenario-based verification, trustworthy AI evaluation, and user-centred safety metrics for HMI assessment. Technically, the project combines virtual simulation, hardware-in-the-loop, vehicle-in-the-loop, and on-road validation environments within a harmonised toolchain to demonstrate safety across diverse contexts. Innovation arises from linking these domains through continuous data exchange and credibility assessment of models and simulations, ensuring that evidence remains valid and traceable throughout system evolution. CERTAIN also pioneers V2X-based safety validation, lifecycle feedback loops, and trustworthy-AI auditability, enabling regulators and industry to transition from static certification to dynamic, data-driven assurance of CCAM safety.

Impact and Next Steps:
CERTAIN will reinforce Europe’s leadership in safe and trustworthy mobility by delivering a validated, human-centric SAF that bridges research, industry, and regulation. The project’s methods and data assets will shape future EU type-approval procedures, consumer testing (Euro NCAP), and international standardisation (UNECE, ISO, SAE, ASAM). Its long-term impact will include improved interoperability between assurance frameworks, reduced duplication of validation efforts, and enhanced societal acceptance of automation. Looking forward, CERTAIN will extend its framework towards in-service monitoring and digital certification, enabling continuous regulatory oversight and strengthening Europe’s Vision Zero ambitions.

What we can offer

Underpinned by scientific evidence, our research helps you put the users at the heart of your technology developments and deploy innovations in the real world at the highest safety standards.