Fraunhofer Institute for Cognitive Systems IKSArtificial intelligence (AI) is of vital importance for autonomous and highly automated driving, because it’s the job of AI-based systems to make sure that autonomous vehicles can cope with complex urban traffic situations. Autonomous vehicles must be able to perceive their environment and respond just right. The systems need to identify pedestrians at all times, for example, and initiate the correct response without fail. Still, the use of AI in autonomous vehicles raises a question: “Is AI demonstrably safe enough for use in safety-critical applications like road traffic?” After all, it’s not always transparent how AI systems reach their conclusions, and even minor changes to the input data or other elements can lead to a completely different result. In road traffic, however, the decisions made by the AI must be clear-cut and easy to verify so that functional safety is guaranteed at all times.
AI assurance: Safe artificial intelligence for autonomous driving
Proving the safety of AI systems must be possible
That’s why the “KI-Absicherung” project for AI assurance, an initiative by the German Association of the Automotive Industry (VDA), has defined its goal of making the safety of in-car AI systems verifiable. To this end, the project partners are developing a stringent, verifiable chain of arguments for the assurance of AI functions in highly automated vehicles. In total, 28 project partners are researching solutions to this challenge. The Fraunhofer Institute for Cognitive Systems IKS is also on board. The project was initiated as part of the VDA’s flagship initiative “Autonomous and Connected Driving” and is funded by the German Federal Ministry for Economic Affairs and Energy (BMWi). It is also part of the German federal government’s AI strategy.
Example: Identifying pedestrians — systematic AI assurance
The companies are working on five subprojects that have one overall objective: the assurance of AI functions in autonomous vehicles.
- Subproject 1: AI function
- Subproject 2: Synthetic data
- Subproject 3: Methods and measures
- Subproject 4: Assurance strategy
- Subproject 5: Project management
The primary use case here is the reliable identification of pedestrians in road traffic. As such, the first subproject involves developing algorithms for identifying pedestrians. In the second subproject, synthetic data is being generated to map the many possible scenarios in which pedestrians can appear in road traffic. This synthetic data can then be used to produce data sets for training and tests, including corner cases — that is, cases where the usual operating parameters don’t apply. The data subsequently serves as the foundation for subproject 3, where researchers are building a toolkit with methods and measures, which they also assess for their safety.
Fraunhofer IKS is contributing its expertise to subproject 4, “Assurance strategy”, which applies and assesses the solutions developed in the preceding subprojects. The objective of this subproject is to develop a methodology that verifies whether the AI can meet the safety requirements and, if so, under what conditions. If the safety requirements cannot be met fully at all times, it is particularly important to have a transparent representation of the remaining risks. These risks, which take both technical and ethical aspects into account, are then incorporated in the reasoning structure. In addition, Fraunhofer IKS is taking the lead in developing a suitable schedule for evaluating the safety requirements and their reasoning structure. Overall, the first four subprojects aim to create a systematic assurance strategy for AI functions.
Lastly, the fifth subproject of the “KI-Absicherung” project engages in an exchange with standardization bodies and certification authorities. The objective here is to reach a common industry consensus as to how the assurance of artificial intelligence can be achieved. With these five elements, the project is working on a crucial prerequisite for autonomous driving in Germany.