Trustworthy AI

Artificial Intelligence (AI) is at the heart of Cognitive Systems. The development of trustworthy cognitive systems therefore starts with trustworthy AI. Recent developments such as the European Union's AI Regulation show the increasing importance of developing strong AI solutions that society can trust. The Fraunhofer Institute for Cognitive Systems IKS focuses its research on aspects that are particularly relevant for cyber-physical systems. Therefore, we research approaches that have a direct impact on the reliability and safety of artificial intelligence for various application areas. These include autonomous driving, production, smart farming, medicine, and quantum computing.

One focus of our research in the area of trustworthy artificial intelligence is on reliable AI-based perception and image recognition, in particular on the interpretation of 2D and 3D data using artificial intelligence. Such approaches are used, for example, for environmental perception in autonomous vehicles and for image interpretation in medical devices. To improve the reliability of perception and image recognition systems, Fraunhofer IKS is investigating approaches for AI monitoring, which records the results of machine learning algorithms during runtime and evaluates their quality.

In order to assess the quality of machine learning algorithms, we also conduct research into the training of artificial intelligence. Here, we focus on data-efficient learning approaches, since most industrial applications and companies do not have sufficient data to train artificial intelligence in the required quality. We consider the use of active-in-the-loop learning, hybrid ML models, and synthetic data.

As part of its work on trustworthy artificial intelligence, Fraunhofer IKS is not only researching reliable AI-based predictions, but also methods for optimal decision making. This is because classical AI-based prediction models are not automatically suitable for generating reliable decision recommendations - instead, causal informed predictive modeling has to be integrated in order to be able to make well-founded decisions.

Using time series analysis, we evaluate series of past data to make reliable AI-based predictions for future developments. The application areas of such approaches are manifold: They are used, for example, to support medical diagnosis and predictive maintenance of production facilities, machines and devices. We develop methods for robust, reliable and early prediction of future events. Such events can be the failure of individual components in machines or the likelihood of a physical illness. In addition, we integrate methods into our AI solutions that provide defensible and actionable recommendations for decision-making through causal inference.