Digital Health

The digitalization of the healthcare sector and the introduction of the electronic health record (EHR) give access to an increasing amount of data and open the possibility of using AI to provide medical decision support in ways that were previously not possible. In the safety-critical sector of healthcare, where lives are at stake, ensuring trustworthiness and robustness of AI is crucial. 

Therefore, we develop explainable and reliable AI models for clinical decision support systems, intended to assist in the analysis of patient data. These systems aim to support healthcare professionals with improved decision-making, disease prediction, and individualized treatment plans for patients. Through previous research projects, we have gained extensive experience in handling the challenges of real-world healthcare problems, where data is frequently incomplete, unstructured, limited in size, or distributed among institutions (Zamanian et al. 2023). In close collaboration with medical experts and industry, we use our scientific expertise to technologically advance the healthcare sector and improve patient care.

• Advanced analysis of EHR data using AI methods (e.g., data-driven analysis of patient data in preterm infants)
• AI-based prediction of health complications (e.g., prediction of target lesion failure in coronary artery stent patients (Pachl et al. 2021))
• Time-series analysis for monitoring disease progression in sequential patient data (e.g., vital signs, lab values)
• Optimization of treatment strategies through treatment effect analysis and causal inference (Artikel: "Medizintechnik...”)
• Federated learning enabling collaborative model training with data distributed across multiple medical institutions without data centralization
• Explainable AI (xAI) to provide transparency of AI-driven decision-making (Schröder et al. 2023a, Schröder et al. 2023b)
• Generative AI to assist in complex information structures (e.g., analysis of causal relationships, diagnoses, EHR data)

In medical imaging and diagnostics, AI is already supporting doctors in practical applications, e.g., in the evaluation of X-ray, MRI or CT images. Despite the ongoing progress of digital transformation in Germany and the digitization of the healthcare sector, the development of trustworthy AI, especially in medical imaging remains challenging as only small sample sizes (little data) are available, such as in the case of rare disease. Generally, large or very large sample sizes (big data) are required for the training and testing of AI models, in order to achieve high accuracy and reliability of AI decisions. 

Our research approaches show that using the right AI methods allows to develop very reliable and explainable AI models for image classification, with just a small amount of data. The question of how little data is enough highly depends on the use case and can range from a handful to thousands of images. Besides "classic" AI methods, we are also exploring future technologies like quantum computing, e.g., by developing hybrid quantum-enhanced AI models to classify breast cancer images.

• Explainable AI (xAI) to provide transparency of AI-based image classification and diagnostics  (e.g., Prototype Learning, Concept Learning)
• Human-interpretable AI concepts
• AI models based on domain expert know-how
• Trustworthy AI (despite small sample sizes)
• Sub-task learning
• Quantum-enhanced AI

Ensuring safety and trustworthiness of AI systems is a crucial step before deployment in healthcare facilities. This concern is reflected in the recent trend of publishing standards and regulations such as the EU law for regulating artificial intelligence (AI ACT 2021). The challenge for AI developers and providers is thus operationalizing the high-level standards in the intricate technical level, arguing how an AI system complies with the propounded norms, and compiling valid and transparent assurance cases that supports the tedious and costly process of certification of the AI system.  

With our AI verification framework, we have developed a solution to verify the trustworthiness of AI models based on scientific findings and experience in applied projects. Covering various types of use cases, AI models and data sources, it generates a thorough documentation with arguments and assurances, specific to the AI project, to substantiate compliance with standards and regulations. What is then left for safety and AI experts is to revise and fine-tune the documentation by minimal time and effort. Relying upon the “safeAI” expertise at Fraunhofer IKS which is already proven in automotive and railway industries, the framework provides roadmaps for safe development and effective verification of AI. 

• Validation & verification of AI models, considering regulation like the EU AI Act
• Standardised approaches and benchmarks for AI verification
• Forschung zu sicheren und vertrauenswürdigen KI-Qualitätsmerkmalen