Solutions for intelligent Testing in Medical Technology

This invention describes a new computer-assisted method of testing automated systems using AI-supported risk analysis.

• Test system with AI that checks the safety of semi-automatic and autonomous systems, providing risk assessments and optimized scenarios.     
• It is used to improve existing test systems based on IEC 62304, ISO 14971, and FDA GLP.
• It is time-saving and cost-effective, because tests can be limited to a few relevant scenarios in a risk-oriented manner (MVTS).
• It is applicable to different levels of automation.
• It demonstrably reduces the effort required for risk assessment and regression testing.

Efficient and transparent testing and safety certification for automated and autonomous medical technology systems.

The growing use of AI for automation in medical technology necessitates innovative methods for verifiably safe and reliable testing of these systems. Testing methodologies must address different situations in medical environments, such as critical feature correlations and exceptional cases. Growing cost pressure and the need for assurance in agile development processes require efficient, risk-optimized regression testing.

Currently, new medical devices and systems must undergo extensive testing in clinical environments over long cycles to validate their safety and effectiveness. This process is extremely time-consuming and costly. Therefore, intelligent, data-driven testing and validation methods are needed to shorten development times while meeting regulatory requirements.

The University of Stuttgart, in collaboration with the Institute for Automation and Software Technology and the Robo-Test Incubator, developed an AI-based testing procedure. The procedure has been patented and is used in various environments. Representative clinical cases are aggregated with the help of AI for these tests, which are used for validation and as an MVTS (minimum viable test set). 

AI-driven, targeted selection of relevant test scenarios drastically reduces the number of scenarios in which a system must be tested. This improves the efficiency of complex and time-consuming tests on test benches or in virtual environments. Linking concrete system responses with normative requirements makes it possible to verify the regulatory compliance and safety of medical systems with a minimal, specifically selected number of test scenarios. The system can be integrated into existing test architectures and quality assurance systems in the medical technology industry.

The intelligent testing process supports medical standards and regulatory requirements. It does so by enabling targeted verification and validation based on clinical scenarios. Examples of these scenarios include ISO 13485 for quality management, IEC 62304 for developing safety-critical software, and ISO 14971 for creating risk-based test scenarios (e.g., edge or worst-case scenarios). For software as a medical device (SaMD), the process provides efficient regulatory support in accordance with the EU Medical Device Regulation (MDR/IVDR) and the FDA's Good Machine Learning Practice (GMLP), which promotes transparency in AI systems.

Thus, the process provides the basis for the efficient, transparent, and cost-optimized quality assurance and approval of intelligent and partially automated medical technology systems.

Ebert, C. et al. (2023). AI-Based Testing for Autonomous Vehicles. ResearchGate link
IAS (2022). Testing Software Systems. University of Stuttgart. Link
Synthetic data generation for the continuous development and testing of autonomous construction machinery, Alexander Schuster, Raphael Hagmanns, Iman Sonji, Andreas Löcklin, Janko Petereit, Christof Ebert, and Michael Weyrich. Link