Accelerated innovation cycle of a medical robotic device through test automation
A joint success story with Carl Zeiss Meditec AG - Find out how the test duration could be reduced by 96.25%.
Many medical technology companies face the challenge of making their product innovation cycles shorter and accelerating market launches. However, modern medical devices are highly complex and must meet the highest security and regulatory requirements for the quality of the software. For this reason, quality assurance during development plays a crucial role, but is also time-consuming and costly. The increased use of test automation reduces this effort considerably, but standard tools for customer-specific requirements are often missing.
We used the 'Scalable Automated Framework for Firmware Tests'. This is a test automation tool, that fully meets the customer-specific requirements of Carl Zeiss Meditec AG. The tool was developed and validated for the firmware test of robotic devices. Using test automation, time and costs for development and integration tests for the client could be significantly reduced. At the same time the level of software quality could be further increased. Conceived as a platform solution, the framework can be expanded to other components and products at low cost and can therefore be used for a broad product portfolio.
Due to the strong interdisciplinary character of developing medical devices, a deep technical understanding from different areas such as e.g. embedded engineering, software development, software architecture, testing and test automation is needed. In addition, the high regulatory requirements of medical technology had to be considered during development and documented in a meaningful way.
Our test team, which specializes in medical technology, was responsible for the complete conception and development of the test automation solution. In close cooperation with the embedded software developers, architects, and reliability managers at Carl Zeiss Meditec AG, we analyzed the customer requirements and planned and implemented the technical implementations in an agile manner. Efficient scalability and expandability of the test automation were considered right from the start. The framework was validated as a test tool in the form of IEC 62304 and ISO 13485 in cooperation with the customer. The high quality and complete documentation of the framework was ensured throughout the development phase. It can therefore be used both during development and for acceptance tests.
The 'Scalable Automated Framework for Firmware Tests' is used to verify and validate the embedded firmware of robotic surgical microscopes. For this purpose, classic black box test cases were implemented, which run in a hardware-in-the-loop test environment.
The framework communicates directly with the device under test via CAN bus and sends control signals to the device firmware or receives data and signals from the device and records them. The complex device logic was abstracted into an easy-to-use interface so that test cases can be implemented quickly and efficiently.
Two types of test cases - fully automated and semi-automated tests - were implemented based on the framework.
- Fully automated tests do not require any user interaction and can e.g. be run overnight as regression tests. This gives the developers direct feedback on their changes in a short time. For this purpose, the test framework was integrated into the customer's continuous integration environment and can thus be triggered with each update of the firmware or by a timed trigger.
- Semi-automated test cases are largely automated, but also have individual test steps that require user interaction. For this purpose, the framework was extended by a Graphical User Interface (GUI). The test GUI guides the tester through the test case and informs him about necessary manual interventions. The evaluation of the test results and the generation of reports continue to be fully automatic. The tester is thus relieved of a large part of the manual steps, which significantly reduces the complexity and duration of the test execution.
Test reports and associated data, such as e.g. recorded measurement data and logs, are automatically stored in the customer's test management tool. This ensures that the test results are archived and traceable.
With the help of the test framework, the test duration could be reduced from up to 2 weeks to approx. 3 hours (for fully automatic tests), which corresponds to an increase in efficiency of 96.25%. For semi-automated tests, the test duration was reduced to approximately 1 day. These high time and cost savings not only have an effect during development, but also when the product is already being used in the field. For necessary firmware updates, all automated test cases can be carried out within a few hours. Updates can be rolled out faster and more efficiently, necessary impact and Risk analyzes are greatly simplified, further costs are saved. The higher test coverage also increases confidence in the quality of the update.
Another major advantage of test automation is that the tests and the load on the firmware can be much more intensive than if the tests were only performed manually. This additionally increases the robustness of the firmware and confidence in its quality.
With the implementation and use of the 'Scalable Automated Framework for Firmware Tests', we not only achieve an improved quality of the FW components, but also significantly reduce the effort for integration as well as error and regression analyses. A clear step forward for us in terms of efficiency and effectiveness in development. The test team from ZEISS Digital Innovation supported us very professionally and collaboratively - in the implementation and as an advisory partner.
ZEISS Digital Innovation
Health & Life Science Solutions
Contact us to learn more about our holistic quality assurance services and discover more about ZEISS Digital Innovation Health & Life Science Solutions.