Medical Device Integration for Software: What Errors Really Cost You

Medical Device Integration for Software: What Errors Really Cost You

In September 1999, NASA’s Mars Climate Orbiter missed its intended orbit and disintegrated due to atmospheric stresses. An investigation indicated that the Orbiter’s Device failure resulted from two pieces of software (one on earth and one on the orbiter) miscommunicating with different interface setups. One software was set to interpret data as imperial measurements (inches, yards, feet, etc.), and the other was set to interpret data in metric standards. As a result, the orbiter completely miscalculated the point of firing its thrusters, moved too close to the planet’s hostile atmosphere, and was lost. 

NASA’s Root Cause Analysis

From root cause analysis, NASA identified these main issues:

  1. Lack of communication between project teams on units and measurements, plus
  2. NASA’s insufficient integration testing due to the unit oversight

Hindsight is a Wonderful Thing!

The total cost and loss of this project were estimated at a staggering $327 million! If only they had completed on small integration test, it would have saved them a fortune.

Thankfully, there was no loss of human life in this incident. But within this field and the field of medical devices, a small error like this can have catastrophic consequences. 

Wider Validation & Verification Testing:

This is why it is vital to have robust integration testing as a wider part of validation which verifies that software modules within a medical device software are communicating correctly.

Integration Testing: What is it?

Integration testing is defined as a series of tests with the aim of either discovering internal errors in the interaction between software units of the medical device software or errors in the interaction between the medical device software and other software. Integration testing can also include discovering errors in the interaction with other hardware and medical devices.

Integration testing is very important to provide clinical evidence that medical device software demonstrates correct interoperability with hardware and other medical devices or IVDs.

What to Consider:

  1. A well-defined Integration test plan must be written up on what tests and/or test cases will be performed. This needs to include what the requirements are for the tests to pass and what resources or testers will be assigned for the testing. 
  2. There are two main types of integration testing:
      1. The first is Incremental testing which is subdivided into a top-down or bottom-up approach.
    1. The second is non-incremental testing which is also referred to as ‘the big bang method’. Ensure that the testers are competent and trained for the testing required. Deliverables and the timelines for completion and delivery need to be defined in the plan.
    2. Where changes have been made to the software to allow better interoperability between software and hardware, consider regression testing as per IEC ISO 62304. This testing verifies that no defects or errors have occurred after integrating any changes or applications into the medical device software.
    3. Ensure that the test reports generated lays out the scope of the test. The criteria for testing to pass needs to be defined, as well as the data generated from the test, and then conclude with the explanation of whether the test passed or failed. The test report needs to note the developer who carried out the test plus a sign-off from the reviewer/approver for document control purposes.
    4. An investigation will need to be initiated for all failures or defects found at the end of testing. There will be a need to determine a root cause, carry out a risk assessment to establish the severity of the failure, and raise a change control where necessary to document the actions to be taken to fix the issue. Then revise the test documentation to the next version number and re-run the testing. Traceability of the changes made between versions of test documentation is vital. Therefore, ensure that the revision history of the test documents is recorded and easily available.

 

Need help? LFH Regulatory has the expertise in SaMD to provide support to manufacturers on Integration and other testing elements for medical device software. Contact us today.

References

IEC ISO 62304-2015 Medical device software – Software life cycle processes

NASA Solare System Exploration https://solarsystem.nasa.gov/missions/mars-climate-orbiter/in-depth/#:~:text=Mars%20Climate%20Orbiter%20was%20designed%20to%20arrive%20at%20roughly%20the,23%2C%201999.

Written by Ashfaaq Ismail, Regulatory & Quality Consultant. Ashfaaq has over 10 years of experience within the medical device/IVD industry from a regulatory and quality perspective, including GMP and ISO13485. He has spent over 4 years of this concentrating on software as a medical device/IVD.

 

SHARE POST
TWEET POST