What is Performance evaluation and why is it important?
Under the IVDR Regulation (EU) 2017/746 (IVDR), all IVDs must undergo a Performance Evaluation before being placed on the market. The Performance Evaluation is a continuous process conducted throughout the lifecycle of an IVD, that assesses the Analytical and Clinical Performance, Scientific Validity which forms the clinical evidence for an IVD. The Performance Evaluation is an important aspect of the IVDR, as it ensures that IVDs are safe, accurate, and effective for their intended use.
Steps that Manufacturers should perform for performance evaluation creation
Here are 7 steps that are typically involved in the Performance Evaluation process to help ensure that your device is compliant with IVDR requirements:
Step 1: Assessing Data and Clinical Evidence
An important step in the performance evaluation process is to determine the level of clinical evidence required to demonstrate IVDR conformity for your device. As part of the Performance Evaluation, you are required to demonstrate conformity to the general safety and performance requirements (GSPRs) as set out in Annex I of the IVDR, taking into account its intended purpose, and should include the appropriate justification, validation, and verification of the methods implemented to meet those requirements.
If your device is new to the market, you should assess the key characteristics such as the intended purpose, device classification, state of the art, and risks of your device which will determine the level of clinical evidence required.
For legacy IVDs (those already marketed under the IVDD), you should evaluate your existing data by conducting a gap assessment to determine if the data is compliant with the IVDR requirements. If any gaps are identified with your data, you will need to address if additional evidence is required in your Performance Evaluation documentation.
Step 2: Performance Evaluation Plan (PEP)
The Performance Evaluation process begins with the establishment of the Performance Evaluation Plan (PEP) which documents how you will conduct the Performance Evaluation for your device. The PEP specifies key characteristics of your device which defines the procedure and the approach that will be used to demonstrate conformity to the GSPRs. It should identify the relevant GSPRs and the specification of methods that will be used to demonstrate the Analytical and Clinical Performance as well as the Scientific Validity. The requirements for the PEP are set out in the IVDR under Annex XIII – Performance evaluation, performance studies and post-market performance follow-up, Part A, section 1.1.
Step 3: Scientific Validity
Scientific validity is the association of the analyte that is used to identify a clinical condition or physiological state measured by the device and is documented in a Scientific Validity Report (SVR).
The SVR should establish your device as state-of-the-art and demonstrate the clinical benefit of its use. It should provide evidence of the clinical background, clinical condition, significance of the analyte, diagnostic options currently used, similar and/or equivalent devices, and recommendations from professional societies to support the IVDs’ performance and intended use.
A systematic literature search will need to be conducted to provide evidence of the scientific validity. The literature search method strategy, selection criteria, appraisal, and analysis of the obtained data should be documented in the SVR. The approach and methods that will be used to demonstrate Scientific Validity should also be documented in the PEP in Step 2 – Performance Evaluation Plan (PEP).
Scientific validity may also be demonstrated through the use of existing data, where available while taking into account the generally acknowledged state-of-the-art and includes sources such as but not limited to;
- statements by professional societies,
- expert opinions
- results from proof-of-concept studies.
If any gaps are identified or there is an insufficient level of evidence, it may be necessary to provide a scientific rationale and to generate new or additional data. Ensure to align your SVR with the requirements set out in Annex XIII Part A sections 1.2-1.2.1 of the IVDR.
Step 4: Analytical Performance
Analytical Performance is the ability of an IVD to correctly detect or measure a particular analyte and should be demonstrated using analytical performance studies to verify analytical performance parameters. These Analytical Performance parameters include the following and are set out in Annex I, section 9.1 a) of the IVDR:
Where any of the above parameters are not applicable for your device, a justification must be provided in your documentation.
For IVD Medical device software (MDSW), the following characteristics may need to be considered:
- confidentiality
- integrity
- reliability
- generalisability
- expected data rate or quality
- usability engineering
The Analytical Performance studies should be adequately planned and documented in an Analytical Performance plan to detail the study methods, acceptance criteria, statistical design, and results should be summarised in the Analytical Performance Report (APR). The approach and methods that will be used to demonstrate Analytical Performance should be documented in the PEP under Step 2.
Step 5: Clinical Performance
Clinical Performance refers to the ability of an IVD to accurately identify a particular disease, condition, physiological or pathological process or state in accordance with the target population and intended user in a clinical setting.
Clinical Performance must be demonstrated for your device in relation to all the parameters described in point (b) of Section 9.1. of Annex I of the IVDR, unless any omission can be justified as not applicable.
Demonstration of the clinical performance of a device is based on one or a combination of the following sources:
- clinical performance studies
- scientific peer-reviewed literature
- published experience gained by routine diagnostic testing.
Clinical Performance studies should be performed unless due justification is provided for relying on other sources of clinical performance data such as peer-reviewed literature. Clinical Performance testing can be characterised by the demonstration and verification of the following clinical performance characteristics that are applicable to your device:
- diagnostic sensitivity
- diagnostic specificity
- positive predictive value (PPV)
- negative predictive value (NPV)
- likelihood ratio
- expected values in normal and affected populations.
Where any of the above parameters are not applicable for your device justification must be provided in your documentation.
The Clinical Performance studies should be adequately planned and documented in a Clinical Performance Study Plan to detail the study methods, acceptance criteria, and statistical design, and results should be summarised in the Clinical Performance Report (CPR). Refer to Annex XIII Part A, 2) for requirements on conducting clinical performance studies and Annex I, chapter II, section 9.1 b) for Clinical Performance requirements. The approach and method that will be used to demonstrate Clinical Performance should be documented in the PEP from step 2.
Step 6: Performance Evaluation Report (PER)
In the Performance Evaluation report (PER) you will need to summarise all the results on Scientific Validity, Analytical Performance, and Clinical Performance to verify the demonstration of conformity with the relevant GSPRs to your device as referred to in Annex I of the IVDR. In the PER, you should evaluate the clinical evidence in light of the current state of the art in medicine, the technology, the intended purpose, and the device’s positive benefit-risk ratio. The PER requirements are defined in Annex XIII, part A, section 1.3.2 of the IVDR.
Step 7: Post-Market Surveillance (PMS) and Post-Market Performance Follow up (PMPF)
Post-market surveillance data for your device should be reviewed as part of your PMS system (in accordance with Chapter VII – Post-Market Surveillance, Vigilance and Market Surveillance, Section 1, Article 78 of the IVDR), this should be done regularly and is used to determine the potential impact on the risks, clinical benefit and whether there is a need to update the PER.
As part of PMPF, a plan needs to be created and should consist of planning for any additional testing or monitoring of your IVD’s safety and performance after it has been released onto the market in accordance with Annex XIII, Part B of the IVDR. The results need to be summarised in a PMPF report and these results should also be updated in the PER. If you deem that PMPF is not required, then a justification should be provided and documented within the PER.
Final Thoughts…
The Performance Evaluation should be considered as a continuous process conducted over the lifecycle of the IVD. The safety, effectiveness, and performance of an IVD is maintained by data that is actively and continuously monitored and collected by the manufacturer. This may include developments in the state-of-the-art, post-market information such as complaints, post-market performance follow-up (PMPF) data, direct end-user feedback or newly published research, guidelines, and harmonised standards. Therefore, the PEP and PER should be updated accordingly and, the clinical evidence for your device should be developed as necessary as new data becomes available.
Still unsure what to do? LFH Regulatory can help…
Whether you need help with preparing performance evaluation documentation in compliance with the IVDR, reviewing technical documentation, or performing a gap analysis our team of consultants can provide medical writing or consultancy support to help get your IVD onto the market. Contact us today phone +441484662575 or via info@lfhregulatory.co.uk. Alternatively, you can also simply complete the ‘Contact Us’ section below to see how we can help.
References:
- Regulation EU 2017/ 746 of the European Parliament and of the Council of April 5, 2017 on in vitro diagnostic medical devices
- MDCG 2022-2 – Guidance on general principles of clinical evidence for In Vitro Diagnostic medical devices (IVDs) – January 2022
