Proposed Guidance on IVD Clinical Evidence and Performance Evaluation

The Global Harmonization Task Force (GHTF; www.ghtf.org [6]), created in 1992 in an effort to promote the international harmonisation of medical device regulation, is a voluntary group of representatives from national medical device regulatory authorities and the regulated industry. GHTF develops guidance documents that provide a model for the regulation of medical devices that can then be adopted by regulatory authorities. The model, which is based upon a set of Essential Principles of Safety and Performance, is closely aligned with the European medical device regulatory framework.

The GHTF soon will be replaced by the International Medical Device Regulators Forum (IMDRF), created in February 2011 to discuss future directions in medical device regulatory harmonisation. In contrast to the GHTF, membership is limited to representatives from medical device regulatory authorities; however, IMDRF may permit nonregulator stakeholders to join working groups. Fortunately, IMDRF has stated that it will develop a formal strategy for the management and maintenance of GHTF documents. Therefore, proposed documents, such as the ones discussed in this article, should be made final either under the GHTF during a transition period or under the responsibility of the IMDRF. Additional information on this organisation can be found at www.imdrf.org [7].

Guidance on key definitions and concepts
The proposed document, Clinical Evidence for IVD Medical Devices–Key Definitions and Concepts, dated 16 September 2011, can be obtained from the GHTF website at www.ghtf.org/sg5/sg5-proposed.html [8]. As with all three proposed documents discussed here, the contents could change as a result of comments received or further deliberations during development of the final document. Readers should remain alert regarding the issuance of the final document.

The proposed document provides definitions and explanations for the following terms:

• Clinical evidence of an IVD medical device
• Scientific validity of an analyte (measurand)
• Performance of an IVD medical device, including analytical performance and clinical performance
• Performance evaluation of an IVD medical device
• Clinical utility of the IVD medical device

Intended use/purpose
In addition, it attempts to illustrate, with a figure consisting of two overlapping circles, that scientific validity and clinical performance are common elements of clinical evidence and clinical utility.

For example, “clinical evidence of an IVD medical device” is defined as “all the information that supports the scientific validity and performance for its use as intended by the manufacturer.” The term “performance evaluation of an IVD medical device” is defined as the “assessment and analysis of data to establish or verify the performance of an IVD medical device.” The proposed document also explains that performance evaluation data “are typically generated from verification and validation studies (including, where appropriate, clinical performance studies using human specimens) or obtained from a literature review that confirms the performance characteristics of the product.” This explanation is particularly useful in clearly placing clinical performance studies using human specimens within the scope of validation studies. It is also consistent with the requirements specified in the In Vitro Diagnostic Medical Devices Directive (IVDD; 98/79/EC). That is, the conformity assessment procedures of the IVDD, such as Section 3, indent 11 of Annex III, specify that performance evaluation “data should originate from studies in a clinical or other appropriate environment or result from relevant biographical references.”    

Scientific validity and performance evaluation
The proposed document, Clinical Evidence for IVD Medical Devices - Scientific Validity Determination and Performance Evaluation, dated 16 September 2011, can be obtained from the GHTF website at www.ghtf.org/sg5/sg5-proposed.html [8]. The purpose of the document is to provide guidance on how to collect and document clinical evidence for an IVD medical device as part of the conformity assessment procedure before placing the device on the market, as well as to support its ongoing marketing. It is also intended to provide guidance to regulators and others when assessing clinical evidence provided by manufacturers.

The principal areas covered in this document are: definitions, general principles of clinical evidence, scientific validity determination, performance evaluation (including analytical performance and clinical performance) and clinical evidence report. Appendix A provides a possible format for a literature search report. Appendix B includes a flow chart illustrating possible methodology for documenting, screening and selecting literature within a literature search report. The definitions provided in the document, such as “diagnostic sensitivity,” “diagnostic specificity,” “examination” and others are consistent with the European harmonised standard, EN ISO 18113-1:2011, In vitro diagnostic medical devices - Information supplied by the manufacturer (labelling) - Part 1: Terms, definitions and general requirements (ISO 18113-1:2009).

Section 5.0, General Principles of Clinical Evidence, provides useful concepts regarding clinical evidence and its importance and also information on methods for generating and documenting clinical evidence for an IVD medical device. For example, the section points out that gathering information to support clinical evidence begins during the research process for the IVD device, consisting of two major phases: identification of the scientific validity of the analyte (measurand) and the performance evaluation of the device.

A flow chart in this section provides an overview of the stages involved in assessing clinical evidence. Section 6.0, Scientific Validity Determination, discusses when it is necessary to determine scientific validity, the potential sources upon which to base scientific validity and important principles in the appraisal and analysis of scientific validity information.

Performance evaluation
Section 7.0, Performance Evaluation, of the proposed document on scientific validity determination and performance evaluation, places IVD medical devices into three categories: established and standardised tests, established and nonstandardised tests, and novel tests, referring to these categories when providing guidance on the type of analytical performance data, and, where appropriate, clinical performance data that should be generated. Section 7.1, Analytical Performance, explains that analytical performance, namely technical test performance, generally using human specimens, is always expected for IVD medical devices. The document states that analytical performance data may include data to demonstrate accuracy (trueness and precision), analytical sensitivity, analytical specificity, linearity, limit of detection and limit of quantitation, cut-off, measuring range, carry-over, determination of appropriate specimen collection and handling and endogenous interference on assay results.

The guidance provided in Section 7.2, Clinical Performance, is analogous to guidance on clinical evaluation ¹, ² applicable to medical devices other than IVD medical devices. The section begins by stating that for many IVD medical devices, clinical performance data typically would not be expected, such as for established and standardised tests and assay migration between instruments that meet certain criteria. It may be required for established and nonstandardised tests and typically required for novel tests. In addition, for high-risk IVD medical devices, design changes that may affect the performance claims of the device also may require clinical performance studies. Another important point is that clinical performance only should be conducted once the analytical performance of the device has been established and determined to be acceptable. The potential sources of clinical performance data can be derived from clinical performance studies, literature and experience gained by routine diagnostic testing. Each of these data sources is discussed in this section; however, a separate guidance document has been developed for clinical performance studies and is discussed next. Once clinical performance data have been identified, the data must be appraised to determine the relevance and quality to address questions about the IVD medical device and the data’s contribution to demonstrating the device’s clinical performance. A clinical performance analysis collectively evaluates all of the appraised information, in terms of weight and significance.

Clinical performance studies
The proposed document, Clinical Performance Studies for In Vitro Diagnostic Medical Devices, dated 26 January 2012, can be obtained from the GHTF website at www.ghtf.org/sg5/sg5-proposed.html [8]. It begins by stating that the purpose of a clinical performance study is to validate aspects of IVD medical device performance, which cannot be determined by analytical testing, literature or previous experience gained by routine diagnostic testing. A discussion of all sections of the document is beyond the scope of this article; however, three are discussed below.

Section 6, Clinical Performance Study Design Type, discusses the types of clinical performance study designs as observational or interventional. An observational study refers to a study in which test results are not used for patient management and do not impact treatment decisions. Interventional studies are those in which test results may influence patient management decisions and may be used to guide treatment. An observational study is further characterised as having a single time-point, longitudinal, retrospective or prospective design. Readers should refer to the proposed document for a full description of these types of designs. The proposed document also lists several practical examples of each type.

Section 7, Clinical Performance Study Design Considerations, discusses the effect that the test purpose, such as diagnosis, screening or monitoring, will have on the study design, such as sample size. This section also discusses the use of different types of samples in clinical performance studies, including purposefully-collected, leftover and archived specimens. Other parts of this section provide guidance regarding site location, statistical design, potential risks and ethical considerations, including informed consent and ethics committee involvement.

Section 8, Clinical Performance Study Protocol, provides a useful list of important information that should be provided regarding study design such as: purpose, objectives, study population, description of test method(s) and interpretation of results, site training and monitoring, specimen type, specimen collection, preparation, handling and storage, inclusion and exclusion criteria, limitations, warning and precautions, data collection/management, data analysis, required materials and number of study sites.

Word of caution
The content of the GHTF IVD medical device proposed documents discussed in this article could change before publication as final documents. Nonetheless, even as proposed documents, they provide useful interpretations concerning important principles and concepts that are consistent with European regulatory expectations regarding IVD performance evaluation data. Manufacturers using these proposed documents, and eventually their final versions, in their efforts to meet European requirements will need, however, to always base their actions on the laws and regulations in member states which transpose the requirements of the IVDD, European harmonised standards, relevant European guidance documents when they become available and the results of any consultations with Notified Bodies or competent authorities.
 
References
1)    Global Harmonisation Task Force, Clinical Evaluation (May 2007), www.ghtf.org/sg5/sg5-final.html [9]
2)    Clinical Evaluation: A Guide for Manufacturers and Notified Bodies (MEDDEV 2.7.1 Rev.3, December 2009); http://ec.europa.eu/enterprise/sectors/medical-devices/files/meddev/2_7_... [10]