The Complex Role of Literature Reviews for Medical Devices

Literature Reviews and Comparative Effectiveness Why worry about comparative effectiveness? Because the American Recovery and Reinvestment Act of 2009mandated $1.1 trillion dollars to "accelerate the development and dissemination of research assessing the comparative effectiveness of health care treatments and strategies, through efforts that: (1) conduct...research that compares the clinical outcomes, effectiveness, and appropriateness of... procedures that are used to prevent, diagnose, or treat diseases, disorders, and other health conditions; and (2) encourage the development and use of clinical registries, clinical data networks, and other forms of electronic health data that can be used to generate or obtain outcome data...." You can compare the effectiveness of your device to other devices or procedures, or you can wait for others to do it for you. And it may be possible to do it with a literature review. The US Agency for Healthcare Research and Quality (AHRQ) states: "...systematic reviews of existing research on the effectiveness, comparative effectiveness, and comparative harms of different health care interventions, are intended to provide relevant evidence to inform real-world health care decisions for patients, providers, and policymakers. In an effort to improve the transparency, consistency, and scientific rigor [of these reviews]...the Agency has developed a Methods Guide for Comparative Effectiveness Reviews. We intend that [this guide] will serve as a resource for...investigators interested in conducting Comparative Effectiveness Reviews." The first draft of the AHRQ guide was offered for public comment in November of 2007. You may obtain a free copy of the original guide by contacting CDG. This evolving, 128+ page guide contains chapters on: 1) TOC, 2) Topic development, 3) Selecting evidence: controlled trials, 4) Selecting evidence: observational studies of beneficial effects, 5) Finding evidence, 6) Assessing the quality and applicability of included studies, 7) Assessing diagnostic technologies, 8) Harms, 9) Quantitative synthesis, 10) Avoiding potential biases in conducting systematic reviews and meta-analysis, 11) Grading the strength of a body of evidence, and 12) References. In addition, Using Existing Systematic Reviews to Replace de Novo Processes in CERs was posted for public comment in May 2009. Do Literature Reviews Have Different Purposes? Both Europe and the US require literature reviews in the course of conducting clinical trials and commercializing devices. But Europeans and Americans have a fundamentally different view of of their purpose. Europeans believe that a well-done literature review will point you in a singe direction: 1) whether or not you need a clinical trial, 2) a justification for the trial, if you need one, 3) or whether or not your device is ready for commercialization. Americans (and FDA) come from a different perspective. The literature review serves a particular purpose: 1) to summarize the state of world knowledge through a Report of Prior Investigations, 2) to establish a device is substantially equivalent to another device on the market, 3) to establish a device can be used safely under the watchful eye of an investigator, 4) to establish a device can be used safely in the day-to-day business of healthcare delivery, 5) to compare the effectiveness of one device against another or against an alternative procedure--i.e. comparative effectiveness. The European view anticipates and demands an exhaustive review of the published (and unpublished) literature; for an old product like a wound dressing this could consist of thousands of articles going back to the time of the ancient Greeks, while for a familiar product like a tongue depressor there may be only a limited amount of scientific literature available. The American view expects that a balanced and honest review will be conducted, but once the purpose of the review is fulfilled the job is done. A Brief History of Literature Reviews Report of Prior Investigations The first regulatory requirement for a literature review was circa 1980, when FDA brought forth the concept of a Report of Prior Investigations, 21 CFR Part 812.20(b)(2). The Report of Priors is a complete report of prior investigations of the device. Its purpose was to give the Agency a history of device use. The Report of Priors is a requirement for significant risk devices that require a full IDE, the concept is useful but not imperative for clinical trials of non-significant risk devices. Medical Device Directives Then, circa 1993, the Europeans raised the bar and said that a literature review constituted clinical evidence and could, in many cases, substitute for new clinical data. The Medical Device Directive of 1993, Annex X, read "...the adequacy of the clinical data must be based on either a compilation of the relevant scientific literature currently available on the intended purpose of the device and the techniques employed as well as, if appropriate, a written report containing a critical evaluation of this compilation, or..." Now we were in new territory because the regulators were stating overtly that retrospective data could adequately substitute for new, prospective data. The provision may have arisen from a European aversion to clinical research. ISO 14155-2 (2003) Clinical Investigation Plans In 2003, the international standard for Clinical Investigation of Medical Devices for Human Subjectsfurther clarified the role of a literature review in clinical trials. Part 2 states in 4.5.1 "The clinical investigation plan shall contain a critical review of the relevant scientific literature and/or unpublished data and reports together with a list of the literature reviewed. The conclusions from this review shall justify the design of the proposed investigation. The review shall be relevant to the intended purpose of the device to be investigated and the proposed method of use. It should also help in the identification of relevant end-points and confounding factors that should be considered, and the choice and justification of control methods." Even further guidance was offered in Annex A of ISO 14155 Part 1, where performance of a literature review took on the aspects of the scientific method in and of itself, and was based on the groundbreaking work done by the Cochrane Collaboration. (Please see "How to Do a Literature Review" below.) It's useful to remember that the ISO 14155 standards are adopted by CEN(Committee for European Normalization) and incorporated by reference into the Directives. They therefore become de facto regulation in Europe and the European device industry thus has a huge stake in how the standard is worded. The Global Harmonization Task Force In 2007, the GHTF Study Group 5 published their guidances on clinical evaluations. They furthered the preference for using retrospective data in lieu of prospective clinical data with the statement on page 9: "Clinical evaluation of medical devices that are based on existing, well-established technologies and intended for an established use of the technology is most likely to rely on compliance with recognized standards and/or literature review and/or clinical experience of comparable devices." Harmonizing the EU Directives with GHTF An important little policy that has slipped past most Americans is the European plan to harmonizethe Medical Device Directives and the European Commission's Medical Device Guidance Documents (MEDEVs) with the GHTF. In an open statementissued by the Notified Bodies, it is "Europe's intention to integrate GHTF guidance into the European legal framework to as full an extent as possible...(and) to replace the first part of the MEDEV on clinical evaluation with the GHTF document." The significance is that as Europe and the GHTF become ever closer it strengthens the preference for retrospective data over prospective data for device commercialization. Furthermore, it forces the hand of TC 194 WG4, the working group struggling to bring the 2008 ISO standard into the 21st century, to accommodate policies in the GHTF guidance and the politicizing of the science of clinical research. While fewer clinical trials are required in Europe, and devices come to market years and even decades sooner than in the States, some European citizens feel they are the guinea pigs of the Western world. The European Medicines Agency, EMEA,made a play for taking over regulation of device industry with backing from the European Commission. However, the move was blocked, and Europe is now in regulatory disarray. How to Do a Literature Review One thing Europeans and Americans do agree on is how to do a scientifically sound literature review. Here's how to go about it: [1] Identify the purpose of the review. [2] Identify the sources from which you'll take your articles: e.g., Medline, Google, abstracts or full articles, reviewed scientific literature or unreviewed or unpublished data, etc. [3] Decide on any filters or limitations, e.g., no articles before 1985, only articles on a well-defined topic, only articles that relate to your indication for use, etc. [4] Gather the articles and list them in a table. [5] Read the articles, some may be excluded--perhaps there are two articles about the same study, state the reason for exclusion in the table. [6] Read the remaining articles again and exclude the articles that are redundant or irrelevant or unusable for other reasons, state the reason for exclusion in the table. [7] Weight each article according to relevant criteria. [8] Analyze the data with regard to the number of patients, success or failure of device, adverse events, etc. [9] Draw conclusions based on original purpose of the review. [10] Prepare a report. [Buy a CD webcast on this topic.]

Report Format The report will have a predictable format. [1] Purpose of the review. [2] Description of the device, [3] Intended use, indications for use, and claims. [Example] [4] Selection criteria for literature articles. [5] Summary and grading of clinical data. [6] Data analysis. [7] Conclusions. We Can Help Literature reviews are even more important in Europe today than they ever were, and The American Recovery and Reinvestment Act is likely to raise the bar of expectation in the States as well. If conducting a well-designed literature review seems daunting and time-consuming, especially when the real-time status of the controversy must be considered, let CDG help you. With trained professionals on both sides of both ponds (we have experts in Japan, EU, and the States), we have the capability to review the literature on your behalf. Email us with your request at cdginc@clinicaldevice.com.