CDG Whitepapers

Literature Evaluations for Biological Safety

A Whitepaper by Dr. Nancy J Stark

During the course of product development there are three times when you should do a literature evaluation: 1) to justify animal use, 2) to justify a clinical trial design, and 3) to apply for certification and CE Mark or Medicare reimbursement. I've written previous whitepapers about doing literature evaluations as a part of clinical evaluations. In this whitepaper I'll do a brief overview and then look at literature evaluations to justify animal use.

Three literature evaluations
Selective Clinical Evaluations
ISO 14155 requires a clinical evaluation to justify the design of a clinical trial. 0. ISO 14155. A European colleague showed me several literature evaluations that were done for this purpose. The most important point I noted was that they focused only on the selected indication being investigated, and resulted in a relatively short, compact literature review utilizing only 5-10 publications.

 

Comprehensive clinical evaluations
The literature evaluations intended for certification and CE Mark are supposed to be comprehensive, "The clinical evaluation is based on a comprehensive analysis of available pre- and post-market clinical data relevant to the intended use of the device in question…." 1. MEDDEV Clause 5.5.1. However, CDG has been successful with Notified Bodies by averaging ~40 articles per review for Class IIa and IIb devices. The number of articles reviewed depends on the number available, the quality of the articles, the age of the technology, the volume of competition, and other similar factors. I observed one European Class III device that got certification and CE Mark based on a review of only 9 case series and 7 publications.

Literature reviews for Medicare reimbursement application should be larger and will be more persuasive if they follow the examples found at www.ahrq.gov. The Agency for Healthcare Research and Quality (AHRQ) has been publishing technology assessment literature reviews for years. If you look at the technology assessment for negative pressure wound therapy (NPWT) devices, you'll see that 1078 citations were identified, 288 full-text articles retrieved, and 137 articles reviewed. There were also more than 1400 items submitted for review by independent "interested parties". The authors point out that none of the studies compared an NPWT system head-to-head to another system. 2. Negative Pressure.

Similarly, AHRQ recently released for comment a comparative effectiveness review of devices used in diagnosing cardiac disease in women, including stress electrocardiography, echocardiography, single proton emission computed tomography, cardiac magnetic resonance, and computed tomography angiography. A total of 7163 citations were identified, 1452 full-text articles retrieved, and 101 articles (98 studies) reviewed. The authors note that none of the studies were randomized controlled trials. 3. Diagnosing CAD. This review is currently open for public comment and device manufacturers should absolutely submit publications and data based on research they have done.

In 2010 AHRQ received $500,000,000 from the federal government to prepare comparative effectiveness reviews on a wide variety of products. These reviews will surely affect the availability of new and old technologies to the American patient as Medicare revisits its reimbursement decisions based on these reviews. The message to device manufacturers is that we will have to rethink how we design clinical trials and be prepared to submit our own clinical evaluations if we are to obtain reimbursement in the American marketplace.

Literature evaluations for biological safety
I haven't yet written about literature evaluations to justify the use of animals for biological safety testing or preclinical animal testing. The beauty is that the way you un-justify the use of animals is by demonstrating the work has already been done and does not bear repeating—saving you costly resourses and the unnecessary use of animals. Let's look more closely at literature evaluations to justify (or un-justify) animal tests.

How to Write A Clinical Evaluation—CD Workshop

A new CD workshop is now available on "How to Write a Clinical Evaluation". This well-researched presentation takes you step-by-step through the complicated process of writing a clinical evaluation by making it organized and understandable. The 100+ content slides and supporting handouts explain the logistics of clinical evaluations, how to define objectives and key questions, identify and retrieve abstracts, filter and screen abstracts to identify articles, filter again to zone in on the articles that are applicable, assess and weight the literature, and write the evaluation to address the key questions originally posed. The presentation discusses the three major purposes for evaluations, discusses their regulatory foundation, and proposes a strategy to allow each evaluation to build upon the previous ones. Finally, some average metrics are disclosed.

If you want a learning guide to help you write a clinical or literature evaluation, this workshop is the place to turn. A procedure, protocol, and template—meeting the requirements of MEDDEV 2.7.1 r3—are available for a small, additional cost. Click here to purchase or more information.

Authority—who says so
Take out your current copy of ISO 10993-1 "Biological evaluation of medical devices—Part 1: Evaluation and testing within a risk management process" (2009) and turn to Annex C, "Suggested procedure for literature review." 4. ISO 10993-1.

The introduction tells us, "A review and evaluation of the literature is essential for justification and planning of any biological evaluation of a material or a medical device. The aim of such a review is to determine scientific background for the biological evaluation. It also provides essential information for assessing risks/benefits and achieving the ethical conduct of the planned evaluation as required by ISO 10993-2.

NOTE Such a literature review can be helpful to assess whether the relevant data available in the literature are sufficient to demonstrate biological safety of the device in question without the need to generate further data from actual testing or to conclude that the available data are not sufficient.

Performing a literature review is a scientific activity that should be done with rigor and objectivity, and
should allow verification by third parties."

Methodology
[1] Establish a plan
The methodology, taken from Annex C, sounds a lot like the methodology described by GHTF Study Group 5 and MEDDEV 2.7.1 r3. The first step is to establish a plan or protocol for the identification, selection, collation and review of all available studies and data. The plan should be documented in writing and based on recognized practices for the systematic review of the scientific literature.

[2] Define objectives or key questions
The objectives of the literature evaluation should be clearly defined up-front. Sometimes it is easier to phrase the objectives in the form of key questions, for example, which, if any, animal safety tests from the G95-1 Matrix should be performed? 5. FDA BlueBook G95-1. Does the chemical characterization of the materials justify not doing animal safety tests? Do literature data satisfactorily explain the negative outcomes of certain animal tests? Do literature data justify the design of preclinical animal tests, selection of species, route of exposure, duration of exposure or follow-up, or other design parameters?

[3] Document sources and selection criteria
The next step is to identify the sources of literature and other databases that will be searched. The selection will depend on what information you need. For example, you might need evidence regarding the safety of chemicals such as additives or adulterants, information about chemical properties such as glass transition points or vaporization, data about adverse effects related to the device or technology, published studies about in vivo animal use, or information about 'active water' concentration. The type of data you need will influence where you look for it.

You will probably need to contact suppliers directly (called manual searching) to ask for specific information. For example you might need, say, formulation or processing information that isn't published.

In addition, you'll need to specify criteria for the data. For example, you might accept data from rodents but not from fish, you might require data from certain bacterial strains, large mammals, or ex vivo studies. You might need data with a certain sensitivity, such as parts per billion rather than parts per million. I find it easier to specify what I need than what I don't need until I have a feeling for what kind of information is out there.

[4] Assessment of documents
The documents are then valued in such a way as to group them by low, moderate, and high quality, depending on pre-determined assessment criteria; and then the values for each criteria are summed to give the document a weight. It may be that some high quality documents are unfavorable to the objectives, but these documents must be included in the evaluation nonetheless.

A 'document' may be anything from a MAUDE report of an adverse event to a full-text published article to information provided directly by a supplier. Documents might be assigned a value on: 1) the similarity of the device in the document to the device under consideration, based on technology, critical performance, design, and principles of operation; 2) the relevance of the particular experimental animals used in the selected studies for the biological evaluation of the device under consideration; and 3) conditions of use of the material or device in the selected documents and the intended use of the device in question.

Documents might also be valued on, 4) whether the author's conclusions are substantiated by the data, 5) whether the document reflects the current medical practice and state of the art technology, 6) whether documents are taken from recognized scientific publications and whether or not they have been reported in peer reviewed journals, and 7) the extent to which the data were acquired through established scientific literature.

[5] Critical evaluation of the literature
Note that valuation means the act of setting the value of something, while evaluation means a diagnosis or diagnostic study based on those values. 6. Dictionary.com.

Applying these definitions to literature reviews, we can say that we assess a document through the process of assigning it a separate value for each of several criteria, we weight the document by adding up the values, and we evaluate the body of documents (the literature) by determining if they address the key questions, placing the most emphasis on the document with the highest weights. The idea is to evaluate high quality data whether or not it is favorable to our technology.

The hoped-for outcome is to conclude than no new animal testing is justified, but it may be that we identify that new animal testing is indeed needed.

[6] Format and content
The literature evaluation itself may follow the standard format and content of other literature reviews.

[a] A short description of the material and device, including its intended use. I like to include the European and US classification, if known.
[b] Discussion of whether or not the literature supports the objectives or key questions.
[c] Evaluation of the hazards, risks, and appropriate safety measures that should be taken by the manufacturer.
[d] Description of the method of document selection, reasons for omitting documents, and methods of weighting documents.
[e] List of documents (full citations) used in the evaluation and appropriate cross-references.
[f] Conclusion with a justification for whether additional animal testing should be performed or is not necessary. The conclusion should include a discussion of probable risks and benefits, considering the 'state of the art' of the technology. And the conclusion should identify any gaps in evidence regarding safety or performance that should be addressed by future testing.
[g] The printed names, signatures, and responsibilities of the reviewers, and the dates of signature. I recommend attaching a biosketch or curriculum vitae of the reviewers as evidence of their experience.

Conclusion
Literature evaluations for biocompatibility are used to determine the need for animal biological safety testing or preclinical testing. Documents both favorable and unfavorable to the technology should be reviewed, but only those documents of the greatest assessed value need be evaluated. Naturally, the hoped-for outcome is that no further testing is needed.

The more common situation, however, is for the manufacturer to do a bunch of tests and then turn to the literature to defend an unfavorable outcome. While I'm not a proponent of this strategy, it is wiser to turn to the literature to explain an unwanted test result than to do even more animal testing. If you have unfavorable animal test results, find a knowledgable medical device toxicologist to assess what is currently known about your material, technology, or device, and then do whatever testing is still needed.

References
0. ISO 14155 "Clinical investigation of medical devices for human subjects—good clinical practice" (2011).
1. MEDDEV 2.7.1 r3 "Clinical Evaluation: A guide for manufacturers and Notified Bodies," (2009); clause 5.5.1.
2. AHRQ "Negative Pressure Wound Therapy Devices", 2009, http://www.ahrq.gov/Clinic/ta/negpresswtd/references.htm#ref65.
3. AHRQ "Comparative Effectiveness of Noninvasive Technologies for the Diagnosis of Coronary Artery Disease in Women", 2011, http://effectivehealthcare.ahrq.gov/index.cfm/research-available-for-comment/comment-draft-reports/?pageaction=displaydraftcommentform&topicid=202&productid=770&documenttype=draftReport.
4. ISO 10993-1 "Biological evaluation of medical devices—Part 1: Evaluation and testing within a risk management process" (2009).
5. FDA BlueBook Memo G95-1, "Use of International Standard ISO-10993, 'Biological Evaluation of Medical Devices Part 1: Evaluation and Testing'" (1995).
6. Dictionary.com, http://dictionary.reference.com/browse/evaluation.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

Clinical Trial Agreements

A Whitepaper by Nancy J Stark

Clinical trial agreements (CTAs) are the legally binding agreements between a sponsor and an institution (site) as to how certain business and property rights will be dealt with between the parties. These agreements are separate from Investigator Agreements and Confidentiality Agreements and are not regulated by FDA or disclosable to FDA. CTAs are important because they allocate risk, responsibility, financial support, and obligations of the parties; and they protect the rights of the parties.

Usually the sponsor presents an initial draft of the agreement to the institution, and then each of the terms and conditions are negotiated by the party's attorneys. Agreements follow a typical format of: preamble, acknowledgements and responsibilities, term and termination, payment and reimbursement of costs, HIPAA and patient privacy, publication, data ownership and use, intellectual property, indemnification and insurance, subject injury, best efforts, waiver of consequential damages, miscellaneous (export controls, governing law, and dispute resolution/arbitration/mediatio, exhibits), and signatories. (1. Perry-Northwestern University)

 

The most common problem areas are publication, data ownership and use, intellectual property, indemnification and insurance, subject injury, best efforts, and signatories. In the text below I explore these problems and solutions suggested through various model CTA templates. (2. Weill Medical College of Cornell.) (3. UIDP) (4. MAGI) (5. Arizona template) (6. IOM template)

Conflicting missions
Universities, teaching hospitals, and other not-for-profit institutions are grounded in the principles of academic freedom and the discovery and sharing of information. Furthermore, in order to maintain a tax-exempt status, they must fulfill certain duties such as education of students, creating generalizable knowledge, or dissemination of knowledge. (7. Guiding Principles)

Sponsors, manufacturers, and other for-profit firms have commitments grounded in creating value for their shareholders, providing useful goods and services, and expanding the state of the art. They contribute to society by developing and providing new technologies to ease the medical conditions of mankind, but they must be profitable to survive.

Since neither party can live without the other, the challenge is to find contractual language that supports the mission of each within the constraints that limit them.

[F] Publication
The publication clause outlines who can publish what, and when. The clause is important to industry because they need to hold early developmental information confidential until the design of the device has matured into a functional, manufacturable, and marketable configuration. Revealing information too early, or revealing the wrong information, may allow your competitors to learn from your mistakes. Sponsors are also concerned about the accuracy of reporting, both of data and of product descriptions. And for multi-center studies, solutions are needed that allow for a complete manuscript incorporating results from all sites, and which pre-determines the order of authors.

Conversely, a non-profit institution needs to publish new knowledge. Publication creates recognition of the investigator's efforts, allows results to be used in future trials, enhances the site's prominence, contributes to public knowledge, and allows graduate students to use the data for their theses.

A typical industry draft clause might say, in essence, "Institution shall submit to Sponsor a copy of any proposed publication at least 90 days prior to submission for review and approval."

Compromise language suggested by the University of Arizona is, "Sponsor has no objection to publication by Institution of the results of the Study based on information collected or generated by Institution, whether or not the results are favorable to Sponsor. However, to ensure against inadvertent disclosure of Confidential Information or unprotected Inventions, Institution will provide Sponsor the opportunity to review any proposed publication or other type of disclosure at least thirty (30) days before it is submitted for publication or otherwise disclosed. If any patent action is required to protect Inventions, Institution agrees to delay the disclosure for a period not to exceed ninety (90) days from the date Institution initially submitted the proposed publication, or other type of disclosure, to Sponsor for review. Institution will, on request, remove any Confidential Information before disclosure.

If Study is part of a multi-center study, Institution agrees that the first publication is to be a joint publication covering all centers. However, if a joint manuscript has not been submitted for publication within twelve (12) months of completion or termination of Study at all participating sites, Institution is free to publish its results separately…." (5. Arizona template)

[G] Data ownership and use
Data ownership and use is another common sticking point. Sponsors may argue that they "own" the data because they have paid for it, but Institutions counter that they cannot retain their non-profit status if they engage in contract research or "work-for-hire". Institutions want to protect the right to use study data for non-commercial, research, development, and educational, purposes.

A typical industry draft clause might read, "All study data shall be the sole and exclusive property of Sponsor."

MAGI offers this compromise clause, "Site owns all Source Documents. Sponsor owns all Sponsor Data. Site owns all Specimens and Genetic Data, and grants Sponsor access to such property only for purposes of the Study. Sponsor and Site may freely use their own data subject to Subject consent and IRB approval. In addition, Site may freely use Sponsor Data after first multicenter Publication is published or Sponsor waives right to a multicenter Publication. Sponsor has no right to Site’s pre-existing biological samples, genomic database, or other proprietary database." Sponsor Data is defined as, "Study lab test results, CRFs and other reports completed by Site or Investigator per the Protocol, Agreement or other written instruction by Sponsor, excluding Source Documents and Genetic Data. Specifically excludes results derived from analysis of data." (4. MAGI)

ISO 14155 "Clinical investigation of medical devices in human subjects—
good clinical practice" (2011)

What makes this ISO 14155 training so special? The author served on the Editing Committee of Working Group 4, the inner circle of individuals who put finger-to-keyboard and wrote the standard. In this three-hour recorded training Dr. Stark discusses the standard in a topic-by-topic format; identifying what is new, what were the controversies, and what was the Working Group's thinking in adopting the language it did. It is this background information that helps you apply the standard appropriately to your study.

You can buy this newly released training on CD on our website cdginc@clinicaldevice.com or by phoning 773-489-5706. Private training from Dr. Stark is also available.

[H] Intellectual property
Intellectual property is a little different than data ownership, in that it covers who has the right to obtain a patent, or own an invention, development, or discovery.

A typical industry draft clause may read, "Any inventions made as a result of Institution's performance of this Agreement will be disclosed promptly to Sponsor and shall be deemed the property of Sponsor." But again, the sale of information is at cross-purposes with the non-profit mission of the institution.

The University of Arizona offers this compromise clause, "Inventorship of any inventions, developments, or discoveries, whether patentable or not (collectively referred to as "Intellectual Property"), resulting from the performance of the Study, shall be allocated according to U.S. Patent law (Title 35 U.S. Code) in effect at the time the Intellectual Property was created…." (5. Arizona template)

[I] Indemnification and insurance
Indemnification is language whereby one party agrees to protect another against an anticipated loss or damage; insurance is a policy through a third-party which guarantees payment to cover the costs of loss or damage. Although FDA does not require sponsors to carry insurance to cover the possibility of loss or damage to the Institution resulting from the study, most Institutions will have insurance requirements for sponsor-initiated clinical trials. In the event a sponsor in unable to meet the insurance requirement, the study will probably undergo a clinical risk assessment via which the institution attempts to assess the likelihood of loss or damage.

A typical industry draft clause might read, "Sponsor shall indemnify and hold harmless Institution from any damages and liability that are caused by Sponsor's negligence."

A compromise clause might read, "Sponsor shall indemnify, defend, and hold harmless Institution from any damages and liability that arise out of the proper administration of the Study device, Study-required procedures, Sponsor's use of the results, or Sponsor's negligence or breach of the Agreement."

[J] Subject injury
While indemnification and insurance have to do with loss and damage to the Institution, the issue of subject injury has to do with loss and damage to the subject. Sponsors readily agree to provide subject injury coverage if it is determined the injury is related to the investigational device, but who gets to make the determination? Good clinical practice tells us the principal investigator makes this determination.

Some sponsors like to add a clause that they will not pay for injuries until insurance is billed and denied. But billing third-party payors, whether Medicare or private insurance, should be at the Institution's discretion. If the injury is related to the medical device there should be no billing.

Some sponsors are concerned with injuries that are, in fact, attributable to the underlying illness. A solution is to qualify underlying illness by saying, "to the natural progression of the underlying illness."

Some sponsors try to limit their liability if the injury is attributable to the negligence of the study subject or the failure of the study subject to follow the instructions of the Principal Investigator. But such thinking is contrary to current ethical standards. Subjects are donating the use and study of their bodies for other people's gain; there may be little or no benefit to them beyond the satisfaction of contributing to the world's knowledge. Either the institution, sponsor, or third-party payors should bear the cost of their injury regardless of it origins.

IOM's suggested language is, "The Sponsor shall reimburse actual and reasonable medical expenses incurred in treating any injury or illness to a Study Subject that is directly related to the administration of the Investigational device or the proper performance of any other procedure, each in accordance with the Protocol and the Sponsor’s written instructions to the Institution (or to the extent that the Sponsor’s written instructions conflict with the Protocol, the Sponsor’s written instructions to the Institution only). The Sponsor is not required under this Section to provide compensation for (a) other injury- or illness-related costs (such as lost wages), (b) medical expenses that are paid for by a third party (provided that neither the Institution nor the Study Subject shall be obligated to seek reimbursement from a third party insurer), (c) medical expenses that are incurred as the result of a violation of the Protocol or other misconduct or negligence, in each case by any agent or employee of the Institution (including the Study Staff), or (d) medical expenses for injury or illness unrelated to the Investigational Device and unrelated to the proper performance of any other procedure required by the Protocol or Sponsor’s written instructions to the Institution." (6. IOM template)

Arizona offers a simpler suggestion, "Should a subject suffer any adverse effect caused by the Study drug/device/biologic, the Study procedures, or laboratory work required by the Protocol, Sponsor will pay for all medical and hospital costs required for the diagnosis and treatment of such adverse effect." (5. Arizona template)

[K] Best efforts
Consultants (such as Clinical Device Group) know that research projects are dynamic. No wise Institution, consultant, or research contractor should agree to work under time, cost, and performance constraints. Deadlines and results cannot be guaranteed. Furthermore, work product is constrained by the laws and standards of the United States.

One might think the Sponsor is left defenseless, but this is not the case. Institutions will agree to a "best efforts" clause, a contractual provision which requires the institution to use its highest efforts to perform its obligations and to maximize the benefits to be received by the sponsor. Sponsor's are obligated to terminate a site's participation in a study if non-compliances to protocol or regulations are discovered. And sponsors always have the right to terminate a study for business or ethical reasons. Cost overruns, project delays, or non-performance (of investigator or device) are all legitimate business or ethical reasons for terminating a study or an institution's participation in a study.

[N] Signatories
Industry sponsors sometimes don't realize the employee-employer relationship between investigators and investigative sites and they propose to have the investigator sign the CTA. They're also concerned about the performance of the investigator because of their financial investment in the trial.

A typical industry suggestion for the signatory clause is, "This Agreement is entered into by and among Sponsor, Investigative Site, and Principal Investigator." But if you think about it, an investigator—as an employee of the institution—has no legal authorization to sign the CTA.

A compromise clause is, "This Agreement is entered into by and between Sponsor and Investigative Site. The Principal Investigator shall conduct the trial as an employee of the institution and not as a party to this Agreement." Then the Principal Investigator is asked to sign the Agreement to indicate their understanding and intention to comply.

Conclusion
Sponsors and Institutions both have a vested interest in performing clinical research, and each party has much to gain from the process. By thinking through the issues and appreciating the needs of the other party, it is possible to come to an Agreement that is fair, balanced, and meets everyone's needs.

References
1. My thanks to Sean Perry, JD, Sr. Contract and Grant Officer, Office for Sponsored Research, Northwestern University, Chicago for his insights into this topic.
2. The format is taken from Clinical Trial Agreements, Weill Medical College of Cornell University, Office of Clinical Trials Administration. Contact CDG for link.
3. University-Industry Demonstration Partnership (UIDP), National Academy of Sciences, http://sites.nationalacademies.org/PGA/uidp/PGA_060368.
4. MAGI Model Clinical Trial Agreement, https://www.magiworld.org/standards/.
5. Template for Clinical Trial Agreement, University of Arizona Board of Regents, http://www.orca.arizona.edu/ctas.html.
6. Template for Clinical Trial Agreements, Developed by Jim Snipes, Covington & Burling LLP for the Institute of Medicine, April 2009. Contact CDG for pdf copy.
7. Guiding Principles for University-Industry Endeavors, National Academy of Science and the National Council of University Research Administrators and the Industrial Research Institute, April 2006.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc