CDG Whitepapers

Risk-Based Monitoring—The New FDA Guidance

An Op-Ed by Dr. Nancy J Stark

FDA's draft guidance, "An Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring" (August 2011) references the ICH-GCPs six times. The ICH-GCPs were written in 1996 by pharmaceutical sponsors and regulators from the US, Europe, and Japan. It is 15 years old and is showing a few signs of age—there is no discussion of central data management or electronic security issues, for example. But because the phrase 'good clinical practice' is in the public domain, ICH was free to adopt it as part of the title of document E6, "Good Clinical Practice: Consolidated Guidance." The Center for Drugs published the guidance in the Federal Register giving it high standing in the regulatory community.

Although used by device manufacturers, the guidance does not work well. It's description of an Investigator's Brochure is chemically-centered not mechanically-centered, investigational devices are stored and dispensed but never installed, there is no mention of training an investigator to use the investigational product, there is no mention of software-controlled devices or software collected data, no mention of caregiver or healthcare provider safety; in other words, for medical devices, the ICH-GCPs are inadequate and out-of-date.

ISO 14155 'Clinical investigation of medical devices in human subjects—Good clinical practice' (2011)—the internationally definitive document for medical device clinical studies—is not mentioned even once; yet the international standard was closely harmonized with ICH-GCP to promote international continuity. I could live with the lack of acknowledgement in FDA's guidance if the same FDA Office hadn't taken such a heavy hand in writing the ISO standard. It is wrong to insist on practice-changing language (such as moment of consent versus moment of enrollment) and then turn your back on the work product as FDA seems to have done.

The draft guidance draws heavily on the ever-increasing presence of electronic data collection. A really cool EDC system can spot a lot of site problems before the human eye ever gets to the data. But these systems are expensive (think in terms of a $1M for a modest study) and unaffordable to small start-ups.

Finally, it was the Offices of Compliance from CDRH, CDER, and CBER who wrote this guidance. While they promise us the inspection manual will be promptly updated to match the guidance, they do not make assurances that the reviewers in the Office of Device Evaluation will follow along. Sponsors are strongly urged to get buy-in on the monitoring plan from reviewers before initiating a clinical trial.

While device manufacturers can use the new guidance to their benefit, often to defend existing practices, they should make certain FDA's Office of Device Evaluation buys into any IDE monitoring plan.

 

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Risk-Based Monitoring—The New FDA Guidance

A Whitepaper by Dr. Nancy J Stark

In August of 2011 FDA released a draft guidance document titled "Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring." At first I was very excited because the monitoring guidelines had not been updated since the 1988 "Monitoring of Clinical Investigations." But then, as I began to consider the ramifications of implementing the new guidance for medical device investigations I started to have mixed feelings. The first part of this whitepaper is a factual review of the draft guidance with ideas about how device manufacturer's can take advantage of it. The second part (next blog) is an op-ed expressing my views about what it will mean for the device industry. [1.Risk-Based Monitoring Guidance] [2. Monitoring of Clinical Trials]

Two points stand out as noteworthy before we begin: 1) the draft guidance was written by the Office of Compliance not the Office of Evaluation, and buy-in from an IDE reviewer is uncertain, and 2) the draft guidance addresses itself to drugs with a mere mention of devices as an afterthought, so most concepts have to be 'retrofitted' to device needs.

Part One: Review of Guidance
Risk-Based Monitoring
The draft guidance is based on the premise that most sponsors conduct on-site monitoring visits every 6-8 weeks with the goal of source verifying 100% of the data 100% for 100% of the subjects. It uses this premise to argue that sponsors will save money by moving to risk-based monitoring because such a move will result in fewer monitoring visits. Risk-based monitoring probably will result in fewer on-site visits for big pharmaceutical firms, who may be 'conducting hundreds to thousands of trials in locations around the world'. [3. Quality Management in Clinical Trials] But start-up medical device companies have never monitored as heavily as the guidance assumes and, in that sense, the principles risk-based monitoring won't save on monitoring costs.

The guidance suggests that we use fancy statistical algorithms and analyze real-time data as it comes into electronic data capture systems to identify high-risk sites or high-risk data. The algorithm could be something like 'Risk Priority Number = severity x likelihood of occurrence x detectability'. A Risk Priority Number is calculated for each identified risk. When high-risk sites or data exceed a trigger, they merit immediate, onsite monitoring visits. [3. Quality Management in Clinical Trials]

CDG's Risk-Based Monitoring Course on CD—Available Now!

Training on risk-based monitoring is now available from CDG! The course covers the details of the FDA guidance 'A Risk-Based Approach to Monitoring', recommendations on using it for device studies, include a sample monitoring plan and template, and features QnA addressing issues unique to device manufacturers. The well-researched course is designed and presented by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

High risk sites and high risk data
A high-risk site is a site with a high probability of having a problem; the 'symptom' might be that they are enrolling at a slower or faster rate, experiencing a higher or lower rate of adverse events, or having a higher rate of non-compliances than other sites. For example, one site I worked with suddenly stopped enrolling subjects altogether. The sudden change in enrollment pattern should have triggered an on-site monitoring visit. We eventually learned the site had enrolled the same subjects in multiple trials and compromised everyone's data. The site stopped enrolling because they were being audited by other sponsors for fraud.

High-risk data might be a particular type of event or outcome that is a 'symptom' of a problem with the study. For example, a high rate of post-procedure embolism might indicate a poorly trained investigator or a problem with subject selection.

The guidance suggests that any site in the start-up phase of the study is a high-risk site and that on-site monitoring should be performed at every site in the study's early stages.

Device companies can adopt the same principles using low-cost methods. Have the sites fax or scan the case report forms to the monitor once a week for off-site monitoring (i.e. remote monitoring). With one trial and maybe 100 subjects, a monitor can easily examine the case report forms manually for high-risk sites or data without the need for statistical algorithms.

Critical data and processes
Critical data and critical processes are the types of data and processes most likely to be high risk. Critical data include: [1] evidence that the subject really exists, [2] evidence that subject has the disease or condition being studied, [3] evidence that IRB approval was obtained, [4] evidence that informed consent was signed before the intervention or procedure, [5] endpoint data that address the hypothesis, and [6] serious adverse device effect data. For each study you'll need to make a unique list of endpoint data and adverse device effect data that is critical.

Critical processes include [1] the process of obtaining informed consent, [2] the intervention or procedure, [3] timely reporting of regulatory non-compliances or study deviations, [4] timely correspondence with a data safety monitoring board (if one exists), and the like. Because studies vary so much from each other, you'll need to make a unique list of critical processes for each one.

Critical data and processes are most likely to be high risk and should be checked for closely during the off-site monitoring process.

CDG has monitors experienced in risk-based monitoring

Our outsourced monitors come in with the attitude of service, impermanence, and an enthusiasm for the new; they are accustomed to off-site monitoring and problem spotting. They also carry a breadth of experiences that can help solve the issues de jour. Click here to request a proposal for monitoring services from CDG or phone 773-489-5721.

Methods of monitoring
Next the guidance considers two different methods of monitoring; on-site monitoring and 'centralized monitoring', meaning the real-time analysis of data by an application running on a server in some data management center.

I propose that for device studies there are four methods of monitoring: [1] on-site monitoring, [2] off-site monitoring of faxed or scanned case report forms, [3] off-site monitoring of electronic ally captured data, and [4] off-site monitoring of source data that has been captured by the device (usually in vivo or in vitro diagnostic devices.) Often data card is changed out on-site and the data card analyzed off-site. Sometimes the data card may transmit data electronically via the internet—consider a digital camera used to take photographs of the procedure. In the future, of course, it may even be possible to remotely access electronic medical records.

The idea, then, is to use the most expensive method of monitoring (on-site) for sites or data or processes that are critical and high risk and to use less expensive methods of monitoring (off-site or remote) for sites and data that are less so. It is a welcome concept to device manufacturers who are cash-starved and looking for ways to economize.

Triggers
The idea of a trigger is that some server at a data management center somewhere is programmed to alert you if critical data or a critical process has become high-risk; i.e. has the value or frequency of a critical data element or critical process step changed in such a way as to demand attention. Indeed, if hundreds or thousands of sites are reporting data every day, you will need a program to screen the information for anomalies or outliers.

But since most device firms are doing only a few studies at a time, having the case report forms faxed or scanned to the monitor for remote viewing is an inexpensive alternative. The monitor checks the forms for, [1] completeness (no missing data), [2] contemporaneousness, [3] legibility, [4] logic, [5] adverse device effect reports. The forms can be checked for everything except source verification. The monitor issues and resolves queries for any issues observed. Most triggers are found by simple observation.

Then the case report forms are forwarded to the data center where the data are entered into a local (often non-networked) computer. Here the data may be statistically analyzed to see if a site, data element, procedure or process step is at risk and merits an unscheduled, on-site monitoring visit.

Monitoring Plan
A monitoring plan should be developed for every study. This requirement is not new to sponsors conducting IDE studies or European studies under ISO 14155, but the guidance makes clear recommendations for the format. I recommend you develop a template for a separate document that can be referenced in the protocol, and don't make the monitoring plan a part of the protocol itself.

Section One—Study Description
As a separate document, the first section describes the protocol, investigational device, purpose of the study, and other obvious reference information. You need to describe, [1] the monitoring approaches you will use for the study, [2] criteria for determining the timing, frequency, and intensity of planned monitoring activities, [3] specific activities required for each monitoring method employed during the study, including
reference to required tools, logs, or templates, [4] definitions of events or results that trigger changes in planned monitoring activities for a particular clinical investigator, and [5] identification of possible deviations or failures that would be critical to study integrity and how these are to be recorded and reported.

A second opinion is a phone call away

Sometimes to you want to discuss an issue, but implement the solution yourself. If you want a second opinion about clinical, regulatory, biological safety, or reimbursement strategies, sign us up as consultants. Email us here or call Nancy at 773-489-5721.

Section Two—Reporting
Section two of the monitoring plan should discuss the communication of monitoring results to management, review boards, and regulatory bodies. It should describe the format, content, timing, and archiving requirements for reports and other documentation. For example, if the monitor receives case report forms by fax or email, it makes sense to require, say, a weekly or monthly report. Section two should discuss the process for appropriate communication of the findings to management, review boards, and regulatory bodies.

Section Three—Noncompliances
Section three of the plan should discuss the management of noncompliance. How, and who, will follow up with investigators found to be non-compliant with the regulations, protocol, or IRB requirements? If non-compliances persist will the site be retrained, terminated from the study, or will some other action be taken? If protocol deviations are detected you need a plan for root cause analyses. Problem-solving has always been a part of a monitor's responsibilities in device trials and it may require an on-site visit to really assess the problem. I have had the root cause to be such simple things as the data transcriber needing spectacles, a three-hole punch, or a dedicated phone line. Eyes-on may be the only way to detect such problems so they can be solved.

Section Four—Training
Section four of the monitoring plan should describe specific training for monitors and internal data auditors, especially with the detection of triggers and recommendations for on-site monitoring visits. For device studies, I recommend section four also discuss training plans for investigative site personnel. This section should also describe plans for random quality audits.

Section Five—Amendments to Monitoring Plan
Section five describes the process for amending monitoring plans. What events may require review and revision of the monitoring plan and establish processes to permit timely updates when necessary?

Much of sections Two to Five can be incorporated into the clinical research quality management system. By developing procedures for off-site monitoring you won't have to repeat them in every monitoring plan.

References
[1] 'Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring' FDA, August 2011. http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf
[2] 'Guideline for the Monitoring of Clinical Investigations' FDA, January 1988.
[3] 'Quality Management in Clinical Trials', March 2009, Clinical case series from Pfizer.

ISO 14155 (2011) Good Clinical Practices for Medical Devices Is Here

It's time to update your procedures!
Coming into force very, very soon.

The International Standards Organization (ISO) has finally done it. Earlier this week they published the good clinical practices for medical device clinical investigations: ISO 14155 "Clinical investigations of medical devices in human subjects—good clinical practices" (2011). You can buy your very own copy at www.iso.org for 158 Swiss francs, search for 14155.

 

Once the standard is harmonized via publication in the Official Journal of the European Union it comes into force immediately; it will be tough, so you should start updating your procedures for European studies now. What's the "proof" that enforcement is immediate? And what will it mean for your study? Read on.

No phase-in period—the tedious proof
You can take my word for it, or you can follow the tedious proof presented in the next paragraphs to convince yourself about the enforcement date. We will look only at the Medical Device Directive (MDD); you can follow the same logic for the Active Implantable Medical Device (AIMD) directive on your own.

Annex ZA of the standard
Annes ZA of the standard states: "Once this standard is cited in the Official Journal of the European Communities under that Directive and has been implemented as a national standard in at least one Member State, compliance with the normative clauses of this standard confers, within the limits of the scope of this standard, a presumption of conformity with the relevant Essential Requirements 6a of that Directive and associated EFTA regulations."

Which means—in American English—once the standard is cited in the Official Journal, you will meet the Essential Requirements of the MDD for clinical investigations if you conform to the standard.

Essential Requirements of the MDD
Turning our attention to the MDD, in Article 3 we read:

"Article 3
Essential requirements
The devices must meet the essential requirements set out in Annex I which apply to them, taking account of the intended purpose of the devices concerned.”

And in Article 4 of the MDD paragraph 2, we read:

“Article 4
2. Member States shall not create any obstacle to:
— devices intended for clinical investigation being made available to medical practitioners or authorized persons for that purpose if they meet the conditions laid down in Article 15 and in Annex VIII,”

And in Article 5 of the MDD, paragraph 1, we read:

“Article 5
Reference to standards
1. Member States shall presume compliance with the essential requirements referred to in Article 3 in respect of devices which are in conformity with the relevant national standards adopted pursuant to the harmonized standards the references of which have been publishes [sic, published] in the Official Journal of the European Communities; Member States shall publish the references of such national standards.”

Harmonized standards
You can check the MDD at ec.Europa.eu to see if the standard has been harmonized to the directive. Scroll down on the page to see the long list of harmonized standards. As of today, the 2003 versions of the clinical investigation standards are still in force. I'm told it takes 5-6 months before the a new ISO standard is cited in the Official Journal and harmonized to the directive.

 
 

Frequently asked questions—my guess at the answers
Your Notified Body is your final arbiter for enforcement of the new ISO standard for ongoing studies. A good rule of thumb is that the standard applies to your investigation from the time it is harmonized and going forward, but does not apply retrospectively to activities that are already completed.

[1] When the standard comes into force, what happens to clinical investigations that are currently enrolling?

The new rules will apply to the remainder of the investigation. This means investigators and study staff must to be trained to the new standard and their heavier responsibilities, new data forms for collecting device deficiencies should be created, and the like. It may be useful to develop an educational pack for the Ethics Committees.

[2] When the standard comes into force, what happens to clinical investigations that are completed (last subject is out) and I am analyzing the data?

The final report, i.e. clinical investigation report, should follow the format and content of the standard.

[3] When the standard comes into force, what happens to clinical investigations I am setting up? Let's say I have investigators and sites, and I have EC approval, but I have not yet enrolled the first subject?

Your clinical investigation procedures and quality management system should be updated now to meet the intense requirements of the new standard. Because you haven't enrolled subjects yet, you can expect to redo most of your administrative set-up if it wasn't done in conformance to the new standard.

$$$$ advertisement $$$$

CDG Can Update Your Procedures

CDG was actively involved in writing ISO 14155 (2011). We understand the intention behind the words and can help explain standards-speak in a way you can understand. We can review your procedures, provide recommendations to bring them into conformance, or write additional procedures if needed.

Our style is to work collaboratively with a point-person on your side so that you are involved in the process every step of the way. Phone or email us at 773-489-5721 or cdginc@clinicaldevice.com.

 

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc