CDG e Conferences & Workshops

Adverse Event Reporting for Medical Device Trials

A Webinar on CD by Dr. Nancy J Stark

First published 2 September 2008

Events and Effects
When we compare the good clinical practice requirements for medical devices and pharmaceuticals, adverse event reporting is among the top three causes of confusion.

In the device world not all adverse events are reportable, instead the investigator makes the decision as to whether a given adverse event is device related or not. If so it is called an adverse device effect (think of cause and effect) and only adverse device effects are recordable and reportable.

As with drugs, an adverse event is any unwanted medical occurrence in a subject. And an adverse device effect is any device related adverse event. ISO/DIS 14155--Clinical investigations of medical devices in humans, good clinical practices, 2008, defines a serious adverse device effect as one that, just like any serious adverse event, results in hospitalization, permanent impairment, death, or other sequelae.

Unanticipated Effects
But here distinctions really start to diverge. FDA does not recognize a separate category of adverse device effect called a 'serious adverse device effect'. Instead, the concept of serious is embedded within the definition of an unanticipated adverse effect; so we have the definition that an unanticipated adverse device effect is a 'serious adverse effect on health or safety...caused by...a device...if...not previously identified in nature, severity, or degree of incidence in the investigational plan....' [Sec. 812.3(s)].

Serious adverse device effects are no more or less reportable than plain-old-vanilla adverse device effects. But unanticipated (and hence serious) adverse device effects play a special roll in the world of medical devices.

Unanticipated adverse device effects are reportable to the sponsor and IRB within ten days of the investigator's learning of them, and to FDA and other IRBs within ten days of the sponsor's learning of them.

Other Event Categories
The protocol or investigational plan may call out special reporting requirements for adverse events (not related to the device), serious adverse events (serious, but not related to the device), or anticipated serious adverse device effects (serious, but we were expecting them in our population). But these are considered on a case-by-case basis by FDA depending on the nature of the device and the study.

ISO/DIS 14155, 2008, reinforces a concept that has been around for awhile but not fully embraced by regulators or manufacturers: the concept of an adverse event that happens to a non-subject such as a caregiver or bystander.

Given the physical-mechanical nature of devices we can imagine that injury from some devices will occur to persons who are not trial subjects.

Injury to non-subjects will likely be reportable under the 2008 ISO standard. And reporting such an event under the new information clause of 812.40--General responsibilities of sponsors is certainly appropriate, even if not commonly done.

Terminating a trial because an unanticipated adverse device effect presents an unreasonable risk to subjects is a forward-looking concept unique to FDA. [Sec 812.46(b)(2)] There is no corollary in ISO 14155.

Data Safety Monitoring Boards
While not the norm, data safety monitoring boards are finding their way into more and more medical device clinical trials. These boards may be charged with reviewing adverse events for seriousness or for device-relatedness.

If called a Data Monitoring Board the charter might be limited to or expanded to assessing device efficacy as well as safety.

Whatever the charter, DSMBs make recommendations to the sponsor that are usually, but not necessarily, implemented.

Safety Assessment and Adverse Event Report Forms
I recommend using a two-part system for capturing and evaluating adverse events.

The first part consists of a case report form that is used whenever the subject visits the investigative site. It asks, one-by-one, if a risk on the risk assessment has occurred. If the answer is 'no', no further action is required and the sponsor slowly builds up data regarding the (in)frequency with which the risk occurs.

If the answer is 'yes', then an adverse event evaluation form is completed. This form asks for an event description and information about the date of onset, date of resolution, seriousness, and device relatedness. It asks if the event was anticipated or not; if yes, it asks if the event has increased in nature, severity, or frequency from what was anticipated--a change in nature or increase in severity or frequency may cause the anticipated event to become unanticipated. Finally, it asks what medical actions were required for treatment.

Non-significant Risk Studies
Many device clinical trials are non-significant risk studies, meaning the IRB reviews the protocol but FDA does not. Since FDA has no record of these studies the ratio of non-significant risk to significant risk studies is poorly documented. Our private research established this number to be about 50/50.

Because FDA does not review non-significant risk devices there is no reviewer assigned to the study. So the question arises: are unanticipated, serious adverse device effects that occur in non-significant risk studies reportable to FDA? And if so, to what individual?

Yes, unanticipated, serious adverse device effects are reportable to FDA with ten days.

But you can set your fears aside. A quick phone call to CDRH's Office of Device Evaluation will guide you to the right party for reporting. They will ask for the protocol and a description of the event and consider the following issues: 1) should the study have been categorized as significant risk, and 2) should additional safeguards be included in the protocol.

 

Nancy J Stark, PhD
President

Clinical Device Group Inc

Risk-Based Monitoring for Device Trials

A CD Course by Dr. Nancy J Stark
22 November 2011

In August 2011, FDA withdrew the 1988 "Guideline for Monitoring of Clinical Investigations", and in doing so withdrew the very concept of pre-study, study initiation, routine study, and study close-out activities. Instead of monitoring at these highly recognized milestones, FDA now encourages linking on-site monitoring visits to critical data and processes that have gone "high-risk" as identified by a centralized electronic data capture system. This fundamental shift in how we look at study management is described in a new draft guidance for industry, "Oversight of Clinical Investigations—Risk-Based Approach to Monitoring" (Aug 2011).

The guidance flies in the face of the new international standard ISO 14155 "Clinical investigations of medical devices in human subjects—good clinical practices" (2011) and is only marginally consistent with ICH-GCPs. Why is FDA encouraging such a radical shift in our approach to study management? Will we, once again, have to rewrite our quality management systems for clinical investigations? Is there a career future for monitors? And most importantly, how can device firms leverage the new guidance to reduce monitoring costs?

Objectives
The course objectives are:   
[1] To learn the content of FDA's draft guidance "Oversight of Clinical Investigations—Risk-Based Approach to Monitoring" (August 2011).
[2] To learn when on-site, off-site, and centralized monitoring are appropriate.
[3] To understand a world that does not think in terms of pre-study, study initiation, routine study, and study close-out visits.
[4] To gain ideas about how to take advantge of the new guidance in a device-centric industry.

Centralized Monitoring
FDA discusses only two methods for monitoring clinical trials, the on-site visit and centralized monitoring. Historically, on-site visits have been indexed to the timing of the study: pre-study, study initiation, routine study, and study close-out visits. But now, since those concepts no longer exist in FDA's mind, monitoring visits are tied to the occurrance of high-risk events or processes.

Electronic data capture systems are key to centralized monitoring. Data are sent to a central data management center from sites around the world. Algorithms programmed into the database detect when a critical data element or critical study process has gone "high-risk", thus triggering additional monitoring, including a possible on-site monitoring visit. In the scary future, a subject's entire source file might be transmitted to the central server and case report forms source verified electronically, possibly triggering on-site monitoring because source data and study data are inconsistent.

The course discusses the concepts of centralized monitoring, the challenges of integrating it into your existing study management, and how device sponsors have been doing variations on monitoring methods for years.

Courses on CD
The two-hour course is available as a prerecorded CD. CD Courses are a great information bargain. You can stop and start the presentation as often as you like, share the CD with your colleagues, watch it at different times, and build a library of material for training new employees. Certificates and CEUs (0.25 units) are available. Just watch the entire presentation (on your honor) and return the completed Word survey. Your certificate will be emailed to you.

You Will Learn
Specifically, you will learn the contents of the new guidance and how it will affect your study management, such as:
[1] Identifying critical data elements and study processes.
[2] Calculating Risk Priority Numbers.
[3] Assemblying a risk assessment of study data and processes.
[4] The parts of a Monitoring Plan,
    [a] Description of monitoring approaches to be used in the study.
    [b] Communication of monitoring results.
    [c] Management of investigator non-compliance.
    [d] Training for monitors and sites.
    [e] Process for amending the Monitoring Plan.

You Will Receive
[1] A CD with slides, handouts, feedback survey, and Flash playable presentation.
[2] PowerPoint slides as pdfs.
[3] Handouts describing triggers and risk assessment.
[4] Template for a Monitoring Plan.
[5] A renowned speaker, Dr. Nancy J Stark.
[6] Opportunity to give course feedback.
[7] 0.25 CEUs and Certificate of Attendance.

Course Level
This is an intermediate level course. Participants are assumed to have a basic knowledge of clinical research and device regulations.

How to Buy
Navigate to http://www.clinicaldevice.com/mall/eConferenceCD.aspx  and buy online or telephone 1-773-489-5706 to purchase a copy.

 

Nancy J Stark, PhD
President
Clinical Device Group Inc
[email protected]

CDRH's 510(k) Working Group—Finding 4

A GLIMPSE OF THE FUTURE?
A Cycle of 510(k) Whitepapers by Nancy J Stark and Kathleen Johnson

The Center for Devices and Regulatory Health has two important missions: 1) to protect the public health, and 2) to foster medical device innovation. Their difficult job is to keep these missions in balance, never compromising one in favor of the other.

In August 2010, CDRH's 510(k) Working Group published a preliminary report consisting of more than 60 recommendations grouped under seven findings aimed at improving the Center's effectiveness in implementing its missions. Will the recommendations work? Do they comprise a forward-thinking strategic plan for the device industry? Herein you'll find CDG's summation of Finding 3 and part of Finding 4, along with our opinions of their worth. If you wish to comment, please do so below.

Finding 4: It is challenging for reviewers to obtain the information it needs to make well-supported clearance decisions

[5] Improper use of recognized standards—require a summary of testing
Section 514(c)(1) of the Act describes the possibility of using, as part of a 510(k) submission, consensus standards that have been entirely or partially recognized by FDA. This section states, “[FDA] shall, by publication in the Federal Register, recognize all or part of an appropriate standard established by a nationally or internationally recognized standard development organization for which a person may submit a declaration of conformity in order to meet a premarket submission requirement or other requirement under this Act to which such standard is applicable.”

Working Group Issues
The problems are that: 1) a device may raise questions of safety and performance not addressed by the consensus standard; the standard provides only partial evidence of substantial equivalence, 2) submitters may not use the most current version of a standard, they may not be able to determine which sections of the standards are recognized and which are not (you must turn to the supplemental information sheets), and 3) submitters do not always indicate how they have used the standard or deviated from the standard.

CDG Comments: The recognized version of a standard, not the current version, should be used. The 510(k) Manual should be updated to discuss the proper use of consensus standards and the "Standards Data Report Form for 510(k)s" should be modified to provide a place for summary information and all other relevant information FDA requires.

CDG can write your 510(k)

CDG's extensive Network Staff has expertise in writing 510(k)s in a wide variety of therapeutic, management, or diagnostic areas. My co-author, Kathleen Johnson is a regulatory consultant with more than 10 years experience in medical devices. CDG has capabilities in regulatory submissions, clinical research, toxicology, biostatistics, information research, medical writing, design control—we focus exclusively on medical device pre-approval issues.

Our style is to work collaboratively with a point-person on your side so that you are involved in the process every step of the way. Phone or email us at 773-489-5721 or [email protected].

[6] Incomplete information—how much is enough?
Current regulations require 510(k) submissions to provide "data to support the statement" regarding substantial equivalence to a predicate device. Under PMA regulations, PMA applications are explicitly required to contain a summary of all information known concerning the safety and/or effectivenes of the device under review. FDA's concern is that submissions may contain biased information, including only clinical or scientific information that supports device equivalency.

Working Group recommendations
The Working Group recommends that 21 CFR 807.87 be revised to explicitly require 510(k) submitters to provide a list and brief description of all scientific information regarding the safety and/or effectiveness of a new device known to or that should be reasonably known to the submitter.

CDG comments: The recommendation to review all available literature is not in keeping with the original intention of the Act, which was to demonstrate substantial equivalence to a predicate. It sounds like a Clinical Evaluation Report for every medical device in Europe. The result could inundate FDA with so much information that an already stressed Agency would increase its review times rather than become more efficient.

Summarizing data generated by other parties on similar, but not equivalent devices, is not helpful; summarizing data generated by other parties on equivalent devices but dissimilar test methods is not helpful. Much information is developed using early design or prototype devices. Would the recommendation require a review of every document in the design history file and before? It has the feel of a PMA, where safety and performance are established de novo.

Listen to Experts Janice Hogan and Yarmela Pavlovic
Discuss "The Successful 510(k)"

Listen to experts Janice Hogan, Esq and Yarmela Pavlovic, Esq. present the whys and hows of crafting a successful 510(k) on CD or OnDemand and take advantage of their vast experience. Their detailed and thorough presentation covers: 1) the legal framework, 2) finding predicates, 3) chosing the right claim, introducing new features, 4) assessing data requirements, 5) software and combination products, 6) dealing with a not-substantially equivalent (NSE) determination, 7) handling device modifications, and 8) seeing the 510(k) from the reviewers perspective. You'll receive the slide deck, video/audio recording, and links to important guidances. Order here.

[7] Type and level of evidence needed
The reviewer's first step is to determine if the device and predicate have the same intended use. However, once this is done, most devices have either different indications for use and/or different technological characteristics than the predicate, and FDA may want data to show that any differences have not altered the intended use or significantly affected the safety and/or effectiveness of the device. In most cases data from bench and/or animal testing is sufficient to complete the argument of safety and effectiveness, but in some cases the Agency wants additional data.

Working Group recommendations
The Working Group recommends the agency develop guidance defining a subset of devices, i.e. Class IIb devices, for which clinical information, manufacturing information, or post-marketing information would be typically necessary.

CDG has already stated that it disagrees with defining Class IIa and Class IIb devices because it will lead to confusion with European Class IIa and IIb device classification schemes. If clinical, manufacturing, or post-market information is necessary to successfully argue safety and effectiveness, the agency can address this via a "Super" 510(k). As prevsiouly stated, the regulation numbers of devices and the required information for Super 510(k)s should clearly defined; the information requirements cannot change with each submission.

[8] Clinical information
The Working Group acknowledges there is common misunderstanding as to what constitutes clinical data. Yet within the space of four paragraphs on pages 76-77 of the report the Working group refers to "clinical information", "clinical data", "clinical evidence", "clinical investigation", and "clinical study". According to CDRH staff, submitters sometimes cite the definition of “valid scientific evidence” together with the “least burdensome” provisions of the Act, claiming that it is sufficient for them to provide any type or level of evidence that meets the definition of “valid scientific evidence” with the lowest possible level of burden, even if such evidence fails to provide "reasonable assurance" of safety and effectiveness.

 

 
Working Group recommendations
The Working Group recommends a “Class IIb” guidance provide greater clarity regarding the circumstances in which the agency will request clinical data in support of a 510(k), what type and level of clinical data are adequate to support clearance, and to define clinical data. Would this mean that all IVD 510(k)s are Class IIb?

CDG recognizes there are situations in which new clinical data may be necessary to establish substantial equivalence and demonstrate safety and effectiveness.

For the most part, however, substantial equivalence, safety, and performance can be clearly established with valid scientific evidence obtained from bench and animal data. Needlessly requesting new human clinical data is unethical. All clinical trials expose human subjects to risk by their very definition. It is not that not conducting a clinical trial should be justified, but that the very act of conducting a clinical trial should be justified.

By needlessly raising the bar to 510(k) clearance, FDA neglects is obligation to foster medical device innovation.

With regard to clinical information, FDA should harmonize its definitions of clinical information with Europe and GHTF so that communication between submitters and reviewers can be clear. In the Europe and GHTF world clinical information is data from the literature, research in humans is a clinical investigation (study or trial may be used colloquially), clinical evaluations are reviews of clinical information and clinical investigations and the risk management system altogether.

[9] Postmarket information
CDRH feels that for certain devices, including novel or particularly complex technologies, it is not feasible to conduct a large-scale clinical trial prior to clearance [does this mean such devices are now cleared without adequate clinical data?] They feel it is necessary to collect additional data after clearance in order to better evaluate the safety and effectiveness of a device over a longer time period, or in a wider patient population.

CDRH has the authority to require postmarket surveillance studies (also called Section 522 studies) as a condition of 510(k) clearance for devices that are expected to have significant use in pediatric populations, but there is no explicit authority for CDRH to order Section 522 studies as a condition of premarket clearance in other situations.

Working Group recommendations
The Working Group recommends the Agency seek greater authority to require postmarket surveillance studies as a condition of clearance for certain devices.

CDG comments: The public health may benefit from postmarket registry studies for certain long-term use devices or devices with very long-lasting effects, however the decision to conduct such studies should be that of the manufacturer and not a condition for clearance.

The agency should continue to develop a unique device identification (UDI) system. There is a vast store of information collected everyday from Medicare billing records, Medicaid billing records, the Health Care Utilization Program (HCUP), and other public information sources, which will allow "real-world" anonymized data on device use and outcomes to be searched, sorted, and correlated.

[10] Manufacturing process information
Working Group recommendations
Because 37% of device recalls are associated with manufacturing problems, the Working Group recommends CDRH develop guidance to provide greater clarity regarding what situations may warrant the submission of manufacturing process information as part of a 510(k), and include a discussion of such information as part of its “class IIb” guidance. The Working Group further recommends the agency seek authority to withhold clearance on the basis of a failure to comply with good manufacturing practice requirements in certain situations.

CDG comments: There are situations, as with novel materials, when manufacturing process and design control information should be part of the 510(k) and that such situations should be addressed in a guidance.

A requirement to have evidence of a good manufacturing practice system in place prior to clearance moves the 510(k) process into the arena of a PMA. Many of the proposals by the Working Group blur the line between the 510(k) and PMA processes. The distinction is clear in the Act and should be honored by the Agency.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

CDRH's 510(k) Working Group—Findings 3 & 4a

A GLIMPSE OF THE FUTURE?
A Cycle of 510(k) Whitepapers by Nancy J Stark and
Kathleen Johnson

The Center for Devices and Regulatory Health has two important missions: 1) to protect the public health and 2) to foster medical device innovation. Their difficult job is to keep these missions in balance, never compromising one in favor of the other.

In August 2010, CDRH's 510(k) Working Group published a preliminary report consisting of more than 60 recommendations grouped under seven findings aimed at improving the Center's effectiveness in implementing its missions. Will the recommendations work? Do they comprise a forward-thinking strategic plan for the device industry? Herein you'll find CDG's summation of Finding 3 and part of Finding 4, along with our opinions of their worth. If you wish to comment, please do so at our blog.

 
Finding 3: the de novo pathtway is important and has not been optimally used across the Center.

The de novo process is an essential tool but difficult-to-use tool
The de novo classification process was established under FDAMA to allow for the 510(k) clearance of devices for which there is no predicate but whose risks do not warrant a PMA approach. The problem with the current system is that a device must first be classified as a Class III through a formal process. The issue is how to expedite classification.

The Working Group recommendation
CDRH reform its implementation of the de novo classification process to provide a practical, risk-based option that affords an appropriate level of review and regulatory control. While the number of de novo requests increases every year (there were 15 in 2009) the review process has become an incredible 2-3 years!

CDG Comments: We agree with the Working Group recommendation and further recommend that the Agency develop a separate de novo application which would include: 1) evidence of no predicate for the intended use and/or technology, 2) demonstration of low risk through design control and biological safety information, and 3) a mechanism for de novo meetings with the Agency. The application should be reviewed within 60 days of submission.

Typically CDRH develops device-specific guidance documents as special controls for de novo devices. CDRH should develop a single generic set of controls that could serve as the baseline. Device-specific guidelines can be added later, as needed.

This would give the Agency an avenue to either quickly classify a device as Class I or II via the de novo process or determine that the device is actually a Class III device.

 

Finding 4: It is challenging for reviewers to obtain the information it needs to make well-supported clearance decisions.

Class IIb Devices versus a "Big Mac" 510(k)
Working Group recommendations
The Working Group recommends creation of a new "class IIb" device classification.

CDG is baffled that the Working Group thinks a new "class IIb" classification is necessary or would even helpful to their obtaining sufficient levels of information about a device to make fully informed decisions.

First, it would add to global confusion. The European Union has a Class IIa and Class IIb device classification system and the EU Class IIb and US Class IIb are unnlikely to be the same. Because most companies do business globally, we should at least be creative enough to think of a new phrase.

And why not build on what we already have: a Special 510(k), Abbreviated 510(k), and Traditional 510(k). If we follow our existing pattern and instead create a new type of 510(k) called a "Super 510(k)" or "510(k) Plus." The Super 510(k) would describe additional information, over and above a traditional 510(k), that would be required for certain, significant-risk Class II devices. A new category of 510(k) can be structured to meet FDA needs, provide more options for the manufacturer, and avoid confusion with the world's second largest device market. We would like to see clear identification of the devices that require clearance under the "Super 510(k)" and defined requirements for data.

Device modifications: to report or not report?
Under current regulations, manufacturers are only required to submit new 510(k)s for device modifications when the modification "could significantly affect the safety or effectiveness of the device" or there has been a "major change or modification to the intended use of the device." The Agency has observed that minor, incremental modifications sometimes accrue to the point that safety, effectiveness, or intended use are affected. Also, there is concern that manufacturers misinterpret the regulation to mean that a new 510(k) is only necessary if a modification definitely does affect safety and/or effectiveness. There is also concern that manufacturers misinterpret the regulation to mean that a new 510(k) is only necessary if a modification could negatively affect safety and/or effectiveness.

When a 510(k) is submitted for a particular modification, CDRH must evaluate not only the effect of that modification itself on the device’s safety and effectiveness, but also the cumulative effect of any unreported modifications that preceded it. There is no explicit requirement for submitters to list the modifications that have been made since the last clearance. This can create challenges for review staff and may result in requests for additional information in the midst of a review.

Working Group recommendations
The Working Group recommends CDRH revise existing guidance to clarify what types of modifications do or do not warrant submission of a new 510(k), and, for those modifications that do warrant a new 510(k), what modifications are eligible for a Special 510(k) (i.e. declaration of conformity to design controls in lieu of providing detailed performance data to FDA as part of a 510(k).) The guidance should clarify that a new 510(k) is also required for changes that potentially could have a positive effect, because what was expected to have a positive impact may in fact create new risks that would not be detected without an adequate assessment.

Finally, the Working Group recommends that CDRH require each manufacturer to provide regular, periodic updates to the Center listing modifications made without 510(k) submission.

CDG comments: Modernizing guidances as to what type of modification requires a new 510(k) submission would be useful. When a 510(k) is submitted, it is reasonable to expect the submitter to identify all interim modifications in both design and/or labeling made since the previous submission.

Periodic modification reporting would result in a huge work overload for both manufacturers and reviewers. Neither party has the human or financial resources to manage regular reporting.

CDG can write your 510(k)

CDG's extensive Network Staff has expertise in writing 510(k)s in a wide variety of therapeutic, management, or diagnostic medical device areas. My co-author, Kathleen Johnson has more than 12 years experience in medical device regulatory affairs. CDG has capabilities in regulatory submissions, clinical research, toxicology, biostatistics, information research, medical writing, design control—we focus exclusively on medical device pre-approval issues.

Our style is to work collaboratively with a point-person on your side so that you are involved in the process every step of the way. Phone or email us at 773-489-5721 or [email protected].

Quality of submissions—crafting an argument
In this section the Working Group takes great pains to avoid stating outright that some submitters can't write: we do not make logical and scientifically developed arguments for safety and effectiveness (i.e. we don't apply the scientific method when structuring our arguments), we are disorganized, and we omit information that is clearly required.

CDRH reviewers state: "...information contained in the “Device Description” section of a 510(k) is often a general summary, and it frequently does not provide an adequate level of detail about the new device...." and "...or many 510(k)s, information provided in one section is inconsistent with information provided in another."

Working Group recommendations
The Working Group recommends the agency adopt the use of an "assurance case" framework for 510(k) submissions. An assurance case is defined as a documented body of evidence that provides a convincing and valid argument that a specified set of critical claims regarding a device's safety and performance are adequately justified. The definition parallels three steps of the scientific method.

 
Further, the Working Group recommends that both photographs and schematics be included with a submission, and that manufacturers retain at least one unit of a device and make it available to the agency for current or future reviews (e.g. when the device is cited as a predicate.)

CDG comments: FDA should update the 510(k) Manual (FDA publication 95-4158) and make it available under "guidances" in Device Advice. This document provides detailed guidance of what each section of a 510(k) should include. There is no need to develop yet another model, when a sufficient and familiar model already exists. Kathleen has used this manual as her guide in developing numerous, successfully cleared 510(k)s.

 
We searched for "510(k) Manual" in Device Advice and it isn't there. We searched for "510(k) Manual" in Google and it isn't there. Kathleen had a copy in her library so we could find the FDA publication number. When we searched for "FDA publication 95-4158" in Google we found a copy under FDA dockets. We were amazed that the 510(k) Manual was so difficult to find and not available at all under Device Advice.

FDA has a website for most of its databases, but the database for guidance documents—surely a key database for good 510(k) submissions—is not there. The website is called FDA Databases and you can see from the image above that there is no tab for guidances!

Part of the problem is that new companies can't find the information they need to tell them how to prepare a 510(k). Maybe FDA should sponsor a public workshop, live or available free as webcasts, to educate new submitters about requirements and expectations for 510(k) submissions.

Listen to Experts Janice Hogan and Yarmela Pavlovic
Discuss "The Successful 510(k)"

As useful as when it was first recorded, Janice Hogan and Yarmela Pavlovic present the whys and hows of crafting a successful 510(k). Listen on CD or OnDemand and take advantage of their vast experience. Their detailed and thorough presentation covers: 1) the legal framework, 2) finding predicates, 3) choosing the right claim, introducing new features, 4) assessing data requirements, 5) software and combination products, 6) dealing with a not-substantially equivalent (NSE) determination, 7) handling device modifications, and 8) seeing the 510(k) from the reviewers perspective. You'll receive the slide deck, video/audio recording, and links to important guidances. Order here.

CDG disagrees that exemplar devices be retained for current or future reviews. Some devices cost many thousands of dollars, some fill an entire room, and some are not stable. Retaining examples would be unduly burdensome and in some cases an impossibility. Furthermore, one manufacturer should not be compelled to make an example device available to FDA so that FDA can have it available for reviewing a submission from another manufacturer.

Improper use of recognized standards—require a summary of testing
Section 514(c)(1) of the Act describes the possibility of using, as part of a 510(k) submission, consensus standards that have been entirely or partially recognized by FDA. This section states, “[FDA] shall, by publication in the Federal Register, recognize all or part of an appropriate standard established by a nationally or internationally recognized standard development organization for which a person may submit a declaration of conformity in order to meet a premarket submission requirement or other requirement under this Act to which such standard is applicable.”

Working Group Issues
The problems are that: 1) a device may raise questions of safety and performance not addressed by the consensus standard; the standard provides only partial evidence of substantial equivalence, 2) submitters may not use the most current version of a standard, they may not be able to determine which sections of the standards are recognized and which are not (you must turn to the supplemental information sheets), and 3) submitters do not always indicate how they have used the standard or deviated from the standard.

CDG Comments: The recognized version of a standard, not the current version, should be used. The 510(k) Manual should be updated to discuss the proper use of consensus standards and the "Standards Data Report Form for 510(k)s" should be modified to provide a place for summary information and all other relevant information FDA requires.

Where's the rest? The last five recommendations under Finding 4 will come out next week. Got something to add? Please comment below.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc 

 

CDRH's 510(k) Working Group Preliminary Report

TAKE THE 510(K) REVIEWER SURVEY
A Cycle of Whitepapers by Nancy J Stark and Kathleen Johnson

It's been a long time coming, with complaints dating back for several years, but FDA's problems really became public when a group of FDA scientists and physicians wrote to John Podesta, head of then President-elect Barack Obama's Transition Team that "America urgently needs change at FDA because FDA is fundamentally broken...the scientific review process for medical devices...has been corrupted and distorted by current FDA managers...." The open letter was particularly significant because it was copied to six senators and four congressman and published in the national press and on every blog on the internet concerned with medical devices.

It was followed by a host of General Accounting Office (GAO) reports, the resignation of Center Director Daniel Schultz and Office of Device Evaluation Director Donna-Bea Tillman, and initiation of an Institute of Medicine (IOM) review of the 510(k) process due out in 2011.

 
Internal Assessment at CDRH
Meanwhile, the Center itself initiated a series of activities to assess the 510(k) process by establishing a 510(k) Working Group and a Task Force on Utilization of Science in Regulatory Decision Making. The Working Group and Task Force published preliminary reports in August 2010.

The working group, comprised of representatives from across CDRH, was assigned to develop an understanding of the way the 510(k) process currently functioned, the perspectives of CDRH staff and a range of external constituencies on the program, and potential areas for improvement.

As part of its research, the Working Group co-hosted a public town hall meeting called "Strengthening CDRH and the 510(k) Process" on 18 February 2010 in conjunction with the Task Force on the Utilization of Science in Regulatory Decision Making. A CD of the public meeting is available at Strengthening the Center.

510(k) Survey
To assess the consistency of CDRH reviewers’ interpretation and understanding of 510(k) regulations, guidance documents, and review practices, the Working Group conducted a survey of the Center’s premarket reviewers and managers. The survey consisted of twenty questions related to reviewers’ and managers’ knowledge and opinions on a range of areas of concern.

Reviewer Cohort. The survey was sent by email to all reviewers in CDRH’s two premarket review offices, the Office of Device Evaluation (ODE) and the Office of In Vitro Diagnostic Device Evaluation and Safety (OIVD). All reviewers were strongly encouraged to complete it. Out of a total of 308 reviewers, 215 reviewers took the survey, and at least 162 respondents answered each question.

Manager Cohort. Premarket review managers also completed the survey as a separate cohort of respondents. Out of a total of 38 managers (Branch Chiefs, Deputy Division Directors, and Division Directors) in ODE and OIVD, 21 ODE Branch Chiefs and Deputy Division Directors took the survey, and at least 13 respondents answered each question.

Take the Survey Yourself
We've put the Working Group survey into SurveyMonkey so that you can take the it yourself, benchmark your answers against your colleagues', and then compare your answers to Center staff. It's useful and information to see if your understanding of the 510(k) clearance process is the same as the agency's. Click on survey to take the test, click on benchmark to compare your answers to your colleagues, and then Center Reviewers to see the agency's responses.

When you've finished, submit the form at the end of the survey to receive your OnDemand access to "Surviving Medical Device Recalls" presented by the Center's Bioresearch Monitoring staff.

When you need to write a 510(k), CDG has people who can help
CDG has a family of technical professionals, each with 10-15 years experience in medical devices and eager to help you write your 510(k). We have regulatory professionals for in-vivo diagnostic, in-vitro diagnostic, and non-diagnostic medical devices; regulatory engineers; a toxicologist experienced in device biological safety; statisticians; and other professionals who can contribute to 510(k) preparation. We work collaboratively with a point-person on your side so that you are involved in the process every step of the way. View our company brochure and capabilities sheet.

Contact us
Phone or email us at 773-489-5721 or [email protected].

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

Good Monitoring Practices: US and ISO

Global Consensus
A Whitepaper and Workshop by Nancy J Stark, PhD
Workshop available OnDemand or on CD.

It's hard to discuss the global consensus on monitoring in a way that is brief, to-the-point, and meaningful. But in a nutshell: ISO/FDIS 14155 (and hence the European Union), is more like the United States than ever before in their expectations of monitoring a clinical investigation.

Clinical research, clinical investigations, and generalizable knowledge
On more than one occasion, a sponsor has decided that some activity they are doing in the field is not clinical research and doesn't have to be monitored. So let's get the definition of clinical research out of the way now: clinical research is the medical care of a patient for the purpose of gathering generalizable knowledge. Americans know this from the Belmont Report.

Said medical care might involve standard-of-care procedures and tests, following the natural course of a disease, or the study of a medical device. If the medical device is not yet cleared for sale (or does not have a Certificate of Conformance in the EU), and the purpose is to gather generalizable knowledge, then the activity is a clinical investigation. Note that two criteria must be met: 1) the device is not yet cleared by 510k or PMA, and 2) the purpose is to gather generalizable knowledge.

This is so because FDA does not have authority to regulate medical practice. Medical practice is 'regulated' by professional societies such as the American Medical Association. FDA regulates interstate commerce.

Manufacturers and sponsors
Still, things can get dicey if a manufacturer puts an uncleared device in a physician's hands and allows the physician to use it. The manufacturer is still in the legal zone if they don't accept data or information about the device, but as soon as they accept an opinion (It worked great!) they have been transformed from manufacturer to sponsor of a clinical trial.

Manufacturers get caught in one other way, in my experience. They don't accept information about their device, instead they accept money. And because there are are only three ways for a device to be "out there" in the world: commercial, custom, or investigational; our hypothetical manufacturer finds himself the unwitting sponsor of a clinical trial.

More than one manufacturer has been surprised to get a warning letter from FDA accusing them of failing to get IRB approval, failing to get informed consent, and sponsoring a clinical trials of an investigational device.

ONLY THREE WAYS TO BE "OUT THERE"
Commercial, Custom, or Investigational

Data credibility
All sponsors must monitor all clinical trials. But the sponsor and monitor may be the same individual; in other words, a one-person firm is perfectly within the legal zone if they monitor an investigation themselves. In early developmental situations, this overlap of responsibilities makes good economic sense.

As a device moves from early development to late development, where you want the resulting data to persuade another party (such as a venture capitalist, an FDA reviewer, a customer, or a third-party payer) of the device's safety, effectiveness, or performance, then you'll want the data to have more credibility. Creditability comes from independent review, enter the monitor.

Who's the sponsor?
Once a client got into a battle with the investigative site over who was the sponsor of the study. There were data-ownership and publishing rights at stake and the site was possessive because they were not accepting payment for the study. The manufacturer was merely providing software free-of-charge.

I asked regulators from the US, UK, and Sweden how to know for certain who was the sponsor: "The sponsor is the guy who is liable if things go south," they said.

My client was in a tricky situation: they wanted to maintain control of their unapproved software and the data generated from it, but they had already installed it on the site's computers. Buyer beware: get it clear up-front in a research contract who owns what, who can publish what, who will pay for what, who is the sponsor, and who is the investigator. There is no substitute for prior understanding.

Moving forward
Now that our hypothetical manufacturer of a hypothetically investigational device has accepted the fact that their on-site activities are clinical trials; and they've come to a understanding that monitoring and credibility go hand-in-hand, who should do the monitoring?

Monitors are usually employees of the sponsor or they may be independently contracted. Employee or contractor, the monitor reports to the sponsor and is the sponsor's agent. So we find an interesting absence in Part 812 and ISO/FDIS 14155: there are no regulatory responsibilities of the monitor. There are only regulatory responsibilities of the sponsor to assure an investigation is monitored.

When the light-bulb first went on, I felt left out.

Convergence between US and ISO/FDIS
When we make a line-by-line comparison between the sponsor's monitoring responsibilities in the US and ISO/FDIS 14155 we find a happy thing: monitors do the same things on both sides of the pond.

If you liked the whitepaper, take the workshop
If you liked this whitepaper, then take the upcoming workshop. The learning objective is to compare the requirements of monitoring in the US and ISO/DIS 14155 in order to help assure quality in clinical trials. In this intermediate level course participants are expected to have a basic knowledge of clinical research and device regulations. Buy the recording here or phone us at 773-489-5706.

A quiz, of course
The five-hour workshop is followed by a half-hour online quiz, taken on your own time within the following two weeks. The quiz is designed to test your learning of the concepts discussed in the lecture and your ability to apply those concepts to real-life issues.

You will receive, we will discuss
[x] Printable PowerPoint slides.
[x] Comparison of monitoring requirements between US and ISO/FDIS 14155
[x] Clear example distinguishing intended use, indications, claims and warning.
[x] Selected monitoring procedures and templates.
[x] Recent monitoring warning letters.
[x] A graded quiz with immediate test results.
[x] Certificate of Attendance and 0.55 CEUs.
[x] One computer connection for one learner (each learner must log-in individually).

Who should attend
[x] Monitors.
[x] Clinical research associates.
[x] Managers and directors of clinical research.
[x] Start-up companies planning their first clinical trial.

Presenter
Dr. Nancy J Stark is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements for OnDemand or CD 
[x] Personal computer.
[x] Internet Access for OnDemand.
[x] Windows Media Player for CD.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Good Monitoring Practices—"Good Grief! What Now?"

The immense job of monitoring!

I thought I'd have some fun in this whitepaper and talk about what goes wrong in the lives of monitors. Here are some of my favorite "Good grief, what now?" situations they've given me. Later on in the whitepaper I'll give you a PollMonkey address so you can pick your preferred course of action and see how it compares to other peoples.

Subject situations...check all that apply...
[1] No Lab test
A subject did not get a lab test on their final visit. The investigator says the subject refused because it hurt to have a needle stick. What should you do?
[a] Have the investigator report the deviation in a note to the Regulatory Binder.
[b] Remind the subject of their obligation to finish the study, they were told of the lab tests in the consent.
[c] Have the investigator report the deviation to the IRB.
[d] Make sure the case report form is clear that the data are not available.

[2] I'm not sick!
You are monitoring a diabetes study. There is a note in one subject's file that they deny having diabetes, even though their blood work shows they do. You think that a patient who denies having a disease is not a good candidate for a study on that disease. As the monitor, what would you recommend to your boss?
[a] The investigator exit the subject from the study.
[b] The subject be educated about diabetes.
[c] Let the subject complete the study and use the data.
[d] Let the subject complete the study and don't use the data.

Begin Advertisement
Date Wheels Are Useful for Planning Study Schedules

 
This week I had to estimate a timeline for a study. I wrote the task list and estimated the duration for each task. Then I used the date wheel to estimate the initiation date like this:
Receive firm assignment—Day 0, Jan 8
Write protocol—5 weeks, Feb 12
Submit to IRB—3 weeks, Mar 5
IRB review and approval—12 weeks, May 27
Study initiation—begin May 28

Date Wheels are circular, pocket-sized, calendars used to identify some future date (e.g. when the kittens are due.) Ours are 3.75 inches in diameter, fit easily into your pocket, are flexible, water-proof, and precision-printed for clear, crisp lettering. Don't plan a study schedule without one. Buy them here.
End Advertisement

Investigator incidents...check all that apply...
[3] When did they sign?
The subject and the witness signed the informed consent six days later than the investigator. The visit date, accountability log, screening logs and source notes agree with the doctor's date. You suspect the investigator forgot to get the subject's signature on treatment day. As the monitor, what do you do now?
[a] Confront investigator to get the truth of the matter, and then make a note to file.
[b] The investigator used an investigational device without the patient's consent; it should be reported to the IRB within 5 days.
[c] Have the investigator correct the data and initial the change.

[4] The case of the missing consent
Just before implantation the investigator turns to an assistant to confirm the patient has signed the consent form. But there is no consent form to be found. The investigator is certain he had the patient sign the form and implants the investigational device anyway. Later, he cannot find a signed consent anywhere. As the monitor, what should you do?
[a] Confirm the investigator reports to the IRB that a subject has been enrolled without consent.
[b] Remind your management that they have five days to report the incident to FDA.
[c] Confirm the patient/subject is truthfully informed of the situation.
[d] Ascertain if the patient/subject was orally informed of the study before surgery.
[e] All of the above

[5] Fake documents
While monitoring a subject's six-month follow-up data you notice the source document looks fake. All the progress notes are written with the same pen and same handwriting, and the entries slant to the right as you move down the page. You remember monitoring the three-month data and the source documents were real. When you confront the study nurse she confesses that they lost the patient's chart and had to reconstruct it. "We think it was shredded." Can you use the six-month data?
[a] yes
[b] no
[c] not sure

Begin another advertisement
Adverse Event Definitions: At Your Fingertips

 
The Adverse Event mousepad/notepad puts complicated adverse event definitions at your fingertips. Use it as a mousepad and find the definitions quickly as you craft a protocol. Use it as a notepad and find the definitions quickly in a meeting.

Starting at the center, you identify the event's characteristics in a pie-wedge. The outer ring identifies the event by type. ISO and FDA definitions are on the right. The tool helps you think your way through the reporting process. Order me.
End advertisement

IRB antics...check all that apply...
[6] Summaries only, please
The IRB of a famous research hospital refuses to review the full protocol for your non-significant risk study. "We only review summaries of NSR device studies," they say, even though their website procedures state they always review full protocols. You balk, but the IRB stands firm and sends a letter approving the study. What should you do?
[a] Tell the FDA.
[b] Report the incident to AAHRPP, the hospital's research accrediting association.
[c] Stop resisting and initiate the study.
[d] Include a note of objection in your monitoring report.

[7] Annual renewal
The investigator neglected to get an annual renewal from the IRB and continued to enroll subjects in the study. The investigator claims the IRB never notified him that the approval was up, as they are supposed to do. What "offenses" have been committed?
[a] Subjects were enrolled in an unapproved study.
[b] An investigator failed to file an annual report to the IRB.
[c] The IRB failed to conduct continuing review of research.
[d] The sponsor failed to secure investigator compliance.

Sponsor snarls...check all that apply.
[8] Anonymous letters
Your manager writes letters to FDA with no return address, seals them, and asks you to mail them from whatever city you are traveling to. You suspect he may be tattling on some questionable practices by your competitors. Are you participating in an unethical practice?
[a] yes
[b] no
[c] don't know

[9] Where's the data?
Your company used paper CRFs printed and assembled by a copy shop. When the study is over the data center notices that a page was missing from ~30% of the forms. Of course it's the page with the pivotal, success or failure question on it. What would you recommend as the best course of action?
[a] Have the statistician impute the values based on the values from the other subjects.
[b] Drop the subjects with the missing data from the success/failure analysis.
[c] Continue the study and replace the subjects with the missing data.

If you liked the whitepaper, take the workshop
If you liked this whitepaper, then take the upcoming workshop. We'll review the poll consensus and I'll add my opinions to the mix. The learning objective is to review the skills of monitoring in order to help assure quality in clinical trials. In this intermediate level course participants are expected to have a basic knowledge of clinical research and device regulations. Sign up here or phone us at 773-489-5706.

A quiz, of course
The five-hour workshop is followed by a half-hour online quiz, taken on your own time within the following two weeks. The quiz is designed to test your learning of the concepts discussed in the lecture and your ability to apply those concepts to real-life issues.

You will receive, we will discuss
[x] Printable PowerPoint slides.
[x] Clear example distinguishing intended use, indications, claims and warning.
[x] Selected monitoring procedures and templates.
[x] Recent monitoring warning letters.
[x] A graded quiz with immediate test results.
[x] Certificate of Attendance and 0.55 CEUs.
[x] One computer connection for one learner (each learner must log-in individually).
[x] A copy of the CRA Handbook 2010 (electronic) with registration by October 13.

Who should attend
[x] Monitors.
[x] Clinical research associates.
[x] Managers and directors of clinical research.
[x] Start-up companies planning their first clinical trial.

Presenter
Dr. Nancy J Stark is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 5 hour workshop will be presented on Wednesday, 20 October 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at here or phone us at 773-489-5706.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

Good Monitoring Practices

GLOBAL CONSENSUS
A Whitepaper by Nancy J Stark, PhD

It may be a coincidence, but it seems to me that the device industry emerged from a long sleep after the August 31, 2010 CDRH Webinar on Preliminary Internal Evaluations of the 510(k) process. The webinar entertained questions from the audience regarding two preliminary reports published by: 1) the 510(k) Working Group, and 2) the Task Force on Utilization of Science in the Regulatory Decision-Making Process. Both groups were comprised of representatives from across CDRH to assess the challenges faced by Center management and staff in the 510(k) process and to assess how effectively science is used to evaluate 510(k)s.

One recommendation of the 510(k) Working Group was to "define the term 'clinical data' in order to foster a common knowledge among review staff and manufacturers about what constitutes 'clinical data' ". Recommendations by the Task Force on...Science... were to "continue to engage in the development of...international consensus standards, which, when recognized by FDA, could help establish basic guidelines for clinical trial design, performance, and reporting," and for the Center "to expand its ongoing efforts related to clinical trials...to include clinical trials that support 510(k)s". Possibly the most significant and repeated recommendation made in the reports was to improve the overall quality of clinical trials.

Good monitoring is essential
Think about what happens in a clinical trial. A manufacturer located in one state or country contracts with an investigator in another state or country to conduct a clinical trial that may last from a month to as long as several years. Like all human beings, investigators lose interest in the trial, deviate from the protocol to experiment with their own ideas, enroll subjects in competing studies, or generally misbehave, without regular eyeball contact from the sponsor.

Monitoring is the mechanism by which a sponsor remains engaged with an investigator. Monitors are the individuals charged with maintaining that eyeball contact.

Types of monitoring visits
Clinical trials are implemented in 'project management' phases of prestudy design, study initiation, implementation (duration) and close-out. Monitoring activities are often designed to parallel the process of the clinical trial. While the transition from one phase to another is not carved in stone, I've tried to describe transition points that are commonly used and easy to conceptualize.

 

Prestudy 'visits'
Pre-investigation or assessment visits are intended to select investigators and prepare sites for the study. They are better perceived as a process in which the capabilities and business relationship of the sponsor and investigator are established.

Both FDA and ISO envision this as a time for sponsors to assure that investigators understand the investigational nature of the device, the requirement to wait for IRB approval, the need for informed consent, agreement that records will be created and made available for monitoring, and that contractual obligations are mutually understood and agreed to. This is also the time during which budget agreements and financial arrangements are made between sponsor and site. The period of time may take several months, visits, phone calls, and email messages and can be thought of as spanning from project conception to first subject consented.

Initiation visits

The main focus of the initiation process is investigator and study staff training. Training may begin with an investigator's meeting if there are many investigators involved in the study. Or investigative sites may be trained one at a time and may include practice on simulated patients or live animals. The importance of training the investigator to device use cannot be over-emphasized. Even investigators familiar with your type of device will need to review special features, materials, software, hardware, instructions for use, and protocol requirements. Misuse of an investigational device by an investigator may come back on the sponsor as a failure to adequately train the user and may be considered an adverse event.

During this phase the monitor will set up record-keeping systems by building Investigator Regulatory Binders, Coordinator's Study Manuals, Subject Binders, Source Document Verification Logs and sponsor files.

It is a common practice for subjects to be consented but not considered enrolled in a study until they have passed subsequent screening procedures and been exposed to the device. But the practice of viewing consent and enrollment as separate events is now frowned on by FDA. They are concerned that subjects may be consented, and then dropped from the study without documentation about why. The preferred viewpoint is that consent and enrollment take place at the same moment. Then if a subject later fails screening, the reasons are documented and the subject is exited from the study. Study initiation culminates when the first device is placed on the first subject.

Routine monitoring visits
A routine monitoring visit is an on-site visit in which the monitor observes the process of the study, compliance to protocol, compliance to regulations, completeness of documentation, and verifies accuracy of data on the case report forms against source information. Case report forms should be verified for accuracy, logic, completeness, legibility, and contemporaneousness; electronic forms should be checked for traceability (i.e., who filled out the screen.) Monitoring should take place on a periodic basis, but I suggest you allow no more than 2-3 days of work to accumulate at a site.

Monitoring requires an extensive skill set. It requires an intimate knowledge of clinical regulations and company procedures, knowledge of the device function and possible malfunctions, knowledge of the disease or condition being studied, the ability to solve problems and work independently, and the ability to write a comprehensive and lucid report that captures the progress of the study and action items for the future.

Monitoring Reports >> Action Item Database >> Open Action-Item Checklist

Monitors need a special kind of personality. They should be people who can work with busy sites, who love data, who can sift through pieces of information and fit them together, who do not leave details unaddressed, and who cannot tolerate an unbalanced checkbook.

Off-site monitoring
Off-site monitoring is an alternative to travel when a site is known as well-organized. Data can be transmitted by fax, pdf, or electronic means and checked for everything but accuracy to source files.

Close-out visits
Close-out visits deal with ending issues and have the goal of making the site inspection-ready. All open action items are addressed, open queries resolved, and reports filed. Close-out visits are like filing income taxes, if you've done a good job of record-keeping all year long the close-out visit will go quickly; if not, it could take days.

If you liked the whitepaper, take the workshop
If you liked this whitepaper, then take the workshop. The learning objective is to review the skills of monitoring in order to help assure quality in clinical trials. This is an intermediate level course, participants are expected to have a basic knowledge of clinical research and device regulations. Sign up here or phone us at 773-489-5706.

A quiz, of course
The five-hour workshop is following by a half-hour online quiz, taken on your own time within the following two weeks. The quiz is designed to test your learning of the concepts discussed in the lecture and your ability to apply those concepts to real-life issues.

You will receive, we will discuss
[x] Printable PowerPoint slides.
[x] Clear example distinguishing intended use, indications, claims and warning.
[x] Monitoring procedures and templates.
[x] Recent monitoring warning letters.
[x] A graded quiz with immediate test results.
[x] One computer connection for one learner (each learner must log-in individually).
[x] Certificate of Attendance and 0.55 CEUs.
[x] A copy of the CRA Handbook 2010 (electronic) with registration by October 13, 2010.

Who should attend
[x] Monitors.
[x] Clinical research associates.
[x] Managers and directors of clinical research.
[x] Start-up companies planning their first clinical trial.

Presenter
Dr. Nancy J Stark is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 5 hour workshop will be presented on Wednesday, 20 October 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Or you can take the event later by CD or OnDemand. Sign up at here or phone us at 773-489-5706.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

A Plan for EU Clinical Evaluation Reports

What Do Notified Bodies Want?

A Whitepaper by Nancy J Stark, PhD.

We've come a long way in our understanding of what Europe expects in a Clinical Evaluation Report. Where you start depends on where you are: 1) writing a CE Report for a new, pre-approval device, or 2) writing a CE report for a commercial device. In the first case, you are asking the question, "Do I need new clinical data for certification?" In the second case, you are bringing your documentation up-to-date.

Clinical evaluation reports for pre-certification devices
If you're still in the stage of designing your device, you begin by identifying its intended uses. Then you evaluate the existing clinical data to determine if it is sufficient to support safety and performance per the Essential Requirements of the MDD or AIMD, or if you need additional clinical data. If additional clinical data are needed you have the choice of revising the intended use or performing a clinical trial.

Once you believe the data are sufficient, you convince top management to grant a CE Mark and sign the Declaration of Conformity to your quality management system. Finally, you will discover the truth when the Notified Body comes to call.

Clinical evaluation reports for commercial devices
The more common situation is that your device is already on the market in Europe, and the Notified Body has informed you that its clinical evaluation report was due last March. He will give you a grace period, but you are expected to conform. You follow the same process of evaluating the existing data to determine if it is sufficient to support safety and performance for the intended uses. Should the report conclude that the existing data are insufficient you are in a difficult situation: you'll need to suspend sales until additional data are acquired.

Because the consequences of determining the data are not sufficient are so financially significant for your firm, and the consequences of determining the data are satisfactory are so financially favorable, it is wise if the report writers are as far removed from the device designers as possible.

Resources
To write the report you need a team of at least three people: 1) a medical writer, 2) an information specialist, and 3) a statistician.

In addition you need a written procedure for how to proceed, a report template (useful, but not required), and access to literature databases such as Medline and Embase. Medline is easy, it is operated by the US National Library of Medicine and is available to anyone with a browser. Medline provides access to 5400 worldwide journals in 39 languages dating from 1947 to the present. And it is free.

Embase, on the other hand, is not free. It provides access to an additional 2000 European medical journals, and so, is indispensable since you want to sell in Europe. But it is owned by Elsevier Publishing. You should be prepared to pay a hefty access fee ($7000 and up per year) or sub-contract the time from a third-party source.

A Three-Legged Stool
First leg
The first leg of a Clinical Evaluation Report is an evaluation of the existing literature data. The Global Harmonization Task Force (GHTF) Study Group 5 document "Clinical Evaluation" coupled with MEDDEV 2.7.1 Rev 3, which is slightly easier to read, are the guidances that tell us how to do the literature evaluation.

Here's what the medical writer—working together with you—will do:
[1] Define the key questions.
[2] Identify the databases to search (Medline, Embase, MAUDE, Cochrane).
[3] Define scope of search and search strategies using a qualified information specialist.
[4] Scan the abstracts to identify articles for review.
[5] Acquire full-texts of articles.
[6] Weight the articles for technical significance, discard anything with a low weight.
[7] Have a statistician weight the articles for statistical significance, discard anything with a low weight.
[8] Review only those articles with the highest weightings; maybe 5-25 articles per intended use.

The literature data are evaluated to determine if they support safety and performance of the device for its intended use. I like to prepare a separate literature review and attach it to the main CE Report as an annex. The literature review annex has several required elements including the search strings used by the information specialist, a list of abstracts scanned, the articles reviewed, their weights and justification for those weights, and full-text copies of the articles.

Second Leg
The second leg of the CE Report is a review any existing clinical investigation that your firm may have sponsored. These data are reviewed separately from the literature because you have closer access to the details. The writer asks questions such as: "Was the Declaration of Helsinki followed?" or "Were adverse events resolved?" If not, those facts are noted in the report but the data are not used.

Once the writer is satisfied that ethical and administrative requirements were correct, he/she reviews the study results to see if they support safety and performance of the device for its intended use. I like to prepare a separate sponsored study review and attach it to the main CE Report as an annex.

Third leg
In the third leg, the medical writer comes to the most sensitive step of all; examining the complaint file against the risk management system. They review the file to see if complaint handling, as dictated by the risk management system, supports the safety and performance of the device for its intended use. I like to prepare a separate risk management review and attach it to the main CE Report as an annex.

The CE Report
Finally, the medical writer takes the literature review, sponsored study review, and risk management review altogether and reaches an overall conclusion of whether the existing data support the device's safety and performance for the intended use, or if additional clinical data are needed newly. You know what happens if new clinical data are needed.

The Technical File
If the data are sufficient, the CE Report is filed away in the Technical File, a written procedure describing the writer's strategy is incorporated into your quality management system, and you wait with baited breath until the inspector comes.

How long, how much
There are only two questions in top management's mind: 1) how long will it take, and 2) how much will it cost. The answers are simple: "It depends." Let's say you have a very nice ultrasonic toothbrush and you intend to use it for cleaning roadside burns—the kind of burn you get when your motorcycle crashes and your skin scrapes against the pavement. The toothbrush is used to remove the dirt, grit, and tarmac from the wound. It's an unusual intended use and raises all kinds of questions about potential damage from the bristles, safety features to prevent over-brushing, effects of ultrasound on open wounds, comparing gum tissue to wound tissue, instructions for use, and so on. A clinical evaluation report for cleaning wounds will take longer and cost more than a report for brushing teeth with the same device.

CDG can help
Clinical Device Group is poised and ready to help you with writing your reports. We have medical writers, information specialists, experienced statisticians, access to Embase, a written procedure, a standard template, and a proven process. CE Reports written by our specialists have been reviewed by at least three different Notified Bodies and all have been accepted. We work collaboratively with a point-person on your side so that you are involved in the process every step of the way.

Buffet approach
But perhaps you'd like to do your own medical writing in order to keep down costs, but don't have access to an information specialist, an experienced statistician, or Embase. Call us anyway, we like to work flexibly to meet your needs.

Contact us
Phone or email us at 773-489-5721 or [email protected].

A Standard Development Tale: ISO/FDIS 14155 (2010)

ISO/FDIS 14155 "Clinical investigation of medical devices in human subjects—
good clinical practices" (2010) 
A Whitepaper by Nancy J Stark, PhD 

ISO 14155 (2011) to become good clinical practice for medical devices
In this opinion-editorial I'd like to give some background on how the revisions to the 2003 ISO 14155 standards came about, why it took so long, what the economic implications may be, and what it's like to serve on a Working Group. These are my personal opinions and experiences, so please keep that in mind.

The medical device clinical investigation standard has had a varied and difficult past. To emphasize the point, here is the publication history as near as I can piece it together:
(1993) EN-540 "Clinical investigation of medical devices"
(1996) ISO 10993 "Biological testing of medical and dental materials and devices - Part 8: Clinical investigation".
(2003) ISO 14155 "Clinical investigation of medical devices for human subjects—Part 1: General requirements" and "Clinical investigation of medical devices for human subjects—Part 2: Clinical investigation plans."
(2011?) ISO 14155 "Clinical investigation of medical devices for human subjects—good clinical practices."

In 2003, the US voted yes to the revisions of the ISO 14155 (1996) standard. The yes vote did not come lightly. The 1996 standard was fluff, even for those days, and the 2003 standards were not a substantial improvement. The ICH-GCPs were adopted in 2000, FDA expectations were already higher than the 2003 standards, and even large firms such as 3M Company or Hollister Incorporated set higher expectations for themselves than the 2003 standards. Japan abstained from voting for ISO 14155-1 and voted against ISO 14155-2. The US reluctantly agreed to the 2003 standards with the idea of moving the action forward and on the condition that revisions begin immediately.

In January of 2003 the US began work on the contents of a New Work Item Proposal. The most important consideration was to harmonize the 2003 standards with the ICH-GCPs. But by April of that year the US backed away. There were competing political sensitivities between ISO (International Standards Organization), who had taken the lead for Part 1 with a Swiss convener; and CEN (Center for European Normalization), who had taken the lead for Part 2 with a British convener. The relationship between ISO and CEN had only recently been formalized in 2001 with the Vienna Agreement. A new Japanese law had passed, adopting the ICH-GCPs as their standard for medical device investigations in outright rejection of the ISO 14155 (2003) standards. And the US had declared war on Iraq, irritating much of Europe and weakening our international status in leadership roles.

Finally, the standard was originally a European initiative, pre-dating the 2001 Vienna Agreement between CEN and ISO. The original members of the CEN working group have always seemed a little surprised at the American and Asian interest in the standard and usually interpreted "international" to mean "pan-European"; eventually we started using the word "global" to make it clear that the standard would have world-wide consequences.

Possible changes to 2003 standards
Work resumed on a New Work Item Proposal (NWIP) in late 2004 with a list of 42 potential changes. Among the most important points proposed were:
[1] Merge Part 1 and Part 2 to avoid overlap and improve document flow.
[2] Harmonize the ISO standard with the ICH-GCPs.
[3] Harmonize the standard with MedDev guidances on clinical trials.
[4] Expand the principles of the standard.
[5] Clarify the responsibilities of investigator, sponsor, monitor, and IRB/EC more thoroughly and consistently.
[6] Clarify risk and risk assessment with regard to the subject rather then device design.
[7] Update the literature review section to match MedDev and any developing GHTF documents.
[8] Clarify the responsibilities of sponsors, monitors, investigators, institutions, IRBs/ECs, DSMB, auditors, and other parties.
[9] Clarify details of the investigator brochures, final report, adverse event handling, case report forms, and data management.
[10] Distinguish between adverse events and deviations.

NWIP approved, 2006
A New Item Proposal was finally introduced for voting in August 2006; it passed in December 2006 saying only that the ICH guidelines should be considered. Only the United Kingdom voted against revisions, leaving one to ask the question why they would oppose an improvement to the standard: there is always something to be gained from a position taken.

One part or two parts?
Work began in earnest in February 2005 in Berlin. The first item of discourse was whether to retain the standard in two parts or combine it into one; the consensus of the group was to combine the standard into one. By this time CEN had abandoned its working group and couldn't allocate funds to support development of a new standard so the question of who was to take the leadership role was easily resolved: ISO would take the lead and the British convener would step aside. The British did not attend another working group meeting until September 2009.

Project management approach
The working group began by dividing into sub-groups, each sub-group took on a major section of the standard and rewrote it to harmonize with ICH-GCPs and their own experiences of best practices. The result, when we met again in December 2005, was several well-written sections which lacked flow and integration. Someone complained that just looking at responsibilities didn't give a new firm a feel for how to go about getting started, and then someone else suggested modeling the standard after any project management approach: preparation, initiation, implementation, maintenance, and close-out; and then there could be sections on who is responsible for what.

Some countries argued that standards should not be teaching documents, but after much heated debate we agreed that teaching was important if we were to going to elevate the quality of device clinical investigations and gain respect for the standard itself. Whether teaching is a proper purpose of a standard or not would have to take second place to the more important goal of subject protection and data integrity.

Regulatory Authorities
Every country has it's own regulations and we needed to find a way to fit standard into a global structure. Many paragraphs are qualified with variations of the phrase "when requested by regulatory authority" as a way of building in flexibility. The phrase gets a little old, but its importance is indisputable.

Time or no time
The next debate was quickly resolved: should we mention time frames for reporting, say, adverse events. Our unanimous answer was no, because each country had it's own time frames and we didn't want to contradict them.

The same held true for EC/IRB responsibilities. We agreed to leave their job descriptions up to individual regulatory authorities.

Adverse event & effect definitions
Defining adverse events and effects turned out to be a fairly easy affair; we turned to the ICH-GCPs whenever we struggled and adapted the language to work for a medical device scenario. But there were wrinkles. The Japanese needed a definition for an "unanticipated, serious adverse effect". We Americans are so accustomed to having it left out that we didn't care, but certainly the standard meshes better with Part 812 by having a definition included.

Work stalls
By January 2006 we had a fairly complete standard. Then work stalled for reasons known only to the convener, the Secretariat, and ISO top management. No one with authority would call a meeting and from December of 2005 to July of 2008—18 months—there was no Working Group 4 meeting. The meeting in Chicago in July of 2008 was unproductive.

Unbeknownst to most Americans, Europe was in disarray. The admission of a number of Eastern European countries into the European Union left the New Approach in question. With 27 member states, reporting of adverse events, device deficiencies, and other post-market surveillance issues became unmanageable. Regulation of the medical device industry came within weeks of being under the control of the European Medicines Agency. It is my belief that the difficulties in Europe affected the progress of the standard.

First ISO/DIS issued in Sep08
In September 2008, despite the nearly unanimous recommendation of the Working Group, an ISO/DIS draft international standard was issued by the ISO Secretrariat. Most of the changes requested by the working group had been overlooked and the draft standard was eventually voted down by the Member States. 

New leadership—second ISO/DIS issued Sep09
No matter what the circumstances may be, when progress stalls it is the leadership that takes the hit. Working Group 4 is no different, and in September 2009 the group elected a new convener. The new convener, also Swiss, worked us hard for three days, not dismissing the meetings until 7 pm or until we had reached consensus. The work culminated in a second ISO/DIS being issued that same month.

Risk management and device deficiencies
The British rejoined the group for the first time since 2005. They brought with them a background in ISO 14971 "Application of risk management to medical devices" and were quite firm about including risk management language in the standard.

We also owe the reporting requirements for device deficiencies to the British members. This is one example of how the ISO standard differs from the ICH-GCPs; the standard needed to bring in the concept of devices that malfunction, deploy improperly, are misused, subject to use errors, and the like.

Adverse event categorization chart
One issue was not resolved: that of developing an adverse event categorization chart. A sub-committee, known as the Editing Committee, worked from January 2008 on to develop a chart, going through at least 15 different versions and never reaching consensus. Two versions of the chart, one from the British and another from the Japanese, were finally included in the final draft. It's interesting that we agreed on the definitions of the terms, we didn't agree on their relationship or if there was a hierarchy. The long discussions revealed basic differences in our understanding of how adverse events and effects are related to each other, and explain why definitions have always been do difficult.

The publication plan
FDIS stands for 'final draft international standard'. While the document is in the final draft stage it is being translated into French. Once this is complete the FDIS will be presented to the ISO members (Standards Bodies from each member country) for a final vote, but the only changes allowed at this stage will be spelling, punctuation, or grammar. After publication by ISO in 2011, each ISO member will be free to adopt the standard at the national level and publish it as a national adoption; so in the US the standard may be adopted as an ANSI/AAMI/ISO standard of the same number (ANSI is the American National Standards Institute, AAMI is the Association for the Advancement of Instrumentation.)

Oddly enough, an ISO member is free to modify the standard, within limits, before issuing it under its own designation. So buyer beware, the national adoption published in France may be different from the national adoption published in the UK, or the one published in the US!

If you were bored with the whitepaper, take the workshop anyway
If you were bored with this whitepaper, try the previous one on ISO 14155; and then take the workshop. The learning objective of this five-hour workshop is to review the changes from the old, 2003 standards to the new, 2010 FDIS. The recording is available by OnDemand or CD at http://www.clinicaldevice.com/mall/Workshops.aspx. 

You will receive, we will discuss
[x] PowerPoint slides.
[x] A 5 hour presentation.
[x] Hints for writing Clinical Evaluation Reports and their role in clinical investigations.
[x] Hints for writing Risk Analysis Reports and their role in clinical investigations.
[x] The required contents of clinical investigation plans (i.e. protocols).
[x] The required contents of investigator's brochures.
[x] Steps in planning clinical investigations.
[x] Steps in implementing clinical investigations.
[x] Steps in closing-out clinical investigations.
[x] The monitor's role in clinical investigations.
[x] Responsibilities of sponsors and investigators.
[x] A 30-minute quiz to reinforce your learning experience.
[x] CEUs and certificate of attendance.

Who should attend
[x] Clinical research professionals who estimate study costs.
[x] Project managers who want better planning and reporting from their group.
[x] CROs who want to improve their proposal skills.
[x] Executives who are planning the future of their company.

Presenter
Dr. Nancy J Stark has been a member of Working Group 4 since 2001 and was an active participant in shaping the coming ISO 14155 standard. She is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 5 hour workshop will be presented on Wednesday, 22 September 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at registration.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

ISO/FDIS 14155 "Clinical Investigations of Medical Devices in human subjects—good clinical practices" (2010)

New Rules for International Device Trials
A Whitepaper and Workshop by Nancy J Stark, PhD
Wednesday, 22 September 2010, 11 AM Central; or OnDemand soon after. 

Sweeping changes to ISO 14155
The international standard for medical device clinical research is about to undergo sweeping changes. The first and most evident change is the change in title; the standard is now known as ISO/FDIS 14155 " Clinical investigations of medical devices in human subjects—good clinical practices". Harmonized with the ICH-GCPs, 'Good clinical practices' was added to the title to signal to the world the Working Group's intention that this is now the standard to be followed for international medical device investigations.

There are other significant changes as well. The mechanical, physical, and engineering nature of devices are taken into account with the integration of the normative reference ISO 14971 "Application of Risk Management to Medical Devices". The contents of Investigator's Brochures looks like a document that reflects a mechanical device rather than a chemical drug, while the requirements for recording and reporting device deficiencies alongside adverse events complements the essential requirements of the MDD and AIMD.

All studies are created equal
Possibly the hardest thing for Americans to grasp is that there is no risk-based categorization of clinical trials in the standard. The concept of non-significant risk studies versus significant risk studies does not exist—all the rules apply equally to all clinical investigations.

An international standard written by international experts
The new standard was written by a team of experts from the US, Europe, and Japan. The team consisted of industry representatives, consultants, and regulators to assure that it was achievable, consistent with country regulations, and in keeping with accepted ethical and organizational practices. It did not come easily; the working group and editing committee met fourteen times, with a dozen or so teleconferences sprinkled in between the meetings.

One result is a standard that will be an an economic leveler—it will be neither easier nor more difficult to conduct a trial in one country over another. Another result is a standard that will be an ethical leveler—it will be neither easier nor more difficult to conduct trials in developing countries than developed countries.

Format and approach
The Part 1 and Part 2 standards from 2003 are now combined into single, comprehensive, 65-page document. The strategy was to approach clinical investigations from a project management perspective. A look at the layout gives an idea of the breadth and depth of the standard.

First, there are four administrative sections:
[1] Scope.
[2] Normative references.
[3] Terms and definitions.
[4] Ethical considerations.
The scope tells us that "the principles ...apply to all other clinical investigations...." The statement has no legal precedence but was clearly important to the Europeans. The scope also makes it clear that the standard does not apply to in vitro diagnostic devices.

"Normative" is ISO-speak for required. It is the one, small sentence under normative references that elevates ISO 14971 to importance. Being listed as a normative reference means that the sponsor is, de facto, non-compliant with ISO 14155 if they are not compliant with ISO 14971. Scattered throughout the document are the many ways in which Risk Analysis Reports are integrated into clinical investigations.

These are followed by three project planning sections:
[5] Clinical investigation planning.
[6] Clinical investigation conduct.
[7] Suspension, termination, and close-out.
It is not that there is anything new in the project management sections, but there is so much more detail and instructive language. If you have never done a clinical study before, you will find these sections to be a welcome reference.

The need for a Clinical Evaluation Report performed according to the principles of GHTF Study Group 5 to justify the design of the clinical protocol is another example of a requirement included to meet the essential requirements of EU Directive 93/42/EEC on medical devices (MDD) and EU Directive 90/385/EEC on active implantable medical devices (AIMD).

These are followed by two responsibilities sections:
[8] Responsibilities of the sponsor.
[9] Responsibilities of the principal investigator.
Sponsor and principal investigator responsibilities follow closely with FDA's new guidance "Investigator Responsibilities—Protecting the Rights, Safety, and Welfare of Study Subjects" and the ICH-GCPs. Principal investigators (aka "investigators" in the States) are still required to sign final reports under ISO, but investigators (aka "co-investigators" or "sub-investigators" in the States) are off the hook.

While much of the detail is laid out in five annexes:
Annex A (normative)—Clinical investigation plan.
Annex B (normative)—Investigator's Brochure.
Annex C (informative)—Case report forms.
Annex D (informative)—Clinical investigation report.
Annex E (informative)—Essential clinical investigation documents.
Annex F (informative)—Adverse event classification decision tree.
Annexes A and B on Clinical Investigation Plans (i.e. protocols) and Investigator's Brochures are required for compliance, the remaining annexes are for the reader's information. They contain extremely valuable information on organizing case report forms, writing a final report, and understanding the relationships between adverse events and effects.

Other extensive changes
Each section was expanded to include sufficient narrative to teach a new sponsor how to conduct a clinical investigation. This controversial move is meant to educate as well as to set a performance bar. Some notable examples of additions or changes include:
[1] Clearer definitions of adverse events, adverse device effects, and unanticipated device effects.
[2] A new definition, recording, and reporting requirements for device deficiencies.
[3] Requirements for recording and reporting adverse device effects in persons other than subjects.
[4] Implied requirement for a clinical research quality management system.
[5] Requirement for Risk Analysis Report.
[6] Requirement for a Clinical Evaluation Report to justify the study design.
[7] Required content for a protocol (clinical investigation plan).
[8] Required content for an Investigator's Brochure.
[9] Suggested content and organization for case report forms.
[10] Discussion of data monitoring committees.
[11] Requirements for document and data control.
[12] Requirements for electronic data systems.
[13] Auditing recommendations.
[14] Procedures for suspension or premature termination of a trial.
[15] Procedures for working with vulnerable populations.
[17] An extensive list of the documents essential for a clinical trial.
[18] Omission of the annex discussing how to do a literature review.
[19] And, very importantly, two differing attempts at adverse event classification.

The publication plan
FDIS stands for 'final draft international standard'. While the document is in the final draft stage it is being translated into French and German. Once this is complete the FDIS will be presented to the ISO members (Standards Bodies from each member country) for a final vote, but the only changes allowed at this stage will be spelling, punctuation, or grammar. After publication by ISO in 2011, each member Standards Body will adopt the standard into its own system; so in the US the standard will be re-issued as an ANSI (American National Standards Institute) standard of the same number.

Oddly enough, a member Standards Body is free to modify the standard, before issuing it under its own name if it wishes. So buyer beware, the French version may be different from the British version or from the US version!

If you liked the whitepaper, take the workshop
The learning objective of this five-hour workshop is to review the changes from the old, 2003 standards to the new, 2010 FDIS. The recording is available by OnDemand or CD at http://www.clinicaldevice.com/mall/Workshops.aspx.

You will receive, we will discuss
[x] PowerPoint slides.
[x] A 5 hour presentation.
[x] Hints for writing Clinical Evaluation Reports and their role in clinical investigations.
[x] Hints for writing Risk Analysis Reports and their role in clinical investigations.
[x] The required contents of clinical investigation plans (i.e. protocols).
[x] The required contents of investigator's brochures.
[x] Steps in planning clinical investigations.
[x] Steps in implementing clinical investigations.
[x] Steps in closing-out clinical investigations.
[x] The monitor's role in clinical investigations.
[x] Responsibilities of sponsors and investigators.
[x] A 30-minute quiz to reinforce your learning experience.
[x] CEUs and certificate of attendance.

Who should attend
[x] Clinical research professionals who estimate study costs.
[x] Project managers who want better planning and reporting from their group.
[x] CROs who want to improve their proposal skills.
[x] Executives who are planning the future of their company.

Presenter
Dr. Nancy J Stark has been a member of Working Group 4 since 2001 and was an active participant in shaping the coming ISO 14155 standard. She is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 5 hour workshop will be presented on Wednesday, 22 September 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at registration.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Keeping Study Costs under Control

Building and Managing A Realistic Budget Plan A Whitepaper and Workshop by Nancy J Stark, PhD Wednesday, 1 September 2010, 11 AM Central; or OnDemand soon after. Reading someone else's copy? Opt-in for whitepapers to get your own. A modest device study for a million dollars A modest device study costs $1M and a major study $10M or more, big bucks for a start-up venture, not insignificant for big firms. The total costs are divided between investigative sites, centers such as CROs or consultants, and internal costs such as your salary and benefits. In this whitepaper we'll discuss common mistakes made in estimating study costs and steps you can take to keep spending under control. It starts with the protocol  Consider the following study plan: 100 subjects will be implanted with a vascular access port; they will use the port for the duration of their primary treatment, and then have the port removed. Let's say the protocol requires still photographs from a final fluoroscopy and radiologist's report at the time of implant to confirm catheter and port position; and still photographs from a final fluoroscopy and radiologist's report at the time of explant to determine possible migration. These are procedures that are outside the normal standard of care. Mistake [1] The first mistake sponsors make is to ask for too much information; information that is unnecessary to investigating the study objective. Don't ask for nice-to-have procedures, lab tests, or information; limit your requests to information that is essential to analyze your primary hypothesis. The 2010 CRA Handbook is Finally Here The 2010 CRA Handbook is finally here! We delayed production until we were sure that the ISO/FDIS 14155 "Clinical investigations for medical devices—good clinical practices" (2010) was definite. So here you have it: the best and the original field guide to medical device clinical trials, brought to you from Clinical Device Group. If you have used our handbook before, you'll be pleased to know that we are offering it again as either a spiral-bound paper edition (designed to lie flat, fit compactly in your briefcase, and lend itself to field notes) OR the electronic version (designed to load to your laptop, be fully searchable as a pdf document, and annotatable if you have Acrobat Writer). Full descriptions, table of contents, and ordering information are here. Know your investigator's costs Begin by building an investigative site budget on your own. Sophisticated sites expect that the sponsor will provide a study overview and model budget for them to work from. The study overview summarizes the study in a paragraph or two, describes the subject selection criteria, and that includes a list of every task the site must do to fulfill the study requirements. Think in terms of verbs or action items for the site, organized by the site's prestudy, study, and post-study tasks. Mistake [2] The Site's Task List is more than just a procedure list for the subjects, it includes every activity the site must perform in order to implement the study. Such tasks as 'review the protocol', 'prepare the IRB application', 'screen patient medical records', 'meet with monitor', 'enter data onto case report forms', and other administrative duties are included in the list. The Site's Task List might look like this: Site Prestudy Tasks (Study Start-Up): [1] Submit protocol and consent to IRB [2] IRB review fee [3] IRB amendment fee [4] Meet with monitor for study training [5] Set up regulatory binder [6] Set up subject binders x number of subjects [7] Administrative fee Mistake [3] Our human tendency is to under-estimate the time required to perform a task; however long you think it will take, multiply by a correction factor characteristic of you. I usually underestimate the actual time by half. It's hard to get firm numbers because investigative sites don't invoice sponsors by the hour. Site Study Tasks: [1] Screen patient records [2] Offer study to patient [3] Obtain informed consent [x] Implant vascular access port—procedure not paid by sponsor [4] Take implant fluoroscopy and still image, write radiology report [x] Treat subject—procedures not paid by sponsor [5] Investigate adverse event [6] Take explant fluoroscopy and still image, write radiology report Mistake [4] Under-estimating the cost of study-procedures; accurate estimations are hard to find. Practice Management Information Corporation (PMIC) reminds us that third party payers rarely disclose their fee databases, and professional medical associations are prevented by Federal antitrust legislation from disclosing fee survey results. You can obtain Medicare fee schedules from the CMS site if you can find them, and a variety of medical fee schedule databases are available for sale. You can use MEDFEES from PMIC or EncoderPro.com from Ingenix, for example. Site Post-study Tasks: [1] Transcribe/enter data from source files to case report forms/EDC screens [2] Be available to monitor for monitoring visit [3] Answer data queries [4] Notify IRB of study close-out Other: [1] Overhead, ~30% of total site budget Mistake [5] No-one like to pay overhead. My own Clients argue over a 10% overhead. A quick search in Google showed that Vanderbilt charges 29%, Memorial University of Newfoundland charges 30%, the Center for Cancer Care and Research in St. Louis, MO wants 20%, UCLA wants 16% of total direct costs, and NYU wants 30%. Overhead costs are higher if calculated on procedure fees only, lower of based on total direct costs. You should plan on a 25-30% overhead charge against the total site budget. THE TASK TABLE Put the task list into a spreadsheet and create a Task Table by assigning hours, hourly rates, and amounts to each task. For tasks that require time, like preparing the IRB submission, imagine in your own mind how long it would take you to perform the task, this will help you estimate the number of hours associated with the task. Then you'll decide if the investigator can perform this duty (high complexity duty or patient-contact duty) or if a trained member of the study staff can perform it. This will help you determine "fair market value" for the hourly rate. Fair market value for a physician/surgeon will be around $250-$350 and for a study nurse around $70-$90. Finally, sites like to discuss study costs in terms of costs per subject and some sites have a minimum per subject. Northwestern Memorial Hospital in Chicago considers any study that pays less than ~$2000 per subject as a money-loser and will not participate. Center budgets A 'center' is a business that pays its own taxes. First you need to identify all the centers that will contribute to your study. Possible centers are central IRBs, central laboratories, contract research organizations, independent monitors, independent professionals, contract research organizations, data management centers, and anyone else whose time you will charge against the study. You begin in the same way. Center budgets are built around the protocol, so you'll make a task list of all the activities you need each center to perform in order to fulfill their part of the study. Mistake [6] Bring a competent clinical research professional into the project early on. Too many sponsors wait until after they have designed the protocol and contracted with the sites to discuss study issues. CROs can often save you money by suggesting better regulatory strategies, study designs, or sample size calculations. Once you have a protocol you'll go through it page-by-page, as before, and make a table of tasks you want to sub-contract and which center you want to do the task. Mistake [7] Don't chose your center by cost alone. Look at references, experience in the device world, reputation, sub-contractor network, and size. Time and again I see centers hired because their estimates were lowest; and then cost over-runs come in when it is too late to switch vendors. Mistake [8] Don't hire a big firm for a small job, or a small firm for a big job. In the first scenario your study will be lost in the daily shuffle; in the latter scenario the center will not have the resources to implement the project. Then, you estimate a budget for each center. Some centers have fixed fees, such as central IRBs; others charge a fixed fee per subject, such as central laboratories; data management centers charge by the number of data fields; and other centers charge an hourly rate, such as professional consultants. As before, you divide the center's activities up into prestudy, study, and post-study tasks; and then assign a number of hours and hourly rate (or fixed fee) to each task. Mistake [10] Don't cut corners by not monitoring poorly performing sites. Clinical studies are market-driven, and there is always a reason for poor performance. If enrollment is slow there is a good chance a competing study is paying more than you per subject, but you won't know until you monitor face-to-face. Sponsor budgets As the sponsor, your own budget fits into the equation too. Sponsor costs aren't just the clinical trial department's costs, they are costs spread throughout the company but that rightfully should charge back to the clinical trial project. Statistical functions, data management functions, quality design costs, costs for comparator devices, and even manufacturing costs for building investigational devices, are all charges that should be included in the sponsor budget. And, of course, don't forget the salaries and benefits of in-house employees. As before, you organize the activities that in-house personnel are expected to do by prestudy, study, and post-study tasks. Here I like to include a separate category of data management tasks, because these tasks span the study project in time from before study initiation to after study close-out. Mistake [11] Task lists, line items, and budgets are tricky to make. A 'splitter' personality will easily find a couple of hundred things for people to do, a 'lumper' personality may whittle a study down to 30-40 tasks. When splitter personnel work for lumper bosses, or vice versa, sparks fly. Learn the personality preferences of your workmates and accommodate them with sub-lists and super-lists. Total study budget Finally, you will add the budgets for each investigative site, each center, and yourself (the sponsor) to make a total study budget. By getting a good estimate ahead of time you can assess the scale of the study, re-evaluate the cost of a hypothesis, plan for expenditures, give realistic information to top management, and scope back when necessary. Mistake [12] First-time sponsors are always surprised that as much as 50% of the study budget may be spent before the first subject enters the study. Good budget planning, organized by study timing, prepares top management for what lies ahead and helps keep middle-managers out of the 'you didn't tell me' hot-seat. If you liked the whitepaper, take the workshop The objective of the workshop is to learn how to plan and manage clinical study costs. Sign up at registration. You will receive, we will discuss [x] PowerPoint slides. [x] A 4-5 hour presentation. [x] How to build a task list. [x] How to relate task to line items. [x] How to craft a budget. [x] How to estimate the cost-at-completion midway through a study. [x] Examples of site budgets, center budgets, study budgets. [x] A 30-minute quiz to reinforce your learning experience. [x] CEUs and certificate of attendance. Who should attend [x] Clinical research professionals who estimate study costs. [x] Project managers who want better planning and reporting from their group. [x] CROs who want to improve their proposal skills. [x] Executives who are planning the future of their company. Presenter Dr. Nancy J Stark is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com. System requirements [x] Personal computer. [x] Internet Access. [x] Telephone. Date, time, registration The 4-5 hour workshop will be presented on Wednesday, 1 September 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at registration. Best Regards, Nancy J Stark, PhD President, Clinical Device Group Inc

Risk Analysis Reports and EU Clinical Trials

A Whitepaper by Nancy J Stark 
An e-Conference by Barry Sands
Wednesday, 4 August 2010; available OnDemand soon after.

If you are planning a clinical trial in Europe, now is the time to learn about risk management in clinical trials. Risk management plays a critical new role in ISO/FDIS 14155 "Investigation of medical devices in human subjects—good clinical practices". The standard is expected to be published in late 2010 or early 2011 and will be incorporated into the AIMD and MDD immediately; there will be no phase in period.

Compliance with risk management is required
The ISO 14971, "Medical devices—Application of risk management to medical devices" standard is a normative reference for ISO/FDIS 14155; meaning its implementation is indispensable for the conduct of clinical trials. In other words, in order to be compliant with ISO/FDIS 14155, the entire clinical trial process must also be compliant with ISO 14971.

Hazards, harms, and risks in clinical trials
It's simple: in risk management terms a hazard is a situation that could cause harm, a harm is an adverse event, a risk is the likelihood that the harm will occur. For example, a lose connection is a hazard, an unwanted electrical shock is a harm, the risk is the mathematical probability that the harm will occur. In a clinical trial you focus on subject-centric risks: what happens to the subject if the device's handle falls off? What happens to the subject if the procedure takes too long? Then you consider the probability that the adverse event will occur and the severity to the subject if it does occur.

Risk and its application to clinical trials
Applying risk management to making sound/informed decisions on clinical study initiation, suspension, modification, reinitiation and stoppage are among the most intense decisions your organization will make. These decisions affect patient safety, clinician/surgeon reputation/relationships, company reputations and product liability.

As clinical research professionals responsible for these studies you aren't unfamiliar with risk concepts, but you will have to learn a new language. For example, what is the difference between risk estimation, risk analysis, risk assessment, risk evaluation, risk control, residual risk, a risk analysis report, risk management, and a risk management file? Here is a partial list of some of the issues you will face with regard to risk management.

Clinical evaluation report
The ISO/FDIS 14155 (2010) takes a whole new strategy toward initiating a clinical trial. Before you start writing the protocol you review the scientific data to prepare a clinical evaluation report consistent with the principles of GHTF Study Group 5. CDG has written several whitepapers on how to prepare clinical evaluation reports and scientific literature reviews. One outcome of the clinical evaluation report is a justification for the design of your clinical study.

Risk analysis report
Another outcome of the clinical evaluation report is a Risk Analysis Report. "What's that?" you ask? It is a table of potential adverse events (i.e. harms) that may occur to the subject as a result of participation in the trial. The list includes all the adverse events that might occur from use error (say, perforating the bowel during a colonoscopy), device malfunction (a laser that over-delivers energy), or the procedure itself (tooth loss following radiation for head and neck cancer). Then the next columns of the table identify the frequency with which the event will occur, the severity of the event if it does occur, the risk (likelihood) that the event will occur, how you will mitigate the event (protect against occurrence or treat should it occur), and finally any residual risks left after mitigation. The table might look like this:

RISK ANALYSIS REPORT
POSSIBLE ADVERSE EVENT | FREQUENCY | SEVERITY | RISK | MITIGATION | RESIDUAL RISK

The standard is serious about risk management; any serious adverse event that occurs and is not identified in the Risk Analysis Report is considered unanticipated and is subject to expedited reporting.

There is a skill to writing Risk Analysis Reports. One colleague prepared a three page report on the risks of removing medical tape from your skin. Don't include so many minor harms that you obfuscate the important ones. Don't omit serious harms, even when they have only a slight chance of occuring. Update the Report and the Investigator's Brochure whenever there is a change in perceived risks.

Informed consent
It isn't enough to prepare a Risk Analysis Report; you have to share it with the subject as part of the informed consent.

Resulting decisions
You will decide whether or not to establish a data monitoring committee based on the findings in the Risk Analysis Report. You will decide whether or not to audit the study based on the Risk Analysis Report. You will determine the follow-up period based on the Report, the follow-up period should be long enough to asses whether or not a harm occurs. And the decision to suspend or terminate a study due to unacceptable risk to subjects will follow naturally from the Risk Analysis Report.

Final report
The conclusion section of the final report shall assess the risks and benefits of the investigational device and identify any groups of persons that are at particular risk.

Expertise in ISO/EN 14971
I've asked Barry Sands to walk us through ISO/FDIS 14155 and ISO/EN 14971 and examine all the ways in which risk and risk concepts are to be applied to clinical trials. Barry is experienced in both clinical research and risk management and we are lucky to have him. He will provide examples on how this process was and was not effectively applied to actual medical device clinical trials.

If you liked this whitepaper, take the e-conference
In this e-conference you will learn how to apply risk management concepts to the clinical research process. You will learn how risk management allows a manufacturer to make more objective and informed decisions about when to initiate, suspend, restart, modify or stop a clinical study. And you will learn how specific risk management tools can assist in these clinical study decisions. 

You will receive
[x] PowerPoint slides.
[x] An expert speaker who has worked for industry and FDA.
[x] Knowledge essential for ISO/FDIS 14155 compliance.
[x] Chance for Q&A.
[x] CEUs and certificate of attendance.
[x] Please purchase your own copy of ISO/DIS 14155 from DIN and ISO/EN 14971 at www.iso.org.

Who should attend?
[x] Anyone who's considering a device trial outside the United States.
[x] Anyone who monitors clinical trials outside the United States.
[x] Regulatory professionals who prepare documents for CE certification and marking.
[x] Regulatory professionals submitting foreign data to FDA.
[x] FDA personnel who review data from foreign trials.
[x] FDA inspectors who inspect foreign sites or sponsors.

Instructor
Mr. Barry Sands is a biomedical engineer with a chemical engineering concentration. He has seven years experience as a Biomedical Engineer and Sr. Scientific Reviewer at FDA/CDRH/ODE and FDA’s Boston District. This government experience was followed with seventeen (17) years in midlevel and executive regulatory/clinical/quality affairs management positions in small start-up and large multinational medical device companies. His firm provides support in the areas of regulatory submissions (510k, IDE, PMA, HUD/HDE, Design Dossiers), Clinical Study Design/Management, Risk Management, Quality System Design/Audits (FDA QSR and ISO 13485) and FDA Negotiation and Communication (QSR Audits, 483s, Warning Letters, Bioresearch Monitoring, Medical Device Reports, Recalls). He can be reached at [email protected].

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 90-minute e-conference will be presented on Wednesday, 4 August 2010, at 11:00 am Central Time. Event materials will be distributed the day before the event. Sign up at registration.

==============================
Be Trained or Be Left Behind

It's a fact in today's fast-paced world: you are never fully trained or up-to-speed. The regulations, standards, and current thinking about the clinical research process are constantly changing. You need access to think-tankers, thought-leaders, insiders, and researchers in order to maintain your knowledgability.

Happily, you can get the full spectrum of current thought from Clinical Device Group, available on CD or OnDemand, at our website.
============================

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

CRO Capabilities from Clinical Device Group

CRO Services from Clinical Device Group -- Click for Help
Clinical trials are not designed by one person, they take a team. Consider the big-ticket items you need:
[1] Project Management,
[2] Investigator's Brochure or Report of Prior Investigations,
[3] Study documents,
[4] Pre-IDE meeting with FDA (optional but recommended),
[5] Approvals,
[6] Study conduct.

On average, a new clinical trial takes 6-9 months to initiate; as much as 75% of your clinical trial labor will be spent on study start-up. You'll need people from toxicology, regulatory, clinical research, manufacturing, marketing, and R&D to pull this all together.

Project Management of Medical Device Trials

Taking Control of the Process
A Whitepaper and Workshop by Nancy J Stark, PhD
Wednesday, 21 July 2010, 11 AM Central; or OnDemand soon after.
Reading someone else's copy? Opt-in for whitepapers to get your own.

You can only expect what you inspect
As a manager, you know its true. As a monitor, you know its true. A clinical study can be doomed from the start due to poor design, or it can evaporate into a cloud of non-enrollment and no data without watchful nurturing while it matures.

Recently a couple of clients asked me to review their protocols after they had IRB approval. Here are some of the problems I found: the least trained people were recording critical data (patients and home-healthcare providers); subjects received permanent implants with no provision for long-term follow-up; no provisions for documenting patients who were screened but not enrolled; and big-name investigators who only did difficult cases, not the everyday cases required by the study. All of these design problems are fixable, but it's embarrassing, time-consuming, and expensive in lost opportunity costs, to go back to the IRB for changes.

Plans are useless, but planning is essential [Eisenhower]
Everyone manages projects, whether you're planning a trip to China or changing the spark plugs. You know its a project if it has a beginning, middle, and end. You are managing the project if you take specific, planned steps on a defined schedule and within limited costs to achieve a distinct, pre-determined goal.The six steps of project management are as old as the hills and as familiar as a summer day; but it isn't always obvious how to apply the steps to clinical trials.

...[1] Define the clinical project
The first step is to define the project. Why are you conducting the study? Is it an early developmental (feasibility, first-in-man, proof-of-concept) study? Is it to gain 510k or PMA or other regulatory approval? Is it to add a new indication for use? Is it a post-approval study to gain visibility, exposure, or additional clinical history? Is it to show non-inferiority to your biggest competitor?

I insist that my project development team request a clinical trial in writing, and that they answer the following questions: [1] what is the device being tested, [2] what is the business objective of the product development project, [3] who is the target market, i.e. who makes the buying decision, [4] what are the intended use and indications for use, [5] what are the features, functions, benefits, or outcomes you hope to make claims about, and [6] who are the major competitors, including competitive procedures.

Tricky me, I have just gotten the team to design the clinical study for me: number one is the device being studied, two tells me if I should look for a big-name or a first-time investigator, three tells me the medical profession of the investigator, four identifies the subject population, five describes the primary endpoints, and six tells me who or what to use as a comparator.

Do You Need A Clinical Research Quality Management System? CDG's Clinical Research Quality Management System is a comprehensive collection of statements, policies, procedures, templates, and supporting documents that contains everything you need to set up a clinical research function. Each procedure is designed to tell you 'how to' do something and results in a work product such as a document or report. The procedures are organized chronologically, the way you would organize a project schedule, to walk you step-by-step through the process of running a clinical trial and a clinical department.The manual is divided in sections that follow the major activities of a clinical project. For more information click here.

...[2] Clinical project schedules
Next comes the task list and Gantt chart. You make a list of every task to be done. If you're a "lumper" you might see a clinical trial as five easy steps, if you're a "splitter" you might see it as several hundred. I cannot wrap my mind around more than 20-30 tasks, so I prefer to work with tasks and sub-tasks in order to keep things manageable. You want to have enough tasks so the project team understands what comes next, but not so much as to confuse them.

Then you organize the tasks in chronological order. Some tasks will overlap in time, others will be in sequence. For other tasks, what is important is that they end at the same time. I have never thought of a good clinical study example for this, but think of your supper tonight. You hope the chef will have the meat and potatoes done at the same time.

Next you estimate how long it will take to do each task. Writing a new protocol without a prior example to work from takes about 30 working days. Designing the content and layout for case report forms takes about four hours per page. You should look backwards into your company's history to get numbers that are realistic for your product area.

Finally, you link the tasks to make a Gantt chart. The most popular application for this is Microsoft Project, but there are other affordable applications as well. Gantt charts really are important: without them the team doesn't understand why even simple trials on non-significant risk devices take an average of six months from conception to first product on first subject.

...[3] Resource allocation
Subjects, investigators, and monitors are the three most important human resources to allocate. The statistician will tell you how many subjects are required and the protocol will dictate the number of case report form pages that will be generated and over what time frame. Using that information you can estimate the number of investigators (investigative sites) you'll need and the number of monitoring hours and monitors you'll need. The fewer the better.

You may need to allocate some physical resources, too. One company I worked for required the clinical research function to order and 'buy' investigational devices from the manufacturing function. The financial department actually moved money from one function's budget to another.

... [4] Clinical project estimates, why do studies cost so much?
For each task item you'll assign a cost. The cost may be fixed (such as a one-time IRB fee), per unit of work-hour (such as internal or contracted labor), or per unit of material (such as forms or devices.) The most accurate way to do a budget is to look at the actual costs for a similar study and add 5%, but you'll need to do a zero-based budget if you don't have a past one to work from.

As much as 50% of the entire budget may be spent before the first device goes on the first subject! A task-related (line item) budget really is important. It communicates to the the team and to management where the money will go, and why so much will be spent before there is any data. As a clinical researcher, you should talk about this a lot—keep it in front of the team's face—get them used to the idea of spending money first in order to get good data later.

...[5] Site support, study management, and reporting
During the implementation process is when most studies fail. Once study start-up is completed there is a tendency to coast while the site does the work. Keep active contact with the site with emails or phone calls every week, a good study coordinator both expects this and appreciates the attention. Provide the site with updates on the trial as a whole, everyone is part of the team and needs to be included.

Study management means following the events and progress of a study; adding or moving resources as needed to get the job done; advising, coaching, or even intervening if personnel are floundering.

Reporting means making certain that top management, other sites, and regulatory authorities are informed as required.

...[6] Study close-out
Finally the fun—a celebration for study completion. But some work, too, as you assure the site is inspection ready. Study close-out is like doing your income taxes. If you have done a good job of record-keeping all study long the close-out process will be simple. If you have been careless in ongoing record-keeping the close-out process can be expensive and complex.

If you liked the whitepaper, take the workshop
The objective of the workshop is to learn how to plan and manage a clinical study. Sign up at registration.

You will receive, we will discuss
[x] PowerPoint slides.
[x] A 4-5 hour presentation.
[x] How to craft a Clinical Development Plan.
[x] How to craft a Gantt chart.
[x] How to calculate monitoring time (labor).
[x] How to estimate the number of monitors needed.
[x] How to prepare a clinical study budget.
[x] How to estimate the cost-at-completion midway through a study.
[x] A checklist of documents to be retained in the sponsor's study file.
[x] A 30-minute quiz to reinforce your learning experience.
[x] CEUs and certificate of attendance.

Who should attend
[x] Clinical research professionals who want to improve their study management skills.
[x] Clinical research managers who need better planning and reporting from their group.
[x] CROs who want to improve their proposal skills.
[x] Executives who are planning the future of their company.

Presenter
Dr. Nancy J Stark is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 4-5 hour workshop will be presented on Wednesday, 21 July 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at registration.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

The Difficulties of Good Data Management

A Whitepaper by Dr. Nancy J Stark
Reading someone else's copy? Opt in to whitepapers and get your own.

Let's define a mid-sized device study as 100 subjects, 50 pages of paper case report forms per subject, and an average of 10 fields per page. Device sponsors are always surprised to hear that data management for such a study will require in the neighborhood of 2500+ hours. So what's all the time about?

How it is supposed to work
In a perfect world, data management would happen by the seamless convergence of data collection and database design. I'll discuss a paper-based model first, because that's what most start-ups use and it is easier to understand. Then I'll look at some electronic data capture (EDC) variations.

...Data collection
Data collection begins with defining what data to collect in order to support what you want to show. In other words, start with your envisioned indications for use, then determine how to measure device performance to that indication; the parameters you measure are called the endpoints. 'Endpoint' is an old term that used to mean the end of data collection. Today it means the pivotal data point that demonstrates product success or failure.

You progress to a set of case report forms that are easy for the Form Filler (at the study site) to fill out and error-proof for the Data Enterer (at the data management center) to enter. The case report forms should collect all the data that are necessary to support or refute the indication for use under study.

...Data verification
The case report form data is verified against the source files (medical records) by a monitor who is independent of the Form Filler. When the sponsor and investigator are separate parties, the monitor works for the sponsor and independence of the Form Filler is assumed. When the study is sponsored by the investigator (an investigator-sponsored study), the monitoring function should be outsourced to a third party who has no financial interest in the outcome.

...Handling test reports
Much of the data will be presented in the form of reports from, say, gastroenterologists or interventional radiologists. Statisticians can't work easily with the comment data found in reports. You'll need to develop questions that capture the essential elements of these reports in order to have usable data for analysis. You might have a series of questions, such as: "Is there any sign of cancer: yes, no, unsure." "What is the widest dimension of the cancer (enter 0 for no or 99.9 for unsure): __.__ mm." etc.

...Other data sources
There will likely be other sources of data too. Digital data may be recorded directly from the device. These data might be impedance, frequency, milliseconds, watts, or other parameters of device performance. Or these data might be measurements taken from the subject such as resistance, oxygenation, pH, or blood pressure.

Clinical laboratory data may be recorded on a laboratory information system (LIMS). These data may include analyst concentrations like glucose or potassium or cholesterol levels; or enzymatic levels such as alanine transaminase; or red, white, or plasma cell counts in the blood.

...Data dictionary and map
Next it is time to finalize the data dictionary and data map. The dictionary defines each data element. The definition may be self-evident but sometimes a more lengthy definition is required, perhaps the CT with contrast means a cat scan with barium contrast medium. The definition includes the data type, range of accepted values if any, and maps the data to its source. The dictionary and map aren't just for convenience. They make up part of the meta-data that enables the reconstruction of events at a future time, should that be necessary.

Using the correct data type is very important if you want calculations to run properly downstream. For example, to calculate the subject's age at the time of consent you would subtract the birth date from the consent date. If we erroneously enter the consent date as text, the calculation will return an error. It can be easily fixed, but it will probably cost half a day in labor.

...Database design
Meanwhile, back at the data management center, the Database Designer designs a database uniquely built to receive the data from the various sources of data. The structure of questions on the case report form dictates exactly the structure of the database. If the question says: "How many apples did you eat today?" the database must receive integer data. If the question says: "What was your primary source of protein for lunch? Select one." The database will have a drop-down box that allows the Data Enterer to select one option from a predetermined list. If the question says: "Which of the following vegetables did you eat today? Check all that apply," a related table will be designed that allows several answers to be linked back to a specific subject.

In addition, remember the digital data that is coming in from the device? The database must be designed to accept these data when they are electronically imported and to automatically link the data to the correct subject. The same holds true for any clinical laboratory data that will be directly imported.

Finally, a set of management and outcomes reports will be coded. Management reports will enable managers and sponsor to follow the progress of the study without revealing study outcomes. A typical list of manager's reports includes: [1] Data entered by subject, [2] Data queries by subject, by site, by monitor, [3] Missing data, [4] Adverse events, and [5] Data received but not yet entered. One type of summary report is shown below. Here, each subject is represented by a row, and the case report forms make up the columns. The forms are color-coded to show if the data are entered (blue), missing (gray), outstanding due to queries (red), or received but not entered (yellow).

Outcomes reports might consist of, say, [1] time to an event, [2] number of events at day 14, [3] correlation of diagnosis with gold standard, or [4] any other calculated result.

...Database validation
To validate the database, three or four sets of faux patients are entered into the database. The phony data are entered, reports are run, and the answers are checked against manual calculations. In this way it can be confirmed that all the tables are synchronized, edit checks are firing properly, correct data types are being accepted as specified in the data dictionary and map (below), calculated fields are functioning, and database queries are working.

...Version control
The final touch to the database application is to invoke a version control system. Once the design of the database is validated, the Access or Excel file is 'checked in' to a library such as Microsoft Sharepoint or SourceSafe. Then, whenever someone wants to make a change to the code they check it out of the library, make their changes, and check it in again. The version control software will automatically increment the version number, record who made what changes, and provide a space for notes.

...Convergence: merging data and database application
After the database application is validated it is ready to receive the 5000 pages of case report form data from the mid-sized study described above. This involves entering 50,000 data points! In our perfect world, our easy-to-enter forms are forwarded to the data management center where the Data Enterer enters the data without error. The data may also be re-entered, verified, or reviewed by a second person to improve data quality.

Once all data have been entered and reviewed, edits run, and those very expensive queries resolved (see below), there should be no missing or illegible data and no outstanding data queries. The digital data from the device are forwarded and match exactly with the patient numbers when uploaded. Ditto for the laboratory data.

...Data audit trails and other Part 11 issues
As data are entered into the database an independent, but corresponding, record is kept of what data are entered or changed, when, and by whom. This creates an audit trail of the data (as opposed to version control of the database application). Affordable audit trail/Part 11 add-ons are commercially available for both Access and Excel (still the most popular application for data management). The audit trails will track every entry and every change to data fields, generating a complete history of data entry and data changes.

...Data validation and lock
Earlier we validated the database, confirming the fields were the correct data type, which related tables were properly linked, and reports were accurate. After data entry, we validate the data itself. I like to print a report that 'looks' like a case report form. I print out, say, 10% of the forms and manually compare them to the case report forms used for input. And I print out a report of endpoints. A common level of error acceptance is 0% for pivotal endpoints and 0.5% for other fields. Finally, the data can be locked, imported into SAS, reports can be run, statisticians can analyze, and all is right with the world.

Electronic Data Capture
Viewing it simply, in a paper-based system the data are transcribed onto case report forms by the Form Filler at the study site. After source verification, the data are forwarded to the data management center for data entry by a Data Enterer. In electronic data capture (EDC) the validated database is uploaded to a server and study staff are given access via a password and secure connection. Rather than fill out a paper form, the Form Filler is also the Data Enterer. This is where you'll have a cost savings because the work isn't duplicated and a Form Filler/Data Enterer is entering data directly from the source files.

EDC systems are a worthwhile investment if your company expects to do many trials with similar designs and your investigators are comfortable with computers and applications.

How it usually works
Unfortunately, life is rarely a straight-forward path, and we often begin in the middle. The numbers of things that can go wrong are too many to enumerate. But here are a few likely problem-makers.

...Questions change
It seems so minor, changing a question on the case report form so it makes more sense to the Form Filler. But if you change the data type, it might be hard to merge data for that question from before and after the change. Even worse, you are looking at a couple of hours of database re-design and re-verification.

...Differing subject ID numbers
How can it be? Subjects are numbered 1-100 in the database, but 20100607.1-20100607.100 from the laboratory. As humans, we can see it is a date followed by a number, but our database does not understand. Another half-day is lost learning there is a problem, identifying the problem, and fixing the problem.

...Adverse events
Adverse event data can require a lot of time to manage. The event must be followed until it is resolved or stabilized, and that may not happen for several months. Ongoing adverse events keep the database open and require continuing updates for an indeterminate length of time. Ongoing reports to Database Monitoring Committees or Clinical Events Committees also demand extra time.

...Missing data
Unfortunately, some forms will come in with missing values. When a value is missing, i.e., a question is unanswered, the integrity of the data are compromised. If the missing value is a pivotal endpoint, the Form Filler will be asked by data query to track down the missing data. If the value is no longer available, (let's say you wanted the patient's temperature immediately after the procedure), the statistician may be able to guess at a defensible value for the missing data, but it is still a guess.

...Data queries
Paper case report forms are written in handwriting. In spite of the best monitoring and source verification, some illogical or illegible data will slip through. "Now, let's see", thinks the Data Enterer, "is that a six or a zero?" A data query is sent to the monitor, who sends it to the Form Filler, who looks up the correct answer in the source file and sends it back to the monitor, who sends it to the Data Enterer. This is good for 15 minutes of Data Enterer time, 15-30 minutes of monitoring time, and 30 minutes of Form Filler time. Not counting the Form Filler, we can assume $125 per data query. If there is one data query per subject, that would be $62,500!

Thankfully, in the EDC world, illegible handwriting is not an issue for the data manager as the entry has already been reviewed by the site. Typically, in EDC studies, less time is spent dealing with handwriting and missing value queries and effort can be focused on more complex queries.

...Data arrival
When data are coming in from multiple sources, it doesn't always arrive at the same time. You may have all the device data for subjects 1-37, the paper case report forms for subjects 1-30 except for subjects 3 and 23 which were held back, and no laboratory data. You'll need ongoing management reports to show what data has been received for what subjects, and what data the monitors should be tracking down at the site.

Communication
Throughout all of this, communication will be recorded by email messages. Messages should be written clearly and with detail. All phone conversations are summarized with a list of action items. When referring to a database or spreadsheet, use its name and version number. Communication is not only meant for the immediate reader, it is meant to be meta-data (data about the data) which can serve to reconstruct events at a point in the future. For contractors this is especially important; billing issues are usually resolved based on written communications.

Periodically, all the messages are selected and printed (memo style) to Acrobat Writer. The long pdf file is bookmarked with relevant tables of contents, burned to a CD, and stored as a permanent part of the technical file.

Not just CDG's numbers
I used my own data, of course, but I also called upon the goodwill of several friends to assemble the hours estimate below. A perfectly designed, perfectly executed study will not come with free data management.

How Clinical Design Group can help
Building a database, validating the database, invoking audit trails, rendering the database Part 11 compliant, entering and importing data, validating the data, exporting the data for SAS analysis, and documenting communication are all part of the data management services that CDG offers. Whether you want a paper-based system or an electronic data capture system, CDG has the capabilities and experience to help you.

Do you need all of these services, only part of them, or consulting to help you do it yourself? You can contact Clinical Device Group for help. Here are some of the Network Staff and resources who we have at to help you.

...Meet Wessam Sonbol, Database Designer
Wessam Sonbol is the President of Clinical Systems Consultants Inc. Mr. Sonbol graduated from the University of Minnesota’s Institute of Technology with a degree in computer science. Throughout his career he has held positions at Eminent Research/PPD, American Medical Systems and Sorin Group. He developed a global database with which he gathered data from hospitals all over the world to review cardiovascular diseases and understand the demographics of different causes. Wessam also developed http://www.tctmd.com, an informational website for doctors focusing on Trans Catheter Therapeutics; and Global View, a post-market eRegistry tool used by many organizations such as St. Jude and J&J.  He also developed studies using Oracle clinical, Clindex, ERT and Rave and designed a MS Access 21 CRF 11 compliant database. 

...Meet Michelle Secic, Biostatistician
Michelle Secic is the President of Secic Statistical Consulting, Inc. where she works on a variety of medical research projects.  She aids companies around the world with the statistical aspects of their research from small projects to large clinical trials involving drugs or medical devices. She received her Master of Science degree in applied statistics in 1990.  She worked at the Cleveland Clinic Foundation from 1990 through 2001, when she left to continue expansion of her own consulting company.  She authored a book in 1997 titled "How To Report Statistics in Medicine", published by the American College of Physicians. The book was translated into Chinese and published in China in 2002; the second edition was released in 2006.

...Meet Dr. Robert Thiel, Biostatistician
Robert Thiel, PhD is the President of Thiel Statistical Consultants. Dr. Thiel has degrees in physics, clinical counseling and psychometrics. He is an expert in the analysis of IVD data for the Medical Device Industry with more than 75 510K and 5 PMA approvals. He has published papers in the areas of free PSA, breast cancer, ovarian cancer, liver disease and multivariate analyses as applied to sports (baseball and basketball).

His current interests lie in the area of Bayesian statistics, bootstrap and permutation test methodology, and the application of these methods to small sample trials.

...Meet Clindex, a web-based EDC system
Clindex (Fortress Medical Systems LLC, MN) offers a Clinical Data Management System, Clinical Trial Management System, and EDC System in one integrated solution. The software is designed for use with medical devices and diagnostic products (and okay, with pharmaceutical trials; but those of you who know me know I don't do drugs....) Clindex is used from start-up firms to Fortune 500 companies. It has been used in hundreds of clinical studies, international studies, and studies that supported FDA marketing applications. The FDA has inspected numerous companies using Clindex with no feedback of any findings or deficiencies.

I look forward to hearing from you!

Best Regards,

Nancy J Stark, PhD
President, Clinical Device Group Inc
773-489-5706
[email protected]

Deviations and Amendments

Deviations and Amendments Workshop July 23, 2008

What's a deviation, anyway? What, exactly, is a protocol deviation, and is it different from a violation? Deviations are defined in the regulations, but violations are not. IRBs have many opinions, FDA has yet another. The IRB Regulations are little help here, so we'll turn to the IRB professional associations to learn their views on definitions, distinctions, and reporting requirements. The FDA Deviations and violations are fundamentally different from amendments. In the first, we document the event after the fact; in the second we obtain permission before the fact. We will use the rules for IDE supplements as a guideline for managing trial deviations and violations. Learning Objective: The goal is to be able to identify a deviation when it occurs, properly categorize it, and determine its reportability to the IRB and FDA. The Workshop: Workshops are informal, interactive, and about 3 hours in length—they are not lectures. Each learner is required to logon individually at their own computer to enhance and track their participation during the event.  The material is presented with PowerPoint slides accompanied by extensive notes and reference material; in fact, each learner has the entire presentation in writing. The presentation is followed by an online quiz to be taken within two weeks. You must participate in the presentation, take the quiz, and provide feedback in order to receive CEUs and a Certificate, but the good news is you will receive CEUs for time spent on the quiz and giving feedback.    More information: Presented by Nancy J Stark, Deviations>>