CDG Whitepapers

Quality Management for Clinical Research? Yes!

A Whitepaper by Nancy J Stark, PhD

©2015 Clinical Device Group Inc, all rights reserved.

Not everyone agrees that there should be a quality management system for clinical research; that is, the organizational structure, procedures, processes and resources needed to implement quality control, quality assurance, and quality improvement for clinical research. They argue that there should be one quality management system for a company, and quality from all the various functions should flow naturally from the one system.

I disagree. I see a corporate quality management system as having many sub-systems, one for each critical function. Clinical research is one of those critical functions—correctly viewed as a type of manufacturing validation process—but requiring its own unique set of procedures, processes, and resources; operating under a unique mission statement; and set apart from the mainstream of the company because of externally imposed ethical principles. [1. FDA] [2. ISO] [3. OHRP]

Clinical investigations are the study of medical devices in human subjects, where the devices are not cleared or approved for commercialization by regulatory authorities. Hence, the devices are called 'investigational'.

Clinical research is a broader term, used to encompass clinical investigations, registry studies, IVD studies, feasibility studies on new technologies, or any application of a device or technology that can treat or test a human subject or specimen.

A study is a detailed examination and analysis of a subject. [4. dictionary.com] It is not a regulatory word, which makes it useful because it doesn't imply anything beyond its literal meaning. I use the word study to refer to a clinical project with a beginning, middle, and end; regardless of whether the project is a clinical investigation or other clinical research.

 

Crafting a Quality System for Clinical Research

I view the clinical research function as a business within itself. Your "customers" are top management, the product development team, marketing, regulatory, FDA, Notified Bodies, and foreign regulators. Your "products" are data and reports about data. A quality management system provides the written guidance, how-to instructions, forms, tools, and references to implement clinical research from project initiation to completion and to produce reports that are accurate and give management the information they need to make decisions.

You need to know the tasks required to complete a clinical research project, a standard of quality for each task, and a way to measure the quality of each task. The quality management system will lay out the tasks, provide templates and tools for each task, and provide methods for measuring quality as you progress through to project completion.

But on a broader scale, you need to understand the contribution the clinical department is expected to make to the company, what projects are under the departments purview, and how the clinical department interacts with other departments within the firm. In other words, what are the clinical department's vision and mission?

I find it useful to divide a clinical research quality manual (the manual is a binder which houses the quality system) into sections which mimic the progress of a clinical project:    

1. The Quality System    
2. Prestudy Tasks    
3. Study Tasks    
4. Study Close-out Tasks    
5. Database Design    
6. Data Entry & Analysis    
7. Other Tasks
8. Glossary
9. Glossary

 

Section One: The Quality System

The Quality System section describes the management system itself. Here, policy decisions about the scope of the department's commitment, responsibility, and authority are set forth. It begins with a series of statements. First is a statement of commitment issued by the chief executive officer. The statement is a promise that the company stands firmly on the side of quality, and is made by the CEO to the employees.

The second statement sets forth the vision, mission, and quality for the department. A vision is a dream or imagining of the department's function—for example, the function might be to assure that safety, efficacy, and performance claims are supported by adequate human data whenever non-human data are insufficient.

A mission is a purpose, a reason for existence—for example, the mission might be to assure all human data are collected in compliance with good clinical practice.

Quality is when a thing does what it's supposed to do—for example, quality data and reports reports about data, should accurately and clearly present the data and should meet the reader's need for information.

An operations statement describing when clinical investigations are required and how they fit into the product development cycle is very useful. I use a cross-sectional flowchart to describe the clinical department's role in the concept, prototype, prepilot, pilot, and production phases of product development.

Importantly, section one includes a procedure for the style of a written procedure, and a procedure for the rules of communication with team members located in different geographic areas.

 

Section Two: Prestudy Tasks

Eighty percent of your project budget will be spent on prestudy tasks, eighty percent of the project budget will be spent before the first device is applied to the first subject. It's really important that upper management understand this. A prestudy process, executed with care, typically requires six to eight months. It hurts to see time and money being spent and no data coming in, but over and over again I have seen studies fail because of poor prestudy work or too aggressive a timeline.

There are two ways to set up operating procedures: wide and shallow or deep and narrow. A wide and shallow procedure might group all the prestudy tasks into one procedure; with numerous plans, reports, and work products resulting from it.

A deep and narrow procedure is focused on one task and results in one work product. For example, there are procedures for writing a protocol, designing consent forms, designing case report forms, addressing adverse events, and every other pre-study task that results in a work product. We've identified 26 distinct tasks to be executed in the prestudy phase.

The value of a deep and narrow method is that each procedure focuses on a single activity, and the list of procedures—taken together—provide a schedule of prestudy activities.

 

Section Three: Study Tasks

Everyone is excited when study tasks begin; finally you have the first device on the first subject and the prospect of feedback about its performance. The study execution process moves away from the company and takes place primarily at the investigator's sites. Whether your clinical research associates do the monitoring or if you hire separate employees to do the monitoring, someone will have an active travel load.

Don't skimp on the monitoring process. After spending all that time and money setting up a study, make sure a qualified individual travels regularly to the site to assure the device is used per protocol and that problems are spotted early on and fixed. This is the time when data are created; the monitoring process is critical to assuring the data will meet your ultimate goals.

The most important study task procedures are monitoring trips and reports, adverse event reporting, shipping logs, and data monitoring committee meetings. We've identified nine distinct tasks for the study execution process.

 

Section Four: Close-out Tasks

It's tempting to skip study close-out tasks. Everyone is so tired, the focus has turned to analyzing data, and the sites can take care of themselves, right? Almost. Some things really do need oversight: make sure you've recovered all the remaining devices, wrap up financial disclosure reports, follow-up on outstanding adverse effects, and complete the study documentation checklist. We've identified four distinct close-out tasks.

 

Section Five: Database Design

The database is a receptacle for receiving and manipulating data. In very small studies you may be able to use Excel, but for most studies you will use a relational database application such as Microsoft SQL Server. Plan to spend $30-60,000 and six months on database design, even more if you outsource it. Unfortunately, no application comes ready-to-wear, you have to build the database de novo to suit your unique study.

The cost of database design depends on how many sites, how much data, and the duration of data storage: 1) if you want the data to be entered by the investigative site as it is collected, each investigative site needs its own logon portal and each portal costs money, 2) the quantity of data that will be stored on the server is the second cost source. You determine the quantity of data by counting the number of fields in each case report form and multiplying by the number of subjects. The more gigabytes of data the higher the cost. 3) the third cost factor is the duration of the study. The longer server space must house your data the more it will cost.

Procedures for designing a database include planning the tables and records and how you'll relate them, what standard reports you want, creating a change log, assembling a data dictionary, and making a user's manual for the data enterer. We have identified two procedures for database design.

 

Section Six: Data Entry & Analysis

Data may be entered into the database either at the investigative site, at an outsourced data management center, or in-house. The obvious value of having sites enter the data is faster feedback. Don't expect to save money by having investigative sites enter the data. They know that work has been off-loaded from your firm to theirs, and they expect to be paid market-value for their labor.

Entering the data in-house has some special challenges. It means that you take full responsibility for data integrity and Part 11 compliance. Think simple if you are a small firm and want to go this route. Buy a stand-alone computer, keep it off the network, put it in a locked room, and give keys only to the people who have authority to enter data, correct data, and run reports. Management does not get a key.

The procedures in this section discuss who/how/where data will be entered, how data integrity will be verified, how to manage data queries, 'locking' the database, statistically analyzing the data and writing final reports. We include six separate procedures.

 

Section Seven: Other Tasks

This section of the quality manual is a place for housing miscellaneous procedures. We include procedures for using healthy (employee) volunteers, maintaining your own IRB, auditing sites, auditing yourself, and reviewing label copy.

Reviewing label copy closes the loop on a clinical investigation. If the purpose of the investigation is to obtain data to support safety, efficacy, and performance; then assuring there is data on file for each claim in the labeling brings obvious closure to the project.

 

Section 8: Bibliography

The manual itself contains a list of significant references. These references provide the history, background, and necessity for the content of a clinical research quality management system.

I also recommend that even small companies set up a library of training and reading materials. Such books and references often provide useful examples of how to approach problems and make unusual protocol designs.

 

Section 9: Glossary

Every company uses clinical research terms in a manner unique to them. That's fine. The definitions provided in US regulations or international standards may certainly be refined to meet your corporate needs. A glossary of how your firm uses these terms is extremely important, however, because you want everyone in the company assigning the same meaning to the words.

 

References

[1] 21 CFR Part 812 Investigational Device Exemptions.
[2] ISO 14155 Clinical investigations of medical devices in human subjects - good clinical practice (2011).
[3] OHRP - Office of Human Research Protections, www.hhs.gov/ohrp.
[4] Dictionary.com.

 

Formatting Clinical Evaluation Reports

 

A Whitepaper by Charles Hurwitz, PhD, and Nancy J Stark, PhD

A number of clients have commented to me this year that Notified Bodies are rejecting their Clinical Evaluation Reports for nonconformance to the formatting rules of MEDDEV 2.7.1 r3. Some companies are undertaking complete rewrites of their existing clinical evaluations, so concerned are they that the reports will be rejected at the next review cycle. The EU intends to change the formatting rules, however my European colleagues inform me the 2009 version of the guideline is still firmly in effect and that changes are likely to come slowly. In this whitepaper we'll review the format suggested by MEDDEV 2.7.1 r3 and look at some common formatting errors. [1. MEDDEV 2.7.1 rev 3]

A Muddled History

It was people at the Cochrane Library, the US Agency for Healthcare Research and Quality (AHRQ), and the National Library of Congress who first worked with the problem of how to use the existing literature to evaluate safety, efficacy, or performance of products and procedures. The Global Harmonization Task Force Study Group 5 (now defunct) published a document titled "Clinical Evaluation" in May of 2007. (You may download a copy for free from CDG's Vintage Guidances web page.) The Active Implantibles Medical Device Directive of 1990 discussed the essential requirement of clinical evaluations in Annex 7. The Medical Device Directive of 1993 discussed the essential requirement of clinical evaluations in Annex X. But neither directive gave much information about format. Both directives were updated in March 2010 with Directive 2007/47/EC, strengthening the requirement for clinical evaluations and their content, but without much regard to format, saying only that the clinical evaluation shall be documented as a report, regularly updated, and signed appropriately. [2. Directive 2007/47/EC]

Suggested Format from MEDDEV 2.7.1 r3 (2009)

In the year or two following the 2010 revision of the Medical Device Directive many of the Notified Bodies (NB) themselves didn't know how a clinical evaluation report should be written or what their authority was in terms of reviewing them. Any CER that bore even a vague resemblance to what was described in the Directive was deemed to be acceptable. The variety of formats that have crossed our desks has been remarkable.

However as time has passed the NBs have better understood their role in the evaluation process. They have formed their own professional society, the Notified Body Operations Group (NBOG), and have issued best practice guides for their members. With the issuance of a checklist for an audit of a Notified Body's review process, NBs are being held accountable for their decisions and are refusing to accept reports that do not correspond strictly to the MEDDEV 2.7.1 r3 guideline. [3. NBOG CL 2010-1]

Appendix E of the MEDDEV guideline is "a possible" format for a clinical evaluation report. This is the format you are expected to follow for compliance. Reviewers work with a checklist at the back of the guideline when reviewing a CER and if a section or item is missing you may get a nonconformity, costing extra weeks of review time and lost sales opportunities. What follows is a review of the recommended format for a CER. [4. MEDDEV 2.7.1 r3 p32]

1. General Details—A Three-Legged Stool

Think of the clinical evaluation report as a three-legged stool. One leg is the literature review, a second leg is the assessment of your risk management process, and the third leg is the assessment of any clinical investigations your firm has sponsored.

The three "legs" are brought together as attachments to a finalized report, the "seat" of the stool. In the opening paragraphs of the seat you must take care to identify exactly what device(s) are covered by the report; identifying the devices by proprietary name, any code names assigned during device development (one company used the code name "Kermit" because they intended to leap over the competition,) and the manufacturer(s) of the device.

Your firm should assign (or outsource) a medical writer who will act as project manager for preparing the report, have authority to obtain and inspect all necessary records and reports, and have authority to work with external services as needed. In our experience as consultants, it takes about 140 hours of labor over a three month duration to write a clinical evaluation.

2. Description of the device and its intended application in or on the body

Provide a concise physical description of the device, cross referencing to relevant sections of the manufacturer’s technical information as appropriate. The description should cover information such as:

• materials, including whether the device incorporates a medicinal substance (already on the market or new), tissues, or blood products;

• the device components, including software and accessories;

• mechanical characteristics; and

• other characteristics, such as sterile vs. non-sterile, radioactive, and the like.

You should think like the sales department of a successful department store: "there shall be no barriers to customer buying." In a CER case, there should be no barriers to the Notified Body quickly understanding your technology. Anticipate the questions a naïve reviewer will have and address the questions in the seat of the report. It's useful to include diagrams, drawings, photographs, comparative tables, lists of animal safety testing, text boxes, and the like. These tools help the reviewer to grasp the essence of your technology without doing a lot of independent research.

Don't forget to state the intended application of the device – single use or reusable; invasive or non invasive; implantable or not; duration of use or contact with the body; organs, tissues or body fluids contacted by the device; is the device used in the home – and describe how the device achieves its intended purposes.

Sometimes clients forget about the processing aids or biological substances used during the manufacturing process. Metal parts are routinely washed with solvents to remove oily residues, biological products may have their origin from unidentified sources of tissue (fish from the sea.) Sometimes the final processing takes place in the physician's office or lab. You should be able to demonstrate the absence of any inadvertent residue, or that any such residue does not interfere with safety or performance.

3. Intended therapeutic and/or diagnostic indications and claims

Devices are physically applied on or in the body for a reason: to treat, mitigate, cure or diagnose. You must clearly state the medical conditions (indications) and discuss the medical background for each condition. Reviewers are not experts in all fields of medicine. Your goal as the medical writer is to remove all barriers to clarity, understanding, and full information.

You then identify the target treatment groups and diseases for the medical conditions. Who are the patient groups that will be exposed to the device? Let's say the medical condition is pregnancy. The likely patient group is young, healthy women with no concomitant illnesses or genetic variants.

What are the diseases or disease subgroups for the medical conditions? Again, suppose the medical condition is pregnancy. You may be targeting pregnant women with diabetes, preeclampsia, or other issues. Finally, list the specific safety or performance claims (intended uses) you will make for the device and defend them.

You will have some strategic issues to deal with along the way. For example, do you want to mix device indications and, say, cosmetic indications in the same clinical evaluation? Or is it a smarter to submit an evaluation for medical use today and amend it for cosmetic use in the future?

4. Context of the evaluation

Outline the developmental context for the device. The information should include whether the device is based on a new technology, a new clinical application of an existing technology, or the result of incremental change of an existing technology. The amount of information will differ according to the history of the technology. Where a completely new technology has been developed, this section would need to give an overview of the developmental process and the points in the development cycle at which clinical data have been generated. For example, a manufacturer developed an resorbable container in which a patient's tissue was held in place until it could take hold and grow, as opposed to being held in place with a sling of sutures. The CER included data about both the resorbable container and preparation of the patient's tissue.

For a long standing technology, a shorter description of the history of the technology (with appropriate references) could be used. For example, the CER for a new wound dressing would not need to trace the history of wound dressings all the way back to the Romans.

Clearly state if the clinical data used in the evaluation are for an equivalent device. Identify the equivalent device(s) and provide a justification of the equivalency, cross-referenced to the relevant non-clinical documentation that supports the claim. Not to be confused with substantial equivalence, MEDDEV 2.7.1 r3 equivalence (page 42) means, "1) Clinically: used for the same clinical condition or purpose, at the same site in the body, in similar population (including age, anatomy, physiology); have similar relevant critical performance according to expected clinical effect for specific intended use, 2) Technically: used under similar conditions of use; have similar specifications and properties e.g. tensile strength, viscosity, surface characteristics; be of similar design; use similar deployment methods (if relevant); have similar principles of operation, 3) Biologically: use same materials in contact with the same human tissues or body fluids."

State the Essential Requirements relevant to the device in question. In particular, describe any special design features that pose performance or safety concerns, e.g. the presence of medicinal, human or animal components. These features should be identified in the device risk management documentation as requiring assessment from a clinical perspective.

Sources of data

Outline the considerations used to choose the clinical databases for the evaluation. Usually we see publication sources such as PubMed or MAUDE, but rarely do we see a discussion of why these databases were chosen. Provide an outline of the search and retrieval process for scientific literature, including the actual key words and search strings, and cross-reference these to your standard procedure for searching and reporting the literature (you have one, of course.)

5. Summary of the clinical data and appraisal

Provide a tabulation of the clinical data used in the evaluation, categorized according to whether the data address the performance or the safety of the device in question. (Note: many individual data sets will address both safety and performance.) Within each category, order the data according to the importance of their contribution to establishing the safety and performance of the device and in relation to any specific claims about performance or safety. Additionally, provide a brief outline of the data appraisal methods used in the evaluation, including any weighting criteria, and a summary of the key results.

Include full citations for literature-based data and the titles and investigation codes (if relevant) of any clinical investigation reports. At CDG, we begin an independent Excel spreadsheet as soon as we start reviewing abstracts. The spreadsheet lists the complete title and citation for the abstract, and a justification if the full article is not reviewed further. The spreadsheet is provided as an attachment to the CER.

Cross-reference the entry for each item of data to its location in the manufacturer’s technical documentation.

6. Data analysis

6.1 Performance

Provide a description of the how the literature analysis, or analysis of sponsor-acquired clinical data, was used to assess performance.

Identify the datasets that are considered to be the most important in contributing to the demonstration of the overall performance of the device and, where useful, particular performance characteristics. Outline why they are considered to be “pivotal” datasets and how they demonstrate the performance of the device collectively (e.g. consistency of results, statistical significance, clinically significance of effects.)

6.2 Safety

Describe the total experience with the device, including numbers and characteristics of patients exposed to the device; and duration of follow-up of device recipients. You might include the number of patient-days of exposure, if some patients are exposed or followed longer than others. You should explain if patients were lost to follow-up.

Provide a summary of device-related adverse effects, paying particular attention to serious adverse device effects.

Provide specific comment on whether the safety characteristics and intended purpose of the device require training of the end-user.

6.3 Product literature and instructions for use

State whether the manufacturer’s proposed product literature and Instructions for Use (IFU) are consistent with the clinical data. Does the literature cover all the hazards and other clinically relevant information that may impact on the use of the device? Implanted or home use devices require special consideration, describe how patients will be educated on the care and use of their device and how they can obtain help or additional information, especially outside of business hours.

7 Conclusions [to report]

Outline clearly the conclusions reached about the safety and performance of the device from the evaluation, with respect to the intended use of the device. State whether the risks identified in the risk management documentation have been addressed by the clinical data.

For each proposed clinical indication state whether: • the clinical evidence demonstrates conformity with relevant Essential Requirements; • the performance and safety of the device as claimed have been established; and • the risks associated with the use of the device are acceptable when weighed against the benefits to the patient

A word about risk assessments

The contents and process of assessing risk is poorly discussed in the MEDDEV 2.7.1 r3 guideline. Here we discuss the process as it is usually performed at CDG. We begin with examining the Instructions for Use (IFU) and the most current Risk Assessment Report (RAR) per ISO 14971. Do the clinical data reveal any new risks not already discussed? If yes, you may need to update the RAR and IFU. [5. ISO 14971 2012]

Preparing a spreadsheet listing the clinical benefits and risks is a good strategy for grasping their scope. Describe the nature of each benefit and risk in a column and the anticipated outcome in the next column. Indicate the anticipated frequency of occurrence of each benefit and risk in column three. In column four, identify type of adverse effect likely to occur. Next, indicate if the risk would result in a mild, moderate, or severe adverse effect; for example, a headache rarely meets the criteria of serious, yet it could be severe. Sometimes we indicate if the risk is associated with the procedure or the device itself. Finally, indicate how the risks have been mitigated. We are trying to gather together as much information as possible to understand the overall risk of the device.

Subjectively, we then compare the benefits and risks to determine if one outweighs the other. A number of factors may come into consideration. For example, let's say that two patients suffer from spondylolisthesis, a defect in the spine which causes vertebra to slip to one side of the body. Let's also assume that one patient is in her thirties and the other in her seventies. Knowing that it takes a year to recover from corrective surgery, spinal fusion may make sense for the younger patient for whom pain management over fifty years of remaining life is unreasonable. Spinal fusion may not make sense in the older patient, whose life expectancy is ten more years, and who does not want to spend 10% of their remaining lifetime recovering from surgery.

Notified Body Check List

After you've written the report you should check your work using the Notified Body checklist on page 35 of the MEDDEV guidance. Here you will find a thorough and detailed itemization of the points you should include in the report. If you meet all of the criteria in the checklist, a Notified Body will have little power to reject your evaluation based on format alone. [6 MEDDEV 2.7.1 r3, p 35.]

References

[1] MEDDEV 2.7.1 r3, Clinical Evaluation: A guide for manufacturers and Notified Bodies, 2009.
[2] Directive 2007/4/EC, Official Journal of the European Union, L 247/21.
[3] NBOG CL 2010-1 Checklist for audit of Notified Body's review of Clinical Data/Clinical Evaluation.
[4] MEDDEV 2.7.1 r3, p32.
[5] ISO 14971 Application of Risk Management to Medical Devices, 2012.
[6] MEDDEV 2.7.1 r3, p 35.

 

The 2014 Clinical Research Associate Handbook Is Here!

The 2014 (Paper) CRA Handbook Is Here!

A Medical Device Handbook for Regulatory and Clinical Research Professionals

A Promotional Piece by Nancy J Stark, PhD.

The CRA Handbook was the first "book" I ever edited. I was jealous because, at the clinical and regulatory conferences I attended, little books of Part 312 for pharmaceutical clinical investigations were handed out for free by the vendors. But there was nothing available for the poor Cinderella of medical devices. To balance the scales, I wrote my own handbook for device clinical and regulatory professionals and added some extra reference materials to make it even more useful.

Devices Are Different from Drugs

If you work in the medical device world, you know that good clinical practices for devices are vastly different from pharmaceuticals in implementation.

Size and Storage of Investigational Products
The first big difference is with size and storage. Once I had a device the size of a breadbox. The IRB insisted the pharmacy keep the investigational breadbox-sized devices under double-lock and key, as was their practice with investigational drugs. But pharmacies don't have storage room for 100 breadboxes, and pharmacies don't dispense medical devices. The investigator had storage room, but no double-locked box for large devices. After weeks of negotiation--and costly delays to the investigation--the IRB agreed to allow the investigator to store the large devices in her office with a single lock on the door. The unique storage requirements of devices is unfamiliar to drug folks.

CRA Handbook 2014
I use the CRA Handbook as a desk reference for US regulations, a list of guidances, and regulatory and clinical requirements. When I traveled, I used it as a reference tool during meetings and for late-night hotel work. By the end of the year, my copy was dog-eared, filled with annotations, and colorful with Post-It tags. All my favorite paragraphs were marked.  To learn more, go here.

Initiation of Investigations
The second big difference is that there are two levels of approval for study initiation in devices. In the US, device clinical investigations may be conducted under the abbreviated IDEs or under full IDEs (Investigational Device Exemptions). Abbreviated IDE studies make up 80-90% of the clinical investigations in the States, do not require FDA approval for initiation, and do not require a final report to FDA. There is no registry or list of abbreviated IDE studies. I'm getting my numbers from private interviews; for example, manufacturers of dermal contact products might do 100 clinical investigations a year on healthy, volunteer employees. While full IDE studies get all the attention, so-called 'non-IDE' studies are confusing to drug folks.

Adverse Effect Reporting
The third big difference between device and pharmaceutical investigations is with reporting of adverse effects and events. Adverse effects are device related, adverse events are not. Unless otherwise specified by the protocol, only USADEs (unanticipated, serious, adverse device effects) are reported to FDA. Not reporting anticipated adverse effects to FDA is befuddling to drug folks.

What's New in the 2014 Edition?
Two major things changed in the regulatory world since 2011 and are reflected in the 2014 edition of the Handbook. The Declaration of Helsinki was updated to reflect the changes made in by the World Health Association in 2013, and 21 CFR Part 812.119 on investigator disqualification was expanded and strengthened. What's missing?

Should we add other documents to the Handbook to make it more useful to you? Tell us what to add.

The Table of Contents
00 Foreword
01 CFR Part 812-Investigational Device Exemptions
02 CFR Part 812.20-Premarket Approval Applications
03 CFR Part 11-Electronic Records: Electronic Signatures
04 CFR Part 50-Protection of Human Subjects
05 CFR Part 54-Financial Disclosure
06 CFR Part 56-Institutional Review Boards
07 CFR Part 99-Dissemination of Information
08 CFR Part 809-In Vitro Diagnostic Products
09 CFR Part 820.30-Quality System--Design Controls
10 Expedited Review List
11 Risk Based Monitoring--The New FDA Guidance
12 Declaration of Helsinki (2013)
14 ISO 14155 (2011) Summary
15 Adverse Events and Effects
16 FDA Guidances

How do I get one?
Go to our website.

The electronic version of the handbook is in the works.

Julie is editing as fast as she can to get the electronic copy updated, functional, with working links and pretty colors. She expects it to be completed by June 2014. If you would like to be notified when it is ready, contact Peggy and have your name added to a notice list.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Follow us on twitter 
If the link doesn't work in your system, email us at [email protected] or visit our website at www.clinicaldevice.com 

Medical Devices in the Home—FDA's Home Use Device Initiative

An Op-Ed by Nancy J Stark 23 September 2013 In our lifetimes, each one of us will be either a caregiver or a care-receiver. 1. Brady It's been a year and a half since my last whitepaper. Several of you have asked why I stopped writing. My husband Frank was diagnosed with cancer in late 2010 and my role in life quickly changed from author and business-woman to caregiver. He passed away in April 2012 as the immediate result of a shoddy medical device and a shift in business practices that uses poorly regulated specialty pharmacies as distributors. Care-giving in the Home When I was a care-receiver in 2000, my husband took care of me and my hospital took responsibility for my home-use IV therapy. Over six weeks of treatment, that Abbott IV system never failed once. 2. Abbott When I was the caregiver for my husband in 2012, his Curlin 4000 CMS IV system failed every 6-8 hours around the clock. 3. Moog Curlin An incredible burden is placed on family caregivers today. We are given a burden that goes far beyond basic assisted living duties of bathing, cooking, cleaning, shopping, changing diapers, and routine errands. Today the family caregiver is expected to operate sophisticated medical equipment, recognize the signs of disease progression, treatment failure, and make critical patient assessments, all without adequate training or skills or support. I believe the device industry must take proactive responsibility to ease this burden and protect its reputation. The American Medical Association tells us that 16% of caregivers report their own health worsening since taking on the caregiver role (mine did); 40% incur financial expenses for (non tax-deductable) care related products, services, and activities (I did); and 20% spend up to 10% of their monthly income on caregiving activities. 4. AMA Some lose their jobs when long-term leave expires. Loss of health-care coverage can follow soon after. In my own situation I was forced to stop working for two years; I supported my family and paid insurance premiums with our life savings. Division of Labor On 19 March 2012 my husband was admitted to Northwestern University Hospital with a painful lump on the back of his neck. We already knew he had a type of cancer that almost never metastasized, but nevertheless we were worried. Here is Frank's photograph, goofing around in the ER using his belt to hold a hot-pack in place on the back of his neck. I'm showing you his picture so you'll know I'm talking about a real human being who played a full and critical role in my life. Frank C Suarez, March 2012   After a work-up, the lump was diagnosed as an abscess caused by a Staph infection. Frank was started on an IV and oral antibiotics and began improving immediately. After a few days of treatment, we were ecstatic with hope and confidence. And then, on March 28, he was sent home with a Curlin 4000 CMS IV pump and Curlin IV administration sets. He was to receive an additional six weeks of IV antibiotic therapy at home. We had every hope of a positive outcome and every expectation that the specialty pharmacies in charge of his health would provide quality drugs and devices to support our needs. His meds and tubings were provided by Walgreen's Infusion Therapy. The pump was dispensed from another firm called Greatland Home Health Services but had a Walgreens label on it. I never was clear about who the home health care nurse worked for. A 24/7 support line was provided by someone to answer any questions that might arise at home. The Curlin 4000 IV Pump   At 10 PM the first night the pump started to beep incessantly. A mystery code was displayed on the screen. We couldn't shut the noise off, and there were no instructions for use. My husband was infected, feverish, tired, weak, unable to concentrate, focus, or remember the five minutes of 'training' someone had given him at the hospital. I was tired, worried, scared, untrained, and alone with someone's life in my hands. After extended deliberation, we remembered the promised 24/7 support line for the device. When we called the 1-800-number we got an answering service. The individual said they didn't know anything about IV pumps, they were just an answering service, and they would have someone call us back. Try to imagine the urgency of the situation, the loud screeching and unending noise this machine was making late at night, and our frightened helplessness and vulnerability. Several minutes later, a return caller translated the secret code as meaning we needed to change the battery. Wouldn't you expect a home medical device to be dispensed with a fresh battery? To compound our desperation, a previous user had jammed the lid to the battery closet on so tightly that we could not get it off. My husband was weak and could barely open a can of Pepsi. I have a small frame and arthritis in my hands; I need a pair of pliers to open a bottle of Ensure. The woman on the phone said it didn't matter that we were promised 24/7 support, they were not going to send a technician out after hours to help us. Finally my husband used a hammer and screwdriver to pry off the battery lid. I said a silent prayer for revenge hoping the damned machine would be destroyed. Wouldn't you expect a home medical device to go through a quality check before being dispensed? Two or three times a day over the next few days we would get error messages called "down occlusions," accompanied by loud screeching beeps and a pump that didn't deliver life-saving medicine. Since there were no instructions for use, I had not been trained, and Frank was never educated about error codes, we called the 1-800-number again. The return caller told us that a down occlusion meant there was an air bubble somewhere between the pump and the bag. The air bubble was always in the region of the pump rollers, where the tubing was placed inside the pump. The IV set was built from two lengths of PVC tubing, attached to each other by a short segment of soft silicone-like tubing—a common construction for peristaltic pumps. The soft tubing went inside the pump and over the rollers so the rollers could move the fluid forward. This seemed to be the source of the air leak. I learned to take the tubing out of the pump, massage the air bubbles backward into the bag, replace the tubing, and restart the pump cycle. I worried that if air could be drawn in from the atmosphere, what other contaminants could be drawn in? A Final Failure On Easter evening, 1 April 2012, my husband and I attended mass. Frank wanted to shut the pump off for the duration of the service so that screeching alarms wouldn't disturb the congregation. Because he wanted to be independent, he was the person who monitored the pump for failures. The primary family caregiver is expected to assess the patient and make the most important medical decision of the patient's life: should they be taken to the hospital or can they make it safely through another night? Although I asked the doctors on two separate occasions to educate me about what to watch for, all I was ever told was that if Frank was in trouble he would be 'disoriented' or 'unable to concentrate'. I had visions of a cousin with dementia who ran away from home one cold winter night. To my eternal heartbreak, Frank forgot to turn the pump back on after mass. We discovered the pump was off about noon the next day. The clue was that it hadn't screeched at us for twelve hours. There was an air bubble 18 inches long in the tubing. I didn't know how to disconnect the tubing from a PICC line and replace it so I had no choice but to start the task of massaging the air bubble backwards into the bag. I had to fight to get a health-care nurse come out that same day. At 4 PM the nurse arrived and calmly suggested that Frank be taken to the ER. He simply said, "He is dehydrated." I had no understanding of the significance of those words. Finally in the ICU, the doctors told me the dehydration had caused his liver to shut down and toxins had built up in Frank's body and brain. If he survived there would be brain damage. Frank was calling out "love out, love out" and I thought he was telling me he no longer loved me. Later I realized he meant "lights out", but he couldn't grab the right noun. The doctor and I had conversations about whether or not to "put the paddles" to him. A few days later Frank came home to die. The next day the administration set fell off the PICC line and he started to bleed all over his clothing, blankets, and bed. The specialty pharmacy, who promised to deliver such specialized care, actually asked me to rebuild the PICC line following instructions from them over the phone! My husband passed away on 20 April, exactly four weeks after being diagnosed with an infection. If he hadn't been sent home, if we had been given decent equipment, if we had been given picture-based instructions-for-use, if we had been given training, if the home care nurse had come daily, if the tubing hadn't failed regularly, if I had been told the signs and danger of dehydration, if, if, if... he might have lived a few weeks longer. I used to think that if you have lived for 70 years what difference can one more week make? Now I know that the last week of life may be the most intimate, the most loving, the most important that a couple can share. Thirty-two days after his death, many models of CURLIN IV administration sets were recalled, including the model number dispensed to my husband. Conveniently, the specialty pharmacy did not keep records of which lot numbers Frank received. The Blameless Middle-Man—Specialty Pharmacies We all know the story of the meningitis infections following careless quality practices at the NECC compounding pharmacy in Massachusetts. That's when we learned that FDA has limited authority for inspection and enforcement over these firms. But the issue came to my house when I wrote to Martin J Berardi, President of Moog Medical Devices Group, complaining about the quality of his products. He was sorry for my loss and lack of training, reminded me that Moog has many educational and quality programs in place, "However, in spite of all our efforts...we cannot control what happens with our information after we provide it to the agencies responsible for the use and distribution of the pump." The statement is significant because it illustrates how specialty pharmacies are used as a screen behind which the device industry hides. Moog only makes the devices, what specialty pharmacies do with them is not Moog's problem. In a bipartisan effort, the House introduced HR 3089, the Compounding Clarity Act, last week. The bill would subject large compounding (specialty) operations to regular inspections, fees and other requirements from the Food and Drug Administration, and would mandate FDA to write regulations for these operations. The Senate introduced similar legislation, S 959—the Pharmaceutical Compounding and Accountability Act—in May. 5. House & Senate. FDA's Home Use Device Initiative In 2010 FDA launched its Home Use Device Initiative. The initiative recognizes our aging population and the challenges of using medical devices that are still too complex for an untrained lay person to use safely and effectively. Furthermore, as our health care system changes, hospital stays are becoming shorter, forcing patients who still require sophisticated care to receive that care from untrained or untrainable family members. The Home Use Device Initiative aims to: 1) establish guidelines for manufacturers of home use devices, 2) develop a home use device labeling repository, 3) partner with home health accrediting bodies to support safe use, 4) enhance postmarket oversight, and 5) increase public awareness and education. 6. Home Use Initiative. [1] Practitioner Survey on Medical Device Labeling In 2011, a group of home care practitioners were surveyed for FDA regarding their needs and perceived shortfalls regarding labeling for devices used in the home. The practice areas included respiratory therapy, infusion therapy, and wound care. Practitioners agreed that more pictures and diagrams; use of color, larger font size and more white space; a short and a long version of the labeling; the short version attached to the device; hardcopy and electronic copies; and better paper quality, all were improvements needed in medical device labeling. The practitioners identified instructions for use, troubleshooting, warnings, precautions, contraindications, adverse events, manufacturer's contact information, and lot numbers/serial numbers as the most important items to be included in the labeling. Clinical trials and references were ranked as unimportant. 7. Practitioner Survey. [2] Caregiver Survey on Medical Device Labeling Three months after my husband died I found a Patient Guide for the CURLIN 4000 IV pump, hidden carefully behind Greatland's contract stating we were responsible for any damages. The Patient Guide, originally published in June of 2005, is a copy of a copy of a copy. The guide assumes a level of user sophistication that I find laughable: "...The cursor will be on RESUME, REPEAT Rx, or NEW PROGRAM. Move the cursor to the proper selection...." In my state of stress I could barely function beyond the age of reason, let alone figure out the proper selection. In 2012, a group of 127 caregivers were surveyed for FDA regarding their needs and perceived shortfalls regarding labeling for devices used in the home. These people were caring for adult family members (95%), usually spouses or partners (44%), and the majority had been using their medical equipment for three or more years. The most common medical conditions were stroke, heart disease, diabetes, cancer, lung disease, Alzheimer's or dementia, post-surgery recovery, or simply old age. The most common items of medical equipment used were patient lifts, wheelchairs, blood glucose monitors, specialty hospital beds, nebulizers, respiratory equipment, and assistive equipment. Caregivers indicated that more pictures and diagrams would be useful in the instructions, that most don't have instructions for their equipment, and that the information they need isn't always contained in the manufacturer's instructions; instead, information came from other sources such as friends. 8. Caregiver Survey [3] FDA Accessible Medical Device Labeling Workshop On April 29-30, 2013, FDA held a two-day workshop on Accessible Medical Device Labeling. The purpose of the workshop was to discuss the growing need for medical device labeling to be delivered in a clear, concise, and readily accessible format so that patients, caregivers, and healthcare providers could access and utilize it. In other words, the development of a labeling repository. FDA proposed to develop a standard format and standard content for device labeling. And they proposed that a pdf copy of every version of a device's labeling be maintained in a labeling 'database' to be maintained by FDA. Care receivers often use the same device for many years, but manufacturers commonly replace older versions of labeling as new versions become available. Care receivers using older devices are unable to obtain replacement copies of labeling if the original is lost or unavailable. 9. Accessible Labeling. [4] Draft Guidance on Design Considerations for Home Use Devices The FDA guidance for labeling medical devices for home use, "Guidance on Medical Device Patient Labeling," was written in 2001. The technology changes of the last decade surely need to be addressed in a labeling guidance. An updated draft guidance, "Design Considerations for Devices Intended for Home Use" was published in December 2012. The guidance discusses environmental considerations (pet, children, plants, WiFi), user considerations (cognitive level, arthritis or physical limitations), device considerations (lock-out mechanisms, calibration, mechanical strength, electrical issues, electromagnetic compatibility, wireless technology, and those pesky alarm systems), human factors, labeling, and post-market considerations. 10. Draft 2012 Guidance. For the first time in my career, I submitted comments on a draft guidance to FDA. 11. Stark-Suarez Comments. AAMI Activities The AAMI standards community is working closely with FDA to define healthcare technologies that will meet the higher demands of home use. Joint summits are being held to identify, discuss, and formulate strategic initiatives which will help ensure the safety and effectiveness of medical technologies in nonclinical settings. An AAMI/FDA summit was held on 9-10 October 2013. For more information, contact AAMI at 1-703-253-8270. Caregiver and care-receiver input is especially important to these efforts. 12. AAMI/FDA Summit. Conclusion Training is key. Adults learn in different ways: I need a picture. Some people need a demonstration. Others need the written word. All kinds of training, available on-demand, are critically important and it is the responsibility of manufacturers to make sure this happens. If specialty pharmacies aren't doing a good job representing your device, don't use them. Delivery of training is key. People who are dealing with life and death scenarios cannot Google for instructions. The manufacturer must make certain the labeling is in front of the caregivers face, on top of the contract for service, and permanently affixed to the device. Support is key. If a device fails at 10 PM it is unreasonable to tell the caregiver they'll just have to wait until morning for help. Quality is key. Devices cannot fail in home. Period.   Best Regards, Nancy J Stark, PhD President, Clinical Device Group Inc References 1. Brady,    Mary Brady, Sr. Policy Analyst, CDRH, [email protected] or 1-301-796-608. 2. Abbott.    Abbott Laboratories, Abbott Park, IL. 3. Moog Curlin.    The Curlin 4000 IV pump is manufactured by Moog Medical Inc, 4314 S 670 W  Salt Lake City, UT 84123; 1-801-264-1001. 4. AMA.    http://www.ama-assn.org//ama/pub/physician-resources/public-health/promoting-healthy-lifestyles/geriatric-health/caregiver-health.page 5. House & Senate.     http://beta.congress.gov/bill/113th/house-bill/3089/text     http://www.gpo.gov/fdsys/pkg/BILLS-113s959is/pdf/BILLS-113s959is.pdf 6. Home Use Initiative.     http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/HomeHealthandConsumer/HomeUseDevices/ucm208268.htm 7. Practitioner Survey.    http://www.fda.gov/downloads/MedicalDevices/ProductsandMedicalProcedures/HomeHealthandConsumer/HomeUseDevices/UCM335197.pdf 8. Caregiver Survey.     http://www.fda.gov/downloads/MedicalDevices/ProductsandMedicalProcedures/HomeHealthandConsumer/HomeUseDevices/UCM335198.pdf 9. Accessible Medical Device Labeling Workshop, April 2013.    http://www.fda.gov/MedicalDevices/NewsEvents/WorkshopsConferences/ucm334501.htm 10. Draft 2012 Guidance.     http://www.nancystark.com/FDA_Archives/2012-12-12_Home_Use_Draft_Guidance.pdf 11. Stark-Suarez Comments.    http://www.nancystark.com/FDA_Archives/2013-01_Stark-Suarez_Comments_on_Draft_Home_Use_Guidance.pdf 12. AAMI/FDA Summit    http://www.aami.org/summit2013/index.html

Registry Studies: Just Like Clinical Trials

A Whitepaper by Dr. Nancy J Stark

An old favorite, expanded and enhanced
©2012 Clinical Device Group. All rights reserved.

I see it over and over again. Somehow people think registry studies can have crude protocols, poorly monitored implementation, and insufficient data for analysis and still be good enough to get reimbursement from Medicare. Without solid data we end up with coverage determinations that leave our devices as adjunctive therapies and secondary options, such as: 1) patients with chronic graft versus host disease can only have extracorporeal photopheresis after all standard drug treatment has failed, 1. Extracorporeal photopheresis 2) a complete wound therapy program...must have been...ruled out prior to application of negative pressure wound therapy, 2. NPWT or 3) a CPT code is assigned but no National Coverage Determination is issued for a capsule to study gastric emptying. 3. Ingestible pH and Pressure Capsule You can keep track of these decisions by subscribing to email updates from the Centers for Medicare and Medicaid Services (CMS). 4. CMS

As device people we have to wake up and smell the roses; the decision-makers are only comfortable with randomized, controlled clinical trials. We need to come as close to this mark as possible if we want CMS to issue National Coverage Determination that support robust reimbursement.

Registry Studies versus Clinical Trials
There is only one difference between registry studies and clinical trials: registry studies are observational, they look backwards at what was done without dictating a treatment plan. Clinical trials are investigational, they look forward to what shall be done and dictate and control the treatment plan. Putting it simply, in a clinical trial we tell the investigators how to do it, in a registry study we observe how they actually did it.

Consider this example; a sponsor wants to do an observational study on a commercial device for treating a condition. They need an additional piece of data about the outcome which they forgot to collect in clinical trials. The additional information requires that the patient submit to an unscheduled MRI. This design is not observational. The reason is observational studies—by definition—do not dictate interventions. But the protocol for this study dictates an invasive, unscheduled procedure be given to the subject. It isn't always the primary treatment that moves the study from observational to investigational, it might be an invasive diagnostic procedure you are adding to the mix.

A Few Definitions
There is a reason why there is a hodgepodge of definitions. Many regulatory documents grab onto a word and give it special meaning. For example, Medicare will reimburse for investigational devices but not experimental ones. FDA decides at the time you make an IDE submission whether your technology is investigational or experimental, based on guidelines they have developed. One FDA guidance tried to distinguish between study and trial. Don't take definitions too strictly, think for yourself within the context of what you are reading. For this paper, the following meanings apply.

Observational study: the protocol does not require a specific treatment plan. Used for many purposes, in this whitepaper we focus on their use for reimbursement application.

Investigational study: the protocol does require a specific treatment plan, intervention, or exposure.

Retrospective design: patients are enrolled based on selection criteria after they have received treatment, intervention, or exposure (say, to a contaminated water supply), and then endpoints and outcomes data are collected.

Three Common Errors and Why You Need a Registry Consultant

Device manufacturers make three common errors when they design a registry study: 1) the study is too small, or more accurately, they forget to have a statistician even look at the size and power of the study, 2) the study isn't comparative, a tough one because the standard of practice may be very different from your technology, and 3) the study isn't monitored. A poorly-written protocol obfuscates your thinking, distracts the reader, and frustrates good results.

CDG's experts can review your protocol to help develop a clear hypothesis, a properly sized and powered design, the selection of a comparator or a thoughtful defense of why no comparator is available, suggestions for an affordable monitoring plan, or just an ear so you can double-check your logic. Please phone or email us at 773-489-5721 or [email protected]. Dr. Stark will be happy to discuss a proposal.

Prospective design: patients are enrolled based on selection criteria before they receive treatment, intervention, or exposure (say, to an unscheduled MRI), and then endpoints and outcomes data are collected.

Prospective longitudinal design: endpoints and outcomes data are collected over time (days, weeks, months, years) after treatment, intervention, or exposure.

Prospective cross-sectional design: endpoints and outcomes data are collected in the same hospital stay (i.e. at the same time) as treatment, intervention, or exposure.

Registry: a 'registry' is a list or dataset of records about something, a 'patient registry' is a dataset of records about patients.

Registry study: usually an observational study on a post-approval device, also known as a post-market study. FDA has no role or reason to set forth guidance documents, expect in the rare cases when they have legislative authority under Section 522, because the studies are conducted on 510(k) cleared or IDE approved devices. The registry can be as small or large as your pocketbook will allow, it can be single-armed or comparative depending on how you contrast your technology the standard of practice, it can be monitored or not monitored depending on how much you care about the quality of the data you collect.

Setting up a registry study
Implementing a registry study is just like implementing a clinical trial. All the basic elements of planning, design, and project management are present. There are no international standards for registry studies and FDA cannot provide guidance documents because they don't have regulatory authority over post-market studies (except for Section 522 studies.) Your primary resource for information will be "Registries for Evaluating Patient Outcomes: A User's Guide, Second Edition" from the Agency for Healthcare Research and Quality (AHRQ). Don't discount or ignore this user's guide, AHRQ plays a significant referential role in CMS National Coverage Decisions. 5. Registries

[1] Planning
In the planning phase, you decide what you need to know: identify the hypotheses (mine is better than yours, mine is safer than yours, mine is cheaper than yours), the endpoints or outcomes that will support or refute the hypotheses, and the inclusion and exclusion criteria for who will be in the study. Next you determine if safety monitoring boards, IRBs, or other committees are necessary, and finally plan an exit strategy so you'll know when the study is completed.

A hypothesis is a testable statement about the safety, efficacy, or cost of device use and it sets the design for the rest of the study. Most people falter at this first step. For example, I recently reviewed a protocol that said: "We hypothesize that our device will provide a durable improvement for women with XYZ disease." This hypothesis is a plan, but it is not a testable statement from which treatment success or failure will be clear. How will we know if an improvement was durable, or if there was an improvement at all?

Train Your Staff for Registry Studies per ISO 13485

'Personnel performing work affecting product quality shall be competent on the basis of appropriate education, training, skills, and experience,' states ISO 13485. But training is expensive and you want to get the most bits of information for your dollar that you can. Our CDs are a 'bitful' information buy. You get comprehensive, well-researched information, presented by an internationally respected expert, in a reusable format that you can share with your colleagues and add to your library to train future employees.

CDG offers a five-hour workshop on designing and implementing registry studies for medical devices. It is filled with examples of real-life medical device registries. Designed and recorded by Dr. Nancy J Stark, the workshop is a focused presentation of the AHRQ User Guide, adapted to medical devices. You can find more information on our website. Scroll down to Registry Studies for Medical Devices.

[2] Design
In the design phase, the details of the registry study are worked out and a protocol is written. There are only a few options for study design:

a) Cohort designs follow over time a group of people who possess a common characteristic to see if they develop a particular endpoint or outcome. For example, you might follow a group of women with urinary incontinence, including in your study those women who receive mechanical implants such as collagen and those women who receive implantable electrical stimulators. The hypothesis might be that one treatment is better, safer, or less costly than the other. The outcomes data might measure the number of incontinence episodes per month, the adverse events by month, or the cost of the procedure and lost work time (quality of life) by month. Notice the similarity to a randomized clinical trial? 5. Registries, p38.

b) In case-control designs you gather 'cases' of patients who have a particular outcome or who have had a particular adverse event and 'controls' who have not, and then you look backwards to see what proportion had an exposure or characteristic of interest. For example, in the evaluation of re-stenosis after coronary angioplasty in patients with end-stage renal disease, investigators found both cases and controls from an existing PTCA registry. Alternatively, cases could come from the PTCA registry and controls from outside the registry (say, Medicare data). 5. Registries, p38 and p46.

In another example, in 2004 Cordis began a registry designed to assess stenting outcomes in relation to the outcomes of their SAPPHIRE trial, which was used as the historic comparison group. The research question was to see if non-academic physicians would achieve the same level of success as the academic investigators used in the clinical study (the hypothesis was that they would.) The registry was conducted because of concerns by FDA and the Centers for Medicare and Medicaid Services (CMS) that the device worked only safely and effectively in the hands of experienced clinicians. The study involved 74 sites and 1493 patients. The large number of sites and subjects are characteristic of registry studies. 5. Registries, p38.

[3] Selecting subjects and comparison groups
The target population consists of all the patients with a common disease or condition or a common exposure. For example, the target population might be all people with cataracts, all women with urinary incontinence, or all people who have been exposed to radiation for cancer treatment. Then broad inclusion/exclusion criteria are used to select a representative population of patients. You want to keep the inclusion and exclusion as broad as possible so that the final data will be applicable to the general population.

Selecting comparison groups may be trickier in observational studies than in clinical trials because subjects have a choice as to which intervention they receive. (In theory the patient has a choice between all possible treatments, but not all clinicians offer all treatment options a patient must "doctor-shop" if they really want a particular treatment.) Treatment bias is the notion that, given a choice between your new technology and an existing technology, the sickest patients will choose you, while less-ill patients may choose the comparator. The result will be an unfair imbalance in adverse events for the new technology. Key demographic factors—such as age, lifestyle, and disease advancement—are collected and statistically applied to help correct for treatment bias.

Comparison groups may be "internal" (data collected simultaneously), "external" (data collected outside of the registry, such as a previous clinical trial, Medicare data, or billing data provided by patients), or "historical" (data collected under the registry protocol but not simultaneously). Comparison groups are essential when you want to distinguish between alternative procedures, assess the magnitude of differences, or determine the strength of associations between groups.

[4] What data should be collected?
You collect the same kind of data that you do for randomized, controlled trials. For example, you need data from patient demographics, medical history, health status, and patient identifiers (the 'personal domain'), the patient's experience with the technology or device (the 'exposure domain'), and the primary endpoints, secondary endpoints, adverse events, and technology deficiencies (the 'outcomes domain'.) In addition, you should collect information about potential confounders (say a drug being taken to treat the same condition as the study device.) And of course, the collected data should relate directly to the hypothesis.

'Data elements' refers to the exact data that will be collected. Sometimes there are broadly accepted sets of standard data elements for a disease or condition. Look to the specialty societies to see if they have created clinical data standards that you can use as a guide for selecting data for collection. For example, the American College of Cardiology has created clinical data standards for acute coronary syndromes, heart failure, and atrial fibrillation. 5. Registries, p53. Whenever possible, tie your data elements to established terminology, such as Current Procedural Terminology (CPT) codes, International Classification of Disease (ICD-10), or events related to device deficiencies. 6. ISO 19218-1.

[5] Data Sources for Registries
Contrary to popular belief, registry data is not de-identified. Depending on the data sources, registries may use certain personal identifiers to locate specific patients and link the data to other sources. For example, Social Security numbers (SSN) can be used to identify individuals in the National Death Index (NDI). What peaks my interest is that data may come from many different sources: outpatient clinic records, inpatient hospital records, laboratory records, billing records, and even payer claims data! Data may come from medical chart abstraction, electronic medical records, institutional or organizational databases, administrative databases, death and birth records, census databases, or other registry databases. For example, if you are developing a thermoembolization technology for treating liver cancer, you may want to access data from the Registry of Liver Diseases.

[6] Ethics, Data Ownership, and Privacy
The principles of ethics, data ownership and privacy are the same for registry studies as they are for clinical trials. You need IRB approval to conduct the study, HIPAA waiver to access patient medical records, a financial agreement with the institution regarding payments, data ownership and publication rights, and assurances of patient privacy.

Consider the case study of the National Oncologic PET Registry, a registry developed to collect data about PET scans in cancer management with the goal of obtaining expanded CMS coverage for PET scans. The registry was to be conducted at hundreds of hospitals and free-standing PET facilities. The sponsor's believed the registry was not subject to IRB approval because it was being "conducted by or subject to the approval of Department or Agency heads" for the purpose of evaluating a "public benefits or services program." CMS agreed. One week before starting operation the Office of Human Research Protections (OHRP) issued a letter of disagreement. The study was put on hold while the sponsors contemplated the difficulty of obtaining approval from hundreds of IRBs. Ultimately OHRP conceded that the registry only needed to be approved by one IRB. 5. Registries, p84.

[7] Recruitment
Recruitment of sites becomes a major issue in studies the breadth of registries. Sites should be paid fair-market value for their time and must see a benefit to their operations if they are to join and actively participate in a registry. This is especially true if the registry study is to include community physicians or high-volume specialty centers, as well as academic centers. Community physicians are more likely to participate if the registry is viewed as a scientific endeavor, is endorsed by leading organizations, led by a respected opinion-leader, provides useful self-assessment data to the physician, or helps meet other physician needs such as maintenance of certification, credentialing, or pay-for-performance programs.

Patient recruitment presents the same challenges as clinical studies. The best success comes from recruitment by the patient's own physician. It also helps to communicate that registry participation may help improve care for future patients, to provide written materials in language easily understood by the lay public, keep survey forms short and simple, and provide incentives such as newsletters, reports, and modest monetary compensation.

[8] Data collection and quality assurance
Three sets of documents, together, form the system for data collection. The first is the case report forms, be they paper or electronic. These are the forms whereby data is gathered in the field, entered into coded database fields, and transmitted to a data management center. The second is a data dictionary which contains a detailed description of each variable used in the registry. For example, the question may be: "Do you smoke?" And smoking may be defined as having smoked tobacco within the last year. The third is the set of data validation rules. These are logical checks on data entered to look for inconsistencies such as males taking birth control pills.

A data management manual should be developed to define how missing data will be handled, how invalid entries will be handled, how data will be cleaned, and what level of error will be accepted. The manual should describe how data will be tracked and coded, how query reports will be generated and resolved, and how it will be stored and secured. Finally, the data management manual should describe a quality assurance system for data entry and registry procedures.

[9] Monitoring
Don't confuse observational with not-monitored. It is a common mistake to think that registry studies don't need to be monitored. You should develop a monitoring plan for a registry study the same way you would for a clinical study. At the very least you want to verify that: 1) the subject exists, 2) has the disease or condition under study, 3) met inclusion and exclusion criteria, 4) signed an informed consent and HIPAA authorization, 5) received or declined treatment. Beyond that you may develop an on-site monitoring scheme or a risk-based monitoring scheme based on triggers from electronic data. Most importantly, don't just ignore the issue, develop a monitoring plan that you can defend.

[10] Adverse event reporting
For device and device procedure registries, adverse event detection, collection, and reporting is the same as adverse event reporting for any other post-approval setting. It begins with the "becoming aware" principle; i.e. the clock for reporting adverse events starts at the moment the investigator becomes aware of symptoms or events reported by the patient or signs such as out-of-range laboratory results reported by a lab, or from the moment the manufacturer learns of an event from an investigator or subject.

Investigators (i.e. users) must report serious injuries to manufacturers and to FDA within 10 days. Investigators are responsible to report deaths to both the manufacturer and FDA as soon as possible but within 10 days. 7. 21 CFR Part 803. Interestingly, if an adverse event occurs with a comparator device the investigator must report the event to the comparator's manufacturer. Manufacturers have 30 days to report deaths, serious injuries and malfunctions to FDA, and 5 days to report events that require remedial action to prevent an unreasonable risk of substantial harm to the public health. Events are logged into the Manufacturer and User Facility Device Experience Database (MAUDE).

[11] Analysis n Interpretation
Statistical analysis of registry data is no different than statistical analysis of clinical data. There are a couple of points that deserve mentioning, though. First, you'll need to determine how closely the actual study population represents the target population. Second, there should exist a statistical analysis plan for how the data are to be analyzed and interpreted with regard to accepting or refuting the hypothesis (i.e. with regard to device success or failure.) And third, there should exist a plan for how to handle missing data.

Conclusion
Don't be misled, registry studies are not cheap, second-rate clinical trials. They should be designed with the same respect as the exalted RCT. What they are is different. They are observational studies that assess an approved technology's ability to achieve its intended use in the real world and are usually conducted for the purpose of obtaining a robust Medicare National Coverage Decision.

References
1. Extracorporeal photopheresis, CMS National Coverage Determination. [http://www.cms.gov/medicare-coverage-database/details/ncd-details.aspx?NCDId=113&ncdver=2&bc=AAAAQAAAAAAA&]
2. NPWT, CMS National Coverage Determination on Negative Pressure Wound Therapy. [http://www.cms.gov/medicare-coverage-database/license/cpt-license.aspx?from=~/overview-and-quick-search.aspx&npage=/medicare-coverage-database/details/lcd-details.aspx&LCDId=27025&ContrId=138&ver=12&ContrVer=1&Date=&DocID=L27025&bc=iAAAAAgAAAAA&%3f]
3. Ingestible pH and Pressure Capsule, CPT code. [https://www.bcbsmt.com/medicalpolicies/Policies/Ingestible%20pH%20and%20Pressure%20Capsule%20to%20Evaluate% 20Gastroparesis.aspx]
4. CMS Coverage Pages, email updates, http://www.cms.gov/.
5. Registries for Evaluating Patient Outcomes: A User's Guide, Second Edition. Agency for Healthcare Research and Quality, 2010.
6. ISO/DTS 19218-1 Medical devices—Hierarchical coding for adverse events—Event type codes (2010).
7. 21 CFR 803.20(b)(1) and 803.30(a)(1)Medical Device Reporting.

 

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Choosing A Device CRO-Consultancy

A Whitepaper by Dr. Nancy J Stark

©2012 Clinical Device Group Inc, all rights reserved.

Device contract research organizations (CROs) play a different role in the product development cycle than drug CROs, and it is unrealistic to expect a drug CRO to deliver the service a medical device sponsor needs. In the drug world a CRO is used more like a Kinko's or a dry cleaners: the sponsor drops off the protocol on Monday and picks up the report on Friday. The CRO acts independently and is legally responsible for the sponsor obligations identified in the contract. [1. 21 CFR Part 312.52]

In the device world there is no possibility to legally transfer sponsor obligations to a CRO. The manufacturer may use the services of a CRO, but legal responsibility for sponsor obligations remain with the manufacturer. As a result, medical device CRO must serve as a consultancy; collaborating intimately with the manufacturer on sponsor responsibilities such as recording and reporting adverse device effects, investigator noncompliances, device deficiencies, investigator and study staff training to proper device use, and so forth. The device CRO-consultancy cannot act independently because the legal responsibility of their actions remain with the manufacturer.

Choosing a device CRO-consultancy
As the manufacturer you should choose a CRO-consultancy that complements your corporate personality, fills the knowledge gaps in your organization, and makes themselves readily available to you. Here are some specific characteristics that you can evaluate.

Device experience of top management
Does the CEO have medical device experience? The career history and experience of top management sets the focus of any company. When you are interviewing a CRO, be sure to ask for the CVs of top management. Have they worked for medical device firms? Have they worked in clinical research or regulatory roles? Are they sensitive to the differences in regulations between devices and pharmaceuticals? Without device experience at the top, you are unlikely to get device experience at the level of implementation.

CDG Inc—A Device CRO Here to Stay

CDG is a small, networked CRO offering services in biological safety, quality design, clinical research, statistics, audits, reimbursement, and clinical evaluations for medical devices. We have many clients who use us as a consultancy—they call for advice when they want direction about how to proceed.

We have been in business since 1990, and our President has been in the device industry since before Part 812 was final! Please call us to discuss consulting needs for your firm. Click here for more information or call 773-489-5706.

Size and power
How big is the CRO in terms of number of employees or annual gross revenue? Little sponsors get swallowed-up by big CROs. If your project is small compared to the other projects the CRO is running you will be ignored when bigger clients have problems and demand attention. The disparity in size leaves you with no power to demand immediate help. Small CROs will value your business and are more likely to make themselves available to advise you.

But in fairness, if you are a large device manufacturer you may be more comfortable working with a large CRO. Large CROs can bring more sophisticated electronic data tools, financial reporting tools, or other management advantages that small CROs cannot afford.

Progressive slices of the pie—estimates and schedules
In CDG's experience, device projects tend to be progressive slices of the greater pie. For example, the first project might be to review device biological safety, if that goes well we may be asked to review a protocol, and then to calculate an array of possible sample sizes, then to implement a study, and so forth. One project may turn into five projects or twenty-five project over time.

In any case, once you settle on some specifications the CRO should be able to give you an estimate and schedule. I am using the word "estimate" to mean the number of labor-hours it will take to complete the project and the word "schedule" to mean the overall duration of the project. For example, it might require 40 man-hours to write a clinical evaluation over a duration of ten weeks.

If the estimate and schedule are within 25% of the final cost and duration, the manufacturer has done a good job of planning and the CRO has done a good job of execution. But I promise you, there is no way the project will go to completion without a change in specifications.

Who will do the work?
It is key for the device manufacturer to ask, "Who will do the work?" Some large CROs have been known to show the CVs of device-experienced personnel during the negotiation phase, but then have another—device-inexperienced—individual actually do the work. It is understandable that personnel availability may change from the time a proposal is first discussed to the time the contract is finalized, but it is wrong to assign new individuals who don't have the necessary expertise.

During project negotiations be sure to ask the CRO if they have the expertise to do the work. And then don't hesitate to object later if the assigned individual isn't as skilled, thorough, or knowledgable as you need them to be. From time-to-time every company makes personnel adjustments and it should not become a big issue.

Poll Question: How did you use a CRO last year, if at all?

How did you use a CRO last year, if at all? Respond to the poll question and help us learn about our industry. Your answers are completely confidential and will be used only for my own research. The average responses will be posted in an upcoming whitepaper. Answer me here. —Nancy

Independent experts: one-person firms
Many of the finest medical device consultancies are independent, one-person firms. They have chosen an entrepreneurial lifestyle, preferring the independence of self-employment to the security of a routine paycheck. It is reasonable to ask what experience they have in the device industry, why they chose to work for themselves, and if they can commit to the duration of your project.

I've had some interesting bosses over the years. When I decided to become a consultant in 1990, my soon-to-be-ex boss told me that "consultant" was a code word for "I'm unemployed." I took that as a challenge and knew that I would have to work at building a business persona. It was important that manufacturers perceived Clinical Device Group Inc as a company that was here to stay.

When out-sourcing to a new independent expert, look for signs of someone trying to create a business: do they have a business name, business cards, a marketing effort, a website, a brochure, conference booths, speaking engagements, and the like? Look for people who have actually worked in the device industry and understand how devices are conceived, designed, tested, investigated, and commercialized. Independent experts run businesses themselves and will bring a sensitivity to the business needs of your organization.

Networks
Many independent experts form relationships with other independents and work together to provide services to clients beyond their own specific expertise. Ask what services they can provide to you directly via employee or sub-contractor, and whom they can refer to you. They can often provide you with a full range of clinical and regulatory services just like any other full-service CRO through their strategic alliances.

Communications
If the experts are in a network, who is the project leader on their side? How will communications flow, knowing that the experts may live in different states? Will you have direct access to the project implementers? Think of the management model used in most organizations; the doer in clinical research is free to send an email to the doer in regulatory affairs, but the managers of both functions are copied to keep them in the loop.

Who will bill whom? I follow a general contractor model: my sub-contractors bill CDG, CDG in turn bills the manufacturer.

Do you need (or want) separate non-disclosure agreements (NDAs) from each sub-contractor? Occasionally manufacturers want NDAs from each sub-contractor, but more commonly the primary NDA requires CDG to have NDAs with each sub-contractor so that the requirement for confidentiality flows through.

Contribution to the industry
I have had an independent expert walk away from a contract in the middle of a project in order to take a job. I respect everyone's need for security, but it is dishonest for experts to present themselves as consultants who are here to stay, and then betray that promise.

Businesses who are committed to their industries show it by finding affordable ways to demonstrate that commitment. CROs or consultant may offer public presentations, contribute written articles, participate in standards development, or find other ways to give back to the industry and to mentor younger persons. Looking at the CRO's commitment to the industry is one way to judge their staying power.

Standing and reputation
"Standing" refers to the reputation of the CRO or consultant in the eyes of clients and peers. For older CROs or independents standing may be self-evident; for younger CROs or independents you may want to ask for references.

I have had the unusual situation of a client asking me if I agreed with the opinion of another consultant. I did not, and cited the regulations on which I based my opinion. For the manufacturer it was a question of whom to believe. It is useful if you have confidence in the advice the CRO is giving you.

Project Management for Clinical Studies
Stark NJ,Project Management for Clinical Studies, Clinical Device Group Inc, Chicago, IL (2003).




Probably the most overlooked aspect of device investigations, project management is absolutely essential if a trial is to be implemented on time and with care. The book follows the usual outline of any project: conception, planning, implementation, reporting, and close-out; but focusing on clinical research. It gives sample planning forms, Gantt charts, budgets, methods for cost analysis, reporting metrics, and sample CRO contracts. If you are in management you should insist your staff read this book, it is my personal favorite of all the books I've written.

290 pages, perfect bound, illustrated, ISBN 1-889160-09-1. Table of Contents. $250 USD.

Bidding Process
A bidding process is a process where the submits a bid for a job or project. The process requires strict specifications for the project (i.e. the sponsor must know exactly what they want) and works well for "cookie-cutter" investigations. The idea is that bids are obtained from several potential suppliers and the "best" bid is accepted, where "best" is some combination of people, resources, and cost.

It's a personal choice on the part of CROs, but I rarely participate in bidding situations. I have put many hours of original research and considerable proprietary business practice in a bid, only to have the project go to another party and my proprietary information used as a learning tool for the cheaper firm. In one case the sponsor decided not to out-source at all, but to do the clinical trial in-house based on what they learned from the bids they solicited.

If you use a bidding process, it's important to play fair and ethically. Sponsors and manufacturers get reputations among CROs, too; and you don't want to risk being in a situation where no-one of quality will contract with you.

Capabilities meeting
Larger CROs will often travel to the manufacturer's site and make a capabilities presentation. Smaller CROs will often host a capabilities meeting, but the cost of travel is the burden of the manufacturer. Most frequently the meeting takes place by teleconference. No matter how it is arranged, the CRO should be able to make a presentation reviewing their company history, business model, service capabilities, technology capabilities, and resources.

If I travel to a conference I may learn, say, four new things. If it cost me $2000 in travel expenses to attend, then a new piece of information costs me $500. As a CRO, I have an unspoken rule of thumb. I will give away two new pieces of information to a potential client to establish credibility, but no more.

As the manufacturer, you are engaging the CRO in conversation so as to assess their knowledge, confidence, organization, and corporate personality. You should expect to learn a couple of new things during the meeting.

Website, brochure, company information, references
There are many ways for a CRO to present their capabilities with professional, take-home, printed materials. Whether it is a simple file folder, a four-color glossy, or an electronic brochure, you are looking for a material that describes the CRO's organization, presents its vision and mission, describes it scope of services, and tells you who they are.

References
[1] 21 CFR Part 812.32 Transfer of obligations to a contract research organization.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

A Strategic Approach to Protocol Design

A Whitepaper by Dr. Nancy J Stark, made available by Clinical Device Group

A former boss once told me this story. When I walk into his office he envisions me with a monkey on my back. As we begin our conversation the monkey jumps off my back and starts to create havoc, knocking papers to the floor, turning over the lamp, swinging from the chandelier, and kicking the waste can. My goal is to leave the office with the monkey on my boss's back; my boss's goal is for me to leave his office with the monkey on my back. When the conversation ends, the person who has something to do has the monkey on their back.

The same analogy can help you to write a protocol. The monkey is consensus on several key points about the study design. If you get consensus from the project team and top management, the monkey is on their back. If you have to start writing the protocol without agreement on these points, the monkey is on your back. With committed consensus, you can plan on 20-30 working days over a duration of 2-3 months to write a new protocol for a new technology. Without consensus it can take years.

Authority
To lay some background, let's look at the regulatory requirements for a clinical investigation. FDA regulations tell us that we shall have valid scientific evidence to support the safety and efficacy of medical devices. Valid scientific evidence [consists of] evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsible be concluded by qualified experts that there is a reasonable assurance of safety and effectiveness of a device under its conditions of use. So the purpose of a clinical investigation in the States is to provide reasonable assurance of safety and effectiveness for the investigational device. [1. 21 CFR Part 860.7(c)(1-2) (paraphrased)]

[08] Protocol & Trial Design Workshop on CD (March 2012)

A newly updated version of the Protocol and Trial Design Workshop is available on CD. The 4.5 hour presentation, consistent with both US and EU requirements, gives the backstory to each section of the protocol: why is it there, what are regulators looking for, is it important, can you omit it? You also receive a "fill-in-the-blank" protocol template with detailed instructions so you can focus on the creative work and leave the boilerplate in place. The well-researched course is designed and studio-recorded by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

For those of you concerned with clinical trials in Europe the path of authority is a little more complex. You'll begin with the Medical Device Directive which states the demonstration of conformity with the essential requirements must include a clinical evaluation in accordance with Annex X. [2. MDD 93-42-EEC I.6a)] A clinical evaluation is the assessment and analysis of [existing] clinical data pertaining to the medical device in order to verify the clinical safety and performance of the device. [3. MEDDEV 2.7.1 r3] If existing clinical data cannot verify safety and performance, then a clinical investigation must be done per ISO 14155 or the intended use reworded. The standard states that the justification for the design of the clinical investigation shall be based on the evaluation of the results of a clinical evaluation. [4. ISO 14155 Clause 5.3] This is the logic behind bringing a medical device to market in the EU via the literature route.

Start at the end
To leave the monkey on the project team's back, here are the key points on which you will want consensus.

Intended use
Intended use is a US regulatory term that refers to what you say in the labeling. It ties the physical device, and the words we use to describe it, into a single entity. [5. 21 CFR 801.4] Intended use is a broad concept—think verbs, function, and action. Consider the following examples:
[1] A neurostimulator is intended for stimulating the vagus nerve.
[2] An MRI scanner is intended for soft-tissue imaging using nuclear magnetic resonance.
[3] A fetal oxygen monitor is intended for monitoring oxygen in a fetus.

The intended use dictates the specialty we need in an investigator, say orthopedist or neurologist, and begins to define the subject population.

Indications for use
Indications for use are the diseases or conditions for which the device will be used. Choose diseases, conditions, or procedures that have accepted medical definitions. The indications for use dictate the subject selection criteria. [6. 21 CFR 814.20(b)(3)(i)] Consider the following examples:
    [1] A neurostimulator indicated for prevention of epileptic seizures.
    [2] An MRI scanner indicated for detecting structural abnormalities.
    [3] A fetal oxygen monitor (hopefully) indicated for determining if a C-section is warranted.

Consulting Services

CDG's experts are frequently engaged on a retainer basis to be available for phone calls, email exchanges, document review, good advice, or other consultations. We have experts in most medical device pre-approval issues, including market assessment, biocompatibility, regulatory, clinical evaluations, clinical investigations, statistics and sample size calculations, and audits. To discuss a Service Agreement, call or email us here or ask for Nancy at 773-489-5706.

Performance and device deficiencies
Performance speaks to the physical properties and operation of the device; for example, what range of light-waves does it emit, how hot does it get, do the buttons stick, how easy is it to use? Performance also speaks to the ISO concept of device deficiencies such as inadequacies with respect to the devices identity, quality, durability, reliability, or safety. The intended performance or concerns about lack of performance dictate several of the questions that you should include on the case report forms. [7. ISO 14155] Consider the following examples:
    [1] The neurostimulator delivers xyz amps.
    [2] An MRI scanner has good contrast resolution.
    [3] A fetal oxygen monitor detects oxygen levels in fetal blood.

Safety
Safety refers to the absence of adverse events to subjects and the absence of serious, unanticipated adverse events to caregivers or bystanders. [8. ISO 14155] Recording and reporting of adverse events varies with the host country (the country in which the study is being implemented) and with the study risk. Every protocol requires a "risk assessment". The risk assessment is often written in tabular format with headings of hazard, frequency, severity, risk, mitigation, and residual risk.

A hazard is an adverse event that may occur to a subject, or a serious, unanticipated adverse event that may occur to a caregiver or bystander. Frequency refers to the rate at which the hazard is likely to occur; it can be estimated on any convenient scale such as 1-100%. Severity refers to the intensity of the event, such as mild, moderate, or severe; a three-point scale is useful for estimation. Risk is a function of frequency and severity; you can use any meaningful mathematical relationship, in the example the two scores have been added. Mitigation refers to actions that are taken to lessen or prevent the risk; and because you are writing a protocol you’ll focus on measures that can be taken by the investigator. Residual risk is the amount of risk left over after all mitigation measures are taken. Residual risk can be described as minor, requiring an outpatient procedure, or hospitalization; or by any other meaningful scale. [9. ISO 14155] Clearly the safety questions on the case report forms will come from the risk assessment. In the US, the risk assessment may cause the IRB to see the study as significant risk. Consider the following examples for residual risk:
    [1] Neurostimulalton of the vagus nerve may cause headaches.
    [2] MRI scanner may expose technical staff to NMR.
    [3] Placement of fetal oxygen monitor sensors may injure fetal soft tissue.

Target market
Who is going to influence the buying decision about your device within the hospital or clinic structure? Often this person is a good choice for clinical investigator.

Major competitive products or procedures
Controlled, randomized trials are not usually required for medical devices in order to gain clearance in the US, and a comparison to competitors does not enter into the equation in the EU. Nevertheless, it is useful to know what your competitors are claiming, how they word the intended use and indications for use, what clinical trials they have done, what predicate devices they referenced, and the like. If you do decide to do a randomized, controlled study, a thorough knowledge of your competition will help you decide on what product or procedure to use as a control.

Claims
A claim is an assertion of truth about your product. The claims you want to make dictate the endpoints (clinical measurements that indicate device success or failure), outcome measurements, whether a study is comparative or single armed, and many other features of investigation. The team may list several claims they want to make about the device; limit your selection to two or three. If you're a new company go after the easy claims first to get experience in conducting studies, getting FDA clearance, and commercializing a device. If your firm is experienced you may feel comfortable going after more complex, but desirable, claims in the first investigation.

Frequently the project team will settle on the claims to be investigated. And then you'll begin writing the protocol only to find there is no good methodology for measuring success or failure, the sample size makes the study unaffordable, there are too few patients to maintain a reasonable recruiting rate, there is no market for treating such patients, or some other insurmountable problem. You go back to the project team, rework the wording of the proposed claims, and resume writing the protocol. This is the monkey creating havoc.

Clinical Trials Design
Stark NJ, Clinical Trials Design, Third Edition, Clinical Device Group Inc, Chicago, IL (2000).

This is one of the few books about writing protocols and designing trials that links them to the trial hypotheses and product claims. That simple sentence sums up the guts and substance of the book: first decide what you want to claim about your device, and then design a trial to investigate it. The concept is so important and so rarely discussed that the book has been translated into Japanese and Korean.

Still current, still relevant, and still a useful reference. It is intended for use by medical device manufacturers. Provided in a high-quality three-ring binder so you can add your own notes, illustrated, 300 pages. ISBN 1-889160-02-4. Table of Contents. $250 US.

Protocol Contents
There is no absolute format for a protocol, although FDA issued a guidance document in 1996, [10. FDA Statistical Guidance] and ISO 14155 contains an extensive checklist. Here is a suggested format that is a combination of the two:
[01] Cover page, Table of contents, Executive summary.
[02] Device description, photograph, accessories, description of comparator.
[03] Contention of SR or NSR regulatory status.
[04] Study objective.
[05] Hypotheses, primary and secondary.
[06] Endpoints and procedures, primary and secondary.
[07] Medical risk analysis.
[08] Study design.
[09] Subjects.
[10] Ethical considerations.
[11] Sample Size.
[12] Data collection.
[13] Control of study devices.
[14] Adverse event recording and reporting.
[15] Monitoring plan.
[16] Documentation.
[17] Central laboratories.
[18] Data management.
[19] Data analysis.
[20] Data monitoring board, Data safety monitoring board, Clinical events committee, if one will be used.
[21] Personnel Responsibilities.
[23] Return of unused inventory.
[24] Study registration and publication policy.
[25] Study cancellation.
[26] Definitions.
[27] References.
[28] Attachments.

Internal Approvals
Internal approvals are another great opportunity for the monkey to create havoc. But you can keep him in his cage by using a approval form that identifies and limits each function's responsibilities. For example, the project team leader reviews the protocol to ensure you have described the device accurately and you are investigating the right device; marketing ensures you are investigating the right claims, manufacturing reviews to ensure they can actually make the investigational device, the medical manager assures you have an accurate and complete risk analysis, regulatory assures you have identified the regulatory status of the investigational device correctly, statistics reviews the sample size and data analysis plan, and so forth. Only your manager has the privilege of global review.

References
[1] 21 CFR Part 860.7(c)(1-2) (paraphrased).
[2] MDD 93-42-EEC I.6a).
[3] MEDDEV 2.7.1 r3.
[4] ISO 14155 Clause 5.3.
[5] 21 CFR 801.4.
[6] 21 CFR 814.20(b)(3)(i)
[7] ISO 14155 Clause 3.15.
[8] ISO 14155 Clause 3.2.
[9] ISO 14971 "Medical devices—Application of risk management to medical devices" (2007).
[10] FDA's Statistical Guidance for Clinical Trials of Non-Diagnostic Medical Devices (1996).

Questions or Comments?
Please post your questions and comments below.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

 

 

Uptick in Clinical Evaluations

What Do Notified Bodies Want?

A Clinical Device Whitepaper by Nancy J Stark

We've seen an uptick in requests for clinical evaluations recently, and I thought it would be worthwhile to review the requirements. Where you begin depends on where you are now: 1) writing a clinical evaluation for a new, pre-approval device, or 2) writing a clinical evaluation for a device that is already commercial. In the first case, you are asking the question, "Do I need new clinical data for certification?" In the second case, you are bringing your documentation up-to-date.

Clinical evaluation reports for pre-certification devices
If you're still in the stage of designing your device, you begin by identifying its intended uses. Then you evaluate the existing clinical data to determine if it is sufficient to support safety and performance per the Essential Requirements of the MDD [1 MDD] or AIMD [2 AIMD], or if you need additional clinical data. If additional clinical data are needed you have the choice of revising the intended use or performing a clinical investigation.

Once you believe the data are sufficient, you convince top management to grant a CE Mark and sign the Declaration of Conformity to your previously certified quality management system. Finally, you will discover the truth when the Notified Body comes to make an inspection call.

Clinical evaluation reports for commercial devices
The more common situation is that your device is already on the market in Europe, and the Notified Body has informed you that its clinical evaluation is overdue or needs updating. They may give you a grace period, but you are expected to get busy. You follow the same process of evaluating the existing data to determine if it is sufficient to support safety and performance for the intended uses. Should the report conclude that the existing data are insufficient you are in a difficult situation: you'll need to suspend sales until additional data are acquired.

Because the consequences of determining the data are not sufficient are so financially significant for your firm, and the consequences of determining the data are satisfactory are so financially favorable, it is wise if the report writers are as far removed from the device designers as possible.

Resources
To write the report you need a team of at least three individuals: 1) a medical writer, 2) an information specialist, and 3) a statistician.

In addition you need a written procedure for how to proceed, a report template (useful, but not required), and access to literature databases such as Medline. Medline is easy to use, it is operated by the US National Library of Medicine and is available to anyone with a browser. Medline provides access to 5400 worldwide journals in 39 languages dating from 1947 to the present. And it is free.

Date Wheels for Clinical Evaluation Planning

A typical clinical evaluation involves the following steps and durations:
[1] Identify key questions—1 weeks.
[2] Identify databases to search—overlaps.
[3] Identify search scope, keywords, search strings—1 week.
[4] Review abstracts, discard with reasons—1 week.
[5] Acquire articles—2 weeks.
[6] Read articles—1 week.
[7] Weight articles for relevance, discard with reasons—overlaps.
[8] Weight articles for statistical validity—1 week.
[9] Medical writer summarizes literature—2 weeks.
[10] Review sponsored clinical trials, if any—0-2 weeks.
[11] Review risk management system & complaint file—2 weeks.
[12] Write final evaluation—1 week.

Date Wheels are a handy, pocket-sized calendar used by project managers to quickly plan simple schedules. If the clinical evaluation work begins on March 1, you can hope for a final report about three or so months later, or about June 1. Buy them here.

A Three-Legged Stool
First leg
The first leg of a Clinical Evaluation Report is an evaluation of the existing literature data. The Global Harmonization Task Force (GHTF) Study Group 5 document "Clinical Evaluation" coupled with MEDDEV 2.7.1 Rev 3, which is slightly easier to read, are the guidances that tell us how to do the literature evaluation.

Here's what the medical writer—working together with you—will do:
[1] Define the key questions.
[2] Identify the databases to search (Medline, Embase, MAUDE, Cochrane).
[3] Define scope of search and search strategies using a qualified information specialist.
[4] Scan the abstracts to identify articles for review.
[5] Acquire full-texts of articles.
[6] Weight the articles for technical significance, discard anything with a low weight.
[7] Have a statistician weight the articles for statistical significance, discard anything with a low weight.
[8] Review only those articles with the highest weightings; see averages below. The literature data are evaluated to determine if they support safety and performance of the device for its intended use. I like to prepare a separate literature review and attach it to the main clinical evaluation as an annex. The literature review annex has several required elements including the search strings used by the information specialist, a list of abstracts scanned, the articles reviewed, their weights and justification for those weights, and full-text copies of the articles.

Second Leg
The second leg of the evaluation is a review any existing clinical investigations that your firm may have sponsored. These data are reviewed separately from the literature because you have closer access to the details. The writer asks questions such as: "Was the Declaration of Helsinki followed?" or "Were adverse events resolved?" If not, those facts are noted in the report but the data are not used.

Once the writer is satisfied that ethical and administrative requirements were correct, he/she reviews the study results to see if they support safety and performance of the device for its intended use. I like to prepare a separate sponsored study review and attach it to the main evaluation as an annex.

Third leg
In the third leg, the medical writer comes to the most sensitive step of all; examining the complaint file against the risk management system. They review the file to see if complaint handling, as dictated by the risk management system, supports the safety and performance of the device for its intended use. I like to prepare a separate risk management review and attach it to the main evaluation as an annex.

The Clinical Evaluation
Finally, the medical writer takes the literature review, sponsored study review, and risk management review altogether and reaches an overall conclusion of whether the existing data support the device's safety and performance for the intended use, or if additional clinical data are needed newly. You know what happens if new clinical data are needed.

The Technical File
If the data are sufficient, the clinical evaluation is filed away in the Technical File, a written procedure describing the writer's strategy is incorporated into your quality management system, and you wait with baited breath until the Notified Body inspector comes.

Recently CDG conducted a poll on the average number of
articles used in a clinical evaluation. Here are the results.
Please add your two cents worth below.

How long, how much
There are only two questions in top management's mind: 1) how long will it take, and 2) how much will it cost. The answers are simple: "It depends." Let's say you have a very nice ultrasonic toothbrush and you intend to use it for cleaning roadside burns—the kind of burn you get when your motorcycle crashes and your skin scrapes against the pavement. The toothbrush is used to remove the dirt, grit, and tarmac from the wound. It's an unusual intended use and raises all kinds of questions about potential damage from the bristles, safety features to prevent over-brushing, effects of ultrasound on open wounds, comparing gum tissue to wound tissue, instructions for use, and so on. A clinical evaluation for cleaning wounds will take longer and cost more than a report for brushing teeth with the same device.

CDG can help
Clinical Device Group is poised and ready to help you with writing your evaluations. We have medical writers, information specialists, experienced statisticians, a written procedure, a standard template, and a proven process. Clinical evaluations written by our specialists have been reviewed by at least three different Notified Bodies and all have been accepted. We work collaboratively with a point-person on your side so that you are involved in the process every step of the way.

References
[1] Medical Device Directive 93/42/EEC.
[2] Active Implantable Medical Device Directive 90/385/EEC.
[3] Global Harmonization Task Force, Study Group 5, Clinical Evaluations. http://www.ghtf.org/sg5/sg5-final.html
[4] MEDDEV 2.7.1 rev 3. http://ec.europa.eu/enterprise/sectors/medical-devices/files/meddev/2_7_1rev_3_en.pdf

 
Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

 

 

 

 

 

Relationship Building in Clinical Trials

A Whitepaper by Dr. Nancy J Stark

Relationship building is an integral part of the successful implementation of any clinical trial. You need good relationships with investigators, study staff, your project team, your colleagues, your boss, and your subordinates in order to successfully implement any project. One thing is for certain: they are not going to change. If you want a good relationship you will have to adapt to them.

Myers and Briggs are great heroines and role models to me. Imagine two women psychologists during World War II. Katherine Cook Briggs and her daughter Isabel Briggs Myers had plenty of obstacles stacked against them. They had only master's degrees, they were women, they were playing in a man's world, yet they developed a personality tool that is still used seventy years later. These two did original research extending Carl Jung's theories to practical applications in the workforce.

Why did it matter? Women were entering the workforce en masse without a clue as to what they would be good at: office work (quiet and peaceful), factory work (noisy and lots of yelling), data work (numbers and logic), design work (color, texture, image, spacial relationships), or whatever.

In this whitepaper I will attempt to adapt the work of Myers and Briggs to the critical task of relationship building in clinical trials. I use layman's language because I am not a psychologist. If you are, feel free to add your comments, corrections, contributions, and rebuttals below.

Jungian Background
The theory of personality type originates from the work of Carl Jung, who proposed that people perceive the world through two different functions: 1) the 'rational' (judging) functions of thinking and feeling, and 2) the 'irrational' (perceiving) functions of sensing and intuition. Jung went on to propose that these functions are expressed in either an introverted or extraverted form. From Jung's original concepts, Myers and Briggs developed their own theory of personality types. [1. Myers-Briggs Type Indicator]

Risk-Based Monitoring for Medical Device Firms on CD

We offer training for medical device firms on risk-based monitoring on CD. The course covers the details of the FDA guidance 'A Risk-Based Approach to Monitoring', recommendations on using it for device studies, includes a sample monitoring plan and template, and features QnA addressing issues unique to device manufacturers. CDs are nice because you can stop and start them, share with your colleagues, and build a library for new hires. The well-researched course is designed and presented by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

Understanding Personality Types
Myers and Briggs saw that there are four essential steps in how people relate to the world: 1) we focus on the world of our choice, 2) we perceive information, 3) we judge the information, and 4) we present the information to the outer world with a characteristic attitude. We do each of these four steps as either introverts or extroverts for a total of sixteen possibilities. For the sensors among us, I've tried to represent the concept in a simple, colorful visual image. [2. Gifts Differing]

As children we learn how to survive in the world that we know first: family and parents. And those survival skills are our default personality throughout life. As we gain experience and mature we learn to move out of our preferred personality type and become skilled at all possibilities, allowing us to act appropriately in different situations. But when under stress we tend to revert to the survival skills of our childhood. If we understand our own personality types, and become adept at discerning the personality types of others, we can adjust our behavior to build the best possible relationships with our colleagues.

[1] Focusing on the World: Introversion or Extroversion
People prefer to focus on either the inner world of thoughts and ideas (introversion) or the outer world of people and things (extroversion). Introverts do their best work in their heads in reflection; extroverts do their best work in the outside world, in action. Our focus on the world effects dramatically how we live in the world. Only an introvert would sit alone in their office and write this whitepaper. An extrovert will be impatiently waiting to get to the practical applications.

Introverts organize the facts and principles related to a situation and are good at researching material, writing protocols, or designing investigations; extroverts organize the situation itself, including any idle bystanders, to get things rolling. Extroverts are good at implementing clinical investigations.

Help and advice is a phone call away

Sometimes you want to discuss an issue, but implement the solution yourself. If you want advice, a second opinion, or someone to talk to about clinical, regulatory, biological safety, or reimbursement strategies, sign us up as consultants. Email us here or ask for Nancy at 773-489-5706.

Introverts work out their insights slowly and carefully, searching for eternal truths. Extroverts have an urge to communicate and put their inspirations into practice. Can you imagine the annoyance of an introverted study nurse when an extroverted monitor bombards her with emails to give her the latest study updates; or the annoyance of an extroverted (and eager) investigator when an introverted monitor is slow to communicate the start date?

[2] Perception, Taking in Information by Sensing or Intuition
Perception refers to how you get information from the outside world into your head. People take in information either by sensing or by intuition. Sensing means to become aware of information by our five senses: touch, smell, sight, sound, and taste. Intuition means to become aware of information through the unconscious. With intuition we incorporate our ideas or associations to what we sense with lightning speed and seemingly without logic.

Discuss a new device with a sensing person and they may tell you it is too lightweight, it is smaller than the current model, the materials aren't sturdy enough, but the display panel is clear and easy to read—all the information taken in by their senses. Sensing people are well suited for the demands of clinical trial monitoring. Practical and matter-of-fact, they see the world as it is rather than as it should be. They are often good project managers, following the project with Gantt charts, spreadsheets, drawings, flowcharts, and data. If both you and the investigator are sensors you may focus too closely on the details and lose sight of overall progress.

Hand an intuitive person the same device and they may answer, "This reminds me of my grandmother. She used this kind of device every day when I was a kid, we'd check in on her to make sure it was functioning properly and then play a game of checkers." Instead of focusing on the device, your intuitive colleague focused on her associations with it. [3 Tieger]

Intuitive people often have inspired product ideas, clever solutions to problems, or make associations that no one else would see. Intuitive people are good at picking up undercurrents, they know when a situation is 'off' even if they can't exactly tell you why. If both you and your investigator are intuitives, you may have imaginative conversations but not notice some study problems.

I had an extroverted, sensing boss who had a rule that we could present no more than three possibilities to him. If he asked you what went wrong, you had better not give him a list of five things. If he asked you for a recommendation, you had better not have more than three. His intuitive, data-loving subordinates were terrified.

“Readers who are sensors will tend to confine their attention to what is said here on the page... readers who are intuitive are likely to read between and beyond the lines to the possibilities that come to mind.”

[3] Judging, Processing Information by Thinking or Feeling
Judging refers to how you process the information you take in. Thinkers use a logical process aimed at reaching an impersonal finding. Feelers use an emotional process of viewing information in a personal, subjective manner.

The thinking protocol writer may make a list of all the data elements they think are necessary to meet the purpose of the new clinical trial. As they develop the protocol they use their list of data elements as an absolute guide. They are not swayed by pleas from marketing to include questions they have already dismissed as unessential or exclude questions because "it is knowledge we would rather not have." Thinking writers may try to collect too much data in a clinical trial—oblivious to the impracticality.

If, as a member of the project team, you want additional data elements included in the study you should listen closely to the thinking writer's rationale for omitting them, be respectful of their reasoning, and use facts and data that relate back to the trial's purpose to persuade them to make the change. Thinking people objectify situations—dealing in facts, figures and data; they do not respond well to insincere flattery. (My husband is even suspicious of sincere flattery, I only tell him he has done a good thing when I know he already thinks so himself.)

CDG knows risk-based monitoring for device studies

Our professional monitors are experienced in risk-based monitoring of device studies (because that's how device studies are often done), they know how to work with all personality types, their goal is to worry so you can sleep at night. Our Network Staff has been involved in dozens of device trials and hundreds of issues; their extensive knowledge can solve your problem de jour. Click here to request a proposal for clinical research services from CDG or phone 773-489-5706.

A feeling person may make a list of data elements but it will have little influence on the final protocol design—the decision to collect data is made because it “feels” right. The decision is made first, and a rationale is developed later. A thinking monitor who is ill-at-ease with the site's data collection should be delicate with a feeling investigator who is confident the device works, but they should not back down. They should challenge the data with facts while being thoughtfully respectful of the investigator's mode of processing information.

Another consequence of thinking and feeling personalities is how they relate to each other. Feeling people subjectify situations—anticipating how facts, figures and data will affect the people involved. A feeling monitor (such as myself) might have difficulty enforcing an important requirement, say, for dating an informed consent, because it causes too much inconvenience to the investigator. The feeling person’s prime motivation is harmonious relationships, not regulatory compliance. A sensing boss has no difficulty pointing out the monitor's deficiencies.

“A reader who first considers whether an idea is consistent and logical is using a thinking process. A reader who first considers whether an idea is pleasing or displeasing, supporting or threatening, is using a feeling process.”

[4] Attitude to the World: Perception or Judgment
Our attitude in presenting information to the world is a reflection of which personality type is more strongly developed, perception or judgment. Attitude or "face" refers to how you present information back to the outside world. For extroverts, attitude is a reflection of the personality type that is dominant. For introverts, attitude is a reflection of the personality type that is subordinate because they reserve their favored, dominant preference for the more highly prized, inner world. In other words we know exactly what extroverts think and we never know what introverts think.

People with perceptive attitudes maintain an open mind regarding situations, continuing to seek additional information that can broaden and deepen their understanding. If pressed, they will disclose the facts, but they will refrain from telling you what it means or predicting the outcome. Perceptive people make others feel safe and accepted, and are valued members of any project team because they keep an open mind. Project managers with perception attitudes may be reluctant to move the action forward. They are more likely to delay conclusions or fail to act at all. They can be positively frustrating to bosses who who are at a critical decision-point.

Persons with judging attitudes shut off their perception (at least for the moment) and make a judgment. They may not have all the facts, but they can certainly tell you what they mean. Monitors or project managers with judging attitudes will have strong opinions about the value of a project, work product, piece of data, or whitepaper; and will freely share that opinion with listeners. They are more likely to come to wrong conclusions, a positively frustrating behavior to bosses who want to be sure before acting on a recommendation. But judgers are an essential component of a well-functioning project team; they bring issues to closure and move the project to the next step.

The Queen of Hearts in Lewis Carroll’s classic novel Alice in Wonderland had a typical judgment attitude (as well as being an impatient extrovert). “Off with her head, we’ll have the execution now and the trial later!” she said.

“Readers who are still following this explanation with an open mind are using perception, readers who have decided by now that they agree or disagree with me are using judgment.”

Putting It to Practice
The most important step in building relationships is the first one: know thyself. To help you get over the embarrassment of digging so deep into your psyche, I'll tell you that I am an introverted, sensing, feeling, judger (an ISFJ.) I live in my head, which is far more fascinating to me than anything going on outside; furthermore you will never know what I really think. I take in information by my senses: are there data-gaps, are the facts consistent, do I trust the source, do I need more information? I am a feeler, use the wrong word (mismanagement versus bad economy) and your conversation with me is over. And I am a judger, I have an opinion about almost everything, a quality essential for writing whitepapers or editorials.

Happily we are not stuck on home base. With time and experience I have learned to behave like an extrovert, make a rare intuitive leap of logic, be objective in my assessment of a situation, and withhold judgment until the facts are in—at the cost of a great deal of psychological energy. The hallmark of maturity isn't balance, but being able to move freely from one extreme to the other as the situation demands.

The next step is understanding your colleagues. Changing them is not your goal or your business, and anyway it won't work. The goal is to build relationships that will facilitate the successful completion of your trial. If you're working with an introvert be respectful of their privacy; if you're working with a extrovert enjoy the excitement. If your working with a sensor be prepared with facts and details, if with an intuitive go for the big picture. If your working with a thinker be logical in your presentation, if with a feeler never, never, never say "you". If your working with a perceptive be patient while they work through information on their own, if with a judger you may want to slow them down. The best project teams have someone of every personality type.

Relationships are complicated by supervisor/subordinate, client/consultant, parent/child, wife/husband, boyfriend/girlfriend, addiction, religion, politics, culture, heritage, wealth, and so much more; there is no one simple answer to building good relationships. But understanding personality types and adjusting our own behavior to meet the needs of our colleagues in order to meet the goal, is something we can all do.

References
[1] Myers-Briggs Type Indicator, Wikipedia, the free encyclopedia.
[2] Isabel Briggs Myers with Peter B Myers, Gifts Differing, Understand Personality Type, Davies-Black Publishing, Palo Alto, CA, (1980).
[3] Teiger PD, Barron-Teiger B, The Art of SpeedReading People, Little, Brown & Company, New York (1998).


Questions or Comments?
Please post your questions, comments, contributions, and rebuttals below.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

Risk-Based Monitoring—The New FDA Guidance

An Op-Ed by Dr. Nancy J Stark

FDA's draft guidance, "An Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring" (August 2011) references the ICH-GCPs six times. The ICH-GCPs were written in 1996 by pharmaceutical sponsors and regulators from the US, Europe, and Japan. It is 15 years old and is showing a few signs of age—there is no discussion of central data management or electronic security issues, for example. But because the phrase 'good clinical practice' is in the public domain, ICH was free to adopt it as part of the title of document E6, "Good Clinical Practice: Consolidated Guidance." The Center for Drugs published the guidance in the Federal Register giving it high standing in the regulatory community.

Although used by device manufacturers, the guidance does not work well. It's description of an Investigator's Brochure is chemically-centered not mechanically-centered, investigational devices are stored and dispensed but never installed, there is no mention of training an investigator to use the investigational product, there is no mention of software-controlled devices or software collected data, no mention of caregiver or healthcare provider safety; in other words, for medical devices, the ICH-GCPs are inadequate and out-of-date.

ISO 14155 'Clinical investigation of medical devices in human subjects—Good clinical practice' (2011)—the internationally definitive document for medical device clinical studies—is not mentioned even once; yet the international standard was closely harmonized with ICH-GCP to promote international continuity. I could live with the lack of acknowledgement in FDA's guidance if the same FDA Office hadn't taken such a heavy hand in writing the ISO standard. It is wrong to insist on practice-changing language (such as moment of consent versus moment of enrollment) and then turn your back on the work product as FDA seems to have done.

The draft guidance draws heavily on the ever-increasing presence of electronic data collection. A really cool EDC system can spot a lot of site problems before the human eye ever gets to the data. But these systems are expensive (think in terms of a $1M for a modest study) and unaffordable to small start-ups.

Finally, it was the Offices of Compliance from CDRH, CDER, and CBER who wrote this guidance. While they promise us the inspection manual will be promptly updated to match the guidance, they do not make assurances that the reviewers in the Office of Device Evaluation will follow along. Sponsors are strongly urged to get buy-in on the monitoring plan from reviewers before initiating a clinical trial.

While device manufacturers can use the new guidance to their benefit, often to defend existing practices, they should make certain FDA's Office of Device Evaluation buys into any IDE monitoring plan.

 

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Risk-Based Monitoring—The New FDA Guidance

A Whitepaper by Dr. Nancy J Stark

In August of 2011 FDA released a draft guidance document titled "Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring." At first I was very excited because the monitoring guidelines had not been updated since the 1988 "Monitoring of Clinical Investigations." But then, as I began to consider the ramifications of implementing the new guidance for medical device investigations I started to have mixed feelings. The first part of this whitepaper is a factual review of the draft guidance with ideas about how device manufacturer's can take advantage of it. The second part (next blog) is an op-ed expressing my views about what it will mean for the device industry. [1.Risk-Based Monitoring Guidance] [2. Monitoring of Clinical Trials]

Two points stand out as noteworthy before we begin: 1) the draft guidance was written by the Office of Compliance not the Office of Evaluation, and buy-in from an IDE reviewer is uncertain, and 2) the draft guidance addresses itself to drugs with a mere mention of devices as an afterthought, so most concepts have to be 'retrofitted' to device needs.

Part One: Review of Guidance
Risk-Based Monitoring
The draft guidance is based on the premise that most sponsors conduct on-site monitoring visits every 6-8 weeks with the goal of source verifying 100% of the data 100% for 100% of the subjects. It uses this premise to argue that sponsors will save money by moving to risk-based monitoring because such a move will result in fewer monitoring visits. Risk-based monitoring probably will result in fewer on-site visits for big pharmaceutical firms, who may be 'conducting hundreds to thousands of trials in locations around the world'. [3. Quality Management in Clinical Trials] But start-up medical device companies have never monitored as heavily as the guidance assumes and, in that sense, the principles risk-based monitoring won't save on monitoring costs.

The guidance suggests that we use fancy statistical algorithms and analyze real-time data as it comes into electronic data capture systems to identify high-risk sites or high-risk data. The algorithm could be something like 'Risk Priority Number = severity x likelihood of occurrence x detectability'. A Risk Priority Number is calculated for each identified risk. When high-risk sites or data exceed a trigger, they merit immediate, onsite monitoring visits. [3. Quality Management in Clinical Trials]

CDG's Risk-Based Monitoring Course on CD—Available Now!

Training on risk-based monitoring is now available from CDG! The course covers the details of the FDA guidance 'A Risk-Based Approach to Monitoring', recommendations on using it for device studies, include a sample monitoring plan and template, and features QnA addressing issues unique to device manufacturers. The well-researched course is designed and presented by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

High risk sites and high risk data
A high-risk site is a site with a high probability of having a problem; the 'symptom' might be that they are enrolling at a slower or faster rate, experiencing a higher or lower rate of adverse events, or having a higher rate of non-compliances than other sites. For example, one site I worked with suddenly stopped enrolling subjects altogether. The sudden change in enrollment pattern should have triggered an on-site monitoring visit. We eventually learned the site had enrolled the same subjects in multiple trials and compromised everyone's data. The site stopped enrolling because they were being audited by other sponsors for fraud.

High-risk data might be a particular type of event or outcome that is a 'symptom' of a problem with the study. For example, a high rate of post-procedure embolism might indicate a poorly trained investigator or a problem with subject selection.

The guidance suggests that any site in the start-up phase of the study is a high-risk site and that on-site monitoring should be performed at every site in the study's early stages.

Device companies can adopt the same principles using low-cost methods. Have the sites fax or scan the case report forms to the monitor once a week for off-site monitoring (i.e. remote monitoring). With one trial and maybe 100 subjects, a monitor can easily examine the case report forms manually for high-risk sites or data without the need for statistical algorithms.

Critical data and processes
Critical data and critical processes are the types of data and processes most likely to be high risk. Critical data include: [1] evidence that the subject really exists, [2] evidence that subject has the disease or condition being studied, [3] evidence that IRB approval was obtained, [4] evidence that informed consent was signed before the intervention or procedure, [5] endpoint data that address the hypothesis, and [6] serious adverse device effect data. For each study you'll need to make a unique list of endpoint data and adverse device effect data that is critical.

Critical processes include [1] the process of obtaining informed consent, [2] the intervention or procedure, [3] timely reporting of regulatory non-compliances or study deviations, [4] timely correspondence with a data safety monitoring board (if one exists), and the like. Because studies vary so much from each other, you'll need to make a unique list of critical processes for each one.

Critical data and processes are most likely to be high risk and should be checked for closely during the off-site monitoring process.

CDG has monitors experienced in risk-based monitoring

Our outsourced monitors come in with the attitude of service, impermanence, and an enthusiasm for the new; they are accustomed to off-site monitoring and problem spotting. They also carry a breadth of experiences that can help solve the issues de jour. Click here to request a proposal for monitoring services from CDG or phone 773-489-5721.

Methods of monitoring
Next the guidance considers two different methods of monitoring; on-site monitoring and 'centralized monitoring', meaning the real-time analysis of data by an application running on a server in some data management center.

I propose that for device studies there are four methods of monitoring: [1] on-site monitoring, [2] off-site monitoring of faxed or scanned case report forms, [3] off-site monitoring of electronic ally captured data, and [4] off-site monitoring of source data that has been captured by the device (usually in vivo or in vitro diagnostic devices.) Often data card is changed out on-site and the data card analyzed off-site. Sometimes the data card may transmit data electronically via the internet—consider a digital camera used to take photographs of the procedure. In the future, of course, it may even be possible to remotely access electronic medical records.

The idea, then, is to use the most expensive method of monitoring (on-site) for sites or data or processes that are critical and high risk and to use less expensive methods of monitoring (off-site or remote) for sites and data that are less so. It is a welcome concept to device manufacturers who are cash-starved and looking for ways to economize.

Triggers
The idea of a trigger is that some server at a data management center somewhere is programmed to alert you if critical data or a critical process has become high-risk; i.e. has the value or frequency of a critical data element or critical process step changed in such a way as to demand attention. Indeed, if hundreds or thousands of sites are reporting data every day, you will need a program to screen the information for anomalies or outliers.

But since most device firms are doing only a few studies at a time, having the case report forms faxed or scanned to the monitor for remote viewing is an inexpensive alternative. The monitor checks the forms for, [1] completeness (no missing data), [2] contemporaneousness, [3] legibility, [4] logic, [5] adverse device effect reports. The forms can be checked for everything except source verification. The monitor issues and resolves queries for any issues observed. Most triggers are found by simple observation.

Then the case report forms are forwarded to the data center where the data are entered into a local (often non-networked) computer. Here the data may be statistically analyzed to see if a site, data element, procedure or process step is at risk and merits an unscheduled, on-site monitoring visit.

Monitoring Plan
A monitoring plan should be developed for every study. This requirement is not new to sponsors conducting IDE studies or European studies under ISO 14155, but the guidance makes clear recommendations for the format. I recommend you develop a template for a separate document that can be referenced in the protocol, and don't make the monitoring plan a part of the protocol itself.

Section One—Study Description
As a separate document, the first section describes the protocol, investigational device, purpose of the study, and other obvious reference information. You need to describe, [1] the monitoring approaches you will use for the study, [2] criteria for determining the timing, frequency, and intensity of planned monitoring activities, [3] specific activities required for each monitoring method employed during the study, including
reference to required tools, logs, or templates, [4] definitions of events or results that trigger changes in planned monitoring activities for a particular clinical investigator, and [5] identification of possible deviations or failures that would be critical to study integrity and how these are to be recorded and reported.

A second opinion is a phone call away

Sometimes to you want to discuss an issue, but implement the solution yourself. If you want a second opinion about clinical, regulatory, biological safety, or reimbursement strategies, sign us up as consultants. Email us here or call Nancy at 773-489-5721.

Section Two—Reporting
Section two of the monitoring plan should discuss the communication of monitoring results to management, review boards, and regulatory bodies. It should describe the format, content, timing, and archiving requirements for reports and other documentation. For example, if the monitor receives case report forms by fax or email, it makes sense to require, say, a weekly or monthly report. Section two should discuss the process for appropriate communication of the findings to management, review boards, and regulatory bodies.

Section Three—Noncompliances
Section three of the plan should discuss the management of noncompliance. How, and who, will follow up with investigators found to be non-compliant with the regulations, protocol, or IRB requirements? If non-compliances persist will the site be retrained, terminated from the study, or will some other action be taken? If protocol deviations are detected you need a plan for root cause analyses. Problem-solving has always been a part of a monitor's responsibilities in device trials and it may require an on-site visit to really assess the problem. I have had the root cause to be such simple things as the data transcriber needing spectacles, a three-hole punch, or a dedicated phone line. Eyes-on may be the only way to detect such problems so they can be solved.

Section Four—Training
Section four of the monitoring plan should describe specific training for monitors and internal data auditors, especially with the detection of triggers and recommendations for on-site monitoring visits. For device studies, I recommend section four also discuss training plans for investigative site personnel. This section should also describe plans for random quality audits.

Section Five—Amendments to Monitoring Plan
Section five describes the process for amending monitoring plans. What events may require review and revision of the monitoring plan and establish processes to permit timely updates when necessary?

Much of sections Two to Five can be incorporated into the clinical research quality management system. By developing procedures for off-site monitoring you won't have to repeat them in every monitoring plan.

References
[1] 'Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring' FDA, August 2011. http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf
[2] 'Guideline for the Monitoring of Clinical Investigations' FDA, January 1988.
[3] 'Quality Management in Clinical Trials', March 2009, Clinical case series from Pfizer.