CDG Whitepapers

Choosing A Device CRO-Consultancy

A Whitepaper by Dr. Nancy J Stark

©2012 Clinical Device Group Inc, all rights reserved.

Device contract research organizations (CROs) play a different role in the product development cycle than drug CROs, and it is unrealistic to expect a drug CRO to deliver the service a medical device sponsor needs. In the drug world a CRO is used more like a Kinko's or a dry cleaners: the sponsor drops off the protocol on Monday and picks up the report on Friday. The CRO acts independently and is legally responsible for the sponsor obligations identified in the contract. [1. 21 CFR Part 312.52]

In the device world there is no possibility to legally transfer sponsor obligations to a CRO. The manufacturer may use the services of a CRO, but legal responsibility for sponsor obligations remain with the manufacturer. As a result, medical device CRO must serve as a consultancy; collaborating intimately with the manufacturer on sponsor responsibilities such as recording and reporting adverse device effects, investigator noncompliances, device deficiencies, investigator and study staff training to proper device use, and so forth. The device CRO-consultancy cannot act independently because the legal responsibility of their actions remain with the manufacturer.

Choosing a device CRO-consultancy
As the manufacturer you should choose a CRO-consultancy that complements your corporate personality, fills the knowledge gaps in your organization, and makes themselves readily available to you. Here are some specific characteristics that you can evaluate.

Device experience of top management
Does the CEO have medical device experience? The career history and experience of top management sets the focus of any company. When you are interviewing a CRO, be sure to ask for the CVs of top management. Have they worked for medical device firms? Have they worked in clinical research or regulatory roles? Are they sensitive to the differences in regulations between devices and pharmaceuticals? Without device experience at the top, you are unlikely to get device experience at the level of implementation.

CDG Inc—A Device CRO Here to Stay

CDG is a small, networked CRO offering services in biological safety, quality design, clinical research, statistics, audits, reimbursement, and clinical evaluations for medical devices. We have many clients who use us as a consultancy—they call for advice when they want direction about how to proceed.

We have been in business since 1990, and our President has been in the device industry since before Part 812 was final! Please call us to discuss consulting needs for your firm. Click here for more information or call 773-489-5706.

Size and power
How big is the CRO in terms of number of employees or annual gross revenue? Little sponsors get swallowed-up by big CROs. If your project is small compared to the other projects the CRO is running you will be ignored when bigger clients have problems and demand attention. The disparity in size leaves you with no power to demand immediate help. Small CROs will value your business and are more likely to make themselves available to advise you.

But in fairness, if you are a large device manufacturer you may be more comfortable working with a large CRO. Large CROs can bring more sophisticated electronic data tools, financial reporting tools, or other management advantages that small CROs cannot afford.

Progressive slices of the pie—estimates and schedules
In CDG's experience, device projects tend to be progressive slices of the greater pie. For example, the first project might be to review device biological safety, if that goes well we may be asked to review a protocol, and then to calculate an array of possible sample sizes, then to implement a study, and so forth. One project may turn into five projects or twenty-five project over time.

In any case, once you settle on some specifications the CRO should be able to give you an estimate and schedule. I am using the word "estimate" to mean the number of labor-hours it will take to complete the project and the word "schedule" to mean the overall duration of the project. For example, it might require 40 man-hours to write a clinical evaluation over a duration of ten weeks.

If the estimate and schedule are within 25% of the final cost and duration, the manufacturer has done a good job of planning and the CRO has done a good job of execution. But I promise you, there is no way the project will go to completion without a change in specifications.

Who will do the work?
It is key for the device manufacturer to ask, "Who will do the work?" Some large CROs have been known to show the CVs of device-experienced personnel during the negotiation phase, but then have another—device-inexperienced—individual actually do the work. It is understandable that personnel availability may change from the time a proposal is first discussed to the time the contract is finalized, but it is wrong to assign new individuals who don't have the necessary expertise.

During project negotiations be sure to ask the CRO if they have the expertise to do the work. And then don't hesitate to object later if the assigned individual isn't as skilled, thorough, or knowledgable as you need them to be. From time-to-time every company makes personnel adjustments and it should not become a big issue.

Poll Question: How did you use a CRO last year, if at all?

How did you use a CRO last year, if at all? Respond to the poll question and help us learn about our industry. Your answers are completely confidential and will be used only for my own research. The average responses will be posted in an upcoming whitepaper. Answer me here. —Nancy

Independent experts: one-person firms
Many of the finest medical device consultancies are independent, one-person firms. They have chosen an entrepreneurial lifestyle, preferring the independence of self-employment to the security of a routine paycheck. It is reasonable to ask what experience they have in the device industry, why they chose to work for themselves, and if they can commit to the duration of your project.

I've had some interesting bosses over the years. When I decided to become a consultant in 1990, my soon-to-be-ex boss told me that "consultant" was a code word for "I'm unemployed." I took that as a challenge and knew that I would have to work at building a business persona. It was important that manufacturers perceived Clinical Device Group Inc as a company that was here to stay.

When out-sourcing to a new independent expert, look for signs of someone trying to create a business: do they have a business name, business cards, a marketing effort, a website, a brochure, conference booths, speaking engagements, and the like? Look for people who have actually worked in the device industry and understand how devices are conceived, designed, tested, investigated, and commercialized. Independent experts run businesses themselves and will bring a sensitivity to the business needs of your organization.

Networks
Many independent experts form relationships with other independents and work together to provide services to clients beyond their own specific expertise. Ask what services they can provide to you directly via employee or sub-contractor, and whom they can refer to you. They can often provide you with a full range of clinical and regulatory services just like any other full-service CRO through their strategic alliances.

Communications
If the experts are in a network, who is the project leader on their side? How will communications flow, knowing that the experts may live in different states? Will you have direct access to the project implementers? Think of the management model used in most organizations; the doer in clinical research is free to send an email to the doer in regulatory affairs, but the managers of both functions are copied to keep them in the loop.

Who will bill whom? I follow a general contractor model: my sub-contractors bill CDG, CDG in turn bills the manufacturer.

Do you need (or want) separate non-disclosure agreements (NDAs) from each sub-contractor? Occasionally manufacturers want NDAs from each sub-contractor, but more commonly the primary NDA requires CDG to have NDAs with each sub-contractor so that the requirement for confidentiality flows through.

Contribution to the industry
I have had an independent expert walk away from a contract in the middle of a project in order to take a job. I respect everyone's need for security, but it is dishonest for experts to present themselves as consultants who are here to stay, and then betray that promise.

Businesses who are committed to their industries show it by finding affordable ways to demonstrate that commitment. CROs or consultant may offer public presentations, contribute written articles, participate in standards development, or find other ways to give back to the industry and to mentor younger persons. Looking at the CRO's commitment to the industry is one way to judge their staying power.

Standing and reputation
"Standing" refers to the reputation of the CRO or consultant in the eyes of clients and peers. For older CROs or independents standing may be self-evident; for younger CROs or independents you may want to ask for references.

I have had the unusual situation of a client asking me if I agreed with the opinion of another consultant. I did not, and cited the regulations on which I based my opinion. For the manufacturer it was a question of whom to believe. It is useful if you have confidence in the advice the CRO is giving you.

Project Management for Clinical Studies
Stark NJ,Project Management for Clinical Studies, Clinical Device Group Inc, Chicago, IL (2003).




Probably the most overlooked aspect of device investigations, project management is absolutely essential if a trial is to be implemented on time and with care. The book follows the usual outline of any project: conception, planning, implementation, reporting, and close-out; but focusing on clinical research. It gives sample planning forms, Gantt charts, budgets, methods for cost analysis, reporting metrics, and sample CRO contracts. If you are in management you should insist your staff read this book, it is my personal favorite of all the books I've written.

290 pages, perfect bound, illustrated, ISBN 1-889160-09-1. Table of Contents. $250 USD.

Bidding Process
A bidding process is a process where the submits a bid for a job or project. The process requires strict specifications for the project (i.e. the sponsor must know exactly what they want) and works well for "cookie-cutter" investigations. The idea is that bids are obtained from several potential suppliers and the "best" bid is accepted, where "best" is some combination of people, resources, and cost.

It's a personal choice on the part of CROs, but I rarely participate in bidding situations. I have put many hours of original research and considerable proprietary business practice in a bid, only to have the project go to another party and my proprietary information used as a learning tool for the cheaper firm. In one case the sponsor decided not to out-source at all, but to do the clinical trial in-house based on what they learned from the bids they solicited.

If you use a bidding process, it's important to play fair and ethically. Sponsors and manufacturers get reputations among CROs, too; and you don't want to risk being in a situation where no-one of quality will contract with you.

Capabilities meeting
Larger CROs will often travel to the manufacturer's site and make a capabilities presentation. Smaller CROs will often host a capabilities meeting, but the cost of travel is the burden of the manufacturer. Most frequently the meeting takes place by teleconference. No matter how it is arranged, the CRO should be able to make a presentation reviewing their company history, business model, service capabilities, technology capabilities, and resources.

If I travel to a conference I may learn, say, four new things. If it cost me $2000 in travel expenses to attend, then a new piece of information costs me $500. As a CRO, I have an unspoken rule of thumb. I will give away two new pieces of information to a potential client to establish credibility, but no more.

As the manufacturer, you are engaging the CRO in conversation so as to assess their knowledge, confidence, organization, and corporate personality. You should expect to learn a couple of new things during the meeting.

Website, brochure, company information, references
There are many ways for a CRO to present their capabilities with professional, take-home, printed materials. Whether it is a simple file folder, a four-color glossy, or an electronic brochure, you are looking for a material that describes the CRO's organization, presents its vision and mission, describes it scope of services, and tells you who they are.

References
[1] 21 CFR Part 812.32 Transfer of obligations to a contract research organization.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

A Strategic Approach to Protocol Design

A Whitepaper by Dr. Nancy J Stark, made available by Clinical Device Group

A former boss once told me this story. When I walk into his office he envisions me with a monkey on my back. As we begin our conversation the monkey jumps off my back and starts to create havoc, knocking papers to the floor, turning over the lamp, swinging from the chandelier, and kicking the waste can. My goal is to leave the office with the monkey on my boss's back; my boss's goal is for me to leave his office with the monkey on my back. When the conversation ends, the person who has something to do has the monkey on their back.

The same analogy can help you to write a protocol. The monkey is consensus on several key points about the study design. If you get consensus from the project team and top management, the monkey is on their back. If you have to start writing the protocol without agreement on these points, the monkey is on your back. With committed consensus, you can plan on 20-30 working days over a duration of 2-3 months to write a new protocol for a new technology. Without consensus it can take years.

Authority
To lay some background, let's look at the regulatory requirements for a clinical investigation. FDA regulations tell us that we shall have valid scientific evidence to support the safety and efficacy of medical devices. Valid scientific evidence [consists of] evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsible be concluded by qualified experts that there is a reasonable assurance of safety and effectiveness of a device under its conditions of use. So the purpose of a clinical investigation in the States is to provide reasonable assurance of safety and effectiveness for the investigational device. [1. 21 CFR Part 860.7(c)(1-2) (paraphrased)]

[08] Protocol & Trial Design Workshop on CD (March 2012)

A newly updated version of the Protocol and Trial Design Workshop is available on CD. The 4.5 hour presentation, consistent with both US and EU requirements, gives the backstory to each section of the protocol: why is it there, what are regulators looking for, is it important, can you omit it? You also receive a "fill-in-the-blank" protocol template with detailed instructions so you can focus on the creative work and leave the boilerplate in place. The well-researched course is designed and studio-recorded by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

For those of you concerned with clinical trials in Europe the path of authority is a little more complex. You'll begin with the Medical Device Directive which states the demonstration of conformity with the essential requirements must include a clinical evaluation in accordance with Annex X. [2. MDD 93-42-EEC I.6a)] A clinical evaluation is the assessment and analysis of [existing] clinical data pertaining to the medical device in order to verify the clinical safety and performance of the device. [3. MEDDEV 2.7.1 r3] If existing clinical data cannot verify safety and performance, then a clinical investigation must be done per ISO 14155 or the intended use reworded. The standard states that the justification for the design of the clinical investigation shall be based on the evaluation of the results of a clinical evaluation. [4. ISO 14155 Clause 5.3] This is the logic behind bringing a medical device to market in the EU via the literature route.

Start at the end
To leave the monkey on the project team's back, here are the key points on which you will want consensus.

Intended use
Intended use is a US regulatory term that refers to what you say in the labeling. It ties the physical device, and the words we use to describe it, into a single entity. [5. 21 CFR 801.4] Intended use is a broad concept—think verbs, function, and action. Consider the following examples:
[1] A neurostimulator is intended for stimulating the vagus nerve.
[2] An MRI scanner is intended for soft-tissue imaging using nuclear magnetic resonance.
[3] A fetal oxygen monitor is intended for monitoring oxygen in a fetus.

The intended use dictates the specialty we need in an investigator, say orthopedist or neurologist, and begins to define the subject population.

Indications for use
Indications for use are the diseases or conditions for which the device will be used. Choose diseases, conditions, or procedures that have accepted medical definitions. The indications for use dictate the subject selection criteria. [6. 21 CFR 814.20(b)(3)(i)] Consider the following examples:
    [1] A neurostimulator indicated for prevention of epileptic seizures.
    [2] An MRI scanner indicated for detecting structural abnormalities.
    [3] A fetal oxygen monitor (hopefully) indicated for determining if a C-section is warranted.

Consulting Services

CDG's experts are frequently engaged on a retainer basis to be available for phone calls, email exchanges, document review, good advice, or other consultations. We have experts in most medical device pre-approval issues, including market assessment, biocompatibility, regulatory, clinical evaluations, clinical investigations, statistics and sample size calculations, and audits. To discuss a Service Agreement, call or email us here or ask for Nancy at 773-489-5706.

Performance and device deficiencies
Performance speaks to the physical properties and operation of the device; for example, what range of light-waves does it emit, how hot does it get, do the buttons stick, how easy is it to use? Performance also speaks to the ISO concept of device deficiencies such as inadequacies with respect to the devices identity, quality, durability, reliability, or safety. The intended performance or concerns about lack of performance dictate several of the questions that you should include on the case report forms. [7. ISO 14155] Consider the following examples:
    [1] The neurostimulator delivers xyz amps.
    [2] An MRI scanner has good contrast resolution.
    [3] A fetal oxygen monitor detects oxygen levels in fetal blood.

Safety
Safety refers to the absence of adverse events to subjects and the absence of serious, unanticipated adverse events to caregivers or bystanders. [8. ISO 14155] Recording and reporting of adverse events varies with the host country (the country in which the study is being implemented) and with the study risk. Every protocol requires a "risk assessment". The risk assessment is often written in tabular format with headings of hazard, frequency, severity, risk, mitigation, and residual risk.

A hazard is an adverse event that may occur to a subject, or a serious, unanticipated adverse event that may occur to a caregiver or bystander. Frequency refers to the rate at which the hazard is likely to occur; it can be estimated on any convenient scale such as 1-100%. Severity refers to the intensity of the event, such as mild, moderate, or severe; a three-point scale is useful for estimation. Risk is a function of frequency and severity; you can use any meaningful mathematical relationship, in the example the two scores have been added. Mitigation refers to actions that are taken to lessen or prevent the risk; and because you are writing a protocol you’ll focus on measures that can be taken by the investigator. Residual risk is the amount of risk left over after all mitigation measures are taken. Residual risk can be described as minor, requiring an outpatient procedure, or hospitalization; or by any other meaningful scale. [9. ISO 14155] Clearly the safety questions on the case report forms will come from the risk assessment. In the US, the risk assessment may cause the IRB to see the study as significant risk. Consider the following examples for residual risk:
    [1] Neurostimulalton of the vagus nerve may cause headaches.
    [2] MRI scanner may expose technical staff to NMR.
    [3] Placement of fetal oxygen monitor sensors may injure fetal soft tissue.

Target market
Who is going to influence the buying decision about your device within the hospital or clinic structure? Often this person is a good choice for clinical investigator.

Major competitive products or procedures
Controlled, randomized trials are not usually required for medical devices in order to gain clearance in the US, and a comparison to competitors does not enter into the equation in the EU. Nevertheless, it is useful to know what your competitors are claiming, how they word the intended use and indications for use, what clinical trials they have done, what predicate devices they referenced, and the like. If you do decide to do a randomized, controlled study, a thorough knowledge of your competition will help you decide on what product or procedure to use as a control.

Claims
A claim is an assertion of truth about your product. The claims you want to make dictate the endpoints (clinical measurements that indicate device success or failure), outcome measurements, whether a study is comparative or single armed, and many other features of investigation. The team may list several claims they want to make about the device; limit your selection to two or three. If you're a new company go after the easy claims first to get experience in conducting studies, getting FDA clearance, and commercializing a device. If your firm is experienced you may feel comfortable going after more complex, but desirable, claims in the first investigation.

Frequently the project team will settle on the claims to be investigated. And then you'll begin writing the protocol only to find there is no good methodology for measuring success or failure, the sample size makes the study unaffordable, there are too few patients to maintain a reasonable recruiting rate, there is no market for treating such patients, or some other insurmountable problem. You go back to the project team, rework the wording of the proposed claims, and resume writing the protocol. This is the monkey creating havoc.

Clinical Trials Design
Stark NJ, Clinical Trials Design, Third Edition, Clinical Device Group Inc, Chicago, IL (2000).

This is one of the few books about writing protocols and designing trials that links them to the trial hypotheses and product claims. That simple sentence sums up the guts and substance of the book: first decide what you want to claim about your device, and then design a trial to investigate it. The concept is so important and so rarely discussed that the book has been translated into Japanese and Korean.

Still current, still relevant, and still a useful reference. It is intended for use by medical device manufacturers. Provided in a high-quality three-ring binder so you can add your own notes, illustrated, 300 pages. ISBN 1-889160-02-4. Table of Contents. $250 US.

Protocol Contents
There is no absolute format for a protocol, although FDA issued a guidance document in 1996, [10. FDA Statistical Guidance] and ISO 14155 contains an extensive checklist. Here is a suggested format that is a combination of the two:
[01] Cover page, Table of contents, Executive summary.
[02] Device description, photograph, accessories, description of comparator.
[03] Contention of SR or NSR regulatory status.
[04] Study objective.
[05] Hypotheses, primary and secondary.
[06] Endpoints and procedures, primary and secondary.
[07] Medical risk analysis.
[08] Study design.
[09] Subjects.
[10] Ethical considerations.
[11] Sample Size.
[12] Data collection.
[13] Control of study devices.
[14] Adverse event recording and reporting.
[15] Monitoring plan.
[16] Documentation.
[17] Central laboratories.
[18] Data management.
[19] Data analysis.
[20] Data monitoring board, Data safety monitoring board, Clinical events committee, if one will be used.
[21] Personnel Responsibilities.
[23] Return of unused inventory.
[24] Study registration and publication policy.
[25] Study cancellation.
[26] Definitions.
[27] References.
[28] Attachments.

Internal Approvals
Internal approvals are another great opportunity for the monkey to create havoc. But you can keep him in his cage by using a approval form that identifies and limits each function's responsibilities. For example, the project team leader reviews the protocol to ensure you have described the device accurately and you are investigating the right device; marketing ensures you are investigating the right claims, manufacturing reviews to ensure they can actually make the investigational device, the medical manager assures you have an accurate and complete risk analysis, regulatory assures you have identified the regulatory status of the investigational device correctly, statistics reviews the sample size and data analysis plan, and so forth. Only your manager has the privilege of global review.

References
[1] 21 CFR Part 860.7(c)(1-2) (paraphrased).
[2] MDD 93-42-EEC I.6a).
[3] MEDDEV 2.7.1 r3.
[4] ISO 14155 Clause 5.3.
[5] 21 CFR 801.4.
[6] 21 CFR 814.20(b)(3)(i)
[7] ISO 14155 Clause 3.15.
[8] ISO 14155 Clause 3.2.
[9] ISO 14971 "Medical devices—Application of risk management to medical devices" (2007).
[10] FDA's Statistical Guidance for Clinical Trials of Non-Diagnostic Medical Devices (1996).

Questions or Comments?
Please post your questions and comments below.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

 

Uptick in Clinical Evaluations

What Do Notified Bodies Want?

A Clinical Device Whitepaper by Nancy J Stark

We've seen an uptick in requests for clinical evaluations recently, and I thought it would be worthwhile to review the requirements. Where you begin depends on where you are now: 1) writing a clinical evaluation for a new, pre-approval device, or 2) writing a clinical evaluation for a device that is already commercial. In the first case, you are asking the question, "Do I need new clinical data for certification?" In the second case, you are bringing your documentation up-to-date.

Clinical evaluation reports for pre-certification devices
If you're still in the stage of designing your device, you begin by identifying its intended uses. Then you evaluate the existing clinical data to determine if it is sufficient to support safety and performance per the Essential Requirements of the MDD [1 MDD] or AIMD [2 AIMD], or if you need additional clinical data. If additional clinical data are needed you have the choice of revising the intended use or performing a clinical investigation.

Once you believe the data are sufficient, you convince top management to grant a CE Mark and sign the Declaration of Conformity to your previously certified quality management system. Finally, you will discover the truth when the Notified Body comes to make an inspection call.

Clinical evaluation reports for commercial devices
The more common situation is that your device is already on the market in Europe, and the Notified Body has informed you that its clinical evaluation is overdue or needs updating. They may give you a grace period, but you are expected to get busy. You follow the same process of evaluating the existing data to determine if it is sufficient to support safety and performance for the intended uses. Should the report conclude that the existing data are insufficient you are in a difficult situation: you'll need to suspend sales until additional data are acquired.

Because the consequences of determining the data are not sufficient are so financially significant for your firm, and the consequences of determining the data are satisfactory are so financially favorable, it is wise if the report writers are as far removed from the device designers as possible.

Resources
To write the report you need a team of at least three individuals: 1) a medical writer, 2) an information specialist, and 3) a statistician.

In addition you need a written procedure for how to proceed, a report template (useful, but not required), and access to literature databases such as Medline. Medline is easy to use, it is operated by the US National Library of Medicine and is available to anyone with a browser. Medline provides access to 5400 worldwide journals in 39 languages dating from 1947 to the present. And it is free.

Date Wheels for Clinical Evaluation Planning

A typical clinical evaluation involves the following steps and durations:
[1] Identify key questions—1 weeks.
[2] Identify databases to search—overlaps.
[3] Identify search scope, keywords, search strings—1 week.
[4] Review abstracts, discard with reasons—1 week.
[5] Acquire articles—2 weeks.
[6] Read articles—1 week.
[7] Weight articles for relevance, discard with reasons—overlaps.
[8] Weight articles for statistical validity—1 week.
[9] Medical writer summarizes literature—2 weeks.
[10] Review sponsored clinical trials, if any—0-2 weeks.
[11] Review risk management system & complaint file—2 weeks.
[12] Write final evaluation—1 week.

Date Wheels are a handy, pocket-sized calendar used by project managers to quickly plan simple schedules. If the clinical evaluation work begins on March 1, you can hope for a final report about three or so months later, or about June 1. Buy them here.

A Three-Legged Stool
First leg
The first leg of a Clinical Evaluation Report is an evaluation of the existing literature data. The Global Harmonization Task Force (GHTF) Study Group 5 document "Clinical Evaluation" coupled with MEDDEV 2.7.1 Rev 3, which is slightly easier to read, are the guidances that tell us how to do the literature evaluation.

Here's what the medical writer—working together with you—will do:
[1] Define the key questions.
[2] Identify the databases to search (Medline, Embase, MAUDE, Cochrane).
[3] Define scope of search and search strategies using a qualified information specialist.
[4] Scan the abstracts to identify articles for review.
[5] Acquire full-texts of articles.
[6] Weight the articles for technical significance, discard anything with a low weight.
[7] Have a statistician weight the articles for statistical significance, discard anything with a low weight.
[8] Review only those articles with the highest weightings; see averages below. The literature data are evaluated to determine if they support safety and performance of the device for its intended use. I like to prepare a separate literature review and attach it to the main clinical evaluation as an annex. The literature review annex has several required elements including the search strings used by the information specialist, a list of abstracts scanned, the articles reviewed, their weights and justification for those weights, and full-text copies of the articles.

Second Leg
The second leg of the evaluation is a review any existing clinical investigations that your firm may have sponsored. These data are reviewed separately from the literature because you have closer access to the details. The writer asks questions such as: "Was the Declaration of Helsinki followed?" or "Were adverse events resolved?" If not, those facts are noted in the report but the data are not used.

Once the writer is satisfied that ethical and administrative requirements were correct, he/she reviews the study results to see if they support safety and performance of the device for its intended use. I like to prepare a separate sponsored study review and attach it to the main evaluation as an annex.

Third leg
In the third leg, the medical writer comes to the most sensitive step of all; examining the complaint file against the risk management system. They review the file to see if complaint handling, as dictated by the risk management system, supports the safety and performance of the device for its intended use. I like to prepare a separate risk management review and attach it to the main evaluation as an annex.

The Clinical Evaluation
Finally, the medical writer takes the literature review, sponsored study review, and risk management review altogether and reaches an overall conclusion of whether the existing data support the device's safety and performance for the intended use, or if additional clinical data are needed newly. You know what happens if new clinical data are needed.

The Technical File
If the data are sufficient, the clinical evaluation is filed away in the Technical File, a written procedure describing the writer's strategy is incorporated into your quality management system, and you wait with baited breath until the Notified Body inspector comes.

Recently CDG conducted a poll on the average number of
articles used in a clinical evaluation. Here are the results.
Please add your two cents worth below.

How long, how much
There are only two questions in top management's mind: 1) how long will it take, and 2) how much will it cost. The answers are simple: "It depends." Let's say you have a very nice ultrasonic toothbrush and you intend to use it for cleaning roadside burns—the kind of burn you get when your motorcycle crashes and your skin scrapes against the pavement. The toothbrush is used to remove the dirt, grit, and tarmac from the wound. It's an unusual intended use and raises all kinds of questions about potential damage from the bristles, safety features to prevent over-brushing, effects of ultrasound on open wounds, comparing gum tissue to wound tissue, instructions for use, and so on. A clinical evaluation for cleaning wounds will take longer and cost more than a report for brushing teeth with the same device.

CDG can help
Clinical Device Group is poised and ready to help you with writing your evaluations. We have medical writers, information specialists, experienced statisticians, a written procedure, a standard template, and a proven process. Clinical evaluations written by our specialists have been reviewed by at least three different Notified Bodies and all have been accepted. We work collaboratively with a point-person on your side so that you are involved in the process every step of the way.

References
[1] Medical Device Directive 93/42/EEC.
[2] Active Implantable Medical Device Directive 90/385/EEC.
[3] Global Harmonization Task Force, Study Group 5, Clinical Evaluations. http://www.ghtf.org/sg5/sg5-final.html
[4] MEDDEV 2.7.1 rev 3. http://ec.europa.eu/enterprise/sectors/medical-devices/files/meddev/2_7_1rev_3_en.pdf

 
Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

 

 

 

 

 

 

Relationship Building in Clinical Trials

A Whitepaper by Dr. Nancy J Stark

Relationship building is an integral part of the successful implementation of any clinical trial. You need good relationships with investigators, study staff, your project team, your colleagues, your boss, and your subordinates in order to successfully implement any project. One thing is for certain: they are not going to change. If you want a good relationship you will have to adapt to them.

Myers and Briggs are great heroines and role models to me. Imagine two women psychologists during World War II. Katherine Cook Briggs and her daughter Isabel Briggs Myers had plenty of obstacles stacked against them. They had only master's degrees, they were women, they were playing in a man's world, yet they developed a personality tool that is still used seventy years later. These two did original research extending Carl Jung's theories to practical applications in the workforce.

Why did it matter? Women were entering the workforce en masse without a clue as to what they would be good at: office work (quiet and peaceful), factory work (noisy and lots of yelling), data work (numbers and logic), design work (color, texture, image, spacial relationships), or whatever.

In this whitepaper I will attempt to adapt the work of Myers and Briggs to the critical task of relationship building in clinical trials. I use layman's language because I am not a psychologist. If you are, feel free to add your comments, corrections, contributions, and rebuttals below.

Jungian Background
The theory of personality type originates from the work of Carl Jung, who proposed that people perceive the world through two different functions: 1) the 'rational' (judging) functions of thinking and feeling, and 2) the 'irrational' (perceiving) functions of sensing and intuition. Jung went on to propose that these functions are expressed in either an introverted or extraverted form. From Jung's original concepts, Myers and Briggs developed their own theory of personality types. [1. Myers-Briggs Type Indicator]

Risk-Based Monitoring for Medical Device Firms on CD

We offer training for medical device firms on risk-based monitoring on CD. The course covers the details of the FDA guidance 'A Risk-Based Approach to Monitoring', recommendations on using it for device studies, includes a sample monitoring plan and template, and features QnA addressing issues unique to device manufacturers. CDs are nice because you can stop and start them, share with your colleagues, and build a library for new hires. The well-researched course is designed and presented by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

Understanding Personality Types
Myers and Briggs saw that there are four essential steps in how people relate to the world: 1) we focus on the world of our choice, 2) we perceive information, 3) we judge the information, and 4) we present the information to the outer world with a characteristic attitude. We do each of these four steps as either introverts or extroverts for a total of sixteen possibilities. For the sensors among us, I've tried to represent the concept in a simple, colorful visual image. [2. Gifts Differing]

As children we learn how to survive in the world that we know first: family and parents. And those survival skills are our default personality throughout life. As we gain experience and mature we learn to move out of our preferred personality type and become skilled at all possibilities, allowing us to act appropriately in different situations. But when under stress we tend to revert to the survival skills of our childhood. If we understand our own personality types, and become adept at discerning the personality types of others, we can adjust our behavior to build the best possible relationships with our colleagues.

[1] Focusing on the World: Introversion or Extroversion
People prefer to focus on either the inner world of thoughts and ideas (introversion) or the outer world of people and things (extroversion). Introverts do their best work in their heads in reflection; extroverts do their best work in the outside world, in action. Our focus on the world effects dramatically how we live in the world. Only an introvert would sit alone in their office and write this whitepaper. An extrovert will be impatiently waiting to get to the practical applications.

Introverts organize the facts and principles related to a situation and are good at researching material, writing protocols, or designing investigations; extroverts organize the situation itself, including any idle bystanders, to get things rolling. Extroverts are good at implementing clinical investigations.

Help and advice is a phone call away

Sometimes you want to discuss an issue, but implement the solution yourself. If you want advice, a second opinion, or someone to talk to about clinical, regulatory, biological safety, or reimbursement strategies, sign us up as consultants. Email us here or ask for Nancy at 773-489-5706.

Introverts work out their insights slowly and carefully, searching for eternal truths. Extroverts have an urge to communicate and put their inspirations into practice. Can you imagine the annoyance of an introverted study nurse when an extroverted monitor bombards her with emails to give her the latest study updates; or the annoyance of an extroverted (and eager) investigator when an introverted monitor is slow to communicate the start date?

[2] Perception, Taking in Information by Sensing or Intuition
Perception refers to how you get information from the outside world into your head. People take in information either by sensing or by intuition. Sensing means to become aware of information by our five senses: touch, smell, sight, sound, and taste. Intuition means to become aware of information through the unconscious. With intuition we incorporate our ideas or associations to what we sense with lightning speed and seemingly without logic.

Discuss a new device with a sensing person and they may tell you it is too lightweight, it is smaller than the current model, the materials aren't sturdy enough, but the display panel is clear and easy to read—all the information taken in by their senses. Sensing people are well suited for the demands of clinical trial monitoring. Practical and matter-of-fact, they see the world as it is rather than as it should be. They are often good project managers, following the project with Gantt charts, spreadsheets, drawings, flowcharts, and data. If both you and the investigator are sensors you may focus too closely on the details and lose sight of overall progress.

Hand an intuitive person the same device and they may answer, "This reminds me of my grandmother. She used this kind of device every day when I was a kid, we'd check in on her to make sure it was functioning properly and then play a game of checkers." Instead of focusing on the device, your intuitive colleague focused on her associations with it. [3 Tieger]

Intuitive people often have inspired product ideas, clever solutions to problems, or make associations that no one else would see. Intuitive people are good at picking up undercurrents, they know when a situation is 'off' even if they can't exactly tell you why. If both you and your investigator are intuitives, you may have imaginative conversations but not notice some study problems.

I had an extroverted, sensing boss who had a rule that we could present no more than three possibilities to him. If he asked you what went wrong, you had better not give him a list of five things. If he asked you for a recommendation, you had better not have more than three. His intuitive, data-loving subordinates were terrified.

“Readers who are sensors will tend to confine their attention to what is said here on the page... readers who are intuitive are likely to read between and beyond the lines to the possibilities that come to mind.”

[3] Judging, Processing Information by Thinking or Feeling
Judging refers to how you process the information you take in. Thinkers use a logical process aimed at reaching an impersonal finding. Feelers use an emotional process of viewing information in a personal, subjective manner.

The thinking protocol writer may make a list of all the data elements they think are necessary to meet the purpose of the new clinical trial. As they develop the protocol they use their list of data elements as an absolute guide. They are not swayed by pleas from marketing to include questions they have already dismissed as unessential or exclude questions because "it is knowledge we would rather not have." Thinking writers may try to collect too much data in a clinical trial—oblivious to the impracticality.

If, as a member of the project team, you want additional data elements included in the study you should listen closely to the thinking writer's rationale for omitting them, be respectful of their reasoning, and use facts and data that relate back to the trial's purpose to persuade them to make the change. Thinking people objectify situations—dealing in facts, figures and data; they do not respond well to insincere flattery. (My husband is even suspicious of sincere flattery, I only tell him he has done a good thing when I know he already thinks so himself.)

CDG knows risk-based monitoring for device studies

Our professional monitors are experienced in risk-based monitoring of device studies (because that's how device studies are often done), they know how to work with all personality types, their goal is to worry so you can sleep at night. Our Network Staff has been involved in dozens of device trials and hundreds of issues; their extensive knowledge can solve your problem de jour. Click here to request a proposal for clinical research services from CDG or phone 773-489-5706.

A feeling person may make a list of data elements but it will have little influence on the final protocol design—the decision to collect data is made because it “feels” right. The decision is made first, and a rationale is developed later. A thinking monitor who is ill-at-ease with the site's data collection should be delicate with a feeling investigator who is confident the device works, but they should not back down. They should challenge the data with facts while being thoughtfully respectful of the investigator's mode of processing information.

Another consequence of thinking and feeling personalities is how they relate to each other. Feeling people subjectify situations—anticipating how facts, figures and data will affect the people involved. A feeling monitor (such as myself) might have difficulty enforcing an important requirement, say, for dating an informed consent, because it causes too much inconvenience to the investigator. The feeling person’s prime motivation is harmonious relationships, not regulatory compliance. A sensing boss has no difficulty pointing out the monitor's deficiencies.

“A reader who first considers whether an idea is consistent and logical is using a thinking process. A reader who first considers whether an idea is pleasing or displeasing, supporting or threatening, is using a feeling process.”

[4] Attitude to the World: Perception or Judgment
Our attitude in presenting information to the world is a reflection of which personality type is more strongly developed, perception or judgment. Attitude or "face" refers to how you present information back to the outside world. For extroverts, attitude is a reflection of the personality type that is dominant. For introverts, attitude is a reflection of the personality type that is subordinate because they reserve their favored, dominant preference for the more highly prized, inner world. In other words we know exactly what extroverts think and we never know what introverts think.

People with perceptive attitudes maintain an open mind regarding situations, continuing to seek additional information that can broaden and deepen their understanding. If pressed, they will disclose the facts, but they will refrain from telling you what it means or predicting the outcome. Perceptive people make others feel safe and accepted, and are valued members of any project team because they keep an open mind. Project managers with perception attitudes may be reluctant to move the action forward. They are more likely to delay conclusions or fail to act at all. They can be positively frustrating to bosses who who are at a critical decision-point.

Persons with judging attitudes shut off their perception (at least for the moment) and make a judgment. They may not have all the facts, but they can certainly tell you what they mean. Monitors or project managers with judging attitudes will have strong opinions about the value of a project, work product, piece of data, or whitepaper; and will freely share that opinion with listeners. They are more likely to come to wrong conclusions, a positively frustrating behavior to bosses who want to be sure before acting on a recommendation. But judgers are an essential component of a well-functioning project team; they bring issues to closure and move the project to the next step.

The Queen of Hearts in Lewis Carroll’s classic novel Alice in Wonderland had a typical judgment attitude (as well as being an impatient extrovert). “Off with her head, we’ll have the execution now and the trial later!” she said.

“Readers who are still following this explanation with an open mind are using perception, readers who have decided by now that they agree or disagree with me are using judgment.”

Putting It to Practice
The most important step in building relationships is the first one: know thyself. To help you get over the embarrassment of digging so deep into your psyche, I'll tell you that I am an introverted, sensing, feeling, judger (an ISFJ.) I live in my head, which is far more fascinating to me than anything going on outside; furthermore you will never know what I really think. I take in information by my senses: are there data-gaps, are the facts consistent, do I trust the source, do I need more information? I am a feeler, use the wrong word (mismanagement versus bad economy) and your conversation with me is over. And I am a judger, I have an opinion about almost everything, a quality essential for writing whitepapers or editorials.

Happily we are not stuck on home base. With time and experience I have learned to behave like an extrovert, make a rare intuitive leap of logic, be objective in my assessment of a situation, and withhold judgment until the facts are in—at the cost of a great deal of psychological energy. The hallmark of maturity isn't balance, but being able to move freely from one extreme to the other as the situation demands.

The next step is understanding your colleagues. Changing them is not your goal or your business, and anyway it won't work. The goal is to build relationships that will facilitate the successful completion of your trial. If you're working with an introvert be respectful of their privacy; if you're working with a extrovert enjoy the excitement. If your working with a sensor be prepared with facts and details, if with an intuitive go for the big picture. If your working with a thinker be logical in your presentation, if with a feeler never, never, never say "you". If your working with a perceptive be patient while they work through information on their own, if with a judger you may want to slow them down. The best project teams have someone of every personality type.

Relationships are complicated by supervisor/subordinate, client/consultant, parent/child, wife/husband, boyfriend/girlfriend, addiction, religion, politics, culture, heritage, wealth, and so much more; there is no one simple answer to building good relationships. But understanding personality types and adjusting our own behavior to meet the needs of our colleagues in order to meet the goal, is something we can all do.

References
[1] Myers-Briggs Type Indicator, Wikipedia, the free encyclopedia.
[2] Isabel Briggs Myers with Peter B Myers, Gifts Differing, Understand Personality Type, Davies-Black Publishing, Palo Alto, CA, (1980).
[3] Teiger PD, Barron-Teiger B, The Art of SpeedReading People, Little, Brown & Company, New York (1998).


Questions or Comments?
Please post your questions, comments, contributions, and rebuttals below.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

Risk-Based Monitoring—The New FDA Guidance

An Op-Ed by Dr. Nancy J Stark

FDA's draft guidance, "An Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring" (August 2011) references the ICH-GCPs six times. The ICH-GCPs were written in 1996 by pharmaceutical sponsors and regulators from the US, Europe, and Japan. It is 15 years old and is showing a few signs of age—there is no discussion of central data management or electronic security issues, for example. But because the phrase 'good clinical practice' is in the public domain, ICH was free to adopt it as part of the title of document E6, "Good Clinical Practice: Consolidated Guidance." The Center for Drugs published the guidance in the Federal Register giving it high standing in the regulatory community.

Although used by device manufacturers, the guidance does not work well. It's description of an Investigator's Brochure is chemically-centered not mechanically-centered, investigational devices are stored and dispensed but never installed, there is no mention of training an investigator to use the investigational product, there is no mention of software-controlled devices or software collected data, no mention of caregiver or healthcare provider safety; in other words, for medical devices, the ICH-GCPs are inadequate and out-of-date.

ISO 14155 'Clinical investigation of medical devices in human subjects—Good clinical practice' (2011)—the internationally definitive document for medical device clinical studies—is not mentioned even once; yet the international standard was closely harmonized with ICH-GCP to promote international continuity. I could live with the lack of acknowledgement in FDA's guidance if the same FDA Office hadn't taken such a heavy hand in writing the ISO standard. It is wrong to insist on practice-changing language (such as moment of consent versus moment of enrollment) and then turn your back on the work product as FDA seems to have done.

The draft guidance draws heavily on the ever-increasing presence of electronic data collection. A really cool EDC system can spot a lot of site problems before the human eye ever gets to the data. But these systems are expensive (think in terms of a $1M for a modest study) and unaffordable to small start-ups.

Finally, it was the Offices of Compliance from CDRH, CDER, and CBER who wrote this guidance. While they promise us the inspection manual will be promptly updated to match the guidance, they do not make assurances that the reviewers in the Office of Device Evaluation will follow along. Sponsors are strongly urged to get buy-in on the monitoring plan from reviewers before initiating a clinical trial.

While device manufacturers can use the new guidance to their benefit, often to defend existing practices, they should make certain FDA's Office of Device Evaluation buys into any IDE monitoring plan.

 

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Risk-Based Monitoring—The New FDA Guidance

A Whitepaper by Dr. Nancy J Stark

In August of 2011 FDA released a draft guidance document titled "Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring." At first I was very excited because the monitoring guidelines had not been updated since the 1988 "Monitoring of Clinical Investigations." But then, as I began to consider the ramifications of implementing the new guidance for medical device investigations I started to have mixed feelings. The first part of this whitepaper is a factual review of the draft guidance with ideas about how device manufacturer's can take advantage of it. The second part (next blog) is an op-ed expressing my views about what it will mean for the device industry. [1.Risk-Based Monitoring Guidance] [2. Monitoring of Clinical Trials]

Two points stand out as noteworthy before we begin: 1) the draft guidance was written by the Office of Compliance not the Office of Evaluation, and buy-in from an IDE reviewer is uncertain, and 2) the draft guidance addresses itself to drugs with a mere mention of devices as an afterthought, so most concepts have to be 'retrofitted' to device needs.

Part One: Review of Guidance
Risk-Based Monitoring
The draft guidance is based on the premise that most sponsors conduct on-site monitoring visits every 6-8 weeks with the goal of source verifying 100% of the data 100% for 100% of the subjects. It uses this premise to argue that sponsors will save money by moving to risk-based monitoring because such a move will result in fewer monitoring visits. Risk-based monitoring probably will result in fewer on-site visits for big pharmaceutical firms, who may be 'conducting hundreds to thousands of trials in locations around the world'. [3. Quality Management in Clinical Trials] But start-up medical device companies have never monitored as heavily as the guidance assumes and, in that sense, the principles risk-based monitoring won't save on monitoring costs.

The guidance suggests that we use fancy statistical algorithms and analyze real-time data as it comes into electronic data capture systems to identify high-risk sites or high-risk data. The algorithm could be something like 'Risk Priority Number = severity x likelihood of occurrence x detectability'. A Risk Priority Number is calculated for each identified risk. When high-risk sites or data exceed a trigger, they merit immediate, onsite monitoring visits. [3. Quality Management in Clinical Trials]

CDG's Risk-Based Monitoring Course on CD—Available Now!

Training on risk-based monitoring is now available from CDG! The course covers the details of the FDA guidance 'A Risk-Based Approach to Monitoring', recommendations on using it for device studies, include a sample monitoring plan and template, and features QnA addressing issues unique to device manufacturers. The well-researched course is designed and presented by Dr. Nancy J Stark. Click here for more information or call 773-489-5706.

High risk sites and high risk data
A high-risk site is a site with a high probability of having a problem; the 'symptom' might be that they are enrolling at a slower or faster rate, experiencing a higher or lower rate of adverse events, or having a higher rate of non-compliances than other sites. For example, one site I worked with suddenly stopped enrolling subjects altogether. The sudden change in enrollment pattern should have triggered an on-site monitoring visit. We eventually learned the site had enrolled the same subjects in multiple trials and compromised everyone's data. The site stopped enrolling because they were being audited by other sponsors for fraud.

High-risk data might be a particular type of event or outcome that is a 'symptom' of a problem with the study. For example, a high rate of post-procedure embolism might indicate a poorly trained investigator or a problem with subject selection.

The guidance suggests that any site in the start-up phase of the study is a high-risk site and that on-site monitoring should be performed at every site in the study's early stages.

Device companies can adopt the same principles using low-cost methods. Have the sites fax or scan the case report forms to the monitor once a week for off-site monitoring (i.e. remote monitoring). With one trial and maybe 100 subjects, a monitor can easily examine the case report forms manually for high-risk sites or data without the need for statistical algorithms.

Critical data and processes
Critical data and critical processes are the types of data and processes most likely to be high risk. Critical data include: [1] evidence that the subject really exists, [2] evidence that subject has the disease or condition being studied, [3] evidence that IRB approval was obtained, [4] evidence that informed consent was signed before the intervention or procedure, [5] endpoint data that address the hypothesis, and [6] serious adverse device effect data. For each study you'll need to make a unique list of endpoint data and adverse device effect data that is critical.

Critical processes include [1] the process of obtaining informed consent, [2] the intervention or procedure, [3] timely reporting of regulatory non-compliances or study deviations, [4] timely correspondence with a data safety monitoring board (if one exists), and the like. Because studies vary so much from each other, you'll need to make a unique list of critical processes for each one.

Critical data and processes are most likely to be high risk and should be checked for closely during the off-site monitoring process.

CDG has monitors experienced in risk-based monitoring

Our outsourced monitors come in with the attitude of service, impermanence, and an enthusiasm for the new; they are accustomed to off-site monitoring and problem spotting. They also carry a breadth of experiences that can help solve the issues de jour. Click here to request a proposal for monitoring services from CDG or phone 773-489-5721.

Methods of monitoring
Next the guidance considers two different methods of monitoring; on-site monitoring and 'centralized monitoring', meaning the real-time analysis of data by an application running on a server in some data management center.

I propose that for device studies there are four methods of monitoring: [1] on-site monitoring, [2] off-site monitoring of faxed or scanned case report forms, [3] off-site monitoring of electronic ally captured data, and [4] off-site monitoring of source data that has been captured by the device (usually in vivo or in vitro diagnostic devices.) Often data card is changed out on-site and the data card analyzed off-site. Sometimes the data card may transmit data electronically via the internet—consider a digital camera used to take photographs of the procedure. In the future, of course, it may even be possible to remotely access electronic medical records.

The idea, then, is to use the most expensive method of monitoring (on-site) for sites or data or processes that are critical and high risk and to use less expensive methods of monitoring (off-site or remote) for sites and data that are less so. It is a welcome concept to device manufacturers who are cash-starved and looking for ways to economize.

Triggers
The idea of a trigger is that some server at a data management center somewhere is programmed to alert you if critical data or a critical process has become high-risk; i.e. has the value or frequency of a critical data element or critical process step changed in such a way as to demand attention. Indeed, if hundreds or thousands of sites are reporting data every day, you will need a program to screen the information for anomalies or outliers.

But since most device firms are doing only a few studies at a time, having the case report forms faxed or scanned to the monitor for remote viewing is an inexpensive alternative. The monitor checks the forms for, [1] completeness (no missing data), [2] contemporaneousness, [3] legibility, [4] logic, [5] adverse device effect reports. The forms can be checked for everything except source verification. The monitor issues and resolves queries for any issues observed. Most triggers are found by simple observation.

Then the case report forms are forwarded to the data center where the data are entered into a local (often non-networked) computer. Here the data may be statistically analyzed to see if a site, data element, procedure or process step is at risk and merits an unscheduled, on-site monitoring visit.

Monitoring Plan
A monitoring plan should be developed for every study. This requirement is not new to sponsors conducting IDE studies or European studies under ISO 14155, but the guidance makes clear recommendations for the format. I recommend you develop a template for a separate document that can be referenced in the protocol, and don't make the monitoring plan a part of the protocol itself.

Section One—Study Description
As a separate document, the first section describes the protocol, investigational device, purpose of the study, and other obvious reference information. You need to describe, [1] the monitoring approaches you will use for the study, [2] criteria for determining the timing, frequency, and intensity of planned monitoring activities, [3] specific activities required for each monitoring method employed during the study, including
reference to required tools, logs, or templates, [4] definitions of events or results that trigger changes in planned monitoring activities for a particular clinical investigator, and [5] identification of possible deviations or failures that would be critical to study integrity and how these are to be recorded and reported.

A second opinion is a phone call away

Sometimes to you want to discuss an issue, but implement the solution yourself. If you want a second opinion about clinical, regulatory, biological safety, or reimbursement strategies, sign us up as consultants. Email us here or call Nancy at 773-489-5721.

Section Two—Reporting
Section two of the monitoring plan should discuss the communication of monitoring results to management, review boards, and regulatory bodies. It should describe the format, content, timing, and archiving requirements for reports and other documentation. For example, if the monitor receives case report forms by fax or email, it makes sense to require, say, a weekly or monthly report. Section two should discuss the process for appropriate communication of the findings to management, review boards, and regulatory bodies.

Section Three—Noncompliances
Section three of the plan should discuss the management of noncompliance. How, and who, will follow up with investigators found to be non-compliant with the regulations, protocol, or IRB requirements? If non-compliances persist will the site be retrained, terminated from the study, or will some other action be taken? If protocol deviations are detected you need a plan for root cause analyses. Problem-solving has always been a part of a monitor's responsibilities in device trials and it may require an on-site visit to really assess the problem. I have had the root cause to be such simple things as the data transcriber needing spectacles, a three-hole punch, or a dedicated phone line. Eyes-on may be the only way to detect such problems so they can be solved.

Section Four—Training
Section four of the monitoring plan should describe specific training for monitors and internal data auditors, especially with the detection of triggers and recommendations for on-site monitoring visits. For device studies, I recommend section four also discuss training plans for investigative site personnel. This section should also describe plans for random quality audits.

Section Five—Amendments to Monitoring Plan
Section five describes the process for amending monitoring plans. What events may require review and revision of the monitoring plan and establish processes to permit timely updates when necessary?

Much of sections Two to Five can be incorporated into the clinical research quality management system. By developing procedures for off-site monitoring you won't have to repeat them in every monitoring plan.

References
[1] 'Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring' FDA, August 2011. http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf
[2] 'Guideline for the Monitoring of Clinical Investigations' FDA, January 1988.
[3] 'Quality Management in Clinical Trials', March 2009, Clinical case series from Pfizer.

Literature Evaluations for Biological Safety

A Whitepaper by Dr. Nancy J Stark

During the course of product development there are three times when you should do a literature evaluation: 1) to justify animal use, 2) to justify a clinical trial design, and 3) to apply for certification and CE Mark or Medicare reimbursement. I've written previous whitepapers about doing literature evaluations as a part of clinical evaluations. In this whitepaper I'll do a brief overview and then look at literature evaluations to justify animal use.

Three literature evaluations
Selective Clinical Evaluations
ISO 14155 requires a clinical evaluation to justify the design of a clinical trial. 0. ISO 14155. A European colleague showed me several literature evaluations that were done for this purpose. The most important point I noted was that they focused only on the selected indication being investigated, and resulted in a relatively short, compact literature review utilizing only 5-10 publications.

 

Comprehensive clinical evaluations
The literature evaluations intended for certification and CE Mark are supposed to be comprehensive, "The clinical evaluation is based on a comprehensive analysis of available pre- and post-market clinical data relevant to the intended use of the device in question…." 1. MEDDEV Clause 5.5.1. However, CDG has been successful with Notified Bodies by averaging ~40 articles per review for Class IIa and IIb devices. The number of articles reviewed depends on the number available, the quality of the articles, the age of the technology, the volume of competition, and other similar factors. I observed one European Class III device that got certification and CE Mark based on a review of only 9 case series and 7 publications.

Literature reviews for Medicare reimbursement application should be larger and will be more persuasive if they follow the examples found at www.ahrq.gov. The Agency for Healthcare Research and Quality (AHRQ) has been publishing technology assessment literature reviews for years. If you look at the technology assessment for negative pressure wound therapy (NPWT) devices, you'll see that 1078 citations were identified, 288 full-text articles retrieved, and 137 articles reviewed. There were also more than 1400 items submitted for review by independent "interested parties". The authors point out that none of the studies compared an NPWT system head-to-head to another system. 2. Negative Pressure.

Similarly, AHRQ recently released for comment a comparative effectiveness review of devices used in diagnosing cardiac disease in women, including stress electrocardiography, echocardiography, single proton emission computed tomography, cardiac magnetic resonance, and computed tomography angiography. A total of 7163 citations were identified, 1452 full-text articles retrieved, and 101 articles (98 studies) reviewed. The authors note that none of the studies were randomized controlled trials. 3. Diagnosing CAD. This review is currently open for public comment and device manufacturers should absolutely submit publications and data based on research they have done.

In 2010 AHRQ received $500,000,000 from the federal government to prepare comparative effectiveness reviews on a wide variety of products. These reviews will surely affect the availability of new and old technologies to the American patient as Medicare revisits its reimbursement decisions based on these reviews. The message to device manufacturers is that we will have to rethink how we design clinical trials and be prepared to submit our own clinical evaluations if we are to obtain reimbursement in the American marketplace.

Literature evaluations for biological safety
I haven't yet written about literature evaluations to justify the use of animals for biological safety testing or preclinical animal testing. The beauty is that the way you un-justify the use of animals is by demonstrating the work has already been done and does not bear repeating—saving you costly resourses and the unnecessary use of animals. Let's look more closely at literature evaluations to justify (or un-justify) animal tests.

How to Write A Clinical Evaluation—CD Workshop

A new CD workshop is now available on "How to Write a Clinical Evaluation". This well-researched presentation takes you step-by-step through the complicated process of writing a clinical evaluation by making it organized and understandable. The 100+ content slides and supporting handouts explain the logistics of clinical evaluations, how to define objectives and key questions, identify and retrieve abstracts, filter and screen abstracts to identify articles, filter again to zone in on the articles that are applicable, assess and weight the literature, and write the evaluation to address the key questions originally posed. The presentation discusses the three major purposes for evaluations, discusses their regulatory foundation, and proposes a strategy to allow each evaluation to build upon the previous ones. Finally, some average metrics are disclosed.

If you want a learning guide to help you write a clinical or literature evaluation, this workshop is the place to turn. A procedure, protocol, and template—meeting the requirements of MEDDEV 2.7.1 r3—are available for a small, additional cost. Click here to purchase or more information.

Authority—who says so
Take out your current copy of ISO 10993-1 "Biological evaluation of medical devices—Part 1: Evaluation and testing within a risk management process" (2009) and turn to Annex C, "Suggested procedure for literature review." 4. ISO 10993-1.

The introduction tells us, "A review and evaluation of the literature is essential for justification and planning of any biological evaluation of a material or a medical device. The aim of such a review is to determine scientific background for the biological evaluation. It also provides essential information for assessing risks/benefits and achieving the ethical conduct of the planned evaluation as required by ISO 10993-2.

NOTE Such a literature review can be helpful to assess whether the relevant data available in the literature are sufficient to demonstrate biological safety of the device in question without the need to generate further data from actual testing or to conclude that the available data are not sufficient.

Performing a literature review is a scientific activity that should be done with rigor and objectivity, and
should allow verification by third parties."

Methodology
[1] Establish a plan
The methodology, taken from Annex C, sounds a lot like the methodology described by GHTF Study Group 5 and MEDDEV 2.7.1 r3. The first step is to establish a plan or protocol for the identification, selection, collation and review of all available studies and data. The plan should be documented in writing and based on recognized practices for the systematic review of the scientific literature.

[2] Define objectives or key questions
The objectives of the literature evaluation should be clearly defined up-front. Sometimes it is easier to phrase the objectives in the form of key questions, for example, which, if any, animal safety tests from the G95-1 Matrix should be performed? 5. FDA BlueBook G95-1. Does the chemical characterization of the materials justify not doing animal safety tests? Do literature data satisfactorily explain the negative outcomes of certain animal tests? Do literature data justify the design of preclinical animal tests, selection of species, route of exposure, duration of exposure or follow-up, or other design parameters?

[3] Document sources and selection criteria
The next step is to identify the sources of literature and other databases that will be searched. The selection will depend on what information you need. For example, you might need evidence regarding the safety of chemicals such as additives or adulterants, information about chemical properties such as glass transition points or vaporization, data about adverse effects related to the device or technology, published studies about in vivo animal use, or information about 'active water' concentration. The type of data you need will influence where you look for it.

You will probably need to contact suppliers directly (called manual searching) to ask for specific information. For example you might need, say, formulation or processing information that isn't published.

In addition, you'll need to specify criteria for the data. For example, you might accept data from rodents but not from fish, you might require data from certain bacterial strains, large mammals, or ex vivo studies. You might need data with a certain sensitivity, such as parts per billion rather than parts per million. I find it easier to specify what I need than what I don't need until I have a feeling for what kind of information is out there.

[4] Assessment of documents
The documents are then valued in such a way as to group them by low, moderate, and high quality, depending on pre-determined assessment criteria; and then the values for each criteria are summed to give the document a weight. It may be that some high quality documents are unfavorable to the objectives, but these documents must be included in the evaluation nonetheless.

A 'document' may be anything from a MAUDE report of an adverse event to a full-text published article to information provided directly by a supplier. Documents might be assigned a value on: 1) the similarity of the device in the document to the device under consideration, based on technology, critical performance, design, and principles of operation; 2) the relevance of the particular experimental animals used in the selected studies for the biological evaluation of the device under consideration; and 3) conditions of use of the material or device in the selected documents and the intended use of the device in question.

Documents might also be valued on, 4) whether the author's conclusions are substantiated by the data, 5) whether the document reflects the current medical practice and state of the art technology, 6) whether documents are taken from recognized scientific publications and whether or not they have been reported in peer reviewed journals, and 7) the extent to which the data were acquired through established scientific literature.

[5] Critical evaluation of the literature
Note that valuation means the act of setting the value of something, while evaluation means a diagnosis or diagnostic study based on those values. 6. Dictionary.com.

Applying these definitions to literature reviews, we can say that we assess a document through the process of assigning it a separate value for each of several criteria, we weight the document by adding up the values, and we evaluate the body of documents (the literature) by determining if they address the key questions, placing the most emphasis on the document with the highest weights. The idea is to evaluate high quality data whether or not it is favorable to our technology.

The hoped-for outcome is to conclude than no new animal testing is justified, but it may be that we identify that new animal testing is indeed needed.

[6] Format and content
The literature evaluation itself may follow the standard format and content of other literature reviews.

[a] A short description of the material and device, including its intended use. I like to include the European and US classification, if known.
[b] Discussion of whether or not the literature supports the objectives or key questions.
[c] Evaluation of the hazards, risks, and appropriate safety measures that should be taken by the manufacturer.
[d] Description of the method of document selection, reasons for omitting documents, and methods of weighting documents.
[e] List of documents (full citations) used in the evaluation and appropriate cross-references.
[f] Conclusion with a justification for whether additional animal testing should be performed or is not necessary. The conclusion should include a discussion of probable risks and benefits, considering the 'state of the art' of the technology. And the conclusion should identify any gaps in evidence regarding safety or performance that should be addressed by future testing.
[g] The printed names, signatures, and responsibilities of the reviewers, and the dates of signature. I recommend attaching a biosketch or curriculum vitae of the reviewers as evidence of their experience.

Conclusion
Literature evaluations for biocompatibility are used to determine the need for animal biological safety testing or preclinical testing. Documents both favorable and unfavorable to the technology should be reviewed, but only those documents of the greatest assessed value need be evaluated. Naturally, the hoped-for outcome is that no further testing is needed.

The more common situation, however, is for the manufacturer to do a bunch of tests and then turn to the literature to defend an unfavorable outcome. While I'm not a proponent of this strategy, it is wiser to turn to the literature to explain an unwanted test result than to do even more animal testing. If you have unfavorable animal test results, find a knowledgable medical device toxicologist to assess what is currently known about your material, technology, or device, and then do whatever testing is still needed.

References
0. ISO 14155 "Clinical investigation of medical devices for human subjects—good clinical practice" (2011).
1. MEDDEV 2.7.1 r3 "Clinical Evaluation: A guide for manufacturers and Notified Bodies," (2009); clause 5.5.1.
2. AHRQ "Negative Pressure Wound Therapy Devices", 2009, http://www.ahrq.gov/Clinic/ta/negpresswtd/references.htm#ref65.
3. AHRQ "Comparative Effectiveness of Noninvasive Technologies for the Diagnosis of Coronary Artery Disease in Women", 2011, http://effectivehealthcare.ahrq.gov/index.cfm/research-available-for-comment/comment-draft-reports/?pageaction=displaydraftcommentform&topicid=202&productid=770&documenttype=draftReport.
4. ISO 10993-1 "Biological evaluation of medical devices—Part 1: Evaluation and testing within a risk management process" (2009).
5. FDA BlueBook Memo G95-1, "Use of International Standard ISO-10993, 'Biological Evaluation of Medical Devices Part 1: Evaluation and Testing'" (1995).
6. Dictionary.com, http://dictionary.reference.com/browse/evaluation.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

Clinical Trial Agreements

A Whitepaper by Nancy J Stark

Clinical trial agreements (CTAs) are the legally binding agreements between a sponsor and an institution (site) as to how certain business and property rights will be dealt with between the parties. These agreements are separate from Investigator Agreements and Confidentiality Agreements and are not regulated by FDA or disclosable to FDA. CTAs are important because they allocate risk, responsibility, financial support, and obligations of the parties; and they protect the rights of the parties.

Usually the sponsor presents an initial draft of the agreement to the institution, and then each of the terms and conditions are negotiated by the party's attorneys. Agreements follow a typical format of: preamble, acknowledgements and responsibilities, term and termination, payment and reimbursement of costs, HIPAA and patient privacy, publication, data ownership and use, intellectual property, indemnification and insurance, subject injury, best efforts, waiver of consequential damages, miscellaneous (export controls, governing law, and dispute resolution/arbitration/mediatio, exhibits), and signatories. (1. Perry-Northwestern University)

 

The most common problem areas are publication, data ownership and use, intellectual property, indemnification and insurance, subject injury, best efforts, and signatories. In the text below I explore these problems and solutions suggested through various model CTA templates. (2. Weill Medical College of Cornell.) (3. UIDP) (4. MAGI) (5. Arizona template) (6. IOM template)

Conflicting missions
Universities, teaching hospitals, and other not-for-profit institutions are grounded in the principles of academic freedom and the discovery and sharing of information. Furthermore, in order to maintain a tax-exempt status, they must fulfill certain duties such as education of students, creating generalizable knowledge, or dissemination of knowledge. (7. Guiding Principles)

Sponsors, manufacturers, and other for-profit firms have commitments grounded in creating value for their shareholders, providing useful goods and services, and expanding the state of the art. They contribute to society by developing and providing new technologies to ease the medical conditions of mankind, but they must be profitable to survive.

Since neither party can live without the other, the challenge is to find contractual language that supports the mission of each within the constraints that limit them.

[F] Publication
The publication clause outlines who can publish what, and when. The clause is important to industry because they need to hold early developmental information confidential until the design of the device has matured into a functional, manufacturable, and marketable configuration. Revealing information too early, or revealing the wrong information, may allow your competitors to learn from your mistakes. Sponsors are also concerned about the accuracy of reporting, both of data and of product descriptions. And for multi-center studies, solutions are needed that allow for a complete manuscript incorporating results from all sites, and which pre-determines the order of authors.

Conversely, a non-profit institution needs to publish new knowledge. Publication creates recognition of the investigator's efforts, allows results to be used in future trials, enhances the site's prominence, contributes to public knowledge, and allows graduate students to use the data for their theses.

A typical industry draft clause might say, in essence, "Institution shall submit to Sponsor a copy of any proposed publication at least 90 days prior to submission for review and approval."

Compromise language suggested by the University of Arizona is, "Sponsor has no objection to publication by Institution of the results of the Study based on information collected or generated by Institution, whether or not the results are favorable to Sponsor. However, to ensure against inadvertent disclosure of Confidential Information or unprotected Inventions, Institution will provide Sponsor the opportunity to review any proposed publication or other type of disclosure at least thirty (30) days before it is submitted for publication or otherwise disclosed. If any patent action is required to protect Inventions, Institution agrees to delay the disclosure for a period not to exceed ninety (90) days from the date Institution initially submitted the proposed publication, or other type of disclosure, to Sponsor for review. Institution will, on request, remove any Confidential Information before disclosure.

If Study is part of a multi-center study, Institution agrees that the first publication is to be a joint publication covering all centers. However, if a joint manuscript has not been submitted for publication within twelve (12) months of completion or termination of Study at all participating sites, Institution is free to publish its results separately…." (5. Arizona template)

[G] Data ownership and use
Data ownership and use is another common sticking point. Sponsors may argue that they "own" the data because they have paid for it, but Institutions counter that they cannot retain their non-profit status if they engage in contract research or "work-for-hire". Institutions want to protect the right to use study data for non-commercial, research, development, and educational, purposes.

A typical industry draft clause might read, "All study data shall be the sole and exclusive property of Sponsor."

MAGI offers this compromise clause, "Site owns all Source Documents. Sponsor owns all Sponsor Data. Site owns all Specimens and Genetic Data, and grants Sponsor access to such property only for purposes of the Study. Sponsor and Site may freely use their own data subject to Subject consent and IRB approval. In addition, Site may freely use Sponsor Data after first multicenter Publication is published or Sponsor waives right to a multicenter Publication. Sponsor has no right to Site’s pre-existing biological samples, genomic database, or other proprietary database." Sponsor Data is defined as, "Study lab test results, CRFs and other reports completed by Site or Investigator per the Protocol, Agreement or other written instruction by Sponsor, excluding Source Documents and Genetic Data. Specifically excludes results derived from analysis of data." (4. MAGI)

ISO 14155 "Clinical investigation of medical devices in human subjects—
good clinical practice" (2011)

What makes this ISO 14155 training so special? The author served on the Editing Committee of Working Group 4, the inner circle of individuals who put finger-to-keyboard and wrote the standard. In this three-hour recorded training Dr. Stark discusses the standard in a topic-by-topic format; identifying what is new, what were the controversies, and what was the Working Group's thinking in adopting the language it did. It is this background information that helps you apply the standard appropriately to your study.

You can buy this newly released training on CD on our website cdginc@clinicaldevice.com or by phoning 773-489-5706. Private training from Dr. Stark is also available.

[H] Intellectual property
Intellectual property is a little different than data ownership, in that it covers who has the right to obtain a patent, or own an invention, development, or discovery.

A typical industry draft clause may read, "Any inventions made as a result of Institution's performance of this Agreement will be disclosed promptly to Sponsor and shall be deemed the property of Sponsor." But again, the sale of information is at cross-purposes with the non-profit mission of the institution.

The University of Arizona offers this compromise clause, "Inventorship of any inventions, developments, or discoveries, whether patentable or not (collectively referred to as "Intellectual Property"), resulting from the performance of the Study, shall be allocated according to U.S. Patent law (Title 35 U.S. Code) in effect at the time the Intellectual Property was created…." (5. Arizona template)

[I] Indemnification and insurance
Indemnification is language whereby one party agrees to protect another against an anticipated loss or damage; insurance is a policy through a third-party which guarantees payment to cover the costs of loss or damage. Although FDA does not require sponsors to carry insurance to cover the possibility of loss or damage to the Institution resulting from the study, most Institutions will have insurance requirements for sponsor-initiated clinical trials. In the event a sponsor in unable to meet the insurance requirement, the study will probably undergo a clinical risk assessment via which the institution attempts to assess the likelihood of loss or damage.

A typical industry draft clause might read, "Sponsor shall indemnify and hold harmless Institution from any damages and liability that are caused by Sponsor's negligence."

A compromise clause might read, "Sponsor shall indemnify, defend, and hold harmless Institution from any damages and liability that arise out of the proper administration of the Study device, Study-required procedures, Sponsor's use of the results, or Sponsor's negligence or breach of the Agreement."

[J] Subject injury
While indemnification and insurance have to do with loss and damage to the Institution, the issue of subject injury has to do with loss and damage to the subject. Sponsors readily agree to provide subject injury coverage if it is determined the injury is related to the investigational device, but who gets to make the determination? Good clinical practice tells us the principal investigator makes this determination.

Some sponsors like to add a clause that they will not pay for injuries until insurance is billed and denied. But billing third-party payors, whether Medicare or private insurance, should be at the Institution's discretion. If the injury is related to the medical device there should be no billing.

Some sponsors are concerned with injuries that are, in fact, attributable to the underlying illness. A solution is to qualify underlying illness by saying, "to the natural progression of the underlying illness."

Some sponsors try to limit their liability if the injury is attributable to the negligence of the study subject or the failure of the study subject to follow the instructions of the Principal Investigator. But such thinking is contrary to current ethical standards. Subjects are donating the use and study of their bodies for other people's gain; there may be little or no benefit to them beyond the satisfaction of contributing to the world's knowledge. Either the institution, sponsor, or third-party payors should bear the cost of their injury regardless of it origins.

IOM's suggested language is, "The Sponsor shall reimburse actual and reasonable medical expenses incurred in treating any injury or illness to a Study Subject that is directly related to the administration of the Investigational device or the proper performance of any other procedure, each in accordance with the Protocol and the Sponsor’s written instructions to the Institution (or to the extent that the Sponsor’s written instructions conflict with the Protocol, the Sponsor’s written instructions to the Institution only). The Sponsor is not required under this Section to provide compensation for (a) other injury- or illness-related costs (such as lost wages), (b) medical expenses that are paid for by a third party (provided that neither the Institution nor the Study Subject shall be obligated to seek reimbursement from a third party insurer), (c) medical expenses that are incurred as the result of a violation of the Protocol or other misconduct or negligence, in each case by any agent or employee of the Institution (including the Study Staff), or (d) medical expenses for injury or illness unrelated to the Investigational Device and unrelated to the proper performance of any other procedure required by the Protocol or Sponsor’s written instructions to the Institution." (6. IOM template)

Arizona offers a simpler suggestion, "Should a subject suffer any adverse effect caused by the Study drug/device/biologic, the Study procedures, or laboratory work required by the Protocol, Sponsor will pay for all medical and hospital costs required for the diagnosis and treatment of such adverse effect." (5. Arizona template)

[K] Best efforts
Consultants (such as Clinical Device Group) know that research projects are dynamic. No wise Institution, consultant, or research contractor should agree to work under time, cost, and performance constraints. Deadlines and results cannot be guaranteed. Furthermore, work product is constrained by the laws and standards of the United States.

One might think the Sponsor is left defenseless, but this is not the case. Institutions will agree to a "best efforts" clause, a contractual provision which requires the institution to use its highest efforts to perform its obligations and to maximize the benefits to be received by the sponsor. Sponsor's are obligated to terminate a site's participation in a study if non-compliances to protocol or regulations are discovered. And sponsors always have the right to terminate a study for business or ethical reasons. Cost overruns, project delays, or non-performance (of investigator or device) are all legitimate business or ethical reasons for terminating a study or an institution's participation in a study.

[N] Signatories
Industry sponsors sometimes don't realize the employee-employer relationship between investigators and investigative sites and they propose to have the investigator sign the CTA. They're also concerned about the performance of the investigator because of their financial investment in the trial.

A typical industry suggestion for the signatory clause is, "This Agreement is entered into by and among Sponsor, Investigative Site, and Principal Investigator." But if you think about it, an investigator—as an employee of the institution—has no legal authorization to sign the CTA.

A compromise clause is, "This Agreement is entered into by and between Sponsor and Investigative Site. The Principal Investigator shall conduct the trial as an employee of the institution and not as a party to this Agreement." Then the Principal Investigator is asked to sign the Agreement to indicate their understanding and intention to comply.

Conclusion
Sponsors and Institutions both have a vested interest in performing clinical research, and each party has much to gain from the process. By thinking through the issues and appreciating the needs of the other party, it is possible to come to an Agreement that is fair, balanced, and meets everyone's needs.

References
1. My thanks to Sean Perry, JD, Sr. Contract and Grant Officer, Office for Sponsored Research, Northwestern University, Chicago for his insights into this topic.
2. The format is taken from Clinical Trial Agreements, Weill Medical College of Cornell University, Office of Clinical Trials Administration. Contact CDG for link.
3. University-Industry Demonstration Partnership (UIDP), National Academy of Sciences, http://sites.nationalacademies.org/PGA/uidp/PGA_060368.
4. MAGI Model Clinical Trial Agreement, https://www.magiworld.org/standards/.
5. Template for Clinical Trial Agreement, University of Arizona Board of Regents, http://www.orca.arizona.edu/ctas.html.
6. Template for Clinical Trial Agreements, Developed by Jim Snipes, Covington & Burling LLP for the Institute of Medicine, April 2009. Contact CDG for pdf copy.
7. Guiding Principles for University-Industry Endeavors, National Academy of Science and the National Council of University Research Administrators and the Industrial Research Institute, April 2006.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

 

 

Registry Studies: Why and How

There is only one difference between registry studies and clinical studies: registry studies are observational and clinical studies are investigational. (When clinical studies are randomized they are called randomized clinical studies or RCTs.) To put it another way, in a registry study we tell the physician to treat the condition however they want—as sponsors, we are passive observers; in a clinical study we instruct the investigator to treat the condition in a certain manner—we are active researchers.

There is another, inevitable, feature of registry studies that I want to point out. The words "effectiveness" and "efficacy" are often misused, even by FDA. Effectiveness refers to how well a device performs as intended in the general population of patients and the general chaos of clinical practice. Effectiveness is measured in registry studies. Efficacy refers to how well a device performs in a setting of carefully selected patients and a carefully controlled protocol. Efficacy is measured in clinical studies.

 

Why do a registry study?
[1] Reimbursement data
One common reason for doing a registry study is to obtain data for reimbursement purposes. While CMS prefers comparative effectiveness data obtained from randomized clinical trials, such studies aren't always possible. 1. Definition of Comparative Effectiveness. There may not be a direct comparator for your new technology: the comparators might be an office procedure versus a surgical procedure or a device versus a drug. Imagine the complexities of comparing a device that emits a magnetic field intended to lower the viral load of patients with AIDS or hepatitis C to a multi-drug regimen. Or co-pay policies may be different for the comparators. Martin et. al. described an issue where a new (possibly more effective) drug costing $2000 per month was to be compared to an older drug being used off-label and costing $50 per month. There was concern that patients assigned to receive the expensive drug would drop out of the study. 2. Martin, et. al. In such a case, the sponsor would pick up the copay for every subject, no matter what their third-party coverage might be, in order to level the playing field.

[2] Post-approval effectiveness publications
Before adopting your technology most clinicians will ask about its effectiveness in the real world. Registry studies are ideal for obtaining effectiveness data. By allowing wide patient selection criteria you will include patients with multiple confounding complications, wide age ranges, various socioeconomic backgrounds, and differing healthcare attitudes. Learning how your technology behaves in these complex scenarios can provide valuable information to clinicians and important data for publications.

[3] Section 522
In certain circumstances FDA may require a post-approval study under Section 522 of the Food, Drug, and Cosmetic Act. The so-called Section 522 Postmarket Surveillance authority is limited to Class II or Class III devices the failure of which might lead to serious adverse health consequences, devices implanted for more than one year, life-sustaining or life-supporting devices used outside a device user facility, or devices used in pediatric populations. 3. Section 522. Registry studies are an ideal way to collect the broad surveillance data required.

[4] Off-label uses
Devices aren't always used the way we think they will be or in the populations we anticipate. In registry studies we—as sponsors—are not dictating how our device will be used, so there is always the possibility it will be used off label. Off-label use in a registry study is not a protocol violation since we don't specify in the protocol how to use the device in the first place. A review of how our device is actually used in real-world practice can provide valuable marketing information, hypothesis development for future studies, and indications for use development for future regulatory submissions.

 

How to do a registry study
Implementing a registry study is not much different than implementing a clinical study. All the basic elements of design, planning, and project management are present. There is a lack of consensus standards for registry studies so it is difficult to find guidance on how to do them. Your primary resource for information will be "Registries for Evaluating Patient Outcomes: A User's Guide, Second Edition" from the Agency for Healthcare Research and Quality. 4. Registries.

[1] Planning
In the planning phase, identify your stakeholders, the scope of data required, define the core data set (what do you need to know?), identify the patient outcomes or endpoints, define the target population (i.e. inclusion and exclusion criteria), and most importantly, get your funding. Funding may come from top management, venture capital firms, government grants, private grants, or other resources, but in the end we are all accountable to someone. Next you'll set up the registry team, determine if safety monitoring boards, IRBs, or other committees are necessary, and finally plan an exit strategy so you'll know when the study is completed.

[2] Registry design
In the design phase, the details of the registry study are worked out and a protocol is written. There are only a few options for study design:

a) Cohort designs follow over time a group of people who possess a characteristic to see if they develop a particular endpoint or outcome. 5. Registries, p38.

b) In case-control designs you gather 'cases' of patients who have a particular outcome or who have had a particular adverse event and 'controls' who have not, and then you look backwards to see what proportion had an exposure or characteristic of interest. For example, in the evaluation of re-stenosis after coronary angioplasty in patients with end-stage renal disease, investigators found both cases and controls from an existing PTCA registry. Alternatively, cases could come from the PTCA registry and controls from outside the registry (say, Medicare data). 5. Registries, p38 and p46.

For example, in 2004 Cordis began a registry designed to assess stenting outcomes in relation to the outcomes of their SAPPHIRE trial, which was used as the historic comparison group. The research question was to see if non-academic physicians would achieve the same level of success as the academic investigators used in the clinical study. The registry was conducted because of concerns by FDA and the Centers for Medicare and Medicaid Services (CMS), and involved 74 sites and 1493 patients. The large number of sites and subjects are characteristic of registry studies. 5. Registries, p38.

c) A case-cohort design is a statistical variant of a case-control study. Controls are sampled from a list of people, with each person having an equal probability of being sampled. 5. Registries, p38.

Help with Registry Studies

CDG is pleased to offer a five-hour workshop on designing and implementing registry studies for medical devices. Designed and recorded by Dr. Nancy J Stark, the workshop is a focused presentation of the AHRQ User Guide, adapted to medical device registries. You can find more information on our website. Scroll down to Registry Studies for Medical Devices.

If you're short on human resources and need to outsource the planning and implementation of a device registry study, please phone or email us at 773-489-5721 or cdginc@clinicaldevice.com. Dr. Stark will be happy to discuss a proposal.

[3] Selecting subjects and comparison groups
The target population is all the patients with a common disease or condition or a common exposure. For example, the target population might be all people with cataracts, all women with urinary incontinence, or all people who have been exposed to radiation for cancer treatment. Then broad inclusion/exclusion criteria are used to select a representative population of patients. One common feature of registries is that they have few inclusion and exclusion criteria, thus enhancing their applicability to broader populations.

Selecting comparison groups is more critical in observational studies than in clinical studies, because subjects have a choice as to which intervention they receive. The sickest patients may choose your technology, while less-ill patients may choose the comparator. The result will be an unfair imbalance in adverse events for the new technology. Key demographic factors—such as age, lifestyle, and disease advancement—are collected and statistically applied to help achieve equipoise.

Comparison groups may be "internal" (data collected simultaneously), "external" (data were collected outside of the registry, such as Medicare data), or "historical" (data collected under the registry protocol but not simultaneously). Comparison groups are essential when you want to distinguish between alternative procedures, assess the magnitude of differences, or determine the strength of associations between groups.

Registries do not need comparison groups when the purpose is to characterize the "natural history" of an intervention.

[4] What data should be collected?
Registry enthusiasts have their own language for many of the concepts we are already familiar with from RCTs. For example, they talk about "domains" of data, and by that they mean data should be collected from the personal domain (patient demographics, medical history, health status, and patient identifiers), the exposure domain (patient's experience with the technology or device), and the outcomes domain (primary endpoints, secondary endpoints, adverse events, and technology deficiencies.) In addition, you should collect information about potential confounders (say a drug being taken to treat the same condition as the study device). The collected data should relate directly to the purpose of the registry.

"Data elements" refers to the exact data that will be collected. Currently there are few, if any, broadly accepted sets of standard data elements for most disease areas, making it difficult to use external data as a source of comparison data. Look to the specialty societies to see if they have created clinical data standards that you can use as a guide for selecting data for collection. For example, the American College of Cardiology has created clinical data standards for acute coronary syndromes, heart failure, and atrial fibrillation. 5. Registries, p53. Whenever possible, tie your data elements to established terminology, such as Current Procedural Terminology (CPT) codes, International Classification of Disease (ICD-10), or events related to device deficiencies. 6. ISO 19218-1.

[5] Data Sources for Registries
Depending on the data sources required, registries may utilize certain personal identifiers for patients to locate the specific patients and link the data. For example, Social Security numbers (SSN), as well as a combination of other personal identifiers, can be utilized to identify individuals in the National Death Index (NDI). What peaks my interest is that data may come from many different sources: outpatient clinic records, inpatient hospital records, laboratory records, billing records, and even payer claims data! Data may come from medical chart abstraction, electronic medical records, institutional or organizational databases, administrative databases, death and birth records, census databases, or existing registry databases. For example, if you are developing a thermoebolization technology for treating liver cancer, you may want to access data from the Registry of Liver Diseases.

[6] Ethics, Data Ownership, and Privacy
The principles of ethics, data ownership and privacy are the same for registry studies as they are for clinical studies. You need IRB approval to conduct the study, HIPAA waiver to access patient medical records, a financial agreement with the institution regarding payments, data ownership and publication rights, and assurances of patient privacy.

Consider the case study of the National Oncologic PET Registry, a registry developed to collect data about PET scans in cancer management with the goal of obtaining expanded CMS coverage for PET scans. The registry was to be conducted at hundreds of hospitals and free-standing PET facilities. The sponsor's believed the registry was not subject to IRB approval because it was being "conducted by or subject to the approval of Department or Agency heads" for the purpose of evaluating a "public benefits or services program." CMS agreed. One week before starting operation the Office of Human Research Protections (OHRP) issued a letter of disagreement. The study was put on hold while the sponsors contemplated the difficulty of obtaining approval from hundreds of IRBs. Ultimately OHRP conceded that only the registry was engaged in research and study needed to be approved by only a single IRB. 5. Registries, p84.

[7] Recruitment
Recruitment of sites becomes a major issue in studies the breadth of registries. Sites must be paid fair-market value for their time and must see a benefit to their operations if they are to join and actively participate in a registry. This is especially true if the registry study is to include community physicians or high-volume specialty centers, as well as academic centers. Community physicians are more likely to participate if the registry is viewed as a scientific endeavor, is endorsed by leading organizations, led by a respected opinion-leader, provides useful self-assessment data to the physician, or helps meet other physician needs such as maintenance of certification, credentialing, or pay-for-performance programs.

Patient recruitment presents the same challenges as clinical studies. The best success comes from recruitment by the patient's own physician. It also helps to communicate that registry participation may help improve care for future patients, to provide written materials in language easily understood by the lay public, keep survey forms short and simple, and provide incentives such as newsletters, reports, and modest monetary compensation.

[8] Data collection and quality assurance
Three sets of documents, together, form the system for data collection. The first is the case report forms, be they paper or electronic. These are the forms whereby data is gathered in the field, entered into coded fields, and transmitted to a data management center. The second is a data dictionary which contains a detailed description of each variable used in the registry. For example, the question may be: "Do you smoke?" And smoking may be defined has having smoked tobacco within the last year. The third is the set of data validation rules. These are logical checks on data entered to look for inconsistencies such as males taking birth control pills.

A data management manual should be developed to define how missing data will be handled, how invalid entries will be handled, how data will be cleaned, and what level of error will be accepted. The manual should describe how data will be tracked and coded, how query reports will be generated and resolved, and how it will be stored and secured. Finally, the data management manual should describe a quality assurance system for data entry and registry procedures.

[9] Adverse event reporting
For device and device procedure registries, adverse event detection, collection, and reporting is the same as adverse event reporting for any other post-approval setting. It begins with the "becoming aware" principle; i.e. the clock for reporting adverse events starts at the moment the investigator becomes aware of symptoms or events reported by the patient or signs such as out-of-range laboratory results reported by a lab, or from the moment the manufacturer learns of an event from an investigator.

Investigators are responsible to report serious injuries to manufacturers within 10 days and to FDA within 10 days if the manufacturer is not known. Investigators are responsible to report deaths to both the manufacturer and FDA within 10 days. 7. 21 CFR 803. Interestingly, if an adverse event occurs with a comparator device the investigator must report the event to the comparator's manufacturer. Manufacturers have 30 days to report deaths, serious injuries and malfunctions to FDA, and 5 days to report events that require remedial action to prevent an unreasonable risk of substantial harm to the public health. Events are logged into the Manufacturer and User Facility Device Experience Database (MAUDE).

[10] Analysis and Interpretation
Statistical analysis of registry data is no different than statistical analysis of clinical data. There are a couple of points that deserve mentioning, though. First, you'll need to determine how closely the actual study population represents the target population. Second, there should exist a statistical analysis plan for how the data are to be analyzed and interpreted. And third, there should exist a plan for how to handle missing data.

Conclusion
Don't be misled, registry studies are not cheap, second-rate clinical studies. They are easily as complex and costly than the exalted RCT. What they are is different. They are observational studies that asses a technology's ability to achieve its intended use in the real world. They are used when alternative technologies don't exist, are outdated, or perhaps unethical.

References
1. Draft definition, prioritization Criteria, and Strategic Framework for Public Comment.
2. Identifying and Eliminating the Roadblocks to Comparative-Effectiveness Research, Martin et.al., New England Journal of Medicine, June 2, 2010.
3. Food, Drug, and Cosmetic Act, Section 522 Postmarket Surveillance.
4. Registries for Evaluating Patient Outcomes: A User's Guide, Second Edition. Agency for Healthcare Research and Quality, 2010.
5. Registries for Evaluating Patient Outcomes: A User's Guide, Draft. Agency for Healthcare Research and Quality, 2006.
6. ISO/DTS 19218-1 Medical devices—Hierarchical coding for adverse events—Event type codes (2010).
7. 21 CFR 803 Medical Device Reporting.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Medical Device Biocompatibility

A Whitepaper by Dr. Nancy J Stark and Dr. Dan McLain

There are commonly three scenarios in which manufacturers call on CDG for biocompatibility help: a) FDA has raised questions about the safety tests you performed, b) you disagree that you need sensitization, genotoxicity, or carcinogenicity tests, or c) you have a new device and don't have a clue as to what tests to do. Can an expert opinion help?

Materials and the manufacturing process
We usually think about biological safety when selecting the materials for a new device. Many materials may be under consideration for a particular component. The mechanical properties, manufacturability, availability, aesthetics cost, as well as biological safety issues, are all considerations in the material selection process.

Often manufacturers believe they are using "common" material in their medical devices, but when challenged by FDA to give reasonable assurance of the material's safety they cannot offer specifics. You must establish the material is comparable after your manufacturing process. This is why biological safety tests are performed on final manufactured product. Some companies use final manufactured devices, others run a coupon of the material through the manufacturing process to get a material that is representative of the final process.

Back in the day—when one of us started working in the device industry—there were no recognized standards for device biological safety testing. Manufacturers floundered around looking for tests that seemed applicable. I once used US Pharmacopoeia (USP) tests for drug containers when developing a new syringe. Even today, suppliers and manufacturers sometimes assume that materials which meet USP Class V or VI requirements are sufficiently safe for device applications. (1 USP)

 

 

Step Two: "The Matrix"
Because of its long history, most manufacturers base their biological safety testing on "The Matrix". In 1986, regulators from FDA, the UK, and Canada published the Tripartite Biocompatibility Guidance. The guidance built upon standards that had previously been published by AAMI, ANSI, ASTM, and others. It looked at materials based on the type of tissue the material contacted and the duration of that contact. Then a testing category was recommended for each combination of contact and duration. The problem with the Tripartite Guidance was that nearly every category of biocompatibility test was recommended for nearly every combination of material contact and duration (198 out of a possible 240 categories were recommended.) (2 Tripartite) There were other problems too: it didn't tell you how to handle whole devices, and it didn't apply to metals or ceramics.

In 1992, the International Standards Organization (ISO) published ISO 10993 Part 1 "Guidance on selection of tests". The standard was part of a series of biological safety standards, but Part 1 included a matrix similar to the contact and duration matrix of the Tripartite Guidance. The major difference was that far fewer categories of tests were recommended (117 out of 240). (3 ISO 19003-1 Selection of Tests) This matrix was not accepted by FDA.

  

1995 brought us The Matrix as American manufacturers know it today. It is found in FDA's General Program Memorandum G95-1. (4 G95-1) With clairvoyance and foresight FDA modified the matrix presented in a version of the ISO 10993-1 standard that would not be published for another two years. (5 ISO 10993-1 Testing and Evaluation) The goal was to harmonize the agency's biological safety policy with the expectations of the international community. In the modified matrix, an additional 36 categories of tests are "suggested" for manufacturers to consider (for a total of 153 out of 240).

Step One: ISO 10993 Part 18 'Chemical characterization of materials' (2005)
A more ethical and efficient practice is to chemically characterize materials and compare them to materials in existing clinical use before performing animal tests. This is done in conformance to ISO 10993-18 'Chemical characterization of materials' (2005) and it should be done before the matrix is even consulted. (6 ISO 10993-18)

To put it in context, the chemical characterization of materials justify the performance or omission of animal biocompatibility tests, just as clinical evaluations justify the performance or omission of human clinical investigations.

  

Chemical characterization is the accepted method for comparing one finished, manufactured material to another in order to make the argument that they are clinically equal or that one material is no worse than a currently used material. The methodology has been developing over the years and was described in detail in a book by Dr. Stark in 2003. (7 Stark) The characterization is done in the following manner.

[1] Qualitative information
Describe the material and its intended purpose within the device. The description includes qualitative information such as supplier name, common name, chemical name, batch or lot number, visual and physical characteristics such as color, hardness, flexibility, and chemical family, and other specifications. The intended use of the material (type of tissue contact, duration of contact, and function) are also described.

[2] Material equivalence
The new material is compared to an existing material that is used in a device with the same clinical exposure; the device can be a commercial device from your own firm or it can be from your competitor. You can readily visualize a spreadsheet or database listing each material in the device and comparing it to another known material. If you have sufficient information for an expert to make a toxicological risk assessment you can stop here.

$$$$ Offer for Services $$$$

Expert Opinions and Consultation for Biological Safety

Dr. Dan McLain, is an internationally renowned medical device toxicologist. His position as convener of ISO 10993-11 is evidence of his global stature. Dan is available to assist with pre-clinical strategies and chemical risk assessments for your materials and devices. Our style is to work collaboratively with a point-person on your side so that you are involved in the process every step of the way. Phone or email us at 773-489-5721 or cdginc@clinicaldevice.com.

 

[3] Quantitative data
Quantitative data will be needed if a sound risk assessment cannot be made purely on qualitative information. This means a thorough knowledge of the chemical composition, additives, manufacturing residues, and leachables must be obtained through analytical and physical chemistry. The goal is to determine the amount of identified chemicals (such as monomers, anions, radioactive elements, and the like) that are present in the final, manufactured material.

[4] Quantitative risk assessment
In the quantitative risk assessment the amount (in µg, Gy for radiation, or other unit of exposure) of identified chemicals are compared to existing toxicological information, rather than to another material. The objective is to determine which, if any, of the chemicals in the material might be harmful at the calculated exposure.

[5] Estimate clinical exposure
Finally the amount of potentially harmful chemicals, the 'dose', is compared to the clinical dose a patient might receive in a lifetime, a procedure, or other unit of time. Such a comparison seeks to show that the maximum dose of a chemical one might receive is acceptable in light of the lowest dose that could case medical problems. Detailed methods for risk assessments are discussed in ISO 14155-17 'Methods for the establishment of allowable limits for leachable substances' (2008). (8 ISO 10993-17)

 $$$$ Advertisement $$$$

"The Matrix" Is Available as a Mousepad/Notepad

CDG's talented graphic artists have reconfigured The Matrix and printed it on a high-quality mouse-grade paper pad. Now you can always have The Matrix at your workstation, in plain sight where you can find it. You can quickly look up the test categories needed while the boss is quizzing you on the phone. And at ten sheets to the pad, you can take notes on the conversation (or doodle), tear off the sheet, and keep it safe in the technical file.

Find it here, or contact Peggy at 773-489-5706 or cdginc@clinicaldevice.com.

Consider these examples, they are intended to illustrate the kind of issue you'll be addressing: a) Let's say you have a metallic implant that you clean by washing with perchloroethylene. Known to be highly volatile, but also highly toxic, you must ask if enough perchloroethylene remains on the metal implant to preclude its use as a cleaning agent; b) Let's say you make a wound dressing that contains a potassium salt. You know that potassium affects cardiac function and in sufficiently high concentrations could put a patient into cardiac arrest. You must ask if sufficient potassium can leach out of the dressing and into the wound to affect cardiac function.

References
(1 USP) United States Pharmacopeia.
(2 Tripartite) Tripartite Biocompatibility Guidance G87-1, (1987).
(3 ISO 10993-1 Selection of tests) ISO 10993-1 Biological evaluation of medical devices: Selection of tests (1992).
(4 G95-1) "Use of International Standard ISO-10993-1 Biological Evaluation of Medical Devices: Evaluation and Testing", G95-1 (1995).
(5 ISO 10993-1 Evaluation and Testing) ISO 10993-1 Biological evaluation of medical devices: Evaluation and testing (1997).
(6 ISO 10993-18) ISO 10993-18 Biological evaluation of medical devices: Chemical characterization of materials (2005).
(7 Stark) "Biocompatibility Testing & Management" Fourth Edition, Nancy J Stark, Clinical Device Group Inc, 2003.
(8 ISO 10993-17) ISO 10993-17 Biological evaluation of medical devices: Methods for the establishment of allowable limits for leachable substances (2008).

Questions or Comments?
Do you have questions, comments, or additional information? Please post them below.