CDG e Conferences & Workshops

A Plan for EU Clinical Evaluation Reports

What Do Notified Bodies Want?

A Whitepaper by Nancy J Stark, PhD.

We've come a long way in our understanding of what Europe expects in a Clinical Evaluation Report. Where you start depends on where you are: 1) writing a CE Report for a new, pre-approval device, or 2) writing a CE report for a commercial device. In the first case, you are asking the question, "Do I need new clinical data for certification?" In the second case, you are bringing your documentation up-to-date.

Clinical evaluation reports for pre-certification devices
If you're still in the stage of designing your device, you begin by identifying its intended uses. Then you evaluate the existing clinical data to determine if it is sufficient to support safety and performance per the Essential Requirements of the MDD or AIMD, or if you need additional clinical data. If additional clinical data are needed you have the choice of revising the intended use or performing a clinical trial.

Once you believe the data are sufficient, you convince top management to grant a CE Mark and sign the Declaration of Conformity to your quality management system. Finally, you will discover the truth when the Notified Body comes to call.

Clinical evaluation reports for commercial devices
The more common situation is that your device is already on the market in Europe, and the Notified Body has informed you that its clinical evaluation report was due last March. He will give you a grace period, but you are expected to conform. You follow the same process of evaluating the existing data to determine if it is sufficient to support safety and performance for the intended uses. Should the report conclude that the existing data are insufficient you are in a difficult situation: you'll need to suspend sales until additional data are acquired.

Because the consequences of determining the data are not sufficient are so financially significant for your firm, and the consequences of determining the data are satisfactory are so financially favorable, it is wise if the report writers are as far removed from the device designers as possible.

Resources
To write the report you need a team of at least three people: 1) a medical writer, 2) an information specialist, and 3) a statistician.

In addition you need a written procedure for how to proceed, a report template (useful, but not required), and access to literature databases such as Medline and Embase. Medline is easy, it is operated by the US National Library of Medicine and is available to anyone with a browser. Medline provides access to 5400 worldwide journals in 39 languages dating from 1947 to the present. And it is free.

Embase, on the other hand, is not free. It provides access to an additional 2000 European medical journals, and so, is indispensable since you want to sell in Europe. But it is owned by Elsevier Publishing. You should be prepared to pay a hefty access fee ($7000 and up per year) or sub-contract the time from a third-party source.

A Three-Legged Stool
First leg
The first leg of a Clinical Evaluation Report is an evaluation of the existing literature data. The Global Harmonization Task Force (GHTF) Study Group 5 document "Clinical Evaluation" coupled with MEDDEV 2.7.1 Rev 3, which is slightly easier to read, are the guidances that tell us how to do the literature evaluation.

Here's what the medical writer—working together with you—will do:
[1] Define the key questions.
[2] Identify the databases to search (Medline, Embase, MAUDE, Cochrane).
[3] Define scope of search and search strategies using a qualified information specialist.
[4] Scan the abstracts to identify articles for review.
[5] Acquire full-texts of articles.
[6] Weight the articles for technical significance, discard anything with a low weight.
[7] Have a statistician weight the articles for statistical significance, discard anything with a low weight.
[8] Review only those articles with the highest weightings; maybe 5-25 articles per intended use.

The literature data are evaluated to determine if they support safety and performance of the device for its intended use. I like to prepare a separate literature review and attach it to the main CE Report as an annex. The literature review annex has several required elements including the search strings used by the information specialist, a list of abstracts scanned, the articles reviewed, their weights and justification for those weights, and full-text copies of the articles.

Second Leg
The second leg of the CE Report is a review any existing clinical investigation that your firm may have sponsored. These data are reviewed separately from the literature because you have closer access to the details. The writer asks questions such as: "Was the Declaration of Helsinki followed?" or "Were adverse events resolved?" If not, those facts are noted in the report but the data are not used.

Once the writer is satisfied that ethical and administrative requirements were correct, he/she reviews the study results to see if they support safety and performance of the device for its intended use. I like to prepare a separate sponsored study review and attach it to the main CE Report as an annex.

Third leg
In the third leg, the medical writer comes to the most sensitive step of all; examining the complaint file against the risk management system. They review the file to see if complaint handling, as dictated by the risk management system, supports the safety and performance of the device for its intended use. I like to prepare a separate risk management review and attach it to the main CE Report as an annex.

The CE Report
Finally, the medical writer takes the literature review, sponsored study review, and risk management review altogether and reaches an overall conclusion of whether the existing data support the device's safety and performance for the intended use, or if additional clinical data are needed newly. You know what happens if new clinical data are needed.

The Technical File
If the data are sufficient, the CE Report is filed away in the Technical File, a written procedure describing the writer's strategy is incorporated into your quality management system, and you wait with baited breath until the inspector comes.

How long, how much
There are only two questions in top management's mind: 1) how long will it take, and 2) how much will it cost. The answers are simple: "It depends." Let's say you have a very nice ultrasonic toothbrush and you intend to use it for cleaning roadside burns—the kind of burn you get when your motorcycle crashes and your skin scrapes against the pavement. The toothbrush is used to remove the dirt, grit, and tarmac from the wound. It's an unusual intended use and raises all kinds of questions about potential damage from the bristles, safety features to prevent over-brushing, effects of ultrasound on open wounds, comparing gum tissue to wound tissue, instructions for use, and so on. A clinical evaluation report for cleaning wounds will take longer and cost more than a report for brushing teeth with the same device.

CDG can help
Clinical Device Group is poised and ready to help you with writing your reports. We have medical writers, information specialists, experienced statisticians, access to Embase, a written procedure, a standard template, and a proven process. CE Reports written by our specialists have been reviewed by at least three different Notified Bodies and all have been accepted. We work collaboratively with a point-person on your side so that you are involved in the process every step of the way.

Buffet approach
But perhaps you'd like to do your own medical writing in order to keep down costs, but don't have access to an information specialist, an experienced statistician, or Embase. Call us anyway, we like to work flexibly to meet your needs.

Contact us
Phone or email us at 773-489-5721 or [email protected].

A Standard Development Tale: ISO/FDIS 14155 (2010)

ISO/FDIS 14155 "Clinical investigation of medical devices in human subjects—
good clinical practices" (2010) 
A Whitepaper by Nancy J Stark, PhD 

ISO 14155 (2011) to become good clinical practice for medical devices
In this opinion-editorial I'd like to give some background on how the revisions to the 2003 ISO 14155 standards came about, why it took so long, what the economic implications may be, and what it's like to serve on a Working Group. These are my personal opinions and experiences, so please keep that in mind.

The medical device clinical investigation standard has had a varied and difficult past. To emphasize the point, here is the publication history as near as I can piece it together:
(1993) EN-540 "Clinical investigation of medical devices"
(1996) ISO 10993 "Biological testing of medical and dental materials and devices - Part 8: Clinical investigation".
(2003) ISO 14155 "Clinical investigation of medical devices for human subjects—Part 1: General requirements" and "Clinical investigation of medical devices for human subjects—Part 2: Clinical investigation plans."
(2011?) ISO 14155 "Clinical investigation of medical devices for human subjects—good clinical practices."

In 2003, the US voted yes to the revisions of the ISO 14155 (1996) standard. The yes vote did not come lightly. The 1996 standard was fluff, even for those days, and the 2003 standards were not a substantial improvement. The ICH-GCPs were adopted in 2000, FDA expectations were already higher than the 2003 standards, and even large firms such as 3M Company or Hollister Incorporated set higher expectations for themselves than the 2003 standards. Japan abstained from voting for ISO 14155-1 and voted against ISO 14155-2. The US reluctantly agreed to the 2003 standards with the idea of moving the action forward and on the condition that revisions begin immediately.

In January of 2003 the US began work on the contents of a New Work Item Proposal. The most important consideration was to harmonize the 2003 standards with the ICH-GCPs. But by April of that year the US backed away. There were competing political sensitivities between ISO (International Standards Organization), who had taken the lead for Part 1 with a Swiss convener; and CEN (Center for European Normalization), who had taken the lead for Part 2 with a British convener. The relationship between ISO and CEN had only recently been formalized in 2001 with the Vienna Agreement. A new Japanese law had passed, adopting the ICH-GCPs as their standard for medical device investigations in outright rejection of the ISO 14155 (2003) standards. And the US had declared war on Iraq, irritating much of Europe and weakening our international status in leadership roles.

Finally, the standard was originally a European initiative, pre-dating the 2001 Vienna Agreement between CEN and ISO. The original members of the CEN working group have always seemed a little surprised at the American and Asian interest in the standard and usually interpreted "international" to mean "pan-European"; eventually we started using the word "global" to make it clear that the standard would have world-wide consequences.

Possible changes to 2003 standards
Work resumed on a New Work Item Proposal (NWIP) in late 2004 with a list of 42 potential changes. Among the most important points proposed were:
[1] Merge Part 1 and Part 2 to avoid overlap and improve document flow.
[2] Harmonize the ISO standard with the ICH-GCPs.
[3] Harmonize the standard with MedDev guidances on clinical trials.
[4] Expand the principles of the standard.
[5] Clarify the responsibilities of investigator, sponsor, monitor, and IRB/EC more thoroughly and consistently.
[6] Clarify risk and risk assessment with regard to the subject rather then device design.
[7] Update the literature review section to match MedDev and any developing GHTF documents.
[8] Clarify the responsibilities of sponsors, monitors, investigators, institutions, IRBs/ECs, DSMB, auditors, and other parties.
[9] Clarify details of the investigator brochures, final report, adverse event handling, case report forms, and data management.
[10] Distinguish between adverse events and deviations.

NWIP approved, 2006
A New Item Proposal was finally introduced for voting in August 2006; it passed in December 2006 saying only that the ICH guidelines should be considered. Only the United Kingdom voted against revisions, leaving one to ask the question why they would oppose an improvement to the standard: there is always something to be gained from a position taken.

One part or two parts?
Work began in earnest in February 2005 in Berlin. The first item of discourse was whether to retain the standard in two parts or combine it into one; the consensus of the group was to combine the standard into one. By this time CEN had abandoned its working group and couldn't allocate funds to support development of a new standard so the question of who was to take the leadership role was easily resolved: ISO would take the lead and the British convener would step aside. The British did not attend another working group meeting until September 2009.

Project management approach
The working group began by dividing into sub-groups, each sub-group took on a major section of the standard and rewrote it to harmonize with ICH-GCPs and their own experiences of best practices. The result, when we met again in December 2005, was several well-written sections which lacked flow and integration. Someone complained that just looking at responsibilities didn't give a new firm a feel for how to go about getting started, and then someone else suggested modeling the standard after any project management approach: preparation, initiation, implementation, maintenance, and close-out; and then there could be sections on who is responsible for what.

Some countries argued that standards should not be teaching documents, but after much heated debate we agreed that teaching was important if we were to going to elevate the quality of device clinical investigations and gain respect for the standard itself. Whether teaching is a proper purpose of a standard or not would have to take second place to the more important goal of subject protection and data integrity.

Regulatory Authorities
Every country has it's own regulations and we needed to find a way to fit standard into a global structure. Many paragraphs are qualified with variations of the phrase "when requested by regulatory authority" as a way of building in flexibility. The phrase gets a little old, but its importance is indisputable.

Time or no time
The next debate was quickly resolved: should we mention time frames for reporting, say, adverse events. Our unanimous answer was no, because each country had it's own time frames and we didn't want to contradict them.

The same held true for EC/IRB responsibilities. We agreed to leave their job descriptions up to individual regulatory authorities.

Adverse event & effect definitions
Defining adverse events and effects turned out to be a fairly easy affair; we turned to the ICH-GCPs whenever we struggled and adapted the language to work for a medical device scenario. But there were wrinkles. The Japanese needed a definition for an "unanticipated, serious adverse effect". We Americans are so accustomed to having it left out that we didn't care, but certainly the standard meshes better with Part 812 by having a definition included.

Work stalls
By January 2006 we had a fairly complete standard. Then work stalled for reasons known only to the convener, the Secretariat, and ISO top management. No one with authority would call a meeting and from December of 2005 to July of 2008—18 months—there was no Working Group 4 meeting. The meeting in Chicago in July of 2008 was unproductive.

Unbeknownst to most Americans, Europe was in disarray. The admission of a number of Eastern European countries into the European Union left the New Approach in question. With 27 member states, reporting of adverse events, device deficiencies, and other post-market surveillance issues became unmanageable. Regulation of the medical device industry came within weeks of being under the control of the European Medicines Agency. It is my belief that the difficulties in Europe affected the progress of the standard.

First ISO/DIS issued in Sep08
In September 2008, despite the nearly unanimous recommendation of the Working Group, an ISO/DIS draft international standard was issued by the ISO Secretrariat. Most of the changes requested by the working group had been overlooked and the draft standard was eventually voted down by the Member States. 

New leadership—second ISO/DIS issued Sep09
No matter what the circumstances may be, when progress stalls it is the leadership that takes the hit. Working Group 4 is no different, and in September 2009 the group elected a new convener. The new convener, also Swiss, worked us hard for three days, not dismissing the meetings until 7 pm or until we had reached consensus. The work culminated in a second ISO/DIS being issued that same month.

Risk management and device deficiencies
The British rejoined the group for the first time since 2005. They brought with them a background in ISO 14971 "Application of risk management to medical devices" and were quite firm about including risk management language in the standard.

We also owe the reporting requirements for device deficiencies to the British members. This is one example of how the ISO standard differs from the ICH-GCPs; the standard needed to bring in the concept of devices that malfunction, deploy improperly, are misused, subject to use errors, and the like.

Adverse event categorization chart
One issue was not resolved: that of developing an adverse event categorization chart. A sub-committee, known as the Editing Committee, worked from January 2008 on to develop a chart, going through at least 15 different versions and never reaching consensus. Two versions of the chart, one from the British and another from the Japanese, were finally included in the final draft. It's interesting that we agreed on the definitions of the terms, we didn't agree on their relationship or if there was a hierarchy. The long discussions revealed basic differences in our understanding of how adverse events and effects are related to each other, and explain why definitions have always been do difficult.

The publication plan
FDIS stands for 'final draft international standard'. While the document is in the final draft stage it is being translated into French. Once this is complete the FDIS will be presented to the ISO members (Standards Bodies from each member country) for a final vote, but the only changes allowed at this stage will be spelling, punctuation, or grammar. After publication by ISO in 2011, each ISO member will be free to adopt the standard at the national level and publish it as a national adoption; so in the US the standard may be adopted as an ANSI/AAMI/ISO standard of the same number (ANSI is the American National Standards Institute, AAMI is the Association for the Advancement of Instrumentation.)

Oddly enough, an ISO member is free to modify the standard, within limits, before issuing it under its own designation. So buyer beware, the national adoption published in France may be different from the national adoption published in the UK, or the one published in the US!

If you were bored with the whitepaper, take the workshop anyway
If you were bored with this whitepaper, try the previous one on ISO 14155; and then take the workshop. The learning objective of this five-hour workshop is to review the changes from the old, 2003 standards to the new, 2010 FDIS. The recording is available by OnDemand or CD at http://www.clinicaldevice.com/mall/Workshops.aspx. 

You will receive, we will discuss
[x] PowerPoint slides.
[x] A 5 hour presentation.
[x] Hints for writing Clinical Evaluation Reports and their role in clinical investigations.
[x] Hints for writing Risk Analysis Reports and their role in clinical investigations.
[x] The required contents of clinical investigation plans (i.e. protocols).
[x] The required contents of investigator's brochures.
[x] Steps in planning clinical investigations.
[x] Steps in implementing clinical investigations.
[x] Steps in closing-out clinical investigations.
[x] The monitor's role in clinical investigations.
[x] Responsibilities of sponsors and investigators.
[x] A 30-minute quiz to reinforce your learning experience.
[x] CEUs and certificate of attendance.

Who should attend
[x] Clinical research professionals who estimate study costs.
[x] Project managers who want better planning and reporting from their group.
[x] CROs who want to improve their proposal skills.
[x] Executives who are planning the future of their company.

Presenter
Dr. Nancy J Stark has been a member of Working Group 4 since 2001 and was an active participant in shaping the coming ISO 14155 standard. She is President and Founder of Clinical Device Group, a CRO and consulting firm that has been in business since 1990. Her curriculum vitae can be found at www.nancystark.com.

System requirements
[x] Personal computer.
[x] Internet Access.
[x] Telephone.

Date, time, registration
The 5 hour workshop will be presented on Wednesday, 22 September 2010, at 11:00 Central Time. Event materials will be distributed the day before the workshop. Sign up at registration.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc