Why Do Non-Inferiority Clinical Studies?

A Whitepaper by NJ Stark and e-Conference by Dr. Robert Thiel Learn more Wednesday, 7 July 2010, 11 AM Central; or here OnDemand soon after. Reading someone's elses copy? Opt-in for whitepapers to get your own. I read an interesting comment in a press release this morning. The CEO of Spectranetics said: "Our physician customers prefer randomized, controlled studies to assist them with their clinical decisions...." and "Our clinical trial program ... demonstrates our commitment to evidence-based medicine." Every book or guidance document I read has a different way of categorizing clinical studies. One obvious categorization for medical devices is by whether or not you are doing a single-group (single-arm) study with no comparator, or whether or not you will compare your device to another. My experience is that most device studies are single-arm designs so I read the Spectranetics press release with a loud hurrah. A poll—what's everybody else doing? What is the most common category of clinical trial your company conducted last year? [a] Single arm. [b] Superiority. [c] Equivalence. [d] Non-inferiority. Answer poll 17 to compare yourself to others. Be the 100th person to answer the poll and win a free Adverse Event note pad. Non-inferiority studies use smaller sample sizes In any comparative study you want to show whether your investigational device is better than, the same as, or no worse than the competitor. If you will be filing a 510k, you might think that you need to be the same as the comparator in order to support the argument of substantial equivalence. It makes sense. The difficulty arises when you calculate the sample size. Sample sizes are calculated based on the difference between the average values of an endpoint. The greater the difference between your device and mine, the fewer the number of subjects you'll need to show it. As the difference between your device and mine gets smaller, the sample size gets larger and larger. As the difference becomes infinitesimally small, i.e., the devices become equivalent, the sample size skyrockets to infinity. Enter non-inferiority. Instead of showing that two devices are the same, a clever FDA statistician named Blackwelder looked at things differently and developed a strategy called "Proving the Null Hypothesis". He figured that if we stopped trying to prove the hypothesis and reject the null, and instead rejected the hypothesis and proved the null, we could accomplish the same thing as an equivalence trial but with a much smaller sample size. Blackwelder's ground-breaking paper was published in 1982. FDA's new guidance document In point of fact, device manufacturers have been doing non-inferiority studies for years; FDA kindly points out the opportunity every time a manufacturer proposes implementing a superiority trial. It is just as unethical to do an unnecessarily large study as it is too do too small a study. Recently, FDA issued a draft guidance document titled "Guidance for Industry - Non-Inferiority Clinical Trials" (Mar2010). The guidance was issued by CDER and CBER because non-inferiority trials require active controls rather than placebos—something new for the drug world. We device people are familiar and comfortable with active controls (who would use a placebo heart valve?), and there is much a good biostatistician like Robert Thiel can teach us with regard to medical devices and diagnostics. What device companies already know The guidance document points out some important "issues" in implementing non-inferiority studies. If you've designed non-inferiority studies before, you have probably already encountered them. [1] You have to know the comparator's performance for your 'pivotal' endpoint. You might learn this from the literature, benchtop data, animal data, or even data gathered at your own expense on the comparator device. [2] You have to judge the investigational device's performance for the 'pivotal' endpoint based on data such as benchtop data, animal data, or data from similar, commercial devices. [3] Non-inferiority really means "not too inferior", and how inferior is too inferior must be determined. You should plan on plenty of pre-protocol discussion over this topic. [4] The study design must be able to detect a difference if there is one. [5] The need to show a lack of difference can lead to a lack of incentive to study excellence, i.e., sloppiness. I personally take issue with this statement, FDA, but I'll let it go.

Motivate Your Study Staff: Give an Adverse Event Note Pad/Mouse Pad The Adverse Event note pad is a graphical way for study site staff to categorize adverse events. Starting at the center, they choose the characteristics in a pie-wedge that best describe the event. The outer ring identifies the event by type. ISO definitions, accepted by FDA, are provided on the right. The tool helps study staff think their way through to the reporting requirements described in the protocol. Read more. Learn more: take the e-conference The objective of the e-conference is to learn more about the historical pitfalls in the design and execution of non-inferiority trials and how to avoid them. Addendum: Brief Comments on Missing Data Let's shift gears and speak for a minute about missing data. The clinical trial has not yet been conducted where there hasn't been at least some missing data: the subject can't remember it, the study staff forgot to record it, the dog ate it, and the statistician is leftout with it. There are three basic kinds of missing data: [1] Data may be 'missing completely at random', meaning the probability that an observation is missing is unrelated to its value. For example, if a subject missed a follow-up visit because of jury duty, the data would be missing at random and we could impute the missing values. [2] Data may be 'missing at random' but not completely at random. For example, people who are depressed might be less inclined to report their income than others. If we can account for the variable depression, we can impute the income value. [3] Data may be 'missing not at random'. For example, if we ask people to report their income, low income people are less likely to respond. The income data is definitely not missing at random, it is missing specifically for a particular group of people. We can't simply impute the value, an algorithm must be developed. A good study design has a plan (called a statistical analysis plan or SAP) that, among other things, determines beforehand how missing data will be identified and handled. Dr. Thiel will take a few final moments to discuss this important, but neglected, issue. Dr. Thiel will discuss [1] What resources are needed to design and execute a non-inferiority trial. [2] What the FDA expects to see for a trial of this type. [3] Design problems with active-control (non-inferiority) clinical trials. [4] Statistical issues associated with non-inferiority clinical trials. [5] How the results of non-inferiority clinical trials are interpreted. [6] The all-important margin of error. [7] An addendum: Handling missing values. You will receive [x] PowerPoint slides. [x] An interpretation of the guidance document. [x] Practical examples to learn from. [x] The expertise of a prominent biostatistician. [x] Chance for Q&A. [x] CEUs and certificate of attendance. Who should attend (no, you do not need to be a statistician) [x] Clinical research professionals who design clinical trials. [x] Regulatory professionals who report clinical data. [x] Statistical professionals who analyze clinical data. [x] Managers who sign-off on clinical trials. [x] Executives who are planning the future of their companies. Presenter Dr. Robert P. Thiel has degrees in physics, clinical counseling and psychometrics. He is an expert in the analysis of IVD data for the Medical Device Industry with more than 75 510K and 5 PMA approvals. He has published papers in the areas of free PSA, breast cancer, ovarian cancer, liver disease and multivariate analyses. His current interests lie in the area of bootstrap and permutation test methodology and the application of these methods to small sample trials and Bayesian analysis as applied to diagnostic clinical trials. Dr. Thiel can be contacted at rpthiel@thielstatcon.com. System requirements [x] Personal computer. [x] Internet Access. [x] Telephone. Date, time, registration The 90-minute e-conference will be presented on Wednesday, 7 July 2010, at 11:00 Central Time. Event materials will be distributed the day before the e-conference. Sign up at registration. Best Regards, Nancy J Stark, PhD President, Clinical Device Group Inc Click here to receive CDG news via our XML/RSS feed Follow us on Twitter