Avoiding Common Pitfalls at FDA Device Panel Meetings
The day of an FDA Device Panel Meeting is arguably one of the most important days in the regulatory pathway of a new device. Sponsors have only a few hours to clearly articulate years’ worth of research and development and to communicate the benefits of their new device. Too often, much of the time at an FDA Device Panel Meeting is spent on issues with the pivotal clinical study that could have been avoided with more careful planning at the design and conduct stages. These deliberations on study design and trial conduct detract from the key messages to support benefit/risk and may lower the credibility of the Sponsor’s research in the eyes of the panelists. Below are a list of the most recurrent pitfalls at Device Panel Meetings as well as solutions for how to handle these issues proactively so they don’t become problems during the review process:
Justification of study design. Randomized controlled trials are the gold standard of product evaluation and are the preferred trial design of FDA Device Panel members. The Sponsor and FDA may have agreed that less burdensome scientific evidence is adequate to provide reasonable assurance of safety and effectiveness. Sponsors should be prepared to justify single arm-trials, no-treatment control groups or historical controls as appropriate for their device. They should also weigh the potential benefit of using a more rigorous controlled trial not only to satisfy device panelists, but also for increasing market access and gaining reimbursement from payers after approval.
Inadequate representation of genders or ethnic/racial minorities. Frequently, panelists are concerned by the lack of generalizability of the pivotal study results due to a paucity of patients from a certain gender or ethnic/racial subgroup. While enrollment of certain sub-populations can be difficult, failure to do so will undoubtedly be an issue at the advisory panel. Of note, the need to ensure adequate enrollment of key demographic subgroups has now been highlighted in FDA”s June 2016 draft guidance “Evaluation and Reporting of Age, Race, and Ethnicity Data in Medical Device Clinical Studies.”
In some cases, the panel is explicitly asked to vote whether the data are generalizable, and whether to require additional post-marketing studies to ensure safety and efficacy in the sub-populations. Expanding the study population in pre-market trials can save money if it eliminates the need for an expanded post-market study. A more generalizable representation of subgroups can be attained by:
- Setting enrollment targets for sub-populations of interest that: 1) are in line with the demographics of the target patient population and 2) guarantee a sufficient number of patients are represented in each subgroup to support the generalizability of safety and effectiveness across subgroups.
- Ensuring clinical sites (or a subset thereof) are located in areas where recruitment of historically underrepresented groups is feasible.
- Using targeted advertising/incentives to enrich enrollment.
Failure to prepare for issues raised by FDA during the review process. Often, the Sponsor will dismiss a comment made by the FDA in the IDE review period, or believe that a response to a question raised during PMA review has adequately addressed the issue. Often, we have seen these issues resurface as panel questions because the FDA did not find the Sponsor’s response satisfactory. One opportunity to avoid this situation is to directly ask the FDA reviewers if there are any outstanding concerns regarding any of the questions or the Sponsor’s responses. If so, a call with FDA to better understand their concerns or communicate the response, can either close out the issue or provide better insight into how the question might be posed (and appropriately addressed) at panel.
Missing data. Considerable amounts of missing data lead to uncertainty in the Sponsor’s data, and can call into question the validity of the conclusions of the study. In recent years, the FDA and advisory panels have been raising their expectations for data completeness. While this is one of the most common pitfalls, it is also one of the most easily avoided. See a full list of suggestions for how to avoid missing data in another article, but below are some key points to integrate into the study plan to minimize missing data:
- Do not exit patients from the study if they do not receive the device or have the device removed/explanted prior to the completion of the primary endpoints. Early withdrawal of patients threatens randomization and the intention-to-treat principle of most analysis plans.
- Set benchmarks for study sites on the allowable amount of missing data, and continuously monitor to those targets.
- Offer incentives to sites and patients for excellent follow-up.
- Minimize unnecessary data collection to reduce the burden of study participation.
- Pre-specify reasonable sensitivity analyses for missing data that cover the clinically plausible range for missing values. Panelists often expect to see “tipping point” analyses, which do not rely on some of the assumptions of the model-based approaches to handling missing data.
Clinical meaningfulness of study endpoints. It’s surprising how many times we’ve seen sponsors agree to endpoints for their IDE trial that they don’t think are clinically meaningful, but proceed with the endpoints because they were approved (or suggested) by FDA. Do NOT discount the clinical meaningfulness of a study endpoint. If the primary endpoint doesn’t meaningfully address a considerable improvement in how the patient will function, feel, or survive, then do not agree to run a trial with such an endpoint. Here are some suggestions when considering study endpoints.
- Engage key opinion leaders to ensure that all endpoints would be considered clinically meaningful by clinicians in the field.
- Give careful consideration to secondary endpoints. If secondary endpoints are included, the ranking of these endpoints should be meaningful and they should be adequately powered to be successful.
Lack of validated patient-reported outcomes (PROs). As FDA moves toward a more patient-centric, benefit-risk approach to approval of novel products, it’s becoming increasingly important that the sponsor has a validated PRO to address how much the device has improved patients’ lives. Unfortunately, too often sponsors will either (1) not use a PRO at all, (2) use a quality of life instrument that isn’t well-suited to address the important questions that a more targeted PRO could address, or (3) fail to properly validate the PRO according to FDA guidance. We strongly consider adding a PRO if possible. This should follow the FDA guidance document (Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims) and Sponsors should work with FDA throughout the validation process.
In summary, conducting a thoughtfully designed and well-executed clinical study is the best way to ensure a successful outcome at an FDA Device Panel Meeting. While there are many factors to consider during the design and conduct stage of a clinical study, the clinical and regulatory team supporting a panel-track device ought to give serious considerations to the historical concerns of Device Panel members. At a minimum, these should include the justification for the trial design, responsiveness to FDA review questions, clinical meaningfulness of the study endpoints, representativeness of the enrolled population, and prevention of missing data. A sponsor’s attentiveness to these issues up front has the potential to pay huge dividends down the road not only at the Device Panel Meeting, but also during the PMA review process and after approval in support of positive reimbursement and coverage determinations.
ABOUT THE AUTHORS
Christopher Miller, MS, is a biostatistician with extensive experience in the design, analysis and interpretation of clinical trials. Prior to joining 3D Communications, he was a senior biostatistician at NAMSA. Miller has published more than a dozen articles in peer-reviewed medical journals. Connect with him on LinkedIn.
Susan Resnick, PhD, has 20 years of experience in clinical research, medical device technology and regulatory strategy. As a scientific executive at 3D Communications, she guides clients through critical meetings with FDA, including Advisory Committee meetings. She has published more than 35 articles in peer-reviewed medical journals. Connect with her on LinkedIn
Todd Gross, PhD, has more than 25 years of experience in biostatistics and clinical development in the pharmaceutical and medical device industries. Todd uses his expertise in analysis and interpretation of clinical data, clinical study design, endpoint selection, and patient-reported outcomes (PRO) development and validation, to help clients prepare for FDA interactions and advisory committee meetings.