In its letter to medical device firms, FDA said it has observed an increasing number of third-party test labs generating testing data “that are fabricated, duplicated from other device submissions, or otherwise unreliable.” Specifically, the agency called out “numerous” medical device submissions containing unreliable data from third-party test labs “based in China and India.” Calling the trend “alarming,” FDA said it is has been unable to authorize marketing for certain medical devices whose submissions include such data, or to reach substantial equivalence determinations.
Accordingly, FDA admonished device firms to take proactive steps to qualify third-party test labs and to closely scrutinize all testing data that a firm does not perform itself, especially data relating to biocompatibility and other performance testing. FDA’s letter also notes that third-party labs being voluntarily accredited under the Accreditation Scheme for Conformity Assessment (ASCA) program is beneficial but “does not substitute for conducting an independent assessment of all third-party data.”
FDA further announced it is looking at additional ways to identify and correct data integrity issues, including through the bioresearch monitoring program, which we previously summarized online here. But in the immediate future, FDA asserted the medical device industry must be vigilant and proactive about ensuring the integrity of all data they include in medical device submissions.
After witnessing a rise in falsified/altered data during the COVID-19 pandemic, especially for biocompatibility data, it appears that FDA may be employing heightened scrutiny of all raw data for irregular trends.
Last week’s warning letter underscores the agency’s trend toward cracking down on data validity and integrity issues. It remains to be seen whether FDA would expect companies to detect data issues in the absence of obvious patterns, which may be difficult to do without having conducted the test. It also remains to be seen whether FDA will identify specific “bad actors” among test labs. If you have any questions over the accuracy or validity of your data, or regarding statistical analyses for medical devices more generally, feel free to contact any of the authors of this alert or the Hogan Lovells attorney with whom you regularly work.
For many years, FDA has required submitters to attest to the truthful and accuracy of the content of submission in its entirety and especially as it relates to the data contained therein. Further all submission to FDA are subject to 21 USC Part 1001 regarding prohibitions against false statements to the government. Specifically related to FDA submissions, the agency’s Application Integrity Policy is the tool the agency created in response to manufacturer’s submitting data that is fraudulent, falsified or lacking in sufficient integrity, but it has presumed that such activity was submitted with knowledge of, or a lack of ordinary care exercised by, the manufacturers. This appears to be a problem wherein the manufacturer too is a victim rather than the perpetrator and so other types of tools may well be necessary. Where the bad act occurs inside of a contracted third party facility, it may well be incredibly difficult to detect given that the fraud would be perpetrated by individuals well versed in generating legitimate test data, who would very likely be able to construct test reports that appear on their face to be valid.
The agency tries to raise awareness with manufacturers to be vigilant for this type of conduct, but intentional fraud of this sort will likely be as difficult for the manufacturers to detect as it is for the agency – indeed more difficult as companies would not be privy to as large a data set from a given lab as FDA might. While the agency refers to the perpetrators recently detected as being labs in China or India, the agency does not identify the bad actors by name, leaving companies to do their best through their vendor qualification and monitoring activities and also performing careful reviews of the reports to ferret out possible fraud. Where the agency suggests that companies should be on the lookout for improbable and impossible results, that too is unlikely to be obvious, as it is not typical for a company to send a device it has developed with diligence for outside testing that it does not anticipate as being highly likely to pass testing. We note that the indicia of integrity issues are often subtle, and that solutions to data issues, when encountered will require engagement by both the agency and industry, because as FDA points out, this harms everyone.
At Hogan Lovells, we help clients prepare agreements for vendor services, evaluate supplier qualification and management programs, prepare medical device submissions, investigate suspected wrongdoing, and most importantly, advise clients regarding the “Truthful and Accurate” statement that needs to be signed for all marketing submissions. We can help new technology sponsors when they have suspicions that there may be integrity issues in the data from third party labs, help evaluate indicia of such concerns that require additional evaluation, and interact with sponsors and their contractor regarding the data. Potential indicia or observations of data concerns can include:
-
Identical results are reported for multiple animals or subjects
-
Identical language or conclusions are presented in reports from different labs
-
The same photocopied results are used in multiple studies
-
Statistical analyses show very little variation; e.g., when the standard deviation or variance is very close to the mean, or coefficients of variation are less than 5%.
-
A large number of outliers in the data; e.g., unusually high or low values that don’t fit the typical distribution of the data
-
The data appears to be too perfect
Additionally, should a company become aware that data from a third party test house may well be fabricated, either before or after submission, we have assisted companies in working through the matter with FDA and also navigating instances in which the agency threatens or has invoked its Application Integrity Policy against a company. We have also helped companies perform the investigation and negotiate remedies under the contract or, where that is not possible, engage in litigation.
Authored by Randy Prebula, Jodi K. Scott, Kristin Zielinski Duggan, and Mike Mortillo
