Establishing Credibility: What It Takes to Earn Trust in Real-World Data Studies
By: Dr. Steven Farmer - Senior Partner, ABIG Health
About 10 years ago, a movement started in the U.S. Congress that has improved the medical device market for clinicians and patients. With 2016’s 21st Century Cures Act in 2016, the federal government began to broaden the use of Real-World Evidence (RWE) in clinical studies.
Now, fueled by additional regulatory shifts like the U.S. Food and Drug Administration’s (FDA) National Evaluation System for Health Technology (NEST) program, U.S. Centers for Medicare and Medicaid Service (CMS) guidance, and general industry enthusiasm, Real-World Data (RWD) studies are gaining momentum in the medical device marketplace.
Despite this enthusiasm, skepticism about the credibility and rigor of RWD studies remains high.
That skepticism is unfounded. Yes, high-quality RWD studies are simply cheaper or faster alternatives, but they also require thoughtful design, investment, and transparency to be credible and influential. For innovators and manufacturers, they are an essential complement to clinical trials.
RWD Studies Are Gaining Traction, but Come with Scrutiny
The FDA has proclaimed that it “is committed to realizing the full potential of fit-for-purpose RWD to generate RWE that will advance the development of therapeutic products and strengthen regulatory oversight of medical products across their lifecycle.” The agency initiated NEST in 2019 and, four years later, it issued guidance on RWE and RWD.
Meanwhile, in June 2025, CMS issued RWD Study Protocol Guidance. The medical device industry is eager to leverage RWD for regulatory and reimbursement purposes.
Here is the uncomfortable truth, however: some observers are skeptical. In my time at CMS, I heard comments like “I’ve never seen a negative RWD study.” Like partisan polls, RWD data studies are often seen as biased toward the entity that is releasing its findings. Manufacturers have been accused of cherry-picking data or offering an incomplete look at real world scenarios.
This skepticism means that companies that are gathering RWE and carrying out RWD studies must earn credibility. Even though policymakers are interested in RWD and RWE, their acceptance should not be assumed.
RWD Studies Require a Different Set of Skills and Investments
I will admit: some of the skepticism of RWD studies may be well founded — manufacturers are well-versed in traditional clinical trial design, but not in modern RWD methodologies.
The good news is that this deficit, and the resulting anxiety about it, are correctable, especially with a commitment to investment. (Contrary to popular perception, RWD studies are not “cheap.” According to the Pro Pharma Research Organization, the cost of an RWD study can vary significantly, ranging from approximately $80,000 to $2 million or more.
While certainly more economical than RCTs, meaningful RWD studies require significant investment in study design, data acquisition, analytics, and governance. Specifically, high-quality RWD studies require:
Expertise in causal inference and observational study methods;
And understanding of the strengths and weaknesses of different data sources; and
Statistical fluency in sensitivity analyses and bias reduction techniques.
Fit-for-Purpose Design: The Cornerstone of Credibility
To enhance credibility further, medical device developers and manufacturers should embrace fit-for-purpose design.
Fit-for-purpose design, sometimes calls fitness for purpose, means that manufacturers have worked to ensure a product, service, or system is specifically designed and tailored to meet its intended use and specifications. In terms of medical devices, fit-for-purpose means ensuring an innovation will work for different populations — both women and men, for example — or in certain settings and contexts.
As such, data sources, study design, and analysis all must align with the research question.
Studies are not fit-for-purpose when there is inadequate follow-up, when the wrong population is examined, or when there is insufficient endpoint precision. Each of these oversights can enhance bias and undermine the study’s authority.
Manufacturers and developers can address these problems by embracing propensity score matching, inverse probability weighting, instrumental variables, and other tools, but while these methods help, the only do so when applied thoughtfully and when key assumptions are tested and validated.
Evolving Data Sources and Technologies
Thankfully, over the last several years since the 21st Century Cure Act and the federal embrace of RWE, there has been a vast improvement in the RWD ecosystem. Manufacturers are now able to leverage linkages between insurance claims data, electronic health records, and lab, imaging, and other data, for example.
And new technology is coming online every day.
Artificial intelligence and machine learning will further enhance manufacturers’ ability to find and extract structured data from unstructured records. Additionally, variables once thought impossible to capture retrospectively, such as functional status and symptom scores, are increasingly accessible.
While there is great promise in new technology, manufacturers must walk before they run. Technological evolution comes with its own risks, including adding bias into the process.
Greater complexity demands more transparency and validation.
Building Trust: Transparency and Prespecification
That’s my last note of caution. Now for the helpful news: what process will reduce skepticism and avoid some of the technology-related pitfalls that can occur in RWE gathering?
Here is a five-step process:
Prespecification: define the protocol, endpoints, and clinically meaningful differences in advance.
Protocol registration: publicly register studies to prevent “data dredging.”
Amendments: if changes are needed, document them clearly with justification.
Data source validation: show reliability and completeness of datasets.
Independent oversight: use advisory boards or external review panels when feasible.
Notice that marketing is not on this list. Federal regulators and payers, not to mention private insurance companies, can sniff a rote elevator speech a million miles away. Embrace this mantra and repeat it often: trust is built through rigor and transparency, not a slick sales pitch.
Real-World Data studies hold great promise, but credibility is not automatic. Manufacturers must invest in the right people, methods, and processes to deliver fit-for-purpose evidence. By embracing transparency, methodological rigor, and prespecification, the device industry can overcome skepticism and realize the full potential of real-world evidence.
In other words: to shift perception, RWD must not just be real — it must be credible.
