DNV Sets Out Six Steps For Trustworthy Synthetic Data In Healthcare

A position paper from assurance and risk management provider, DNV, has laid out a six-step method for judging whether synthetic data is fit to be used in healthcare, arguing that trust in artificially generated datasets has to be built on evidence rather than assumed.

Titled “How can I trust my fake data?”, the paper treats data quality as something to be assessed at every stage of the lifecycle, from the real-world records a generator is trained on through to the clinical performance of whatever tool is eventually built with it.

The six steps are: defining the intended use of the dataset; assessing the quality of the training data; evaluating the performance of the generation method; validating the synthetic output against real-world benchmarks; carrying out analytical and clinical validation; and documenting the results in a structured “synthetic data card”.

“Synthetic data holds significant potential, but its safe use depends on structured and transparent evaluation,” commented Principal Researcher at DNV, Serena Elizabeth Marshall (pictured).

“By grounding assessments in evidence and aligning them with regulatory expectations, organisations can better understand whether data is fit for purpose.”

Why healthcare is looking at artificial records

Real-world health datasets are frequently out of reach because of privacy regulation, ethical constraints and questions of ownership. When they are available, they are often fragmented, inconsistent and limited in both volume and quality.

Collecting and labelling that data is slow, costly and sometimes not possible at all, particularly for rare conditions where patient numbers are too low to support robust AI model training.

Synthetic data has been proposed as a scalable alternative that can reflect real-world complexity, allow rapid prototyping and algorithm testing, increase sharing opportunities and simulate edge cases.

The paper points to a range of applications already being explored, including simulation and prediction research, testing during AI development, clinical trial control arms, epidemiology and public health research, and training and education.

Training AI algorithms on synthetic medical imaging data or patient records has been reported to improve the accuracy, robustness and reliability of diagnostic tools. Synthetic datasets containing traits such as age, blood sugar levels and medical history have been used to develop and test treatment protocols, and to build risk prediction algorithms for diabetes.

It also has a role in clinical trials, where it could supplement external control arms in randomised control trials and support the identification of drug candidates and the prediction of treatment outcomes.

The drivers DNV identifies include growing reliance on datasets, the cost and complexity of governing real-world data, privacy and intellectual property concerns, demand for diverse and fair datasets, and the need to investigate scenarios that have not been observed but remain plausible.

“Synthetic data opens up new possibilities across medicine, from improving the development of diagnostic tools to enabling research into rare diseases where real-world data are limited,” according to Team Leader, AI Risk and Assurance Science at DNV, Carla Ferreira.

“With the right validation and safeguards in place, it can support more robust testing, faster innovation, and ultimately better-informed clinical decisions.”

Five dimensions of quality

At the centre of the framework are five dimensions against which a dataset is measured: similarity, utility, privacy, fairness and environmental impact.

Similarity covers how well the synthetic data preserves the statistical properties of the real dataset, compared feature by feature and across the full population, including correlations and underlying patterns.

Utility, also described as usability or broad utility, tests how the data performs when it replaces real data for a specific analytical task.

Privacy evaluation estimates the risk that attributes of individuals can be inferred, or that natural persons in the original dataset can be re-identified.

Fairness addresses the possibility that a generator reproduces or amplifies biases already present in the source data, which the paper warns can produce inequitable outcomes that fall disproportionately on particular subgroups. Handled well, the same process could help rebalance existing biases.

Environmental impact reflects the computational load of training generative models such as GANs or diffusion models, which the paper describes as poorly documented but capable of carrying a high carbon footprint.

How much weight each dimension carries depends on context. Where a synthetic dataset is only being used to augment an internal dataset that will never leave the organisation, privacy may matter less.

The privacy trade-off

Balancing privacy against representativity is one of the harder problems the paper identifies. Heavy privacy safeguards can strip representativity out of a dataset, while light ones risk exposing sensitive information.

The picture is complicated further by the common practice of using the same metrics for both. Statistical closeness measures are often used as a proxy for privacy risk, but the paper observes that the results can pull quality in opposite directions.

Inference attack simulations are presented as a more reliable estimate of practical disclosure risk. Membership inference attacks test whether an attacker can work out if an individual’s record was in the original dataset, while attribute inference attacks test whether sensitive attributes can be predicted.

Some generators build in differential privacy, adding statistical noise to provide a mathematical guarantee of privacy loss. That number does not necessarily translate into a usable read on residual re-identification risk, the paper cautions.

The gap is one practitioners have flagged. “We still lack clear guidance on privacy best practices for combining metrics and defining threshold values,” observed Nico Sieberg-Riedel of the German Federal Institute for Drugs and Medical Devices.

Risks that travel downstream

Bias can be reinforced where the generating algorithm underperforms or the training dataset fails to represent the diversity of the real-world population it is meant to imitate.

Generators can also produce data points that sit outside real distributions, resulting in hallucinated and unrealistic samples. And where an algorithm produces output very close to the original dataset, the result can be false confidence in anonymity.

The paper frames these as risks to both performance and regulatory compliance in whatever application the data ends up supporting. Managing them starts with a risk description for the intended downstream use, covering consequences for stakeholder interests, an assessment of severity and an assessment of the uncertainty and strength of knowledge behind both.

Working through the lifecycle

Training data evaluation is the first technical checkpoint, since the validity of anything derived from it depends on what went in. Metrics here cover accuracy, relevance, completeness, representativity, diversity and selection bias, alongside a record of what preprocessing was carried out and why.

Generation performance is the next, and the paper describes the literature on how to measure it as fragmented and difficult to navigate. Common failure modes include algorithm convergence problems, mode collapse, mode invention, density shifts, noisy data, replication of training data, inadequate coverage and bias amplification.

Synthetic data evaluation then compares the generated dataset directly against its training set. Utility can be checked against established medical knowledge such as disease prevalence, expert assessment or sample-level logical rule engines, while similarity is assessed one variable at a time and across multiple variables to see whether joint distributions, inter-variable dependencies and structural relationships survive.

Analytical validation typically uses split-sample validation, benchmarking a model trained on synthetic data against the same model trained on real data. The paper refers to this as “Train on Synthetic, Test on Real” versus “Train on Real, Test on Real”, and warns that a larger dataset may simply produce better results because of its size rather than the quality of its medical inferences.

Clinical validation and monitoring closes the loop before market access, testing whether the drug, technology or device performs as intended for the target population in a real clinical context. Any drift or adverse impact traceable to the use of synthetic data during development should surface here or through post-market surveillance. Both are mandated under the EU Medical Device Regulation and In-vitro Diagnostic Medical Device Regulation.

Regulation without a synthetic data rulebook

There are no regulations written specifically for synthetic data. Instead, existing rules covering clinical trials, medical devices, health data, AI systems and in vitro diagnostics apply.

On the horizontal side, that means the General Data Protection Regulation, the Data Governance Act, the Data Act and the European Health Data Space in the EU, and the Health Insurance Portability and Accountability Act in the US.

Applying them is not straightforward. GDPR’s broad definition of personal data affects both the use of personal data to generate synthetic data and the handling of the synthetic output itself, particularly where residual re-identification risk is high enough for the result to count as personal data.

The EU AI Act adds requirements for high-risk applications, including AI-based medical software. Structured quality assessments must evaluate and report on utility and similarity, privacy, fairness and bias, and the computational footprint of both high-risk applications and general-purpose AI systems. Providers of systems generating text, audio, video and image data must mark outputs as artificially generated.

Both the Data Governance Act and the AI Act recognise synthetic data generation as a privacy-preservation method.

A judgment in CJEU Case C-413/23 P, appealed under Article 56 of the Statute of the Court of Justice of the European Union, closed a long-running argument over whether pseudonymised data could ever be considered anonymous where a re-identification key exists. The Court decided it can be, where the risk of re-identification is reasonably low.

The practical effect is that a single synthetic dataset can be personal data and anonymous at the same time, depending on who holds it and whether they can reach a re-identification key. The EU Digital Omnibus Regulation proposal seeks to codify that judgment through amendments to GDPR.

A European Commission template published in July 2025 requires providers of proprietary general-purpose AI models to document and disclose the data used during training, including details of synthetic data used to train the model and the precursor data used to generate it.

In December 2025, the US published a presidential executive order establishing a national policy framework for AI governance, aimed at minimal national policy and forbidding what it describes as patchwork regulatory regimes. It cites state laws embedding ideological bias in models and a Colorado law banning algorithmic discrimination, and sets up an AI litigation task force to challenge laws inconsistent with the new policy.

What device and drug regulators want to see

For medical devices, manufacturers must supply documentation on the provenance, generation and relevance of the datasets they have used as part of the technical documentation.

The US Food and Drug Administration’s Center for Devices and Radiological Health has flagged synthetic datasets in its regulatory science priorities, and a January 2025 draft guidance on AI-enabled device software functions states that synthetic data used in support of a regulatory submission should come with a comprehensive explanation of how it was generated and why it is fit for purpose.

An expert working group convened by the UK Medicines and Healthcare products Regulatory Agency and the PHG Foundation has published guiding principles encouraging critical assessment of synthetic data before and after it is used in developing AI as a medical device. Core considerations run from source data provenance and generation model through to post-market surveillance.

On the drug side, the FDA released draft guidance in January 2025 covering the use of AI to generate data, built around a risk-based credibility assessment framework. The European Medicines Agency and Heads of Medicines Agencies network data steering group has a strategic advisory group with a 2025 to 2028 workplan that will consider whether synthetic data can feed into evidence generation alongside clinical data.

The paper also raises a point that sits beyond compliance, noting that ticking regulatory boxes is reportedly not enough to secure public trust in the secondary use of health data. Transparency of process and data provenance is required as well.

Cards on the table

The documentation piece is where DNV puts much of its emphasis. Borrowing from the existing practice of datasheets for datasets and model cards for AI models, the paper proposes synthetic data cards as a standard way to summarise intended purpose, generation method, training data, generator parameters, ethical considerations, suitability for the stated purpose and known limitations.

The concept has been tested through Profiler.SD, an evaluation prototype developed with Fürst Medical Laboratory. That work covered a reporting methodology for the generation process, a quality dashboard, an independent third-party assurance procedure and the data card itself. The prototype has been made available online as a demonstrator.

The paper is careful to draw a line between related terms. A digital twin is a digital model of a physical object, system or process continuously updated with data from its physical counterpart. Synthetic data mimics the statistical properties of real data. Simulated data is a type of synthetic data produced through computational models or simulations of real-world processes, supporting testing, prototyping or training where direct measurement is impractical.

Multi-disciplinary by necessity

DNV’s position is that safe adoption requires structured, evidence-based quality evaluation and transparent documentation, aligned with regulatory expectations and folded into assurance processes.

The five dimensions map onto DNV’s existing assurance terminology and its Recommended Practice for Assurance of AI-Enabled Systems, DNV-RP-0671, with similarity supplying evidence for representativity claims, utility supporting the fit-for-purpose claim, and privacy, fairness and environmental impact feeding the risk description.

Getting there means pulling in expertise from healthcare, data science, ethics and regulation to define quality standards, set acceptance criteria and balance the trade-offs between them.

DNV is participating in the EU SYNTHIA project, contributing research on the assurance, risk and regulatory implications of synthetic data in healthcare. It plans to develop a synthetic data assurance framework, explore the use of AI agents in assessments, and produce guidance for manufacturers, assessors and notified bodies.

The paper’s own framing is blunt about the limits of the technology. Synthetic data can support innovation and widen access to data, but it is not a silver bullet, and users have to actively manage the quality and compliance challenges that come with it.

As Norsk Helsenett’s Cecilie Fossli put it in the paper: “Synthetic data are crucial for the future of healthcare”.

DNV operates in more than 100 countries as an independent assurance and risk management provider. Further detail is available at dnv.com.

Last Updated on August 21, 2026 by Nick Ross