"Governed AI" is having a moment. It's frequently cited at recent industry conferences, it's the language regulators have settled on, and it's increasingly the terminology validation and quality operators use when someone wants to introduce AI anywhere close to a GxP process.
To be clear: “governed AI” is not a regulatory term. It is a concept that illustrates workflows using AI alongside human oversight, audit trails of what the AI did (and why), plus evidence that tethers AI outputs to a documented review step. Humans retain full accountability.
This post explores guidance coming from regulatory bodies and how validation teams that apply critical thinking are well-prepared for governed AI.
FDA’s first guidance
In January 2025, FDA issued its draft guidance Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products. It centers a seven-step, risk-based credibility assessment framework:
- Define the question of interest an AI model must answer
- Define the context of use, such as the model’s role, output use, and evidence sources
- Assess model risk
- Plan credibility assessment proportionate to the risk
- Execute credibility assessment
- Document credibility evidence
- Determine adequacy of evidence
That’s pretty familiar territory. It’s the same philosophy validation teams adopt when using a CSA approach of critical thinking over traditional CSV: proportional rigor instead of exhaustively testing every path.
Principles for AI practice from FDA and EMA
In January 2026, FDA and EMA jointly issued ten Guiding Principles of Good AI Practice in Drug Development.
This guidance is high-level. More interestingly, it signals inter-agency alignment. And although non-binding, it surfaces multiple points worth recentering: human-centric design, a risk-based approach, clear context of use, and model design practices that demand explainability and interpretability.
For those in the validation field, the joint guidance from FDA and EMA gives data governance and documentation its own discrete principle. That sets expectations for traceable, verifiable lifecycle management.
Newer guidance alongside 21 CFR Part 11
Neither the January 2025 draft guidance nor anything that's followed alters or replaces 21 CFR Part 11 (or the predicate rules requiring the record to exist). Part 11 is a binding regulation controlling electronic records and electronic signatures across every FDA-regulated environment. It sets enforceable requirements. By contrast, newer AI-specific guidance introduces non-binding recommendations for establishing model credibility.
Related, but separate. Any AI tool touching regulated records must satisfy Part 11's audit trail and e-signature requirements. Establishing credibility assessment for AI models is serving the same greater purpose of confidence but wholly independent guidance.
In FDA’s own words: “the barrier is a myth”
In an interview with Axendia, FDA Policy Analyst Daniel Walter candidly discussed the agency’s stance on AI and validation.
Mr Walter was clear: "The barrier is a myth”. There is no regulatory barrier to adopting AI. Any perceived barrier is cultural, not regulatory. In fact, FDA's own review of warning letters found no citations for adopting a new technology. Instead, citations occur when a company claims to do something and then fails to actually do it. This is literally the same root cause as other compliance findings.
As further evidence, a recent warning letter involving AI misuse emphasized the finding was for “inappropriate use of AI”, not “use of AI is inappropriate”. The letter targeted how the company implemented and validated the tool, not the technology itself. The principle of strong governance hasn't changed.
Global guidance is codifying
FDA and EMA publicly aligning on governed AI principles points to where enforcement expectations are heading.
Furthermore, this alignment isn't confined to the US and EU. The agencies framed their joint principles as a starting point for international harmonization. It explicitly builds on work like ICH's harmonized clinical and technical standards and is the same motion of PIC/S harmonizing GMP inspection standards across borders. This common ground overcomes the inefficiency of every jurisdiction building its own AI rulebook from scratch.
For a validation program, the message is clear: AI is not special. It’s a tool like any tool. It requires control based on risk assessments. Even for organizations training their AI models on real-world data (RWD) and real-world evidence (RWE), the imperative for validation never recedes.
Equally important, avoiding AI adoption does not mean avoiding risk. Business risk (falling behind competitors who modernize, losing market relevance, margin erosion) is as real as compliance risk.
