Periodic reviews are premised on one core assumption: the system sits still. You validate it. It remains in that state. Anything that changes runs through documented change control. Reviews confirm nothing drifted; for deterministic software, that assumption mostly holds.
It works because “traditional” systems tend to fail conspicuously: thrown errors, stopped processes, complaints. People notice.
Why it breaks for AI
AI breaks all of that because AI can degrade without any visible failure. Small shifts happen without sending out distress signals. No errors, no downtime, no complaints. Output flows with the expected volume and confidence scores into the same workflows and a reviewer signs the same form they signed last quarter.
One practitioner illustrates the failure mode with a complaint-triage model that passed qualification yet disagreed with manual decisions about one time in seven, concentrated in the subgroups the training data covered poorly. Nothing in its logs would have shown it.
Periodic evaluation draws on deviations, incidents, and upgrade history. These assume failure is visible or change is documented. Nothing was documented because it wasn’t known.
The configuration baseline for traditional software lists versions, patch levels, and parameters. These define the system. They are structured, documented, and predictable.
For an AI model, there’s a unique ingredient: the training data. That data defines the system as much as the code. If a model is retrained on a different data set, then the model is functionally new — no matter what the version number says.
The regulatory signals
Several regulatory efforts are converging on the same underlying point.
FDA’s January 2025 draft guidance assesses an AI model’s credibility for a specific context of use, typically at a point in time. A separate section addresses lifecycle maintenance for models that stay in use, such as manufacturing. Organizations should monitor performance on an ongoing basis, with metrics, risk-based monitoring frequency, retesting triggers, and other aspects of the plan held in the site’s quality system and changes routed through its change management.
Also in 2025, FDA finalized its CSA guidance for medical device production and quality system software. It's not AI-specific. But the message still counts: validation scales to risk, and assurance becomes ongoing.
The most explicit statement comes from the draft EU GMP Annex 22. It expects ongoing performance monitoring after deployment. It restricts critical GMP use to static, deterministic models and removes continuously learning and generative models from critical applications.
What replaces the annual check
Pulled together, the direction is starting to come into focus:
- Ongoing monitoring, not calendar-based review. Performance measured against a labeled sample of real output, input drift statistics, and override rate and direction, with an owner and defined review cadence.
- Retraining as its own governed event. A model update requires a trigger, a data-quality gate, and a delta check against the acceptance criteria before approval. The same rigor applied to a traditional change control event.
- Risk tiering that dictates the level of monitoring. Intensive, ongoing scrutiny is required for a model shaping a high-consequence decision. Lower-risk models drafting a first pass for human review doesn't.
- Periodic review refocused on verifying the monitoring. The review checks that the process worked versus trying to catch everything that happened since last time.
All of this evolves how periodic reviews are defined. Instead of proving a state once and checking in occasionally, the validated state has to be actively held, with evidence that it's being held, for as long as the model stays live.
