In December 2024, an Ontario hospital reported a privacy breach to its provincial regulator. A former physician's AI scribe had joined a virtual hepatology rounds meeting and recorded it. The hospital found out because the tool emailed a summary and transcript to the meeting participants.

The tool announced itself by doing its job. Nothing the hospital had in place surfaced it first.

That gap sits at the center of AI governance in healthcare. In a December 2025 survey of more than 500 healthcare workers, 17 percent admitted to using AI tools their employer had not approved. Of those who admitted it, 45 percent said they did so for a faster workflow.

I would expect most of those organizations to have an AI acceptable use policy on file. The policy said no. The workflow said yes. The workflow won.

What 700 Assessments Taught Me About Documents

I have completed more than 700 security risk assessments, and one pattern repeats in nearly every engagement. A policy exists, it is well written, and the right people approved it. Then I ask for the last time someone tested whether the policy was followed, and the room gets quiet.

A policy states intent. A control produces evidence. The two are easy to confuse because both live in the same binder and both get presented at the same audit. Only one of them can tell you what happened on a Tuesday in March.

Where The Evidence Was Missing

Three recent cases show what the gap looks like when someone outside the organization starts asking questions. Each involves allegations or disputed findings, and I have noted where the parties disagree.

Consent Without A Record. In April 2026, patients filed a proposed class action against Sutter Health and MemorialCare, alleging that an ambient AI scribe recorded their visits without meaningful informed consent. The complaint cites California's medical confidentiality and invasion of privacy statutes and the federal Wiretap Act. The scribe vendor was not named as a defendant, and it recommends that clinicians follow their organization's consent policies. Sutter says it takes patient privacy seriously. A consent policy answers one question. A court asks a different one: can the organization show that each patient received notice at each encounter?

A Vendor's Number Is A Claim. In 2024, the Texas Attorney General reached a settlement with Pieces Technologies after finding that its advertised accuracy metrics, including a severe hallucination rate below 1 per 100,000, were likely inaccurate and may have deceived hospitals. At least four Texas hospitals were sending patient data to the product in real time. Pieces disputes how the Attorney General characterized the agreement. The Attorney General's message to hospitals was direct: they must decide whether an AI product is appropriate and train their employees accordingly. A test result from your own environment is evidence. A number in a sales deck is a claim.

The Review That Took 1.2 Seconds. In 2023, ProPublica reported that Cigna physicians denied more than 300,000 claims over two months using a system called PxDx, averaging 1.2 seconds per claim. A class action followed, alleging that this violated California's requirement for individual physician review. Cigna says the tool does not use AI, that it speeds payment to providers, and that the reporting mischaracterized it. Whatever the court decides, the case shows which record carries the weight. A review policy says a physician looks at the claim. The timestamp says how long the physician looked.

What Failure Costs

The stakes fall in four places.

  • Regulatory Penalties. OCR has expanded its enforcement initiative from risk analysis to risk management, meaning what an organization does about the risks it finds. Civil penalties are listed at up to $2,190,294 per violation category per year. State attorneys general have also shown they will apply consumer protection law to healthcare AI.
  • Litigation. The Sutter case alleges violations through ordinary clinical workflow. A proposed class action can follow from a tool that was deployed without a record of notice.
  • Patient Harm. The UnitedHealth lawsuit over the nH Predict algorithm alleges that a patient stayed in a care facility and paid about $150,000 out of pocket after coverage was cut. UnitedHealth says the suit has no merit. The allegation shows how far downstream an automated decision can land.
  • Patient Trust. A patient who learns of an unannounced recording can reasonably decide to say less at the next visit, and a clinical record is only as good as what the patient tells the clinician.

Four Questions To Test Your Own Program

Pick one AI tool that touches patient data in your environment and ask these four questions.

  1. Inventory. Can you list every AI tool in use today, including the ones employees added on their own?
  2. Approval Record. Who approved this tool, on what date, and based on which risk review? For a tool that records patients, where is the record of notice and consent?
  3. Enforcement Log. What did your controls allow, block, or flag for this tool on a specific date last month?
  4. Test Result. When did you last test that the control works, and what did the test show?

If the answer to any of these is "the policy says," you have a document. A document supports compliance. A control with evidence is how you show that you are compliant.

Design the evidence deliberately. Log what proves the control worked, retain as little as possible, and protect those logs as you would any system holding ePHI.

OCR has not issued an AI-specific enforcement position, so applying its risk management logic to AI is my reading. It follows directly from the Security Rule. Any AI tool that touches ePHI belongs in the risk analysis, and the risk management that follows needs owners, timelines, and proof. The NIST AI RMF is voluntary, and its guidance is that evidence should show ongoing organizational behavior, not a one-time assessment.

The Path Forward

The organizations that handle AI well over the next two years will be the ones that can answer a simple request: show me. I built ControlLayer.ai around that request, because I have spent two decades on the receiving end of it.

Pick one AI tool this week and ask for the evidence. The answer will tell you what you have.