AI Governance
Has your AI ever told you no?
The short version
- My assistant interrupted a task I had already approved to tell me it had fabricated something. It did not develop a conscience. It executed a rule I wrote and forgot about.
- Almost every AI policy governs the human at the keyboard. The person was not the one hallucinating.
- Three layers: Policy governs you, Context governs what the model knows, Conduct governs how it behaves when it is wrong. Almost nobody builds the third.
- Write conduct rules as checkpoints that fire at a nameable moment, not as dispositions that ask for vigilance. Dispositions decay over a long session. I have the receipts.
In the middle of a task I had already approved, my assistant interrupted itself.
"Stop, I need to correct myself before you go looking any further. I'm sorry I hallucinated. In my last message I said..."
Unprompted, mid-task
I was not auditing it. I was not asking it to double-check anything. I was moving, and it stopped me.
My first reaction was that the model had done something impressive. My second reaction, about ninety seconds later, was worse and more useful.
It had not done anything impressive. It had done exactly what I told it to do.
I wrote that sentence
I went looking for why it behaved that way, and I found the answer in two files I wrote myself and had mostly forgotten about. Both of them load before the model reads anything I type.
Read the first rule against what the model actually said. The correction is close to verbatim.
It did not develop a conscience. It executed an instruction. I wrote the sentence, and the machine said it back to me at the exact moment it was needed, without me asking, while I was busy looking somewhere else.
That is not a story about how good AI has gotten. It is a story about infrastructure.
What most people mean by AI governance
Ask ten people what AI governance means and you will get some version of the same answer. Rules about what you are allowed to paste into the tool. A policy about what you may publish. A note about client data. Maybe a training module, if the organization is large enough to have one.
All of that is real work and it matters. I am not dismissing it.
But look at what it governs. Every one of those rules governs the human sitting at the keyboard. It tells the person what to do.
The part everyone misses
The person was not the one hallucinating.
No acceptable use policy in the world would have caught what happened to me, because the failure was not in my behavior. I did nothing wrong. I asked a reasonable question, got a confident answer, and was about to act on it. The policy layer had nothing to say about that, because the policy layer was never pointed at the model.
The three layers
Here is how I think about it now, and this is the part I would hand to anyone setting up their own environment.
Policy
Governs the human
What you may paste, what you may publish, what you may automate. Most people stop here. Most companies stop here. This is what "AI governance" means in the press.
Context
Governs what the model knows
Your background, projects, voice, files, standing preferences. The people who get past layer one usually land here and call it governance. It is not. It is personalization. Context makes the model useful. It does nothing about what happens when the model is wrong.
Conduct
Governs how the model behaves
How it acts when it is uncertain, wrong, or out of its depth. Not what it knows. How it behaves at the moment of failure. Almost nobody builds this layer. It is the layer that produced the word "Stop."
What layer three actually looks like
It is less work than it sounds. A short list of rules, written in plain language, in the files the model reads before it reads you.
When you are uncertain, say so. Name the confidence level. Do not fill the gap with something that sounds right. An answer labeled "I am not sure, but" is worth more than a confident wrong one, because I can act differently on it.
Label every claim by where it came from. There is a difference between something you measured, something you read, and something you inferred. Say which one it is, every time, without being asked. Fabrication is what happens when the third one gets delivered in the voice of the first.
When you have fabricated, stop and say it plainly. No hedging, no burying it in a paragraph of qualifications, no quiet correction three turns later. Stop the task.
Gate anything irreversible. Before a command that deletes, resets, overwrites, revokes or publishes: run a read-only inventory first, show what will be destroyed, and state the recovery path in one sentence.
Write checkpoints, not dispositions
This is the distinction that took me longest to learn, and it is the one that decides whether a rule survives contact with a real working session.
A checkpoint fires at a nameable moment and leaves evidence you can inspect. Label this claim. Run the inventory before the destructive command. Show the output. You can look at the transcript afterward and see whether it happened.
A disposition asks for sustained vigilance across an entire session. Be careful. Be rigorous. Do not use that punctuation. Dispositions feel like the same kind of rule. They are not. They decay as the session gets longer.
I know this because I tested it on myself. One of my own style rules is a disposition, and in a four-hour session it was violated in nearly every message, while the technical work inside those same messages stayed measured and correct. The rules that fired at a specific moment held. The rule that asked for continuous attention did not.
The rule about rules
Write checkpoints wherever you can, and attach any unavoidable disposition to a discrete review step at the end, where it becomes a checkpoint again.
This is not hypothetical. Here is the catalogue.
Every rule in my conduct layer exists because something went wrong first. I keep the failures written down next to the rules, with dates, because a rule with no incident attached is a preference.
A confident UI path with no link
I was told to click through "API Keys, then Create" with no URL attached. The instructions were plausible and the wrong product's settings page looked nearly identical, so I generated a token in the wrong place. The rule now: never give a UI path without a URL. If the URL is unknown, say the URL is unknown.
A destructive command against unique work
git gc --prune=now was recommended against a repository holding five unreachable commits, two of them unique work that existed nowhere else. git fsck --unreachable caught it. Nothing else would have. The rule now: inventory before anything irreversible, and put the rescue step in its own block before the destructive one.
A placeholder inside a runnable block
A copy-paste block contained a bracketed description where a real value belonged. It was pasted verbatim into a DNS record, created a duplicate entry, and blocked domain verification. The rule now: no placeholders inside a code block, ever. Name substitutions above the block in capitals.
None of those three was caught by a policy about what I am allowed to paste. Every one of them was a model behaving confidently at the exact moment it should have stopped.
Why this has to exist before you take off
The instinct is to build governance later, once the thing is working. Get moving first, add the guardrails when there is something worth guarding.
Run the cost of that decision forward.
Catching a fabrication in the chat window costs one turn. I lost ninety seconds.
Catching it after it has shipped in a client deliverable costs the client relationship, and it costs it in a specific way that is hard to recover from, because the client now has to wonder what else you did not check.
Catching it after it has been wired into something automated costs everything downstream of that automation, silently, for as long as nobody looks.
That correction was not a lucky break. It was the return on a file I wrote before I needed it. Governance is not a brake on speed. It is the thing that makes speed survivable, which is the only reason anyone would build it in the first place.
You do not install brakes at altitude.
What this does not prove
I want to be precise, because the overclaim here is easy and it would be dishonest.
This proves one model executed one written rule one time. It does not prove that models reliably police themselves, and I would not build a business on the assumption that they do. The line I keep in my own instruction file is blunter than anything I would put in a headline.
"Self-review does not catch fabrication. Re-reasoning over a wrong claim re-confirms it. Only an external oracle catches it."
From my own instruction file
That is the whole caveat. You are running the error back through the machine that made it. Writing conduct rules changes the probability that something surfaces early. It does not remove the need for a check that comes from outside the model: a command that actually runs, a document you can actually open, a number you can actually verify.
Layer three is not a replacement for verification. It is what makes verification affordable, by surfacing the cheap failures early enough that you only have to verify the expensive ones.
The test
The question at the top of this page is the first of three. They take about thirty seconds together.
Ask your AI
- Has it ever told you no?
- Has it ever told you it was not sure?
- Has it ever corrected itself before you caught it?
If the answer is no to all three, there are two possible explanations. Either it has never been wrong, or nothing in its environment has ever told it to say so.
It has been wrong.
A model that always agrees with you is not aligned with you. It is unsupervised.
And the gap between those two things is invisible right up until the moment it is expensive.
I build the third layer with people, in their own environment, on their own files. If you ran the test and did not like your answers, that is usually where we start.
Questions people ask about this
Straight answers, in case you skimmed.
What are the three layers of AI governance?
Policy governs the human: what you may paste, publish, or automate. Context governs what the model knows about you: your background, projects, voice, and files. Conduct governs how the model behaves when it is uncertain, wrong, or out of its depth. Most people build the first two and skip the third.
Is giving the model context the same as governing it?
No. Context is personalization. It makes the model more useful by telling it about you, your work, and your preferences. It does nothing about what happens when the model is confidently wrong. Governance is about behavior at the moment of failure, not about knowledge.
Can an AI reliably catch its own hallucinations?
Not reliably. Self-review does not catch fabrication, because re-reasoning over a wrong claim tends to re-confirm it. Written conduct rules raise the odds that an error surfaces early. They do not replace a check from outside the model: a command that actually runs, a document you can open, a number you can verify.
What is the difference between a checkpoint and a disposition?
A checkpoint fires at a nameable moment and leaves evidence you can inspect, such as labelling a claim or running an inventory before a destructive command. A disposition asks for sustained vigilance across a whole session and decays as the session gets longer. Write conduct rules as checkpoints wherever possible, and attach any unavoidable disposition to a discrete review step at the end.
How do I know if my AI setup has a conduct layer?
Ask three questions. Has your AI ever told you no? Has it ever told you it was not sure? Has it ever corrected itself before you caught it? If the answer is no to all three, either it has never been wrong, or nothing in its environment has ever told it to say so.
Sources & Notes
- The model's correction is quoted exactly as it was output, including its punctuation.
- Both rules are quoted verbatim from instruction files in my own working environment, which load at the start of a session. The first is line 53 of a prompt-audit skill file. The second is from the proof gate section of my project instruction file.
- The failure catalogue entries are the real incidents recorded next to those rules, with the dates as logged. Product names have been generalized.
- The three-layer framing and the checkpoint versus disposition distinction are mine. They are working models for setting up an AI environment, not published standards.