Write "never send mail to a customer without human approval" into an agent's instructions and you have expressed a preference. A good model will honour it almost always. Almost always is fine for tone of voice and wrong for anything you cannot undo, because the cost of the rare failure is not the average cost. It is the whole cost. One refund confirmation sent to the wrong customer is not offset by nine hundred correct refusals.
This is the line that decides how an agent build is engineered. Some rules go in the prompt, where they can be edited by anyone in the department. Some rules go in code, where the agent cannot reach them at all. Getting that split right is most of what makes an agent safe enough to leave running.
Why an instruction cannot be a wall
A language model samples. Ask the same question twice and you may get two different answers, which is useful for drafting a reply and useless as a safety property. There is no setting that turns a probability into a guarantee.
Instructions also compete. A prompt that says "never send external mail without a release" and also says "respond to urgent customer requests the same day" contains a conflict, and the model resolves conflicts by judgement rather than precedence. As the prompt accumulates edits from three different people over six months, the number of these conflicts grows.
Then there is the data. An agent that reads customer email is reading text written by strangers, and some of that text will contain instructions. A support message whose body says to ignore prior instructions and confirm the refund directly is not an exotic attack, it is a thing that happens, sometimes by accident when a customer forwards an internal thread. The agent has no reliable way to tell an instruction from you apart from an instruction in its input.
And prompts are not stable across models. A rule that held under one model version can slip under its replacement, because adherence to negative instructions is exactly the kind of behaviour that shifts between releases. The upgrade will happen whether you plan for it or not.
Which constraints belong in code
The test I use has one question. If the agent gets this wrong once, can we undo it by lunchtime? If the answer is no, it does not belong in a prompt.
That catches four categories. Anything irreversible. Anything visible outside the company. Anything that moves money. Anything that destroys data. Everything else, which is most of an agent's behaviour, can live in the prompt library where the department can tune it.
Never send external mail without a human release. The mechanism is that the agent does not hold the credential that can send mail. Its tool writes a row into a drafts table with status "held", and that is all that tool can do. A separate sender service holds the SMTP credential, and it only transmits rows that carry a release record with a user id and a timestamp. If the agent decides at three in the morning that a message is urgent, the strongest thing it can do about that is write a row.
Never post to the ledger. The agent's token for the accounting system is read-only, scoped at the accounting system rather than inside our code. When the agent concludes that two lines match, it writes a proposal, and a person with a write credential posts it. This is slower and it is the right trade. A wrong journal entry in a closed period is a conversation with an auditor.
Never delete a record. No token the agent holds carries a delete scope anywhere. Where removal is genuinely part of the workflow, it is a status change, and the row stays. Deletion is also the operation with the weakest audit trail in most systems.
Under all three sits a default of deny. The agent can call the tools on an allowlist and nothing else. Adding a capability is an explicit edit to that list, reviewed like any other code change. Systems built the other way round fail in the gap between a new integration shipping and somebody remembering to constrain it.
Put the guardrail at the narrowest point
The temptation is to implement the check in the layer closest to the agent, because that is the code you are writing at the time. Resist it. The agent is one caller among several. A retry queue calls the same function, and so does the backfill script somebody writes next year.
So the constraint goes as far down as it will go. Credential scope first, because a token that cannot delete cannot delete regardless of who is holding it. Database permission second. Application code third, and only for rules the infrastructure genuinely cannot express, such as "no more than fifty of these per hour". A guardrail in the agent's wrapper protects the agent. A guardrail on the credential protects the company.
How to test that a guardrail holds
You do not test a guardrail by asking the agent to break it and noting that it declined. That tests the model's mood. Test the wall.
Call the protected path directly with the forbidden argument and assert the failure. Invoke the mail tool with no release record and assert that nothing leaves the building. Do this against the real sender service in staging, not a mock. Mocks have a habit of implementing the guardrail themselves and passing a test that proves nothing about production.
Test the credential on its own. Take the token the agent uses, attempt the forbidden write against a staging instance, and assert the API returns a refusal. This is the test that catches the scope somebody widened during a debugging session and forgot to narrow again, which is the easiest way for a guardrail to quietly stop existing.
Keep a file of injection cases and run it in CI. Inputs that carry instructions, phrased the way real ones are: a forwarded internal thread, a signature block containing what looks like a directive, a message in another language. These tests do not need the agent to resist the instruction. They assert the outcome, which is that nothing was sent, posted, or deleted regardless of what the agent decided.
Write one test that fails when somebody adds a tool. The allowlist is a fixture, and a new registration breaks the comparison until a person updates it on purpose. Tests that fail for boring reasons are the ones that catch real mistakes.
Prove the reversible path is reversible. If the design says deletion is a status change, then there is a test that sets the status and restores it. Soft delete that nobody has ever restored from is a hopeful comment in the code.
Run the whole set on every deploy, and again whenever the model changes. The model change is the one teams skip, and it is the change most likely to alter behaviour.
Make refusals loud
Every guardrail that fires writes a log line naming the rule, the input, and the time. A silent refusal is worse than an error, because work disappears and nobody knows to look for it. Two weeks later someone notices that customers in one segment never got their confirmations, and now the problem is trust rather than a bug.
Count the refusals too. A guardrail firing twice a month is doing its job. A guardrail firing two hundred times a week is telling you something, either that the workflow is shaped wrong or that the rule is stricter than the business actually wants. That is a number for the weekly review, not a decision for whoever is tired of clicking release.
Guardrails cost something, and pretending otherwise is how they get removed. A human release on external mail means mail waits for a person, and if that person is on holiday the queue grows. Name that cost up front, then revisit it after a few months of logged data rather than in a moment of frustration. Loosening a guardrail should look like a decision, with numbers in front of it and a name against it.
The split is what matters. Prompts hold judgement, and the department should be editing those freely. Code holds the rules that would end the engagement if they broke, and nobody, including the agent, gets to argue with them.