Technology

Three things an AI agent must never do in a lending workflow

Automation belongs everywhere except the acts a regulator holds a person accountable for.

There is real work for automation in this process. Chasing missing evidence, walking a borrower through an approval, following up on stalled draws, drafting a resolution summary — all of it is repetitive, and all of it is currently done by people who would rather be doing something else.

There are also three acts where automation should be refused outright, and being specific about which is more useful than a policy about responsible AI.

The three

  • Signing an attestation. An attestation is a statement by a party. An agent is not a party.
  • Authorizing a release. This is the act a regulator will ask a named person to account for.
  • Resolving a dispute. A consumer whose complaint was closed by software has, correctly, not been heard.

These are precisely the acts that consumer lending enforcement in this market is about. They stay with an accountable person.

Where the refusal has to live

Not in a system prompt, and not in a route guard. A prompt is guidance, and a route guard protects the routes someone remembered to guard. The refusal belongs in the data layer, where it also covers the endpoint nobody has written yet — which matters more than usual here, because agent capability is expanding faster than anyone's threat model.

An instruction not to do something is not a control. A system that cannot do it is.

What good automation looks like in between

  • Its own identity. An agent acts as itself and appears in the record as itself, never as the person who asked. "Which agent did this, on whose instruction" has to be answerable.
  • Read-only by default. Every state-changing request refused before it reaches a handler, so the exceptions are deliberate and enumerable.
  • Propose, do not decide. For anything requiring judgment, the agent drafts and a person approves. The proposal is frozen once raised, or approval means nothing.
  • No self-approval. An automated account cannot approve an automated proposal. Otherwise the review queue manufactures the appearance of oversight, which is worse than having none.

That last point is the one most easily got wrong. A reviewer column full of service accounts reads as scrutiny to anyone skimming.

The practical version

Ask a vendor to name the acts their automation cannot perform, and where that refusal is enforced. Vagueness here is the answer.

Related

All 20 articles

Ask us for the audit pack.

Pick a closed project. We will show you the document you would hand a regulator. If it does not answer the question, nothing else matters.