Shadow mode: how agents earn autonomy

Shadow mode runs an agent beside your team, touching no real record, until a named person decides it has earned a gate. Checklist and exit criteria.

The short answer

Short answer

To run an AI agent in shadow mode, have it do the real workflow alongside the person who does it today.

It reads the same inputs and proposes outputs, but nothing it produces reaches a record or a recipient. Mercury Digital records every difference, and a named gate holder decides from that record when the agent earns a gate.

Definition

Shadow mode

A stage of agent deployment where the agent runs the real workflow on real inputs next to the human doing it, with no ability to write, send or pay. Its outputs are recorded and compared with the human outcome. Autonomy opens only on a human decision made against that record.

Shadow mode vs other ways to test an agent

ApproachReal inputs?Can it affect records?What it provesWhat it doesn't prove
Demo / sandboxNo (sample data)NoThe agent can do the task on chosen examplesHow it handles your actual inputs and exceptions
Shadow modeYesNoHow its proposals compare with your team's real outcomesInputs it never saw; what happens when it writes
Gated productionYesOnly after a named person approves each itemIts proposals hold up under real approvalsUnattended behavior (by design)
Unattended for a stepYesYes, for that step onlyIt performs within agreed limitsSteps it wasn't opened for

How to run shadow mode: checklist

01

Pick one workflow and its consequential step

This is usually one step, such as posting, paying or sending. See approval gates. Name the gate holder who will decide, from the record, whether the agent earns that gate.
02

Capture the human baseline

Record what the person actually did with each item, not what the procedure says they should do.
03

Give the agent read-only access

It reads the same inputs. No write, send or pay permissions exist in shadow mode.
04

Run in parallel on real items

The agent reads, assembles and proposes. The person keeps doing the work.
05

Record every comparison

Each comparison is recorded as a match or a mismatch, with the cause of each mismatch (incomplete rule, agent error, or human baseline error). Track coverage: record which input types and exception types have actually come through, and which haven't.
06

Write down the limits

Before anyone relies on the record, write down its limits: unseen inputs, exceptions that never happened, and the fact that it says nothing about writes.
07

Decide at the gate

The gate holder reviews the record against the exit criteria and records the decision: open, extend shadow, or stop.

Limits

What the shadow record can't tell you

Unseen inputs

It covers only the inputs that actually came through.

The human baseline may be wrong

Some mismatches are the person's error. That's a finding, not an agent failure.

Nothing about writes

Shadow mode never writes, so the first gated writes still need close attention.

Where this fits

Concept

In Mercury Digital deployments, every workflow starts in shadow mode and ships gated by default. Under Mercury's standard policy, the agent acts on its own only after it has run in shadow for at least 50 items of each type and at least 2 weeks, with 95%+ agreement with the human reviewer and zero critical errors. Autonomy is then opened gate by gate, on your decision. It's the Deployment phase of Audit → Design → Deployment → Governance. See How It Works.

Exit criteria (Mercury's standard policy, confirmed by the gate holder before shadow starts)

Shadow mode ends on a recorded decision, not a calendar date. Mercury's standard policy sets the minimums below; the gate holder confirms them, or sets stricter ones, before shadow starts, so the decision is made against them:

  • Coverage: every input type and exception type the step routinely sees has come through at least 50 times, over at least 2 weeks of shadow running.
  • Agreement: the agent's proposals match the human reviewer's decision on at least 95% of items, with every mismatch explained.
  • Zero critical errors across the shadow run.
  • Zero unexplained misses on the consequential field (amount, account, recipient).
  • Rules updated: every mismatch caused by an incomplete written rule has had its rule fixed and re-tested.
  • Limits stated: the record says in writing what it doesn't cover.
  • Rollback ready: how to close the gate again, and who can do it, is documented.
  • Decision recorded: the gate holder's name, date and decision (open / extend / stop).

What a shadow record contains

Every item that comes through shadow mode gets one entry in the record. That's what the gate holder decides from.

FieldWhat it holds
ItemThe input the step handled (an invoice, an entry, a request), with its type
Agent proposalWhat the agent would have done, with the evidence it used
Human decisionWhat the human reviewer actually did with the same item
Agree / disagreeWhether the proposal matched the human decision
ReasonFor a disagreement, the cause: incomplete rule, agent error or human baseline error

Coverage, the agreement rate and any critical errors are read straight from these entries.

How It Works

Questions, answered

Run it in parallel with the person on real inputs, with read-only access, and record every proposal against the human outcome. A named gate holder decides from that record, against exit criteria set in advance, whether to open a gate.
AI Opportunity Audit · Shadow mode

Pick the workflow to watch in shadow mode

Pick the workflow you would want to watch before you trust it, and find out whether it leaves the record shadow mode needs. The AI Opportunity Audit is fixed-price, quoted before we start.