Writing / Consulting lessons

Automate the decision, not the click

The valuable part of an operator's job isn't the clicking — it's the bounded decision. A design for automation that retrieves context, enforces policy, calls narrow tools, and attaches evidence.

Most automation projects start by recording the operator's movements. It's easy to watch and easy to demo: open a dashboard, click a filter, copy a value, update a ticket, send a message.

It's also usually the least valuable part of the job.

Nobody's paying that operator to click “review.” They're paying them to look at a few signals, apply policy, decide whether the case is ordinary or an exception, and choose what happens next. A script that reproduces the clicks makes the operator faster. A system that owns the bounded decision removes the queue. That difference changes the architecture.

Start with a decision inventory

Before I pick a model or a framework, I write down the decisions in the existing workflow. For each one, I record:

Field

Question

Trigger

What makes this case eligible for processing?

Context

Which records, messages, pages, or files does the operator inspect?

Policy

Which outcomes are allowed, forbidden, or always escalated?

Action

Which side effects follow from each outcome?

Evidence

What would let another person verify the decision?

Recovery

What happens when context is missing or a tool fails?

This separates two kinds of work. Mechanical work can become a tool. Judgment can become a decision contract. Anything that fits neither form isn't ready for autonomous execution yet.

The decision loop

The model sits inside a controlled loop. It doesn't own the workflow.

flowchart LR
  Trigger["Workflow trigger"] --> Load["Load case state"]
  Load --> Retrieve["Retrieve relevant context"]
  Retrieve --> Policy["Apply policy and permissions"]
  Policy --> Reason["Model proposes decision"]
  Reason --> Validate["Validate typed decision"]
  Validate --> Tools["Execute narrow tools"]
  Tools --> Evidence["Persist evidence and outcome"]
  Evidence --> Review["Complete or escalate"]
  Review --> Memory["Evaluation data"]

The order matters. Retrieval before reasoning. Policy before side effects. Validation after the model and before tools. Evidence persisted as part of the action, not assembled later from logs.

A decision is not a paragraph

Free-form text is a bad interface between an agent and the rest of a product. The output should describe the decision the system is allowed to make, not just explain what the model thinks.

type EvidenceRef = {
  kind: "record" | "screenshot" | "log" | "source";
  uri: string;
  observedAt: string;
};

type CaseDecision =
  | {
      kind: "act";
      action: "route_case" | "create_ticket" | "mark_passed";
      confidence: "high" | "medium";
      evidence: EvidenceRef[];
    }
  | {
      kind: "needs_review";
      reason: string;
      missingContext: string[];
      evidence: EvidenceRef[];
    }
  | {
      kind: "refuse";
      reason: string;
    };

This contract buys three things:

  1. The model can't invent an arbitrary side effect.
  2. Ambiguity becomes a valid outcome instead of an error hidden behind confident prose.
  3. A consequential decision carries an explicit evidence requirement.

The application still validates permissions, state, idempotency, and rate limits. A valid JSON object isn't permission to mutate the system.

Tools should expose capabilities, not internals

A tool should be narrow enough that its preconditions are understandable. “Call the CRM API” isn't a useful boundary. “Create a review case with these fields” is closer.

type Tool =
  | {
      name: "create_review_case";
      input: {
        accountId: string;
        reason: string;
        evidenceRefs: string[];
      };
    }
  | {
      name: "send_internal_notification";
      input: {
        channel: string;
        message: string;
        caseId: string;
      };
    };

The tool implementation owns authorization and idempotency. The model can request an operation; it can't decide whether the current state permits it.

Autonomy needs a boundary

Not every decision should be automated. A practical rollout has at least three lanes:

  • Automatic: high-confidence decisions with reversible side effects and complete evidence.
  • Suggested: a proposed decision a person approves before the side effect.
  • Escalated: missing context, policy ambiguity, sensitive communication, or an irreversible action.

The threshold is a product decision, not a model parameter. A system that's 95% accurate can still be unacceptable if the 5% of errors are expensive or hard to spot.

SuperSquad as an example

SuperSquad applies this to browser QA. A team describes the behavior that matters. The system turns that intent into coverage, runs browser journeys, captures artifacts, and produces a finding only when it can supply reproducible steps and linked evidence.

The browser click is a tool call. The valuable decision is whether the observed behavior violates the product intent, whether the evidence is sufficient, and whether the finding is confirmed or should be escalated. That's why a screenshot alone isn't enough — a useful finding needs the journey, the expected behavior, the actual behavior, the reproduction steps, and enough browser evidence for an engineer to verify it.

What to measure

Model accuracy isn't enough. Measure the workflow:

Metric

What it reveals

Auto-action precision

Whether the system is safe to leave unattended.

Escalation rate

How often context or policy is insufficient.

Evidence completeness

Whether decisions can be audited.

Tool failure rate

Whether external systems are limiting reliability.

Time to resolution

Whether the system actually removes operational load.

Override rate

Where policy or retrieval needs work.

Every automatic action should become evaluation data. If outcomes aren't recorded, the system can't improve past prompt tweaks and anecdotes.

The engineering conclusion

Automation becomes valuable when it owns a bounded decision, not when it animates a macro. The design work is making the boundary explicit: retrieve the right context, enforce policy, expose narrow tools, validate typed output, keep evidence, and provide a safe way to refuse. The model supplies the reasoning; the product keeps the authority.

Bhavin Kamani

A small command line for a curious visitor.