AI Agents Should Know When Not to Decide

I started Paperclip from a very practical annoyance: AI coding agents were getting useful enough that the bottleneck was no longer whether they could write code.

The bottleneck was everything around the code.

Which task is this? Which repository? Which branch? Which agent is working on it? What decision is still pending? What needs review? What is safe to merge?

This is the first part of a three-part series about building Paperclip toward an AI-operated company. This part is about bounded agency: agents knowing what they own and what they should route elsewhere. The next two parts look at blocking and resumability, then at Paperclip as an organizational layer rather than a swarm of agents.

At first, my mental model looked mostly like software development:

    flowchart TD
    task[Task] --> agent[Coding agent]
    agent --> repo[Repository]
    repo --> implementation[Implementation]
  

That naturally led to questions about coding agents, repositories, worktrees, models, execution environments, and isolated development machines.

Those questions still matter. But they are not the whole abstraction.

Paperclip is starting to look less like a task runner for coding agents and more like the organizational layer for an AI-operated company. And in a company, software development is only one kind of work.

    flowchart LR
    company[Paperclip company]
    company --> product[Product]
    company --> engineering[Engineering]
    company --> sre[Infrastructure / SRE]
    company --> research[Research]
    company --> marketing[Marketing]
    company --> publishing[Publishing]
    company --> ops[Operations]
    company --> finance[Finance]
  

Once agents occupy different roles, technical capability and organizational authority can no longer be treated as the same thing.

That is the beginning of bounded agency.

Capability is not ownership

The fact that an AI agent can make a decision does not mean it should.

This sounds obvious when applied to humans.

A developer might be perfectly capable of selecting a production database topology, changing a price, or approving a security-policy exception. That does not mean those decisions belong to the developer role.

AI agents need the same distinction.

The interesting question is shifting from:

Can the agent solve this?

to:

Is this decision actually owned by this role?

That question showed up very clearly in a recent Paperclip implementation around retention policy.

The retention-policy experiment

One product decision was already settled:

    flowchart LR
    completed[Completed items older than 30 days]
    archive[Archive]
    completed --> archive
  

Completed items older than 30 days should be archived rather than deleted.

That is a product decision. The implementation agent does not get to reopen it just because it found the relevant code. Its job is to implement the product behavior, not to relitigate the policy.

During implementation, though, another decision appeared:

    flowchart TD
    question[How should the retention scan run?]
    question --> inapp[Periodically inside the application]
    question --> job[Separate scheduled/background job]
  

A capable coding agent could inspect the repository, pick one, and start coding.

That would be the wrong move.

The scheduling decision touches more than code. It affects deployment architecture, operational ownership, failure behavior, observability, resource use, testing strategy, and production runtime assumptions.

That means the developer role does not fully own it.

The agent should still keep moving

The right behavior is not panic at the first sign of ambiguity.

An agent should continue everything it legitimately owns.

    flowchart TD
    start[Retention implementation] --> lifecycle[Archive lifecycle]
    lifecycle --> config[Configuration]
    config --> restore[Archive / restore behavior]
    restore --> tests[Independent tests]
    tests --> scheduling{Scheduling architecture}
    scheduling -->|owned by developer| implement[Implement directly]
    scheduling -->|authority boundary| consult[Consult Architecture / Platform / SRE]
  

Archive lifecycle? Continue.

Configuration? Continue.

Archive and restore behavior? Continue.

Independent tests? Continue.

Scheduling architecture? That is the boundary.

Only when further useful progress depends on the unresolved decision should the developer consult another specialist.

And the consultation should not be a vague handoff like “what do you think?”

It should include the real implementation context:

• what was already decided;
• what was already implemented;
• what options emerged;
• what constraints the codebase exposed;
• what decision the developer cannot legitimately own.

The specialist should return a concrete decision, the rationale, constraints, and implementation caveats.

Then the developer implements that decision. It does not invent one because it got bored waiting.

This is not agent timidity

Bounded agency is not about making agents weak.

It is about making them organizationally useful.

The worst version of an AI company is not one where agents occasionally ask for help. The worst version is one where every capable agent quietly expands its own authority because the model can produce a plausible answer.

That does not create autonomy. It creates organizational mush.

Healthy organizations are full of boundaries:

    flowchart LR
    dev[Developer]
    arch[Architecture]
    product[Product]
    security[Security]
    sre[Platform / SRE]

    dev -->|implementation within scope| code[Code]
    dev -->|architecture change| arch
    dev -->|product behavior change| product
    dev -->|security exception| security
    dev -->|production topology| sre
  

The point is not to slow everything down. The point is to make sure decisions are made by the role that owns the consequences.

For Paperclip, that means roles need more than prompts. They need mandates. They need ownership. They need escalation paths. They need a way to say:

I can technically do this, but I do not own this decision.

That sentence is a feature, not a failure.

The goal is not to build agents that can make every decision.

The goal is to make sure every decision is made by the right agent.

The next problem appears immediately after that.

Once the developer consults another specialist, it may be blocked for minutes, hours, or days.

So what should an autonomous agent do while it waits?

That is where Part 2 picks up: blocked work, durable state, and why autonomous agents need to know how to stop. Part 3 then zooms out from individual agents to the larger question of Paperclip as the coordination layer for an AI company.