Justin Ng
← Writing

An agent waiting for permission that would never come

My AI code reviewer ran for six hours doing nothing. In CI, every permission prompt is a hang.

· 2 min read· Operate

Our finance app has an AI code reviewer that runs on every pull request in GitHub Actions. A normal review takes 4 to 6 minutes. One afternoon I noticed a run that had taken 33 minutes, and when I looked closer, an earlier one had run for six hours, until GitHub's job limit killed it.

Neither run was doing anything for almost all of that time.

What happened

The reviewer worked normally for about a minute and a half. Then it tried to read a file in /tmp, outside the repository checkout. The agent's permission setting for directories outside the project defaults to ask: pause and wait for a human to approve.

In a terminal, that's a sensible default. In CI, there is no human. The agent asked, nobody answered, and it waited with no output until the run was cancelled or timed out. The model wasn't even being called; I was paying for a runner to sit still.

The fix

  1. Turn every "ask" into "deny" in CI. Outside-directory access and the agent's loop-detection prompt both defaulted to ask, and in CI any ask is a hang. Denying means the request fails immediately, the agent sees the error, and it carries on with the review.
  2. Tell the agent where to work. One line in the review prompt: stay inside the checkout, and read the diff with git diff.
  3. Add a time limit as a backstop. The agent first proposed 15 minutes. I pushed for less, and it checked the history instead of guessing: 26 of the last 29 successful reviews finished within 8 minutes. We set 10. An occasional slow review may now fail, but a stuck one costs 10 minutes instead of six hours.

The pull request with the fix tested itself: its own review ran with the new settings and finished in about 2 minutes.

The same week, a quieter cost

While I was looking at CI spend, the iOS app's test workflow stood out. It ran on GitHub's macOS runners, which are billed at a multiple of Linux minutes, and it ran on every push, including pushes that only changed docs. We moved the tests to Xcode Cloud, triggered only when app code, tests or the project file change, and removed the macOS job from GitHub entirely. The repo's rules file now says not to add macOS jobs back, and why, so the next agent session doesn't helpfully reintroduce one.

What I took away

  • Permission defaults are designed for a human at a keyboard. When you move an agent into an unattended environment, go through every setting that can ask and decide whether it should allow or deny. Leaving it on ask is a decision too: to wait forever.
  • Fail fast beats fail safe when nobody is watching. A denied request the agent can see and work around is better than a polite pause no one will answer.
  • Timeouts should come from data. "15 minutes sounds generous" was a guess; the run history gave a number.
  • Write the reason down where the next agent will read it. A rule without a reason gets "fixed" by the next session that doesn't know why it's there.