Label In, Production Out: A Claude Code + GitHub Actions Pipeline

For the last few months, my development loop on yesalittle looked like this. I’d refine a story with Claude in chat until we had a handoff spec. I’d paste that spec into Claude Code. I’d open the PR, kick off an AI review loop by hand, read the result, and merge. It worked well, but I was the glue between every step: copying, pasting, starting things, waiting.

So I set out to remove myself from the middle. The goal: I write the story, apply one label, and the next thing I see is the change in production, with Claude Code in GitHub Actions doing the work in between. Or, when something goes wrong, a clear note telling me why.

It works now. Getting there took two days, about fifteen PRs against my own repo, and a surprising amount of human judgment. That irony is most of the story.

What I ended up with

I’m a solo developer, so the shape is simple:

I write a story as a GitHub issue using a Story template: Summary, Acceptance criteria, Spec, Out of scope.
I apply status:ready-for-dev.
A dev agent, Claude Code running in GitHub Actions on Sonnet 5.5, reads the issue, implements it on an agent/issue-N branch, runs the build, opens a PR, and requests auto-merge.
A review agent, a separate Claude Code job on Opus 5.5 with a fresh context, judges the diff strictly against the issue’s spec and writes a pass/block verdict.
GitHub merges the PR only when three required checks pass: CI, the Vercel preview deploy, and the review.

Merging to main deploys to production. The issue closes itself and the branch is deleted.

The last test story replaced a unicode ear emoji with a proper headphones icon. I labeled the issue, and a few minutes later it was live. The agent built a small reusable component, noticed a second ear in an old prototype file outside the build, left it alone, and said so in the PR description. The reviewer passed it and listed a couple of judgment calls as non-blocking for me to read later. I didn’t touch anything between the label and production.

The part nobody puts in the demo

Building a pipeline with limited human involvement took a lot of human involvement.

Not much typing. Claude wrote nearly all the YAML, and Claude Code made most of the edits. What I did was judge. I had Greptile on a trial, and it reviewed every workflow PR I opened. Its findings fell into three buckets:

Real bugs I would have shipped. The best catch: the review gate trusted a verdict file in the workspace, so a PR could have committed a stale "pass" and slipped through if the reviewer failed to write a new one.
Real but low-stakes. Using || true hid label-update failures.
Confidently wrong. It flagged my tightened permission check as a P1 for doing exactly what its previous finding had asked for. Another time it insisted PRs opened by the agent would never trigger CI, which two earlier runs had already disproven.

Sorting those buckets was my job, and it’s the job I don’t think goes away. An AI reviewer chasing a perfect score will happily send you around in circles. That shaped how I built my own reviewer: it blocks only on a fixed list, and it’s told not to invent requirements. More on that below.

The other recurring human task was writing good specs. Every time the pipeline did something surprising, the cause traced back to the spec or the permissions, not the model. A one-line meta tag change taught me to say “the text of the <title> element” rather than “the content value.” A file move taught me to give an exact glob. The agents are only as precise as the story.

The build, step by step

I did it incrementally, and I’d do it that way again. Each step was usable on its own.

1. A merge gate. Before any agents, I made main defensible. A CI workflow runs npm run build, which already includes the tests, both typechecks, the Vite build, and content-page generation. A ruleset on main requires a PR plus two passing checks: verify (CI) and Vercel (the preview deploy). Squash merges only, auto-merge allowed, head branches deleted automatically. I’m the repository admin, so I’m on the bypass list for emergencies. Agents are not.

2. Credentials. I installed the Claude GitHub App on this one repo and stored an OAuth token from claude setup-token as a repo secret, so agent runs bill against my Claude subscription rather than API credits. The important idea: in Actions, the agent acts as claude[bot], not as me. It can’t use my bypass, its work is attributed to it, and its token is scoped to one repo.

3. Labels as a state machine. status:ready-for-dev → status:in-progress → status:in-review, with status:needs-human as the off-ramp. The workflows move the labels; I only apply the first one.

4. The dev agent. A workflow triggers on that label, but only when I’m the one who applied it. It runs one story at a time, with a 45-minute timeout and an 80-turn cap. The agent’s tools are an explicit allowlist: file edits, npm ci, npm test, npm run build, git, mkdir, and a handful of gh commands. If the spec is unclear, it’s told to ask questions on the issue rather than guess. A final shell step checks whether a PR actually exists, rather than trusting the agent’s account of itself.

5. The review agent. It runs on every push to an agent/issue-* PR from my own repo, and nothing else. It reads the issue and the diff and sorts findings into blocking and non-blocking. Blocking means an unmet acceptance criterion, a spec violation, an out-of-scope change, a likely bug, new logic without tests, or a security problem. Everything else is non-blocking. A plain shell step, not the model, decides pass or fail: it requires a freshly written JSON verdict with exactly the expected shape, and anything missing or malformed counts as a block. That job is the third required check, claude-review.

I then opened a deliberately bad PR on an agent-named branch, touching a file the story declared out of scope. The reviewer caught all three violations and the merge button stayed disabled. A gate you’ve only seen pass isn’t tested.

6. Auto-merge, with an off switch. A repository variable, AUTO_MERGE, controls whether the dev agent requests auto-merge after opening its PR. Set it to false and PRs wait for me. I’m honestly not sure which mode I’ll keep. Merging is one click, and it’s the last moment I look before production. But watching a story go from label to live with nothing from me is compelling.

Gotchas, for future me

Each of these cost me at least one confused PR.

A ruleset with no target branch matches nothing. It looks configured and enforces nothing. Check that the Required badges actually appear.
Actions taken with GITHUB_TOKEN don’t trigger other workflows. GitHub suppresses them to prevent loops. My first auto-merge step enabled auto-merge with that token, so the merge was attributed to github-actions, and the linked issue never closed and the branch never got deleted. Having the agent request auto-merge under the Claude app’s identity fixed both.
claude-code-action refuses to run when a bot triggered the event. The reviewer runs on PRs opened by claude[bot], so it needs allowed_bots: "claude". Name exactly that bot rather than *.
“Re-run jobs” reuses the old workflow file. After fixing a workflow, close and reopen the PR to get a fresh run.
Tool denials are silent. The first file-move story failed with zero explanation: two permission denials (almost certainly mkdir) and the agent quietly gave up. Two fixes: the prompt now requires a comment on the issue explaining any stop, and show_full_output: true exposes the full transcript while debugging. My repo is public, so I turned that back off. Anyone can read Actions logs.
pull_request workflows run the PR’s own copy of the workflow. An agent PR could edit the reviewer that’s judging it. A fail-fast step catches accidental .github/ changes, but the real protection is a CODEOWNERS file plus “Require review from Code Owners” in the ruleset. Any change to .github/ needs my approval, and a workflow edit can’t remove that. My own workflow PRs now go through the bypass, which is what the bypass is for.
Fail closed everywhere. When something unexpected happens, the default is to block rather than allow. In the review gate, I want a missing verdict, an error, or anything weird to stop the merge, never to let it through by accident.
Skipped required checks count as passing. Counting skipped checks as passing is what allows my own PRs and Dependabot’s skip claude-review without getting stuck.
A concurrency group holds one running and one waiting run. Label three stories at once and the middle one’s queued run gets cancelled. Its issue keeps the ready-for-dev label with nothing coming to pick it up. Label one story at a time, or re-apply the label on anything that got dropped.

Cost

The repo is public, so Actions minutes are free. The agents run on my existing Claude subscription. The dev agent uses Sonnet 5.5, since it does the token-heavy exploration. The reviewer uses Opus 5.5, because it reads a short spec and a small diff, and its judgment counts most. A different, stronger model also shares fewer of the author’s blind spots. I’ll watch usage over the next few weeks and drop the reviewer to Sonnet if it eats too much.

Greptile’s trial ended partway through. I’m keeping its free plan as an advisory second opinion on my own PRs. It turned out to be most valuable exactly where my Claude reviewer never looks: changes to the pipeline itself.

What’s next

An auto-fix loop. A blocked review currently means I comment @claude on the PR with the fix. Eventually the dev agent should take another pass on its own, capped at a few rounds.
Spec automation. That was the original step 2 of my plan, and I’ve deliberately parked it. Spec quality turned out to be the biggest lever in the whole system, so I want a stack of real stories behind me before I hand spec writing to an agent too.

For now: write the story, apply the label, and go do something else.