Five rungs, one moving part: where the human sits relative to the loop. At every rung, the remaining human job is judgment; each rung moves where judgment is applied.
| Rung | Name | Your role | Judgment applied to | The gate to stand here |
|---|---|---|---|---|
| 1 | Chat | Drive every turn | Every sentence | You instinctively verify factual output |
| 2 | Prompt workflows | Chain the steps by hand | The pipeline design | You reuse instead of rewriting — and feel the copy-paste friction |
| 3 | Supervised action | Approve each action | Each real-world action | You act safely AND can state where your data goes |
| 4 | Single-purpose agent | Set the goal, review the result | The handoff and the review | You trust the gate enough not to watch — but still review |
| 5 | Autonomous / multi-agent | Set guardrails, audit after | The system's boundaries | You can recognize and govern (building optional) |
"Produce a weekly summary of what's happening in our field and send it to the team."
| Rung | Tools | What it looks like | Who fires each action |
|---|---|---|---|
| 1 | None | Paste 3 articles in, get summary; you find and send everything | You — every real-world step |
| 2 | None | Saved reusable templates; you still paste in and copy out | You — you are the wiring between steps |
| 3 | Web search, email | AI searches → you approve → drafts → offers to send → you approve | You, each action |
| 4 | Same tools | "Every Friday, compile the summary" — runs its own searches, reads last week's notes, drafts | The AI; you set the goal, review before send |
| 5 | Same tools | Runs Friday alone — gather, fact-check, format/send agents; you see it after, or on a guardrail trip | The AI runs the loop; you set boundaries, audit |
Read the table vertically: rungs 1–2 have zero tools (a sealed box). The same tools appear at rung 3 and never change through rung 5. The only column that keeps moving is the last one — whose hand is on the trigger. That is the entire framework.
The model is stateless and its attention rots — so at every rung, your job includes deciding what deserves to be on its desk. By hand at rung 1 (prompting); by design at rung 2 (templates); by retrieval at rung 3 (RAG); by memory and compaction at rung 4; by policy at rung 5.
Applying it: pick one recurring task from your own work. Rebuild the second table for it, rung by rung — the worksheet is the blank. Climb only when a rung's gate is true of you, and remember that every gate is a restraint: verify, reuse, account, review, audit. Capability and judgment climb together, or the ladder tips.
Every term in the framework, grouped by the rung where it first matters. Elsewhere on this site, terms appear only when needed; here, everything appears, defined completely, whatever your role or reading path.
{{ t.term }} {{ t.def }}
Several ideas deliberately reappear up the ladder in stronger forms: logging (r3) → observability / audit log (r5); prompt injection (r3) → indirect prompt injection (r4); context window (foundations) → context engineering (r2) → RAG (r3) → memory, compaction, caching (r4).
If you remember only two things: human-in-the-loop vs. human-on-the-loop — the whole autonomy story in two prepositions — and grounding vs. hallucination — the reason tools exist. Everything else in this glossary hangs off one of those two.
Reasonable question; short answer: there are several public framings, none of which does the Judge Ladder's specific job — describing an individual's changing role relative to the task loop. Here is how the ladder sits against the ones you are most likely to encounter.
OpenAI has described a five-level progression: Chatbots → Reasoners → Agents → Innovators → Organizations. It is the framing you will most often meet in the press. The essential difference: OpenAI's levels describe what AI systems can do, culminating in AI operating at the scale of whole organizations. The Judge Ladder describes what one person does — where a single worker's hand and judgment sit as they delegate one task. The frameworks answer different questions, and both questions are real.
| Judge Ladder | OpenAI levels |
|---|---|
| Rung 1 (Chat), Rung 2 (Prompt workflows) | L1 "Chatbots" (with L2 "Reasoners" as the improving engine underneath) |
| Rung 3 (Supervised action) | L3 "Agents" |
| Rung 4 (Single-purpose agent) | L3 "Agents" |
| Rung 5 (Autonomous / multi-agent) | No clean equivalent |
Rungs 3 and 4 both land on OpenAI's "Agents." Their L3 does not distinguish approving-each-action from delegating-the-goal. The Judge Ladder splits exactly there — because that is precisely where individuals get confused, and where the practical experience of using AI changes most. The ladder is deliberately more granular exactly where people need granularity. That is a feature, not a disagreement.
Rung 5 maps to nothing. OpenAI's L4 ("Innovators") and L5 ("Organizations") describe model capabilities that do not yet exist in anyone's hands. Rung 5 describes a practice that exists today: multi-agent pipelines running under guardrails with humans on the loop. A capability forecast and a practice description simply pass each other at that altitude.
Shapiro's five-level model of AI-assisted software development (consciously modeled on the NHTSA driving-autonomy levels, running to a fully "Dark Factory" end state) is the framework closest to the ladder in spirit: it is organized around the human's changing role, not the machine's capability, and its narration of how each level feels to the person living it is excellent. The difference is scope: Shapiro's levels are specifically about writing software. The Judge Ladder generalizes the same instinct to any recurring knowledge task. If you are a developer, read his levels as a domain-specific companion piece.
A list circulates widely on LinkedIn in several variants: use AI to do this for me / do this with me / help me think / find my blind spots / challenge my thinking. It doesn't fit on the ladder — and that is the interesting thing about it. The five asks classify the type of intellectual work you request; the ladder classifies the autonomy arrangement you delegate under. The axes are orthogonal, and every combination is real: you can ask "challenge my thinking" of a sealed rung-1 chat, and a rung-5 pipeline can exist whose entire job is finding blind spots in your team's weekly output. Used as a cross-cutting lens, the list is genuinely useful at any rung. Just never mistake it for a level: it is a second dimension, not a higher rung.
| Framework | Organizing principle | Scope | Relation to the ladder |
|---|---|---|---|
| Judge Ladder | Where the human sits relative to the loop | Any individual's recurring task | — |
| OpenAI five levels | AI capability | Field-wide / organizational | Vocabulary bridge; rungs 3–4 both = their "Agents"; rung 5 has no equivalent |
| Shapiro's five levels | The human's changing role | Software development | Closest in spirit; domain-specific companion |
| "Five asks" list | Type of intellectual work | Any single request | Orthogonal axis; a lens for every rung, never a rung |