Claude Code can write code, but engineers still need a workflow they can trust. Kaido Koort explains how specs, testing harnesses and validation make agent delegation repeatable. The conversation covers Plepic's six-Friday curriculum, agent-led interviews, competing validator sub-agents, session handoffs and parallel work in Git worktrees, plus the limits of Claude certifications and the attention cost of managing multiple agents.

Spotify Apple Podcasts RSS

Your job is managing agents now

If you use Claude Code every day, you already feel this. Most of your day goes on writing context and making decisions, not writing if statements. Kaido Koort has spent 20+ years in product and engineering roles and now runs Plepic, which trains experienced engineers to work this way. He describes it as developers being pushed into becoming agent managers. You guide agents in plain English, and the skill is “a fusion of the engineering work and then like a manager role”. You still need to know what good code looks like. But what you produce is context, decisions and verification.

Not everyone is happy about it. In episode #18, Julien Bisconti’s reaction came down to “this is not what I signed up for”. If you enjoyed writing code by hand, the new job can be less fun. We think the rewards are real, but they sit somewhere else. They come from building the harness that makes agents trustworthy, not from typing the code yourself.

So how do you learn the new job? What you want is a workflow you can explain, check and hand to a colleague. That’s what your team notices on a Tuesday afternoon, and it’s what engineers ask for when they sign up for training.

Engineers want structure and a protected day

What engineers ask for in training is almost entirely practical.

What do engineers actually want from agentic AI training?

They want help with the projects they’re already working on, and protected time to learn, because nobody has time to keep up alone.

Plepic started in 2025, soon after Claude Code came out, by running hackathons for development teams. There were eight or nine of them, each with 20 or more people. The feedback was consistent. Developers wanted a structured course and more time, because they’re busy at work and everyone’s head is spinning. A new model or technique appears every couple of weeks, and nobody has room to absorb it.

The format came out of that feedback: six Fridays, 37.5 hours, about 20 participants, with two or more years of engineering experience required. Friday is the day that’s easiest to take away from normal work.

The experience rule exists because of vibe coding. With so much hype around, it’s hard to tell a real engineer from someone who threw a solution together overnight. Mixing the two slows a cohort down, much like a class slows when one student is still learning the language it’s taught in. Kaido isn’t strict about it, though. He counts any kind of engineering experience. He regularly hears from analysts who have never coded professionally and want to know if they can join. What matters is engineering fundamentals, not job titles.

Seniority doesn’t predict progress either. Kaido has seen experienced engineers who resist the change move more slowly than keen juniors who are excited about AI. Some of the strongest capstone projects came from non-coders who have engineering fundamentals and aren’t afraid of the agent.

Graduates report a median speed-up of five to six times on their capstone projects. That number is self-reported, and it’s presented that way. Demand is growing. There was one cohort in each of the first three quarters, and five or six are planned for Q4. The bottleneck now is finding instructors.

Six Fridays, from brownfield to parallel agents

The course order is worth copying even if you never sign up. It follows the order in which a team builds trust in agents.

What should an agentic engineering curriculum cover first?

Safety, and getting oriented in a codebase you didn’t write. Features and parallelism come after that.

The first four Fridays simulate joining a brownfield project. The previous lead developer has left, and you inherit about 150,000 lines of code. Everyone works on the same unfamiliar codebase:

  1. Harness and safety. Set up a dev container, turn the friendly fire off, and run Claude Code in YOLO mode inside that boundary.
  2. Making sense of the code. Build layered summaries and a C4 architecture model of the codebase, and have the agent write tests across the testing pyramid.
  3. Spec-driven build. The agent implements a feature, and nobody reads the code. Engineering loops let agents work on their own, which only works when you have a target you can measure the agent’s success against.
  4. Parallelism. Several agents work on one codebase at once, each in its own Git worktree.

The last two Fridays are for the capstone. You pick a problem or project of your own, commit to it in front of the cohort, and then present what you built, what problems you solved and what you learned. Seeing twenty other engineering projects is part of the value.

It isn’t an advanced course, and it doesn’t claim to be. It gives a solid foundation, and the order matters. Day three’s “never look at the code” only works because days one and two built the safety boundary and the test coverage it relies on.

For Estonian employers, the cost can drop sharply. On air, the price was given as €2,500 plus VAT per developer. The training page currently lists €2,520 plus VAT. The Unemployment Insurance Fund, Töötukassa, runs a training grant for employers developing employees’ ICT skills. It covers 80 percent of the invoice, capped at €2,500 per employee over three years. Plepic says you must apply before training starts.

Kaido’s figure of €20.16 per developer depends on two assumptions. First, the 80 percent applies to the VAT-inclusive invoice: 80 percent of €3,124.80 is €2,499.84. Second, the company then recovers the full €604.80 of VAT. Banks and insurers typically can’t recover VAT, and the grant uses up that employee’s allowance for three years.

Let the agent interview you, then let validators argue

Day three depends on the spec. This is where Kaido’s process and ours turned out to be almost the same.

Who should write the spec, you or the agent?

The agent. Switch roles, so the agent interviews you and writes the spec, and a set of validator sub-agents raises the contradictions for you to resolve.

The process Plepic teaches works like this. The agent asks you about what you want to build. It writes the spec and an implementation plan. Then you start around ten validator sub-agents, with whatever lenses fit: security, domain, code quality, performance, simplification. Each one reviews the spec and reports decisions that conflict. The security validator insists on two-factor authentication. The simplification agent calls it over-engineering. You make the trade-off, as engineers always have, and the agent rewrites the spec.

The review doesn’t need to drag on. A couple of hours is usually enough, because by then the plan has been reviewed so thoroughly. You should still read it yourself. Most people use one of two skills for the interview: Matt Pocock’s grill-me, or the brainstorming skill in obra’s Superpowers. The principle is the same in both. The results improve when the agent is well connected: it knows your codebase and coding standards, and it can reach Jira, Confluence or wherever your team keeps its knowledge. That’s the point about context we made in episode #17.

Our version is nearly identical. The agent explores first, then interviews us. Sub-agents and other harnesses then review the plan to find gaps. The finished plan goes to a fresh session for implementation. We prefer this to the Kiro spec workflow, which was early to specs. Kiro turns a prompt into requirements.md, then design.md, then tasks.md, and you iterate at each stage. It works. Our problem is that you have to take in everything at once, and the agent spends too little time preparing before it talks to you. With the interview approach, the agent does its reading before it asks you anything.

Handoffs keep sessions clean, and wrap-ups catch loose ends

A good spec gets you through implementation. It doesn’t cover what you forget between sessions.

How do you stop small decisions getting lost across agent sessions?

End each piece of work with a wrap-up step. It reads the transcripts for open questions and files each one as an issue.

We don’t run the whole flow in one session. A handoff skill saves the current state to a file, and a clean session picks it up from there. Each handoff records the sessions before it, so every new agent knows it’s continuing work that has run across several sessions. It’s the same deliberate handoff at the context limit we described in episode #18.

The newer piece is a wrap-up skill. It checks that everything is merged and the worktrees are cleaned up. Then it reads the session transcripts for anything left open: a question from the interview, a point raised in review, code the session touched and flagged as a problem. Each one becomes an issue. You pick up the next batch of issues, which may produce more work, and the loop continues.

It can feel endless. But the codebase does get better, because the loose ends people complain about are the small things nobody wrote down. We expect the skill to change again within a couple of weeks. The idea should last longer.

This way of working also changed our standing instructions. A global CLAUDE.md used to feel unnecessary. What changed our minds was the agent referring to issues and decisions by number: “decision 4”, “lens 2”, a bare issue number. When you switch between five sessions, you come to each one with a fresh mind. The instruction now says: whenever you show me something, give the full context. That means what happens today, what changes, the pros and cons, and the issue’s title rather than its number.

Verification checks the spec, validation checks the problem

Agents have multiplied the amount of code in the world, so the gates matter more than they did.

What’s the difference between verification and validation for agent output?

Verification checks the implementation against the spec. Validation checks whether the solution solved the problem you started with.

Kaido puts agent-written code at 70 to 80 percent, and says the number of lines out there has multiplied. You need both gates. A verification gate compares the implementation with the spec, so it’s only as good as the spec. A validation gate asks whether the result solved the original problem. A feature can pass the first gate and fail the second. A green CI run won’t catch that.

How strict to be depends on what’s at stake. For a hobby project, vibe code it. If it works, fine. If it doesn’t, drop the thread and move on. For business-critical systems, or anything where lives are at stake, you can’t trust the agent blindly. Either you keep reviewing the code, or you invest in a testing harness mature enough to reject work that fails your team’s standards: coding standards, the linter, end-to-end tests, performance results.

This is also why the course order matters. Teams that jump straight to “never look at the code” without the gates are the ones stuck in that review queue.

Measure where you are, and pace yourself

With gates in place, the next question is how far to push. Conference talks are a poor guide. Kaido has learned to tell marketing talk from what teams actually do.

How do I measure my own agentic AI adoption?

Rate yourself on three separate axes: how much autonomy you give the agent, how many agents you run in parallel, and how good your skills are.

Plepic’s self-assessment at skill.plepic.com has three branches. Autonomy is how far you trust the agent to act alone. Parallelism is how many agents work on one codebase at once. Skill usage covers what tools the agent has, how good your defined skills are, and how well you reflect on them and improve them over time. Each level describes how you work at that stage, what makes it hard and how to reach it, and you tick what matches you. We’d have liked a questionnaire that gives you a score, but seeing where you sit on each branch is useful too. Kaido doesn’t know of anyone else doing this, and he wants disagreement: maybe it needs four or five branches, or a graph.

His own parallelism level is “manual parallel”: two to five agents in the Claude Code agents dashboard. You give one agent a command, press enter, switch to the next, and keep cycling.

The hope was that we’d work less. Instead you work on five projects and three features at once, making decisions the whole time. If more agents are running and your afternoons are getting worse, the constraint is your attention.

That tiredness is the price of learning on the first wave. On the second wave, things are more mature. If you arrive late, things work more smoothly, but you understand less about why they work. Some of what we learn this year will be obsolete in a year or two. The understanding stays. People from DevOps have an advantage here, because new tools have always kept arriving, and most LLM development is automation. You’re writing shell scripts in plain English now.

The Claude Developer exam tests fundamentals, not your workflow

Completing a course and passing a vendor exam are different kinds of evidence. Plepic’s course ends with a certificate of completion, which says you practised the workflow above over six Fridays. Anthropic’s new certifications say you answered questions about the material correctly.

Is a Claude certification worth taking?

Take the Developer exam if you want proof that you understand how LLMs behave and how to build on the API. Don’t read any of the exams as proof that you can run agents on a real codebase.

Anthropic’s program is delivered through Pearson VUE, online or at a test centre, as AWS exams are. There are four exams: Claude Certified Associate Foundations, Developer Foundations, Architect Foundations and Architect Professional. Pearson VUE’s page also says the program is open to organisations in the Claude Partner Network. Check that against your own situation before you plan around it.

One of us took the Developer exam: 53 questions in two hours, and not very complicated. What prepared us was a year or two of building with LLMs, plus the simple chat application from Anthropic’s Building with the Claude API course. The questions we got were mostly about fundamentals. How compaction works. Which model to pick for classification and which for open-ended work. Who stores the conversation in a chatbot. The answer to that last one is you, the developer. The Messages API doesn’t keep conversation state for you, so your application sends the full message history with every call.

The Architect exam is harder: 60 questions, most of them multiple-answer, and we know people who failed it. We haven’t taken it, so we can’t say what it feels like. Its scope is wider on paper. The Architect Foundations page says it tests Claude Code, the Claude Agent SDK, the Claude API and MCP, and its exam guide gives Claude Code configuration and workflows 20 percent of the weighting. So workflow knowledge is tested, but answering questions about a workflow isn’t the same as running one on 150,000 lines of someone else’s code.

Some Developer questions still need work. One asked what to do after deploying a new AI-enabled application, with options like running CI, running end-to-end tests or reviewing the pull request. Honestly, it depends on the organisation. After the exam you can leave feedback on every question, so there’s a channel for fixing questions like that.

Kaido has the same problem with his own curriculum. Plepic revises its course every two to three weeks, using feedback collected after the first four days and again at the end. Any fixed set of 60 questions is chasing a moving target.

Assistant prefill is model-specific history

One technique from the exam course shows how quickly this material goes out of date.

How do you force Claude to return JSON?

On current Sonnet and Opus models, use structured outputs. The prefill trick from the course returns a 400 error on those models, though Claude Haiku 4.5 still accepts it.

The course taught a hack. You add a half-written assistant turn to the conversation that opens a JSON code fence. The model assumes it has already started answering in JSON and carries on. You set a stop sequence on the closing fence and cut the output there. When we first saw it, it felt awful. You’re tricking the model into thinking it already agreed. It’s close to prompt injection, just aimed at your own request.

Newer models reject it. Anthropic’s Sonnet 5 migration guide says that “Prefilling assistant messages returns a 400 error on Claude Sonnet 4.6 and later models”. The Opus 5.5 migration guide says the same for “Claude Opus 4.6 and later Opus models”. The Sonnet guide also notes that prefill “works on Claude Haiku 4.5”. Both guides point to structured outputs, system prompt instructions or output_config.format instead.

Structured outputs constrain the response to your schema, with documented exceptions. A refusal can come back outside the schema, and a response that hits max_tokens gets cut off partway through. Check stop_reason before you parse anything. Agent SDKs offer the same idea through typed models. Strands Agents, for example, supports typed structured output, and that’s the approach we use.

The lesson isn’t really about JSON. A course is a snapshot, and the API keeps moving. Check anything a course teaches against the current docs for the exact model you run before it goes into production code.

What this means for your team

A certificate or a seat licence doesn’t make someone an agentic engineer. A workflow you can explain, check and hand to the next person does. The hard part is the time. Teams have to balance a culture of shipping with a culture of learning, and that means taking developers away from keeping the lights on long enough to change their habits. Four things to try this week:

  • Switch roles on your next feature. Have the agent interview you with grill-me or Superpowers brainstorming. Then run several validator sub-agents with opposing lenses, such as security against simplification, and resolve their conflicts yourself.
  • Add a wrap-up step. Before you close a piece of work, have an agent read the session transcripts for unresolved questions and file each one as an issue.
  • Separate your gates. Write down which check verifies against the spec and which one validates against the original problem. If you only have the first, you can ship features that meet their spec and still miss the problem.
  • Rate yourself on autonomy, parallelism and skills separately. Then work on whichever branch is weakest for you.

Common questions, answered

How do I prepare for the Claude Developer certification exam?

Build with LLMs first. What prepared us was a year or two of building with LLMs, plus Anthropic’s Building with the Claude API course and the simple chat application it walks you through. The exam has 53 questions in two hours and focuses on fundamentals. Expect questions on compaction, on choosing a model for classification or for open-ended work, and on the fact that your application sends the conversation history with every Messages API call. The Architect exams are harder and cover more, including Claude Code, so budget more time for those.

Why does my Claude API request with a prefilled assistant message return a 400 error?

Because assistant prefill isn’t supported on Claude Sonnet 4.6, Claude Opus 4.6 or later models. Switch to structured outputs, system prompt instructions or output_config.format. Claude Haiku 4.5 still accepts prefill, so the old trick may keep working there. Structured outputs constrain the response to your schema. A refusal or a response cut off at max_tokens can still fall outside it, so check stop_reason before you parse.

How do I write a spec for Claude Code without spending days on it?

Let the agent interview you, using a skill such as grill-me or Superpowers brainstorming. Then have it write the spec and implementation plan. Run validator sub-agents with different lenses, such as security, performance and simplification, and resolve the conflicts they raise. A couple of hours is usually enough. Read the final plan yourself, then hand it to a fresh session to implement.

Resources

Join B.O.R.I.S Slack Playground