AI coding agents write the code, leaving engineers to prove their value. Julien Bisconti explains how context, verification and domain expertise turn faster output into useful work. The discussion covers reference implementations, review limits, changing harnesses, AWS and Google Cloud context, transcript security, and the limits of token spend as a productivity measure.

Spotify Apple Podcasts RSS

Summary

Spotify Apple Podcasts RSS

A year ago the job fit on one line. Cloud engineer, SRE, the person who makes the containers run and keeps them running. Now the code gets written by an agent, the first three reviews are done by more agents, and the honest answer to what you did all day has gone fuzzy.

Julien Bisconti calls that an identity crisis, and he is not being dramatic. He is a Stockholm-based software engineer and Site Reliability Engineer who works through his own consultancy, Eijlnu Software AB, a Google Developer Expert for Google Cloud specializing in Go, Kubernetes and distributed systems, maintainer of the awesome-docker list, and a former co-host of The DevSecOps Talks Podcast. “When I see how my job has changed since a year ago, I would say it’s a little bit of an identity crisis right now. So let’s see in one more year what my job title would be.”

Almost none of the constraints that bite in a working day are about the model any more. They are about consistency, about context, and about knowing which problem is worth solving. The work got faster and the beach never arrived. What arrived instead was ten projects in parallel, all of them harder than the one project any of us would have taken on before, a pile of generated text somebody has to read, and a set of dashboards that count what is easy to count.

Consistency beats configuration

Start with the machine, because that is where the first chunk of time goes missing.

Does a better setup make you faster?

Mostly no. The gain you can actually feel comes from not changing the setup.

Julien lives in the terminal and opens a browser only when there is no CLI. He moved to Ghostty, which describes itself as fast, feature-rich and native, with a shared libghostty core in Zig behind a platform GUI, early and for one reason: “It was clean. It was nice. I didn’t have to configure it much, which I like.” The Vim keybindings are muscle memory. Everything else is settled and he wants it to stay settled. “From the days of university I could spend days customizing and ricing everything. Then I started working in the industry and I was like, I’m never configuring anything ever again.”

We went the other way and landed in the same place. We moved off Ghostty to cmux after the terminal conversation in episode #13, and the honest description is that cmux is Ghostty in different wrapping with some bells and whistles on top. Julien installed it, looked at the configuration surface, and closed it. “I’m fine. It won’t boost my productivity that much.”

The second monitor goes the same way. Catch yourself dragging windows between screens and what you are watching is motion, not work. Julien bought a 43-inch Dell during COVID, big enough for four tiled work areas, and ran Rectangle, the free macOS window manager, to keymap windows around it. That made sense when there was third-party software and a dozen applications to arrange. These days it is the laptop keyboard and the laptop screen, at home or in a coffee shop, and nothing about the workflow changes. “The productivity boost came from consistency, I would say.” Travel enough with only a laptop and the laptop quietly becomes the environment. The tell is watching people in an airport who feel unproductive the moment their fancy screens are gone, on exactly the same machine they were productive on yesterday.

The one thing Julien did change, fifteen years ago, was the keyboard layout. He is Belgian, so he grew up on AZERTY, and he could feel the strain in his fingers reaching for curly braces and parentheses. “You almost have to remember the combination to get to that, and that is not a flowing experience. You have to stop and think, and that breaks my workflow.” He switched to a US layout, which is the same reasoning as the terminal choice: anything that makes you stop and think about the tool is a tax on the thing you were trying to think about. He still types on a Swedish Mac keyboard, because a real US board puts Enter somewhere his hands refuse to accept.

Voice input runs into the same flow argument from two directions. Julien finds dictation fidgety on technical vocabulary, especially with an accent, and switching between voice and keyboard annoys him more than typing does, so he types everything. The wall we hit is more specific: you cannot copy and paste with your voice. You stop, wait for the dictation to catch up, paste, continue, and the transcription model loses the first part of what you were saying. It works well for short commands. The version that does work is voice for dispatch: say the whole problem out loud, send an agent off on it, switch to the next one, and keep the keyboard for editing.

Context is the whole job now

If the setup is settled, the next thing that decides output quality is what you hand the agent, and what previous sessions left lying around for it to find.

When the output is bad, is the model the problem?

Usually not. It is the context: what you gave it, how much of the window is left, and which stale artifacts it picked up on the way.

Point it at an example, not an adjective

The most transferable technique here is not a prompt. Julien had work with advanced authentication and security requirements, so he built an example project first: real abstractions, modularized, concerns separated, and different stages depending on how secure a given path needed to be. Then he stopped explaining and started pointing.

“I made an example project where I really focus and explain and make the right abstraction. And then even in other languages, I was like, hey, point your agent to this repo and tell it what you want. And it will copy and follow the rule. You just boosted the context to a level where, okay, it can read Go so it understands very well what the code does, and you just need to do it in another language. It works perfect.”

The exemplar does not have to be in the target language. The model reads the structure in Go, understands what the code does, and carries the shape into whatever you are writing next. Explicit instructions are not the losing option here: Anthropic’s prompting best practices tells you to be clear and direct, and puts examples right next to that, calling them “one of the most reliable ways to steer Claude’s output format, tone, and structure”. What the reference project bought him was reliability on a complicated requirement. Right abstractions, separated concerns and different stages depending on how secure a path needs to be is a lot to pack into a paragraph, and working code that already does it leaves less to interpret than a description of it. The trade-off is worth stating plainly: building the reference implementation is real engineering work that ships nothing on the day you do it, and it only pays back across repetitions, so the preparation cost has to be justified by how often you expect to reuse it.

The oldest rule in the business still applies, with a bigger blast radius. If the agent has the right context it can follow the right direction, and if the context is wrong you end up somewhere else entirely. Crap in, crap out. The corollary Julien adds is that bloated context is its own failure mode, and he puts a number on where it starts for him: “Even those top models, after 200,000 tokens, they choke on themselves.” So the splitting has to be yours. Left to decompose a task itself, an agent handed him a fifty-step migration plan for something he already knew how to do and did not care about. This is the same window arithmetic we measured in episode #2 and argued from in episode #12: the constraint is not how smart the model is, it is how much attention you left it for the actual problem.

The prescription that follows is one word, and it should be familiar from episode #12: a glossary. “You need to have a glossary of the terms you use and have that in common with whatever the model thinks you’re talking about.” That is a semantic layer described from the user’s side of the interface. If you know the problem space, you can define the problem cheaply. If you do not, you get what Julien expects to get if you drop him into finance tomorrow, which is a vague question about where the money goes and a correspondingly vague answer.

Everything the agent writes becomes the next agent’s input

The productivity gain is real and the prose is still unbearable. “I find the wording to be super weird. The way they speak is just insane to me. English is not my first language, and I read that and think, who in the world understands what that means?” Somebody has started calling it Cloudish, as if it were its own language.

Julien’s theory for why is watermarking: that nudging token selection to leave a detectable signal turns output into “just a probabilistic result” rather than a sentence. The mechanism is documented, but the documentation does not reach as far as the theory needs it to. Anthropic’s announcement of Claude’s text watermark describes the SynthID-Text method, which changes the source of randomness when the model chooses between equally viable words. On scope it is a plan rather than a description of the fleet: “Future Claude models will generate text that contains a watermark”, while models launched before August 2, 2026 fall under an EU transition period, with watermarking for those “rolled out over the coming months”. Code is carved out where an exact output is required, and the page says the watermark can be used where the choice between terms is arbitrary, “such as comments within code”, with a negligible effect on the code itself. So the source settles the mechanism and the roadmap. It does not establish that the models producing the prose Julien is describing were watermarked when he read it, which has to be the first link in the chain for the theory to hold.

The published research does the same thing: Kirchenbauer and colleagues select a randomized set of “green” tokens before each word and softly promote them during sampling. What neither source supports is the effect on the prose. Anthropic states that “watermarking does not impact the quality of Claude’s output” and reports no impact on content, creativity or readability in internal testing, and the arXiv paper claims “negligible impact on text quality.” Vendor self-assessment is not independent evidence and neither is a paper about a research implementation, so treat the register as observed and the cause as unproven, with the rollout timing as one more reason not to settle on it.

Our own explanation for the verbosity specifically is duller, and it is a hypothesis rather than a finding: hosted models are trained toward long output because more tokens bill for more money, with the side benefit that long traces help the people trying to work out what the system is doing. The vendor account runs the other way. Anthropic’s update on recent Claude Code quality reports calls verbosity a notable behavioral quirk of Claude Opus 4.7 relative to its predecessor, and names model training, prompting and improving thinking UX in the product as the tools it has for bringing verbosity down. One of those attempts was a system prompt instruction capping text between tool calls at 25 words and final responses at 100 unless the task needed more, and it was reverted after it hurt coding quality. So take the billing incentive as our guess, not as a finding, and note there is documented effort pointing the other way. Whatever the cause, the consequence is the same, and Julien has acted on it. “Everything from summaries to documentation to freaking comments, and they put comments in everything. I banned that from the code. No comments anymore. If you cannot read the code and understand what it does, the comment on top is not going to help.”

The part with teeth for anyone running agents over a long-lived repository is that the comments are not just noise, they are the next session’s context. Generated comments describe things that no longer exist, and the next agent reads them as current: here we used to have a different thing, but now it is gone. Julien’s version is watching an agent fetch a decision made six months ago, on models that are now obsolete, and reopen it. “Why are we even talking about this?” Every artifact an agent leaves behind is an input to the next agent, so generated documentation carries a maintenance cost even when no human ever reads it.

Layers of reviewers, and knowing when to stop

Bad context produces bad code quietly. The only thing standing between that and a release is whatever verification you built, and that has grown into more machinery than most teams have admitted to running.

How many review rounds are enough?

Nobody has a principled answer, so the limit has to be a number you pick. Ours is two.

Our own pipeline starts with a grilling session on the plan. Three sub-agent reviewers in Claude and one in Codex then review the proposal in parallel and come back with what is wrong and what is missing. If the plan changes, another round. It can go a long way into those cycles and leave you having shipped nothing, which is why the cap is two. Then implementation, with deliberate handoffs: as a session approaches 200k we start a new one, with traceability carried in handoff files. Then three to five sub-agents plus Codex review the implementation, the findings come back, the session fixes them, and there is potentially one more pass. And then you actually have something that works.

We would rather not look inside it, because we know we will not be happy, and what substitutes for looking is extensive testing. It is pretty much a black box.

Julien is in the same position from the other side. “I don’t think I’ve written a single line of code in the last six months. But I still read the code, and it’s scary sometimes, and the over-abstraction.” He is blunt about what that has done to the day: “the work is freaking boring now. It’s really just banging this AI, trying to fit it into what you want.” The job it most resembles is project management, which is its own answer to why it feels the way it does.

Generated tests are where this gets concrete. Julien’s verdict is that half of them are useless, and his distinction is between tests that enforce behavior and tests that verify the concept: “okay, you validated that this function works.” That distinction matters because two different jobs are being done with one artifact. A test written to fix a requirement is documentation with a failure mode, so when you change the code in six months it tells you who broke what. A test an agent writes for itself is a feedback loop: it runs the test, sees the branches execute, and treats that as evidence the code works. You can prompt for the first kind, but the default behavior is the second, which means a suite nobody steered is optimized for the agent’s confidence rather than your regressions.

The harness is the other half, and it moves under you

Verification machinery runs inside a harness, and the harness is not a neutral container. It is half the system, and it changes under you.

Are agent harnesses interchangeable?

No. “They’re not all the same,” and the differences land on your skills, your config and your upgrade path rather than on benchmark scores.

Julien’s pain is more mundane than capability differences: “It’s painful that there is not one place I can put all my skills.” He runs oh-my-pi, the coding-first fork that wires an IDE into the agent with LSP, a real debugger over DAP, subagents and a plan mode, having used Pi itself for a while first. Pi is the deliberately minimal original, four tools and a system prompt under a thousand tokens, with no MCP, no plan mode and no sub-agents by design. What pushed him off it was billing rather than features, the Claude subscription route closing for third-party harnesses, which is the same shift we covered in episode #6. “I don’t know if they fixed it. I just hate changing my workflow. I would rather do something very painful than do that, because I put so much energy into focusing on the problem, and then I’m stopped by some config in whatever nonsense DSL they came up with.” He also uses Claude and Codex, and rates Pi and especially oh-my-pi warmly: “They have a planning mode that is actually pretty good.”

Two complaints about the whole category are easy to verify yourself. “Even the harnesses, most of them don’t know their own config. They have to go look it up on the internet.” And on upgrades: “How is programming soft, but you cannot auto-update my stuff?”

This is not a side issue. The model is half the equation and the harness is the other half, because raw models make mistakes that the harness exists to catch, right down to emitting JSON that does not validate. And harness work does not stay done, because it is tuned against a specific model’s habits. One skill in B.O.R.I.S had to be rewritten when the latest Claude developed a preference for piping output through head and reading the first sixty lines. The skill now says the output is already optimized, not to pipe it, that there is a flag if you want it smaller, and that the tool will say when there is more to read. Without that last part, a truncated result and a complete result look identical to the model, which is the entire reason the instruction exists. Julien generalizes it: “Every three months the model changes, so I have to relearn everything. You have to delete whatever instruction you have. You have to review your skills non-stop.”

We have a small worked example of exactly that, from our own CLI. Working on bmcp, we hit a case where credentials were present in the terminal but expired. The tool could not recognize the expiry, so it reported itself unauthenticated, and the agent concluded the tools were broken. The fix was a flow decision rather than a technical one: use the credentials if they are valid, otherwise fall back to the pre-configured SSO profile, so nobody has to think about it. That had to be spelled out, repeatedly, because the model kept optimizing for something else. A model will happily optimize a flow no human would tolerate, because it has no sense of what using the thing feels like.

What Julien has built around his harness is small, and each piece removes one manual step. He runs Herdr, a background server that keeps coding agents running across projects, shows which ones are working, blocked or idle, and restores layouts and sessions after a restart. He picked it, consistent with everything else, because it installs with one command and nothing else about the setup changes. He wrote a skill so that instead of producing a handoff document he then pastes into a new window, the handoff opens a new session in a new tab already primed with it. He uses escape to rewind a conversation when he wants a different path from an earlier point. And he forces everything into worktrees, with “a small hook that prevents any kind of commit to main. That’s the only way I think it allows me to keep things sane.” Note what that hook actually does: it blocks direct commits to main. Work still reaches main, through a merge you make on purpose.

He also flags Worktrunk for teams that want the worktree lifecycle managed by tooling. Its merge command is explicitly aimed at the default branch: it squashes and rebases the current branch, fast-forwards the target, then removes the worktree, and it runs pre-merge hooks after the rebase, pre-remove hooks before cleanup, and post-merge and post-remove hooks in the background. So the safeguards are the hooks you configure, such as tests and lint that abort the merge, not a built-in refusal to touch main.

The smallest and most relatable complaint in this whole area is permissions. “Just figuring out the permissions makes me shout at the screen more than I would like to admit. It’s just bonkers.” The answer that stuck for us is one line: auto mode solved it, so we stay in auto mode.

Which account has what, across two clouds

Harness churn is friction you can feel. Missing infrastructure context is friction you pay without noticing, and it gets worse the moment a second cloud shows up.

Why does a multi-cloud estate break an agent’s context?

Because loading everything at once does not tell it which cloud you mean. In Julien’s two-cloud work the default was AWS, so the cloud and the account belong in the request rather than in your assumptions.

Julien specializes in Google Cloud and his work also spans AWS, which produces a specific daily friction: “it’s very hard for me to know which account contains what.” What changed for him was a different kind of latency. “With B.O.R.I.S it was like, wow, I don’t have to wait half a day for the US to wake up. It was just there. That has boosted my productivity a lot.” The second use is translation between the two: how does this service work in AWS, and how would I adapt that to Google Cloud. The questions themselves are ordinary, and that is the point. Give me all the external IPs for that account. Can we do least privilege for that service account based on the logs. Small lookups that used to mean clicking through a console or waiting on a colleague in another timezone. The same which-account-has-what problem is the one we took apart in episode #17.

Two gotchas come with it, and both are worth knowing before you build on anything new. The first is that a model has no training data for a tool that did not exist when it was trained. “When you create a new tool the model hasn’t been trained on, you’re going to suffer. It’s going to try and try and do really simple typo mistakes, and you waste plenty of your session tokens on something like, hey man, you just have to write the query.” What he did not expect is that this is not what happened to him with bmcp: “surprisingly, the model usually doesn’t make a lot of mistakes.” So the warning is the thing to plan for with any new tool, and the exception is his own, one estate’s experience rather than a general result.

The second is cloud ambiguity, and it is the one that generalizes. “The hard part is that if everything is loaded in the context, it doesn’t know which cloud I’m talking about. It’s going to default to AWS. So you have to block it a little bit, but not too much.” He calls it a game of whack-a-mole: you have to be careful which one you slack on. The cheap half of the fix is not in the instructions at all: name the cloud, the account or project and the region in the request itself, so nothing has to be inferred. The expensive half is tuning the standing instructions, because adding context does not improve behavior monotonically, and a multi-cloud estate just makes that trade-off visible in a way a single-cloud one hides. His overall verdict is still positive and specific: “I think the quality of things has improved a lot, or allowed us to improve the quality of the product that we’re building.”

What runs on your laptop

Two of these problems have a local answer, and both come with costs that are easy to state badly.

Is a local model useful for real work yet?

For short private checks, yes. For coding on a laptop, manage your expectations first.

Julien’s read, from time with the Google DeepMind people about the Gemma models and the optimization work needed to deliver them, is that a recent Qwen release is the better local coder while Gemma 4 is plenty good enough, “you just need to manage your expectation.” The expectation management is the useful part. On a laptop it is slow and there is not a lot of intelligence in it. His plan to get work done on a plane with a local model ended the way you would guess: there went the battery.

His other warning is that you will grill your SSD running that for a month. Worth being precise, because that is a warning rather than a measurement. Flash endurance is rated as cumulative writes: Kingston describes TBW as drive capacity multiplied by the program/erase cycles each NAND block supports, divided by the write amplification factor, with DWPD expressing the same budget per day across the warranty. Inference itself mostly reads weights. The writes come from memory pressure, when a model does not fit and the machine starts swapping hard, so the thing to watch is memory pressure and your drive’s rating, not the calendar.

The other local risk is the one nobody classified as a secret store. Julien’s warning: “In the .claude directory on your laptop there are transcript files saved, and those files will have all the credentials Claude ever got its hands on. If you had temporary credentials that expired, that’s fine. But if it got hold of your GitHub token, you probably want that deleted, because any exploit that gets hold of your disk, this is where it would go to search for credentials.” He contrasts it with the care everyone already takes elsewhere, SSH keys in 1Password or the Secure Enclave, and suggests deleting them or setting a cron job.

Anthropic’s own documentation confirms the mechanism and offers better levers than cron, along with two exceptions that undo the naive version of the fix. The .claude directory reference says these files are plaintext and that anything passing through a tool lands in a transcript on disk: file contents, command output, pasted text. CLI session transcripts under ~/.claude/projects/ are swept once they are older than cleanupPeriodDays, which defaults to 30 days, has a minimum of 1, and rejects 0.

The first exception is the desktop one. Transcripts from sessions you started or last continued in Claude Desktop or Cowork are kept at any age by default, so lowering cleanupPeriodDays does nothing for them. Those need desktopSessionCleanupPeriodDays, which requires Claude Code v2.1.248 or later, unless managed settings set cleanupPeriodDays, in which case that period applies to them instead.

The second exception outlives both. ~/.claude/history.jsonl holds every prompt you have typed, with its timestamp and project path, and the documentation lists it under the paths the retention sweep does not remove and Claude Code keeps until you delete them. So the credential you pasted into a prompt, rather than the one a tool printed, is still sitting there after every transcript that mentioned it has aged out.

Clearing it is deliberate work rather than a setting, and the obvious tool for it is blunter than the job. claude project purge <path> does filter that project’s prompt lines out of history.jsonl, but it is a project-state command rather than a history command. The same run also deletes the project’s transcripts and auto memory under projects/, its per-session tasks/, debug/ and file-history/ entries, which is checkpoint restore for those sessions, and the project’s entry in ~/.claude.json, which carries trust, history and MCP server configuration. --all purges every project at once and deletes history.jsonl outright instead of filtering it. The command prints its full deletion plan and asks for confirmation, and --dry-run prints the same plan without touching anything, so read it before you answer y. If prompt history is the only thing you want gone, delete ~/.claude/history.jsonl yourself and leave the rest of the project state alone. Upstream of all of it, CLAUDE_CODE_SKIP_PROMPT_HISTORY skips writing transcripts and prompt history in any mode, with --no-session-persistence alongside -p or persistSession: false in the TypeScript Agent SDK as the non-interactive equivalents. What you give up by deleting or never writing that file is up-arrow recall, Ctrl+R history search and ! shell-command completion, which is a small price next to a long-lived token in plaintext. Rotation is still the backstop for both files, because deletion removes the copy on your disk and does nothing about the one that already leaked.

Token spend is not a performance metric

Zoom out from the laptop to the organization and the same question shows up in dashboard form: all of this is obviously faster, so how would you prove it is worth anything.

Does burning more tokens mean getting more done?

Not reliably. Spend tells you what the tooling cost, not what the work was worth.

Start with the expectation, stated as a joke that is not really a joke:

“The hard thing about increased productivity is, I thought by now we would be on a beach, pressing yes to the AI and just letting it roll. But no, man, suddenly we have like ten projects in parallel, all of them hard, way harder than the one project we would have done before. And at the end of the day my brain is drained from reading AI slop. Text slop, not code slop.”

He takes responsibility for it immediately: “I think it’s a problem with us, not with AI. We have some free time and we stuff it with more projects.”

The next escalation is what he calls software factories. Somebody formalizes an existing framework like Agile, puts a model on it, and builds a small harness that synchronizes state through an event loop, so events trigger prompts and the prompts are wired together. His objection is not that it cannot work, it is the debugging. “That work is insanely hard and insanely complicated to debug. How would you even know that something is wrong?” And the way you find out is by asking an AI what was done, which means more text slop in your face about things you did not think you needed to know.

Underneath is the loss that actually stings. “A year ago we were not fast, but I knew everything that was going on. Now I feel like I don’t know half of the stuff that is being shipped.” Fifty pull requests merged in two days is exactly why an automated Slack status update is such a good fit for automation, and also why the generated summaries are not good enough: “you can tell that a human needs to look at it and have a judgment about what someone else would like to know. The AI has absolutely no clue. It’s still statistics.” Statistics is also where the limit sits: once you are in the tail end of things, everything falls apart, and you have to drive that thing very much manually.

So teams reach for the dashboard, and the first thing to correct is the assumption that one of these products reports something simpler than the other. Org-deployed Claude and Cursor both ship admin leaderboards, and neither leaderboard is denominated in tokens. Cursor’s Usage Leaderboard ranks people on three counts: Chats, the number of messages a user sent in the chat interface, Tab Completions, the number of Tab suggestions the user accepted, and Agent Lines of Code, lines written by the Agent and accepted by the user. Only two of those three are acceptances at all. Chats counts what somebody sent, whether or not anything came of it. Usage metrics is the fair label for the set, which is also the label Anthropic puts on its own equivalents. Token and dollar figures live on a different surface, in the usage and spend reporting and the spending limits documented for the team dashboard, and they answer what the tooling cost rather than what came out of it.

The Claude dashboard reports lines, and a line turns out to be two different things depending on which number you are looking at, because they come from two different families. Anthropic’s analytics documentation splits usage metrics from contribution metrics and defines each line count separately. Lines of code accepted is a usage metric: what Claude Code wrote and the user accepted inside a session, excluding rejected suggestions and without tracking anything deleted afterwards. Lines of code with Claude Code is a contribution metric, which needs a GitHub integration to produce at all: effective lines across merged pull requests attributed back to a session, where effective means more than three characters after normalization, with empty lines and bracket-only lines dropped, and where anything a developer rewrote by more than twenty percent stops being attributed at all.

That is a real distinction and worth knowing before somebody quotes a number at you. The usage side counts keystrokes that survived an accept, which is where Cursor’s Agent Lines of Code also sits. The contribution side counts qualifying lines in merged pull requests, regardless of target branch, with Anthropic calling those metrics deliberately conservative. The branch part is easy to over-read: Anthropic’s attribution notes say the algorithm does not consider the pull request’s source or destination branch, so a PR merged into a long-lived feature branch scores the same as one merged into your default branch. What none of these metrics counts, in either family, is whether the change was worth making. A merged pull request that should never have been opened scores the same as one that fixed a customer-facing failure.

The spend data, which is the separate surface, makes the same point from the other end. Across three or four organizations the same distribution came back: at the top sat both the people you would call top performers and people whose output you would throw away, burning comparable amounts, with a solid middle underneath and a group not using any of it.

The part that matters most for anyone tempted to act on this: none of those organizations had token targets or goals announced by leadership. The numbers were simply there, people looked at them, and anyone who knows the names behind them does a double take. Julien’s analogy makes the absurdity obvious: “Imagine you let your kids loose and say, let’s see who spent the most. And then afterwards you try to evaluate whether what they bought makes sense.”

The measured version is in DORA’s ROI of AI-assisted Software Development report, which argues AI acts as an amplifier of an organization’s existing strengths and dysfunctions rather than a productivity input of its own, and that the returns come from the surrounding system: platform quality, workflow clarity, team alignment. It also names the cost this keeps circling back to, the verification tax, the effort of checking whether generated code is reliable, secure and architecturally sound, which scales with volume. And it describes the initial productivity dip that follows adoption, before any of it nets out.

There is a second adoption problem that gets far less attention than the first, and it never shows up on a dashboard either: the people who tried AI once, had it fail, and will not touch it again. Both groups are drag, and only one of them is counted. One caveat belongs on all of it, which more of these conversations should include: we are in a bubble. Talk to people outside it and the reaction is, what are you even talking about. The hamster wheel is one we built ourselves and then started running in.

What stays valuable when the container part is done

If spend does not measure value, the question turns personal: which part of this work is still worth paying a human for.

What is left that an agent cannot take over?

Knowing which problem is worth solving, in a domain you understand well enough to tell someone their ask is wrong.

The old model was the generalist. We are cloud engineers, we do cloud for whoever asks. That no longer clears the bar. What does is a pairing: a business vertical you actually understand, with the vocabulary that goes with it, plus a technical discipline you are deep in. Cloud for fintech comes with its own requirements, its own compliance standards, its own language and its own sense of what is important. You could get by as a generic cloud specialist before.

The reason the bar moved is specific. The upper hand used to be that developers handed over Docker containers: whatever is in the container, make it run, make sure it does not fall over, make sure it is available and it scales. Done, and it looked more or less the same in every industry. An agent can do that now. What remains is going one layer deeper into the business, because the solution you are building may not survive a change in requirements, and anticipating how requirements will change comes from time spent in that industry. Julien’s note is that this is only strange to engineers: legal, sales and marketing already work this way, “it’s just us cloud engineers who do whatever comes.”

Specialization has a sharper edge than domain vocabulary, because the ask itself is often wrong. “Sometimes what people ask is wrong and they don’t realize it. They say we should build a data pipeline, and they start making it super expensive and reliable and throwing stuff at it. And it’s like, man, you just need a message queue there.” Then the consequence:

“If you only want the answer, that’s what you get. AI gives it to you. But you kind of need the lesson here.”

The human version arrives as a story about a colleague in the gaming industry with a habit of saying “yes, sir” to his manager and then doing nothing. The manager loved him, because they felt validated. He knew perfectly well the ask was nonsense and that the pressure was coming from somewhere else. The conclusion Julien draws is that the human aspect is still number one, and the business-level version of it is the same trap one floor up. AI speeds up the software development part. Lead generation gets better with AI tooling, but the sales process still takes the time it takes, procurement included, and the decision to buy is still made by humans. You can run faster in the hamster wheel of software development efficiency, and if the business is not moving faster it is all for nothing. Which leaves the open question neither of us pretends to answer, about what happens if the value never materializes against what is being spent. “Man, your opinion is as good as mine. I have no clue.”

Two bills arrive later, and both are easy to leave out of the enthusiasm for replacing bought software with homemade equivalents now that the first version is cheap to generate.

“Have you seen the cost of maintaining software? It’s basically like somebody gave you a Ferrari or a mansion, but they never talked about the bill of the water. You didn’t win the lottery. You just got an expensive gift that you don’t have to pay for, but all the maintenance costs come onto you. And I’m not sure you have the income to actually pay for it.”

The same shape as taking out credit for an expensive BMW and then meeting the service schedule that a sophisticated suspension comes with. Build got cheaper on the build side of build-versus-buy. The maintenance obligation did not go anywhere: whatever you generate, somebody owns its upgrades, its security patches and its bugs, and that cost belongs in the decision.

The second bill is the one Julien says he is really worried about. “Where is the next generation of senior engineers if all the juniors use that stuff as if it were the truth? How are you going to learn?” He then argues against himself: “It’s not that bad, because I didn’t learn how a CPU worked on my first day. When you program, that’s not exactly what you need. You just need to fix a business need, and you have a codebase, and you have people who know that codebase and can help you.” The counter is the abstraction shift. Almost nobody working today knows assembler and we still solve problems, just at a different layer, and when compilers arrived productivity rose because people moved up to higher-level languages. His reply is one sentence and it is the whole objection: “Well, so we are moving to non-deterministic, anyhow.” A compiler gave you a new layer that behaved the same way every time. That is the property this transition does not carry, which is why the analogy comforts less than it should.

Where the learning still happens

Specialization has to come from somewhere, and the answer is less about conference programmes than it looks.

On AI conferences specifically, Julien gives a qualified no. The last one he attended was AI Engineer Europe, Europe’s first flagship edition of that event, three days at the Queen Elizabeth II Centre in London across eleven tracks, advertised with an expected attendance of a thousand or more engineers rather than a count of who turned up. His read is that the field has not settled enough for the format to work. “It’s all over the place. It’s too early. The landscape is changing, it’s like a tectonic shift, and I cannot recommend saying oh yeah that’s good, because things are in flux right now.” He is fair about the execution: it was packed, you had to arrive early to get a seat in any room, and there were the organizational wrinkles anyone who has run a first-year event will recognize. “But you know what, I had a blast.” What he got out of it was tech rather than talks, including time with the Google DeepMind people about Gemma and the optimization work needed to ship it.

On conferences in general he is unequivocal. “Go to a conference, it’s a very good investment of time. You meet people, you make friends, you hear interesting stories, you get inspired, you get new ideas. And your employer should pay for that.” His preference is smaller and local: AWS days, Google Cloud days, one or two days, a hundred to a hundred and fifty people, enough room to actually make connections. The mega-conferences he steers away from for a reason worth knowing before you buy the ticket. “The last time I went to KubeCon I think I saw maybe three talks. The rest of the time I was speaking or reconnecting with people I only meet at KubeCon.” That is a perfectly good reason to go. It is just a different reason than the programme advertises.

What this means for your team

None of this is a tooling problem you can buy your way out of in an afternoon. The leverage sits in a handful of decisions that are cheap to make and expensive to keep postponing.

Four things worth doing this week:

  • Build one reference implementation for the pattern you repeat most, with the abstractions and the security stages you actually want, then point agents at it alongside your written instructions. Do it for something you will apply ten times, not once, because the preparation only pays back across repetitions.
  • Write down your stopping rule for reviews. An agent asked to find problems will keep finding them. Pick a cap, two rounds on the plan and two on the implementation is a defensible starting point, and treat extensive testing as the thing that substitutes for reading every line.
  • Check what your transcripts and your prompt history are holding, then set retention deliberately. Lower cleanupPeriodDays for CLI sessions and set desktopSessionCleanupPeriodDays separately, because Desktop and Cowork transcripts are kept at any age by default. Then treat ~/.claude/history.jsonl as its own job: the sweep never touches it, so delete that file yourself, or stop writing it in the first place with CLAUDE_CODE_SKIP_PROMPT_HISTORY. Reach for claude project purge only when you also want that project’s transcripts, auto memory, checkpoint state and ~/.claude.json entry gone, and run it with --dry-run first. Rotate any long-lived token that has passed through a session or a pasted prompt.
  • Stop reading dashboards as performance. Learn what your numbers actually count. Neither the Claude nor the Cursor leaderboard reports tokens, lines accepted in a session and lines attributed to merged pull requests are different measurements from different families, one of Cursor’s three columns counts messages sent rather than anything accepted, and token or dollar spend sits on a separate billing surface. Then ask what shipped, whether anyone can still explain it, and what the verification is costing you. If you want a number to argue with, the DORA ROI framing gives you a better one than a leaderboard does.

None of that is a one-time setup, and the trigger for redoing it is not a date on the calendar. It is a change underneath you: a model that starts reading your skills differently, or a harness that moves your config out from under them. When either lands, re-read the instructions you wrote for the previous one before you trust them.

FAQ

How do I get an AI coding agent to write code in my team’s style?

Give it a worked example alongside the instructions. Build a small reference project that shows the abstractions, the module boundaries and the security handling you want, then point the agent at that repository and tell it what to build. Anthropic’s prompting guidance recommends both being clear and direct and using examples, which it calls one of the most reliable ways to steer output format, tone and structure. The example earns its place when the standard is complicated enough that a paragraph describing it leaves too much to interpret. It also works across languages, because the model can read the structure in one language and carry the shape into another. The cost is real engineering time that produces nothing shippable on the day, so it makes sense for patterns you expect to reuse many times.

Should I let an AI agent add comments to my code?

Julien Bisconti banned them from his own code, and his stated reason is that a comment cannot rescue unreadable code: “if you cannot read the code and understand what it does, the comment on top is not going to help.” The argument with wider reach is staleness. Generated comments describe the code as it was when they were written, so after a few changes they describe things that no longer exist. The next agent reads them as current context and acts on them, which is how a stale note becomes a wrong decision. The same applies to generated design documents and old decision records: every artifact an agent leaves behind is an input to the next agent, so it carries a maintenance cost even if no human reads it.

Is token spend a good measure of AI productivity?

No. Across several organizations, the top of the spend ranking held both strong performers and people whose output was close to unusable, with most people bunched in the middle. Spend tells you what the tooling cost, not what the work was worth. It also helps to know that the admin leaderboards are not spend reports at all, and that the numbers on them are not one kind of measurement. Cursor’s Usage Leaderboard ranks Chats, which counts messages sent in the chat interface rather than anything accepted, alongside accepted Tab completions and accepted agent lines. Anthropic’s analytics documentation splits usage metrics, such as lines accepted inside a session, from contribution metrics, which are effective lines attributed to merged pull requests through a GitHub integration, and that attribution ignores which branch a pull request targets. So they are usage and contribution metrics, token and dollar figures live in separate usage and billing reporting, and no number in any of those families answers whether the change was worth making. DORA’s ROI of AI-assisted Software Development report makes that point from data, describing AI as an amplifier of an organization’s existing strengths and dysfunctions and naming the verification tax, the effort of checking that generated code is reliable and secure, as a cost that grows with volume.

How many review rounds should an AI coding agent do before I ship?

Pick a cap and stick to it, because there is no natural stopping point. An agent asked to find problems will keep finding them, and each round that changes the plan invites another round, which is how a day disappears with nothing shipped. A workable shape is a grilling session on the plan, parallel reviewers across two different models, a hard limit of two planning cycles, then implementation with a second review pass on the code and extensive testing as the real safety net.

Why does my agent assume the wrong cloud provider?

Because when everything is loaded into context at once, nothing in the question tells it which cloud you mean. In Julien’s two-cloud work, AWS and Google Cloud side by side, the observed default was AWS. The cheapest fix is to stop relying on inference: name the cloud, the account or project and the region in the request itself. Standing instructions help too, but over-constraining degrades other answers, so that half is a balancing exercise rather than a fix. A tool that did not exist when the model was trained can compound it, because the model may make simple syntax mistakes on your queries and burn session tokens doing it, until the tool or your instructions make the right call obvious.

Are Claude Code transcripts a security risk on my laptop?

They deserve treating as sensitive, and so does the prompt history sitting next to them. Anthropic’s documentation says the files under ~/.claude are plaintext and that anything passing through a tool lands in a transcript on disk, including file contents, command output and pasted text, so any long-lived credential an agent touched is sitting there. CLI session transcripts are deleted once they are older than cleanupPeriodDays, default 30 days, minimum 1. Two things survive that sweep. Transcripts from Claude Desktop or Cowork sessions are kept at any age unless you set desktopSessionCleanupPeriodDays, which needs Claude Code v2.1.248 or later. And ~/.claude/history.jsonl, which records every prompt you have typed with its timestamp and project path, is documented as kept until you delete it, so a secret you pasted into a prompt outlives the transcripts that quoted it. When prompt history is all you want gone, delete that file by hand, or stop writing transcripts and prompt history in the first place with CLAUDE_CODE_SKIP_PROMPT_HISTORY. There is a claude project purge command that filters one project’s prompt lines out of the file, but it is a project-state command: the same run deletes that project’s transcripts and auto memory, its per-session task, debug and checkpoint-restore entries, and its entry in ~/.claude.json, which holds trust, history and MCP server configuration, while --all deletes history.jsonl outright across every project. It prints a deletion plan and asks for confirmation, and --dry-run shows that plan without deleting anything, so use it only when you want the wider clear-out. The cost of losing prompt history either way is up-arrow recall, Ctrl+R search and ! completion. Then rotate any token that has already been through a session, because deleting the file does not un-leak it.

Can I run a local model for coding on my laptop?

You can, with lowered expectations. Julien’s own read of the open-weight models he has tried is that a recent Qwen release is the stronger coder and Gemma 4 is “plenty good enough,” which is one engineer’s experience rather than a benchmark result, so test the pair on your own work before committing. On laptop hardware it is slow either way, the effective intelligence is limited, and battery life disappears fast. The job local models are clearly good at is a short private check: run a small prompt against a local model to ask whether a command is safe or whether secrets are about to be included. The screening itself runs locally, though anything it approves still goes on to the hosted model, so treat it as a filter rather than a guarantee. On drive wear, the concern is not inference itself, which mostly reads weights, but the writes caused when a model does not fit in memory and the system swaps heavily. SSD endurance is rated in cumulative writes, so watch memory pressure and your drive’s TBW rating.

Which conferences are worth attending right now?

Smaller and local ones, if the goal is to learn and meet people. Events of a hundred to a hundred and fifty people, such as AWS or Google Cloud community days, leave room for actual conversations, while the largest conferences turn into reconnecting with people you only see there, which is valuable but not what the programme sells. AI-specific conferences are worth treating with caution at the moment, because the field is changing fast enough that the content ages between the call for papers and the event.

Resources

  • Julien Bisconti - the guest, with his resume and talks page for the Google Cloud, Go and Kubernetes background behind his read of the tooling, his GitHub profile and the awesome-docker list he maintains, and The DevSecOps Talks Podcast, the show he co-hosted with Andrey Devyatkin before this one. Useful for understanding why his answers skew toward workflow durability rather than novelty.
  • The tools on his machine, each linked to its own documentation: Ghostty settles what it actually is, fast, feature-rich and native with a shared Zig core, which is the low-configuration promise he bought. Rectangle is the free macOS window manager from the 43-inch-monitor era he has since abandoned. Herdr documents the working, blocked and idle status tracking and the layout restore that made it worth one install command. Worktrunk is the correction worth reading if you assume a worktree manager keeps work off main: wt merge squashes, rebases and fast-forwards the current branch into the target, which defaults to the default branch, then removes the worktree, with pre-merge, pre-remove and post-merge hooks as the place you add your own gates.
  • Pi and oh-my-pi - the two harnesses he rates alongside Claude and Codex, and the clearest illustration of his point that the harness is not interchangeable with the model. Mario Zechner’s post explains Pi’s radical minimalism, four tools and a sub-1,000-token system prompt, with MCP, plan mode and sub-agents left out on purpose. oh-my-pi is the coding-first fork that puts them back and wires in LSP and a real debugger, including the planning mode he singles out.
  • Explore the .claude directory (Anthropic) - the primary source behind the transcript warning, and more specific than the warning is. It confirms the files are plaintext and that tool output lands on disk, gives the CLI sweep rule (cleanupPeriodDays, default 30, minimum 1, 0 rejected), and documents the two things a shortened retention period does not cover. Claude Desktop and Cowork transcripts are retained at any age until you set desktopSessionCleanupPeriodDays, on v2.1.248 or later. And history.jsonl, every prompt you have typed with timestamp and project path, sits in the “kept until you delete them” table, so it needs deleting by hand or CLAUDE_CODE_SKIP_PROMPT_HISTORY upstream. Read the Clear local data section before reaching for claude project purge: it lists the full blast radius, transcripts and auto memory, per-session task, debug and file-history entries, and the project’s ~/.claude.json entry, with --all deleting history.jsonl outright across every project and --dry-run to see the plan first.
  • Claude’s text watermark (Anthropic), A Watermark for Large Language Models (Kirchenbauer et al.) and An update on recent Claude Code quality reports (Anthropic) - the three pages to read before repeating either theory about why generated prose reads the way it does. The first two settle the mechanism, biased sampling among viable token choices, and also settle its limits: the announcement is forward-looking, watermarking “future Claude models” with a phased rollout for models launched before August 2, 2026, so nothing on it establishes that the output being complained about carries a watermark at all. Neither source supports the effect either, Anthropic reporting no impact on content, creativity or readability and the paper claiming negligible impact on text quality. The postmortem is the counterweight to our own billing-incentive guess about verbosity: it calls verbosity a notable behavioral quirk of Claude Opus 4.7, names training, prompting and thinking UX as the levers for reducing it, and describes a word-limit system prompt instruction that was added and then reverted because it hurt coding quality.
  • Track team usage with analytics (Anthropic) and Usage Leaderboard (Cursor) - read both before anyone quotes a leaderboard at you, because together they say what a number on these dashboards is and is not. Anthropic separates usage metrics from contribution metrics: lines of code accepted is what was accepted inside a session, rejected suggestions excluded and later deletions untracked, while lines of code with Claude Code is effective lines in merged pull requests, over three characters after normalization, brackets and blanks dropped, anything rewritten by more than twenty percent no longer attributed, and the whole contribution set dependent on a GitHub integration. The attribution notes add the detail that is easy to assume wrongly: the algorithm ignores the pull request’s source and destination branch, so nothing in the number tells you the code landed on main. Cursor’s page corrects the common recollection that its leaderboard counts tokens. It ranks Chats, messages sent in the chat interface, Tab Completions, suggestions the user accepted, and Agent Lines of Code, lines the Agent wrote and the user accepted, so one of its three columns is not an acceptance at all. Spend on both products lives elsewhere, in Anthropic’s usage and spend reporting and on Cursor’s team dashboard.
  • The ROI of AI-assisted Software Development (DORA) - the measured counterpart to the leaderboard argument. Its framing of AI as an amplifier of existing organizational strengths and dysfunctions, the initial productivity dip after adoption, and the verification tax all map onto what a leaderboard cannot show you. Useful if you need to argue against a token-spend target with something other than an anecdote.
  • Claude prompting best practices (Anthropic) - the source behind the reference-implementation advice, and the reason it is not framed as examples instead of instructions. It puts being clear and direct alongside multishot prompting, calling examples one of the most reliable ways to steer output format, tone and structure.
Join B.O.R.I.S Slack Playground