AI adoption stalls when licenses and quotas replace trust, context, and verification. Fernando Gonçalves and Vladimir Samoylov explain how DevOps makes agentic coding safe to scale through checkable starter tasks, organization-specific context, independent tests, Terraform plans, and deterministic gates—while exposing why token metrics, AI-written tests, and advisory Markdown can create false confidence.
Summary
Somewhere right now a manager is telling the board that every engineer has AI access, and an engineer on the same payroll has a Copilot tab open that they quietly don’t trust. This episode of Agentic AI in DevOps puts Fernando Gonçalves and Vladimir Samoylov in that gap, and the conclusion they reach is that AI adoption is not a thing a company can switch on. Vladimir’s framing sets the tone: management treats it like the lights — “you open the lights, but in the same time you open AI, you just click it” — while nobody hands engineers an actual path, only a subscription. The path the two of them sketch is unglamorous and specific: start where the input and the expected output are already known and only the black box between them changes, keep the tests you trust and let the agent touch the code, feed it your Terraform docs and your organization’s real topology before reaching for anything clever, then replace advisory markdown with checks that either pass or fail. The sharp moments earn the runtime — a policy Vladimir has seen requiring 75% of code to be written by AI, token leaderboards that reversed by 2026 into something “like a food allowance,” a reported $10 million annual AI budget burned in three months, Fernando discovering his agent had written tests that faithfully reinforced a broken implementation up to 97% coverage, and Vladimir’s warning that with agents, degradation compounds at machine speed: “every next agent will follow on whatever was created by previous agent.”
Key Topics
AI adoption is not a light switch
The episode opens on a disconnect both hosts keep running into. Vladimir Samoylov describes how AI adoption looks from the top of an organization: one single button, a switch on the wall. You flip it, and the company is now AI-enabled. Fernando Gonçalves hears the same thing from the management side of his network — we have AI everywhere, all my employees have access — and something very different from the engineers: yes, there’s a Copilot license, or whatever agentic tool the company picked, but “I sort of don’t trust what comes out of it. I don’t use it that much.”
Vladimir’s explanation for the gap is procedural rather than cultural. An engineer gets told access has been granted, tries a couple of prompts copied from X or YouTube with no configuration and no setup behind them, gets a poor result for that particular case, and stops. Nobody provided guidance on how adoption is supposed to work: “it’s just a subscription which someone has to purchase from the popular company, and that’s it.” Fernando’s own early experience followed the same curve — buggy code, output that didn’t follow the way he likes to work, and the reasonable conclusion that he’d rather do it himself.
The industry numbers around this disconnect are blunt, if not a perfect match for what the hosts describe. The 2025 Stack Overflow Developer Survey put 84% of respondents using or planning to use AI tools, while distrust of output accuracy (about 46%) ran ahead of trust (about 33%) — a plurality who don’t trust it, not yet a majority — and the top frustration was answers that are “almost right, but not quite.” That survey measures use and intent, not how many engineers hold a license they quietly avoid, which is the specific population Fernando is describing. But the trust picture it does capture is the shape of the problem the episode is really about: the license is not the hard part, and neither is the model. The missing artifact is a path from access granted to this is how I actually use it on my work.
When the mandate becomes the metric
Pressure makes that gap worse, and both hosts have seen the pressure. Vladimir mentions companies establishing policies that 75% of code must be written by AI — a target that removes whatever room an engineer had to learn the tool at their own pace. “You must use it right away, somehow.” Fernando’s addition is that the metric says nothing about quality: churn out code as fast as possible, and it doesn’t matter much whether it’s good, as long as the usage shows up. The documented extreme of this management pattern is public: TechCrunch reported that Coinbase’s CEO fired engineers who hadn’t onboarded to AI coding tools by an end-of-week deadline, and called his own approach heavy-handed. “You must use it right away” is not a strawman.
Fernando traces the arc of that thinking through what he calls the token-maxing phase — internal leaderboards ranking developers by token consumption, on the theory that more tokens meant more adoption. Then the bill arrived. He recalls a story from another podcast (secondhand, and worth treating as such) of a company that budgeted roughly $10 million for AI for a full year and burned it in three. Vladimir’s read on 2026 is that the leaderboard has reversed: heavy token use now looks bad for the budget, teams are setting limits, and the whole thing has come to resemble “a food allowance.” The economics behind that reversal were the subject of episode #6, where the hosts worked through what happens when subsidies fade and token spend becomes a line item.
What replaces the metric, in Vladimir’s view, is efficiency of use. You can run the most powerful available model to make a tiny change to your code and feel great about it. You can pick up a big hammer and swing it the wrong way, and it will break more than you intended. He also names a habit worth recognizing in yourself: people mashing enter every time the agent finishes a step — continue, continue, continue — which is the same babysitting pattern that motivated the loop engineering discussion in episode #14. Volume of interaction is not adoption. Sitting in front of the screen approving each step is the thing adoption is supposed to remove.
There is external support for treating mandates carefully. Google’s 2025 DORA report concluded that “AI’s primary role is as an amplifier, magnifying an organization’s existing strengths and weaknesses,” and that the biggest returns come from the surrounding organizational system rather than the tools. A usage quota is a tool-level lever aimed at a system-level problem.
Start where you already know the answer
The concrete on-ramp Vladimir proposes is to look for the routine. Not all development work is interesting: CRUD endpoints, another third-party API integration, the parts where you mostly follow documentation one-to-one. And the category he flags as both maximally annoying and maximally mechanical is version migration — moving a codebase from Java 11 to Java 17 because a library dropped support, or absorbing breaking changes in a major upgrade.
His framing of why these tasks are the right first step is the most portable idea in the episode: start in places where you know the input and you know the output, and only the black box in between needs to change. In coding terms that has a practical corollary — leave the existing tests in place and untouched, and let the agent modify only the implementation. You know what was tested and how it behaved before, so you have a check that doesn’t depend on trusting the agent’s self-report. Fernando’s version of the payoff is a task that would take two days of understanding and mechanical editing landing in something closer to twenty minutes, validated with tools engineers already use, without reading every line or re-deriving the logic.
Both are honest that this still creates uncertainty — you don’t yet know how the tool behaves — which is exactly why the recommendation is a small, controlled scope rather than a general policy. Worth knowing before you spend tokens on it: for some of these migrations, deterministic tooling already exists and may be the better instrument. OpenRewrite’s Migrate to Java 17 recipe automates many common Java-to-17 changes through a repeatable recipe rather than a generative guess — its own guide is explicit that some cases are out of scope and need manual fixes. The transferable principle isn’t “use an agent for migrations,” it’s “pick a first task whose outcome you can check without reading the diff line by line.”
Terraform, plan output, and plugging in real context
For DevOps work specifically, Fernando makes the case that infrastructure as code is a friendlier target than general application code, because the shape of a correct answer is more constrained. Terraform is structured; there aren’t that many ways to declare a given resource. And there’s a checkpoint built into the workflow: terraform plan.
Fernando calls the plan “the real validation of what was created,” and that phrasing is worth tightening. HashiCorp describes the command as creating “an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure” — a reviewable preview of proposed changes, not a guarantee about the end state. Attributes that can only be resolved during apply show up as (known after apply) rather than concrete values, and HashiCorp’s own guidance on speculative plans is that other changes to the target system in the meantime “might cause the final effect of a configuration change to be different than what an earlier speculative plan indicated, so you should always re-check the final non-speculative plan before applying.” What the preview does give you is a machine-generated diff a human can actually read, which is more than reviewing generated HCL line by line gives you on its own. Fernando is careful about where the responsibility stays: the plan shows you which resources appear, and it’s still on the engineer to read the attributes and catch settings that were quietly set wrong.
Vladimir’s next step from there is context, in two layers. The first is documentation: let the agent read how resources are actually declared in the current provider version instead of relying on training data. HashiCorp now ships a Terraform MCP server that gives agents live access to registry provider docs, module inputs and outputs, and policies — the mechanical version of what he’s describing.
The second layer is the one that doesn’t come from any vendor’s docs: how things are built inside your organization. His example is small and recognizable — you add a database in Terraform, but that database is going to be connected to other services somewhere in the estate, and the agent has no way to know that from the file in front of it. This is the job Vladimir describes B.O.R.I.S doing as a context layer for engineering teams, and it’s the through-line from episode #12, where the hosts separated what things mean from what’s actually running. His sequencing advice is the part to take away: plug in the context first, “even before you go wild with some skills or whatever else in this rabbit hole of AI” — a deliberate ordering against the temptation to start with the clever layer the series covered in episode #3.
Describing the outcome beats describing the wish
Fernando’s account of moving from refactoring to greenfield work is where the episode gets its clearest failure mode. His first attempt was winging it, and he sees a scaled-up version of that everywhere now: people — often not engineers — opening a session and saying create me a dashboard that does XYZ, then opening the next one and doing the same thing again. The agent goes off and makes every architectural decision on the way to that stated business goal, because a goal is all it was given. His verdict on the result is short: “That’s not what I wanted.”
What changed the output was passing down the standards he’d otherwise apply himself. Write the tests first. Here is an existing project whose structure I want you to follow. Here are the company policies that apply. Agents are good at reading an existing example, recognizing the pattern, and reproducing it in the new thing — so the design intent gets encoded once and reused. The division of labor that emerges is the one both hosts endorse: the engineer owns the solution design, the technologies, and how information flows; the agent owns the mechanical expression of that in code or in a Terraform module. This is the same discipline episode #14 framed as constraints plus a definition of done, applied at the level of a single session.
Fernando is deliberate about how much trust that buys — not 100%, but he points out that 100% isn’t the standard applied to people either. You ask a colleague for something and they might miss a detail; Vladimir extends it to the normal working pattern where you find something yourself, feed it back to your teammate, and both of you update. The useful reframe is that “can I trust it” was never a yes-or-no question about the tool. It’s a question about what verification you have in place, which is why the rest of the episode is about gates.
Bad conventions now spread at machine speed
Vladimir names the failure that worries him most, and it is not a bug. It’s Terraform that gets formatted however each agent felt like formatting it, resources scattered across files with no consistent home, and a codebase that slowly stops feeling like something the team can maintain or reason about. Losing the ability to understand your own infrastructure is a slower failure than an outage and considerably harder to reverse.
His argument for why this is more urgent with agents than with people is the sharpest technical point in the episode. When one person on a team ignores the conventions, the degradation is slow and bounded — the rest of the team still cares, and review catches some of it. With agents, the effect compounds: “every next agent will follow on whatever was created by previous agent.” Yesterday’s shortcut becomes today’s in-repo example, and the next agent treats it as the house style. Fernando’s word for the result is the salad — everything mixed together — and he notes how fast it can spiral. DORA doesn’t establish this successive-agent mechanism, but the pattern is a concrete example consistent with its amplifier framing, running at a much higher clock rate.
Which leads to a genuinely open question the two of them chew on. Readable code was always for the next human, so they could understand the design well enough to extend or fix it. If an agent is the primary reader now, Fernando wonders how much of that still applies — variable names could be far shorter, the way front-end bundlers minify code into something no person can follow (Vladimir supplies the name: Uglify) while the machine reads it perfectly well. Fernando doesn’t push the idea past its evidence: “I don’t think we are there yet, and that’s where comes this trust challenge.” Vladimir’s reframe is the practical one, and it’s the reason conventions matter more rather than less — you establish rules for how code gets written so the next machine understands the codebase and keeps improving it the same way. Style guides just acquired a second audience.
Fix-forward and the review that keeps getting skipped
Fernando returns to the colleague analogy and finds pull request review sitting inside it. In theory the arrangement transfers cleanly: the agent writes the code, the human reviews the PR, and the second pair of eyes catches what the first missed. That is how many teams started. What breaks it is speed — when the next change is thirty seconds away, the temptation is to merge and fix forward if something surfaces later.
His example is a public one, offered as an observation rather than an indictment: looking at the Claude status page, his impression is that in any given week “there is always like three red blocks at least.” His read is that the team keeps moving anyway — fix forward, keep shipping — where in an earlier era a human had to sit down, understand the logic, and find the bug, and that cost alone imposed care on the release process. Two caveats belong on this: it’s one person’s impression of a status history rather than a measured reliability claim, and Anthropic’s own engineering culture is unusually far along the curve. Fortune reported in June 2026 that Claude Code lead Boris Cherny hasn’t written a line of code by hand in months, and Fernando recalls hearing that the harness itself has been rewritten from scratch multiple times — something you would rarely attempt when a rewrite costs quarters of human effort. Harness engineering was the subject of episode #4.
Vladimir adds a corrective worth keeping: rewrites are not new. Software has always reached the point of being labeled legacy and rebuilt in another language or another framework — it just happened on a multi-year cycle. What’s new is the cycle time, and his answer to it is not slowing down but encoding judgment. If the team is going to talk about good code and bad code at this speed, someone has to define which is which, in a form the harness can apply. That is the setup for the episode’s most useful lesson.
Markdown files are suggestions. Gates are not.
Fernando ran the experiment on himself. He built something that grew genuinely complex, deliberately optimizing for outcomes and not reviewing the code in detail, to see what would happen. His guidelines lived where most people’s live now — markdown files, skills, an AGENTS.md. As the codebase grew, those guidelines got ignored.
His conclusion is the sentence to take out of this episode: instructions in markdown are suggestions. The model might follow them, and in a long session under pressure it might not. What he wants instead are deterministic gates — checks like tests that either pass or fail, where there is no room for “the AI forgot that I wanted it done this way.” Anthropic’s own guidance now draws the same line: rules files are advisory, and “a real guardrail needs to be deterministic,” which is what hooks and permissions are for. When the tool vendor and the practitioner arrive at the same distinction independently, it’s worth acting on.
Fernando is candid about the limit: not everything can be validated that way, and he doubts you can gate every property you care about. But the trade is clear enough to plan around. The more of your standards you can express as a gate rather than a paragraph, the more you can let go of line-by-line review and spend your attention on the domain — what should be built, and what it has to satisfy — instead of how each line got written. He’s continuing that experiment and promised to report back in a future episode on whether the higher-level approach holds up.
The bottleneck moves to QA
Vladimir spots the structural consequence: push AI adoption through the whole software development lifecycle and the constraint has to land somewhere. Code gets written fast, reviewed fast — so quality assurance becomes the next place everything piles up, with someone still expected to answer whether the result is actually correct.
Fernando’s experiment produced a precise illustration, and it’s the most quietly alarming moment in the episode. One of his gates was that the agent should write the tests, test-driven style. The problem is that the agent can see the implementation. So it writes tests that reinforce the implementation rather than tests derived from the requirement. You end up with something like 97% coverage and a dashboard that looks excellent, while, in his words, “you’re covering a broken solution.” It works technically and fails against the outcome, and the coverage number gives you no way to tell the difference. His framing of the open problem: how do you ensure quality without that bias, when the thing writing the checks has already read the answer?
There is a partial instrument for this, and it predates agents. Mutation testing injects faults into the code and asks whether the test suite notices — a suite that keeps passing against mutated code was measuring execution, not behavior. Meta has published on using LLMs to generate relevant mutants and the tests that kill them, including for compliance-relevant faults. Treat it as a supplement rather than the answer to Fernando’s problem: mutation score exposes weak assertions, but it can’t tell you whether the assertions encode the requirement, and Meta’s write-up explicitly leaves the Test Oracle Problem — deciding what correct behavior is in the first place — open. So the gate that’s actually missing is not a better statistic. It’s acceptance tests derived from the requirement by someone who hasn’t read the implementation, kept independent of the agent that wrote the code, with mutation testing layered on top to check those tests bite. Coverage is a statistic; mutation score is a stronger signal; neither is an oracle.
What this means for teams and for individual engineers
Vladimir’s closing summary is the argument the episode has been building all along: every organization has to go through this journey, and “it’s not just a button, just enable AI for everyone.” Different departments, different teams, and different types of engineers hit different walls — the platform team’s problem is not the QA team’s problem, and neither is solved by the same rollout email.
Fernando’s message to the management side is that giving out the tool was the easy half. The other half is helping people learn to use it efficiently, and it can’t be delegated to enthusiasm. Some engineers are curious, will build their own process, and will figure it out unprompted. Others resist, often for a reason worth respecting: the change takes away the part of the job they enjoy, because some people genuinely like writing the code. A mandate does not address that; a supported path might.
For individual engineers, his advice is deliberately low-stakes and sequenced: practice on your own side project, take something you already built and ask the agent to optimize it so you can see what comes back, then move up to designing an outcome — this is the solution, this is the design, implement it — and judge the result against your own intent. That’s how the fluency gets built. And he ends on the blunt version, which is the closest thing this episode has to a warning: it is becoming very hard to stay “a relevant engineer in the market” while holding the position that I don’t trust it, I don’t want to use it, because companies now expect that benefit to show up in the work.
One honest note to carry alongside that: perceived benefit and measured benefit are not the same number, at least not in every setting that has been measured. METR’s randomized trial of 16 experienced open-source developers working on their own mature repositories found tasks took about 19% longer with early-2025 AI tools, while the same developers estimated afterward that AI had made them roughly 20% faster. METR calls that “a snapshot of early-2025 AI capabilities in one relevant setting” and does not claim its developers or repositories represent most software work. Its February 2026 follow-up is, if anything, more cautious: the team thinks it is “likely that developers are more sped up from AI tools now,” but describes its new data as “an unreliable signal” with confidence intervals wide enough to span slowdown and speedup. What survives is narrower than a verdict on the tools — in that specific setting, self-reported speedup diverged sharply from measured time. That’s a reason to measure on your own codebase instead of assuming the gain, which is the same instinct the episode arrives at from the other direction: start on tasks whose outcome you can check, and build gates you can trust.
Resources
- AI section, 2025 Stack Overflow Developer Survey — the trust gap the episode opens on, quantified: 84% of respondents using or planning to use AI tools, distrust of accuracy (about 46%) ahead of trust (about 33%), and “almost right, but not quite” as the top frustration. Read it as evidence that a large group shares Fernando’s skepticism — it does not measure how many engineers hold a license and avoid it.
- State of AI-assisted Software Development 2025 (DORA) — the source of the amplifier finding: AI magnifies an organization’s existing strengths and weaknesses, and the returns come from the surrounding system rather than the tool. Useful framing for Vladimir’s compounding-degradation warning, though the report does not test the agent-follows-agent mechanism he describes.
- Steering Claude Code: when to use CLAUDE.md, skills, hooks, and subagents (Claude blog) — vendor confirmation of Fernando’s hardest-won lesson: prompted rules can fail in long sessions or ambiguous situations, and “a real guardrail needs to be deterministic.” Read this if you are currently relying on a markdown file to enforce something that matters.
- Terraform MCP server and the terraform plan command reference (HashiCorp) — the first is the productized version of Vladimir’s context step: live provider docs, module inputs and outputs, registry access, so generated Terraform reflects the current provider instead of training data. The second is the correction to a common assumption in the episode — plan produces a reviewable preview of proposed changes, with unknowns resolved at apply time, so re-check the final plan before applying.
- Migrate to Java 17 (OpenRewrite) — the exact migration Vladimir names as ideal first-agent work, with many of the common changes automated by a repeatable recipe. Read the guide’s own limits before assuming it covers your codebase: it lists unsupported components and cases that still need manual fixes. Worth comparing before you spend tokens — sometimes the mechanical part of the task already has a mechanical tool, and the transferable lesson is the checkable outcome, not the agent.
- LLMs are the key to mutation testing and better compliance (Meta Engineering) — a partial answer to Fernando’s 97%-coverage-on-a-broken-solution problem: inject realistic faults and see whether the suite notices. Note what it does not claim — the article leaves the Test Oracle Problem open, so requirement-derived acceptance tests are still the gate you have to supply yourself.
- METR on AI and developer productivity: the early-2025 randomized trial (19% slower measured, 20% faster estimated by the same developers) and the February 2026 update (likely more speedup now, but the team calls its own new signal unreliable). Use the pair as evidence of a perception gap in one specific setting and an argument for measuring your own adoption, not as a verdict on the tools.
- Prior episodes this one builds on: #12 — Semantic Layers, Context Layers, and Agents That Stop Guessing is the deep version of the context step Vladimir puts before everything else; #14 — Loop Engineering in DevOps is where the enter, enter, continue babysitting pattern gets replaced with constraints and a definition of done; #6 — The Big AI Squeeze covers the token economics behind the leaderboard reversal.