Is Your CI/CD Ready for Six Repos an Hour?

CI/CD approvals leave security fixes waiting while agents push to six repositories an hour.

Subscribe at Apple Podcasts Spotify RSS

CI/CD built for one push a day can't keep pace with agents changing six repos an hour. Faster triage and verification can help security fixes reach production sooner. The hosts examine why merging a Dependabot pull request leaves production exposed, how infrastructure context narrows thousands of alerts to relevant vulnerabilities, and where agents can verify dependency bumps and assess deployment risk. They also cover restartable services, local code review, and cross-model checks followed by before-and-after explanations.

Summary

Finding a vulnerability used to be the slow part. Now it’s the fast part. More than 100,000 CVEs have been published this year, over twice last year’s total. For vulnerabilities that affect you, available fixes still have to be validated and delivered into production, often on systems that were never built for that pace.

That’s where teams get stuck. When you merge a Dependabot pull request into main, it updates the affected dependency in that branch. Production stays vulnerable until the fixed version is deployed. A fix that sits between environments waiting for a human approval stays exposed the whole time. And the pipelines we’ve worked with were built for one person pushing about once a day. Agents can now push to six repositories an hour.

Our argument is that the bottleneck has moved from finding to delivering. The foundations still decide whether you keep up: immutable infrastructure, services you can restart at any time, and an inventory that tells you what you actually run. Once those are in place, agents earn a place in four spots: triage, risk estimates before deploy, mechanical verification of dependency bumps, and review on the laptop before a pull request is opened. In our case triage meant 4,000 open vulnerabilities, of which two actually affected us.

Four Thousand Vulnerabilities, Two That Matter

You can’t patch everything at this volume, so the first job is to find out whether you’re affected at all.

How do you tell which vulnerabilities actually affect you?

Match them against an inventory of what you run, and let an agent do the matching. An RSS feed, Dependabot and a scanner all produce a flood. Triage means answering whether that package, version and configuration exist in your environment, and how far the damage would spread if they do.

You won’t do that by hand at this volume. Either a product or an agent will estimate the blast radius, and it does much better when the inventory already exists and it doesn’t have to discover your estate first. That’s the job a context layer like B.O.R.I.S does for an agent. We covered why inventory comes first with Mohit Bansal in episode #24.

We don’t agree on Dependabot. Used one way, it’s the start of an automated fix: the pull request is already waiting for you. In our own repositories, the free setup turned into noise.

“There’s a four thousand vulnerabilities, but only two of them are actually affecting affecting us.”

A couple of weeks ago we shut down free Dependabot across our repositories. We now rely on a third-party security tool that says exactly what needs patching. Four thousand alerts and a scatter of pull requests nobody looked at train everyone to ignore the two that matter.

To be fair to GitHub, Dependabot alerts and security updates are available on every plan, and so are GitHub’s preset auto-triage rules, which filter out a share of false positives. The paid layer is GitHub Code Security, part of GitHub Advanced Security. It adds custom auto-triage rules, so you can write your own conditions for which alerts get dismissed, snoozed or turned into pull requests. That’s a much cleaner experience than the free presets alone. Either way, triage only works if whatever does it has your context. A model reading CVE descriptions without knowing what you run is guessing.

Merged Is Not Deployed

Say triage confirms you’re affected and the fix is merged. Much of the security tooling we’ve used stops counting there.

Where do security fixes get stuck after the merge?

Between environments. Dependabot opens the pull request, you merge it, and the tooling turns green. That updates the affected dependency in the branch. Production remains vulnerable until the fixed version is deployed, and in the tooling we’ve seen, whether the fix gets there is nobody’s metric. Fixes pile up in the pipeline. If your release policy requires human approval for production, the clock runs while the fix moves through the pipeline and keeps running while you look for someone to press the button. Time to fix is the wrong measure. Measure time to deploy.

Approvals are already drifting. Many reviewers will use AI to review, and some will have an agent approve through the GitHub CLI. So decide on purpose what the gate is. Write the policy for what can go out without a human and what can’t, instead of letting a rubber stamp decide by default.

We see agents speeding up the pipeline in three places:

  • Risk estimate before production. An agent reads every patch staged for release and estimates its impact. With a context layer and live read tools such as B.O.R.I.S, it can check against the infrastructure that’s actually deployed instead of guessing, so the release decision rests on evidence.
  • Smoke tests after deploy. An agent runs once the deploy finishes and looks for changed behavior in production. Episode #25 walks through running that job unattended.
  • Mechanical verification of dependency bumps. An agent takes a snapshot of the repository, merges the pull request, runs the scripts, does some exploratory testing, then merges and pushes.

The third one has a trap we learned the hard way. When Dependabot triggers an ordinary pull request workflow (pull_request, push and a few related events), GitHub Actions secrets aren’t available. The GITHUB_TOKEN is read-only by default, and only secrets you stored as Dependabot secrets get populated. pull_request_target follows different rules: secrets are withheld there only when the pull request’s base ref was itself created by Dependabot. If your CI depends on secrets, it fails on exactly the pull requests you most want to merge quickly. You can script around that per repository, but those scripts break because every repository is slightly different. An agent handles the differences better, and the job doesn’t need a big model, so the tokens are cheap.

Patching an RCE Isn’t the End of It

Getting the fix deployed closes the door. It doesn’t tell you whether someone already walked through it.

What should you do after patching a remote code execution bug?

First distinguish routine patching from suspected compromise. For routine patching, roll out the fix. For suspected compromise, contain the affected systems and preserve evidence before recovery. The usual flow skips that question: a tool or an agent flags the asset, you patch it, close the ticket and move on. If someone already exploited the remote code execution (RCE) bug, they may be past that asset and hopping to the next system.

Our routine in one system is blunt. Once an RCE fix is deployed, we restart all of production, around 80 to 90 microservices. With no sign of compromise, that makes sure every process runs the fixed code and drops anything that might be living in a running process. It’s also a test many teams can’t pass: can you restart production right now, without asking anyone, and be fine?

That routine is not incident response. If you have any sign someone got in, restarting first destroys the volatile evidence you need, and it doesn’t remove an attacker who has stolen credentials or planted persistence somewhere a restart doesn’t touch. The Canadian Centre for Cyber Security’s guidance covers the first steps: keep potentially compromised systems running, isolate them and preserve the evidence. What comes after is our recommendation: investigate, then recover onto clean systems rather than the ones you no longer trust. Rotate any credentials the attacker could have read as part of that recovery, because a restart leaves stolen credentials just as valid as before.

“We don’t want to restart it because if you restart, you don’t know what’s not going to come up.”

One of us has worked in teams with exactly that Windows server, the one everybody’s afraid to touch. Then an RCE lands on it, and you’re afraid of restarting it and afraid of leaving it alone.

The fix is immutable infrastructure as code, and it isn’t new. It’s also what makes clean recovery possible: you rebuild from a known-good definition instead of nursing a box you no longer trust. Chaos engineering takes it further, because if any service can be restarted at any time, a restart is a non-event. Spot capacity forces the same discipline. Fargate Spot interrupts tasks with a two-minute warning and a SIGTERM, and Fargate also restarts containers when AWS patches the underlying hosts.

The gotcha is state. Teams run containers and still keep sessions inside them, then find out on the first restart that users lose whatever they were doing. Make services stateless before you need to restart them in a hurry.

Your Pipeline Was Built for One Push a Day

Everything so far assumes your pipeline can carry the extra load. Many can’t.

Is your CI/CD ready for agent-speed delivery?

Probably not. The continuous delivery pipelines we’ve built and inherited assume that one person pushes maybe once a day. With a good, detailed plan, parallel agents can push to six repositories every hour. The pipeline becomes a bottleneck, and so does the manual approval.

AI is a multiplier, so you need something to multiply. A team with a good software development lifecycle can build on it: CI/CD, blue-green deployments, immutable infrastructure as code. A team with a manual, fragile process gets more manual, fragile process, because the agents copy what you do. We made the same argument in episode #16. If the basics aren’t there yet, use AI to put them in place now. The volume of security work is only going to grow, and people are surely training open-weight models specifically for offense.

We don’t see many teams reinventing their pipelines yet. Paul Stack’s essay AI Broke the Assumptions Behind CI is one of the few concrete attempts. It cites CircleCI data showing throughput up 59% year over year while main-branch success rates fell to 70.8%. His answer is to run verification before the pull request and have CI check the evidence instead of redoing the work. We’d like to have him on the show.

There’s a counterpoint to “foundations first”. Much of Africa skipped landlines and went straight to mobile. If a new delivery model is emerging, some of today’s foundations may turn out not to be needed. We don’t know which ones yet. Restartable services and a real inventory don’t look optional, though.

Push Review Left, to the Laptop

The oldest rule in delivery is to push verification left, toward the developer, because problems cost more to fix the further right you find them. Agents give that rule a new twist.

Where should AI code review run, on the pull request or locally?

Both, but treat the pull request review as the last line of defense, not the first. Running an AI reviewer on every agent-written pull request is fighting fire with fire. It’s worth doing, because you don’t know what checks happened before the push. The deeper review belongs on the laptop.

The reason is partly economics. Locally you run on a subscription. In CI you most likely pay API token prices. Skills and Markdown instruction files let local agents go past unit tests into exploratory testing, so you can afford much deeper review before anything leaves the machine. Episode #23 covers why tokens and review run out before Git does.

CI still has a job. It keeps the main line green: the code compiles and the tests pass, so whoever checks out next doesn’t clean up after you. We remember teams that couldn’t run long regression suites because they never had a green baseline. The build itself was broken. Flaky tests, green one run and red the next, are still the enemy there.

Some AI-native teams go the other way. They push forward as fast as possible and let agents watch production, open the fix and redeploy. Greenfield projects started with AI tend to have better structure and tests when professional developers set them up, and some organizations now start AI-only projects to get that. Full rewrites, such as Python to Go or Rust, are suddenly on the table, when nobody would have considered them while they took a year. Fix-forward is a valid approach, but it depends on deploys being cheap and restarts being safe, which brings you back to the foundations.

Coordinating Agents Without a Pull Request

Once review moves to the laptop, the next question is why you need the shared main line at all.

Do agents still need pull requests and a main branch?

For teams, yes, for now. The main line exists so several people’s work can be integrated continuously. If all the parties are agents on one laptop or a couple of servers coordinating with each other, you might not need it. In the meantime we’ve improvised.

“We basically use like a Slack as a message bus between the agents”

One of us was pushing hard on a single system alongside a collaborator, and their changes kept rubbing elbows. So each of them told their agents to post what they were doing to a shared Slack channel and to check what the other person’s agents had posted, because they might be building in parallel without knowing it. It wasn’t fancy. At some point the agents were posting to each other, and you could ask your agent to go and check what the other one was up to. It’s a hacky hack and we’re not recommending it. It does show what’s missing from the tools.

We do the same thing in Linear. Linear marks changes made through MCP with your user plus an MCP label. That’s handy, because you can filter out the agent noise. Nobody can read all of it anyway.

Cross-Review, Then Ask for Before and After

Agents writing code, agents reviewing it and a human clicking approve can add up to nobody having reviewed anything.

How do you review a 5,000-line change you didn’t write?

Run a cross-model review, then ask an agent to explain the change as before and after, and read that. For the cross-review, we run several models over the change and merge their findings. It finds a lot. The more rounds you run, the more it finds, so the limit is your patience. Running it at least once is a must. The tooling behind it came up in episode #10.

The second step is the explanation. Front-end developers often get a preview to look at. On the back-end and infrastructure side, the equivalent is asking the agent how the world looks before the change and after it. Then you go by smell. If something doesn’t make sense, you dig into why it was done that way. That sense of smell comes from experience, unfortunately. You learn what wrong looks like by having seen it.

There’s a catch. You’re trusting that the story the model tells you matches the code. With a 5,000-line change, that story is your best chance, so more and more of the work ends up being trust in the agents. You can narrow the gap by giving the model your own conventions alongside public best practices for Dockerfiles or Terraform, and asking it to check the difference. Best practice isn’t always right for your case, and the agent needs to know where you deviate on purpose.

Why the Fix Queue Is Growing Now

Everything above matters more because the queue feeding your pipeline is getting longer every month.

Why is the number of CVEs exploding in 2026?

Because AI models now find vulnerabilities faster than humans can, and more people have access to them. VulnCheck counted 100,222 CVEs published in 2026 by November 13, more than double 2025, and its disclosure lead told The Register that AI vulnerability hunting is driving most of that.

This is the wave Anthropic warned about in April, when it announced Claude Mythos Preview and limited access to organizations that maintain critical software. The frontier labs still gate this work. Anthropic’s generally available models carry conservative cyber safeguards that block most cyber work, and the stronger capabilities sit behind a verification program. Our read is that open-weight models, including Chinese ones, have caught up enough that those gates matter less. Anyone who doesn’t want to get on an allowlist has an alternative without the guardrails. We can’t prove that’s the main driver, but it fits the curve.

You can feel it without reading the news. One of us gets the AWS Security Bulletins RSS feed in a Slack channel. Last year it got a post or two a year. Now something shows up every week. The current page lists eight bulletins in the three days up to November 19, including a PHP Foundation security release and an nginx HTTP/2 issue. AWS publishes what it finds rather than keeping it under the rug, and the volume is the signal.

It isn’t one ecosystem either. PHP, Java, Node and Go have all had their turn. nginx had two unauthenticated RCE bugs in its HTTP/2 handling disclosed on November 18, one present since version 1.9.5 in 2015. If you run PHP on Windows behind nginx, it may be time to consider your life choices. They catch up with you.

The hard part is validation. Microsoft shipped 1,094 fixes in November’s Patch Tuesday after 1,474 the month before. Claude or Codex will write a fix. Neither tells you whether it breaks something three months later. The 24-hour Microsoft 365 disruption on November 12 traced back to a July update to network gateway servers that passed load testing and ran for over three months. Then a failover hit while about 14% of gateway capacity was down, and the latent defect took out more than 80% of the servers. Microsoft hasn’t said what the July change was meant to fix. Either way, more patches mean more changes that can look fine on day one.

What This Means for Teams

The CVE wave won’t wait for your pipeline. Treat security patching as a delivery problem: detect, triage against what you run, get the fix through the pipeline, activate it, then watch for anything odd in the blast radius. Each stage has a place for an agent, and each stage fails without the foundation under it. Keep a separate playbook for suspected compromise, where you contain, preserve evidence and rotate exposed credentials before you restart anything.

Four things to do this week:

  • Test a restart. Pick one production service and restart it without asking anyone. If you can’t, or something fails to come back, that’s the first fix to make.
  • Measure time to deploy for your last security fix. Count from merge to running in production, including the time it waited for approval.
  • Check your Dependabot workflows for secrets. If CI on pull_request events needs secrets, add them as Dependabot secrets or those pull requests will keep failing.
  • Add a before-and-after explanation to your review. After the cross-model review, ask the agent to describe how the system behaves before and after the change, and read it before you approve.

Common Questions, Answered

Why do my Dependabot pull requests fail in GitHub Actions?

Add the secrets your CI needs under Dependabot secrets, with the same names. When Dependabot triggers a workflow on events such as pull_request or push, GitHub Actions secrets aren’t available, only Dependabot secrets are, and the GITHUB_TOKEN is read-only by default. pull_request_target workflows follow different rules and lose secrets only when the base ref was created by Dependabot. An agent can also take over the merge-and-verify step for dependency bumps, and a small model is enough for that job.

Does restarting a compromised server remove an attacker?

Not reliably, and it can destroy the evidence you need. A restart clears what lives in running processes, but stolen credentials and persistence outside the process survive it. If you suspect compromise, keep the systems running and isolate them, and preserve evidence. From there, we’d investigate, then recover onto clean systems built from immutable infrastructure as code, and rotate any credentials the attacker could have read. Restarting everything after an RCE patch makes sense as routine activation when there’s no sign of compromise, not as a substitute for incident response.

How do I cut down Dependabot vulnerability noise?

Triage alerts against an inventory of what you actually run, and patch what affects you first. Turn on GitHub’s preset auto-triage rules, which are free. Custom rules come with GitHub Code Security. In our case, 4,000 open vulnerabilities came down to two that affected us. A security tool that understands reachability, or an agent with access to your infrastructure inventory, can tell you which ones matter.

Resources

Give your AI tools the context they're missing

Partners

Need a HIPAA foundation first?

Talk to our partner network