With AI code generation, software developers create code faster than before. But how can the QA engineers and the CI/CD system absorb this increased workload? In this article, Sergei Cherednichenko suggests some approaches on how to deal with this situation.
Author: Sergei Cherednichenko, https://linkedin.com/in/sergei-cherednichenko/
Software delivery teams produce changes faster than many continuous integration and continuous delivery (CI/CD) systems can comfortably absorb. The pressure appears in familiar places: pull-request queues grow, reviews become less thorough, flaky tests interrupt builds, merge conflicts remain unresolved, and resource-intensive validation slows the path to production.
Across CircleCI projects, average throughput grew 59% year over year. For the median team, feature-branch throughput increased 15% while main-branch throughput declined 7%. Main-branch success rates fell to 70.8%, the lowest in over five years and well below CircleCI’s recommended 90% benchmark (Figure 1). DORA’s 2024 research found a similar pattern: increased artificial intelligence (AI) adoption was associated with an estimated 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability. AI-assisted development amplifies the problem by increasing the volume of code entering the system, but the larger issue is throughput: the delivery pipeline needs to control flow as carefully as it processes individual changes.
(click on figure to enlarge)
Figure 1. The delivery gap: year-over-year change in CI throughput by branch type. (Source: CircleCI, The 2026 State of Software Delivery.) Image courtesy of Sergei Cherednichenko.
Distributed and data-streaming systems offer a useful model. When producers send work faster than consumers can process it, the system applies back pressure: the lagging consumer signals upstream, and producers slow down instead of flooding a buffer that will eventually overflow. CI/CD can respond to congestion the same way, slowing the intake of new work and directing effort toward whatever is preventing existing work from moving forward.
Put back pressure into the delivery flow
A developer who previously created one or two pull requests a day may now, with AI assistance, produce five or ten. Review capacity does not automatically increase at the same rate, so changes queue up waiting for human attention. In a streaming system with back-pressure support, the gap between what has been produced and what has been processed is called consumer lag, and it is watched as closely as throughput. The age of the oldest open pull request is the delivery pipeline’s equivalent.
Back pressure changes that dynamic. A CI/CD environment can limit work in progress or the number of outstanding pull requests. Once that threshold is reached, automated feedback points contributors to work that increases the system’s overall throughput: resolving merge conflicts, investigating build failures, repairing tests, or integrating feature branches that are blocking other changes. A developer fixing a blocking test may contribute more to throughput than one generating another feature branch (Figure 2).
Figure 2. Back pressure in a streaming system (A) and its CI/CD equivalent (B). Image courtesy of Sergei Cherednichenko. (click on figure to enlarge)
Little’s Law makes the cost explicit: average lead time equals work in progress divided by throughput. If a team can review 13 pull requests a day and 40 are open, the average change waits about three days, no matter how quickly it was written.
Treat the CI/CD pipeline as a throughput system
Build failures become more informative when they are examined as signals about pipeline capacity rather than isolated incidents. A failure may represent a legitimate defect introduced upstream. It could indicate that the infrastructure is struggling with increased throughput. It may come from a security or policy gate triggered by a change. Or it may be noise: at Google, almost 16% of 4.2 million tests showed some flakiness, and 84% of pass-to-fail transitions involved a flaky test. Classifying those causes helps teams determine where effort will have the greatest effect.
When defects repeatedly enter the pipeline, earlier validation and stronger review can reduce downstream rework. When infrastructure is the source, attention can move toward building capacity and reliability. When security checks repeatedly stop changes late, earlier automated checks can expose those issues sooner. Flaky tests deserve the same treatment streaming systems give a “poison message” that repeatedly fails: move it to a dead-letter queue, so it stops blocking the work behind it and can be repaired on its own track. The objective is not to complete more builds but to identify where work accumulates and remove the constraint limiting overall flow.
Make testing faster without simply adding more tests
Testing can become a significant throughput constraint. Adding more tests indiscriminately worsens that constraint. A more selective approach includes:
- Run fast tests first. Deterministic and unit tests provide early feedback before expensive integration tests consume additional time and infrastructure. Data pipelines apply cheap filters as early as possible so expensive stages never process work that will be discarded. Fast, deterministic tests play the same role in CI.
- Select tests according to the change. Rather than executing every available test, the pipeline identifies the subset most relevant to a particular modification. AI can augment this selection when the task is sufficiently bounded.
- Evaluate test value. Some tests consume substantial resources while rarely providing useful information; others run infrequently but protect against consequential failures.
- Use automation selectively. AI helps identify flaky tests, recommend relevant tests, and assist with straightforward fixes, but its output needs verification too. One study found security weaknesses in 29.5% of AI-generated Python snippets in GitHub projects.
A smaller, more relevant, and more reliable test path can shorten feedback cycles while reserving expensive validation for changes that warrant it.
Route changes according to risk
Not all changes need the same validation path. Treating a documentation update and a complex cross-service feature the same way consumes review capacity that is better reserved for higher-risk work.
A risk-based CI/CD process classifies changes according to scope and historical behavior. Limited changes, such as test updates or mechanical migrations, receive greater automated review. Changes affecting complex system behavior, user experience, or multiple services receive closer human attention (Figure 3).
The system then records what happens after automatically reviewed changes are merged. If a particular category consistently produces safe outcomes, automation expands gradually. If incidents follow, policies are tightened.
Figure 3. Risk-based routing by blast radius and ease of validation. Placement is illustrative. Image courtesy of Sergei Cherednichenko. (click on figure to enlarge)
AI agents fit best into bounded work inside this model: resolving simple merge conflicts, checking schemas, updating tests, improving documentation, or performing repetitive migrations. In the .NET runtime repository, an AI coding agent’s pull requests merged 84.7% of the time for cleanup tasks but only 54.5% for performance work. End-to-end integration and judging whether a change delivers the intended result still depend on broader system knowledge and human judgment.
Measure delivery performance, not code production
Greater throughput matters when useful changes make it through the system. Pull request counts, lines of code, build counts, and similar activity metrics become less informative as AI makes code generation easier. A streaming system is judged by end-to-end latency and delivered events, not by how fast producers write, and its tail latency matters more than its average. Delivery pipelines deserve the same scrutiny. A stronger measurement model tracks queue time, end-to-end delivery time, flaky-test rates, and deployment failures, helping teams identify bottlenecks without sacrificing reliability.
The emerging CI/CD challenge is less about generating software faster and more about balancing the entire delivery stream. Back pressure, work-in-progress limits, selective testing, risk-based routing, and automated feedback give teams methods to regulate that flow. AI fits into the model by removing specific constraints, but it does not define the model.
Like a well-designed stream processor, it’s critical to evaluate a delivery pipeline by how reliably and quickly it moves work through to completion. Pipelines that let work move smoothly through verification, integration, and deployment without overwhelming the next stage will shorten delivery time without compromising release quality.
About the Author
Sergei Cherednichenko works on distributed systems and real-time data infrastructure. A software engineer and open-source contributor, he has spent 11 years building large-scale platforms at startups and internet-scale companies and writes about the challenges of operating them at scale. He holds an M.S. in Computer Engineering from ITMO University. Connect with Sergei on LinkedIn.
Leave a Reply