DoorDash has built a multi-agent LLM system to automate stale feature flag cleanup across its codebase, reporting that the approach can turn a task that often takes engineers one to two hours into a process that averages 13.8 minutes and $4.79 per cleanup. In an evaluation of 50 stale flags, the system generated usable pull requests for 45 of them, with no bugs or regressions reported in the tested changes.
DoorDash’s feature flag cleanup problem at scale
The project exists because DoorDash operates at a very large feature-flag footprint. Its experimentation platform manages more than 60,000 feature flags across roughly 623 repositories and adds about 2,300 new flags each month. Within that environment, the company identified more than 1,000 stale flags that still needed cleanup.
DoorDash classifies a flag as stale when it has not been modified for 90 days, remains referenced in code, is neither archived nor retired, and is not explicitly excluded. A daily process then creates Jira tickets for the flags that match those criteria, which gives engineers a continuous queue of cleanup work.
On paper, removing a flag can sound simple. In practice, DoorDash said the work is complicated by dependency-injected wrappers, where the flag definition, client call, and business logic may be spread across multiple files. Even a basic Boolean flag can require changes in five to 20 files, including tests.
Why DoorDash did not rely on a simple search-and-delete approach
DoorDash compared its approach with existing tools, including Uber’s open-source Piranha. Piranha uses abstract syntax tree-based transformations to identify and remove stale feature flag code. DoorDash found that approach did not fit its dependency injection patterns, where the relationship between a flag and the application logic is semantic rather than directly expressed through matching syntax.
That distinction matters because not all feature flag cleanup can be done safely by looking for a literal reference and deleting it. If the flag is threaded through wrappers, interfaces, or indirect calls, the code changes may require understanding how the flag behaves across files and layers. DoorDash’s system was designed to handle that broader context with LLM-based reasoning and validation steps.
How DoorDash’s multi-agent LLM workflow works
The company built its workflow using Google’s Agent Development Kit and divided the process into two phases. The first phase begins with an orchestrator running Claude Sonnet. That orchestrator retrieves stale flag tickets from Jira, searches the relevant repositories, and queries DoorDash’s experimentation platform through Model Context Protocol, or MCP.
Through MCP, the system collects metadata such as rollout percentage and target value. An engineer then reviews the generated report and confirms the target value before code changes begin. DoorDash’s description makes the approval step a deliberate checkpoint rather than a fully autonomous action, which helps keep the system aligned with the intended cleanup.
MCP plays an important role here because it offers a standardized way to connect AI applications to external tools and resources. In this case, it gives the agents a structured path to the experimentation data they need before making code edits.
Isolated worktrees and concurrent cleanup agents
The second phase uses Claude Opus cleanup agents operating in isolated Git worktrees. DoorDash ran up to four agents concurrently per repository, allowing multiple stale flag cleanups to proceed in parallel without sharing mutable state.
In this phase, the agents locate flag references, decide on a cleanup strategy, modify source code and tests, and then run validation checks. Those checks include builds, tests, JaCoCo patch coverage, and Detekt static analysis. A pull request is opened only after those checks pass.
DoorDash also imposed a one-hour timeout for each agent. Gradle runs without its daemon in order to avoid state sharing between worktrees, which is important when several agents are operating at once.
What the evaluation showed
DoorDash evaluated the system on 50 stale flags. The results included 31 first-pass merges, 14 revisions, and five engineer interventions. Of the 45 usable pull requests produced, the company said the system averaged 13.8 minutes and $4.79 per cleanup.
The evaluation also broke down results by complexity. Simple flags achieved a 100% single-pass cleanup rate, while medium complexity flags reached 94% and complex flags reached 85%. The five engineer interventions were needed in cases involving deep call chains and cross-interface parameter threading.
DoorDash reported no bugs or regressions in the 50 evaluated changes. That is a meaningful result for a system that is being asked to remove code, update tests, and preserve behavior across a large production codebase.
What the numbers suggest for engineering teams
The headline result is not just that DoorDash used LLMs, but that it applied them to a specific maintenance burden with measurable output. Feature flag cleanup is a common source of technical debt because flags accumulate quickly, and stale ones can make code harder to read, test, and evolve.
- More than 60,000 feature flags are managed across roughly 623 repositories.
- About 2,300 new flags are created each month.
- DoorDash identified more than 1,000 stale flags.
- The evaluated system produced usable pull requests for 45 of 50 stale flags.
- Average time per cleanup was 13.8 minutes, with a reported cost of $4.79 each.
Those numbers show why the company is interested in automation. Even a modest reduction in manual effort can matter when cleanup work spans thousands of flags and many repositories. The approach also suggests that AI agents may be most useful when paired with strong guardrails, explicit validation, and human approval at key decision points.
Planned improvements and next steps
DoorDash said it plans to add confidence scoring for lower-risk cleanups. That would help the system identify which tasks are likely safe to automate with less oversight and which ones may need more caution.
The company also wants a post-cleanup code quality pass to catch issues such as misleading variable names after a flag is removed. That is a good reminder that deleting a flag is only part of the maintenance story; surrounding code often needs refinement too.
The work has been accepted for the ICSME 2026 industry track, which places it in a software maintenance and evolution context rather than as a general AI showcase. In that setting, DoorDash’s result is notable because it combines LLM reasoning, experimentation metadata, isolated execution, and automated checks to solve a very practical codebase hygiene problem.
For teams dealing with large numbers of stale toggles, the takeaway is not that feature flag cleanup should be handed over blindly to agents. It is that a carefully constrained, well-instrumented multi-agent workflow can meaningfully reduce repetitive maintenance while still leaving room for engineering oversight.
Source: Original report
Was this helpful?
Explore more: AI Automation Services More AI & Automation Tech News
Last Modified: September 19, 2026 at 10:33 pm
0 views
