Your AI Tools Were Supposed to Make You Faster

TL;DR: The seats are bought, adoption is green, and the roadmap is moving at last year's pace, because the speedup is real at the keystroke level and leaks out downstream. Research shows AI's gains are largest on isolated greenfield tasks and can invert on mature codebases, while churn, duplication, and bug rates climb with heavy adoption. The fix isn't canceling the seats; it's measuring delivery instead of adoption, and closing the leaks (review load, rework, oversized PRs) that are eating your gains.

You bought the seats. Rolled them out properly, even - onboarding, champions, the internal demo where the agent scaffolded a service in ninety seconds and everyone made the same impressed noise. The adoption dashboard has been green for two quarters.

And the roadmap is moving at exactly last year's pace. Somewhere between the autocomplete and production, the speedup is leaking out, and someone on the board is about to ask for the ROI slide you don't have.

The gains are real. So are the leaks.

Hold two research results in your head at once, because both are true.

First: Peng et al. (2023) ran a controlled experiment where developers with Copilot completed a greenfield task - an HTTP server from scratch - 55.8% faster than the control group. On isolated, well-specified, from-zero work, the acceleration is real and large.

Second: the METR randomized trial put 16 experienced open-source developers on 246 real tasks in mature codebases they knew deeply. With AI allowed, they were 19% slower. The detail that should worry you most: before the study they expected a 24% speedup, and after experiencing the slowdown firsthand they still believed they'd been sped up by 20%. Your developers telling you AI makes them faster are being completely sincere. Sincerity is not measurement.

So which study is your org? If your engineers mostly write small greenfield services, closer to the first. If they mostly evolve a large, aging codebase - most orgs - closer to the second, and the leaks compound downstream. GitClear's analysis of 623M code changes found code-block duplication up 81%, copy/paste rising from 9.4% to 15.7% of commit content, refactoring line-moves down 70%, and short-term churn up 15%: code written faster, rewritten sooner, and duplicated instead of designed. Faros's 2026 AI Engineering Report found bugs per developer up 54% at high AI adoption. And the 2024 DORA report estimated that a 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability - individual productivity up, system throughput down.

The mechanism is no mystery: your bottleneck was probably never typing speed. Pour faster authoring into a fixed review capacity and you get longer review queues of bigger PRs; add more defects and duplication and you get more rework and debt that compounds. The tools accelerated the one stage of your pipeline that wasn't the constraint. That's the whole productivity paradox in a sentence.

Do this first - free, this month

1. Retire adoption as your headline metric. Seats active and suggestions accepted measure usage, not return. Nobody reports "Jira logins" as project health.
2. Baseline delivery, then watch it move. Cycle time, change failure rate, rework - the four keys plus a rework rate you trust, compared before and after AI-heavy periods. This is the only ROI frame a skeptical board accepts, and the full method is in measuring AI coding ROI.
3. Watch the quality canaries. Duplication trend, churn (code rewritten within weeks of landing), and review load. GitClear's numbers above tell you exactly which gauges drift first.
4. Cap the blast radius. AI makes it effortless to open a 900-line PR that no human can actually review. Keeping changes reviewable is the cheapest leak-fix available, and the guardrails for agent-written code are mostly review discipline. If you're earlier in the rollout, run the readiness checklist before scaling seats.
5. Ask, but pair asking with data. Developer experience matters and surveys catch things metrics miss. Just don't let sentiment be the ROI slide. METR showed a 39-point gap between felt and measured speed in expert hands.

What "good" looks like: AI gains visible in cycle time and change-failure rate, plus an honest map of where AI helps your codebase and where it hurts it.

How you'd actually see it

Everything above is measurable from data you already produce (git, PRs, CI) if something correlates delivery and quality metrics against AI-assisted work over time. That's what Busfactor's AI impact view does: AI-versus-human rework cohorts, each tool's merged-PR cohort compared to the baseline on cycle time, review latency and size, the areas where AI churn concentrates, and a judged readiness verdict, so the ROI slide writes itself from receipts instead of sentiment.

The honesty note, because it's load-bearing: identifying which code is AI-written is an estimate. No tool can attribute it perfectly, ours included. We label it probabilistic rather than quoting it as fact, and never use it to grade a person. The delivery and quality metrics around it, though, are hard numbers. If a vendor shows you a precise "AI productivity percentage" with no error bars, you're looking at the vibes slide with extra steps.

The door

Nothing here says cancel the seats. The authoring gains are real; the research says the downstream leaks are where the money goes. Fix the leaks and you keep both: baseline delivery this month, watch the canaries, cap the PR sizes. If the deeper issue is that the speedup has nowhere to go, find your real bottleneck first. And if the board question is really a budget question, that one has its own answer. Or connect your repos and let the AI impact view tell you in a week which study your org is living in. The seats were the easy part. The return is a system property, and system properties can be engineered.