The AI Developer Productivity Study: 19% Slower

May 29, 2026

An AI developer productivity study by METR, led by Joel Becker and Nate Rush, found that using AI coding assistants increased task completion time for experienced open-source developers by 19%, despite developers perceiving a 20% speedup. This finding contradicts the common expectation that generative AI inherently boosts developer efficiency and highlights a productivity paradox where perceived benefits do not align with actual output. The study suggests that factors such as increased cognitive load from prompt engineering and code review may contribute to this unexpected slowdown.

The Core Finding: AI's Unexpected Impact on Developer Productivity

The METR study, conducted by Joel Becker and Nate Rush, revealed that experienced open-source developers using AI coding assistants experienced a 19% increase in task completion time. This finding directly contradicts both the developers' initial expectations and their post-task perceptions. Prior to the study, developers anticipated a 24% decrease in task time, and even after using AI tools, they still perceived a 20% speedup. Nate Rush, a lead author, had personally expected a "2x speed up" before the results were analyzed. This productivity paradox indicates that while developers felt more efficient, their actual output was slower. The study highlighted that generative AI, despite its promise, introduced new activities such as prompt engineering and reviewing AI output, which consumed any time saved on active coding, searching, testing, or debugging. This increased cognitive load contributed to the overall slowdown, challenging the prevailing belief that AI invariably boosts developer productivity.

Discrepancy Between Perception and Reality

The METR study uncovered a significant gap between developers' perception of AI’s impact and its measured effect on productivity. Before the study, open-source developers anticipated a 24% reduction in task completion time when using AI coding assistants. After utilizing these tools, their perception remained positive, with developers believing they had achieved a 20% speedup. However, the actual data revealed a 19% increase in task completion time, meaning developers were slower, not faster. This cognitive disconnect highlights a productivity paradox where the subjective experience of using generative AI tools like GitHub Copilot or other web LLMs feels beneficial, but objective metrics show a detriment. Neil Thompson of MIT Sloan noted that developers thought the AI tool had increased their speed by at least 20%, despite being 19% slower. This suggests that while AI coding assistants may make the development experience "cognitively easier," this ease does not necessarily translate to improved efficiency, and can even mask a slowdown. The increased cognitive load from prompt engineering and code review contributes to this discrepancy.

Identifying the Causes of the Productivity Slowdown

The METR study identified several factors contributing to the 19% slowdown experienced by developers using AI coding assistants. While AI tools reduced active coding time, searching, testing, and debugging, these gains were offset by new activities introduced by generative AI. One significant factor was the increased cognitive load associated with prompt engineering. Developers spent additional time crafting precise prompts to guide the AI, a task that requires a different skillset than traditional coding. Another cause was the necessity of reviewing AI output. Developers had to meticulously check AI-generated code for correctness, efficiency, and adherence to project standards, often involving debugging AI-generated code. This review process could be time-consuming, especially when the AI produced suboptimal or incorrect suggestions. Furthermore, waiting for generative AI responses also contributed to idle time, consuming minutes that might otherwise have been spent on productive tasks. These new categories of work, including prompting, reviewing AI output, and waiting for responses, collectively negated any time saved in other areas, ultimately increasing overall task completion times.

Contextualizing the METR Study's Methodology and Demographics

The METR study, led by Joel Becker and Nate Rush, distinguished itself from prior research by focusing on experienced open-source developers working on real-world tasks with contemporary AI systems (greater than or equal to GPT-4 capabilities as of early 2025). This methodology contrasts with earlier studies that often used less experienced developers, synthetic tasks, or older AI models, leading to different findings. For instance, some previous research indicated significant speedups: Peng et al. reported developers were 56% faster, and Weber et al. found a 65% speed increase. These studies typically did not involve experienced developers on non-synthetic tasks or use fixed outcome measures for productivity.

The METR study's unique approach involved:

Feature	METR Study (Becker & Rush)	Other Studies (e.g., Peng et al.)
AI System Capability	≥ GPT-4	Older/Less Capable
Task Type	Real-world, non-synthetic	Synthetic or less complex
Developer Experience	Experienced, high-familiarity	Varied, often less experienced
Outcome Measure	Fixed before treatment	Often varied
Result (Task Completion)	↓ 19% slower	↑ 56-65% faster

This rigorous design, particularly the use of experienced developers and real-world scenarios, allowed the METR study to capture a more nuanced impact of generative AI on developer productivity, revealing a slowdown that other methodologies might have missed.

Implications for AI in Software Development and Future Outlook

The METR study's findings, particularly the 19% slowdown for experienced developers, highlight a critical "productivity paradox" in AI adoption: developers perceive increased speed (20% faster) even when objectively slower. This suggests a trade-off where "trading speed for ease" becomes a factor, with developers potentially valuing the reduced cognitive load or perceived effort of using AI coding assistants over actual task completion time. This perception gap poses a challenge for effective integration, as teams might misallocate resources based on subjective developer perception rather than objective metrics. For instance, if developers feel faster using tools like Cursor or GitHub Copilot, organizations might overlook the actual increase in task completion time. The study also implies that the "technical debt" generated by AI-assisted code, requiring more rigorous code review and debugging of AI output, could become a significant problem if not addressed. Future outlooks must focus on refining generative AI to minimize prompt engineering overhead and improve code quality, thereby reducing the time spent on review and correction. This shift requires developing AI tools that are not just easy to use, but genuinely augment productivity for experienced professionals on complex, real-world tasks.

Frequently Asked Questions

What was the "19% slower" AI study?

The "19% slower" AI study, known as the METR study led by Becker and Rush, found that experienced open-source developers using advanced AI systems (GPT-4 level) were 19% slower at completing real-world tasks compared to those not using AI.

Who conducted the study that found AI slowed down experienced developers?

The study that found AI slowed down experienced developers was conducted by Joel Becker and Nate Rush, and is referred to as the METR study.

Why did the AI developer productivity study find developers were slower?

The study found developers were slower because new tasks like prompting, reviewing AI output, and waiting for responses negated any time saved, ultimately increasing overall task completion times.

How does developer perception of AI productivity differ from reality?

Developers perceived themselves as 20% faster when using AI, even though the METR study objectively found them to be 19% slower, highlighting a significant "productivity paradox" or perception gap.

What are the limitations of AI coding assistants for experienced developers?

For experienced developers, AI coding assistants can introduce new categories of work like prompt engineering and extensive review of AI-generated code, which can lead to increased task completion times and potential "technical debt."

What is the METR study on AI developer productivity?

The METR study is a research project led by Becker and Rush that investigated the impact of advanced AI systems on experienced open-source developers working on real-world tasks, finding a 19% decrease in productivity.

Conclusion

The METR study offers a crucial, data-driven perspective on AI's current impact on experienced developer productivity, challenging the widespread perception of immediate efficiency gains. It underscores the importance of objective measurement over subjective feeling when integrating new technologies. As AI continues to evolve, future advancements must prioritize true augmentation, minimizing overhead and maximizing genuine productivity for complex tasks.

Sources & References

Want to actually learn Engineering?

Curo turns topics like this into a personalized, guided learning board - built around what you already know. Free to start.

Try Curo

Or jump straight in:

AI developer productivity study METR study Joel Becker Nate Rush developer perception