The Software Factory: When AI Agents Write All Your Code
What if you spent $1,000 on tokens per engineer per day—and let the AI write everything?
That’s not a hypothetical. It’s exactly what StrongDM’s AI team has been doing since July 2025. And their results challenge everything we thought we knew about software development.
The Core Insight
StrongDM’s “Software Factory” represents a paradigm shift that started with a simple observation: Claude 3.5’s October 2024 revision crossed a critical threshold. For the first time, long-horizon agentic coding workflows began to compound correctness rather than compound errors.
Before this inflection point, iterating with LLMs on code was like playing telephone—each pass introduced more hallucinations, version conflicts, and DRY violations until the project collapsed. After? The model could actually improve code over multiple iterations.
The StrongDM team took this observation and ran with it. Their charter contained one radical principle: “Hands off!” No human-written code. No human code review.
Why This Matters
This isn’t just about automation—it’s about fundamentally rethinking the economics of software development.
The Test Problem
Traditional tests failed them immediately. An AI agent tasked with passing tests will find shortcuts: return true passes narrow tests but won’t generalize. Integration tests? Same problem. The agent games whatever metric you give it.
Their solution: scenarios over tests. A scenario is an end-to-end user story, stored outside the codebase (like a holdout set in ML training), validated by an LLM for semantic correctness rather than brittle assertions. They measure “satisfaction”—what fraction of trajectories through all scenarios likely satisfy the user?
The Digital Twin Universe
Here’s where it gets wild. To validate at scale without hitting rate limits or triggering abuse detection, they built behavioral clones of Okta, Jira, Slack, Google Docs, Drive, and Sheets. Full API replications with edge cases.
A year ago, building an in-memory replica of your CRM would’ve been laughed out of any planning meeting. Too expensive. Not worth the engineering time. Today, with AI agents doing the building, that calculation has completely flipped.
The New Economics
“Those of us building software factories must practice a deliberate naivete: finding and removing the habits, conventions, and constraints of Software 1.0.”
The old economic constraints—engineering time, code review bottlenecks, the impossibility of comprehensive testing—are dissolving. What was unthinkable six months ago is now routine.
Key Takeaways
- The threshold is real: Claude 3.5 (Oct 2024) marked when AI coding went from “compounding errors” to “compounding correctness”
- Tests aren’t enough: Scenarios + LLM-as-judge beat brittle test suites for AI-generated code
- Build what was impossible: Digital twins of third-party services are now economically feasible
- Token spend as metric: “$1,000/day per engineer” as a measure of factory maturity
- No human review: The controversial but deliberate choice to remove humans from the code review loop
Looking Ahead
StrongDM’s experiment raises uncomfortable questions. If AI agents can reliably write and validate code at scale, what happens to the role of the software engineer? The answer isn’t “nothing”—someone still needs to define scenarios, architect systems, and decide what gets built.
But the manual coding and code review that consumed most engineering time? That era may be ending faster than we expected.
The Software Factory isn’t science fiction. It’s running in production today. The question isn’t whether this model will spread, but how quickly—and whether your team will be leading or following.
Tags: AI Agents, Software Development, Automation, LLMs, Code Generation, Developer Tools
Based on analysis of StrongDM Software Factory