The development process I run, and where it has failed
Every engineering studio publishes a process diagram. They are all roughly the same diagram, they are all drawn left to right, and none of them include the parts where the process fails. That omission is the reason they are useless for choosing between studios.
Here is the sequence I run. It is not unusual. What follows each stage is the specific way it goes wrong, which is the part worth reading.
The engagement that this sits inside is described on the custom software development page; this is what happens inside it.
Stage one: establish what “working” means
Before anything is designed, we write down how we will know the system works. Not “the tests pass” — what observable thing has to be true.
For a matching engine that might be: a reviewer agrees with the top result in some stated proportion of cases. For a pipeline: the nightly run completes within a window and reconciles to a known total.
Where it fails: the definition gets written in language vague enough that everyone can agree with it. “The system should be accurate” passes review and settles nothing. The check I apply is whether a disinterested third party could take the definition and independently declare the project failed. If they could not, it is not a definition.
Stage two: settle the four architectural decisions
Permissions, concurrency, source of truth, and release cost. These are covered in more depth in application development, and the reason they come this early is that three of the four are expensive to reverse.
Where it fails: stakeholders find these boring compared to the interface, so the conversation drifts to screens. Screens are easy to discuss because everyone can see them. I have learned to refuse to show a single mockup until these are written down, which is unpopular in week one and correct by month four.
Stage three: build the narrow thing end to end
Not a prototype of the interface. One complete path through the real system, with real data, real auth, and real persistence — the least impressive version that touches every layer.
Where it fails: the pressure to demo something broad instead of something complete. A broad demo of five half-built screens is more impressive in a meeting and tells you nothing about whether the parts connect. The narrow version is where integration surprises surface, and they always surface.
Stage four: adversarial review
Someone tries to break it, deliberately, with a brief to find problems rather than to approve. This is where most processes have a rubber stamp instead.
The fine-tuning pipeline audit is the public example — an audit of my own work that surfaced 84 issues, including the reason the model recited instead of reasoned. That number is worth something only because the audit was built to disagree with the build.
The pipeline replay demo shows the mechanism itself: a recorded multi-agent run where critics reject work and hand it back, including a critic that rejected the same work three times and never signed off.
Where it fails: when the reviewer is the builder, or reports to the builder, or is asked “does this look right” instead of “find what is wrong.” The framing of the question determines the result.
Stage five: operate it
Ship, watch, and stay responsible. Instrumentation is decided here, and it should answer “is it still working” without anyone reading code.
Where it fails: this stage is where a one-person studio has a real constraint, and I would rather state it than have you discover it. I can hold operational responsibility for a small number of systems at once. When capacity is the binding constraint, the right answer is a team, and I will say so rather than take the work and be slow.
The stage I removed
I used to include a discovery workshop as a distinct stage. I removed it because in practice it produced a document that nobody consulted after week three, and the useful content of it was already covered by stage one done properly.
I mention it because a process that has never changed is a process nobody has tested. The evidence that this one gets tested is that a feature of it was deleted on the strength of what it produced — the same standard I would apply to any system.
What this process does not fix
It does not fix an unclear commercial goal. If nobody can say what the system is for in business terms, stage one produces a definition of working that is technically sound and beside the point. That failure is upstream of engineering and no development process rescues it.
Related reading: product development on whether you are building a product or a project. Enterprise software on the work that a security review will add regardless of process. And custom software on the question that comes before all of this — whether the thing should be built at all, which stage one will happily produce an excellent definition of working for, right up until someone notices you could have bought it.