TL;DR
- The Question: If this ran a hundred times faster to a hundred times more people, would it get better or worse?
- The Mistake: Most people chase the model and add more agents, and skip the filtering entirely.
- The Fix: Restraint. The system has to know what not to do, like negative keywords in paid media.
- The Order: Memory first. Build the thing that remembers, or nothing else compounds.
Most people build AI systems wrong, and the mistake is not the model they pick. It is what they skip. They chase the latest model, they bolt on more agents, and they never do the one thing that makes the whole system hold together: filter. The system that wins is the one that knows what not to do.
The Question Most People Skip
There is a question I run everything through before it gets built: if this ran a hundred times faster to a hundred times more people, would it get better or worse. That single question filters out most of what people are tempted to automate, because a lot of processes are fine at small scale and catastrophic at large scale.
This is the filtering process, and it is the part almost everyone skips. They do not want to slow down and ask whether the thing should exist. They want to build, add, ship. But the filter is the whole discipline. A system without a filter is just a machine that does a hundred things badly and fast.
The question works because it forces you to separate what scales from what does not. Some work gets better as it compounds: memory, data, audience. Some work gets worse: noise, bad targeting, unfiltered output. If you do not know which is which, you will scale the wrong one and the whole system turns into an amplifier for your mistakes.
The reason most AI systems fall over is the missing filter, not the model. Run everything through one question first: would this get better or worse at a hundred times the scale. Then tell the system what not to do, and build memory before you add agents.
The System Needs to Know What Not to Do
The clearest way to understand restraint is a paid media campaign. You do not just tell the platform who to target. You give it negative keywords, the list of what not to show your ad for. Without that, you waste budget on the wrong clicks. The negative is not an afterthought. It is half the setup.
AI systems work the same way. Most people only define what the system should do, and never define what it should not do. So the system happily does everything, including the things that damage it. The discipline of restraint is writing down the negatives: do not email this segment, do not publish this kind of post, do not trust this source, do not run this when this condition is false.
This feels unglamorous, which is why people skip it. Adding an agent is exciting. Writing a list of prohibitions is not. But the prohibitions are what keep the system honest, and they are the difference between an assistant you can trust and an assistant you have to babysit.
Build Memory First
Here is the build order I give everyone, and it has not changed: memory first. Build the thing that remembers, or nothing else compounds. Every agent, every automation, every workflow you add on top of a memory-less system is starting from zero every single time, and that is why it never gets smarter.
For me, that memory is an Obsidian vault. I dropped in a GitHub repo, told Claude to figure it out and turn it on, and it did the rest. The point is not the tool. The point is that the memory layer exists before anything else sits on top of it. Once the memory is there, the agents have shared context, the work compounds, and the system actually learns.
The order is the whole insight: filter first, memory second, agents last. Reverse it and you get the thing most people have: a pile of impressive agents that do not remember anything, doing a hundred things badly and fast.
What I Actually Think
The AI industry is obsessed with capability, and almost nobody talks about restraint. But the systems that actually survive real work are the restrained ones. They are smaller than they could be, do fewer things than they could, and are better for it. Restraint is not a lack of ambition. It is the difference between a system that compounds and a system that collapses.
I would rather have one agent with memory and a clear list of what not to do than ten agents with neither. The first gets better every week. The second gets louder every week. And this is exactly the discipline behind the four stages I climbed building my own AI staff, and why the eight agents I run stay lean on purpose.
Restraint is the feature. Filter everything through the 100x question, tell the system what not to do, and build memory before you add agents. That order is what separates a system that compounds from one that collapses.
Build your system the right way →
