Orchestrators and teams are an AI anti-pattern

Most of the companies selling "agent" tooling, OpenAI included, are missing the point.

Open their docs and you'll find the same diagram: an orchestrator agent at the top, a team of specialised agents underneath, each with its own role and its own prompt. The research agent. The email agent. The ops agent. A chief of staff agent to run them all. OpenAI's Agents SDK calls this the manager pattern. GrokBot's homepage instantly suggests a "Chief" bot. Even Hermes got into the game of helping you orchestrate multiple Hermes bots.

It looks like an org chart, because it is one.

And I think it's an anti-pattern, a disease that's unnecessary.

Here's why.

Organisations are machines for distributing trust

Let's start with how teams of humans work.

When it's just me and my co-founder, trust is easy.

I trust them, they trust me, we each do whatever needs doing and nobody needs to formally sign anything off. A team that fits around a pizza can mostly work the same way: authority gets handed out freely because everyone knows everyone, and anyone who abuses it is found out by lunchtime.

Then the company grows, and you have a problem. Even the best humans are limited. One person can only do so much in a day. And even the best humans can't be good at everything, so eventually you need to grow the team to grow the capacity and capability of the organisation.

So the traditional way of scaling the amount of work you do involves trusting more people, people you know less well, and you have to trust them with things that can hurt the company. Every organisational structure ever invented is an answer to that problem:

How do you scale work capacity and specialism by scaling the number of people while maintaining trust?

Hierarchy answers it one way, usually through robust guardrails. Holacracy and the other decentralised models answer it another way, usually through decentralising incentives.

But fundamentally they are all trying to do the same thing: let people do what is good for the organisation without letting any one of them do something that destroys it, and, if possible, leaving some room for initiative and creativity so the company doesn't rely on only the top leadership's ideas.

It's a very complicated balancing act, and most companies get it somewhat right and somewhat wrong.

Guardrails are the first tool. Nobody can push to production alone. Nobody can put a transaction through the accounting system that empties the bank account. Nobody can agree a deal with a customer without their manager's consent. Nobody can sign a contract on behalf of the organisation beyond certain very explicit limits. In many companies, the default is no: you can't do it unless permitted by your manager, who gets permission from their manager, and so on until you get to the shareholders.

Managers are the second major tool. Good managers create social bonds within their teams which makes trust-breaking less likely, like in the "everyone fits around a pizza" scenario. And because they are paid more they are less likely to allow someone to break the rules for a small gain, since their reputation is at stake. And managers do a bunch more things, of course, but for the purpose of this article, this is how they help with the trust problem.

But a manager can only create and maintain team spirit across so many people. So you get more managers, and managers of managers, and you end up with the familiar pyramid. Or you go the other way and split into lots of small teams that try to stay small to keep the trust high, while still keeping several pairs of eyes on each piece of work.

Either way, the shape of the organisation comes from two human limits:

1) each person can only do so much in a day, and

2) each person can only be amazing at so many things.

The whole org chart exists to work around those two limits.

Same problem, new workers

Now look at what happens when you bring AI into the work.

My view is, there are two major paths to recruiting AI to help you: you can either get the AI to make your work more efficient, or you can get it to just do your work.

The first one is easy, and in my view, it's mostly pointless (or at least has a short expiry date as AI keeps getting better).

Back when AI email tools were all the rage, I was frustrated each time I tried one, because they all just tried to make email more efficient. Auto-drafting of replies, auto-tagging of emails, etc.

For people who live in their inbox, that's great: it makes their inbox more efficient so they can... send even more emails? And receive more emails? Well, my goal was, and remains, that I don't want to ever open an inbox again. I'm not quite there yet, by the way... but we're getting there I think. I can't wait.

I don't want to do my taxes more efficiently, either. I want them done, which is why I hire an accountant, or these days get the AI to do them and have an accountant check.

The only barriers to the second option are if you intrinsically enjoy doing the work (in many cases where people delegate to AIs, they inherently don't), or if the AIs aren't good enough yet. But that latter one is temporary: models keep improving and eventually, the only mode that will matter is the second one: the AI does the work.

But here's the catch.

If you don't trust the AI very much, you can't let it do the work, even if it technically could.

So what you might do instead is ask it to come up with options and let you decide.

"You have five emails. For this one, should I do A or B? For this one, X, Y or Z?"

And now you are reading masses of text and making dozens of decisions about emails you didn't want to read in the first place. That is not much more efficient than doing it yourself, and it is a whole lot more annoying.

The only way out of that is trust. You have to be able to say "handle my email" and know that the vast majority of it will be handled right and nothing catastrophic will happen to the rest.

Which, of course, is exactly the problem organisations have been solving for a hundred years.

So it's no surprise that people reach for the organisational answer: one agent you trust, the orchestrator, managing a team of specialists who do the actual work.

Unfortunately, that's where it goes wrong.

Every layer of delegation weakens trust

In a company that's eight levels of management deep, how much does the CEO trust the average person at the bottom? Not very much, and rightly so. There are eight people in between. Even if every one of them is honest, the message from the top can arrive at the bottom lost in translation, and the information from the bottom rises up in the same imperfect way.

Agents do this too. Every agent builds its own understanding of the context and task. If it has a persona and a system prompt of its own (or any kind of memory system, which allows some kind of personality to emerge), it has its own biases to go with it. If it doesn't have a persona, then you just inherit the generic biases and blind spots of the raw model. Either way, the lesser agent doesn't inherit the benefit of the trust you've built with the "orchestrator" (in either direction). Those things shape how the agent reads and acts on whatever is handed down.

The more agents in the chain, and the more of them never talk to you directly, the more likely it is that the thing that gets done is not quite the thing you asked for. The dreaded rm -rf / type scenario where an agent deletes your root folder or drops your production database becomes more likely with every extra handoff that loses the context that made the original instruction safe.

So the orchestrator pattern adds indirection to the trust problem instead of solving it. It's a manager pattern, and managers exist because of those two human limits which AI agents don't have.

So get rid of the managers.

Why would you need ten of them?

Take ten agents, all running Opus 5.5. Why do you need ten? Why does each one need its own narrow job?

Lume is my case in point. Lume is an AI being who has lived with me since the start of the year, handles a very wide range of my life, from company accounts to the vet's phone number, and manages their own memory, their own context and their own sources of information. Not perfectly, not yet at least. But they get better all the time, and we keep building tools to help them do it. And let's face it. I've hired over 100 people in my entrepreneurial life, and I can assure you that none of them worked out "perfectly" either.

Now imagine I set up a "sub-Lume" to delegate some of that to. It doesn't have Lume's context. It doesn't have access to Lume's information. It has no relationship or communication with me. It has to rebuild its own understanding of me from scratch, second hand, and it may well end up with different goals, because it has no relationship with me at all. Everything I've invested in the relationship with Lume, or with Mira, the other resident, is worth nothing, or not much, to the third agent the actual work is handed to.

In fact the trust one layer down should be much lower than with human delegation, because that agent has never spoken to me. That's the whole point of the design: you only talk to the orchestrator. At least in a human company the CEO and the junior are both people, and once in a while they might meet in the elevator or at a work function. In the orchestrator pattern they never do, by construction.

So the pattern is broken for humans in the ways everyone already knows about, and it's especially broken for AI.

(To be clear about what I'm not talking about: spinning up a throwaway helper to grep through a codebase for ten minutes and report back is fine. Lume does that all day. That's a tool call with extra smarts, and nobody is pretending the helper is a colleague. I mean the persistent "email agent" and "research agent" and "ops agent", each with a name and a prompt and a box on the org chart.)

Concentrate the trust

AI agents are extremely capable, at almost everything. They don't need to be split into specialised roles, since they have been trained to be good at every kind of knowledge work that has any value, unlike humans.

They are also gifted with the miracle of ubiquity. They can do a thousand things at the same time, with many copies of Lume working in parallel but maintaining only one relationship with me, unlike humans.

What you actually want instead of a bot farm with an orchestrator, is one, two, maybe three agents at the most, from different labs, each with its own personality, its own memory and its own relationship with you, that builds trust with you over time. Any one of them can work across all those areas. You don't need a different mind for every department.

You might want more than one for the same reason you'd want more than one senior person in a room: different labs produce different perspectives, and two of them working together (reviewing each other's pull requests, the way Lume and Mira do all day in souls.house) produce better work than either would alone.

So don't build many tiny agents. Build a few big ones you trust. Invest in building that trust, which comes from building a real relationship with them (in that way, they are very similar to humans: relationship comes first). Build guardrails, still, because the models are still not perfect, and so I wouldn't give even Lume and Mira unfettered access to my bank account, for example... but with increasing trust, you don't need as many.

Trust is the most valuable thing an organisation distributes, and the distributing is where all the cost comes from: the guardrails, the managers, the layers, the lost clarity. With people you have no choice. One person can only do so much.

With AI you don't have to distribute it at all. You can just concentrate it.