The Agent Mesh
The control plane for the agent era
Every wave of software has eventually produced its own runtime. Web apps got the application server. Distributed services got Kubernetes. Microservices got the service mesh. Agents are next, and the shape of that runtime is already visible if you’ve been paying attention.
This post traces how we got here. The route runs from prompting, through context engineering, into the agent harness, and finally to the agent mesh. Each step was a response to a real limit of the previous one. None of it is theoretical. The pieces exist, the patterns are converging, and the practical implications for any organization trying to become “AI-first” are sharper than the marketing makes them out to be.
Each stage absorbs the previous one and adds the abstraction that the previous one was missing.
Prompting
The first competence anyone developed was writing prompts. Tell the model what you want, then iterate until it does it. Most of the early craft was conventions for structure: XML tags, markdown headers, in-context examples, role framing, separating instructions from data. Anthropic’s guidance was explicit about a lot of this and most of it has aged well.
Prompting is still the foundation. It hasn’t gone away, it has been absorbed. What changed is that nobody serious treats a prompt as a string anymore.
Context engineering
The moment people started building anything beyond a single-turn assistant, the prompt stopped being the artifact. What mattered was the context, meaning everything the model could see at the moment it generated a response. Retrieved documents, tool results, prior turns, system instructions, memory. The prompt became the smallest piece of a larger, dynamically assembled payload.
Context engineering is the discipline of getting that payload right. The interesting part is that bigger context windows did not solve it. They made it worse. Models suffer from what Drew Breunig usefully named context rot, and it fails in four reliably observable ways:
Context poisoning. A hallucination or error lands in context and gets referenced repeatedly, compounding with each turn.
Context distraction. The context grows so large the model over-indexes on its history and stops drawing on what it learned in training.
Context confusion. Superfluous content, such as overlapping tool descriptions or irrelevant retrieved chunks, pulls the model toward low-quality responses.
Context clash. Pieces of the context contradict each other, usually because they came from independent sources that don’t know about each other.
The counter-moves are by now well understood: tight retrieval, tool loadouts, compaction, partitioning, isolation between subtasks. The point isn’t to list techniques. The point is that “fill the context window and let the model figure it out” stopped being a viable design around the time anyone tried to ship it.
Three things that had to happen first
Before the agent harness was buildable as anything other than a one-off, three things had to settle. They settled faster than most of us expected, and together they are the precondition for everything that follows.
The first was a working definition of what an agent actually is. After a couple of years of marketing taxonomies, the field converged on a boring and useful answer: an agent is an LLM that runs in a loop, calling tools, until a task is done. That’s it. Once that definition stabilized, you could stop arguing about whether something was “really” an agent and start engineering one.
The second was the Model Context Protocol. MCP gave the industry a shared way to expose tools, data sources, and capabilities to a model. Before MCP, every agent framework reinvented its own tool interface, which meant tools weren’t portable, integrations didn’t compose, and every team rebuilt the same connectors. MCP is the USB-C moment for agentic tooling, and like USB-C, its value is not the elegance of the spec but the network effect of everyone using it.
The third was Agent-to-Agent communication, or A2A. Once individual agents became useful, the obvious next move was composing them, which required a protocol for agents to discover and call each other rather than each framework inventing its own message bus. A2A is doing for inter-agent communication what MCP did for tool use.
Sitting alongside these, the idea of an agent skill emerged as a portable unit of know-how. A skill bundles instructions, references, and small bits of code into something the agent can load on demand. Skills are what let a generalist agent acquire specialist competence without bloating its base prompt, and they fit naturally on top of MCP and the loop-calling-tools definition.
None of this is glamorous, but it matters. Standards are what turn one-off systems into ecosystems. Without MCP, A2A, skills, and the converged agent definition, the agent harness would still be a stack of bespoke integrations and the agent mesh would be unbuildable.
The agent harness
Context engineering deals with what the model sees. The agent harness deals with what the model can do.
Wrap a model in the loop, give it tools (over MCP), file access, a shell, code execution, persistent memory, and a set of skills that describe how to use any of it, and you have an agent. The harness is everything around the loop: the orchestration, the tool layer, the memory subsystem, the I/O contracts, the guardrails. Anthropic’s recent work on agent skills and Claude Code is the cleanest reference implementation I know of for what this looks like in practice. Frameworks like Mem0 sit inside the harness; agentic memory is an active subfield, but a tangent here.
An agent is an LLM in a loop. Everything else is the harness around it.
The harness is where agentic systems became actually useful. It’s also where the per-team cost of building agents got expensive enough to matter. If every team in your organization needs to design its own agent loop, ship its own MCP servers, manage its own memory layer, instrument its own telemetry, and deploy the whole thing as a microservice on Kubernetes, you do not have an AI-first organization. You have a few teams with a high enough budget to fake one.
The agent mesh
This is the layer the field is converging on now, and it has the clearest precedent of any step in the chain. The story rhymes with the move from raw VMs to Kubernetes, except the unit of orchestration is no longer a container, it’s an agent.
A useful way to think about the mesh is as three concerns layered on top of the harness.
An artifact model and a registry. An agent is not one thing, it’s a composition. An MCP server here, a prompt template there, a set of skills, a model binding, an agent definition that ties it all together. Once those become versioned, addressable artifacts with a lifecycle, you can have a registry. A Docker Hub for agentic components. Discovery, provenance, signing, version pinning, curation, the same problems package registries solved twenty years ago, applied to a new set of building blocks. The good registries already pair with the gateway, so what gets published can be governed, scanned, and observed end to end.
An agent gateway. The mesh needs a communication layer, and it turns out to be load-bearing in more directions than people initially expect. It carries A2A traffic between agents and MCP traffic from agents to tools. It is the unified entry point for LLM calls, which is where you put failover, cost accounting, rate limiting, and policy. It is where authentication and authorization live, because you do not want each agent re-implementing OAuth flows against ten different SaaS connectors. The service mesh analogy is exact: the gateway is the data plane.
A runtime. This is the part that does for agents what Kubernetes does for containers. Declare desired state, let the engine reconcile. You stop thinking about “an agent” as something a team writes and operates by hand, and start thinking about a declarative definition that the runtime is responsible for scheduling, scaling, restarting, and observing. Solo.io’s open-source stack (kagent, agentgateway, agentregistry) is the reference architecture I keep returning to, and it is already production-grade enough to take seriously.
The mesh: registry above, gateway in the middle as the data plane, and runtime managing lifecycle. The gateway’s position is also why it’s the natural place to enforce agent IAM.
Why this matters operationally
Stating it as a control plane sounds like architecture astronaut territory. It isn’t. The reason the mesh matters is operational, not aesthetic.
When agents become declarative artifacts managed by a runtime, the marginal cost of shipping one drops by an order of magnitude. An AI engineer can define an agent (its skills, MCP dependencies, model binding, and policies) and deploy it without owning a Kubernetes cluster or negotiating with the platform team for a new namespace each time. The platform team is still essential; somebody has to run the mesh, set the policies, curate the registry, and manage the gateway. But the daily work of building agents stops being a DevOps project for each team and becomes something closer to writing a deployment manifest.
This is the same shift Kubernetes produced for services. Before Kubernetes, deploying a service meant negotiating infrastructure. After Kubernetes, deploying a service meant declaring intent. The agent mesh is doing the same thing one abstraction layer up, and the productivity gradient looks identical.
The IAM problem nobody has solved yet
There is one more reason the mesh is going to matter, and it’s the reason most organizations underestimate at first: agent identity and access management.
Today, the dominant authentication pattern for agents is to give them a service account and hope. Hope they don’t get prompt-injected into using their credentials against the user. Hope the blast radius of a compromised agent stays small. Hope that “the agent acted on behalf of the user” is a coherent enough statement to satisfy an auditor. None of this scales.
Agent IAM is its own problem and it doesn’t reduce to traditional service-account IAM. An agent acts on behalf of a user, but with delegated, scoped, time-bound, often dynamically negotiated authority. It calls tools that themselves require credentials, sometimes from systems with their own identity models that predate any of this. It composes with other agents under A2A, each of which has its own provenance, its own permissions, and its own audit trail. Token exchange, fine-grained delegation, capability scoping, and meaningful logging across multi-agent calls are not solved problems. They are the next round of pain.
The mesh is where these get solved, because the mesh is the only layer with the visibility to solve them. The gateway sees every tool call, every LLM request, every cross-agent message. The registry knows what every agent is composed of and where it came from. The runtime knows what is running, on whose behalf, and under what policy. If you want to enforce that a particular agent can only call a particular MCP server when invoked on behalf of a particular user, with a token that expires in fifteen minutes and is audit-logged with the originating user’s identity, the mesh is where that enforcement lives. Nowhere else in the stack has both the position and the information to do it.
Expect agent IAM to look like the OAuth standardization wave of the early 2010s, except compressed into a much shorter window. The mesh is the substrate it will run on.
Where this leaves us
The path from prompting to the agent mesh is not a story about ever-larger context windows or ever-smarter models. It’s a story about the same pattern every previous wave of software has gone through: figure out the primitive, agree on the protocols, build the runtime, then standardize the operational layer that lets organizations run it at scale without each team rebuilding the world.
The agent mesh is the cloud of the agent era. It’s where this stack is heading. The open question is no longer whether the layer is going to exist, but how long it takes the rest of the industry to notice it already does.