mobile logo

Search

It worked in demos. It drifted in reality.

AI

AI Governance

AWS

Over the last year, almost every serious conversation I have had about AI has started the same way.

We are doing GenAI.
We want agents.
Our board wants an AI roadmap.
We need an AI factory.
Our teams are confused.

Different sectors. Same story.
Insurance carriers. Regulators. Public sector. Healthcare. Startups. Large consultancies.

Everyone feels pressure to move fast. Very few people feel confident they understand what is really happening under the surface.

And I realised something uncomfortable.

The problem is not lack of ambition.
The problem is not lack of tools.
The problem is lack of context.

We are debating vendors, frameworks, and architectures without having a shared mental model of where different parts of this ecosystem sit in their maturity.

So, I stepped back and did what I often do when things feel messy. I mapped it.

A maturity lens for the AI landscape

I used a maturity lens inspired by Simon Wardley and mapped what I see across real projects into four stages. His approach has been used widely by large enterprises and global bodies including the World Economic Forum to map advanced technology maturity.

Genesis
Custom built
Product
Commodity

This is not theory. This is grounded in what I see in delivery rooms, architecture reviews, incident reviews, and board conversations.

And once you see the ecosystem this way, the noise starts to make sense.

Commodity is already here

Some capabilities are no longer innovation, even if they are still marketed that way.

  1. Chat interfaces
  2. Embeddings
  3. OCR
  4. Speech to text
  5. Summarisation
  6. Translation
  7. Basic LLM usage
  8. Standard ML techniques
  9. Deployment pipelines

These are utilities now. Necessary. Expected. There is no functional differentiation here, only operational efficiency.

If teams are still celebrating this as transformation, they are solving yesterday’s problems.

Product is where most enterprises are operating today

This is the layer where platforms are stabilising and buying decisions start to matter.

  1. RAG platforms
  2. Enterprise copilots
  3. Vector databases
  4. Agent frameworks such as CrewAI, Microsoft Agent Framework, Amazon Bedrock AgentCore, GCP ADK etc..
  5. Observability tooling at surface level – Langsmith, Langfuse, FiddlerAI etc..
  6. AI factory playbooks

This is where pilots happen. This is where procurement gets involved. This is where maturity starts to look real. It is a sensible place to invest, if expectations are grounded.

Custom built is where the real engineering work lives

This is where serious advantage is created and where most organisations underestimate the complexity.

  1. Multiple agent orchestration
  2. Memory architecture
  3. Evaluation harnesses
  4. Governance automation
  5. Human plus agent operating models
  6. Domain knowledge graphs
  7. Context graphs grounded in business meaning

This work does not feel like plugging tools together. It feels like designing a new kind of system. Because it is.

And it is exactly where most of the hard problems live today.

Genesis is the frontier

This is research territory. Fascinating. High upside. High uncertainty.

  1. Self evolving agents
  2. Autonomous collectives
  3. Living knowledge systems
  4. Agent native organisations
  5. Emergent reasoning architectures

Important to explore. Dangerous to sell as production ready.

A moment from the field that changed my thinking

A few months ago, I was working with a team building an agentic workflow for a regulated insurance process.

On paper it was excellent.
Emails ingested automatically.
Documents parsed.
Risk attributes extracted.
Rules queried.
Summaries drafted for underwriters.

The demo was strong. Leadership was excited.

The pilot went live.

Week one, people loved it.
Week two, usage increased.
Week three, something shifted.

Not complaints. Something worse. A quiet loss of confidence.

“It feels different.”
“It used to be more cautious.”
“It is skipping things it used to flag.”

Same models. Same prompts. Same architecture.

We investigated.

What had changed was not the components. What had changed was behaviour.

A small upstream change altered how context was passed.
A tweak in memory retrieval increased overconfidence.
A timeout caused silent fallbacks.

Nothing broke loudly. Everything degraded quietly.

Dashboards looked fine.
Latency was fine.
Accuracy metrics were fine.

But trust was eroding.

When we looked deeper, the issue was not the model or the tools. The issue was architecture maturity.

The system had no explicit understanding of context.

Entities were implicit.
Relationships were fuzzy.
Business constraints lived in prompts.
Memory was loosely stitched together.
Provenance was unclear.

It worked in demos. It drifted in reality.

We changed the approach.

We introduced a real domain knowledge graph.
Explicit entities like policy, risk, clause, broker, control.
Explicit relationships.
Explicit provenance.
Explicit constraints.

Not for academic elegance. For engineering stability.

Suddenly behaviour became more predictable.
Reasoning became traceable.
Evaluation became meaningful.
Governance became practical.

This was the difference between a clever system and a trustworthy system.

Context graph and knowledge graph are not buzzwords. They are structural maturity.

Where maturity mapping really helps leaders

I now see three failure modes everywhere.

  1. Teams overinvest in commodity capabilities and call it innovation.
  2. Leadership pushes experimental ideas into production and burns trust.
  3. Organisations try to standardise things that are still fundamentally custom.

Knowledge graph is a perfect example of this confusion.

Some teams claim to have a knowledge graph when they really have tagged tables.
Some buy a graph platform and expect intelligence to magically appear.
Some are quietly building deep domain context graphs that become strategic assets, but they underestimate the investment required.

These sit in completely different maturity layers.
They carry completely different risk profiles.
They demand completely different expectations.

This is why maturity mapping is not a technical exercise. It is a leadership discipline.

It informs investment decisions.
It shapes architecture choices.
It guides hiring.
It influences governance.
It protects organisational trust.

Where Agentic Engineering Body of Practices fits

Most of the meaningful challenges today sit in the Custom built zone.

Not prompts.
Not models.
Not vendor selection.

But questions like these.

How do we design multiple agent systems that behave predictably
How do we represent context explicitly instead of hoping retrieval works
How do we structure memory so systems stay consistent over time
How do we test behaviour rather than just outputs
How do we detect agent drift before users feel it
How do we govern actions without killing speed
How do humans and agents collaborate in real workflows

These are not theoretical questions. These are problems I see weekly inside real organisations.

This is why Agentic Engineering Body of Practices exists.

Not to publish hype.
Not to create another framework.
But to capture what is actually working in practice.
To document emerging engineering disciplines before the industry hardens around poor defaults.
To help teams move from experimentation to professionalism.

Most of this work sits right in the uncomfortable transition between Custom built and Product. That is where patterns solidify. That is where standards emerge. That is where discipline matters.

The organisations that will lead will not be the ones with the most tools.
They will be the ones with the clearest understanding of maturity and the courage to invest deliberately.

This is the work I care about.
This is the work I see every day.
This is the work esynergy is trying to make easier for others through Agentic Engineering Body of Practices.

If you would like to run a one-hour AI capability and practices mapping session with your leadership team, we offer a structured exercise that gives you a clear AI strategy in one page. It highlights focus areas, priorities, what to measure, and where to invest.