For the last decade, DevOps covered most operational questions. The practices were clear enough, the tooling was mature, and teams saw real returns from investing in it.
It is still useful. It is no longer enough.
Where the DevOps model falls short
DevOps assumes that the same input produces the same output. That makes its tooling possible: CI passes or fails, tests reproduce, and a deployment succeeds or it doesn't.
Large language models break that assumption. The same model and prompt can produce different answers across requests. So the operational questions change.
What does "this deployment is healthy" mean when the output cannot be directly compared? How do you version a prompt the way you version code, when a small change in phrasing can shift behaviour substantially? What does a rollback look like when you have switched models? How do you test for quality when "correct" is context-dependent?
Teams running LLMs in production are dealing with these questions now, with mixed results.
A growing operations stack
The industry has added more labels: MLOps for machine learning systems, LLMOps for large language models, and AgentOps for systems that take actions over time.
The labels overlap, but the concerns do differ.
LLMOps covers prompt versioning, evaluation, token costs, retrieval quality, guardrails, and model serving. CPU usage and error rates still matter, but now teams also track output quality and user corrections.
AgentOps adds another layer. If an agent calls APIs, executes code, or manages infrastructure, you need to know what it decided, why, and what it did next. Permissions become critical. So do traces across multiple agents.
What seems to work
The teams handling this well aren't adopting every new framework. They first work out what changed and what stayed the same.
The fundamentals still hold: version control, testability, observability, gradual rollout, and rollback. The hard part is adapting them to a probabilistic system.
A 2026 analysis of enterprise AI operations found that organisations fluent across the full stack were running 30 to 40 per cent faster than those applying only traditional DevOps patterns. That matches what I see in practice, though the field is still young.
Where things stand
Most of this is still being worked out. AgentOps tooling is early, and we don't yet know which patterns will hold up. Anyone claiming otherwise is probably describing a narrow case or extrapolating.
The direction is clearer. Software will contain more AI components, agentic behaviour, and probabilistic output. Treating that as a temporary anomaly will get expensive.
Teams that understand the operational differences now will have an easier time later. It isn't a glamorous conclusion, but I think it is the useful one.
If you're working out where your organisation sits, I'm happy to compare notes.