What Is an AI Agent, Really? Agentic AI Explained Beyond the Buzzword

An AI agent is a piece of software that is given a goal rather than an instruction, and that pursues it by deciding what to do next, acting on the world through tools, observing the result and deciding again, until the goal is reached or it stops. That loop, and not the presence of a language model, is what makes something an agent. A chatbot answers; an agent acts, checks and acts again. Agentic AI is the discipline of building systems that way, and most of what is sold under that name today does not pass the test.

I have been on both sides of this word for a while. In 2016 three colleagues and I filed a patent at IBM that described, without ever using the term, a system that sensed a live event, decided what mattered and acted on it in real time; I wrote about that for Towards Data Science. Today I spend my days with public sector organisations that are being offered “agents” by every vendor on their shortlist. This explainer is the one I wish I could hand them before the meeting starts.

The definition, and the test that comes with it

Strip away the marketing and an agent has four parts. A reasoning component, today almost always a large language model, that reads the situation and chooses the next step. A set of tools it is allowed to call: search a database, send an email, open a ticket, run a query, call another agent. A memory, so that what happened three steps ago, or three weeks ago, informs what happens next. And a control loop that connects them: observe, decide, act, observe again, with a stopping condition and, in any serious deployment, a human somewhere in the loop with the right to say no.

The test follows directly. Ask three questions of anything called an agent. Does it decide the sequence of steps itself, or is the sequence fixed by a developer in advance? Does it act on systems through tools, or does it only produce text for a human to act on? Does it observe the result of its actions and change course? Three yes answers and you have an agent. One or two and you have something useful, but it is a different thing, and it should be priced, governed and trusted as a different thing.

What an AI agent is not

It is not a chatbot. A chatbot, however good the model behind it, produces a response to a message and waits. The user is the control loop. The moment you give the chatbot a tool and let it call that tool without asking, you have taken the first step toward an agent, and you have also taken on the first governance question: what is it allowed to do when nobody is watching?

It is not retrieval-augmented generation. RAG fetches documents relevant to a question and gives them to the model before it answers. That is a single, fixed step, decided by the developer. It makes the answer better grounded; it does not make the system agentic. Many “agents” in enterprise demos are RAG pipelines with a friendlier name.

It is not a workflow, and it is not robotic process automation. A workflow executes a sequence of steps someone designed, with branches someone anticipated. It is deterministic, auditable and often exactly what you need. An agent decides the sequence at run time. That is the source of its power and of its risk, and anyone who tells you an agent is “just a smarter workflow” is hiding the second half of the sentence.

And it is not a bigger model. Upgrading the model inside a system does not make it an agent, and upgrading the model inside an agent does not fix the agent. I argued this at length in the piece on agent memory: the failures I see in production are failures of what the system remembers, what it is permitted to do and how it is checked, not of how clever the model is.

Agentic AI versus AI agents

The two terms are used interchangeably and they should not be. An AI agent is a component: one system with one goal and one loop. Agentic AI is an architectural style: building applications out of agents, often several of them, with an orchestrator that assigns sub-goals, tools that are shared and governed, and memory that outlives any single conversation. You can have an agent without agentic AI, a single assistant that files expense reports. You cannot have agentic AI without agents, and you cannot have it without the plumbing that lets agents find tools and data in a controlled way, which is what protocols such as MCP are for; I covered how that works for a database elsewhere.

Five levels of autonomy

Autonomy is not binary, and the most useful conversation to have with a vendor or a team is about where on the ladder a system sits. At the bottom, the model suggests and a human executes: a copilot. One rung up, the model executes a single tool call per turn, with the human approving each one. Then the model chains several steps toward a goal but stops at defined checkpoints for approval. Then it runs to completion on bounded tasks, with humans reviewing outcomes rather than steps. At the top, agents delegate to other agents, run on schedules or triggers, and the human role shifts to setting policy and auditing. Almost every production system I have seen that works sits on the second or third rung. The demos sit on the fifth. The gap between them is not model capability; it is trust, and trust is built from memory, permissions and evaluation.

Why demos work and production fails

Three things separate the agent that impressed the steering committee from the one that was quietly turned off six months later.

Memory. In the demo, the context window is the memory and the task fits inside it. In production the agent must know what it did yesterday, what the organisation already decided, what it is not allowed to know about this citizen, and what it must forget after thirty days. None of that comes from the model. It comes from a data layer designed for it: state that persists, context that is retrieved with permissions applied, history that can be audited and expired. Most agent projects discover this in month four.

Permissions. In the demo the agent has the developer’s credentials. In production it needs its own identity, scoped to the minimum set of tools and data for the task, with every action logged against that identity. A read-only agent that cannot send anything outside the perimeter is a different risk class from one that can write to a system of record, and the architecture should make that difference visible, not bury it in a prompt.

Evaluation. A chatbot that is wrong produces a wrong sentence. An agent that is wrong produces a wrong action, and then another one based on it. Production agents need a test set of realistic tasks, a definition of done that a machine can check, and a measured success rate before anyone widens their permissions. Teams that cannot state the success rate of their agent do not have an agent in production; they have an experiment with a budget line.

What this means for the public sector

Public administrations are a natural home for agents, because so much of their work is a goal pursued through many systems: a benefit claim that touches four registries, a permit that needs three opinions, a procurement that follows a procedure. They are also the place where the definition matters most, because an agent that decides about a citizen is, in regulatory terms, an automated decision, and the questions that apply are the ones I set out in five questions to ask any system that decides about you: can we understand it, contest it, leave it, correct it, and does it leave room for the case that does not match the past.

The practical consequence is that the first agents in government should sit low on the autonomy ladder and high on the value of the task: preparing the case file, not deciding it; drafting the response, not sending it; finding the inconsistency across registries, not resolving it. That is not timidity. It is how you earn the right to climb.

The practical questions, answered

What is the difference between an AI agent and a chatbot?

A chatbot produces a response and waits for the human to decide what happens next. An agent decides what happens next itself: it chooses and calls tools, observes the result and continues until the goal is met. The same model can power both; the difference is the control loop and the tools.

What are examples of AI agents in use today?

Coding agents that take a ticket, edit a repository, run the tests and open a pull request. Operations agents that triage an incident, query the monitoring stack and propose or apply a fix. Research agents that plan a search, read sources and compile a sourced brief. Back-office agents that reconcile records across systems and prepare a case for a human decision. The common thread is a bounded goal, a small set of tools and a human review point.

Are AI agents safe to use with enterprise data?

As safe as the permissions you give them and the logging you put around them. The sound pattern is an agent with its own identity, read-only access by default, tools exposed through a governed layer rather than raw credentials, every action logged, and no path for data to leave the perimeter that a human did not design. Agents that inherit a developer’s credentials are not safe in any environment that matters.

Do I need an agent, or is a workflow enough?

If you can write down the steps and the branches in advance, build a workflow: it is cheaper, deterministic and auditable. Build an agent when the path to the goal depends on what is found along the way, when the inputs are too varied to enumerate, or when the sequence of tool calls has to be chosen at run time. Many good systems are a workflow with one agentic step inside it.

Why do agents need memory?

Because a goal pursued over many steps, many sessions and many users cannot be held in a context window. Agents need working memory for the current task, long-term memory for what the organisation knows and has decided, and rules for what they may know and must forget. This is where most production agents fail, and it is the subject of the book this site keeps coming back to.

The one sentence that matters

An agent is not defined by how intelligent its model is, but by the loop it runs, the tools it may touch, the memory it carries and the permission it has to act. Judge the systems you are offered by those four things, and the buzzword takes care of itself.