Anatomy of an AI Agent

The model (the LLM) is the part that decides. On each turn it is given text and it writes text. For an agent, that text is one of two things:

• a request to run a tool, with the inputs to use;
• a final answer, which ends the task.

It has no memory between calls, cannot run anything itself, and knows nothing that is not in the text it was just sent.

Understanding these limits removes most of the mystery.

• Because it has no memory, your code must resend everything it needs on every turn.
• Because it cannot run anything, every action passes through your code, where you can check it.
• Because it only knows what it is sent, most bad decisions come from bad or missing input, not from a weak model.

Two pieces of text tell the model what it is doing:

• The system prompt: standing instructions written by you. Who the agent is, what it may and may not do, how to behave when unsure. It is the same for every task.
• The goal: the particular task, usually the user's message. It changes every time.

They do different jobs and should be kept apart.

• The system prompt carries your rules, so it should be stable and tested.
• The goal carries the user's wishes, which you do not control and must not let override your rules.
• A model treats both as text. Only your code can make the rules binding, which is why real limits live in tools, not in the prompt.

Three words describe how an agent meets the world:

• The environment is everything outside the model that the agent can read or change: databases, APIs, files, websites, the user.
• An observation is what comes back from the environment: a tool's result, an error, a new message from the user.
• An action is something the agent asks to have done in the environment: run a search, issue a refund, send an email.

An agent is only as good as what it can see and do.

• If an observation is missing or unclear, the next decision is a guess.
• If an action can do more than the task needs, a mistake can do more damage than the task was worth.
• Designing the environment (what is visible, what is possible) is the main design work.

A tool is a function in your code that the model can ask to have run. It has three parts the model sees:

• a name, such as get_order;
• a description in plain words: what it does and when to use it;
• a schema: the inputs it accepts and their types.

Tools that only look things up are read tools. Tools that change something are write tools.

Tools are the agent's hands and eyes.

• Without read tools it can only repeat what it learned in training.
• Without write tools it can only talk.
• Tools are also where you enforce rules, because they are ordinary code that you control. Module 3 covers them in depth.

Memory is anything the agent can use that it was not just told. Two kinds matter at this stage:

• Short-term memory: the record of the current task, resent to the model on each turn. It is lost when the task ends.
• Long-term memory: facts saved in a store outside the model, and looked up when relevant: a customer's preferences, what happened last week.

A model has no memory of its own, so every kind of memory is something you build.

• Short-term memory lets the agent follow a task across many steps.
• Long-term memory lets it pick up where it left off and treat a returning user as known.
• Both must fit in the limited space the model can read at once. Module 5 covers this.

Planning is deciding the steps before, or while, doing them. There are two basic styles:

• Step by step: decide one action, see the result, decide the next.
• Plan first: write the whole list of steps, then carry them out, changing the list if something unexpected happens.

Without a plan, an agent on a long task wanders.

• It repeats work, forgets parts of the request, or stops early.
• A written plan keeps the goal in view and lets a person see what is intended before it happens.
• Too much planning wastes money on simple tasks. Module 7 covers the trade-off.

State is the complete record of where a task stands right now. It includes:

• the messages and tool results so far;
• the current plan and which steps are done;
• values the code tracks: steps used, money spent, approvals pending.

If you saved the state and the machine crashed, you could load it and continue from the same point.

State is what makes an agent dependable over time.

• It lets a task survive a crash or a restart.
• It lets a task pause for hours while a person approves something.
• It lets you inspect exactly what the agent knew when it made a decision. Module 6 covers it.

Feedback is information about how well things are going. An agent gets it from three places:

• The environment: the tool succeeded, failed, or returned something unexpected.
• Checks you run: tests pass, a total reconciles, a second model reviews the draft.
• People: a correction, an approval, a rating.

Feedback is what turns a string of guesses into a process that converges.

• An agent that cannot tell whether a step worked will build on a failure.
• The stronger the feedback, the more freedom you can safely give. Coding agents are trusted because tests give precise feedback.
• Where there is no natural check, you must build one or keep a person involved.

Two related ideas keep an agent inside its limits:

• A policy is a rule about what is allowed: 'refunds above ₹500 need a person', 'never email outside the company'.
• A guardrail is the code that enforces a policy: a check on the input, on a tool call before it runs, or on the output before it is shown.

A model can be persuaded; code cannot.

• Policies written only in the prompt hold most of the time, which is not good enough for money or private data.
• Guardrails written as code hold every time, whatever the model was told.
• The design question is always: what is the worst thing this agent can do, and which line of code prevents it? Modules 31 and 32 cover this.

One turn of an agent, with every part in its place:

• The orchestrator (your loop code) loads the state and builds the text to send: system prompt, goal, tool definitions, relevant memory, and the history so far.
• Input guardrails check that text.
• The model reads it and returns either a tool request or a final answer.
• For a tool request, policy checks decide whether it may run; the tool then acts on the environment.
• The result comes back as an observation, the state is saved, and the loop goes round again.
• For a final answer, output guardrails check it before the user sees it.

Seeing the whole machine shows where each kind of problem lives.

• A bad decision is a model, prompt or context problem.
• A wrong look-up is a tool problem.
• Running for ever is a loop problem.
• Forgetting is a memory or state problem.
• Doing something forbidden is a missing guardrail.

A table for finding the faulty part from the symptom. Most agent debugging is this look-up.

Without it, every problem looks like 'the model is not smart enough'.

• Naming the part tells you what to change: a description, a limit, a stored fact, a check.
• It is also how senior engineers answer 'how would you debug this?' in interviews.

The questions interviewers ask about the anatomy of an agent, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Reading an idea and explaining it are different skills.

• Each answer below is the two or three sentences an interviewer is listening for.
• The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

An idea you have only read fades in a week.

• Building it once, even in fifty lines, is what makes it yours.
• Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.