Building an AI Agent: What "Agent" Means Beyond the Buzzword

The Word Got Overloaded

Search for "AI agent" and you'll find the term applied to a single API call that looks up today's weather, a chatbot that remembers your last five messages, and a system that spends six hours refactoring a codebase overnight with no one watching. Those aren't the same thing, and treating them as three points on one scale hides what actually matters: what changes when you build one, and when you shouldn't bother. The word describes a mechanism, not a product category, and once you can name the mechanism, most of what makes an agent hard to reason about stops being mysterious.

The Loop Is the Whole Idea

A plain call to a language model API is a single round trip: you send text in, the model sends text back, and the conversation is over unless you send another message yourself. An agent changes exactly one thing about that shape - instead of the model's output always being the final answer, its output can also be a request for more information or permission to take an action, which your code then carries out and reports back on before the model produces anything the user sees. Everything people mean by "agent" - tool use, browsing, coding assistants, autonomous research - is a variation on that one loop: the model's output decides what happens next, not a human clicking through a fixed script.

The Tool-Use Loop, Step by Step

Here's what that loop actually looks like on the wire. Along with your prompt, you send the model a list of tools it's allowed to use - each one a name, a plain-English description of what it does, and a schema describing its inputs, no different in spirit from documenting a function signature for a colleague. A minimal tool definition looks like this:

{
  "name": "get_order_status",
  "description": "Look up the current shipping status of a customer order by its order ID.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string", "description": "The order ID, e.g. ORD-48213"}
    },
    "required": ["order_id"]
  }
}

The model reads the conversation and either answers directly in text, or emits a structured call to one of those tools: a name and a set of arguments, not free-form prose. Your code - not the model, which has no ability to touch a database, hit an API, or run a shell command on its own - executes that specific call, then sends the result back as part of the next request. The model reads the result alongside everything that came before and decides again: answer now, or ask for another tool. That's the entire mechanism. Providers differ on the exact field names - one might call it a tool call and a finish reason, another a tool-use block and a stop reason - but the shape underneath is identical everywhere.

A Workflow Isn't the Same Thing as an Agent

It's worth being precise here because the two get conflated constantly. In a workflow, you write the sequence: call the search tool, then always call the summarizer, then always format the output a certain way. The model fills in specific steps, but the order and the branching are yours, fixed in code. In an agent, the model chooses which tools to call, in what order, and when to stop, based on what it's already seen - you supply the tools and the goal, not the plan. A workflow is more predictable and cheaper to debug; an agent is more flexible and can handle cases you didn't specifically write a branch for. Most production systems that call themselves "agents" are actually a workflow with one or two genuinely agentic steps inside it, and that's usually the right call, not a compromise.

Should You Actually Build One?

Before reaching for an agent loop, it's worth checking four things, because each added iteration is a fresh charge to the API and a fresh chance for the model to go somewhere you didn't intend. Is the task genuinely multi-step and hard to fully specify in advance - "find this fact and summarize it" is not, "triage this support ticket, gather whatever context is missing, and either resolve it or escalate it with a clear reason" plausibly is. Does the outcome justify the extra cost and latency an agent loop adds over one API call - a background job that runs for two minutes and saves someone twenty is an easy yes; a live chat reply someone is staring at a spinner for is a much harder one. Is the model actually capable at this specific kind of task, not just at language tasks in general - an agent loop amplifies a model's judgment, for better and for worse. And can errors be caught and recovered from - a broken agent that only ever drafts an email for a human to approve is low-risk; one that sends the email itself and can't be undone is a different proposition entirely.

A Worked Example: Two Tasks, Two Answers

Take two concrete tasks through those four questions. A bot that answers "what are your business hours" by returning a fixed string fails all four: there's no multi-step ambiguity to resolve, no meaningful cost saved by looping, no real judgment being exercised, and nothing that needs recovering from because there's nothing to get wrong. Now take a support ticket that needs an account looked up, an order history checked, a refund policy applied to a specific dollar amount, and a decision about whether to resolve it automatically or hand it to a human - that passes all four: it's genuinely multi-step and not fully knowable in advance, the time saved per ticket adds up fast at any real volume, the model's judgment is doing real work deciding which policy applies, and a mistake can be caught by requiring human sign-off above a certain refund amount. Same underlying model, same API, completely different answer to "should this be an agent" - because the answer was never about the technology.

What Breaks That Doesn't Break With One Call

A single request has an obvious failure mode: it returns a bad answer, and you see it immediately. A loop introduces failure modes that only show up after several iterations, quietly. The most expensive one is a loop with no exit condition - a model that keeps calling tools because nothing tells it to stop, racking up a charge for every round trip until you notice a bill or a timeout. The subtlest one is a tool call that never actually runs: some models occasionally write out what looks like a tool call as plain text instead of the structured format your code is watching for, and if nothing checks for that shape mismatch, the call silently does nothing and the model moves on as if it had happened. And errors compound: a wrong tool result at step two becomes an input the model reasons from at step five, and by the time the final answer is wrong, the actual mistake is buried several turns back in a transcript nobody reads unless something breaks.

MAX_ITERATIONS = 8

def run_agent_loop(user_message, tools, model_client):
    messages = [{"role": "user", "content": user_message}]

    for iteration in range(MAX_ITERATIONS):
        response = model_client.send(messages, tools=tools)

        if not response.tool_calls:
            return response.text  # the model answered; loop is done

        for call in response.tool_calls:
            result = execute_tool(call.name, call.arguments)
            messages.append(call.as_message())
            messages.append({"role": "tool", "content": result})

    raise RuntimeError(
        f"agent did not finish within {MAX_ITERATIONS} iterations - "
        "bail out instead of paying for an unbounded loop"
    )

Who Runs the Loop

So far this describes the loop conceptually, but somebody has to actually write the code that sends the request, checks for tool calls, executes them, and loops - and that job now sits on a real spectrum rather than being one choice. At one end, you write that loop yourself: full control, and you own every bug in it. In the middle, a library or SDK provides the loop for you and calls your tool functions directly, so you write the tools and skip the boilerplate. At the far end, a hosted runtime runs the entire loop on infrastructure you don't manage at all, including the sandbox where a tool like a shell or a file system actually executes - you configure it and read the results, you don't run the process yourself. None of these options change the underlying mechanism from this lesson; they just move the boilerplate of the four steps above somewhere else. Which one is right depends on how much you need to control what happens inside the loop versus how much you'd rather not maintain that code at all.