what is an agent? | Benning Lab

← back to the course page

This is the ten-minute version of everything the seminar assumes. If you have never used an AI agent, or have used one without being sure what made it an agent, start here. No computer-science background is needed and none is assumed.

The short version

A large language model predicts text. That is the whole of what it does. Give it some words and it produces the words that plausibly follow, one piece at a time, based on patterns in an enormous amount of training text.

An agent is that same model placed in a loop, handed a set of tools it can actually operate — a shell, a web browser, a Python interpreter, your file system — and given a goal rather than a question. It decides what to do, does it, looks at the result, and decides what to do next, repeating until it thinks it is finished or you stop it.

Everything interesting and everything alarming follows from that one change: the model stopped producing sentences about the world and started taking actions in it.

Four rungs

The word “AI” now covers systems that differ enormously in what they can do to your data. It helps to see them as a ladder. The same task runs through all four.

These are rungs of capability, not brands. One product can sit on several of them depending on what is switched on, and the box you type into looks identical either way. The useful habit is asking which rung you are on right now, not which company made the tool.

A language model

Text in, text out. No memory between uses, no access to anything, no way to check what it said. It is not looking anything up. It is producing what sounds right.
Our taskWe asked one for herbarium records of Clarkia xantiana subsp. parviflora collected before 1980. It said it had no live access to any database, then listed specimens anyway — Abrams 5361, Howell 5021, Munz 13345, Lewis 412 — with dates, localities and herbaria. Every collector is real. Not one of those collection numbers appears among the 452 herbarium specimens of this species in GBIF, and the real dates for those collectors miss the claimed ones by decades. Asked again, it named a different botanist as the original collector. Nothing in the output looks wrong. Only the database says so.
A chatbot

The same model wrapped in a conversation, so it remembers what you said earlier in the session and has been trained to be helpful and to follow instructions. With no tools switched on it still has no access to anything outside itself: it cannot look anything up, and it cannot check its own claims.
Our taskYou can now refine across turns — "only Californian records," "only after 1980." The answers get more responsive to you. They are not any more likely to be true.
A model with tools

Now it can call things: search the web, run a snippet of code, query a database. Crucially, it can bring real results back into its own context and use them. This is the rung where output can start being checkable, because there is now something outside the model that produced it.
Our taskIt queries GBIF and returns records that do exist, because it fetched them, and you can click through and check. What it says about them is still its own summary. And you are still driving: one request, one answer.
An agent

You give it a goal instead of an instruction, and it plans its own route. It chooses which tools to use and in what order, runs them, reads what came back, notices problems, and tries again. It may take fifty actions before it says anything to you. That stretch of unsupervised action is the thing that makes it an agent.
Our task"Get the occurrence records for this species, clean them, and fit a species distribution model." It pulls the records, drops the ones at (0, 0) and in the ocean, quietly throws out everything georeferenced only to a county centroid, picks a climate dataset you have not heard of, fits the model on its defaults, and hands you a map. Twenty minutes, no input from you. Two of those choices were routine. Two it made on your behalf. Which two, and how would you know?

So which rung am I on? Today's assistants slide between rungs two and four inside the same chat window. A plain reply is rung two. The moment one searches the web, runs code, or reaches a connected database it is on rung three — and when it does that over and over, deciding each time what to try next, it is on rung four whatever the interface calls itself. The tell: did it go and get something, and did what came back change what it did next?

The loop

Every agent, whatever it is called and whoever built it, is running the same cycle.

Plan — decide the next step toward the goal. Act — call a tool: run code, query an API, edit a file. Observe — read what came back, including errors. Judge — decide whether that worked and whether the goal is met. If not, loop. The number of times round this loop before it reports back to you is the amount of autonomy you have granted it.

The interesting failures live in judge. Judging whether a result is scientifically acceptable is the step that requires knowing what a right answer would look like, and it is the step agents are worst at. When there is a clear pass/fail criterion — the tests pass, the file parses — agents iterate well. When the criterion is “does this look reasonable,” they will often decide that it does.

Reading and writing

The loop tells you how an agent works. It does not tell you what it can do to you. For that, ask a second question: what is this thing allowed to change?

Some agents only read. They search, fetch pages, query databases, run an analysis on a copy. An assistant looping over web searches is doing entirely genuine agentic work and cannot alter anything you own. That is not the same as harmless: anything an agent reads, and any page it fetches, is a route by which your own data can leave. Others write. They edit your files, run shell commands, modify databases, send mail, commit to your repository.

That distinction cuts across products rather than along them, and it is the axis that actually governs risk. Two agents can run the identical loop the identical number of times: the read-only one wastes your afternoon and may mislead you, the read-write one leaves your data in a state you did not choose and may not notice. A difference in kind, not degree.

So the loop is not the dangerous part. The loop combined with write access is. Before letting any agent near real work, know which of the two you have — and if it can write, know exactly what it is allowed to write to.

What they are good and bad at

Horstmann and colleagues handed coding agents the stages of a real neuroscience analysis pipeline and scored them against the standards of the scientists who built it. It is the Week 2 reading, and the two lists below come from it.

Reliably good at

Well-specified stages with a checkable criterion
Writing and debugging code against a test
Mechanical transformation: reformatting, reshaping, converting
Reading and extracting from large volumes of text
Doing all of the above much faster than you

Reliably bad at

Deciding whether their own output is scientifically sound
Sustained visual reasoning — they will plot a result, look at it, and miss an obvious problem
Stringing many stages together without drift
Saying "I don't know" instead of producing something
Knowing when a task needed judgement they do not have

Two patterns from that study are worth carrying into every session. Stages the agents solved perfectly in isolation broke once composed into a full pipeline. And when the authors classified every occasion an agent looked at a plot of its own output, the times it misread or explained away a real problem outnumbered the times it caught one, on every task where it looked at a plot at all.

Why “it acts” changes everything

A chatbot that is wrong produces a wrong sentence, and you are the last line of defence before it becomes a wrong claim. That is a familiar risk and we already have habits for it.

An agent with write access that is wrong has already run the code, already rewritten the file, already sent the query. The error is not in a draft you are reviewing; it is in your data directory. Three consequences follow, and the seminar returns to them all term:

Errors become silent. A hallucinated citation is visible if you look. A dropped subset of rows during a merge is not, and neither is a coordinate system quietly reprojected. The agent will report success either way, because from inside the loop it succeeded.

Provenance thins out. The agent made forty decisions to produce that file, and the file records none of them. Agent tools do keep a transcript of every action and its output, and reading that transcript is what this seminar means by an audit. But nothing attaches it to the result, and nobody reads it unless they decide to.

Speed removes the pause. The friction of doing analysis by hand is also the occasion on which you notice things. Removing the work removes the noticing, unless you deliberately put the noticing back.

The one rule this course adds. Everything else about responsible AI use in research follows the standards you already know: you are accountable for your work regardless of the tools you used, and you never paste unpublished data or personal information into a free external tool. Because agents act, we add one more: gate every irreversible action — delete, overwrite, submit, send — behind human confirmation. An agent should be able to propose deleting something. It should not be able to delete it.

The audit, in four questions. Every demo this term ends the same way. What did it read? What did it change? What did it decide for us? How would we know if it was wrong? Treat an agent the way you treat a new field assistant in their first week — quick, capable, keen to report success — and read the first datasheets in full before you start spot-checking.

Vocabulary

Everything below will come up. You do not need to memorise it; skim it now and come back.

The model itself

Large language model (LLM): A model trained on very large amounts of text to predict what comes next. Everything else here is built on top of one.
Token: The unit a model reads and writes — roughly a short word or word-fragment. Everything is counted, priced, and limited in tokens.
Context window: How much the model can hold in mind at once, in tokens. Anything outside it may as well not exist. Long agent sessions can fill it, which is when behaviour gets strange.
Prompt: What you send it. System prompt is the standing instruction set the tool's builders wrote, which you usually cannot see.
Hallucination: Fluent, confident, false. Not a bug that will be patched — a direct consequence of a system that generates plausible continuations rather than retrieving facts.
Nondeterminism: The same prompt can give different answers on different runs. This is why "I ran it again and it was fine" is not evidence, and why reproducibility is genuinely hard here.
Reasoning model / extended thinking: A model that generates a long internal working-out before answering. Better on hard problems, slower, more expensive. The visible "thinking" is not a reliable account of what actually determined the answer.

What makes it an agent

Tool / tool call / function calling: Something outside the model it can invoke — run a shell command, execute Python, fetch a URL, query a database. The bridge between generating text and doing things.
Agent: A model given tools and permission to loop: plan, act, observe, judge, repeat, until it decides it is done. The autonomy is the definition.
Workflow (as opposed to an agent): A fixed sequence of steps you wrote, that happens to call a model at points. Far more predictable. Most tasks that people build agents for are better served by a workflow, and knowing which you need is a real skill.
Autonomy / permission gating: How many actions it may take before checking with you, and which actions require your say-so. The single most important setting in any agent tool.
Read-only / read-write: Whether an agent's tools can change anything. Searching and fetching are read-only; editing files, running shell commands and sending mail are read-write. The best single predictor of how much damage one mistake can do.
Memory: What persists after the context window or the session ends — usually notes the agent writes to a file and reads back later.
MCP (Model Context Protocol): A common standard for plugging tools and data sources into an agent, so the same connector works across different assistants.
Prompt injection: An agent cannot fully separate the text it reads from the instructions it follows. A web page, a README or a data file can carry wording aimed at the agent, and it may act on it. This is why an agent that only reads still needs watching.
Subagent / multi-agent: An agent that spawns other agents to work in parallel and reports back. More throughput, and correspondingly more places for an error to hide.

Making it more reliable

Retrieval-augmented generation (RAG): Fetch relevant real documents first, then have the model answer using them. Reduces fabrication because the answer is anchored to retrieved text — it does not eliminate it.
Fine-tuning: Further training of an existing model on your own specialised data. Expensive, and usually not the answer; a better prompt or retrieval normally is.
Human in the loop: A person deliberately placed at a checkpoint to review or approve before the process continues. Only meaningful if the person has enough information to actually judge.
Benchmark / eval: A standard task set used to score models. Treat published scores the way you would treat a p-value from an analysis you did not see: informative, and not the same as the thing you care about.
Out-of-distribution: Input unlike anything in training. Model performance falls off, often without any corresponding drop in confidence. Your specific study system is more likely to be here than you think.

Before Friday

Skim the Week 1 readings on the course page. The skim instructions are there for a reason; do not read them all in full.
Bring a laptop.
Bring one question you would genuinely want an agent to answer from your own research — something real from your own data or your own literature, not a demo question. We will try some of them live, and decide together what is safe to put in before anything runs.

You are not expected to have used any of this before. The point of the seminar is to look carefully at what these systems actually do, together, with our own work as the test case.