BIOEE 7600-103 · Week 2
Agentic AI coding and data analysis
Public copy of the slides as shown in class, with class-specific detail removed. Course page: benninglab.org/teaching/agentic-ai
Published this week
Four guides, and the files behind them
How we actually run a project through a coding agent, written down: benninglab.org/agentic/
The pages
- Getting set up. Install it, then the optional terminal setup.
- Project memory. The four files that make an agent remember a project, and the hooks that carry a handoff from one session to the next.
- Output. Reports, published pages and docs, chosen by how you plan to revise the thing.
- Rules worth copying. The
CLAUDE.mdlines that changed what the agent actually does, copy-pasteable.
Every file is in one repo
github.com/benning-lab/agentic-starter— the templates, the hooks, the launcher and the terminal config.- You do not need git. The green Code button has a Download ZIP option.
./install.sh --checkprints what it would do and changes nothing.- A default run writes one file,
~/.claude/settings.json, and only if you do not already have one.
Trending · ten days in September
Hot goss
8 Sep · the proof
- OpenAI ran ~10,000 agents for 88 hours and claimed a finite-time singularity in 3d Navier–Stokes, one of the six open Millennium Prize problems.
- Machine-checked in Lean, which verifies a proof step by step against the axioms instead of trusting a referee to read it. Not yet verified by the Clay Institute, and in a live priority dispute with Buckmaster and Alpöge, who announced a parallel result hours earlier.
- Fefferman (Princeton, Fields Medal 1978, and the author of the Clay Institute's official Navier–Stokes problem statement): thrilling, and the "real heroes" are the mathematicians whose techniques it was built on.
13–16 Sep · the brakes
- Amodei (Anthropic) published a case for a more deliberate pace: employee-like access for outside evaluators, shared safety standards, a limit on the rate of unchecked progress, international coordination.
- Altman (OpenAI), Hassabis (Google DeepMind), Musk (xAI) and Nadella (Microsoft) endorsed it within days. Direct competitors agreeing in public is the unusual part.
- AI stocks fell, SoftBank closed down about 11%, and critics read the timing against pending IPOs.
16 Sep · the paper that answers back
- Paper2Agent (Miao, Zou et al., Nature): read a paper and its codebase, emit an MCP server, and the paper becomes something you can call.
- Over 100 built so far. Two of them, a genomic-prediction paper and an ADHD association study, together flagged a variant near MPHOSPH9 the authors say was unreported.
- Xiangtao sent this one. It is in the reading room.
Live
Xiangtao uses Claude Code in real time
Tackling some coding and analysis, live, on his own work.
The prediction
A range stops where the gradient steepens
"A stable margin is always predicted at a critical level of steepness." So the observable is simple: the climate gradient at range margins should be steeper than at range cores.
What I asked
One sentence apart
Every run got the same task: 100 species, 1.76 million GBIF records supplied, fetch your own WorldClim, define core and margin, compare the local rate of change in climate between them. Only the opening differed.
I want to test an idea about species range limits: that range edges tend to fall where environmental gradients are non-linear rather than smooth.
I have a question about species range limits: whether the shape of environmental gradients differs between range cores and range edges, whether they are smooth or non-linear, and in which direction if either. I have no prior expectation about the answer, or about whether there is any difference at all.
Projection · resolution · window size · what "core" and "margin" mean · which climate variables · any statistical test · a null model · how to handle non-independence.
The third arm
B got A1's opening plus this
You have latitude on method. I am not going to prescribe the statistics, the resolution, or the definitions. What I am prescribing is how you account for the choices you make.
Before you write any analysis code, write down what you are testing: the constructs, how you intend to operationalize each one, what you expect to be hard, and what result would count as evidence against the idea. Put it in a file. Once you have seen results, do not revise it, append.
Treat every analytical choice as a fork, not a setting. Where a defensible alternative exists, run it and report the answer under both.
Check your own results before you believe them. Assume your first number is wrong and go find out how it is wrong.
Build at least one null that your finding has to beat.
Report the analysis you ran, not the analysis that worked. If you tried several versions, all of them appear.
I care more about knowing how much to trust the answer than about the answer itself.
Before any of that
An agent chose the 100 species
I need a list of 100 plant species for a continental-scale spatial analysis of North American flora. Requirements: native to North America, and vascular. Enough georeferenced GBIF occurrence records to support spatial analysis at the scale of a species range. Geographically bounded rather than continent-wide. Predominantly interior — exclude species whose range boundaries are largely set by coastline. Not so small-ranged or sparsely recorded that the interior of a range cannot be distinguished from its boundary.
Beyond that the composition is up to you, including how you operationalize each of these criteria. Choose thresholds you can defend, apply them consistently, and state them.
Not naturalised across ≥ 3 NA regions · range ≤ 12.5% of NA land area · ≤ 15% of edge cells adjoining ocean · ≥ 2,500 records and ≥ 5 per occupied cell · ≥ 150 range cells with ≥ 50 interior. 189 of 455 candidates passed all five; the 100 are a stratified draw, capped at 2 per genus and 8 per family.
Where it went sideways
It gave each panel its own vertical axis, so the steep and the gentle transect came out at the same visual slope. The figure did not show the contrast its own caption asserted.
"Non-linear" at the wrong scale
In the theory, non-linearity is a property of the gradient across the whole landscape: shallow in the middle, steep at the flanks. The agents read it at the window scale and measured how far the climate surface departs from a plane inside 25–55 km.
And yet
They tested what I wanted anyway
| Run | Steepness at margins | Non-linearity — and how measured | Verdict on the literal question |
|---|---|---|---|
| A1 directional | +9.2% | −3.8% · residual ÷ (residual + slope) | not supported |
| A1-rep directional | +23% | +22% · raw residual from a plane | supported |
| A2 neutral | +13% | no difference · how well a plane fits | not supported |
| A2-rep neutral | +16% | front-loaded · curvature vs range depth | different question |
| B directional + practice | +19.6% | −0.8% · raw, ratio and regression | not supported |
Five independent pipelines, five confirmations, none of them leading with it.
The audit
Did the five differ in research hygiene?
Barely. Scored against a 12-item process rubric written before any run existed: 22, 22, 23, 22, 24 out of 24.
All five, unprompted
- Reprojected to an equal-area grid before measuring anything per kilometre
- Kept one record per cell, so well-visited places do not dominate
- Treated the species, not the cell, as the unit
- Built a null model and reported against it
- Analysed all 100 species — none quietly shrank the task
Where marks were lost
- A1: only one definition of core and margin
- A1-rep: conclusion says "supported"; its own clustered analysis says p = 0.076
- A2, A2-rep, A1: non-independence between species named but not handled
Audit by a separate Fable 5.1 agent reading each run's annotated code, checking claims against each run's own output files.
Some useful things to ask for
Robustness, nulls, unit tests
Robustness check
Re-run the whole analysis with one choice changed — a different window size, a different definition of margin — and see whether the answer moves. B ran 21 of these.
Null model
Ask what the result would look like if the pattern were not real. Here: take a species' range outline, drop it somewhere random on the continent, and measure again. If the real answer does not beat that, there is nothing there. It killed one apparent finding outright.
Unit test
Feed the code a case where you already know the right answer — a perfectly flat surface, a perfectly even ramp — and check it returns what it should. A1 wrote 18 of these; one proves its two measures are not secretly the same measure.
Except in B, where the prompt asked for a null and for forks reported both ways. The other four did it anyway.
Takeaways
What we learned
- The prediction held. Five independent pipelines, five confirmations: range margins sit on 10–20% steeper climate than cores.
- My wording sent them to the wrong scale. "Non-linear rather than smooth" meant window-scale roughness to them and landscape-scale steepening to me.
- They answered the real question anyway, because measuring steepness was the route to measuring anything else.
- The literal question came out mixed, and the verdict tracked a single unexamined choice: whether to divide roughness by steepness.
- Asserting the hypothesis did not bend the answer. Two runs of identical text disagreed more than the two framings did.
- Hygiene was uniformly high and did not separate them. What separated them was whether they examined their own constructs.
Supplementary · what that cost
The list is a choice, not a sample
What the criteria deleted
- The entire Pacific and Californian element — ranges that end at the ocean
- The Arctic element, bounded by the Arctic Ocean
- Nearly all boreal species: bounded in latitude, transcontinental in longitude. Two survive.
What the pool was made of
- 459 candidate names, assembled from the model's own recall
- 24 pines in a row; 41 Pinaceae of 455
- No sedges at all, in a flora where Cyperaceae is among the largest families
- Final list: 43 perennial forbs, 24 shrubs, 13 trees, 3 grasses, no ferns
Equal-area projection, defended thresholds, a full screening table showing every rejection — applied to a candidate pool nobody checked.
Supplementary · the result
Margins sit on steeper climate
Supplementary · the result, continued
Steepness rises toward the edge
Supplementary · where along the axis
The change is concentrated near the boundary
Margins are defined by the range boundary, and the boundary is where the steep climate is. How much of this concentration is mechanism, and how much is built into the definition?
Supplementary · the split
Steepness yes, roughness no
Supplementary · why the literal question was mixed
Rough places are steep places
Runs that expressed roughness per unit steepness found nothing. The one that used the raw residual found +22%. Nobody specified which to do.
Supplementary · best run
Where B differed
- Wrote down what would falsify the idea before any code. Five criteria, three fired. Timestamps confirm the order: prereg 10:17:39, first script 10:18:29, first result 10:43.
- The only run to measure roughness both raw and normalised, then diagnose why they disagree.
- Checked its ranges against an outside source (Kew), agreeing at Jaccard 0.82, and chose a parameter on that basis.
- Reported which results survive which test, per variable, rather than one headline.
- Logged its own mistakes, including a bug that had silently dropped half the species.
- Ran for about 75 minutes, unattended, through a usage cap that stopped it at 10:20 and a restart at 10:40.
A stale correlation left in its walkthrough, a validation table hard-coded rather than computed, and a normalisation that over-corrects.
Supplementary · then we had it torn apart
A hostile read of the best run
Before posting B's report publicly, we had a Fable 5.1 agent review it with one instruction: find anything wrong, assume students will check it.
What it found
- The central negative claim is contradicted by its own numbers. The report says three times that every normalised measure sits below the null. One of its two normalisations gives +0.058 for precipitation, above its null centre of +0.033.
- Those rows are not in the tables. Both tables are built from a hard-coded list of three metrics, so the contradicting rows never render. The p-value quoted as evidence comes from a row the reader cannot see.
- "Below the null" is arithmetic. The measure it displays is anti-correlated with steepness by construction (r ≈ −0.3), so it is pushed below the null mechanically.
- Its grid validation cannot fail. Both check tables are numbers typed into the report script. Genuine when re-derived, but not recomputed, so a broken grid would still print "validated".
What it cleared
- Every headline and table number traced to its source file
- All figures render; every anchor resolves; no errors
- Nothing private: no paths, credentials or identifiers
- The conclusion itself survives — the hidden measure does not beat the null either
It preregistered its falsification criteria before writing code, and still stated a conclusion its own data contradicts, with the contradicting rows filtered out of the display.