BIOEE 7600-103 · Week 4

Agentic literature synthesis

2 October 2026

Public copy of the slides as shown in class, with class-specific detail removed. Course page: benninglab.org/teaching/agentic-ai

Trending · this week

Hot goss

1 Oct · “Claude-shaped science”

Matthew Schwartz (Harvard physicist, visiting researcher at Anthropic): 36 manuscripts in 18 fields with 19 coauthors over three months.
His claim: many fields have problems that a technique from math, physics or CS would solve outright, if anyone knew it existed. Pick those, rather than using the model as a general collaborator.
EEB example: on Barro Colorado Island, tree species composition changes 4.5× faster than neutral theory allows.
“Good at science, but they are not scientists.” He had to “correct every sentence it wrote,” and “‘Done, with one asterisk’ is often ‘not done at all.’”
Published by Anthropic. In the reading room, with two new EEB papers: Ferrari et al. (seven guidelines for GenAI in ecology) and Pollock et al. (AI and biodiversity shortfalls).

Something to think about

Would a project-centred follow-on to this course be useful next semester? Each person brings a project of their own and builds it with an agent over the term.

What we asked

One prompt, your topic

Four lines to fill in (topic, scope, cutoff date, Chrome yes/no), then the agent works alone.

Plan: Opus splits the topic into up to 5 sub-questions.
Search: one Sonnet subagent per sub-question searches OpenAlex in parallel, reads at most 20 abstracts, and reads open-access full text. About 25 papers kept in all.
Full text via Chrome (optional): up to 10 more papers read through the Cornell library.
Evidence table and draft: one row per paper, with design and strength of evidence. Opus writes a ~3,000-word review: what the evidence shows, where people disagree, what would settle it, ways forward.
Critic: a fresh Opus subagent reviews it as an Annual Reviews editor would; Opus revises once.
Check and publish: every DOI looked up at Crossref; the review goes up as a page, with an untouched copy kept as review_original.html in case you revise it later.

Deliberately left out

The name of the review you compared against. Naming it invites the model to recite it.

Small groups · 20 minutes

How did your review do?

Mix people who ran it with people who did not. Walk through one review together.

What did it miss?
Are the references real, and do they say what the review says?
Is the disagreement section real, or balance it made up?
Does it judge evidence, or count it?
Would this be useful to you?

Whole class · 15 minutes

Where is this helpful, and where is it harmful?

Where would this speed up real work: a first pass at a new field, a reading list, a check on your own review?
Where could it do damage, and how might one guard against that?
Did reading full text instead of abstracts change anything you could see?

Next week · 10 minutes

Agentic scientific writing: what should we try?

Which writing task: a methods section from a dataset and script, a Discussion, a response to reviewers?
How should we evaluate Claude's output?

Extra, if time

John’s runs: one review topic, scored

A light and a heavier run on adaptation at range edges, scored against a published review.

The test

A test we can score

Adaptation at range edges, against Angert, Bontrager & Ågren 2020 (Annual Review of Ecology, Evolution, and Systematics).

Only literature published by 31 August 2020, the newest thing the review cites.
A rubric written from the review before any run: 20 points it makes, 5 on how critical it is, plus citation checks.
Two runs: a light one on an earlier draft of the class prompt (Chrome off), and a heavier version. Next slide: exactly what differed.

Same task, two budgets

What differed in the heavier run

	Light run earlier draft of the class prompt	Heavier run
Search subagents	3, on Sonnet	5, on Opus
Abstracts read	~88	134
Papers kept	26	62
Read in full	6, open access only (Chrome off)	40: 25 through the Cornell library in Chrome, 15 open access
Evidence table	design, strength, abstract or full text	adds edge type and result direction
Draft length	~3,000 words	~4,000 words
Analysis	none	a vote count in R, with a figure; what to count written down before looking
Critic	1 round, Sonnet	2 rounds, a fresh Opus subagent each

Same in both: Opus 5.5 as the main agent, the plan step, the critic’s brief, and the Crossref check. The prompt you were given differs from the light run in two ways: its critic is on Opus, and it allows up to 5 search subagents.

The scorecard

Light vs. heavier run, against the review

	No tools memory only	Light run 21 min	Heavier run 40 min
Review's 20 points: hit / partial / miss	6 / 7 / 7	5 / 11 / 4	11 / 5 / 4
Critical synthesis (of 10)	–	8	9
References that exist	–	37 of 37	63 of 63
References after the cutoff	–	0	0
Of the 20 papers you would expect	–	11	14
Cost at list price	–	$3.70	$24.70
Papers read in full	–	6 of 26	40 of 62

What the checks caught

Its first-draft reference list was typed from memory, with at least one error. The Crossref step fixed it.
It called one study "the one direct test" of gene swamping. It never found the stronger test Angert cites (Fedorka et al. 2012).
It mentioned that an edge on the map is not necessarily an edge in climate only in passing. The model with no tools stated it outright. (A 2019 result, so there was little literature to find by the cutoff.)

What more compute bought

Seven times the cost: better coverage, same blind spots

Bought

The theory and gene-flow substance the light run only gestured at, including the stronger swamping case (Fedorka et al. 2012).
Corrections from reading the papers. A famous "no variance at the edge" study selected an interior population; a reported gene-flow asymmetry was not significant.
Critics that fixed real errors, with every change and refusal logged.

Not fixed

Map edge vs. climate edge is still only in passing (new in 2019, little to find).
"No study has done X" stated confidently, and wrong: Sheth & Angert 2016 did it.
Its vote count leans on a split it chose after seeing the critics' comments. It said so, which is the right behaviour, and the reason to check.

Live demo

Reading the paper, not the abstract

Claude in Chrome, through the Cornell library proxy.

The agent opens login.proxy.library.cornell.edu/login?url=… in your browser, using your library login.
It reads methods and results, then updates the evidence table.
It never types a password. A login page stops it.

Recorded fallback in case the live demo fails.