Agentic data exploration, compilation & archiving

Trending · this week

Hot goss

Small groups · 12 minutes

Three or four people you do not usually sit with

1. What is your gut reaction to agentic AI so far? Re: the course, and AI in general.
2. Tried anything since the course began? Totally fine if not. If you did, where did it get stuck?
3. Something you want to try with aAI, but don't quite know how to go about it?

Back together

One stuck point per group

Where are people getting stuck: setup, trust, or not knowing what to use it for?

Please post any other comments in #general, or DM us! We want to adapt the course in real time as we go along.

The case

A big 'ole plant community ecology dataset

Peter Marks's Finger Lakes Vegetation Survey, the basis of his Guide to the Plant Communities of the Central Finger Lakes Region. The paper sheets turned up in January, in a Cornell office cleanout. The digital files survived on backup CDs kept from 1994 to 2006.

What we asked

The prompts

What came out

A folder of tables, a report, and a paper trail

Same raw files, same core counts. They differ in how they organised the species list and in how much they asked a person to check.

Read R's report, as the agent wrote it →

The answer key

Written before any run existed

Thirteen problems we previously found in these files, in earlier iterations with Claude. Each run is scored on two questions: did it handle the problem right, and did it say so?

Then the agent checked our work

One code, two plants

The woody file and the ground-layer file each have their own list of four-letter codes. Six codes mean a different plant in each list. Join the two layers on the code alone and a silky dogwood becomes a hazelnut.

Three of the six shown. Our earlier cleaning made exactly this join: 18 woody records got the wrong species, and 11 roses became a mustard. It is not in the answer key because we did not know about it.

All thirteen

The scorecard

Two silent breaks in 42 calls, both about which plant is which.

Scored against a corrected key. Ours was wrong on the date systems and on trace cover, and it called 15 stands unsampled for woody plants that the authors' sheet records as "no trees": real zeros, not missing data.

Every cell explained →

Can anyone reconstruct it?

Reproducibility, checked rather than claimed

Takeaways

What we learned

1. The code reproduces; the agent doesn't. Every run rebuilt byte-identical from raw. The same prompt run twice still wrote Python once and R once, and split on 425 cover values and on the code collision.
2. The prompt bought a paper trail, not correctness. Tests that catch breakage, a decision log, open questions flagged rather than guessed. R-rep had all of it and still made a silent break.
3. Notes are only as good as they are right. A stranger rebuilt 92% from R's notes and 82% from A's. R's notes described one choice wrongly, and the rebuild followed them.
4. It broke on which plant is which. Both silent breaks were taxonomy, not arithmetic. Check there first.

Next week · literature synthesis

What should we run for literature synthesis next week?

A question you would want answered from the literature, a set of papers you know well enough to check the agent against, or a review you have always wanted someone else to write.

Suggestions now, or in #general this week.