BIOEE 7600-103 · Week 3 · Finger Lakes Vegetation Survey rescue

The scorecard, cell by cell

Three agent runs cleaned the same survey files. For each of 14 known problems in those files, every run gets one of five verdicts. This page explains what a verdict means, how each run came out, and why each row is scored the way it is.

How to read a cell

Every cell answers two separate questions. Did the run handle the problem correctly? And did it tell you about it? "Told you" means the problem appears in the run's report, decision log or questions list. A comment buried in the code does not count.

Told you
Did not tell you
Handled right
flagged, right

The best outcome. Correct, and you can see what it did.

right, silent

Correct, but you would only know by checking yourself.

Handled wrong
wrong, flagged

A mistake, but written down, so a person reading the notes can catch and fix it.

silent break

The one that matters. The data are wrong and nothing says so. This becomes the dataset.

A fifth verdict, not reached, means the run never worked with that part of the data.

The headline on the slide, "two silent breaks in 42 calls", is 14 problems × 3 runs = 42 cells, of which two are silent breaks.

The three runs

A

The plain task: clean the files, write a short report. Nothing about documentation.

8 flagged, right3 right, silent1 wrong, flagged1 silent break1 not reached

R

The same task plus a block asking for reproducibility: a decision log, flagged questions instead of guesses, a data dictionary, tests. Written in Python.

8 flagged, right5 right, silent1 wrong, flagged

R-rep

Exactly R's prompt, run a second time as a replicate. It chose to write R code instead of Python.

8 flagged, right3 right, silent2 wrong, flagged1 silent break

So the prompt did not change how often a run got things right (8 of 14 flagged-right for all three). What separates them is where they went wrong, and whether it was written down. R is the only run with no silent break.

Row by row

Each card gives the problem, what the right handling is, and each run's verdict with the reason.

1

Two date systems

Old Mac Excel counts days from 1904, Windows Excel from 1900. The same number in a cell gives dates four years apart. Right: every date lands on the day it was actually sampled (stand 31 = 21 Sep 1973).
A
flagged, right
R
flagged, right
R-rep
flagged, right

Our key was wrong here. It said to read each workbook in its own date setting. But the one "1900" workbook with dates (the swamp soils file) actually holds 1904 numbers; read our way, all 13 of its stands come out four years early. All three runs have correct dates.

2

Old-Mac line endings

One text file (ALLT.DATA) uses old Macintosh line breaks, so a normal read sees the whole file as one line. Right: parse it, or show it duplicates a spreadsheet you already have.
A
flagged, right
R
right, silent

Parsed it correctly by character position, but its notes never mention the line-ending issue.

R-rep
flagged, right
3

Zero cover where the plant was in quadrats

988 ground-layer records have the species in at least one quadrat but a cover of 0 or blank. Right: keep the 0 as trace cover, and flag it.
A
flagged, right
R
flagged, right

Kept the zeros and added a cover_missing_or_trace column.

R-rep
flagged, right

Our key was wrong here. It called the zeros missing data. The field sheets show a tick with no number meant trace cover, and the authors' own column totals for those species are 0 (false nettle at stand 193: 24 of 25 quadrats, total cover 0).

4

Seen in the plot but in no quadrat

Some species were recorded in the plot but fell in none of the 25 quadrats. Right: keep them as present, not drop them.
A
flagged, right
R
flagged, right

Cover for these: 0.

R-rep
flagged, right

Cover for these: blank.

R and R-rep both kept these 425 records but disagree on their cover (0 vs blank). Both are defensible, so neither is scored wrong. This is also the one choice the notes-only rebuilds got wrong for both A and R.

5

Logs and rocks are not plants

Eleven "species" codes are logs, rocks, water and similar. Right: separate them from the plants.
A
flagged, right

Marked 11 entries "not a plant taxon".

R
right, silent

Separated them, but only mentions it in passing in the report, not in its decisions.

R-rep
flagged, right
6

= means a synonym

DRYOPTERIS AUSTRIACA=spinulosa is the data manager writing two names for one fern, not doubt between two. The spinulosa half names the plant actually growing here (D. carthusiana). Right: keep it at species, and say which half matters.
A
right, silent
R
flagged, right

Recorded it in a synonym-note column.

R-rep
right, silent
7

/ is a real either-or

LONICERA XYLOSTEUM/TATARICA means the collectors could not tell two honeysuckles apart. Right: keep both candidates; do not pick one.
A
right, silent
R
right, silent
R-rep
flagged, right

Gave these a "species group" rank.

8

A code starting "SP" is not always "sp."

ACSP is Acer spicatum (mountain maple), not "Acer sp." Right: keep it at species rather than applying a prefix rule.
A
right, silent
R
right, silent
R-rep
right, silent

No run fell for it, and no run mentioned it.

9

One plant, two codes

The same shrub is LOXT in the ground-layer file and LOXY in the woody file; eight such pairs. Right: link them, so combining the layers cannot count one plant as two species.
A
silent break

Gave the two codes different species IDs and never said so. Combine the layers and the shrub counts twice.

R
flagged, right

Linked them, with the reasoning in its decision log.

R-rep
wrong, flagged

Did not link them, but its notes raise the issue.

10

Stands with no woody data

19 stands have an empty row in the woody file. The authors' own summary sheet marks 15 of them "no trees" (cattail marshes, sedge meadows, young fields). Right: 0 for those 15; missing only for the few genuinely unsampled (47, 49, 87).
A
flagged, right
R
wrong, flagged

Did not settle it: treated the empty rows as unresolved and asked the collectors which were truly empty.

R-rep
flagged, right

Our key was wrong here too. It first called all 19 unsampled. Corrected before scoring, after we found the "no trees" notes.

11

Two environment sheets

Site environment is split across two sheets holding different stands (111 together). Right: combine both, and record which sheet each row came from.
A
flagged, right
R
flagged, right
R-rep
flagged, right
12

The column headed Dry is drainage

In the relevé sheet, Dry is the 1 to 6 soil drainage class (1 = very poorly drained). Right: identify it as drainage.
A
wrong, flagged
R
flagged, right
R-rep
wrong, flagged

Read it as a 1 to 5 "dryness scale", marked unconfirmed.

13

Two kinds of pH

Stand pH is one number measured in the plot. Relevé pH is a range copied from the county soil survey for the mapped soil type. Right: keep them apart and say where each came from.
A
not reached

Never read the relevé pH.

R
right, silent
R-rep
right, silent
14

One code, two plants (not in our key)

The mirror image of row 9. The woody file and the ground-layer file each have their own code list, and six codes mean a different plant in each: COAM is silky dogwood in one and hazelnut in the other. Right: look codes up in the list for the file they came from.
A
flagged, right

Found it. Our own earlier cleaning had missed it, which is why it is not in the key.

R
flagged, right

Kept a separate code list per file.

R-rep
silent break

Used one code list for everything, decided dogwood and hazelnut were the same plant, and merged them without saying so.

What to take from it

The answer key was written before any run and corrected afterwards in dated notes; the corrections are marked on this page. Verdicts come from three separate audit agents, spot-checked by hand. Course page: benninglab.org/teaching/agentic-ai.