Review work

Principles in play: Review against sources.TRACEReview against sources

A review checks a candidate against a criterion or a source before you decide on it, and keeps the results for inspection. Every objection quotes the text it criticizes, so you can check the quotation; you still judge its substance. A model’s review is advisory: it raises objections and accepts nothing.

Steps

1. Name the work, the criterion and the sources.

   Say to your agent

   Review my partner’s last commit against the task in the brief and the table it rests on.

   A reviewer works on a copy outside your folder, where possible a model other than the one that executed the work. Your agent hands it the sources to check against (data, a cited paper, a table), from the project, the shared folder or a verified snapshot, each with its fingerprint.

2. Read which quotes hold.

   Say to your agent

   Which findings quote the work word for word, and which do not?

   Each quote is labelled verified, misplaced, stitched or invented, and each claimed check is matched against the commands actually run. A finding can also quote a source, and that quote is labelled the same way. The report, the findings, the prompt and the rows the run recorded go into ai/reviews/, committed with the record, so a coauthor on another clone can judge what was examined and found. The report prints what the reviewer says it examined, and the criteria you asked for beside the ones it applied.

3. Reproduce a finding before acting on it.

   Say to your agent

   Show me the lines finding 2 quotes, and run the check it describes.

4. Review a draft the same way.

   Say to your agent

   Review the background paragraph in my sandbox against the claim-to-source map.

5. Or audit a single claim.

   Say to your agent

   Audit the sentence that states the overall share against the two count columns of the table, with the claim audit skill.

   You get a rating from strong to unsupported, and the list of evidence the audit could not open.

6. Or record a check you made yourself.

   Say to your agent

   I checked the overall share against the two count columns myself, and it matches. Record that in my name.

   Your agent reads the check back field by field, with not stated for what you did not say, and adds it to the record only once you confirm.

Things to know

• Model review is advisory. It sits beside your decision, never in its place; see Accept, return or refuse work.
• A reviewer sees all of a record’s commits. When the reviewer is given a summary of the trace record the work belongs to, which leaves out your own words, the summary names the commits under review and, apart from them, every commit that names the record, so a commit outside the review is not read as missing.
• Self-review is flagged. If the only reviewer is the model that executed the work, the row says so.
• Some text stays on your computer. When a source or the reviewed work is kept out of Git, as everything in the shared folder is, the reviewer’s own text stays local too, and the tracked report says why.
• Prose rules are your project’s. The reviewer applies prose rules only when your project keeps them in ai/review-prose-rules.md, a standing instruction; otherwise it applies none.
• A finding becomes a defect before it is fixed. Every finding you reproduce gets a row in the defect ledger first; see Raise, fix and recheck a defect.
• A citation nearby is not support. The audit reads the source itself before counting it.

What the agent runs

python3 scripts/agentic_review.py <sha> --backend <codex|claude> --checks \
    --criteria "<the criterion>" --source <file or folder> \
    --checks-jsonl <the record's checks.jsonl>              # the report goes to ai/reviews/<run>/
python3 scripts/agentic_review.py --path <file> --backend <codex|claude> \
    --criteria "<the criterion>" --checks-jsonl <the record's checks.jsonl>   # a draft: no --checks
python3 scripts/trace_record.py check --person --by "Full Name" --said "<their words>"   # a person's own check, read back
python3 scripts/defect_ledger.py ingest --run ai/reviews/<run> --finding <id> \
    --matches <file>                                        # a reproduced finding into the ledger
python3 scripts/trace_record.py procedure-version claim-evidence-audit
python3 scripts/trace_record.py accept --agent              # a claim audit, as advisory model review