Make embedded references canonical

This commit is contained in:
hz
2026-07-13 22:32:31 -05:00
parent 292f8ff9b4
commit 37218d9f45
4 changed files with 21 additions and 1366 deletions
+15 -10
View File
@@ -25,10 +25,10 @@ distinctive fact combinations, answer wording, or images.
mismatches.
Known gap in that historical run: model-based medical review was completed, but
authoritative references were not added to `QUESTION_REFERENCE_REVIEWS.md` for
the `nbme5-*` records. Future runs must complete the reference-review gate below
before setting `isPublished` to `true`. The historical records should be treated
as awaiting that separate reference review.
authoritative references were not embedded in the `nbme5-*` question documents.
Future runs must complete the reference-review gate below before setting
`isPublished` to `true`. The historical records should be treated as awaiting
that separate reference review.
## Prerequisites
@@ -169,6 +169,8 @@ The generation prompt must require:
- Celsius and Fahrenheit together for every temperature;
- labs only when useful and only when their values also appear in the stem;
- `media: []` unless separately reviewed, licensed media will be added later;
- a `references` array that the reference-review gate populates with complete
authoritative source records before publication;
- selection percentages for every authored choice initialized to zero;
- established Step categories, 3-6 tags, and a unique deterministic ID.
@@ -236,8 +238,9 @@ Model agreement is not authoritative medical verification. Before publication:
1. Verify each keyed answer and material explanatory claim against current,
direct, authoritative sources such as professional guidelines, government
agencies, official labeling, or primary research.
2. Add an entry for every question to `QUESTION_REFERENCE_REVIEWS.md`, including
title, publisher, direct URL, supported claim, access date, and reviewer.
2. Add every verified source to the question document's `references` array,
including title, publisher, direct HTTPS URL, supported claim, ISO access
date, and reviewer. Do not maintain a separate repository reference ledger.
3. Search the entire question bank for exact and near-duplicate learning tasks,
not just duplicate text or IDs.
4. Record duplicate-review dispositions and substantially differentiate or
@@ -280,6 +283,7 @@ Verification is complete only when all of these pass:
- the API reports the expected upsert count;
- the database contains every expected external ID exactly once;
- every stored `correctChoiceId` matches the final payload;
- every stored `references` array exactly matches the verified payload;
- a targeted deployed-API search returns at least one newly inserted question;
- the returned question has the expected choices, answer key, and publication
state;
@@ -304,9 +308,10 @@ API_READ_RESULTS=<count>
## 10. Retain Reviews, Then Clean Up
Retain the reference ledger, duplicate-review record, validation summary, audit
decisions, and final ID/answer-key manifest according to project policy. These
are review records, not disposable generation scratch files.
Retain authoritative references in each question document. Retain the
duplicate-review record, validation summary, audit decisions, and final
ID/answer-key manifest according to project policy. These are review records,
not disposable generation scratch files.
After production verification:
@@ -332,7 +337,7 @@ generated-artifact cleanup.
- [ ] All anchoring and source-overlap warnings reviewed.
- [ ] Independent content audit approved every question.
- [ ] Every rejected question regenerated and re-audited.
- [ ] Authoritative references recorded for every question.
- [ ] Authoritative references embedded in every question document.
- [ ] Full-bank duplicate review completed.
- [ ] Image rights cleared for every media-bearing question.
- [ ] Final payload contains only intended new or intentionally updated IDs.