Make embedded references canonical

This commit is contained in:
hz
2026-07-13 22:32:31 -05:00
parent 292f8ff9b4
commit 37218d9f45
4 changed files with 21 additions and 1366 deletions
+2
View File
@@ -52,6 +52,8 @@ existing IDs. For each question, verify:
for the correct choice;
- every follow-up has exactly A-C choices and a valid keyed answer;
- new `choiceSelectionPercentages` entries start at zero;
- authoritative sources supporting approved additions are merged into the
question's `references` array without duplicating existing URLs;
- no unresolved coverage issue remains unless documented as a waiver.
Deserialize the payload as `BulkUpsertQuestionsRequest` and run the actual
+15 -10
View File
@@ -25,10 +25,10 @@ distinctive fact combinations, answer wording, or images.
mismatches.
Known gap in that historical run: model-based medical review was completed, but
authoritative references were not added to `QUESTION_REFERENCE_REVIEWS.md` for
the `nbme5-*` records. Future runs must complete the reference-review gate below
before setting `isPublished` to `true`. The historical records should be treated
as awaiting that separate reference review.
authoritative references were not embedded in the `nbme5-*` question documents.
Future runs must complete the reference-review gate below before setting
`isPublished` to `true`. The historical records should be treated as awaiting
that separate reference review.
## Prerequisites
@@ -169,6 +169,8 @@ The generation prompt must require:
- Celsius and Fahrenheit together for every temperature;
- labs only when useful and only when their values also appear in the stem;
- `media: []` unless separately reviewed, licensed media will be added later;
- a `references` array that the reference-review gate populates with complete
authoritative source records before publication;
- selection percentages for every authored choice initialized to zero;
- established Step categories, 3-6 tags, and a unique deterministic ID.
@@ -236,8 +238,9 @@ Model agreement is not authoritative medical verification. Before publication:
1. Verify each keyed answer and material explanatory claim against current,
direct, authoritative sources such as professional guidelines, government
agencies, official labeling, or primary research.
2. Add an entry for every question to `QUESTION_REFERENCE_REVIEWS.md`, including
title, publisher, direct URL, supported claim, access date, and reviewer.
2. Add every verified source to the question document's `references` array,
including title, publisher, direct HTTPS URL, supported claim, ISO access
date, and reviewer. Do not maintain a separate repository reference ledger.
3. Search the entire question bank for exact and near-duplicate learning tasks,
not just duplicate text or IDs.
4. Record duplicate-review dispositions and substantially differentiate or
@@ -280,6 +283,7 @@ Verification is complete only when all of these pass:
- the API reports the expected upsert count;
- the database contains every expected external ID exactly once;
- every stored `correctChoiceId` matches the final payload;
- every stored `references` array exactly matches the verified payload;
- a targeted deployed-API search returns at least one newly inserted question;
- the returned question has the expected choices, answer key, and publication
state;
@@ -304,9 +308,10 @@ API_READ_RESULTS=<count>
## 10. Retain Reviews, Then Clean Up
Retain the reference ledger, duplicate-review record, validation summary, audit
decisions, and final ID/answer-key manifest according to project policy. These
are review records, not disposable generation scratch files.
Retain authoritative references in each question document. Retain the
duplicate-review record, validation summary, audit decisions, and final
ID/answer-key manifest according to project policy. These are review records,
not disposable generation scratch files.
After production verification:
@@ -332,7 +337,7 @@ generated-artifact cleanup.
- [ ] All anchoring and source-overlap warnings reviewed.
- [ ] Independent content audit approved every question.
- [ ] Every rejected question regenerated and re-audited.
- [ ] Authoritative references recorded for every question.
- [ ] Authoritative references embedded in every question document.
- [ ] Full-bank duplicate review completed.
- [ ] Image rights cleared for every media-bearing question.
- [ ] Final payload contains only intended new or intentionally updated IDs.
File diff suppressed because it is too large Load Diff
+4 -3
View File
@@ -104,9 +104,10 @@ For each question, document:
References must support the keyed answer and any management recommendations,
diagnostic thresholds, formulas, or numerical claims. A citation does not grant
permission to copy wording or media; the separate text-originality and image-
copyright requirements still apply. If the question schema does not expose a
reference field, retain this information in the associated authoring or review
record until it can be stored with the question.
copyright requirements still apply. Store each verified source in the question
document's `references` array; published question writes require at least one
complete reference. Legacy questions with an empty array remain readable while
awaiting reference review.
## Variable Distractor Pools