Make embedded references canonical
This commit is contained in:
@@ -52,6 +52,8 @@ existing IDs. For each question, verify:
|
||||
for the correct choice;
|
||||
- every follow-up has exactly A-C choices and a valid keyed answer;
|
||||
- new `choiceSelectionPercentages` entries start at zero;
|
||||
- authoritative sources supporting approved additions are merged into the
|
||||
question's `references` array without duplicating existing URLs;
|
||||
- no unresolved coverage issue remains unless documented as a waiver.
|
||||
|
||||
Deserialize the payload as `BulkUpsertQuestionsRequest` and run the actual
|
||||
|
||||
+15
-10
@@ -25,10 +25,10 @@ distinctive fact combinations, answer wording, or images.
|
||||
mismatches.
|
||||
|
||||
Known gap in that historical run: model-based medical review was completed, but
|
||||
authoritative references were not added to `QUESTION_REFERENCE_REVIEWS.md` for
|
||||
the `nbme5-*` records. Future runs must complete the reference-review gate below
|
||||
before setting `isPublished` to `true`. The historical records should be treated
|
||||
as awaiting that separate reference review.
|
||||
authoritative references were not embedded in the `nbme5-*` question documents.
|
||||
Future runs must complete the reference-review gate below before setting
|
||||
`isPublished` to `true`. The historical records should be treated as awaiting
|
||||
that separate reference review.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -169,6 +169,8 @@ The generation prompt must require:
|
||||
- Celsius and Fahrenheit together for every temperature;
|
||||
- labs only when useful and only when their values also appear in the stem;
|
||||
- `media: []` unless separately reviewed, licensed media will be added later;
|
||||
- a `references` array that the reference-review gate populates with complete
|
||||
authoritative source records before publication;
|
||||
- selection percentages for every authored choice initialized to zero;
|
||||
- established Step categories, 3-6 tags, and a unique deterministic ID.
|
||||
|
||||
@@ -236,8 +238,9 @@ Model agreement is not authoritative medical verification. Before publication:
|
||||
1. Verify each keyed answer and material explanatory claim against current,
|
||||
direct, authoritative sources such as professional guidelines, government
|
||||
agencies, official labeling, or primary research.
|
||||
2. Add an entry for every question to `QUESTION_REFERENCE_REVIEWS.md`, including
|
||||
title, publisher, direct URL, supported claim, access date, and reviewer.
|
||||
2. Add every verified source to the question document's `references` array,
|
||||
including title, publisher, direct HTTPS URL, supported claim, ISO access
|
||||
date, and reviewer. Do not maintain a separate repository reference ledger.
|
||||
3. Search the entire question bank for exact and near-duplicate learning tasks,
|
||||
not just duplicate text or IDs.
|
||||
4. Record duplicate-review dispositions and substantially differentiate or
|
||||
@@ -280,6 +283,7 @@ Verification is complete only when all of these pass:
|
||||
- the API reports the expected upsert count;
|
||||
- the database contains every expected external ID exactly once;
|
||||
- every stored `correctChoiceId` matches the final payload;
|
||||
- every stored `references` array exactly matches the verified payload;
|
||||
- a targeted deployed-API search returns at least one newly inserted question;
|
||||
- the returned question has the expected choices, answer key, and publication
|
||||
state;
|
||||
@@ -304,9 +308,10 @@ API_READ_RESULTS=<count>
|
||||
|
||||
## 10. Retain Reviews, Then Clean Up
|
||||
|
||||
Retain the reference ledger, duplicate-review record, validation summary, audit
|
||||
decisions, and final ID/answer-key manifest according to project policy. These
|
||||
are review records, not disposable generation scratch files.
|
||||
Retain authoritative references in each question document. Retain the
|
||||
duplicate-review record, validation summary, audit decisions, and final
|
||||
ID/answer-key manifest according to project policy. These are review records,
|
||||
not disposable generation scratch files.
|
||||
|
||||
After production verification:
|
||||
|
||||
@@ -332,7 +337,7 @@ generated-artifact cleanup.
|
||||
- [ ] All anchoring and source-overlap warnings reviewed.
|
||||
- [ ] Independent content audit approved every question.
|
||||
- [ ] Every rejected question regenerated and re-audited.
|
||||
- [ ] Authoritative references recorded for every question.
|
||||
- [ ] Authoritative references embedded in every question document.
|
||||
- [ ] Full-bank duplicate review completed.
|
||||
- [ ] Image rights cleared for every media-bearing question.
|
||||
- [ ] Final payload contains only intended new or intentionally updated IDs.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -104,9 +104,10 @@ For each question, document:
|
||||
References must support the keyed answer and any management recommendations,
|
||||
diagnostic thresholds, formulas, or numerical claims. A citation does not grant
|
||||
permission to copy wording or media; the separate text-originality and image-
|
||||
copyright requirements still apply. If the question schema does not expose a
|
||||
reference field, retain this information in the associated authoring or review
|
||||
record until it can be stored with the question.
|
||||
copyright requirements still apply. Store each verified source in the question
|
||||
document's `references` array; published question writes require at least one
|
||||
complete reference. Legacy questions with an empty array remain readable while
|
||||
awaiting reference review.
|
||||
|
||||
## Variable Distractor Pools
|
||||
|
||||
|
||||
Reference in New Issue
Block a user