Improve dynamic question learning loop

This commit is contained in:
hz
2026-07-23 16:50:53 -05:00
parent c91e70deb5
commit 32c455bba0
14 changed files with 422 additions and 16 deletions
+21 -1
View File
@@ -11,6 +11,24 @@ Effective ability begins as an equal blend of overall and category question-bank
Elo, then shifts toward dynamic objective Elo up to an 80% objective weight after
10 attempts. Aggregate outcomes calibrate each case family's Elo over time.
Quality-standard-4 blueprints add Step 3 coverage and lifecycle metadata. They
must declare the organ system, physician task, encounter frame, site of care,
tested concepts, medical reviewer, review date, and review expiration. Every
distractor must identify the misconception it represents. An expired medical
review prevents publication.
## Adaptive learning loop
New questions are chosen using a 50/30/20 policy:
- 50% scheduled concept reviews, when any are due.
- 30% underexposed blueprint areas, adjusted by `blueprintWeight`.
- 20% interleaved maintenance from the learner's weakest objectives.
Incorrect answers enter concept review; skips and correct first attempts do not.
Reviews use a freshly generated variant at 1, 3, 7, 21, and 45 days. An
incorrect review resets the interval to one day.
## Content requirements
- At least three curated case families per objective.
@@ -20,6 +38,8 @@ Elo, then shifts toward dynamic objective Elo up to an 80% objective weight afte
- A complete explanation, key takeaway, USMLE pearl, review follow-up, and
wrong-answer remediation.
- At least one authoritative reference.
- Quality-standard-4 content includes misconception IDs and current medical
review metadata.
Promotion validates 10,000 seeds for every quality-standard-2 blueprint. Existing
blueprint versions are immutable; content changes require a version increment.
@@ -37,7 +57,7 @@ dotnet QuestionsApi.dll export-dynamic-review \
Keep `answer-key.json` separate from `review-packet.json`. Two clinical reviewers
score realism, distractor quality, explanation quality, and Step 3 relevance from
1–5 and predict whether each item is authored or dynamic.
1-5 and predict whether each item is authored or dynamic.
The pilot passes when there are no critical medical or ambiguity findings, every
dimension averages at least 4/5, and source-identification accuracy is no greater