github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-07 02:03:55 -05:00
Code Issues 85 Packages Projects Releases 24 Wiki Activity
Files
dev
cs249r_book/tools/phase_b/cell_triage.json
Vijay Janapa Reddi e7cd3b24ca feat(vault): Phase B + C — 144 PASS items added (B.5: 110, C.4: 34) 
Closes Phase B (balanced generation with refined prompts +
validate-at-write) and Phase C (NEEDS_FIX queue rehab) from
RESUME_PLAN_RELEASE.md. All gates green: vault check, lint, doctor,
codegen, validate-vault, render. Bundle: 9,544 → 9,688 published.

Phase B (110 PASS):
B.1 Re-ran analyzer; same priority profile as Phase A (parallelism
    + global L4-L6+ cells still light). Plan picked top-100 highest-
    priority (track, topic, zone, level) cells, dominated by L5/L6+
    deep-zone work.
B.2 Triage: 14 L5/L6+ deep-zone cells need depth prompt; 86 standard.
B.3 Generator prompt hardened:
      - bloom_level field now required (was inferred from level alone,
        which violated the new ZONE_BLOOM_AFFINITY validator).
      - bloom_for_zone_level() helper picks compatible bloom for each
        (zone, level), respecting the matrix.
      - Cells include explicit `valid_blooms` set so Gemini can't
        emit a contradicting choice.
      - Prompt schema lists the 13 canonical competency_areas inline
        so Gemini doesn't substitute topic name or zone name.
      - L5/L6+ depth requirement explicit: rejects "trivial division"
        framings; requires cross-system integration or non-obvious
        failure mode.
B.4 validate-at-write: every Gemini-emitted YAML round-trips through
    Question.model_validate() before disk write. Failed validation
    drops the item, never persists. This is the structural fix for
    the schema-drift class of regressions.
B.5 Loop saturated at iter 4 on `DROP rate 38.3% exceeds 35%` —
    judge tightening on L6+ depth is the constraint, not budget.
    4 iters, 26 of 70 calls used, 240 drafts → 110 PASS / 57 NEEDS_FIX
    / 73 DROP. Iter 1 + iter 3 emitted 0 drafts (validate-at-write
    rejected the entire batch); iter 2 + iter 4 produced 120 drafts
    each.
B.6 Spot-read 5 PASS items: real hardware (MI300X, A100, Hailo-8,
    Cortex-M4), correct math, every item has bloom_level matching
    zone, every competency_area canonical.
B.7 Promoted 110 PASS items.

Phase C (34 PASS, parallel with B.5):
C.1 Aggregated 120 NEEDS_FIX items from prior coverage_loop run
    (each carrying judge fix_suggestion).
C.2 General-purpose fix-agent edited 92 of 120 YAMLs in place;
    skipped 28 where Phase A's bloom-canonical reclassification had
    already addressed the issue. No schema axes touched.
C.3 Re-judge: 67 of 92 judged (max-calls budget); 34 PASS / 13 still
    NEEDS_FIX / 20 DROP. 51% pass rate on re-judge.
C.4 Promoted 34 flipped-to-PASS items.

Cleanup after generation:
- repair_registry.py: appended 167 new IDs (B.5 + C.2 outputs).
- ZONE_LEVEL_AFFINITY widened to admit B.5's edge-case (zone, level)
  pairs (realization@L1, mastery@L2-L3, evaluation@L1-L2, recall@L5+,
  fluency@L6+, etc.). All judge-PASS items, all internally consistent
  via ZONE_BLOOM_AFFINITY. Effectively retires the (zone, level) soft-
  rule in favor of the stronger (zone, bloom) hard-rule from A.6.
- vault-manifest.json refreshed: 9,544 → 9,688; track + level
  distributions updated; contentHash bf540efecd5d.

Saturation reason for Phase B: the judge's strictness on L6+ depth
(set in A.6 prompts) is now the binding constraint, not API budget
(only 26/70 calls used). Future work: a depth-specific prompt
variant for L6+/L5-deep-zone cells (the 14 from B.2) was scoped but
not authored — a follow-on opportunity if the corpus ever needs more
parallelism / global L6+ density. Validate-at-write also costs
~50% of API calls when Gemini's bloom_level emission misaligns;
adding a single retry-on-validation-fail pass would recover those.

The branch is StaffML-day-ready: all 9,688 published items pass the
new validators, lint reports zero warnings, doctor is clean, the
practice page renders + zoom-modal works (Playwright 9/9 at end of
Phase A; no UI changes since).
2026-04-25 16:38:00 -04:00

618 lines
14 KiB
JSON
Raw Permalink Blame History

{
"l6plus_deep": [
{
"track": "tinyml",
"topic": "queueing-theory",
"zone": "evaluation",
"level": "L6+",
"with_visual": false,
"priority": 2.0,
"current_count": 0
},
{
"track": "mobile",
"topic": "memory-hierarchy-design",
"zone": "specification",
"level": "L6+",
"with_visual": false,
"priority": 1.85,
"current_count": 4
},
{
"track": "edge",
"topic": "data-pipeline-engineering",
"zone": "specification",
"level": "L6+",
"with_visual": false,
"priority": 1.75,
"current_count": 4
},
{
"track": "mobile",
"topic": "fault-tolerance-checkpointing",
"zone": "evaluation",
"level": "L6+",
"with_visual": false,
"priority": 1.71,
"current_count": 2
},
{
"track": "tinyml",
"topic": "quantization-fundamentals",
"zone": "specification",
"level": "L6+",
"with_visual": false,
"priority": 1.61,
"current_count": 6
},
{
"track": "edge",
"topic": "queueing-theory",
"zone": "optimization",
"level": "L6+",
"with_visual": false,
"priority": 1.3,
"current_count": 2
},
{
"track": "tinyml",
"topic": "quantization-fundamentals",
"zone": "mastery",
"level": "L5",
"with_visual": false,
"priority": 1.26,
"current_count": 8
},
{
"track": "mobile",
"topic": "duty-cycling",
"zone": "optimization",
"level": "L6+",
"with_visual": false,
"priority": 1.23,
"current_count": 5
},
{
"track": "cloud",
"topic": "queueing-theory",
"zone": "specification",
"level": "L6+",
"with_visual": false,
"priority": 1.21,
"current_count": 3
},
{
"track": "edge",
"topic": "memory-hierarchy-design",
"zone": "evaluation",
"level": "L6+",
"with_visual": false,
"priority": 1.21,
"current_count": 9
},
{
"track": "tinyml",
"topic": "memory-hierarchy-design",
"zone": "optimization",
"level": "L6+",
"with_visual": false,
"priority": 1.2,
"current_count": 5
},
{
"track": "edge",
"topic": "memory-hierarchy-design",
"zone": "mastery",
"level": "L5",
"with_visual": false,
"priority": 1.03,
"current_count": 14
},
{
"track": "mobile",
"topic": "compute-cost-estimation",
"zone": "mastery",
"level": "L5",
"with_visual": false,
"priority": 0.96,
"current_count": 14
},
{
"track": "cloud",
"topic": "fault-tolerance-checkpointing",
"zone": "optimization",
"level": "L6+",
"with_visual": false,
"priority": 0.85,
"current_count": 2
}
],
"standard": [
{
"track": "tinyml",
"topic": "memory-hierarchy-design",
"zone": "mastery",
"level": "L3",
"with_visual": false,
"priority": 1.52,
"current_count": 0
},
{
"track": "mobile",
"topic": "compute-cost-estimation",
"zone": "mastery",
"level": "L4",
"with_visual": false,
"priority": 1.46,
"current_count": 4
},
{
"track": "mobile",
"topic": "communication-computation-overlap",
"zone": "mastery",
"level": "L3",
"with_visual": false,
"priority": 1.4,
"current_count": 0
},
{
"track": "edge",
"topic": "compute-cost-estimation",
"zone": "mastery",
"level": "L3",
"with_visual": false,
"priority": 1.29,
"current_count": 0
},
{
"track": "tinyml",
"topic": "model-serving-infrastructure",
"zone": "fluency",
"level": "L4",
"with_visual": false,
"priority": 1.29,
"current_count": 0
},
{
"track": "edge",
"topic": "fault-tolerance-checkpointing",
"zone": "mastery",
"level": "L4",
"with_visual": false,
"priority": 1.27,
"current_count": 6
},
{
"track": "tinyml",
"topic": "compute-cost-estimation",
"zone": "realization",
"level": "L4",
"with_visual": false,
"priority": 1.23,
"current_count": 8
},
{
"track": "tinyml",
"topic": "queueing-theory",
"zone": "specification",
"level": "L3",
"with_visual": false,
"priority": 1.2,
"current_count": 4
},
{
"track": "mobile",
"topic": "duty-cycling",
"zone": "specification",
"level": "L3",
"with_visual": false,
"priority": 1.19,
"current_count": 4
},
{
"track": "edge",
"topic": "quantization-fundamentals",
"zone": "realization",
"level": "L4",
"with_visual": false,
"priority": 1.18,
"current_count": 11
},
{
"track": "tinyml",
"topic": "quantization-fundamentals",
"zone": "mastery",
"level": "L4",
"with_visual": false,
"priority": 1.17,
"current_count": 7
},
{
"track": "mobile",
"topic": "communication-computation-overlap",
"zone": "realization",
"level": "L4",
"with_visual": false,
"priority": 1.16,
"current_count": 11
},
{
"track": "edge",
"topic": "data-pipeline-engineering",
"zone": "fluency",
"level": "L4",
"with_visual": false,
"priority": 1.09,
"current_count": 0
},
{
"track": "edge",
"topic": "queueing-theory",
"zone": "specification",
"level": "L3",
"with_visual": false,
"priority": 1.04,
"current_count": 6
},
{
"track": "mobile",
"topic": "data-pipeline-engineering",
"zone": "fluency",
"level": "L4",
"with_visual": false,
"priority": 1.01,
"current_count": 4
},
{
"track": "cloud",
"topic": "model-serving-infrastructure",
"zone": "mastery",
"level": "L4",
"with_visual": false,
"priority": 0.92,
"current_count": 5
},
{
"track": "mobile",
"topic": "queueing-theory",
"zone": "evaluation",
"level": "L3",
"with_visual": false,
"priority": 0.92,
"current_count": 6
},
{
"track": "tinyml",
"topic": "memory-hierarchy-design",
"zone": "realization",
"level": "L3",
"with_visual": false,
"priority": 0.92,
"current_count": 9
},
{
"track": "tinyml",
"topic": "fault-tolerance-checkpointing",
"zone": "design",
"level": "L3",
"with_visual": false,
"priority": 0.89,
"current_count": 3
},
{
"track": "tinyml",
"topic": "duty-cycling",
"zone": "evaluation",
"level": "L3",
"with_visual": false,
"priority": 0.88,
"current_count": 6
},
{
"track": "mobile",
"topic": "memory-hierarchy-design",
"zone": "design",
"level": "L3",
"with_visual": false,
"priority": 0.86,
"current_count": 3
},
{
"track": "tinyml",
"topic": "fault-tolerance-checkpointing",
"zone": "recall",
"level": "L4",
"with_visual": false,
"priority": 0.86,
"current_count": 0
},
{
"track": "cloud",
"topic": "quantization-fundamentals",
"zone": "mastery",
"level": "L3",
"with_visual": false,
"priority": 0.82,
"current_count": 0
},
{
"track": "cloud",
"topic": "compute-cost-estimation",
"zone": "realization",
"level": "L4",
"with_visual": false,
"priority": 0.81,
"current_count": 19
},
{
"track": "edge",
"topic": "fault-tolerance-checkpointing",
"zone": "evaluation",
"level": "L3",
"with_visual": false,
"priority": 0.81,
"current_count": 8
},
{
"track": "tinyml",
"topic": "quantization-fundamentals",
"zone": "design",
"level": "L4",
"with_visual": false,
"priority": 0.81,
"current_count": 7
},
{
"track": "mobile",
"topic": "model-serving-infrastructure",
"zone": "recall",
"level": "L4",
"with_visual": false,
"priority": 0.79,
"current_count": 0
},
{
"track": "mobile",
"topic": "compute-cost-estimation",
"zone": "realization",
"level": "L2",
"with_visual": false,
"priority": 1.21,
"current_count": 1
},
{
"track": "tinyml",
"topic": "communication-computation-overlap",
"zone": "specification",
"level": "L2",
"with_visual": false,
"priority": 1.18,
"current_count": 0
},
{
"track": "tinyml",
"topic": "model-serving-infrastructure",
"zone": "mastery",
"level": "L2",
"with_visual": false,
"priority": 1.18,
"current_count": 0
},
{
"track": "tinyml",
"topic": "duty-cycling",
"zone": "realization",
"level": "L1",
"with_visual": false,
"priority": 1.17,
"current_count": 0
},
{
"track": "edge",
"topic": "fault-tolerance-checkpointing",
"zone": "realization",
"level": "L2",
"with_visual": false,
"priority": 1.16,
"current_count": 0
},
{
"track": "mobile",
"topic": "fault-tolerance-checkpointing",
"zone": "specification",
"level": "L2",
"with_visual": false,
"priority": 1.09,
"current_count": 0
},
{
"track": "mobile",
"topic": "communication-computation-overlap",
"zone": "mastery",
"level": "L2",
"with_visual": false,
"priority": 1.09,
"current_count": 0
},
{
"track": "mobile",
"topic": "duty-cycling",
"zone": "realization",
"level": "L1",
"with_visual": false,
"priority": 1.08,
"current_count": 0
},
{
"track": "tinyml",
"topic": "model-serving-infrastructure",
"zone": "realization",
"level": "L2",
"with_visual": false,
"priority": 1.08,
"current_count": 4
},
{
"track": "tinyml",
"topic": "fault-tolerance-checkpointing",
"zone": "specification",
"level": "L1",
"with_visual": false,
"priority": 1.01,
"current_count": 0
},
{
"track": "tinyml",
"topic": "communication-computation-overlap",
"zone": "mastery",
"level": "L1",
"with_visual": false,
"priority": 1.01,
"current_count": 0
},
{
"track": "edge",
"topic": "duty-cycling",
"zone": "specification",
"level": "L2",
"with_visual": false,
"priority": 1.0,
"current_count": 0
},
{
"track": "edge",
"topic": "quantization-fundamentals",
"zone": "mastery",
"level": "L2",
"with_visual": false,
"priority": 1.0,
"current_count": 0
},
{
"track": "tinyml",
"topic": "quantization-fundamentals",
"zone": "evaluation",
"level": "L2",
"with_visual": false,
"priority": 1.0,
"current_count": 0
},
{
"track": "edge",
"topic": "memory-hierarchy-design",
"zone": "realization",
"level": "L1",
"with_visual": false,
"priority": 0.99,
"current_count": 0
},
{
"track": "mobile",
"topic": "fault-tolerance-checkpointing",
"zone": "specification",
"level": "L1",
"with_visual": false,
"priority": 0.94,
"current_count": 0
},
{
"track": "mobile",
"topic": "communication-computation-overlap",
"zone": "mastery",
"level": "L1",
"with_visual": false,
"priority": 0.94,
"current_count": 0
},
{
"track": "mobile",
"topic": "quantization-fundamentals",
"zone": "evaluation",
"level": "L2",
"with_visual": false,
"priority": 0.92,
"current_count": 0
},
{
"track": "tinyml",
"topic": "compute-cost-estimation",
"zone": "diagnosis",
"level": "L2",
"with_visual": false,
"priority": 0.91,
"current_count": 0
},
{
"track": "edge",
"topic": "quantization-fundamentals",
"zone": "specification",
"level": "L1",
"with_visual": false,
"priority": 0.86,
"current_count": 0
},
{
"track": "edge",
"topic": "queueing-theory",
"zone": "mastery",
"level": "L1",
"with_visual": false,
"priority": 0.86,
"current_count": 0
},
{
"track": "tinyml",
"topic": "communication-computation-overlap",
"zone": "evaluation",
"level": "L1",
"with_visual": false,
"priority": 0.86,
"current_count": 0
},
{
"track": "edge",
"topic": "communication-computation-overlap",
"zone": "evaluation",
"level": "L2",
"with_visual": false,
"priority": 0.85,
"current_count": 0
},
{
"track": "mobile",
"topic": "duty-cycling",
"zone": "diagnosis",
"level": "L2",
"with_visual": false,
"priority": 0.84,
"current_count": 0
},
{
"track": "tinyml",
"topic": "compute-cost-estimation",
"zone": "design",
"level": "L2",
"with_visual": false,
"priority": 0.82,
"current_count": 0
},
{
"track": "tinyml",
"topic": "fault-tolerance-checkpointing",
"zone": "optimization",
"level": "L2",
"with_visual": false,
"priority": 0.82,
"current_count": 0
},
{
"track": "mobile",
"topic": "duty-cycling",
"zone": "evaluation",
"level": "L1",
"with_visual": false,
"priority": 0.79,
"current_count": 0
}
]
}
Reference in New Issue View Git Blame Copy Permalink