[PR #1433] [MERGED] feat(d1): cutover production D1 + live worker serving v1.0 schema #9102

New Issue

GiteaMirror · 2026-05-03T01:20:37-05:00

GiteaMirror commented

2026-05-03 01:20:37 -05:00

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1433
Author: @profvjreddi
Created: 4/22/2026
Status: ✅ Merged
Merged: 4/22/2026
Merged by: @profvjreddi

Base: dev ← Head: feat/vault-d1-cutover

📝 Commits (1)

8d385b0 feat(d1): cutover production D1 to schema v1.0 + live worker serving

📊 Changes

5 files changed (+181 additions, -5 deletions)

View changed files

📝 interviews/staffml-vault-worker/src/rate_limit.ts (+5 -1)
📝 interviews/staffml-vault-worker/src/types.ts (+10 -2)
📝 interviews/staffml-vault-worker/wrangler.toml (+2 -2)
➕ interviews/vault-cli/scripts/ship_d1.py (+132 -0)
📝 interviews/vault-cli/src/vault_cli/compiler.py (+32 -0)

📄 Description

Summary

Cutover of the production Cloudflare D1 database + staffml-vault Worker to the schema v1.0 corpus. Worker is live and serving v1.0 + Phase 2 reclassifications end-to-end.

Live verification (run at commit time):

$ curl https://staffml-vault.mlsysbook-ai-account.workers.dev/manifest
{"release_id":"d1-cutover","release_hash":"997747a8f43bbd89…","schema_version":"1","published_count":9199,"schema_fingerprint_ok":true}

$ curl .../questions/cloud-0185
{"id":"cloud-0185","level":"L6+","zone":"mastery","competency_area":"latency","bloom_level":"create","phase":"both",…}

$ curl '.../questions?track=cloud&level=L6%2B&limit=1'
{"items":[{"id":"cloud-0179","level":"L6+","zone":"mastery",…}],"next_cursor":"…"}

Four fixes

compiler.py — populate chains table from chains.json. Was silently empty; only surfaced when D1 (FKs enforced by default, unlike SQLite) rejected chain_questions inserts with FOREIGN KEY constraint failed.
types.ts (worker) — add competency_area, bloom_level, phase, human_review_* fields. Worker SQL was already SELECT * so columns flowed through; only the TS row interface needed updating.
rate_limit.ts — floor expirationTtl at 60s. Old calc could emit 11s, which D1 KV rejects. Was throwing error 1101 on every request post-deploy.
wrangler.toml — bump SCHEMA_FINGERPRINT to match v1.0 vault.db (b97218dae…).

New script

interviews/vault-cli/scripts/ship_d1.py — end-to-end vault build → SQL dump → wrangler d1 execute --file. Handles FK ordering. Repeatable for future schema bumps. Used for this cutover; can be wired into vault ship as a leg later.

Site cutover status — hybrid today

The bundled corpus.json is still the primary data path for the Next.js site. The worker path works but is opt-in via env vars. Full sync→async conversion of all getQuestions() callers is a separate refactor (deferred because every caller needs an audit pass).

To flip when ready:

export NEXT_PUBLIC_VAULT_API=https://staffml-vault.mlsysbook-ai-account.workers.dev
unset NEXT_PUBLIC_VAULT_FALLBACK

Still outstanding

DNS / custom domain staffml-vault.mlsysbook.ai isn't mapped; currently on workers.dev default
Site callers → async migration for full bundle drop
Paper figures regen (pre-v1.0 counts still baked)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1433 **Author:** [@profvjreddi](https://github.com/profvjreddi) **Created:** 4/22/2026 **Status:** ✅ Merged **Merged:** 4/22/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `feat/vault-d1-cutover` --- ### 📝 Commits (1) - [`8d385b0`](https://github.com/harvard-edge/cs249r_book/commit/8d385b0c1ad20d09bc3b71bf723f23ddb094bed3) feat(d1): cutover production D1 to schema v1.0 + live worker serving ### 📊 Changes **5 files changed** (+181 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `interviews/staffml-vault-worker/src/rate_limit.ts` (+5 -1) 📝 `interviews/staffml-vault-worker/src/types.ts` (+10 -2) 📝 `interviews/staffml-vault-worker/wrangler.toml` (+2 -2) ➕ `interviews/vault-cli/scripts/ship_d1.py` (+132 -0) 📝 `interviews/vault-cli/src/vault_cli/compiler.py` (+32 -0) </details> ### 📄 Description ## Summary Cutover of the production Cloudflare D1 database + staffml-vault Worker to the schema v1.0 corpus. Worker is live and serving v1.0 + Phase 2 reclassifications end-to-end. Live verification (run at commit time): ``` $ curl https://staffml-vault.mlsysbook-ai-account.workers.dev/manifest {"release_id":"d1-cutover","release_hash":"997747a8f43bbd89…","schema_version":"1","published_count":9199,"schema_fingerprint_ok":true} $ curl .../questions/cloud-0185 {"id":"cloud-0185","level":"L6+","zone":"mastery","competency_area":"latency","bloom_level":"create","phase":"both",…} $ curl '.../questions?track=cloud&level=L6%2B&limit=1' {"items":[{"id":"cloud-0179","level":"L6+","zone":"mastery",…}],"next_cursor":"…"} ``` ## Four fixes 1. **`compiler.py`** — populate `chains` table from `chains.json`. Was silently empty; only surfaced when D1 (FKs enforced by default, unlike SQLite) rejected `chain_questions` inserts with `FOREIGN KEY constraint failed`. 2. **`types.ts` (worker)** — add `competency_area`, `bloom_level`, `phase`, `human_review_*` fields. Worker SQL was already `SELECT *` so columns flowed through; only the TS row interface needed updating. 3. **`rate_limit.ts`** — floor expirationTtl at 60s. Old calc could emit 11s, which D1 KV rejects. Was throwing error 1101 on every request post-deploy. 4. **`wrangler.toml`** — bump `SCHEMA_FINGERPRINT` to match v1.0 vault.db (`b97218dae…`). ## New script `interviews/vault-cli/scripts/ship_d1.py` — end-to-end `vault build` → SQL dump → `wrangler d1 execute --file`. Handles FK ordering. Repeatable for future schema bumps. Used for this cutover; can be wired into `vault ship` as a leg later. ## Site cutover status — hybrid today The bundled `corpus.json` is still the primary data path for the Next.js site. The worker path works but is opt-in via env vars. Full sync→async conversion of all `getQuestions()` callers is a separate refactor (deferred because every caller needs an audit pass). To flip when ready: ``` export NEXT_PUBLIC_VAULT_API=https://staffml-vault.mlsysbook-ai-account.workers.dev unset NEXT_PUBLIC_VAULT_FALLBACK ``` ## Still outstanding - DNS / custom domain `staffml-vault.mlsysbook.ai` isn't mapped; currently on `workers.dev` default - Site callers → async migration for full bundle drop - Paper figures regen (pre-v1.0 counts still baked) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-03 01:20:37 -05:00

GiteaMirror closed this issue

2026-05-03 01:20:38 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#9102