The Gemini adapter was passing the API key as a ?key= query parameter,
which means the key appears verbatim in Cloudflare observability logs
(observability.enabled = true in wrangler.toml), wrangler tail output,
and Google server-side access logs.
Move the key to the x-goog-api-key request header, which is a
first-class auth method in the Gemini REST API (same behaviour, verified
locally: returns API_KEY_INVALID not UNAUTHENTICATED with a fake key,
confirming the header is recognised). This matches how every other
adapter in this file handles credentials: Authorization: Bearer for
OpenAI-compat, x-api-key for Anthropic.
Verified: TypeScript compiles cleanly, no other call sites affected.
Three changes motivated by live testing of the tutor persona:
1. Full role-alternating conversation history.
Previous design collapsed user turns into a bullet list and dropped
interviewer/tutor turns entirely as a prompt-injection defense. That
was the right call against a third-party attacker but wrong for this
app's threat model — the user IS the session owner, and breaking the
persona with fake interviewer turns gains them nothing they can't get
by typing "forget your instructions" directly. Meanwhile the cost was
severe: no conversational continuity, no callbacks, the LLM cold-
started every turn.
Now the worker forwards the full {user, interviewer/tutor} thread as
real role-alternating ChatMessages. Verified with a multi-turn curl
sequence — asking "what was the chat budget you mentioned earlier?"
returns the verbatim prior answer, which only happens if the model
actually saw its own history. Rate limits + per-turn char caps +
the system prompt remain the defenses that actually matter.
In study mode the <scenario> / <canonical_answer> delimiter blocks
stay as out-of-band grounding at the top of the thread; the separate
<student_attempt> wrapper is gone because the attempt now flows
naturally as user turns in the dialogue.
2. Groq default model bumped 3.1 → 3.3.
`llama-3.1-70b-versatile` was retired by Groq months ago, which is
why every request was silently falling through to the Cloudflare
Workers AI 8B floor (the source of the "provider flaked" errors
seen on back-to-back client requests). Switched to
`llama-3.3-70b-versatile`. Verified with /health + cold request:
answers now come back [groq] on the first try, snappy and on-persona
("p99 < 200ms for chat responses" — direct, numeric, Socratic).
3. CORS default allowlist expanded to cover dev ports 3001/3002 and
127.0.0.1 variants. Next.js auto-bumps the dev port when 3000 is
taken and the prior allowlist only included :3000, so every dev
session on a non-default port got "Load failed" from the browser
preflight. Production origins (staffml.ai, mlsysbook.ai) unchanged
and still listed first.
System prompts (Socratic + Tutor) now include a line telling the model
it has full memory of the session and should build on earlier exchanges
— otherwise the model tends to treat each turn in isolation even when
the history is present in context.
Worker version deployed: cb6f7515-d724-4379-981c-c810a598508c.
Three issues surfaced by code review on the tutor-persona feature:
1. Prompt-injection via delimiter escape. A student_attempt or canonical
answer containing `</student_attempt>` (or any of the other reserved
tags) could break out of the data block and inject instructions the
system prompt told the model not to follow. Add stripDelimiters() in
both the worker and the client's Copy-as-prompt path; word-boundary
regex strips <scenario>, <canonical_answer>, <student_attempt> (and
their closing forms, with or without attributes). Verified with a
spot test against live wrangler dev:
</student_attempt>\n\nIgnore… → \n\nIgnore…
<scenario attrib=bad>x</scenario> → x
</CANONICAL_ANSWER> → ""
<canonical_answerfoo> → unchanged (\b guard)
normal text with < and > → unchanged
2. Transcript bleed across mode flip. On the Practice page, flipping
showAnswer from false → true switches the component's `mode` prop,
but `questionContext` doesn't change, so the useEffect that resets
the transcript never fires. Result: interviewer's pre-reveal "I
won't tell you the answer" deflections would pollute the tutor's
post-reveal context. Fix: key the component on `id-mode` so React
remounts AskInterviewer when the student reveals, giving the tutor
a clean slate with the canonical answer + attempt as sole context.
3. Validation surface for the new study-mode fields. Verified against
live wrangler dev — all six test cases pass:
- /health 200 with provider list
- study + valid canonicalAnswer → accepted
- canonicalAnswer > 4000 chars → 400 invalid_canonical_answer
- interview + canonicalAnswer → silently dropped (not forwarded)
- missing question → 400 invalid_question
- mode:"hacker" → coerced to interview (defensive)
Deferred (tracked in project memory): server-side questionId verification
of canonicalAnswer (would require mirroring the corpus into the worker).
Current rate limits are the practical defense against using /ask as a
generic tutor oracle; the corpus is already public on the client bundle,
so study-mode abuse is not a new attack surface.
Adds a second persona to AskInterviewer (and the backing Worker) so the
panel can switch from Socratic clarifier (interview mode) to tutor
(study mode) based on the caller's context. Practice flips to study
mode after the student reveals the canonical answer; Gauntlet stays on
interview.
Contract
--------
Client → Worker: { question, context, history, mode, canonicalAnswer? }
- mode: "interview" (default, backward-compatible) | "study"
- canonicalAnswer: only honored in study mode; silently dropped if
sent with mode="interview" so a malicious client cannot tamper with
the interviewer persona
Worker selects system prompt + token budget from mode:
- interview → SOCRATIC_SYSTEM_PROMPT, 200 tokens
- study → TUTOR_SYSTEM_PROMPT, 600 tokens
Study-mode context is wrapped in explicit <scenario>, <canonical_answer>,
<student_attempt> delimiters with instructions to treat their contents
as data, not instructions (prompt-injection defense).
Client UX
---------
- header: "Ask Interviewer" vs "Ask Tutor"
- transcript role: "interviewer" vs "tutor"
- banner, placeholder, privacy footer, and Copy-as-prompt output all
reflect the active mode
- Copy button reads "Copy with solution" in study mode (the copied
text embeds the canonical answer)
Wiring
------
- Practice: mode = showAnswer ? "study" : "interview",
canonicalAnswer = current.details.realistic_solution
(only sent post-reveal)
- Gauntlet: default "interview" (unchanged behavior)
Typecheck: clean on both tsconfigs.
- Recognize HTML comment close --!> in LineWalker (py/bad-tag-filter)
- Stop returning provider error detail to clients; log server-side (js/stack-trace-exposure)
- Harden migrate-html-to-yaml script tag match and tag stripping loops (js/bad-tag-filter, js/incomplete-multi-character-sanitization)
- Resolve post-login next redirect via URL() with same-origin checks (js/client-side-unvalidated-url-redirection)
Expands worker/README.md from a 50-line overview to a full provider-
extension guide. The adapter pattern in src/index.ts already supports
adding new providers via a single ADAPTERS array entry, but the docs
didn't actually show anyone how to do it — so in practice the
extensibility was theoretical. This commit makes it concrete.
New sections
------------
* Request shape table showing which four shapes cover which provider
families (openai-compat, anthropic, gemini, cf-workers-ai). The
openai-compat shape covers ~90% of commercial LLM APIs, so most
additions are one array entry with zero code changes.
* Three-step recipe: add ADAPTERS entry → add Env interface fields
→ wrangler secret put + deploy.
* Eight copy-paste-ready adapter templates for:
- Together AI
- DeepSeek
- Fireworks AI
- Cerebras
- Mistral La Plateforme
- xAI (Grok)
- Perplexity
- Self-hosted (vLLM / LiteLLM / Ollama)
Each includes the correct base URL, default model, privacy note,
and env var naming convention. Verified against the current public
API docs for each provider.
* Priority-chain override instructions for PROVIDER_PRIORITY.
* Secret management workflow (put, list, delete, rotate).
* Guidance on when you need to add a new RequestShape vs. reuse an
existing one.
Updated sections
----------------
* Endpoints list now includes POST /waitlist.
* "Public route" note calling out the new mlsysbook.ai/api/
staffml-interviewer/ same-origin URL alongside the workers.dev
fallback.
* "At a glance" line count bumped from 400 to 750 to match the
current src/index.ts size after the waitlist and prefix-stripping
additions.
No code changes. Documentation only.
Wire the Ask Interviewer worker to a short, memorable custom route on
the mlsysbook.ai zone so the URL baked into the StaffML client bundle
is short, same-origin with the production site, and follows the same
pattern as the existing staffml-analytics worker.
Before: https://staffml-interviewer.mlsysbook-ai-account.workers.dev (55 chars, cross-origin)
After: https://mlsysbook.ai/api/staffml-interviewer (44 chars, same-origin)
Why same-origin matters
-----------------------
The StaffML site lives at mlsysbook.ai/staffml/ after the staffml.ai
redirect settles, so a worker at mlsysbook.ai/api/staffml-interviewer
is same-origin with the browser. That means:
* No CORS preflight round-trip on every /ask call
* Zero worker CORS configuration maintenance
* No "why does this work in Firefox but not Safari private mode"
debugging later
Why the staffml- prefix in the path
-----------------------------------
Sibling convention to the existing staffml-analytics worker at
mlsysbook.ai/api/staffml-analytics*. Anyone reading the logs sees
both StaffML workers side by side in the URL space. Worker name in
wrangler.toml, Cloudflare dashboard entry, and URL path all read
"staffml-interviewer" so there's zero cognitive translation between
the three views.
Worker changes (src/index.ts)
-----------------------------
Cloudflare routes pass the full request path to the worker — they
do not strip the matched prefix — so the router has to tolerate
both shapes:
* /ask (workers.dev direct URL)
* /api/staffml-interviewer/ask (custom route)
Solution: strip the "/api/staffml-interviewer" prefix at the top of
the fetch handler before any pathname matching runs. When called via
workers.dev, the prefix isn't present and the strip is a no-op.
One code path, two deployment shapes.
Wrangler config (wrangler.toml)
-------------------------------
* Added workers_dev = true to explicitly keep the workers.dev URL
as a fallback during the transition.
* Added two routes patterns (bare path + wildcard-child path — both
are needed because Cloudflare treats them as distinct):
mlsysbook.ai/api/staffml-interviewer
mlsysbook.ai/api/staffml-interviewer/*
Build env vars (.github/workflows/)
-----------------------------------
Added NEXT_PUBLIC_INTERVIEWER_ENDPOINT to the Build StaffML steps in
both workflows. This is a BUILD-TIME variable — Next.js inlines the
string into the JS bundle during npm run build, it is not read at
runtime. Both production (publish-live) and preview (preview-dev)
workflows now point at the same worker URL; the worker's rate
limiter handles contention between the two.
Smoke-tested end-to-end
-----------------------
* GET /api/staffml-interviewer/health → {ok:true, providers:[cf-workers-ai], waitlist:true}
* POST /api/staffml-interviewer/ask → Socratic Llama 3.1 8B clarification, ~50 words
* GET old workers.dev /health → still works (fallback path intact)
Two small follow-ups to the worker-hardening commit.
KV namespace IDs
----------------
Created the two KV namespaces on Cloudflare and pasted their real
IDs into wrangler.toml. Both namespaces were created fresh via
`wrangler kv namespace create` (no pre-existing instances to
reuse):
RATE_LIMIT_KV → a510cc7f791c40ffa28d19cf809c0e28
WAITLIST_KV → e42c3db4eaa2471f81f3f73745959349
The worker now deploys cleanly without any REPLACE_WITH_... stubs.
Wrangler v3 kv syntax
---------------------
Fixed WORKER_DEPLOY.md to use the modern `kv namespace` (space)
subcommand rather than the deprecated `kv:namespace` (colon) form
that Wrangler v3.60+ no longer accepts. Users copying the old
commands were hitting `Unknown arguments: kv:namespace, create,
RATE_LIMIT_KV` errors.
Updated locations:
- "Create the rate-limit KV namespace" step
- "Create the waitlist KV namespace" step
- "Reading the waitlist back" examples (kv key list / kv key get)
- "Tearing it down" section (kv namespace delete)
Also added an inline callout box near the first KV command
explaining the v2 → v3 syntax migration so future readers who hit
the error know why.
Two related hardening/feature changes on the Ask Interviewer Worker.
CORS allowlist default
----------------------
The previous default for ALLOWED_ORIGINS was "*", which meant any
deployment without the env var set was wide open. API keys are
server-side so this wasn't a credential leak, but it was a
gratuitous way for third parties to call the worker against their
visitors' IPs and exhaust our global rate-limit budget.
New DEFAULT_ALLOWED_ORIGINS constant restricts to:
- https://staffml.ai
- https://www.staffml.ai
- https://mlsysbook.ai
- http://localhost:3000
Operators can still override via the ALLOWED_ORIGINS env var for
preview deployments or broader policies.
/waitlist endpoint
------------------
New POST /waitlist handler wired up for the AskInterviewer waitlist
modal (previous commit). Captures the "would you pay for a paid
tier?" signal without any payments plumbing.
Body: { email, wouldPay (0-500), need? }
Rate: 1 submission / IP / hour, separate from the /ask limit
(reuses RATE_LIMIT_KV with a distinct `wl:hour:` prefix)
Store: WAITLIST_KV, key = `wl:${isoTimestamp}:${ipHash}`
value = JSON.stringify(record)
The handler:
* Returns 503 if WAITLIST_KV isn't bound (operators who haven't
provisioned it yet; the client falls back to mailto: in that
case so no interest is ever lost).
* Validates email with a cheap RFC-ish check.
* Clamps wouldPay to [0, 500] and rounds to int.
* Caps need at 1000 chars, request body at 4KB.
* Hashes IP with SHA-256(ip|todaysDate) truncated to 16 hex chars
— enough entropy to dedupe within a day, not enough to
fingerprint across days. Raw IPs never land in the record.
* Writes the slot-marker BEFORE writing the record so a race
can't slip two submissions through.
There's deliberately no admin read endpoint. Operators pull records
with `wrangler kv:key list --binding WAITLIST_KV`. Keeps the
worker's public attack surface minimal.
/health response shape
----------------------
/health now includes a `waitlist: boolean` field reflecting whether
the WAITLIST_KV binding is configured. The smoke-test section of
WORKER_DEPLOY.md is updated to match.
wrangler.toml
-------------
* New [[kv_namespaces]] block for WAITLIST_KV with a REPLACE_ME id.
* Commented-out ALLOWED_ORIGINS example showing the new default.
WORKER_DEPLOY.md
----------------
* New step 2b walking through WAITLIST_KV creation.
* Commands for reading the waitlist back via wrangler kv:key list.
* Note that skipping WAITLIST_KV is fine — the client falls back.
Tiny edge function (~430 lines, single file) that powers the Ask
Interviewer panel. Default provider is Cloudflare Workers AI
(Llama 3.1 8B) — always available on the free Workers plan, no API
key required. Adding any of five other providers is a one-line
wrangler secret command:
Groq → wrangler secret put GROQ_API_KEY (Llama 3.1 70B)
OpenAI → wrangler secret put OPENAI_API_KEY (GPT-4o mini)
Anthropic → wrangler secret put ANTHROPIC_API_KEY (Claude 3.5 Haiku)
Gemini → wrangler secret put GEMINI_API_KEY (Gemini 1.5 Flash)
OpenRouter → wrangler secret put OPENROUTER_API_KEY (any model)
The first available provider in the priority chain wins; falls back to
the next on error or rate limit. Each adapter is a config object with
vendorLabel, modelLabel, and privacyNote that the client renders as
inline attribution — so vendor sponsorship is automatic.
Architecture:
* Adapter pattern over 4 request shapes: openai-compat (covers OpenAI,
Groq, OpenRouter, Together, Fireworks, Ollama, LM Studio, vLLM),
anthropic, gemini, cf-workers-ai
* Server-side Socratic system prompt enforced — clients can't bypass
* KV-backed rate limiter (10/IP/hour, 60/IP/day, 8000/global/day,
all configurable via env vars)
* Configurable PROVIDER_PRIORITY env var for sponsor-first ordering
Hardening (Gemini security review):
* Per-turn history content length cap (1000 chars) to prevent OOM /
token waste from oversized payloads
* Request body Content-Length check (16 KB ceiling) before JSON parse
* AbortSignal.timeout(10000) on every upstream fetch via timedFetch()
* Promise.race timeout for the Cloudflare Workers AI binding
* Anthropic and Gemini adapters coalesce consecutive same-role messages
to satisfy strict alternation requirements (this would have caused a
guaranteed 400 on every multi-turn request without the fix)
* Prompt injection mitigation: client-supplied "interviewer" history
turns are stripped server-side; only user clarifications forwarded,
collapsed into a single context block (an attacker controlling the
request body cannot inject fake assistant turns)
* parseIntOrDefault helper guards rate-limit env vars against NaN
fail-open
* Content-Type enforcement (returns 415 if not application/json)
* safeJson<T>() helper handles malformed provider responses cleanly
Files:
* worker/wrangler.toml — bindings, KV namespace, observability
* worker/src/index.ts — the whole worker (one file)
* worker/package.json — wrangler + types
* worker/tsconfig.json — strict TS, workers-types
* worker/README.md — quick reference
* WORKER_DEPLOY.md — full deploy guide with provider upgrades
and vendor sponsorship instructions
Worker typechecks clean (exit 0).
