Files
cs249r_book/interviews/vault-cli/tests/test_hashing.py
Vijay Janapa Reddi 42f4d1ca8b fix(vault): Round-3 correctness + vault ship + authoring contract
Round-3 review (4 reviewers on v2.1) surfaced two code-correctness
Criticals that this commit fixes, plus the contracted-but-missing
`vault ship` coordinator and David's authoring-UX gaps.

Critical fixes (real bugs in landed code):

worker/src/index.ts
- SCHEMA_FINGERPRINT placeholder fail-closed (Chip R3-C1 / Dean R3-NH-3).
  Was: placeholder auto-passed and silently disabled the fingerprint
  check. Now: placeholder forces degraded mode until operator sets
  real fingerprint.
- DDL hash now includes triggers (FTS5-aware).
- release_id change invalidates schema-fingerprint memoization
  (Dean R3-NH-4).
- wrangler.toml now pins the real fingerprint.

staffml/public/sw.js
- /manifest polling TTL-throttled to 5min (Chip R3-C2). Was:
  per-request fetch nullified the §10.4 cost model.
- API origin persisted to IndexedDB; rehydrated on activate so cold
  offline wake-ups serve cached content (Chip R3-H3).

vault-cli/src/vault_cli/release.py
- emit_migrations diffs all 4 tables via PRAGMA-driven column
  introspection (Dean R3-NC-1 + R3-NH-2). Was: only questions table,
  silently missing chains/chain_questions/tags. Rollback-symmetry
  test extended to populate + verify all tables.

vault-cli/src/vault_cli/commands/release.py
- vault verify --git-ref reconstructs release from 'git archive <ref>'
  into a tempdir (Dean R3-NC-2). Was: always rebuilt from HEAD, so
  verifying a historical release always failed post-authoring.
  Academic-citability contract (C-3) now actually holds.

vault-cli/src/vault_cli/ship.py (NEW)
- vault ship composed verb with journaling (Dean R3-NH-1):
  * Legs run D1 → Next.js → paper-tag-last (§6.1.1 ordering).
  * Journal at releases/<v>/.ship-journal.json records per-leg state;
    --resume continues interrupted ships idempotently.
  * Pre-paper failure auto-rolls back in reverse order.
  * Paper-leg failure pages operator; does NOT auto-rollback earlier
    legs (git tag is remote-durable per §6.1.1).
- 4 unit tests cover happy path, pre-paper failure auto-rollback,
  paper-leg needs-manual, --resume across interruptions.

vault-cli/src/vault_cli/commands/authoring.py
- vault new appends to id-registry.yaml (David R3-H3 + C-5
  enforcement); `git pull --rebase` before allocation.
- authors: auto-populated from git config user.email (David R3-H4 /
  M-15). Was: field never set.
- vault edit injects validation-error comment block at top of YAML
  and re-opens up to --retries=3 times (David R3-H1). Was: terminal
  traceback mid-authoring session.
- vault move refuses dirty tree, chained question, excluded-cell
  per applicability matrix (David R3-H2). Was: unchecked git mv.
- vault renumber command (NEW): post-rebase seq-collision recovery.
  Bumps seq, renames file, updates id field, appends registry
  (David R3-N-2, was spec-only).
- vault mark-exemplar command (NEW): promotes to vault/exemplars/
  with provenance + human_reviewed_at gate (David R3-N-9).

vault-cli/src/vault_cli/compiler.py
- FTS5 virtual table + sync triggers added to DDL (B.5). Triggers
  keep questions_fts in sync via AFTER INSERT/UPDATE/DELETE.
  schema_fingerprint accounts for triggers now.

tests/test_hashing.py
- Nested-dict hash-stability fixture (Soumith R3-F-4). Was: test
  only reordered top-level keys + collapsed details to one key.

All 28 tests pass (22 → 28: +4 ship journaling, +1 multi-table
migration symmetry, +1 nested-dict hash stability). release_hash
unchanged at 1b304282... — FTS5 addition doesn't affect content
Merkle per §3.5 input-only design.
2026-04-16 13:10:16 -04:00

124 lines
4.0 KiB
Python

"""Tests for the canonical hashing layer.
Key invariants:
- Same semantic content hashes identically regardless of YAML key order.
- Whitelist fields drive the hash; metadata doesn't.
- Merkle construction stable across re-ordering of leaves.
"""
from __future__ import annotations
from vault_cli.hashing import CANON_VERSION, content_hash, release_hash
def _base_question() -> dict:
return {
"id": "global-0000",
"title": "Example",
"topic": "kv-cache-management",
"chain": {"id": "global-chain-000", "position": 1},
"status": "published",
"scenario": "Explain KV-cache.",
"details": {"realistic_solution": "Paged attention."},
"tags": ["a", "b"],
"provenance": "human",
}
def test_content_hash_stable_across_key_reorder() -> None:
"""Soumith M-NEW-4 / R3-F-4: top-level AND nested-dict hashing must be
key-order-invariant.
The prior version of this test only reordered top-level keys and collapsed
``details`` to a single-key dict, so it didn't actually exercise the
nested-dict claim. Extended to also reorder ``details`` and ``chain``.
"""
q1 = _base_question()
# Reorder top-level keys AND nested dicts.
q2 = {k: q1[k] for k in reversed(list(q1.keys()))}
q2["details"] = {
"napkin_math": "X", # add a second nested key so the order matters
"common_mistake": "Y",
"realistic_solution": q1["details"]["realistic_solution"],
}
q2["chain"] = {"position": q1["chain"]["position"], "id": q1["chain"]["id"]}
# Mirror on q1 to keep the semantic payload identical.
q1 = dict(q1)
q1["details"] = dict(q2["details"])
# Now q1 and q2 share the same semantic fields; only key insertion order differs.
assert content_hash(q1) == content_hash(q2)
def test_content_hash_nested_dict_order_invariance() -> None:
"""Explicit nested-order test with a 3-key ``details`` — must hash identical
regardless of ``details`` key insertion order.
"""
base = _base_question()
base["details"] = {
"realistic_solution": "ANS",
"common_mistake": "WRONG",
"napkin_math": "2 + 2 = 4",
}
reordered = dict(base)
reordered["details"] = {
"napkin_math": "2 + 2 = 4",
"realistic_solution": "ANS",
"common_mistake": "WRONG",
}
assert content_hash(base) == content_hash(reordered)
def test_content_hash_excludes_metadata() -> None:
"""Hash must NOT change when last_modified or file_path changes."""
q1 = _base_question()
q2 = dict(q1)
q2["last_modified"] = "2050-01-01T00:00:00Z"
q2["file_path"] = "/tmp/foo.yaml"
q2["authors"] = ["someone"]
assert content_hash(q1) == content_hash(q2)
def test_content_hash_changes_with_semantic_edit() -> None:
"""Hash MUST change when scenario changes."""
q1 = _base_question()
q2 = dict(q1)
q2["scenario"] = "An edited scenario."
assert content_hash(q1) != content_hash(q2)
def test_release_hash_includes_canon_and_policy_leaves() -> None:
"""Chip N-H5: release_hash must bind canon version and policy."""
leaves = [("a", "1" * 64), ("b", "2" * 64)]
base = release_hash(
per_question=leaves,
taxonomy_hash="t" * 64,
chains_hash="c" * 64,
zones_hash="z" * 64,
policy_hash="p" * 64,
)
# Different policy_hash → different release_hash
different_policy = release_hash(
per_question=leaves,
taxonomy_hash="t" * 64,
chains_hash="c" * 64,
zones_hash="z" * 64,
policy_hash="P" * 64,
)
assert base != different_policy
# Different canon_version → different release_hash
different_canon = release_hash(
per_question=leaves,
taxonomy_hash="t" * 64,
chains_hash="c" * 64,
zones_hash="z" * 64,
policy_hash="p" * 64,
canon_version=999,
)
assert base != different_canon
def test_canon_version_is_pinned() -> None:
assert isinstance(CANON_VERSION, int)
assert CANON_VERSION >= 1