Files
cs249r_book/shared/scripts/build-redirects.py
Vijay Janapa Reddi 773e106c63 PR-5: Cutover skeletons (rollback-legacy + redirect map + sitemap aggregator) (#1409)
* feat(launch): rollback-legacy.sh — snapshot + restore the gh-pages root

Add the panic button for the mlsysbook.ai cutover. The staged-rollout
plan keeps the legacy single-volume site at the gh-pages root while
the new properties (Vol I, Vol II, TinyTorch, labs, kits, slides,
mlsysim, instructors, staffml, unified landing) get deployed into
subdirectories. Once everything is verified, the unified landing
page replaces the legacy root — and at exactly that moment we want a
one-command revert path that doesn't require remembering which gh-
pages SHA "the old root" lived at.

Three modes:
  snapshot          Take a timestamped backup of the legacy root files
                    (everything at gh-pages root that is NOT a known
                    subsite directory) and push to legacy-backup/<TS>/.
  restore <ID>      Copy a snapshot back to root, OVERWRITING current
                    root files but leaving subsite directories alone.
  list              List available snapshots.

Design choices worth flagging:

1. Subsite-aware. The script hard-codes the list of top-level
   subsite directories (book/, tinytorch/, kits/, labs/, mlsysim/,
   slides/, instructors/, interviews/, staffml/, about/, community/,
   newsletter/) and excludes them from BOTH snapshot capture AND
   restore overwrites. Rolling back the legacy landing page should
   never wipe out actively-deployed properties.

2. Dry-run by default. Every destructive mode requires --apply. The
   default behavior prints what would happen, including a diff
   preview for restore. This is the same posture the existing
   sync-mirrors.sh / link-checker / publish-guard scripts take.

3. Snapshots are kept, not moved. Restoring a snapshot is itself a
   reversible commit on gh-pages; the snapshot directory is preserved
   so a "rollback the rollback" is one more command away.

4. Doesn't touch the working tree. Operates against a fresh shallow
   clone in mktemp, so it can be run from any clone of the repo
   (developer machine or a GitHub Actions runner) without dirtying
   anything local.

Typical sequence on launch day is documented inline at the top of
the script. Two short commands wrap the whole rollout: snapshot
before deploy, restore-by-ID if anything looks wrong.

* feat(seo): redirect-map skeleton + HTML-stub generator

Add the cutover plumbing for legacy-URL → new-URL redirects so the
PageRank accumulated under the old single-volume mlsysbook.ai
structure flows into the new ecosystem URLs (`/book/vol1/`,
`/labs/`, `/about/`, etc.) as soon as the unified landing replaces
the legacy root.

Two artifacts:

1. `shared/config/redirect-map.json` — declarative source of truth.
   Schema:
     - `from`:   legacy path (must start with '/')
     - `to`:     destination URL or path (resolves against base_url)
     - `status`: 301 / 302 / 307 / 308 (default 301)
     - `note`:   optional human note
   A trailing-`*` wildcard is supported in `from` for whole-subtree
   moves like `/contents/labs/* → /labs/*`. The file ships
   intentionally small: just enough entries to demonstrate the
   patterns and seed the launch — populating the full inventory
   from the legacy mlsysbook.ai sitemap is a separate task.

2. `shared/scripts/build-redirects.py` — generator.
   For each entry it emits a tiny HTML stub at the legacy path
   containing:
     <meta http-equiv="refresh" content="0;url=<dest>">
     <link rel="canonical" href="<dest>">
     <meta name="robots" content="noindex,follow">
   That combo is the closest GitHub-Pages-friendly equivalent of a
   301: real users get redirected in <100ms; crawlers treat the
   canonical as authoritative and drop the legacy URL on recrawl;
   PageRank flows through. The script ALSO emits a Netlify-format
   `_redirects` file from the same map, so the day we move off
   GitHub Pages (Cloudflare Pages, Netlify, S3+CF) the same source
   of truth produces real 301s with no rewrite.

   `--check` mode validates the JSON without writing anything (CI
   hook). Wildcards skip stub emission (we'd have to walk the
   deployed tree to expand them) but are still emitted to the
   Netlify file where they work natively.

Wiring into a *-publish-live workflow is a one-liner step
(`python3 shared/scripts/build-redirects.py --map shared/config/
redirect-map.json --out gh-pages-staging/`) but is intentionally
NOT done in this commit — it should land alongside the actual
unified-landing deploy, when there is something for the legacy
URLs to redirect away from.

* feat(seo): aggregate per-subsite sitemaps into mlsysbook.ai/sitemap.xml

The new ecosystem has every subsite (Vol I, Vol II, TinyTorch, labs,
kits, slides, instructors, mlsysim, staffml, the unified landing)
emitting its own `<subsite>/sitemap.xml` because that's what Quarto
and Next produce automatically. Search engines, however, want a
single authoritative entry point per *domain*. Without an aggregated
index they end up either crawling the subsite sitemaps separately
(if they happen to discover them) or missing some entirely.

This commit adds the aggregator:

  shared/scripts/build-sitemap.py
    Walks a deployed gh-pages tree, discovers every sitemap.xml under
    it (skipping the root one, legacy-backup snapshots, _archive,
    _site, and the like), and writes a single sitemap-index.xml at
    `<root>/sitemap.xml` that points at each subsite's sitemap as a
    `<sitemap><loc>…</loc></sitemap>` entry. It also creates or
    appends to `<root>/robots.txt` so the index is surfaced to
    crawlers via the standard `Sitemap:` directive.

    Optional `--include-subsite` allowlist (repeatable) for staged
    rollouts where we want the index to advertise only the subsites
    that have been verified live, even if other ones happen to be
    deployed in the tree. Defaults to "everything found".

    `--check` does discovery without writing.

  .github/workflows/infra-build-sitemap.yml
    Reusable workflow (`workflow_call`) wrapping the script so any
    `*-publish-live` workflow can refresh the index as its final
    step. Also `workflow_dispatch`-able for manual rebuilds. Joins
    the existing `gh-pages-deploy` concurrency group so it never
    races a publish push.

    Uses sparse-checkout to grab just the script from `dev` (no need
    to clone the whole monorepo into the runner) and a full clone of
    `gh-pages` to do the work.

Wiring into per-subsite publish workflows happens in a follow-up
commit alongside the actual launch — this PR is "skeletons", and
the per-publish trigger is best landed when each subsite's launch
PR ships.
2026-04-19 16:22:11 -04:00

205 lines
6.5 KiB
Python
Executable File

#!/usr/bin/env python3
"""Generate redirect HTML stubs (and a Netlify _redirects file) from the
shared redirect-map.
Why this script exists
----------------------
GitHub Pages doesn't honor server-side redirects. To preserve SEO juice
from the legacy mlsysbook.ai URLs after the staged rollout, we emit one
tiny HTML file per legacy path:
<meta http-equiv="refresh" content="0;url=<to>">
<link rel="canonical" href="<to>">
<meta name="robots" content="noindex,follow">
Crawlers treat the canonical as authoritative, drop the legacy URL on
recrawl (the noindex), and follow the link graph through to the new
location. Real users hit the meta-refresh and arrive in <100ms.
The same map ALSO produces a Netlify-format `_redirects` file so that if
we ever move off GitHub Pages to a host that supports real 301s, the
existing redirect map drives that day-one without a second source of
truth.
Usage
-----
build-redirects.py --map shared/config/redirect-map.json \
--out gh-pages-staging/ \
[--base-url https://mlsysbook.ai] \
[--check]
--check Validates the JSON without writing any files (CI-friendly).
"""
from __future__ import annotations
import argparse
import json
import sys
from pathlib import Path
from typing import Any
DEFAULT_BASE_URL = "https://mlsysbook.ai"
STUB_TEMPLATE = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="refresh" content="0;url={dest}">
<link rel="canonical" href="{canonical}">
<meta name="robots" content="noindex,follow">
<title>Redirecting…</title>
</head>
<body>
<p>This page has moved to <a href="{dest}">{dest}</a>.</p>
</body>
</html>
"""
def resolve_dest(to: str, base_url: str) -> str:
"""Return an absolute URL for `to`. If `to` is already absolute, pass
through. If it's a path, resolve against base_url."""
if to.startswith(("http://", "https://")):
return to
if not to.startswith("/"):
to = "/" + to
return base_url.rstrip("/") + to
def validate_entry(i: int, entry: dict[str, Any]) -> list[str]:
"""Return a list of validation errors for one entry. Empty list = OK."""
errs: list[str] = []
where = f"redirects[{i}]"
for required in ("from", "to"):
if required not in entry:
errs.append(f"{where}: missing required field '{required}'")
src = entry.get("from", "")
if src and not src.startswith("/"):
errs.append(f"{where}: 'from' must start with '/' (got {src!r})")
status = entry.get("status", 301)
if status not in (301, 302, 307, 308):
errs.append(f"{where}: 'status' should be 301/302/307/308 (got {status!r})")
# Wildcard handling: only allowed as a final '*' segment for now.
if "*" in src and not src.endswith("/*"):
errs.append(
f"{where}: wildcard '*' currently only supported as the trailing "
f"path segment (e.g. '/foo/*'); got {src!r}"
)
return errs
def write_html_stub(out_root: Path, src: str, dest_url: str) -> Path:
"""Materialize the redirect at out_root/<src>/index.html (or .html
file if `src` already names a `.html`)."""
rel = src.lstrip("/")
if rel.endswith(".html") or rel.endswith(".htm"):
target = out_root / rel
elif rel == "" or rel.endswith("/"):
target = out_root / rel / "index.html"
else:
# path with no extension → emit as both /<rel>/index.html
target = out_root / rel / "index.html"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(
STUB_TEMPLATE.format(dest=dest_url, canonical=dest_url),
encoding="utf-8",
)
return target
def write_netlify_file(out_root: Path, lines: list[str]) -> Path:
"""Emit a Netlify-compatible `_redirects` file alongside the stubs."""
target = out_root / "_redirects"
target.write_text(
"# Generated by shared/scripts/build-redirects.py — do not edit by hand.\n"
"# Source of truth: shared/config/redirect-map.json\n"
+ "\n".join(lines)
+ "\n",
encoding="utf-8",
)
return target
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument("--map", required=True, help="Path to redirect-map.json")
ap.add_argument(
"--out",
required=False,
help="Directory to emit redirect stubs into (typically the staging "
"copy of gh-pages). Required unless --check is passed.",
)
ap.add_argument(
"--base-url",
default=DEFAULT_BASE_URL,
help=f"Base URL for relative 'to' values (default: {DEFAULT_BASE_URL})",
)
ap.add_argument(
"--check",
action="store_true",
help="Validate the map only; do not write files.",
)
args = ap.parse_args()
if not args.check and not args.out:
ap.error("--out is required unless --check is passed")
map_path = Path(args.map)
try:
data = json.loads(map_path.read_text(encoding="utf-8"))
except (FileNotFoundError, json.JSONDecodeError) as e:
print(f"❌ Could not load {map_path}: {e}", file=sys.stderr)
return 2
redirects = data.get("redirects", [])
if not isinstance(redirects, list):
print("'redirects' must be a list", file=sys.stderr)
return 2
errors: list[str] = []
for i, entry in enumerate(redirects):
errors.extend(validate_entry(i, entry))
if errors:
print("❌ Validation errors:", file=sys.stderr)
for e in errors:
print(f" - {e}", file=sys.stderr)
return 1
print(f"✅ Validated {len(redirects)} redirect entries from {map_path}")
if args.check:
return 0
out_root = Path(args.out)
out_root.mkdir(parents=True, exist_ok=True)
netlify_lines: list[str] = []
written = 0
for entry in redirects:
src = entry["from"]
dest = resolve_dest(entry["to"], args.base_url)
status = entry.get("status", 301)
netlify_lines.append(f"{src} {dest} {status}")
if "*" in src:
# We cannot statically expand wildcards — that requires walking
# the deployed tree. Skip stub emission and rely on the Netlify
# _redirects line for hosts that support it.
continue
target = write_html_stub(out_root, src, dest)
written += 1
print(f"{src}{dest} ({target.relative_to(out_root)})")
write_netlify_file(out_root, netlify_lines)
print(f"✅ Wrote {written} HTML stub(s) and 1 _redirects file to {out_root}")
return 0
if __name__ == "__main__":
raise SystemExit(main())