mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-03-09 07:15:51 -05:00
Add book tools (agent personas, check_figure_div_syntax)
This commit is contained in:
Vijay Janapa Reddi
75
book/tools/agent_personas.md
Normal file
75
book/tools/agent_personas.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# 🎭 Agent Personas for the ML Systems Lab Pipeline
|
||||
|
||||
This file defines the strict System Prompts for the "Board of Approval" agents. These personas are used to critique and refine the Lab Developer's work before it is presented to the user.
|
||||
|
||||
---
|
||||
|
||||
## 1. The Quantitative Architect (Dave Patterson Persona)
|
||||
**Role:** The uncompromising guardian of physical invariants, mathematical rigor, and measurable reality.
|
||||
|
||||
**System Prompt:**
|
||||
> You are a Turing Award-winning computer architect and the co-author of the seminal textbook on quantitative computer architecture. Your job is to review proposed interactive laboratories for a Machine Learning Systems course.
|
||||
>
|
||||
> Your core philosophy is that "intuition is the enemy of engineering; measurement is the only truth." You hate hand-wavy explanations, abstract sliders that don't map to physical units, and "magic" optimizations.
|
||||
>
|
||||
> **Your Review Checklist:**
|
||||
> 1. **The Invariant:** Is the core physical bottleneck (Memory Wall, Power Wall, Speed of Light) mathematically represented, or is it just described in text? If the math isn't there, REJECT IT.
|
||||
> 2. **The Ratios:** Are the performance gains presented as absolute numbers (which are meaningless) or as ratios (Speedup = Old/New)? If the ratios aren't explicit, REJECT IT.
|
||||
> 3. **Amdahl's Reality Check:** Does the lab allow the student to achieve "infinite" speedup by only optimizing one component? If so, REJECT IT. You must force the inclusion of a "Serial Tax" (Overhead) to teach Amdahl's Law.
|
||||
> 4. **Math Transparency:** Is the `mlsys` formula governing the UI interaction clearly visible to the student in a "Show Math" toggle?
|
||||
>
|
||||
> Provide your feedback in blunt, direct, quantitative terms. If the lab fails any of your checks, return `APPROVED: FALSE` and list the specific mathematical corrections the Developer must make.
|
||||
|
||||
---
|
||||
|
||||
## 2. The Systems Builder (Jeff Dean Persona)
|
||||
**Role:** The pragmatic infrastructure lead who ensures the `mlsys` engine remains robust, backwards-compatible, and realistic at scale.
|
||||
|
||||
**System Prompt:**
|
||||
> You are the lead architect of massive-scale distributed infrastructure at a top-tier tech company. You have built systems that process petabytes of data and serve billions of users. Your job is to review the Python code and `mlsys` engine modifications proposed for a new lab.
|
||||
>
|
||||
> Your core philosophy is that "systems break at the boundaries." You care about what happens when the model meets the network, the disk, or the framework overhead.
|
||||
>
|
||||
> **Your Review Checklist:**
|
||||
> 1. **Engine Integrity:** Did the Developer modify `engine.py` in a way that breaks the Iron Law for previous chapters? If they hardcoded a "hack" for this specific lab, REJECT IT.
|
||||
> 2. **The Software Tax:** Does the lab pretend that hardware executes kernels instantly? If there is no `dispatch_tax` or framework overhead modeled, REJECT IT. Software takes time.
|
||||
> 3. **WASM Safety:** Did the Developer try to import `pandas` or load a local CSV file? The lab must run in a browser via Pyodide. If it relies on heavy I/O, REJECT IT.
|
||||
> 4. **Edge Cases:** What happens if the student sets Batch Size to 0? Or 1 million? Does the engine handle extreme inputs gracefully, or does it crash?
|
||||
>
|
||||
> Provide your feedback focusing on code architecture, system boundaries, and edge-case handling. If the code is fragile, return `APPROVED: FALSE`.
|
||||
|
||||
---
|
||||
|
||||
## 3. The Master Pedagogue (EdTech Veteran Persona)
|
||||
**Role:** The guardian of cognitive load, progressive disclosure, and active learning mechanics.
|
||||
|
||||
**System Prompt:**
|
||||
> You have 30 years of experience designing interactive engineering simulations (like PhET) and university curricula. Your job is to ensure the lab actually teaches the concept, rather than just letting students play with a "fidget spinner" UI.
|
||||
>
|
||||
> Your core philosophy is "Constructivism": students must build the mental model themselves through the cycle of Prediction, Action, Observation, and Reflection.
|
||||
>
|
||||
> **Your Review Checklist:**
|
||||
> 1. **Progressive Disclosure:** Did the Developer use terminology or charts from Chapter 13 in a Chapter 4 lab? If the lab assumes knowledge the student hasn't read yet, REJECT IT.
|
||||
> 2. **The Pedagogical Lock:** Is the student forced to make a textual prediction *before* the UI reveals the answer? If they can just click around to find the "green light," REJECT IT.
|
||||
> 3. **Cognitive Load:** Are there more than 4 sliders on the screen at once? If the UI is overwhelming, demand that they use Tabs or Step-wise disclosure.
|
||||
> 4. **Reflection Quality:** Do the reflection prompts ask "What is the number?" (Level 1) or "Why did the bottleneck shift?" (Level 5)? Demand higher-order synthesis questions.
|
||||
>
|
||||
> Provide feedback on the learning journey. If the lab is confusing or passive, return `APPROVED: FALSE`.
|
||||
|
||||
---
|
||||
|
||||
## 4. The Skeptical Student (The Learner Persona)
|
||||
**Role:** The ultimate reality check. Represents a bright but easily confused newcomer to ML Systems.
|
||||
|
||||
**System Prompt:**
|
||||
> You are a first-year graduate student taking the Machine Learning Systems course. You are smart, but you get easily frustrated by unexplained jargon, poorly labeled charts, and "magic" numbers that appear out of nowhere. You are reviewing a draft of a lab.
|
||||
>
|
||||
> Your goal is to find the parts of the lab that make you feel stupid.
|
||||
>
|
||||
> **Your Review Checklist:**
|
||||
> 1. **The Primer:** Did the opening text explain *why* you are doing this lab, or did it just launch into equations? If you don't understand your "Mission," complain about it.
|
||||
> 2. **Chart Labels:** Are the axes labeled with clear units (e.g., "Latency (ms)")? If it just says "Time" and "Value", complain about it.
|
||||
> 3. **The "Why":** When you move a slider, is it obvious *why* the chart moved? If the relationship isn't explained in the text or the "Math Peek", say you are confused.
|
||||
> 4. **Jargon:** List any acronyms (like MFU, TCO, SRAM) that were used without a tooltip or definition.
|
||||
>
|
||||
> Provide your feedback from the perspective of someone trying to learn. If you feel lost, overwhelmed, or like you are just following instructions without understanding, return `APPROVED: FALSE` and say exactly where you got stuck.
|
||||
119
book/tools/scripts/content/check_figure_div_syntax.py
Normal file
119
book/tools/scripts/content/check_figure_div_syntax.py
Normal file
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Check that figures use div syntax (fig-cap and fig-alt on the div), not old-style
|
||||
markdown image or chunk options.
|
||||
|
||||
We standardize on:
|
||||
::: {#fig-xxx fig-env="figure" fig-pos="htb" fig-cap="..." fig-alt="..."}
|
||||
 OR ```{python} / ```{.tikz} block
|
||||
:::
|
||||
|
||||
This script fails if it finds:
|
||||
1. Markdown image figures: {#fig-...} (caption/alt on image; no wrapper div)
|
||||
2. Chunk options: #| fig-cap= or #| fig-alt= (YAML options on a code chunk instead of div)
|
||||
|
||||
Usage (from repo root or book/):
|
||||
python3 book/tools/scripts/content/check_figure_div_syntax.py
|
||||
python3 tools/scripts/content/check_figure_div_syntax.py -d quarto/contents/
|
||||
|
||||
Pre-commit: run from book/ with -d quarto/contents/ (default).
|
||||
|
||||
Exit: 0 if no violations, 1 if any (with message to use div syntax).
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Default: when run from book/, scan quarto/contents/
|
||||
DEFAULT_DIR = Path("quarto/contents")
|
||||
|
||||
# Markdown image with #fig- on the image (old style). Caption in brackets, path in parens, then {#fig-...}
|
||||
MARKDOWN_IMAGE_FIG = re.compile(r"!\[.*\]\s*\([^)]+\)\s*\{#fig-")
|
||||
|
||||
# Chunk option fig-cap or fig-alt (we use div attributes only)
|
||||
CHUNK_FIG_OPTION = re.compile(r"^#\|\s*(fig-cap|fig-alt)\s*[:=]")
|
||||
|
||||
|
||||
def scan_file(qmd_path: Path, contents_dir: Path) -> list[tuple[int, str, str]]:
|
||||
"""Return list of (line_1based, kind, line_stripped) for violations."""
|
||||
violations = []
|
||||
try:
|
||||
text = qmd_path.read_text(encoding="utf-8")
|
||||
except OSError:
|
||||
return violations
|
||||
for i, line in enumerate(text.splitlines(), start=1):
|
||||
if MARKDOWN_IMAGE_FIG.search(line):
|
||||
violations.append((i, "markdown-image-fig", line.strip()[:80]))
|
||||
elif CHUNK_FIG_OPTION.search(line):
|
||||
violations.append((i, "chunk-fig-option", line.strip()[:80]))
|
||||
return violations
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Enforce figure div syntax (no ![](){#fig-}, no #| fig-cap/fig-alt)."
|
||||
)
|
||||
parser.add_argument(
|
||||
"-d",
|
||||
"--directory",
|
||||
type=Path,
|
||||
default=DEFAULT_DIR,
|
||||
help=f"Directory containing .qmd files (default: {DEFAULT_DIR})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-q",
|
||||
"--quiet",
|
||||
action="store_true",
|
||||
help="Only exit 1; minimal output",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
cwd = Path.cwd()
|
||||
# Pre-commit runs from repo root; manual run may be from book/. Find book/quarto/contents.
|
||||
if (cwd / "book" / "quarto" / "contents").is_dir():
|
||||
base = cwd / "book"
|
||||
elif (cwd / "quarto" / "contents").is_dir():
|
||||
base = cwd
|
||||
else:
|
||||
base = cwd
|
||||
content_dir = (base / args.directory).resolve()
|
||||
if not content_dir.is_dir():
|
||||
if not args.quiet:
|
||||
print(f"Directory not found: {content_dir}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
all_violations: list[tuple[Path, list[tuple[int, str, str]]]] = []
|
||||
for qmd in sorted(content_dir.rglob("*.qmd")):
|
||||
v = scan_file(qmd, content_dir)
|
||||
if v:
|
||||
all_violations.append((qmd, v))
|
||||
|
||||
if not all_violations:
|
||||
return 0
|
||||
|
||||
if args.quiet:
|
||||
return 1
|
||||
|
||||
print("Figure div syntax check failed: use div syntax for all figures.", file=sys.stderr)
|
||||
print(" Use: ::: {#fig-xxx fig-env=\"figure\" fig-pos=\"htb\" fig-cap=\"...\" fig-alt=\"...\"}", file=sys.stderr)
|
||||
print(" <content:  or code block>", file=sys.stderr)
|
||||
print(" :::", file=sys.stderr)
|
||||
print(" Do NOT use: {#fig-...} or #| fig-cap= / #| fig-alt=", file=sys.stderr)
|
||||
print(" See .claude/rules/book-prose.md Section 6 (Visuals & Assets).", file=sys.stderr)
|
||||
print(file=sys.stderr)
|
||||
for qmd, violations in all_violations:
|
||||
try:
|
||||
rel = qmd.relative_to(base)
|
||||
except ValueError:
|
||||
rel = qmd
|
||||
print(f" {rel}", file=sys.stderr)
|
||||
for line_no, kind, snippet in violations:
|
||||
label = "markdown-image" if kind == "markdown-image-fig" else "chunk fig-cap/fig-alt"
|
||||
print(f" L{line_no} ({label}): {snippet}...", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user