mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-05-08 18:01:20 -05:00
0afc384282
Two new pieces close the generation→validation→saturation feedback loop: 1. gemini_cli_llm_judge.py — multi-criteria validator. For each draft, judges math correctness, cell-fit (does it actually target the declared track/zone/level?), scenario realism, uniqueness vs canonical questions, and visual-asset alignment. Returns PASS/NEEDS_FIX/DROP per item. Batched (default 15 per call) for budget efficiency. 2. iterate_coverage_loop.py — drives the full loop: analyze → plan → generate → render → judge → apply → re-analyze. Self-paced: stops when (a) top priority gap drops below threshold, (b) DROP rate exceeds the saturation/hallucination threshold, (c) total API calls exceed budget, or (d) the same cell is top priority for two iterations in a row (convergence). The user no longer specifies "how many questions" — the loop generates until the corpus reaches a measurable steady state. Plus 25 round-1 visual questions generated by the new batched generator (5 batched calls × 5 cells each, zero failures). The loop is the answer to "we need balance, not just volume": every iteration's plan derives from a fresh analysis of where coverage is weakest, so generation can never over-fill an already-saturated cell.
18 lines
771 B
Python
18 lines
771 B
Python
import os
|
|
import matplotlib.pyplot as plt
|
|
import numpy as np
|
|
mu = 150
|
|
lambda_vals = np.linspace(0, 140, 100)
|
|
latency = 1 / (mu - lambda_vals) * 1000
|
|
fig, ax = plt.subplots(figsize=(6, 4))
|
|
ax.plot(lambda_vals, latency, color='#4a90c4', linewidth=2)
|
|
ax.scatter([50, 125], [10, 40], color='#c87b2a', zorder=5)
|
|
ax.annotate('10ms @ 50 req/s', (50, 10), xytext=(20, 10), textcoords='offset points')
|
|
ax.annotate('40ms @ 125 req/s', (125, 40), xytext=(-90, 10), textcoords='offset points')
|
|
ax.set_xlabel('Arrival Rate (req/sec)')
|
|
ax.set_ylabel('Average Latency (ms)')
|
|
ax.set_title('M/M/1 Queue Latency Hockey-Stick')
|
|
ax.grid(True, linestyle='--', alpha=0.6)
|
|
plt.tight_layout()
|
|
out = os.environ.get("VISUAL_OUT_PATH", "out.svg")
|
|
plt.savefig(out, format="svg", bbox_inches="tight") |