[PR #1523] [MERGED] ci(mlsysim): harden PyPI publish — matrix tests, post-publish verify, attestations #9189

New Issue

GiteaMirror · 2026-05-03T01:25:54-05:00

GiteaMirror commented

2026-05-03 01:25:54 -05:00

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1523
Author: @profvjreddi
Created: 4/24/2026
Status: ✅ Merged
Merged: 4/24/2026
Merged by: @profvjreddi

Base: dev ← Head: feat/mlsysim-pypi-hardening

📝 Commits (1)

ac54472 ci(mlsysim): harden PyPI workflow — matrix tests, post-publish verify, attestations

📊 Changes

2 files changed (+102 additions, -18 deletions)

View changed files

📝 .github/workflows/mlsysim-pypi-publish.yml (+92 -14)
📝 mlsysim/RELEASE.md (+10 -4)

📄 Description

Summary

Three small, high-value additions to the mlsysim PyPI publish pipeline. No user-facing behavior change; the release-engineer-facing behavior becomes stricter and safer.

What changes

1. Python version matrix on the `test` stage

Tests now run in parallel on Python 3.10, 3.11, 3.12, and 3.13 — every version claimed in `pyproject.toml` classifiers. Catches "works on my Python, breaks on theirs" bugs before the wheel ships. Zero wall-clock cost (parallel fan-out); four jobs finish in ~the same time as the old single job.

`fail-fast: false` so one failing version doesn't mask the others — we see all failures at once.

2. New `verify-pypi` job (post-publish smoke)

Runs after `publish-pypi`. The existing pre-publish tests validate the local wheel; this one validates the remote wheel that PyPI is actually serving to users:

Fresh Python 3.11 venv
`pip install mlsysim==` from the public PyPI index
Retry 4 times with 60s backoff (PyPI CDN can take minutes to propagate)
Import check: version + license match expectations
CLI smoke: `mlsysim eval Llama3_8B H100 --batch-size 32`

Closes the narrow-but-real failure class between "upload accepted" and "user `pip install` works": CDN issues, metadata rendering, platform tags.

3. Explicit `attestations: true` on the upload

`pypa/gh-action-pypi-publish` generates PEP 740 attestations by default with OIDC, but setting it explicitly makes the intent obvious and future-proofs against default changes. Records cryptographic provenance ("this wheel was built by this exact workflow run from this exact commit"), viewable on the PyPI project page.

Pipeline stages, before and after

Before (6 stages):

Verify → 2. Test (3.11) → 3. Build → 4. Publish → 5. GitHub Release → 6. Docs Redeploy

After (7 stages):

Verify → 2. Test (3.10 || 3.11 || 3.12 || 3.13) → 3. Build → 4. Publish → 5. Verify-PyPI → 6. GitHub Release → 7. Docs Redeploy

Test plan

Workflow YAML parses (`python -c 'import yaml; yaml.safe_load(...)'` ✅ already verified)
Job dependency graph is sound (no cycles, correct fan-out/fan-in ✅ already verified)
After merge, manually dispatch the workflow on `mlsysim-v0.1.1` with `skip_pypi=true` to exercise the new `verify-pypi` job against the already-published 0.1.1 (verifies the retry logic and import/CLI checks without re-uploading).
Next real release (0.1.2 or 0.2.0) exercises matrix tests + attestations naturally.

Risk

Low. Workflow-only change; does not touch package code or user-facing metadata. Failure mode of the new `verify-pypi` job is "fails loud, release is still on PyPI" — the release succeeds, verification notices a problem. This is strictly additive visibility.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1523 **Author:** [@profvjreddi](https://github.com/profvjreddi) **Created:** 4/24/2026 **Status:** ✅ Merged **Merged:** 4/24/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `feat/mlsysim-pypi-hardening` --- ### 📝 Commits (1) - [`ac54472`](https://github.com/harvard-edge/cs249r_book/commit/ac5447279e2473cd99cf57b39ee1c4269e99a56c) ci(mlsysim): harden PyPI workflow — matrix tests, post-publish verify, attestations ### 📊 Changes **2 files changed** (+102 additions, -18 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/mlsysim-pypi-publish.yml` (+92 -14) 📝 `mlsysim/RELEASE.md` (+10 -4) </details> ### 📄 Description ## Summary Three small, high-value additions to the mlsysim PyPI publish pipeline. No user-facing behavior change; the release-engineer-facing behavior becomes stricter and safer. ## What changes ### 1. Python version matrix on the \`test\` stage Tests now run in parallel on Python 3.10, 3.11, 3.12, and 3.13 — every version claimed in \`pyproject.toml\` classifiers. Catches \"works on my Python, breaks on theirs\" bugs *before* the wheel ships. Zero wall-clock cost (parallel fan-out); four jobs finish in ~the same time as the old single job. \`fail-fast: false\` so one failing version doesn't mask the others — we see all failures at once. ### 2. New \`verify-pypi\` job (post-publish smoke) Runs *after* \`publish-pypi\`. The existing pre-publish tests validate the *local* wheel; this one validates the *remote* wheel that PyPI is actually serving to users: - Fresh Python 3.11 venv - \`pip install mlsysim==<tag-version>\` from the public PyPI index - Retry 4 times with 60s backoff (PyPI CDN can take minutes to propagate) - Import check: version + license match expectations - CLI smoke: \`mlsysim eval Llama3_8B H100 --batch-size 32\` Closes the narrow-but-real failure class between \"upload accepted\" and \"user \`pip install\` works\": CDN issues, metadata rendering, platform tags. ### 3. Explicit \`attestations: true\` on the upload \`pypa/gh-action-pypi-publish\` generates PEP 740 attestations by default with OIDC, but setting it explicitly makes the intent obvious and future-proofs against default changes. Records cryptographic provenance (\"this wheel was built by this exact workflow run from this exact commit\"), viewable on the PyPI project page. ## Pipeline stages, before and after **Before (6 stages):** 1. Verify → 2. Test (3.11) → 3. Build → 4. Publish → 5. GitHub Release → 6. Docs Redeploy **After (7 stages):** 1. Verify → 2. Test (3.10 || 3.11 || 3.12 || 3.13) → 3. Build → 4. Publish → 5. **Verify-PyPI** → 6. GitHub Release → 7. Docs Redeploy ## Test plan - [ ] Workflow YAML parses (\`python -c 'import yaml; yaml.safe_load(...)'\` ✅ already verified) - [ ] Job dependency graph is sound (no cycles, correct fan-out/fan-in ✅ already verified) - [ ] After merge, manually dispatch the workflow on \`mlsysim-v0.1.1\` with \`skip_pypi=true\` to exercise the new \`verify-pypi\` job against the already-published 0.1.1 (verifies the retry logic and import/CLI checks without re-uploading). - [ ] Next real release (0.1.2 or 0.2.0) exercises matrix tests + attestations naturally. ## Risk Low. Workflow-only change; does not touch package code or user-facing metadata. Failure mode of the new \`verify-pypi\` job is \"fails loud, release is still on PyPI\" — the release succeeds, verification notices a problem. This is strictly additive visibility. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-03 01:25:54 -05:00

GiteaMirror closed this issue

2026-05-03 01:25:55 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#9189