github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-16 06:07:17 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[PR #1933] fix(mlperf-edu): stop silently bypassing anti-cheat when mlperf CLI is missing #34657

New Issue
Open
opened 2026-07-14 19:27:47 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-07-14 19:27:47 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1933
Author: @Shashank-Tripathi-07
Created: 7/7/2026
Status: 🔄 Open

Base: dev ← Head: fix/grader-anticheat-silent-bypass

📝 Commits (1)

  • d1f0c16 fix(mlperf-edu): stop silently bypassing anti-cheat when mlperf CLI is missing

📊 Changes

1 file changed (+3 additions, -3 deletions)

View changed files

📝 mlperf-edu/scripts/autograder/grade_all.py (+3 -3)

📄 Description

Summary

  • GraderPipeline._process_zip() runs mlperf verify as the only anti-cheat check. If the mlperf CLI isn't on PATH (the common case for a bare grading checkout that hasn't pip install -e'd the package), the FileNotFoundError was caught and silently passed, and the pipeline fell straight through to parsing the submission's self-reported JSON metrics with "Cheating": "NO" unconditionally.
  • Concrete impact: a student can hand-edit their submission JSON to claim any accuracy/throughput they want, and it gets graded as VALIDATED with no verification at all.

Test plan

  • Confirmed mlperf is not on PATH in a clean grading environment (where mlperf fails).
  • Built a forged submission zip (achieved_accuracy: 0.99, throughput_qps: 99999) and ran the real _process_zip() against it before the fix: returned {'Status': 'VALIDATED', 'Score': 0.99, 'Cheating': 'NO'}.
  • After the fix, the same forged zip returns {'Status': 'VERIFY_UNAVAILABLE', 'Score': 0.0, 'Cheating': 'UNVERIFIED'} and logs the failure instead of silently grading it.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1933 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 7/7/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `fix/grader-anticheat-silent-bypass` --- ### 📝 Commits (1) - [`d1f0c16`](https://github.com/harvard-edge/cs249r_book/commit/d1f0c1685c1a9aff3f27dfdbd6cebdf9a26f6e64) fix(mlperf-edu): stop silently bypassing anti-cheat when mlperf CLI is missing ### 📊 Changes **1 file changed** (+3 additions, -3 deletions) <details> <summary>View changed files</summary> 📝 `mlperf-edu/scripts/autograder/grade_all.py` (+3 -3) </details> ### 📄 Description ## Summary - `GraderPipeline._process_zip()` runs `mlperf verify` as the only anti-cheat check. If the `mlperf` CLI isn't on PATH (the common case for a bare grading checkout that hasn't `pip install -e`'d the package), the `FileNotFoundError` was caught and silently `pass`ed, and the pipeline fell straight through to parsing the submission's self-reported JSON metrics with `"Cheating": "NO"` unconditionally. - Concrete impact: a student can hand-edit their submission JSON to claim any accuracy/throughput they want, and it gets graded as `VALIDATED` with no verification at all. ## Test plan - [x] Confirmed `mlperf` is not on PATH in a clean grading environment (`where mlperf` fails). - [x] Built a forged submission zip (`achieved_accuracy: 0.99`, `throughput_qps: 99999`) and ran the real `_process_zip()` against it before the fix: returned `{'Status': 'VALIDATED', 'Score': 0.99, 'Cheating': 'NO'}`. - [x] After the fix, the same forged zip returns `{'Status': 'VERIFY_UNAVAILABLE', 'Score': 0.0, 'Cheating': 'UNVERIFIED'}` and logs the failure instead of silently grading it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-07-14 19:27:47 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#34657
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?