github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-15 21:28:33 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[PR #1934] [MERGED] fix(mlperf-edu): CSV leaderboard export crashes on mixed grade result shapes #32198

New Issue
Closed
opened 2026-07-12 08:51:31 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-07-12 08:51:31 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1934
Author: @Shashank-Tripathi-07
Created: 7/7/2026
Status: ✅ Merged
Merged: 7/10/2026
Merged by: @profvjreddi

Base: dev ← Head: fix/grader-csv-fieldnames-crash

📝 Commits (1)

  • 3cefb94 fix(mlperf-edu): CSV leaderboard export crashes on mixed grade result shapes

📊 Changes

1 file changed (+6 additions, -2 deletions)

View changed files

📝 mlperf-edu/scripts/autograder/grade_all.py (+6 -2)

📄 Description

Summary

  • _export_leaderboard() derived csv.DictWriter fieldnames from grades[0].keys() alone. Non-validated results (CORRUPT_ZIP, NO_PAYLOAD, JSON_ERROR) have fewer keys than a VALIDATED result (missing Throughput), so whenever a non-validated submission happened to sort first from glob.glob() (filesystem-order dependent, not guaranteed alphabetical), writing a later VALIDATED row raised ValueError: dict contains fields not in fieldnames, crashing the export for the entire batch.

Test plan

  • Reproduced: called the real _export_leaderboard() with a CORRUPT_ZIP result first and a VALIDATED result second -- confirmed ValueError: dict contains fields not in fieldnames: 'Throughput'.
  • After the fix (fieldnames built from the union of keys across all grades, first-seen order preserved): export succeeds, CSV has all columns with blanks for missing fields on rows that lack them.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1934 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 7/7/2026 **Status:** ✅ Merged **Merged:** 7/10/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `fix/grader-csv-fieldnames-crash` --- ### 📝 Commits (1) - [`3cefb94`](https://github.com/harvard-edge/cs249r_book/commit/3cefb948b5e766867aaa3a1f45097598fa4dbaa8) fix(mlperf-edu): CSV leaderboard export crashes on mixed grade result shapes ### 📊 Changes **1 file changed** (+6 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `mlperf-edu/scripts/autograder/grade_all.py` (+6 -2) </details> ### 📄 Description ## Summary - `_export_leaderboard()` derived `csv.DictWriter` fieldnames from `grades[0].keys()` alone. Non-validated results (`CORRUPT_ZIP`, `NO_PAYLOAD`, `JSON_ERROR`) have fewer keys than a `VALIDATED` result (missing `Throughput`), so whenever a non-validated submission happened to sort first from `glob.glob()` (filesystem-order dependent, not guaranteed alphabetical), writing a later `VALIDATED` row raised `ValueError: dict contains fields not in fieldnames`, crashing the export for the entire batch. ## Test plan - [x] Reproduced: called the real `_export_leaderboard()` with a `CORRUPT_ZIP` result first and a `VALIDATED` result second -- confirmed `ValueError: dict contains fields not in fieldnames: 'Throughput'`. - [x] After the fix (fieldnames built from the union of keys across all grades, first-seen order preserved): export succeeds, CSV has all columns with blanks for missing fields on rows that lack them. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-07-12 08:51:31 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#32198
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?