github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-16 06:07:17 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[PR #1940] [MERGED] fix(mlperf-edu): dataset download race on shared fixed temp filename #36550

New Issue
Closed
opened 2026-07-16 00:27:48 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-07-16 00:27:48 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1940
Author: @Shashank-Tripathi-07
Created: 7/7/2026
Status: ✅ Merged
Merged: 7/10/2026
Merged by: @profvjreddi

Base: dev ← Head: fix/assets-download-race-condition

📝 Commits (1)

  • e0e43e6 fix(mlperf-edu): dataset download race on shared fixed temp filename

📊 Changes

1 file changed (+24 additions, -17 deletions)

View changed files

📝 mlperf-edu/src/mlperf/assets.py (+24 -17)

📄 Description

Summary

  • _download() wrote directly to the caller-provided destination -- a fixed .download temp path with no PID/UUID component (tmp = raw.with_suffix(".download"), used identically by ensure_tinyshakespeare, ensure_cifar100, ensure_movielens_100k).
  • Two concurrent fetches of the same asset (plausible since multiple workload training runs each call the dataset factory on first use) both urlretrieve/curl to the identical path with independent file handles -- a slower writer's in-progress download could be clobbered mid-write by the other process before either side renamed to the final path.

Fix

Download to a process- and call-unique temp file first (destination.download.<pid>.<uuid8>.part), then atomically replace the destination. Concurrent callers now only race on the final atomic rename (last-writer-wins, both writes are complete valid downloads), never on the write itself.

Test plan

  • python -m py_compile passes on the edited file.
  • Simulated two "concurrent" downloads targeting the same destination with the new unique-temp-then-atomic-replace pattern: final content is a complete, uncorrupted write from whichever finished last, and no leftover .part temp files remain.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1940 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 7/7/2026 **Status:** ✅ Merged **Merged:** 7/10/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `fix/assets-download-race-condition` --- ### 📝 Commits (1) - [`e0e43e6`](https://github.com/harvard-edge/cs249r_book/commit/e0e43e6784e55cfae0866856e1bc45f55a6331d3) fix(mlperf-edu): dataset download race on shared fixed temp filename ### 📊 Changes **1 file changed** (+24 additions, -17 deletions) <details> <summary>View changed files</summary> 📝 `mlperf-edu/src/mlperf/assets.py` (+24 -17) </details> ### 📄 Description ## Summary - `_download()` wrote directly to the caller-provided destination -- a fixed `.download` temp path with no PID/UUID component (`tmp = raw.with_suffix(".download")`, used identically by `ensure_tinyshakespeare`, `ensure_cifar100`, `ensure_movielens_100k`). - Two concurrent fetches of the same asset (plausible since multiple workload training runs each call the dataset factory on first use) both `urlretrieve`/`curl` to the identical path with independent file handles -- a slower writer's in-progress download could be clobbered mid-write by the other process before either side renamed to the final path. ## Fix Download to a process- and call-unique temp file first (`destination.download.<pid>.<uuid8>.part`), then atomically replace the destination. Concurrent callers now only race on the final atomic rename (last-writer-wins, both writes are complete valid downloads), never on the write itself. ## Test plan - [x] `python -m py_compile` passes on the edited file. - [x] Simulated two "concurrent" downloads targeting the same destination with the new unique-temp-then-atomic-replace pattern: final content is a complete, uncorrupted write from whichever finished last, and no leftover `.part` temp files remain. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-07-16 00:27:48 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#36550
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?