[PR #297] [MERGED] fix: resume interrupted jobs after startup (not only at boot) #6904

Closed
opened 2026-07-14 20:19:35 -05:00 by GiteaMirror · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/RayLabsHQ/gitea-mirror/pull/297
Author: @seanmousseau
Created: 5/22/2026
Status: Merged
Merged: 5/23/2026
Merged by: @arunavo4

Base: mainHead: fix/orchestrator-resume-after-startup


📝 Commits (1)

  • a5563e1 fix: resume interrupted jobs after startup (not only at boot)

📊 Changes

4 files changed (+181 additions, -15 deletions)

View changed files

📝 src/lib/helpers.ts (+17 -4)
src/lib/orchestrator-resume-after-startup.test.ts (+128 -0)
📝 src/lib/recovery.ts (+3 -2)
📝 src/middleware.ts (+33 -9)

📄 Description

Summary

Stops mirrorJobs rows from getting permanently stuck at inProgress=true, lastCheckpoint=never when a sync fails mid-flight after server boot. Found while debugging a separate dedup issue (#296) and observing endless Found 1 interrupted jobs: log spam with no recovery action ever taking place.

The bug

The middleware-level recovery trigger looked like this:

let recoveryInitialized = false;
let recoveryAttempted = false;
// ...
if (!recoveryInitialized && !recoveryAttempted) {  // ← once per process, ever
  recoveryAttempted = true;
  if (needsRecovery) initializeRecovery(...);
  recoveryInitialized = true;
}

Both flags get set on the first request. After that, recovery never re-engages — even if new interruptions appear later in the process lifetime.

So the failure mode is:

  1. Server boots cleanly. Middleware runs initial recovery pass on the first request. No stuck jobs. Both flags flip to true.
  2. User triggers a sync. Job row is created with inProgress=true.
  3. The sync crashes mid-flight (deadlock retry hitting maxRetries, network blip, container restart of a dependency, bun runtime quirk — anything that exits the per-item callback without reaching the resume/finalize codepath).
  4. findInterruptedJobs (called periodically from the health endpoint via hasJobsNeedingRecovery) detects it and logs Found 1 interrupted jobs: on every poll.
  5. But resumeInterruptedJob is only invoked from inside initializeRecovery, which the middleware gate prevents from re-running. The job is stuck forever.

The detector kept finding the row but the resumer never re-fired, producing the symptom: a mirroring-state repo in the UI plus log spam on every health-check tick.

The fix

Middleware (primary fix)

Replace the one-shot gate with an in-flight latch released in a finally block. The actual "don't thrash" guarantee is delegated to the existing 5-minute skipIfRecentAttempt throttle inside initializeRecovery(), which is the right place for it:

let recoveryInFlight = false;  // per-process mutex
// ...
if (!recoveryInFlight) {
  recoveryInFlight = true;
  try {
    if (await hasJobsNeedingRecovery()) {
      await initializeRecovery({ skipIfRecentAttempt: true, ... });
    }
  } finally {
    recoveryInFlight = false;
  }
}

recoveryInitialized is kept and only used to control whether the "first run" log lines fire (so the log doesn't say "startup script may not have run" on every request).

findInterruptedJobs logging (secondary fix)

It previously logged Found N interrupted jobs: unconditionally on every call, including from passive polls (hasJobsNeedingRecovery from the health endpoint and middleware). That produced one log line per poll per stuck job — minutes of log spam for a single stuck job. Made it opt-in:

export async function findInterruptedJobs(options: { logFound?: boolean } = {}) {
  const { logFound = false } = options;
  // ... detect ...
  if (logFound && interruptedJobs.length > 0) { /* log per-job details */ }
  return interruptedJobs;
}

The active recovery cycle in initializeRecovery() passes { logFound: true } so operators still see which jobs are being worked on.

Tests

Adds src/lib/orchestrator-resume-after-startup.test.ts using the structural-source test pattern from gitea-mirror-failure-recovery.test.ts (issue #268's regression test). 4 assertions:

  • Middleware no longer references the once-per-process recoveryAttempted flag
  • Middleware uses an in-flight latch (recoveryInFlight) released in a finally block (so an exception during recovery doesn't permanently jam the latch)
  • findInterruptedJobs accepts a { logFound } option that defaults to false
  • The active recovery path in initializeRecovery() explicitly opts in with { logFound: true }

bun test — 251 pass / 4 skip / 0 fail.

Related

  • #268 — partially addressed. The PR #280 / v3.15.7 fix was about a JS scoping bug that made the initial migrate call's catch path crash before transitioning the repo to failed. That was one cause of stuck mirroring state; this PR addresses the orthogonal issue that even when a job is correctly detectable as interrupted, the post-startup recovery path never re-engages.
  • #296 — companion PR fixing dedup-on-retry. Independent code paths; ship either independently. The duplication symptom in #296 is what made the "stuck job" log spam visible in our test environment, but the two bugs are unrelated.

Test plan

  • bun install && bun test — all green (251/4/0)
  • Confirmed the symptom on stock v3.16.0: Found 1 interrupted jobs: repeating with no recovery action
  • Confirmed the patched build re-engages recovery on subsequent requests when a runtime-interrupted job is detected

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/RayLabsHQ/gitea-mirror/pull/297 **Author:** [@seanmousseau](https://github.com/seanmousseau) **Created:** 5/22/2026 **Status:** ✅ Merged **Merged:** 5/23/2026 **Merged by:** [@arunavo4](https://github.com/arunavo4) **Base:** `main` ← **Head:** `fix/orchestrator-resume-after-startup` --- ### 📝 Commits (1) - [`a5563e1`](https://github.com/RayLabsHQ/gitea-mirror/commit/a5563e1300e13e51581bdc9fdddae78d7092ea29) fix: resume interrupted jobs after startup (not only at boot) ### 📊 Changes **4 files changed** (+181 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `src/lib/helpers.ts` (+17 -4) ➕ `src/lib/orchestrator-resume-after-startup.test.ts` (+128 -0) 📝 `src/lib/recovery.ts` (+3 -2) 📝 `src/middleware.ts` (+33 -9) </details> ### 📄 Description ## Summary Stops `mirrorJobs` rows from getting permanently stuck at `inProgress=true, lastCheckpoint=never` when a sync fails mid-flight after server boot. Found while debugging a separate dedup issue (#296) and observing endless `Found 1 interrupted jobs:` log spam with no recovery action ever taking place. ## The bug The middleware-level recovery trigger looked like this: ```ts let recoveryInitialized = false; let recoveryAttempted = false; // ... if (!recoveryInitialized && !recoveryAttempted) { // ← once per process, ever recoveryAttempted = true; if (needsRecovery) initializeRecovery(...); recoveryInitialized = true; } ``` Both flags get set on the first request. After that, recovery never re-engages — even if new interruptions appear later in the process lifetime. So the failure mode is: 1. Server boots cleanly. Middleware runs initial recovery pass on the first request. No stuck jobs. Both flags flip to true. 2. User triggers a sync. Job row is created with `inProgress=true`. 3. The sync crashes mid-flight (deadlock retry hitting `maxRetries`, network blip, container restart of a dependency, `bun` runtime quirk — anything that exits the per-item callback without reaching the resume/finalize codepath). 4. `findInterruptedJobs` (called periodically from the health endpoint via `hasJobsNeedingRecovery`) detects it and logs `Found 1 interrupted jobs:` on every poll. 5. But `resumeInterruptedJob` is only invoked from inside `initializeRecovery`, which the middleware gate prevents from re-running. **The job is stuck forever.** The detector kept finding the row but the resumer never re-fired, producing the symptom: a `mirroring`-state repo in the UI plus log spam on every health-check tick. ## The fix ### Middleware (primary fix) Replace the one-shot gate with an in-flight latch released in a `finally` block. The actual "don't thrash" guarantee is delegated to the existing 5-minute `skipIfRecentAttempt` throttle inside `initializeRecovery()`, which is the right place for it: ```ts let recoveryInFlight = false; // per-process mutex // ... if (!recoveryInFlight) { recoveryInFlight = true; try { if (await hasJobsNeedingRecovery()) { await initializeRecovery({ skipIfRecentAttempt: true, ... }); } } finally { recoveryInFlight = false; } } ``` `recoveryInitialized` is kept and only used to control whether the "first run" log lines fire (so the log doesn't say "startup script may not have run" on every request). ### `findInterruptedJobs` logging (secondary fix) It previously logged `Found N interrupted jobs:` unconditionally on every call, including from passive polls (`hasJobsNeedingRecovery` from the health endpoint and middleware). That produced one log line per poll per stuck job — minutes of log spam for a single stuck job. Made it opt-in: ```ts export async function findInterruptedJobs(options: { logFound?: boolean } = {}) { const { logFound = false } = options; // ... detect ... if (logFound && interruptedJobs.length > 0) { /* log per-job details */ } return interruptedJobs; } ``` The active recovery cycle in `initializeRecovery()` passes `{ logFound: true }` so operators still see which jobs are being worked on. ## Tests Adds `src/lib/orchestrator-resume-after-startup.test.ts` using the structural-source test pattern from `gitea-mirror-failure-recovery.test.ts` (issue #268's regression test). 4 assertions: - Middleware no longer references the once-per-process `recoveryAttempted` flag - Middleware uses an in-flight latch (`recoveryInFlight`) released in a `finally` block (so an exception during recovery doesn't permanently jam the latch) - `findInterruptedJobs` accepts a `{ logFound }` option that defaults to `false` - The active recovery path in `initializeRecovery()` explicitly opts in with `{ logFound: true }` `bun test` — 251 pass / 4 skip / 0 fail. ## Related - **#268** — partially addressed. The PR #280 / v3.15.7 fix was about a JS scoping bug that made the *initial* migrate call's catch path crash before transitioning the repo to `failed`. That was one cause of stuck `mirroring` state; this PR addresses the orthogonal issue that even when a job *is* correctly detectable as interrupted, the post-startup recovery path never re-engages. - **#296** — companion PR fixing dedup-on-retry. Independent code paths; ship either independently. The duplication symptom in #296 is what made the "stuck job" log spam visible in our test environment, but the two bugs are unrelated. ## Test plan - [x] `bun install && bun test` — all green (251/4/0) - [x] Confirmed the symptom on stock v3.16.0: `Found 1 interrupted jobs:` repeating with no recovery action - [x] Confirmed the patched build re-engages recovery on subsequent requests when a runtime-interrupted job is detected 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-07-14 20:19:35 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea-mirror#6904