mirrorGitRepoMilestonesToGitea had three compounding bugs that produced duplicate Gitea milestone rows on every sync. mirrorGitRepoLabelsToGitea had the last two on its own.
(1) Missing state=all on the existing-milestones GET. Gitea's /milestones endpoint defaults to state=open, so the in-memory existingMilestones Set never contained any closed milestone title. Every CLOSED GitHub milestone was misclassified as missing and re-POSTed on every sync — one new duplicate per closed milestone per sync run.
(2) Single unpaginated GET. Gitea caps response size at [api].MAX_RESPONSE_ITEMS (default 50). Same Gitea-side cap that bit the issues / PRs pre-fetch in commit b76073b. Even with state=all fixed, any repo with > ~50 milestones in a state would silently truncate.
(3) Pagination signal mismatch. Gitea's /milestones and /labels endpoints do NOT emit a Link header — they only emit X-Total-Count. A strict Link-only check (the shape used by the issues/PRs fix in b76073b) terminates after page 1 and re-POSTs every item past index 50. Fix: prefer Link when present, fall back to fetched < X-Total-Count when absent.
Production impact (observed before fix): 11,847 duplicate closed milestones accumulated across 4 mirrored repos:
Repo
Closed milestones
Duplicates
Simple-PHP-IPAM
6,771 (61 distinct)
6,710
Subnet-Calculator
3,534 (31 distinct)
3,503
Simple-WP-Helpdesk
1,445 (17 distinct)
1,428
S3-to-SMB
208 (2 distinct)
206
Same fix applied to mirrorGitRepoLabelsToGitea. Labels hadn't shown duplicates in production yet only because no mirrored repo had crossed 50 distinct labels — bug would have triggered the moment one did.
Both functions also cache newly-created items into the in-memory dedup set after a successful POST so a duplicate entry within the incoming GitHub list (defensive, shouldn't happen) wouldn't trigger a second POST in the same loop.
Test plan
bun test src/lib/gitea-milestone-label-dedup.test.ts — 7 new structural tests pass
bun test src/lib/gitea.test.ts src/lib/gitea-enhanced.test.ts src/lib/gitea-issue-dedup-on-retry.test.ts src/lib/gitea-mirror-failure-recovery.test.ts — 41/41 pass, no regressions
Production validation (2026-05-24): built image from this branch, deployed via Komodo, ran clean syncs against 3 repos that exercise the relevant code paths:
Simple-WP-Helpdesk (delta-add case, 26 GitHub milestones, 22 in Gitea): Mirroring 26 milestones from … ✅ Mirrored 0 new milestones to Gitea — only the 4 actually-new ones POSTed; no false positives.
Subnet-Calculator (v1-regression case — broke an earlier Link-header-only iteration of this fix with 9 false-positive dups on a 74-row /milestones response): Mirroring 34 milestones from … ✅ Mirrored 0 new milestones to Gitea on this commit.
Simple-PHP-IPAM (multi-page case, 86 GitHub milestones, 88 in Gitea spanning 2 pages): Mirroring 86 milestones from … ✅ Mirrored 0 new milestones to Gitea — page 2 correctly fetched via X-Total-Count fallback (/milestones emits no Link header).
Post-sync DB check: 0 milestone / label / issue / release dups across all 4 mirrored repos.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/RayLabsHQ/gitea-mirror/pull/299
**Author:** [@seanmousseau](https://github.com/seanmousseau)
**Created:** 5/24/2026
**Status:** ✅ Merged
**Merged:** 5/25/2026
**Merged by:** [@arunavo4](https://github.com/arunavo4)
**Base:** `main` ← **Head:** `fix/dedup-milestones-and-labels-pagination`
---
### 📝 Commits (4)
- [`b76073b`](https://github.com/RayLabsHQ/gitea-mirror/commit/b76073bcf1baf0baa42492d58ee143121c4a7aa6) fix: prevent duplicate issues & PRs on retry-after-deadlock + Link-header pagination
- [`c093c4c`](https://github.com/RayLabsHQ/gitea-mirror/commit/c093c4c2aee964759795b869fabd2879f2265d47) fix: resume interrupted jobs after startup (not only at boot)
- [`aad8688`](https://github.com/RayLabsHQ/gitea-mirror/commit/aad86884871a2decbebde100c7bc4891da63c39a) fix: prevent duplicate milestones & labels on every sync
- [`711a3bd`](https://github.com/RayLabsHQ/gitea-mirror/commit/711a3bd2b714d4990e95c8acfccdbe8d56299734) fix: paginate /milestones and /labels via X-Total-Count fallback
### 📊 Changes
**7 files changed** (+922 additions, -63 deletions)
<details>
<summary>View changed files</summary>
➕ `src/lib/gitea-issue-dedup-on-retry.test.ts` (+282 -0)
➕ `src/lib/gitea-milestone-label-dedup.test.ts` (+200 -0)
📝 `src/lib/gitea.ts` (+259 -48)
📝 `src/lib/helpers.ts` (+17 -4)
➕ `src/lib/orchestrator-resume-after-startup.test.ts` (+128 -0)
📝 `src/lib/recovery.ts` (+3 -2)
📝 `src/middleware.ts` (+33 -9)
</details>
### 📄 Description
## Summary
`mirrorGitRepoMilestonesToGitea` had **three** compounding bugs that produced duplicate Gitea milestone rows on every sync. `mirrorGitRepoLabelsToGitea` had the last two on its own.
**(1) Missing `state=all` on the existing-milestones GET.** Gitea's `/milestones` endpoint defaults to `state=open`, so the in-memory `existingMilestones` Set never contained any closed milestone title. Every CLOSED GitHub milestone was misclassified as missing and re-POSTed on every sync — one new duplicate per closed milestone per sync run.
**(2) Single unpaginated GET.** Gitea caps response size at `[api].MAX_RESPONSE_ITEMS` (default 50). Same Gitea-side cap that bit the issues / PRs pre-fetch in commit b76073b. Even with `state=all` fixed, any repo with > ~50 milestones in a state would silently truncate.
**(3) Pagination signal mismatch.** Gitea's `/milestones` and `/labels` endpoints do NOT emit a `Link` header — they only emit `X-Total-Count`. A strict Link-only check (the shape used by the issues/PRs fix in b76073b) terminates after page 1 and re-POSTs every item past index 50. Fix: prefer Link when present, fall back to `fetched < X-Total-Count` when absent.
Verified via direct API probe:
```
/repos/.../issues → headers: Link, X-Total-Count (Link works)
/repos/.../milestones → headers: X-Total-Count only (Link absent)
/repos/.../labels → headers: X-Total-Count only (Link absent)
```
**Production impact (observed before fix):** 11,847 duplicate closed milestones accumulated across 4 mirrored repos:
| Repo | Closed milestones | Duplicates |
|---|---:|---:|
| Simple-PHP-IPAM | 6,771 (61 distinct) | 6,710 |
| Subnet-Calculator | 3,534 (31 distinct) | 3,503 |
| Simple-WP-Helpdesk | 1,445 (17 distinct) | 1,428 |
| S3-to-SMB | 208 (2 distinct) | 206 |
Same fix applied to `mirrorGitRepoLabelsToGitea`. Labels hadn't shown duplicates in production yet only because no mirrored repo had crossed 50 distinct labels — bug would have triggered the moment one did.
Both functions also cache newly-created items into the in-memory dedup set after a successful POST so a duplicate entry within the incoming GitHub list (defensive, shouldn't happen) wouldn't trigger a second POST in the same loop.
## Test plan
- [x] `bun test src/lib/gitea-milestone-label-dedup.test.ts` — 7 new structural tests pass
- [x] `bun test src/lib/gitea.test.ts src/lib/gitea-enhanced.test.ts src/lib/gitea-issue-dedup-on-retry.test.ts src/lib/gitea-mirror-failure-recovery.test.ts` — 41/41 pass, no regressions
- [x] **Production validation (2026-05-24):** built image from this branch, deployed via Komodo, ran clean syncs against 3 repos that exercise the relevant code paths:
- **Simple-WP-Helpdesk** (delta-add case, 26 GitHub milestones, 22 in Gitea): `Mirroring 26 milestones from … ✅ Mirrored 0 new milestones to Gitea` — only the 4 actually-new ones POSTed; no false positives.
- **Subnet-Calculator** (v1-regression case — broke an earlier Link-header-only iteration of this fix with 9 false-positive dups on a 74-row `/milestones` response): `Mirroring 34 milestones from … ✅ Mirrored 0 new milestones to Gitea` on this commit.
- **Simple-PHP-IPAM** (multi-page case, 86 GitHub milestones, 88 in Gitea spanning 2 pages): `Mirroring 86 milestones from … ✅ Mirrored 0 new milestones to Gitea` — page 2 correctly fetched via `X-Total-Count` fallback (`/milestones` emits no Link header).
- [x] Post-sync DB check: 0 milestone / label / issue / release dups across all 4 mirrored repos.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/RayLabsHQ/gitea-mirror/pull/299
Author: @seanmousseau
Created: 5/24/2026
Status: ✅ Merged
Merged: 5/25/2026
Merged by: @arunavo4
Base:
main← Head:fix/dedup-milestones-and-labels-pagination📝 Commits (4)
b76073bfix: prevent duplicate issues & PRs on retry-after-deadlock + Link-header paginationc093c4cfix: resume interrupted jobs after startup (not only at boot)aad8688fix: prevent duplicate milestones & labels on every sync711a3bdfix: paginate /milestones and /labels via X-Total-Count fallback📊 Changes
7 files changed (+922 additions, -63 deletions)
View changed files
➕
src/lib/gitea-issue-dedup-on-retry.test.ts(+282 -0)➕
src/lib/gitea-milestone-label-dedup.test.ts(+200 -0)📝
src/lib/gitea.ts(+259 -48)📝
src/lib/helpers.ts(+17 -4)➕
src/lib/orchestrator-resume-after-startup.test.ts(+128 -0)📝
src/lib/recovery.ts(+3 -2)📝
src/middleware.ts(+33 -9)📄 Description
Summary
mirrorGitRepoMilestonesToGiteahad three compounding bugs that produced duplicate Gitea milestone rows on every sync.mirrorGitRepoLabelsToGiteahad the last two on its own.(1) Missing
state=allon the existing-milestones GET. Gitea's/milestonesendpoint defaults tostate=open, so the in-memoryexistingMilestonesSet never contained any closed milestone title. Every CLOSED GitHub milestone was misclassified as missing and re-POSTed on every sync — one new duplicate per closed milestone per sync run.(2) Single unpaginated GET. Gitea caps response size at
[api].MAX_RESPONSE_ITEMS(default 50). Same Gitea-side cap that bit the issues / PRs pre-fetch in commitb76073b. Even withstate=allfixed, any repo with > ~50 milestones in a state would silently truncate.(3) Pagination signal mismatch. Gitea's
/milestonesand/labelsendpoints do NOT emit aLinkheader — they only emitX-Total-Count. A strict Link-only check (the shape used by the issues/PRs fix inb76073b) terminates after page 1 and re-POSTs every item past index 50. Fix: prefer Link when present, fall back tofetched < X-Total-Countwhen absent.Verified via direct API probe:
Production impact (observed before fix): 11,847 duplicate closed milestones accumulated across 4 mirrored repos:
Same fix applied to
mirrorGitRepoLabelsToGitea. Labels hadn't shown duplicates in production yet only because no mirrored repo had crossed 50 distinct labels — bug would have triggered the moment one did.Both functions also cache newly-created items into the in-memory dedup set after a successful POST so a duplicate entry within the incoming GitHub list (defensive, shouldn't happen) wouldn't trigger a second POST in the same loop.
Test plan
bun test src/lib/gitea-milestone-label-dedup.test.ts— 7 new structural tests passbun test src/lib/gitea.test.ts src/lib/gitea-enhanced.test.ts src/lib/gitea-issue-dedup-on-retry.test.ts src/lib/gitea-mirror-failure-recovery.test.ts— 41/41 pass, no regressionsMirroring 26 milestones from … ✅ Mirrored 0 new milestones to Gitea— only the 4 actually-new ones POSTed; no false positives./milestonesresponse):Mirroring 34 milestones from … ✅ Mirrored 0 new milestones to Giteaon this commit.Mirroring 86 milestones from … ✅ Mirrored 0 new milestones to Gitea— page 2 correctly fetched viaX-Total-Countfallback (/milestonesemits no Link header).🤖 Generated with Claude Code
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.