mirror of
https://github.com/RayLabsHQ/gitea-mirror.git
synced 2026-11-16 14:45:02 -06:00
[GH-ISSUE #355] PR mirroring re-fetches all pull requests each sync without conditional requests (ETag) #7737
Reference in New Issue
Block a user
Originally created by @joshfree on GitHub (Aug 5, 2026).
Original GitHub issue: https://github.com/RayLabsHQ/gitea-mirror/issues/355
Originally assigned to: @arunavo4 on GitHub.
Summary
On each scheduled sync,
mirrorGitHubPullRequestsToGiteare-lists a repository's pull requests withstate: "all"and then fetches per-PR details, with noIf-None-Match/ETag conditional requests. Unchanged pull request lists are re-downloaded in full every run.Where
src/lib/github.ts(createGitHubClient).src/lib/gitea.ts—octokit.paginate(octokit.rest.pulls.list, { state: "all", per_page: 100, ... }), followed bypulls.get/pulls.listCommits/pulls.listFilesper PR.3600s) —src/lib/env-config-loader.ts.Impact on users
GitHub's REST API supports conditional requests: sending the previously returned
ETagasIf-None-Matchyields a304 Not Modifiedwhen nothing changed, and304responses do not count against the token's primary rate limit. Because the mirror doesn't use them, every sync spends full rate-limit budget re-downloading unchanged PR data. For users with many mirrored repositories or short sync intervals, this makes it easier to hit the token's hourly rate limit (leading to throttling and slower/failed syncs) and uses more bandwidth than necessary.Suggested change
Store the
ETagreturned for each(repo, endpoint)and send it asIf-None-Matchon the next sync, reusing the cached result on304. Octokit ships no cache by default. This can be done without new heavyweight dependencies — e.g. persist ETags in the existing SQLite database and set/readIf-None-Matchin anoctokit.hookaround the current client, keeping in line with the project's "minimal dependencies / SQLite" guidelines. Optionally, narrowing the list tostate: "open"(or an incrementalsincequery) where appropriate would further reduce redundant work.This keeps sync results identical while lowering rate-limit pressure and bandwidth for self-hosted users.