On each scheduled sync the mirror re-lists a repository's pull requests (octokit.paginate(octokit.rest.pulls.list, { state: "all", per_page: 100, ... })) and then fetches per-PR details. The Octokit client in createGitHubClient was built with the throttling plugin but no conditional-request/ETag layer, so unchanged pull request data was downloaded in full on every run and consumed full rate-limit budget.
This adds ETag-based conditional requests. For every GET the client now replays the previously stored ETag as If-None-Match; GitHub then returns 304 Not Modified when nothing changed, which does not count against the token's primary rate limit, and the cached body is returned in its place. Sync results are unchanged — only redundant full downloads are removed.
What changed
New src/lib/github-conditional-requests.ts: an octokit.hook.wrap("request", ...) that attaches If-None-Match on GETs, caches 200 bodies keyed by user scope + method + URL, and serves the cached body (as a 200) when GitHub replies 304. The paginate link header is preserved on a cache hit so pagination still follows subsequent pages. Non-GET requests are passed through untouched.
The cache store is an interface with a bounded in-memory implementation, wired into createGitHubClient. Gitea Mirror runs as a long-lived server whose scheduler re-syncs on an interval, so a process-lifetime store is enough to turn each repeated poll into a cheap 304. No new dependencies and no schema/migration changes. The store is injectable, so a persistent (e.g. SQLite-backed) implementation could be added later for cross-restart reuse.
Entries are scoped per user so one token never reads another's cached data.
Testing
src/lib/github-conditional-requests.test.ts drives a real Octokit instance with a stubbed fetch and verifies: the second GET replays If-None-Match; a 304 is transparently served from cache as a 200 with the same body; responses without an ETag are re-fetched; non-GET requests are never made conditional; and cache entries are isolated by scope.
Notes
Value to self-hosted users: fewer full downloads and lower pressure on the token's hourly rate limit, so large mirror sets or short sync intervals are less likely to hit throttling.
Optional follow-up (not included here to keep this focused): narrowing the list to state: "open" or an incremental since query where appropriate would further reduce redundant work.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/RayLabsHQ/gitea-mirror/pull/356
**Author:** [@joshfree](https://github.com/joshfree)
**Created:** 8/5/2026
**Status:** ✅ Merged
**Merged:** 8/6/2026
**Merged by:** [@arunavo4](https://github.com/arunavo4)
**Base:** `main` ← **Head:** `feat/conditional-requests-etag`
---
### 📝 Commits (3)
- [`79ccd0d`](https://github.com/RayLabsHQ/gitea-mirror/commit/79ccd0d026685f7901967d397d30eb1922d7dc82) feat(github): reuse ETags across syncs via conditional requests
- [`81b5f03`](https://github.com/RayLabsHQ/gitea-mirror/commit/81b5f030b227edfca94f685afc570452321bba9d) test(github): cover conditional-request ETag cache
- [`e427500`](https://github.com/RayLabsHQ/gitea-mirror/commit/e427500412ff307bf39a3e8518760d39aac7311a) fix: key conditional-request cache by expanded URL
### 📊 Changes
**3 files changed** (+322 additions, -0 deletions)
<details>
<summary>View changed files</summary>
➕ `src/lib/github-conditional-requests.test.ts` (+177 -0)
➕ `src/lib/github-conditional-requests.ts` (+140 -0)
📝 `src/lib/github.ts` (+5 -0)
</details>
### 📄 Description
## Summary
Fixes #355.
On each scheduled sync the mirror re-lists a repository's pull requests (`octokit.paginate(octokit.rest.pulls.list, { state: "all", per_page: 100, ... })`) and then fetches per-PR details. The Octokit client in `createGitHubClient` was built with the throttling plugin but no conditional-request/ETag layer, so unchanged pull request data was downloaded in full on every run and consumed full rate-limit budget.
This adds ETag-based conditional requests. For every GET the client now replays the previously stored `ETag` as `If-None-Match`; GitHub then returns `304 Not Modified` when nothing changed, which does **not** count against the token's primary rate limit, and the cached body is returned in its place. Sync results are unchanged — only redundant full downloads are removed.
## What changed
- New `src/lib/github-conditional-requests.ts`: an `octokit.hook.wrap("request", ...)` that attaches `If-None-Match` on GETs, caches `200` bodies keyed by user scope + method + URL, and serves the cached body (as a `200`) when GitHub replies `304`. The paginate `link` header is preserved on a cache hit so pagination still follows subsequent pages. Non-GET requests are passed through untouched.
- The cache store is an interface with a bounded in-memory implementation, wired into `createGitHubClient`. Gitea Mirror runs as a long-lived server whose scheduler re-syncs on an interval, so a process-lifetime store is enough to turn each repeated poll into a cheap `304`. No new dependencies and no schema/migration changes. The store is injectable, so a persistent (e.g. SQLite-backed) implementation could be added later for cross-restart reuse.
- Entries are scoped per user so one token never reads another's cached data.
## Testing
`src/lib/github-conditional-requests.test.ts` drives a real `Octokit` instance with a stubbed `fetch` and verifies: the second GET replays `If-None-Match`; a `304` is transparently served from cache as a `200` with the same body; responses without an `ETag` are re-fetched; non-GET requests are never made conditional; and cache entries are isolated by scope.
## Notes
- Value to self-hosted users: fewer full downloads and lower pressure on the token's hourly rate limit, so large mirror sets or short sync intervals are less likely to hit throttling.
- Optional follow-up (not included here to keep this focused): narrowing the list to `state: "open"` or an incremental `since` query where appropriate would further reduce redundant work.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/RayLabsHQ/gitea-mirror/pull/356
Author: @joshfree
Created: 8/5/2026
Status: ✅ Merged
Merged: 8/6/2026
Merged by: @arunavo4
Base:
main← Head:feat/conditional-requests-etag📝 Commits (3)
79ccd0dfeat(github): reuse ETags across syncs via conditional requests81b5f03test(github): cover conditional-request ETag cachee427500fix: key conditional-request cache by expanded URL📊 Changes
3 files changed (+322 additions, -0 deletions)
View changed files
➕
src/lib/github-conditional-requests.test.ts(+177 -0)➕
src/lib/github-conditional-requests.ts(+140 -0)📝
src/lib/github.ts(+5 -0)📄 Description
Summary
Fixes #355.
On each scheduled sync the mirror re-lists a repository's pull requests (
octokit.paginate(octokit.rest.pulls.list, { state: "all", per_page: 100, ... })) and then fetches per-PR details. The Octokit client increateGitHubClientwas built with the throttling plugin but no conditional-request/ETag layer, so unchanged pull request data was downloaded in full on every run and consumed full rate-limit budget.This adds ETag-based conditional requests. For every GET the client now replays the previously stored
ETagasIf-None-Match; GitHub then returns304 Not Modifiedwhen nothing changed, which does not count against the token's primary rate limit, and the cached body is returned in its place. Sync results are unchanged — only redundant full downloads are removed.What changed
src/lib/github-conditional-requests.ts: anoctokit.hook.wrap("request", ...)that attachesIf-None-Matchon GETs, caches200bodies keyed by user scope + method + URL, and serves the cached body (as a200) when GitHub replies304. The paginatelinkheader is preserved on a cache hit so pagination still follows subsequent pages. Non-GET requests are passed through untouched.createGitHubClient. Gitea Mirror runs as a long-lived server whose scheduler re-syncs on an interval, so a process-lifetime store is enough to turn each repeated poll into a cheap304. No new dependencies and no schema/migration changes. The store is injectable, so a persistent (e.g. SQLite-backed) implementation could be added later for cross-restart reuse.Testing
src/lib/github-conditional-requests.test.tsdrives a realOctokitinstance with a stubbedfetchand verifies: the second GET replaysIf-None-Match; a304is transparently served from cache as a200with the same body; responses without anETagare re-fetched; non-GET requests are never made conditional; and cache entries are isolated by scope.Notes
state: "open"or an incrementalsincequery where appropriate would further reduce redundant work.🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.