[GH-ISSUE #284] Repos exist in Gitea but missing from Mirror DB #2779

New Issue

GiteaMirror · 2026-05-17T19:17:55-05:00

GiteaMirror commented

2026-05-17 19:17:55 -05:00

Originally created by @nikhilbadyal on GitHub (May 4, 2026).
Original GitHub issue: https://github.com/RayLabsHQ/gitea-mirror/issues/284

I ran into a state where many repos existed in Gitea, but Gitea Mirror didn’t have them in its database. Because cleanup only checks the Mirror DB, these repos were never detected as orphaned and cleanup reported “0 orphaned repos.”

What happened / evidence

Manual cleanup API reports:
- orphanedCount: 0, processedCount: 0

Docker logs show cleanup running successfully but finding no orphaned repos.
DB query showed zero rows for affected repos:
- select ... from repositories where full_name like '%shoutrrr%' → []
Gitea still had those repos, including nikhil/abc.

This makes it impossible to keep Gitea and the Mirror DB consistent. Cleanup and UI state both rely on the DB, so “untracked” repos become invisible to maintenance features.

Expected behavior
If repos exist in Gitea, there should be a way to reconcile or detect DB mismatches.

Actual behavior
Cleanup reports zero orphans because the repos are not in the Mirror DB, even though they exist in Gitea.

Request / Feature Idea
Please add a “force reconcile” or “sync sanity check” feature that:

Compares Gitea repos vs Mirror DB.
Flags missing DB rows or missing Gitea repos.
Optionally fixes the DB or archives/deletes the extra repos.
Reports a summary of abnormalities.

This is what i did to fix.

docker exec -i gitea-mirror sh -lc 'cat > /tmp/cleanup.mjs' <<'EOF'
import { Database } from "bun:sqlite";

const url = process.env.GITEA_URL;
const token = process.env.GITEA_TOKEN;
const owner = process.env.GITEA_OWNER;
const ownerType = process.env.GITEA_OWNER_TYPE || "users";
const mode = process.env.MODE || "dry";

if (!url || !token || !owner) {
  console.error("Missing env vars");
  process.exit(1);
}

const db = new Database("/app/data/gitea-mirror.db");
const trackedRows = db.query(
  "select mirrored_location from repositories where mirrored_location is not null and mirrored_location != ''"
).all();

const tracked = new Set(
  trackedRows.map((r) => String(r.mirrored_location).trim().toLowerCase())
);

async function fetchPage(page) {
  const api = url + "/api/v1/" + ownerType + "/" + owner + "/repos?limit=50&page=" + page;
  const res = await fetch(api, {
    headers: { Authorization: "token " + token },
  });
  if (!res.ok) throw new Error("Gitea " + res.status + " " + res.statusText);
  return res.json();
}

let page = 1;
const candidates = [];

while (true) {
  const repos = await fetchPage(page);
  if (!Array.isArray(repos) || repos.length === 0) break;
  for (const repo of repos) {
    const repoOwner = (repo.owner && repo.owner.login ? repo.owner.login : owner).toLowerCase();
    const full = (repoOwner + "/" + repo.name).toLowerCase();
    if (repo.mirror === true && !tracked.has(full)) {
      candidates.push({ owner: repoOwner, name: repo.name });
    }
  }
  page++;
}

console.log("Candidates: " + candidates.length);
for (const c of candidates) console.log(c.owner + "/" + c.name);

if (mode !== "delete") process.exit(0);

for (const c of candidates) {
  const api = url + "/api/v1/repos/" + c.owner + "/" + c.name;
  const res = await fetch(api, {
    method: "DELETE",
    headers: { Authorization: "token " + token },
  });
  console.log(c.owner + "/" + c.name + " -> " + res.status);
}
EOF





docker exec -it gitea-mirror sh -lc '
export GITEA_URL=""
export GITEA_TOKEN=""
export GITEA_OWNER=""
export GITEA_OWNER_TYPE="users"
export MODE="delete"
bun /tmp/cleanup.mjs
'


> Candidates: 40




docker exec -it gitea-mirror sh -lc '
export GITEA_URL=""
export GITEA_TOKEN=""
export GITEA_OWNER=""
export GITEA_OWNER_TYPE="users"
export MODE="delete"
bun /tmp/cleanup.mjs
'
Candidates: 40

Originally created by @nikhilbadyal on GitHub (May 4, 2026). Original GitHub issue: https://github.com/RayLabsHQ/gitea-mirror/issues/284 I ran into a state where many repos existed in Gitea, but Gitea Mirror didn’t have them in its database. Because cleanup only checks the Mirror DB, these repos were never detected as orphaned and cleanup reported “0 orphaned repos.” **What happened / evidence** - Manual cleanup API reports: - `orphanedCount: 0`, `processedCount: 0` <img width="962" height="562" alt="Image" src="https://github.com/user-attachments/assets/5d6c7c10-bc4a-4827-86e9-c89ca9ca25f0" /> - Docker logs show cleanup running successfully but finding no orphaned repos. - DB query showed zero rows for affected repos: - `select ... from repositories where full_name like '%shoutrrr%'` → `[]` - Gitea still had those repos, including `nikhil/abc`. This makes it impossible to keep Gitea and the Mirror DB consistent. Cleanup and UI state both rely on the DB, so “untracked” repos become invisible to maintenance features. **Expected behavior** If repos exist in Gitea, there should be a way to reconcile or detect DB mismatches. **Actual behavior** Cleanup reports zero orphans because the repos are not in the Mirror DB, even though they exist in Gitea. **Request / Feature Idea** Please add a **“force reconcile”** or **“sync sanity check”** feature that: - Compares Gitea repos vs Mirror DB. - Flags missing DB rows or missing Gitea repos. - Optionally fixes the DB or archives/deletes the extra repos. - Reports a summary of abnormalities. This is what i did to fix. ``` docker exec -i gitea-mirror sh -lc 'cat > /tmp/cleanup.mjs' <<'EOF' import { Database } from "bun:sqlite"; const url = process.env.GITEA_URL; const token = process.env.GITEA_TOKEN; const owner = process.env.GITEA_OWNER; const ownerType = process.env.GITEA_OWNER_TYPE || "users"; const mode = process.env.MODE || "dry"; if (!url || !token || !owner) { console.error("Missing env vars"); process.exit(1); } const db = new Database("/app/data/gitea-mirror.db"); const trackedRows = db.query( "select mirrored_location from repositories where mirrored_location is not null and mirrored_location != ''" ).all(); const tracked = new Set( trackedRows.map((r) => String(r.mirrored_location).trim().toLowerCase()) ); async function fetchPage(page) { const api = url + "/api/v1/" + ownerType + "/" + owner + "/repos?limit=50&page=" + page; const res = await fetch(api, { headers: { Authorization: "token " + token }, }); if (!res.ok) throw new Error("Gitea " + res.status + " " + res.statusText); return res.json(); } let page = 1; const candidates = []; while (true) { const repos = await fetchPage(page); if (!Array.isArray(repos) || repos.length === 0) break; for (const repo of repos) { const repoOwner = (repo.owner && repo.owner.login ? repo.owner.login : owner).toLowerCase(); const full = (repoOwner + "/" + repo.name).toLowerCase(); if (repo.mirror === true && !tracked.has(full)) { candidates.push({ owner: repoOwner, name: repo.name }); } } page++; } console.log("Candidates: " + candidates.length); for (const c of candidates) console.log(c.owner + "/" + c.name); if (mode !== "delete") process.exit(0); for (const c of candidates) { const api = url + "/api/v1/repos/" + c.owner + "/" + c.name; const res = await fetch(api, { method: "DELETE", headers: { Authorization: "token " + token }, }); console.log(c.owner + "/" + c.name + " -> " + res.status); } EOF docker exec -it gitea-mirror sh -lc ' export GITEA_URL="" export GITEA_TOKEN="" export GITEA_OWNER="" export GITEA_OWNER_TYPE="users" export MODE="delete" bun /tmp/cleanup.mjs ' > Candidates: 40 docker exec -it gitea-mirror sh -lc ' export GITEA_URL="" export GITEA_TOKEN="" export GITEA_OWNER="" export GITEA_OWNER_TYPE="users" export MODE="delete" bun /tmp/cleanup.mjs ' Candidates: 40 ```

GiteaMirror commented

2026-05-17 19:17:56 -05:00

@nikhilbadyal commented on GitHub (May 4, 2026):

Stuck in this state.

GitHub Status

@nikhilbadyal commented on GitHub (May 4, 2026): Stuck in this state. <img width="1280" height="673" alt="Image" src="https://github.com/user-attachments/assets/810d760f-01ac-45f5-abc2-a74682e4bbb3" /> GitHub Status <img width="1280" height="646" alt="Image" src="https://github.com/user-attachments/assets/d42b0a0e-9ba5-49e9-89bf-a60bf08a7735" />

GiteaMirror commented

2026-05-17 19:17:57 -05:00

@arunavo4 commented on GitHub (May 5, 2026):

So when building this the expectation was that a clean new gitea would be paired with the application which acts as a backup tool

@arunavo4 commented on GitHub (May 5, 2026): So when building this the expectation was that a clean new gitea would be paired with the application which acts as a backup tool

GiteaMirror commented

2026-05-17 19:17:58 -05:00

@nikhilbadyal commented on GitHub (May 5, 2026):

This was fresh gitea setup only. Created exclusively to be used with gitea-mirror.

@nikhilbadyal commented on GitHub (May 5, 2026): This was fresh gitea setup only. Created exclusively to be used with gitea-mirror.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea-mirror#2779