[GH-ISSUE #309] UX - edge case handling failures - numerous #6116

Closed
opened 2026-07-12 09:46:42 -05:00 by GiteaMirror · 7 comments
Owner

Originally created by @nyrixx on GitHub (Jun 2, 2026).
Original GitHub issue: https://github.com/RayLabsHQ/gitea-mirror/issues/309

Originally assigned to: @arunavo4 on GitHub.

I seem to be a professional edge case locator. Love the idea of what you have built here and I am very hopeful to be able to use it to manage my starred repo to do or follow up process in general.

I configured everything to perform the following:
Github config:

  • fine grained token from github to just have readonly permissions for stars.
  • only public repos for mirroring stars
  • setup everything for automatic stars and mirroring of stars (338 stars for reference)
  • enabled git LFS
  • enabled all repo metadata
  • destructive update protection set to: Smart

Gitea config:
fresh gitea org, initially made an admin team that had permissions to create repos, this failed and I think is where some of the craziness started with gitea-mirrors automation. When the activity logs were being spammed with the error of user is not an org owner I corrected by adding the gitea-mirror user to the owner team instead.
gitea server is on the same docker host as gitea-mirror, I use the local private space IP as reference (gitea is also configured as a tailscale service). this seems to have cause issues with how Issues/PRs are handled when converting from GH to Gitea as gitea-mirror gets an error referencing the tailscale magicdns address in it even though it has not been configured to use this DNS.

  • Single organization
  • Dedicated Organization
  • Private org visibility

At some point when troubleshooting but when gitea-mirror was still not able to mirror to gitea I changed some settings, it would seem during each of these setting changes a new pending list of mirror copies were added to a pending job list in particular around none/org-owner/owner-org naming convention selection. gitea-mirror proceeded to triplicate all of the repos it could successfully mirror, there is no where in the UI I can find to clear/stop this process, so I had to revoke its access to gitea for now. I even tried disabling all the automatic sync settings within gitea-mirror and it still is filling the activity page with failure entries.

Side note it is not great in a scenario like this to not be able to multi-select repositories and delete all at once from the repositories page.

The action messaging on the activity log page when selecting delete all uses language like this is clearing all pending jobs when it seems this is not actually the case.

I was also experiencing during all this some oof the larger repos failing to mirror once and then having 3-6 copies of them be created within the same naming scheme just appended with the -1,-2,-3, etc with gitea.

I also noticed an issue where I have a few public forks on my gh account. these get listed as successfully mirror within gitea-mirror but are never created within gitea at all.

Originally created by @nyrixx on GitHub (Jun 2, 2026). Original GitHub issue: https://github.com/RayLabsHQ/gitea-mirror/issues/309 Originally assigned to: @arunavo4 on GitHub. I seem to be a professional edge case locator. Love the idea of what you have built here and I am very hopeful to be able to use it to manage my starred repo to do or follow up process in general. I configured everything to perform the following: Github config: - fine grained token from github to just have readonly permissions for stars. - only public repos for mirroring stars - setup everything for automatic stars and mirroring of stars (338 stars for reference) - enabled git LFS - enabled all repo metadata - destructive update protection set to: Smart Gitea config: fresh gitea org, initially made an admin team that had permissions to create repos, this failed and I think is where some of the craziness started with gitea-mirrors automation. When the activity logs were being spammed with the error of user is not an org owner I corrected by adding the gitea-mirror user to the owner team instead. gitea server is on the same docker host as gitea-mirror, I use the local private space IP as reference (gitea is also configured as a tailscale service). this seems to have cause issues with how Issues/PRs are handled when converting from GH to Gitea as gitea-mirror gets an error referencing the tailscale magicdns address in it even though it has not been configured to use this DNS. - Single organization - Dedicated Organization - Private org visibility At some point when troubleshooting but when gitea-mirror was still not able to mirror to gitea I changed some settings, it would seem during each of these setting changes a new pending list of mirror copies were added to a pending job list in particular around none/org-owner/owner-org naming convention selection. gitea-mirror proceeded to triplicate all of the repos it could successfully mirror, there is no where in the UI I can find to clear/stop this process, so I had to revoke its access to gitea for now. I even tried disabling all the automatic sync settings within gitea-mirror and it still is filling the activity page with failure entries. Side note it is not great in a scenario like this to not be able to multi-select repositories and delete all at once from the repositories page. The action messaging on the activity log page when selecting delete all uses language like this is clearing all pending jobs when it seems this is not actually the case. I was also experiencing during all this some oof the larger repos failing to mirror once and then having 3-6 copies of them be created within the same naming scheme just appended with the -1,-2,-3, etc with gitea. I also noticed an issue where I have a few public forks on my gh account. these get listed as successfully mirror within gitea-mirror but are never created within gitea at all.
Author
Owner

@nyrixx commented on GitHub (Jun 2, 2026):

Had a second attempt at it. Removed all repos from gitea-mirror, all activities. removed every repo from the gitea org. sync'd stars list from github repolist populated successfully.

Just picked one repo to mirror, ended up with a duplicate with naming style of:
reponame
reponame-ownername
gitea-mirror hit my rate limit on github during operation, when it picked back up it has now finished mirror to both the repos on my gitea with the only difference being an issue that was closed in the last hour.

<!-- gh-comment-id:4606842835 --> @nyrixx commented on GitHub (Jun 2, 2026): Had a second attempt at it. Removed all repos from gitea-mirror, all activities. removed every repo from the gitea org. sync'd stars list from github repolist populated successfully. Just picked one repo to mirror, ended up with a duplicate with naming style of: reponame reponame-ownername gitea-mirror hit my rate limit on github during operation, when it picked back up it has now finished mirror to both the repos on my gitea with the only difference being an issue that was closed in the last hour.
Author
Owner

@nyrixx commented on GitHub (Jun 2, 2026):

thought I had narrowed it down to failures associated with the metadata settings, removed that and started everything over again. it was going well but as i started to including larger batches of repos I have ended up at 53 repos of gitea-mirror claiming 41 mirrored.

<!-- gh-comment-id:4606991485 --> @nyrixx commented on GitHub (Jun 2, 2026): thought I had narrowed it down to failures associated with the metadata settings, removed that and started everything over again. it was going well but as i started to including larger batches of repos I have ended up at 53 repos of gitea-mirror claiming 41 mirrored.
Author
Owner

@arunavo4 commented on GitHub (Jun 3, 2026):

@nyrixx Thanks for the incredibly detailed report. You genuinely did the hard part for me here. I dug through the code and almost everything you hit traces back to a few root causes. Breaking it down so you know what's confirmed vs. what I still need from you:

Confirmed in code, fixing these:

  • Duplicates when changing strategy/naming (reponame vs reponame-ownername, the triplication): the mirror path only checks whether a repo exists at the current strategy's computed location and ignores where the repo was previously mirrored. So switching strategy/naming creates a second copy instead of relocating or skipping the existing one. The -1/-2/-3 variants fall out of the same collision handling.
  • "Delete All" not actually stopping work: that action only clears the activity/job history rows. Pending work isn't stored as a queue. It's re-derived from each repo's status on the next run, so the scheduler/recovery rediscovers the same repos and refills the log. The wording is misleading and there's no real "cancel pending" action today. I'll fix both the copy and add a way to actually stop pending work.
  • Disabling auto-sync doesn't fully stop it: auto-start also fires when a mirror interval is set (independent of the enabled toggle), and startup recovery resumes interrupted jobs regardless. Tightening both.
  • Tailscale MagicDNS in Issue/PR errors: during sync we use the clone_url returned by the Gitea API, which reflects Gitea's configured ROOT_URL (your MagicDNS host) rather than the LAN IP you gave us. That fails the clone and aborts metadata mirroring. Fix is to prefer your configured Gitea URL.
  • "User is not an org owner": when we create the org we don't add the mirror user to the Owners team, so repo migration into that org is rejected until you join it manually (exactly your workaround). I'll make org setup grant ownership.

Also taking your two UX points: multi-select + bulk delete on the repositories page, and the misleading "clearing pending jobs" language.

Where I need a bit more from you, specifically the public forks showing "mirrored" but never created in Gitea. My leading theory is a name collision: forks keep their upstream's name, and under single-org/flat-user a fork can map to the same owner/name as another repo you've already mirrored, at which point we mark the fork "mirrored" but it's actually pointing at the other repo. To confirm:

  1. What version are you on? (current is 3.17.0)
  2. Your config: mirror strategy, starred repos mode, duplicate-naming strategy, and whether skip forks is on/off.
  3. For one fork that "succeeded" but isn't in Gitea: the repo name, whether a same-named repo already exists in your Gitea org, and the container log lines for that repo around the mirror attempt.

Quick heads-up on logs: tokens are encrypted, but logs can still contain repo/org names, URLs, and your Tailscale hostname, so give them a skim before pasting (or redact the hostname).

None of this is blocking the fixes above. I'm starting on the duplicate/idempotency and org-ownership pieces now. Really appreciate you stress-testing this so thoroughly.

<!-- gh-comment-id:4608163608 --> @arunavo4 commented on GitHub (Jun 3, 2026): @nyrixx Thanks for the incredibly detailed report. You genuinely did the hard part for me here. I dug through the code and almost everything you hit traces back to a few root causes. Breaking it down so you know what's confirmed vs. what I still need from you: **Confirmed in code, fixing these:** - **Duplicates when changing strategy/naming** (`reponame` vs `reponame-ownername`, the triplication): the mirror path only checks whether a repo exists at the *current* strategy's computed location and ignores where the repo was previously mirrored. So switching strategy/naming creates a second copy instead of relocating or skipping the existing one. The `-1/-2/-3` variants fall out of the same collision handling. - **"Delete All" not actually stopping work:** that action only clears the activity/job *history* rows. Pending work isn't stored as a queue. It's re-derived from each repo's status on the next run, so the scheduler/recovery rediscovers the same repos and refills the log. The wording is misleading and there's no real "cancel pending" action today. I'll fix both the copy and add a way to actually stop pending work. - **Disabling auto-sync doesn't fully stop it:** auto-start also fires when a mirror interval is set (independent of the enabled toggle), and startup recovery resumes interrupted jobs regardless. Tightening both. - **Tailscale MagicDNS in Issue/PR errors:** during sync we use the `clone_url` returned by the Gitea API, which reflects Gitea's configured `ROOT_URL` (your MagicDNS host) rather than the LAN IP you gave us. That fails the clone and aborts metadata mirroring. Fix is to prefer your configured Gitea URL. - **"User is not an org owner":** when we create the org we don't add the mirror user to the Owners team, so repo migration into that org is rejected until you join it manually (exactly your workaround). I'll make org setup grant ownership. Also taking your two UX points: **multi-select + bulk delete on the repositories page**, and the **misleading "clearing pending jobs" language**. **Where I need a bit more from you,** specifically the **public forks showing "mirrored" but never created in Gitea**. My leading theory is a name collision: forks keep their upstream's name, and under single-org/flat-user a fork can map to the same `owner/name` as another repo you've already mirrored, at which point we mark the fork "mirrored" but it's actually pointing at the other repo. To confirm: 1. What **version** are you on? (current is 3.17.0) 2. Your config: **mirror strategy**, **starred repos mode**, **duplicate-naming strategy**, and whether **skip forks** is on/off. 3. For one fork that "succeeded" but isn't in Gitea: the **repo name**, whether a same-named repo already exists in your Gitea org, and the **container log lines** for that repo around the mirror attempt. Quick heads-up on logs: tokens are encrypted, but logs can still contain repo/org names, URLs, and your Tailscale hostname, so give them a skim before pasting (or redact the hostname). None of this is blocking the fixes above. I'm starting on the duplicate/idempotency and org-ownership pieces now. Really appreciate you stress-testing this so thoroughly.
Author
Owner

@nyrixx commented on GitHub (Jun 6, 2026):

I have attempted a few more times and the general mirroring of stars use case, all resulting in duplicates.
I have not been able to directly replicate the unmirrored but showing mirrored existing forks I have, the setting you mention was skipping forks has been disabled the whole time.
both gitea-mirror and gitea are currently on the latest available versions of their respective docker images.

I am in the middle of transitioning to forgejo so I will be attempting this use case setup again once that transition target is setup properly.

<!-- gh-comment-id:4639173689 --> @nyrixx commented on GitHub (Jun 6, 2026): I have attempted a few more times and the general mirroring of stars use case, all resulting in duplicates. I have not been able to directly replicate the unmirrored but showing mirrored existing forks I have, the setting you mention was skipping forks has been disabled the whole time. both gitea-mirror and gitea are currently on the latest available versions of their respective docker images. I am in the middle of transitioning to forgejo so I will be attempting this use case setup again once that transition target is setup properly.
Author
Owner

@nyrixx commented on GitHub (Jun 6, 2026):

as a side note or potential UX improvement giving the option to require the mirror to use the selected naming scheme in a forced manner without duplicating and just error handling could be a path to address some items. also generally It would be great to be able to just have reponame-repoowner as a selectable default.

<!-- gh-comment-id:4639657445 --> @nyrixx commented on GitHub (Jun 6, 2026): as a side note or potential UX improvement giving the option to require the mirror to use the selected naming scheme in a forced manner without duplicating and just error handling could be a path to address some items. also generally It would be great to be able to just have reponame-repoowner as a selectable default.
Author
Owner

@arunavo4 commented on GitHub (Jun 13, 2026):

@nyrixx Big update on this. Most of what you ran into is fixed and out now, and there are a couple of spots where my earlier diagnosis was just wrong, so I want to correct those too.

Fixed in v3.18.0:

  • The duplicate mess (#317): the reponame vs reponame-ownername thing, the -1/-2/-3 triplication, and the duplicates you got after hitting the rate limit. Root cause was that name generation treated a repo's own previous mirror as a naming collision and grabbed a new suffix every run, because it only asked "does a repo with this name exist?" and never "is this already a mirror of the same source?". Now it reuses the existing mirror when the source matches (it compares against the repo's original_url), and only adds a suffix on a real collision with a different source. A nice side effect: your recorded mirror location is now authoritative when you change strategy or naming, so switching no longer spawns a second copy. That's basically the "force the chosen naming scheme instead of duplicating" behavior you suggested at the end.
  • Forks showing as mirrored but never actually created (#317): before, any mirror sitting at the expected name was trusted as "ours". Now it only counts if it actually mirrors that fork's source, otherwise the fork gets its own repo.
  • The misleading "Delete All" wording, and the fact that nothing actually stopped pending work (#319): the activity action is now labeled "Clear Activity History" and only clears the log, and there's a separate "Stop Pending Mirrors" button that flips queued/failed repos to Ignored so the scheduler quits picking them back up.
  • Turning off auto-sync not fully stopping it (#319): boot-time auto-start now follows the schedule-enabled toggle, so a configured mirror interval won't override a disabled scheduler anymore.

Improved in v3.19.0:

  • Multi-select on the repositories page (#316): you can click a row and then shift+click another to grab the whole range instead of one at a time. Bulk mirror/sync/retry/skip already work on the selection. A bulk delete specifically isn't there yet (more on that below).

Two corrections to what I told you earlier:

  • The MagicDNS/Tailscale thing: I said issue/PR mirroring was using the URL Gitea hands back, but it isn't. That path already uses the Gitea URL you configured. The real culprit was the pre-sync backup step, which preferred Gitea's clone_url (the one that reflects ROOT_URL, i.e. your MagicDNS host). Fixed in v3.18.0 (#319), it now always builds the URL from your configured address. So if you saw a MagicDNS hostname show up in an error, that backup step is where it came from.
  • "User is not an org owner": I was wrong that we forget to grant ownership. Gitea automatically makes the token's user an owner of any org it creates, so there was never anything to fix there. The error actually shows up when you migrate into an org that already existed and was created by a different account (your admin-team setup), where the mirror user isn't an owner. A proper upfront check for that case is still on my list. Until then, adding the mirror user to that org's Owners team, which is what you did, is the right move.

Still open: a bulk delete on the repositories page. The multi-select work from #316 makes it pretty straightforward to add now.

One more thing since you're heading to Forgejo: the duplicate fix keys off the mirror's original_url, and Forgejo populates that the same way Gitea does, so it should carry over fine. If you end up redoing your starred setup on v3.19.0 at some point, I'd really appreciate hearing whether it holds up. Your 338-star setup is honestly the best stress test this code has gotten.

<!-- gh-comment-id:4697412626 --> @arunavo4 commented on GitHub (Jun 13, 2026): @nyrixx Big update on this. Most of what you ran into is fixed and out now, and there are a couple of spots where my earlier diagnosis was just wrong, so I want to correct those too. **Fixed in v3.18.0:** - **The duplicate mess** (#317): the `reponame` vs `reponame-ownername` thing, the `-1/-2/-3` triplication, and the duplicates you got after hitting the rate limit. Root cause was that name generation treated a repo's own previous mirror as a naming collision and grabbed a new suffix every run, because it only asked "does a repo with this name exist?" and never "is this already a mirror of the same source?". Now it reuses the existing mirror when the source matches (it compares against the repo's `original_url`), and only adds a suffix on a real collision with a different source. A nice side effect: your recorded mirror location is now authoritative when you change strategy or naming, so switching no longer spawns a second copy. That's basically the "force the chosen naming scheme instead of duplicating" behavior you suggested at the end. - **Forks showing as mirrored but never actually created** (#317): before, any mirror sitting at the expected name was trusted as "ours". Now it only counts if it actually mirrors that fork's source, otherwise the fork gets its own repo. - **The misleading "Delete All" wording, and the fact that nothing actually stopped pending work** (#319): the activity action is now labeled "Clear Activity History" and only clears the log, and there's a separate "Stop Pending Mirrors" button that flips queued/failed repos to Ignored so the scheduler quits picking them back up. - **Turning off auto-sync not fully stopping it** (#319): boot-time auto-start now follows the schedule-enabled toggle, so a configured mirror interval won't override a disabled scheduler anymore. **Improved in v3.19.0:** - **Multi-select on the repositories page** (#316): you can click a row and then shift+click another to grab the whole range instead of one at a time. Bulk mirror/sync/retry/skip already work on the selection. A bulk delete specifically isn't there yet (more on that below). **Two corrections to what I told you earlier:** - **The MagicDNS/Tailscale thing:** I said issue/PR mirroring was using the URL Gitea hands back, but it isn't. That path already uses the Gitea URL you configured. The real culprit was the pre-sync backup step, which preferred Gitea's `clone_url` (the one that reflects `ROOT_URL`, i.e. your MagicDNS host). Fixed in v3.18.0 (#319), it now always builds the URL from your configured address. So if you saw a MagicDNS hostname show up in an error, that backup step is where it came from. - **"User is not an org owner":** I was wrong that we forget to grant ownership. Gitea automatically makes the token's user an owner of any org it creates, so there was never anything to fix there. The error actually shows up when you migrate into an org that already existed and was created by a different account (your admin-team setup), where the mirror user isn't an owner. A proper upfront check for that case is still on my list. Until then, adding the mirror user to that org's Owners team, which is what you did, is the right move. **Still open:** a bulk delete on the repositories page. The multi-select work from #316 makes it pretty straightforward to add now. One more thing since you're heading to Forgejo: the duplicate fix keys off the mirror's `original_url`, and Forgejo populates that the same way Gitea does, so it should carry over fine. If you end up redoing your starred setup on v3.19.0 at some point, I'd really appreciate hearing whether it holds up. Your 338-star setup is honestly the best stress test this code has gotten.
Author
Owner

@arunavo4 commented on GitHub (Jun 14, 2026):

Quick follow-up: the bulk delete on the repositories page shipped in v3.19.1 (#322), so the last item from your UX notes is now covered. Select repos (shift+click for a range) and there's a Delete action with a confirm dialog. It only removes them from gitea-mirror, not from Gitea itself.

<!-- gh-comment-id:4700945907 --> @arunavo4 commented on GitHub (Jun 14, 2026): Quick follow-up: the bulk delete on the repositories page shipped in v3.19.1 (#322), so the last item from your UX notes is now covered. Select repos (shift+click for a range) and there's a Delete action with a confirm dialog. It only removes them from gitea-mirror, not from Gitea itself.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea-mirror#6116