Learnings from Manual Migration of Gitea Data to Fresh Gitea Instance #14139

New Issue

GiteaMirror · 2025-11-02T11:04:09-06:00

GiteaMirror commented

2025-11-02 11:04:09 -06:00

Originally created by @theAkito on GitHub (Feb 14, 2025).

Feature Description

Migrated from Gitea ~1.18 to 1.23.3. Plain Standard Kubernetes Single-Node Installations.

Tasks

For permission issues with data inside volumes, where a repository cannot be accessed, the Web UI should complain specifically about permission issues, not just say to "contact the administrator". I am the administrator...
When gitea-repositories is not found, then error out. Do not just show no repositories, even though the database knows better, that there should be repositories, but they are missing.
When lfs folder is not found, stop deleting all repositories from database!! Error out and tell the user, that expected LFS files are missing!
When lfs folder is not found, stop wiping repositories, that do not even use lfs, at all. Just error out on those, which actually use LFS.
Generally improve errors regarding configuration issues, whenever some folder is not found or whatever.
Support Git data exports, where, for example, you get a JSON with a list of pull-based mirrors with all Gitea related metadata. On the new instance, those mirrors can just be re-pulled from remote, rather than me copying them all over the place.
Document orphan repository adoption.
Expose REST API for adopting and managing orphan repositories.
Give more options, when adopting orphanded repositories. For example, optionally set creation date, mirror boolean, template boolean, etc.
Ideally, re-vamp adoption of orphan repositories. If orphanded repositories match ones, that are already in database, then merge the metadata of the same-named repositories with the new adopted one. For example, if I adopt akito/myrepo and the database already knows akito/myrepo at a different gitea-repositories folder, than optionally merge the metadata from the "old" repo to the "new" one.

https://forum.gitea.com/t/migrating-repository-folder/10775

Additional Information

Since adopting orphaned repositories is half broken in Gitea, I decided to just 1:1 migrate repositories and LFS files, so I don't have to deal with the problems associated with adoption of orphans.

Originally created by @theAkito on GitHub (Feb 14, 2025). ### Feature Description Migrated from Gitea ~1.18 to 1.23.3. Plain Standard Kubernetes Single-Node Installations. ## Tasks * For permission issues with data inside volumes, where a repository cannot be accessed, the Web UI should complain specifically about permission issues, not just say to "contact the administrator". I am the administrator... * When `gitea-repositories` is not found, then error out. Do not just show no repositories, even though the database knows better, that there *should* be repositories, but they are missing. * When `lfs` folder is not found, stop deleting all repositories from database!! Error out and tell the user, that expected LFS files are missing! * When `lfs` folder is not found, stop wiping repositories, that do not even use `lfs`, at all. Just error out on those, which actually use LFS. * Generally improve errors regarding configuration issues, whenever some folder is not found or whatever. * Support Git data exports, where, for example, you get a JSON with a list of pull-based mirrors with all Gitea related metadata. On the new instance, those mirrors can just be re-pulled from remote, rather than me copying them all over the place. * Document orphan repository adoption. * Expose REST API for adopting and managing orphan repositories. * Give more options, when adopting orphanded repositories. For example, optionally set creation date, mirror boolean, template boolean, etc. * Ideally, re-vamp adoption of orphan repositories. If orphanded repositories match ones, that are already in database, then merge the metadata of the same-named repositories with the new adopted one. For example, if I adopt `akito/myrepo` and the database already knows `akito/myrepo` at a different `gitea-repositories` folder, than optionally merge the metadata from the "old" repo to the "new" one. ## Related https://forum.gitea.com/t/migrating-repository-folder/10775 ## Additional Information Since adopting orphaned repositories is half broken in Gitea, I decided to just 1:1 migrate repositories and LFS files, so I don't have to deal with the problems associated with adoption of orphans.

GiteaMirror added the type/proposal label 2025-11-02 11:04:09 -06:00

GiteaMirror commented

2025-11-02 11:04:10 -06:00

@wxiaoguang commented on GitHub (Feb 15, 2025):

IIRC it should only use 2-3 steps?

Copy all data to new server and chown
Copy app.ini to new server and update db connection/paths

@wxiaoguang commented on GitHub (Feb 15, 2025): IIRC it should only use 2-3 steps? 1. Copy all `data` to new server and `chown` 2. Copy `app.ini` to new server and update db connection/paths

GiteaMirror commented

2025-11-02 11:04:10 -06:00

@theAkito commented on GitHub (Feb 15, 2025):

IIRC it should only use 2-3 steps?

1. Copy all `data` to new server and `chown`

2. Copy `app.ini` to new server and update db connection/paths

Not at all.

I use Ansible to manage app configurations for the installation via Helm Charts on Kubernetes.

My app.ini is set declaratively, as are all other settings, via Ansible and ultimately via Helm Chart.

If you check out the official Gitea Helm Chart, over the versions, many key-value Settings have changed paths, names, etc.
The same way, the default location for gitea-repositories has already changed at least once over the versions, which is related to core Gitea, not the Helm Chart, as far as I know.
The locations for other directories in data have already changed, as well. I needed to move them around, until I found the correct, "new", location...

So, just copying all data and chown will never be sufficient, if your Gitea data structure is at least 6 years old.

Additionally, app.ini has deprecated some features, added new ones. So, I had to adjust it and I also extended it with new settings, which improve my Gitea usage experience.

That said, I tried to change the gitea-repositories path in the app.ini, which caused most the issues, shown in the original issue description. Ultimately, I decided to leave the gitea-repositories & lfs paths as-is, as changing the gitea-repositories has too many advantages, as explained previously.

@theAkito commented on GitHub (Feb 15, 2025): > IIRC it should only use 2-3 steps? > > 1. Copy all `data` to new server and `chown` > > 2. Copy `app.ini` to new server and update db connection/paths Not at all. I use Ansible to manage app configurations for the installation via Helm Charts on Kubernetes. My `app.ini` is set declaratively, as are all other settings, via Ansible and ultimately via Helm Chart. If you check out the official Gitea Helm Chart, over the versions, many key-value Settings have changed paths, names, etc. The same way, the default location for `gitea-repositories` has already changed at least once over the versions, which is related to core Gitea, not the Helm Chart, as far as I know. The locations for other directories in `data` have already changed, as well. I needed to move them around, until I found the correct, "new", location... So, just copying all `data` and `chown` will never be sufficient, if your Gitea data structure is at least 6 years old. Additionally, `app.ini` has deprecated some features, added new ones. So, I had to adjust it and I also extended it with new settings, which improve my Gitea usage experience. That said, I tried to change the `gitea-repositories` path in the `app.ini`, which caused most the issues, shown in the original issue description. Ultimately, I decided to leave the `gitea-repositories` & `lfs` paths as-is, as changing the `gitea-repositories` has too many advantages, as explained previously.

GiteaMirror commented

2025-11-02 11:04:10 -06:00

@wxiaoguang commented on GitHub (Feb 15, 2025):

So it's not simply a "migration". It contains 2 parts:

upgrade to the latest version
migrate to a new server

(or in reverse order: migration to a new server with the same version, then upgrade to the latest version)

@wxiaoguang commented on GitHub (Feb 15, 2025): So it's not simply a "migration". It contains 2 parts: 1. upgrade to the latest version 2. migrate to a new server (or in reverse order: migration to a new server with the same version, then upgrade to the latest version)

GiteaMirror commented

2025-11-02 11:04:11 -06:00

@theAkito commented on GitHub (Feb 15, 2025):

So it's not simply a "migration". It contains 2 parts:
1. upgrade to the latest version

2. migrate to a new server
(or in reverse order: migration to a new server with the same version, then upgrade to the latest version)

The title of this issue says, that I migrated Gitea Data to a Fresh Gitea Instance...

There was no "upgrade", as in upgrade. I migrated data from an older version to a newer version. A simple upgrade wouldn't have caused the path issues with gitea-repositores, for example.

This is a migration. Not purely an "upgrade". In fact, I wouldn't say the name "upgrade", at all. It was migrated from an older version, to a newer version. It's not an "upgrade", it's a big migration.

@theAkito commented on GitHub (Feb 15, 2025): > So it's not simply a "migration". It contains 2 parts: > > 1. upgrade to the latest version > > 2. migrate to a new server > > > (or in reverse order: migration to a new server with the same version, then upgrade to the latest version) The title of this issue says, that I migrated Gitea **Data** to a **Fresh** Gitea Instance... There was no "upgrade", as in upgrade. I migrated data from an older version to a newer version. A simple upgrade wouldn't have caused the path issues with `gitea-repositores`, for example. This is a migration. Not purely an "upgrade". In fact, I wouldn't say the name "upgrade", at all. It was migrated from an older version, to a newer version. It's not an "upgrade", it's a big migration.

GiteaMirror commented

2025-11-02 11:04:11 -06:00

@wxiaoguang commented on GitHub (Feb 15, 2025):

I see, IMO it (to a fresh instance) is not a widely-used approach ...... (TBH I don't quite understand why it should do so, it seems easier to copy the old data to a new server and run the old version first then upgrade to a newer version).

@wxiaoguang commented on GitHub (Feb 15, 2025): I see, IMO it (**to a fresh instance**) is not a widely-used approach ...... (TBH I don't quite understand why it should do so, it seems easier to copy the old data to a new server and run the old version first then upgrade to a newer version).

GiteaMirror commented

2025-11-02 11:04:11 -06:00

@theAkito commented on GitHub (Feb 15, 2025):

I see, IMO it (to a fresh instance) is not a widely-used approach ...... (TBH I don't quite understand why it should do so, it seems easier to copy the old data to a new server and run the old version first then upgrade to a newer version).

I migrated the data to a new location and improved the setup of the Gitea Helm Chart by a lot. I basically re-vamped the whole installation. It would be a widely-used approach, if there were more people using Gitea as long as I have.

That said, I'm not sure what your point with all that is, because it has little to do with the original issue. The original problems described in this issue, are still valid. For example, all the communication issues in the Web UI about the actual problems behind the scenes are perfectly valid issues to work on. It does not matter, whether it was triggered by a major migration, in this specific case.

@theAkito commented on GitHub (Feb 15, 2025): > I see, IMO it (**to a fresh instance**) is not a widely-used approach ...... (TBH I don't quite understand why it should do so, it seems easier to copy the old data to a new server and run the old version first then upgrade to a newer version). I migrated the data to a new location and improved the setup of the Gitea Helm Chart by a lot. I basically re-vamped the whole installation. It would be a widely-used approach, if there were more people using Gitea as long as I have. That said, I'm not sure what your point with all that is, because it has little to do with the original issue. The original problems described in this issue, are still valid. For example, all the communication issues in the Web UI about the actual problems behind the scenes are perfectly valid issues to work on. It does not matter, whether it was triggered by a major migration, in this specific case.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#14139