mirror of
https://github.com/go-gitea/gitea.git
synced 2026-07-15 20:09:18 -05:00
feature, bulk importing of repositories #1087
Open
opened 2025-11-02 03:47:53 -06:00 by GiteaMirror
·
50 comments
No Branch/Tag Specified
main
release/v1.25
release/v1.24
release/v1.23
release/v1.22
release/v1.21
release/v1.20
release/v1.19
release/v1.18
release/v1.17
release/v1.16
release/v1.15
release/v1.14
release/v1.13
release/v1.12
release/v1.11
release/v1.10
release/v1.9
release/v1.8
v1.25.3
v1.25.2
v1.25.1
v1.25.0
v1.24.7
v1.25.0-rc0
v1.26.0-dev
v1.24.6
v1.24.5
v1.24.4
v1.24.3
v1.24.2
v1.24.1
v1.24.0
v1.23.8
v1.24.0-rc0
v1.25.0-dev
v1.23.7
v1.23.6
v1.23.5
v1.23.4
v1.23.3
v1.23.2
v1.23.1
v1.23.0
v1.23.0-rc0
v1.24.0-dev
v1.22.6
v1.22.5
v1.22.4
v1.22.3
v1.22.2
v1.22.1
v1.22.0
v1.23.0-dev
v1.22.0-rc1
v1.21.11
v1.22.0-rc0
v1.21.10
v1.21.9
v1.21.8
v1.21.7
v1.21.6
v1.21.5
v1.21.4
v1.21.3
v1.21.2
v1.20.6
v1.21.1
v1.21.0
v1.21.0-rc2
v1.21.0-rc1
v1.20.5
v1.22.0-dev
v1.21.0-rc0
v1.20.4
v1.20.3
v1.20.2
v1.20.1
v1.20.0
v1.19.4
v1.21.0-dev
v1.20.0-rc2
v1.20.0-rc1
v1.20.0-rc0
v1.19.3
v1.19.2
v1.19.1
v1.19.0
v1.19.0-rc1
v1.20.0-dev
v1.19.0-rc0
v1.18.5
v1.18.4
v1.18.3
v1.18.2
v1.18.1
v1.18.0
v1.17.4
v1.18.0-rc1
v1.19.0-dev
v1.18.0-rc0
v1.17.3
v1.17.2
v1.17.1
v1.17.0
v1.17.0-rc2
v1.16.9
v1.17.0-rc1
v1.18.0-dev
v1.16.8
v1.16.7
v1.16.6
v1.16.5
v1.16.4
v1.16.3
v1.16.2
v1.16.1
v1.16.0
v1.15.11
v1.17.0-dev
v1.16.0-rc1
v1.15.10
v1.15.9
v1.15.8
v1.15.7
v1.15.6
v1.15.5
v1.15.4
v1.15.3
v1.15.2
v1.15.1
v1.14.7
v1.15.0
v1.15.0-rc3
v1.14.6
v1.15.0-rc2
v1.14.5
v1.16.0-dev
v1.15.0-rc1
v1.14.4
v1.14.3
v1.14.2
v1.14.1
v1.14.0
v1.13.7
v1.14.0-rc2
v1.13.6
v1.13.5
v1.14.0-rc1
v1.15.0-dev
v1.13.4
v1.13.3
v1.13.2
v1.13.1
v1.13.0
v1.12.6
v1.13.0-rc2
v1.14.0-dev
v1.13.0-rc1
v1.12.5
v1.12.4
v1.12.3
v1.12.2
v1.12.1
v1.11.8
v1.12.0
v1.11.7
v1.12.0-rc2
v1.11.6
v1.12.0-rc1
v1.13.0-dev
v1.11.5
v1.11.4
v1.11.3
v1.10.6
v1.12.0-dev
v1.11.2
v1.10.5
v1.11.1
v1.10.4
v1.11.0
v1.11.0-rc2
v1.10.3
v1.11.0-rc1
v1.10.2
v1.10.1
v1.10.0
v1.9.6
v1.9.5
v1.10.0-rc2
v1.11.0-dev
v1.10.0-rc1
v1.9.4
v1.9.3
v1.9.2
v1.9.1
v1.9.0
v1.9.0-rc2
v1.10.0-dev
v1.9.0-rc1
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.8.0-rc3
v1.7.6
v1.8.0-rc2
v1.7.5
v1.8.0-rc1
v1.9.0-dev
v1.7.4
v1.7.3
v1.7.2
v1.7.1
v1.7.0
v1.7.0-rc3
v1.6.4
v1.7.0-rc2
v1.6.3
v1.7.0-rc1
v1.7.0-dev
v1.6.2
v1.6.1
v1.6.0
v1.6.0-rc2
v1.5.3
v1.6.0-rc1
v1.6.0-dev
v1.5.2
v1.5.1
v1.5.0
v1.5.0-rc2
v1.5.0-rc1
v1.5.0-dev
v1.4.3
v1.4.2
v1.4.1
v1.4.0
v1.4.0-rc3
v1.4.0-rc2
v1.3.3
v1.4.0-rc1
v1.3.2
v1.3.1
v1.3.0
v1.3.0-rc2
v1.3.0-rc1
v1.2.3
v1.2.2
v1.2.1
v1.2.0
v1.2.0-rc3
v1.2.0-rc2
v1.1.4
v1.2.0-rc1
v1.1.3
v1.1.2
v1.1.1
v1.1.0
v1.0.2
v1.0.1
v1.0.0
v0.9.99
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
Mirrored from GitHub Pull Request
Milestone
No items
No Milestone
Projects
Clear projects
No project
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/gitea#1087
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @bradder555 on GitHub (Sep 22, 2017).
Recently i hacked about in the back-end and database to migrate all of our stash repositories and jam these into gitea.
It's not an operation that i'll need to repeat, as far as i can tell in the documentation there's no procedure for doing this sort of thing.
I was thinking, as a feature, it might be handy to have an "upload zip" for migration to facilitate easy migration from other git servers.
I'm sure there're higher priorities, but figured it would make it easier for people to migrate and adopt gitea.
Thanks
@lunny commented on GitHub (Sep 22, 2017):
You can do migrations via API. Write some scripts to do that.
@lafriks commented on GitHub (Sep 22, 2017):
Such migrations should be done in scripts or external tools but should not be part of main gitea repo
@tacotexmex commented on GitHub (Jun 9, 2018):
Any pointers to valid Github-to-Gitea scripts? I come up empty.
@techknowlogick commented on GitHub (Jun 9, 2018):
@tacotexmex currently @JonasFranzDEV has a really good one: https://git.jonasfranz.software/JonasFranzDEV/gitea-github-migrator
@tacotexmex commented on GitHub (Jun 9, 2018):
Thank you very much @tecknowlogick. * quietly thinking how I could’ve discovered this selfhosted gem without your help, dreaming of a federation of Gitea instances, Mastodon-style *
@techknowlogick commented on GitHub (Jun 9, 2018):
@tacotexmex you are very welcome. Currently there is a working group that is dedicated to solving the issue of federated services. I actually just spent the morning catching up on discussions from the group. If you want to follow along you can see: https://github.com/git-federation/gitpub/issues/5 Unrelated to federation we have a discord channel where the above tool was posted to.
@davidak commented on GitHub (Jul 9, 2018):
@lafriks why? i think migration should be as simple as possible.
imagine a workflow like this:
if load on the gitea server could be a problem, a queue could be implemented.
@lafriks commented on GitHub (Jul 11, 2018):
@davidak while I agree that it should be as simple as possible but if we implement imports from github/bitbucket/gitlab etc it will be hard keep up with their api changes. There is example that it can be easily done outside of Gitea main code base - https://git.jonasfranz.software/JonasFranzDEV/gitea-github-migrator
Of course there are few things that we need to implement in Gitea API to have possibility to import more data but that's other thing
@xeddmc commented on GitHub (Sep 10, 2018):
Anyone have a mirror or copy of https://git.jonasfranz.software/JonasFranzDEV/gitea-github-migrator ? The site is throwing a 500 error when trying to either clone the repo locally or even download the source code.
@lafriks commented on GitHub (Sep 10, 2018):
@JonasFranzDev your server has problems ;)
@techknowlogick commented on GitHub (Sep 10, 2018):
Edit, use at your own risk and don't randomly download zips from the internet.
@xeddmc commented on GitHub (Sep 11, 2018):
Cheers guys, thanks 😄
@nodiscc commented on GitHub (Jul 15, 2019):
https://git.jonasfranz.software/JonasFranzDEV/gitea-github-migrator is back up and states:
I just tested it, it worked flawlessly (Debian 9/10)
Edit: limitation: doesn't import pull requests/release notes. Didn't create the project as "mirror" (no automatic pulls in the future).
Repository, issues, tags, wiki were imported correctly. Operation took 11 minutes on a github project with 1300 issues.
@techknowlogick commented on GitHub (Jul 15, 2019):
For others coming to this issue, the migration functionality is now built into gitea itself.
@nodiscc commented on GitHub (Jul 15, 2019):
@techknowlogick you mean using the
+ New migrationaction?This doesn't import issues/wiki for me, only the git repository.
@lunny commented on GitHub (Jul 16, 2019):
@nodiscc @techknowlogick is right. You can migrate almost all things from github.
When you input a github URL on migration UI, and put a username/password or token/, then you will find some migrations options displayed, you can check issues/wiki/pull requests/milestones/labels .
Of course you have to have v1.9 or above.
@nodiscc commented on GitHub (Jul 17, 2019):
@lunny my bad, I'm on 1.8.0, still have to upgrade. I will definitely try this (my gripe was it didn't set up the repository as mirror and there was no way to change it after the fact).
By the way, thank you and all contributors for the hard work on Gitea. I'm in the process of moving all my projects to a self-hosted Gitea instance (with automatic Github/Gitlab mirrors through hooks), it has been great so far. You can find my ansible role to install gitea here (inspired from [1], [2]).
@nodiscc commented on GitHub (Jul 17, 2019):
I think this issue can be closed as bulk import/migration is perfectly doable using https://gitea.com/gitea/migrator/ (which is linked at https://docs.gitea.io/en-us/third-party-tools/) - or one by one from the web interface.
@nodiscc commented on GitHub (Dec 15, 2019):
@lunny ping
@lunny commented on GitHub (Dec 16, 2019):
@nodiscc Currently we can import almost all things of one repository from github, but maybe this issue is importing many repositories on one operation.
@6543 commented on GitHub (Dec 17, 2019):
I would say this is a feature for tea link: https://gitea.com/gitea/tea/issues/22
@6543 commented on GitHub (Dec 17, 2019):
PS: when tea reaches v1.0 we can add it to gitea docs as tool to automate/do stuff via CLI ...
@csarn commented on GitHub (Sep 25, 2020):
Currently the suggested "https://gitea.com/gitea/migrator/" shows:
But "new migration" inside gitea does only migrate a single repository, so this does not help much if one has to migrate hundreds of repos.
@ameeno commented on GitHub (Mar 5, 2021):
@csarn and others. this is true I wanted to bulk migrate repo's but migratory is 1 repo at a time???
what to do?
@6543 commented on GitHub (Mar 5, 2021):
@ameeno best way at the moment: use API via go-sdk or curl ... script
@zeripath commented on GitHub (Mar 5, 2021):
The other option is to use adopt repository if the repositories are in the right place with the correct names.
@ameeno commented on GitHub (Mar 6, 2021):
anyone have a script ready for batch migration? I found a Chinese one on awesome-gitea but I am having problems with it it only working for orgs.
@andreas-bulling commented on GitHub (Oct 30, 2022):
I need to bulk import 80+ repositories from GitLab to Gitea - any pointers for how this could be done? Thanks a lot!
@lunny commented on GitHub (Oct 31, 2022):
You can do it via some scripts with migration API.
@andreas-bulling commented on GitHub (Oct 31, 2022):
sure, I know this is possible. Has anybody written any such script? I couldn't find one... :/
@MTRNord commented on GitHub (Feb 4, 2023):
Hi any update on it?
I have like 300 repos I want to migrate and doing this one by one isnt an issue. the only script I found was at dev.to which doesnt support migration of public vs not or multiple repos. does someone have a github to gitea script for that? :/
@timmb commented on GitHub (May 14, 2023):
Try something like this (assumes you have Github cli installed - on Mac you can
brew install gh)What it doesn't do which I wish it did is set up push forwarding of each repo back to Github so I can use Gitea as my primary and maintain a backup on Github.
@CodeShakingSheep commented on GitHub (Jun 18, 2023):
I stumbled across this thread too and didn't find a suitable solution for my usecase. So, I created this gist: https://gist.github.com/CodeShakingSheep/5dc2cf6ac3b6d265218a7214f8f1210b
No access to Github CLI needed and it works for organization and user repos. If you need user repos, just leave
GITHUB_ORGANISATIONempty.I also created a script for removing the repos from Gitea in case something went wrong. https://gist.github.com/CodeShakingSheep/65a7f46ab6067a72835e13b8459c0f7d
@paskal007r commented on GitHub (Feb 10, 2024):
+1 on this one, in my case it's gitLab not github that I need to migrate from and needing to study a new scripting language or curl just to get it "the right way" it's a bigger nuissance than migrating 20 repos manually, so I'll be doing that for the next half an hour.
@boydaihungst commented on GitHub (Mar 17, 2024):
Currently, only the first 100 repos are imported. This script will import all repos.
https://gist.github.com/boydaihungst/809861ff696deceefbb89a8a075f6b02
@aneurinprice commented on GitHub (Sep 2, 2024):
I have made some slight tweaks to @boydaihungst 's gist and now it includes setting the org fullname,avatar,link and description (if set). This will also set them to be a mirror by default, behaviour can be changed at the bottom of the script.
how-to
./import-org go-gitea@aneurinprice commented on GitHub (Sep 2, 2024):
If I get the time, I might create a dedicated tool for stuff like this
@aneurinprice commented on GitHub (Sep 4, 2024):
If anyone cares, I have a basic prototype up and running. Import orgs or users along with their github avatars. Work in progress, but it works.
https://gitlab.nyeprice.space/aneurinprice/gitea-bulk-exporter
Edit: Add screenshot
@Shinmera commented on GitHub (Oct 6, 2024):
@aneurinprice Looks like your link got 404d?
@aneurinprice commented on GitHub (Oct 6, 2024):
@Shinmera Updated link, Will provide documentation soon, but if you are at all familiar with golang then you'll probably be able to figure it out.
https://github.com/aneurinprice/gitea-bulk-importer
@aneurinprice commented on GitHub (Oct 6, 2024):
@Shinmera credentials should go
gitea-bulk-exporter/internal/types.gofor now. Once again, will fix when I have the time. The logging is somewhat verbose and not always helpful, but if it breaks, you will certainly know :)@Shinmera commented on GitHub (Oct 8, 2024):
Cool, worked well, though it seems like the DryRun and Verbose arguments don't do anything. Not a big deal in my case, but yeah.
@aneurinprice commented on GitHub (Oct 8, 2024):
Currently they do not do anything, you are quite right. They are on my list of things to fix when I have time. Glad it was useful! Is there any functionality you wish it had? Did the user/org avatars import correctly?
Thank you :)
@Shinmera commented on GitHub (Oct 8, 2024):
I didn't import any users and the org already existed, so I can't comment on that, sorry!
As for features, it would be very good if we could "redirect" which user the repos end up at. After all, it's just a happy circumstance that they are named the same in my case, but they might very well not be. It would also be nice if we could exclude repositories, ideally even by repo type so that forks can be skipped. I have a few trash forks that I should delete now.
Wrt docs it would be good if it was described which permissions are needed for the access keys. I kinda guessed with that.
@aneurinprice commented on GitHub (Oct 8, 2024):
As per the repository types, this is something I'm already working on. As I, like yourself have loads of old forks. Thank you for your feedback, I'll do my best to get all of these features in :)On 8 Oct 2024, at 11:42, Yukari Hafner @.***> wrote:
I didn't import any users and the org already existed, so I can't comment on that, sorry!
As for features, it would be very good if we could "redirect" which user the repos end up at. After all, it's just a happy circumstance that they are named the same in my case, but they might very well not be. It would also be nice if we could exclude repositories, ideally even by repo type so that forks can be skipped. I have a few trash forks that I should delete now.
Wrt docs it would be good if it was described which permissions are needed for the access keys. I kinda guessed with that.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>
@aneurinprice commented on GitHub (Oct 9, 2024):
@lunny Both features currently mostly working on my laptop, will get this tidied up and pushed tomorrow.
Just finding a nice way to handle users/orgs with lots of repos.
@aneurinprice commented on GitHub (Oct 10, 2024):
@Shinmera The above was meant for you
@Shinmera commented on GitHub (Oct 10, 2024):
Nice! Great job!
@aneurinprice commented on GitHub (Oct 12, 2024):
https://github.com/aneurinprice/gitea-bulk-importer
@Seele0oO commented on GitHub (Feb 4, 2025):
I rewrote a program to complete the migration, available here
https://github.com/Seele0oO/gitea-migrate-scripts