Proposal: Import/Migration of all repos in your data-dir should work with a single cli or web-command #2073

New Issue

GiteaMirror · 2025-11-02T04:23:16-06:00

GiteaMirror commented

2025-11-02 04:23:16 -06:00

Originally created by @Cyber1000 on GitHub (Jul 19, 2018).

Edit ❗ : Changing the title, since there is just one point left, my proposal to imports of local repos can be read from starting here: https://github.com/go-gitea/gitea/issues/4474#issuecomment-583788178

Gitea version (or commit ref): fe78154
Operating system: ubuntu (gitea within docker)
Database (use [x]):
- PostgreSQL
- [ X] MySQL
- MSSQL
- SQLite

Description

In https://docs.gitea.io/en-us/backup-and-restore/ "gitea dump" is introduced. Wouldn't it be the same if I just save my docker-volume and make backup of mysql-db? I don't wan't to zip my whole repo-dir, it is backuped daily file per file.
Is it fine to have a cleartext password in app.ini (which is also saved with gitea dump all the time). Wouldn't it be better to cache this in another file (inside docker or also outside if you don't use docker) to exclude this from daily backup.
Is it somehow possible to place all your git-repos into /data/git/repositories/[username]/ and then do some kind of import? (Webgui or command line). If not is there an easy way to implement this? For example with changes to migration-task?

Thanks,
Cyber1000

Originally created by @Cyber1000 on GitHub (Jul 19, 2018). Edit :exclamation: : **Changing the title**, since there is just one point left, my proposal to imports of local repos can be **read from starting here**: https://github.com/go-gitea/gitea/issues/4474#issuecomment-583788178 ----------------------------------------------------------- - Gitea version (or commit ref): fe78154 - Operating system: ubuntu (gitea within docker) - Database (use `[x]`): - [ ] PostgreSQL - [ X] MySQL - [ ] MSSQL - [ ] SQLite ## Description - In https://docs.gitea.io/en-us/backup-and-restore/ "gitea dump" is introduced. Wouldn't it be the same if I just save my docker-volume and make backup of mysql-db? I don't wan't to zip my whole repo-dir, it is backuped daily file per file. - Is it fine to have a cleartext password in app.ini (which is also saved with gitea dump all the time). Wouldn't it be better to cache this in another file (inside docker or also outside if you don't use docker) to exclude this from daily backup. - Is it somehow possible to place all your git-repos into /data/git/repositories/[username]/ and then do some kind of import? (Webgui or command line). If not is there an easy way to implement this? For example with changes to migration-task? Thanks, Cyber1000

GiteaMirror added the type/proposal label 2025-11-02 04:23:16 -06:00

GiteaMirror commented

2025-11-02 04:23:17 -06:00

@Cyber1000 commented on GitHub (Jul 20, 2018):

I've done some research and came to this solution (base is a docker gitea configuration):

Backing up my volume with all the configs (except repositories), backing up my repositories and dumping the mysql-database should include everything. So I may go with this ...
I came to a solution. Having a script with following content:

#!/bin/bash
echo "#File is created automatically, don't change things in here" > ./data/gitea/conf/app.ini
cat ./data/gitea/conf/app.part.ini>>./data/gitea/conf/app.ini
cat ./data/gitea/conf/app.pwd.ini>>./data/gitea/conf/app.ini
docker-compose up -d

app.part.ini is everything except password
app.pwd.ini is:

[database]
PASSWD = xxxxxx

On merging, two database-sections are in the resulting file, but that doesn't matter. Solution could be better (I have to think to start this script instead of a simple docker-compose up, everytime I change something), but for my purposes it's sufficent.

This is somehow my mainpoint for now, since I have a handful of small repos. For now I've turned on IMPORT_LOCAL_PATHS and I'm migrating one by one directly from filesystem. But it's a little bit anoying. A think it would be nothing more than to traverse the directory of a user (or all directories in repositories?), check if the path is entered in database and if it is not -> internally execute the Migrate-Command and pass the name of the found folder, path and user as arguments. More or less, I don't have the time to go deeper, at least for now,

Thanks,
Cyber1000

@Cyber1000 commented on GitHub (Jul 20, 2018): I've done some research and came to this solution (base is a docker gitea configuration): 1. Backing up my volume with all the configs (except repositories), backing up my repositories and dumping the mysql-database should include everything. So I may go with this ... 2. I came to a solution. Having a script with following content: > #!/bin/bash > echo "#File is created automatically, don't change things in here" > ./data/gitea/conf/app.ini > cat ./data/gitea/conf/app.part.ini>>./data/gitea/conf/app.ini > cat ./data/gitea/conf/app.pwd.ini>>./data/gitea/conf/app.ini > docker-compose up -d app.part.ini is everything except password app.pwd.ini is: > [database] > PASSWD = xxxxxx On merging, two database-sections are in the resulting file, but that doesn't matter. Solution could be better (I have to think to start this script instead of a simple docker-compose up, everytime I change something), but for my purposes it's sufficent. 3. This is somehow my mainpoint for now, since I have a handful of small repos. For now I've turned on IMPORT_LOCAL_PATHS and I'm migrating one by one directly from filesystem. But it's a little bit anoying. A think it would be nothing more than to traverse the directory of a user (or all directories in repositories?), check if the path is entered in database and if it is not -> internally execute the Migrate-Command and pass the name of the found folder, path and user as arguments. More or less, I don't have the time to go deeper, at least for now, Thanks, Cyber1000

GiteaMirror commented

2025-11-02 04:23:17 -06:00

@Cyber1000 commented on GitHub (Feb 8, 2020):

Just a follow up, cause I did a cleanup to my configuration:

The approach I used is just fine, with backing up the mysql-db and my repo separately without gitea dump
I have my own template in ./data/etc/templates/app.ini
I can put environment variables in there (more can be found here 70d2244e49/docker/root/etc/s6/gitea/setup):

[database]
DB_TYPE  = mysql
HOST     = database
NAME     = gitea
USER     = $DB_USER
PASSWD   = $DB_PASSWD

I didn't make everything variable (like the database), since I just wanted to mask the password/user, and some other small things.
My docker-compose.yaml (just relevant parts):

environment:
      - USER_UID=1000
      - USER_GID=1000
      - DB_USER=${DB_USER}
      - DB_PASSWD=${DB_PASSWD}
   volumes:
      - ./data/etc/templates/app.ini:/etc/templates/app.ini

Make sure that the real app.ini isn't present when starting your docker-container, there is some magic within gitea, which takes the template from /etc/templates/app.ini, replaces all environment variables and saves in gitea/conf/app.ini
I'm backing up just my template file and not the app.ini (since it can be recreated from my template)
Passwords are stored in an .env file which isn't backuped either

@Cyber1000 commented on GitHub (Feb 8, 2020): Just a follow up, cause I did a cleanup to my configuration: 1. The approach I used is just fine, with backing up the mysql-db and my repo separately without gitea dump 2. I have my own template in ./data/etc/templates/app.ini I can put environment variables in there (more can be found here https://github.com/go-gitea/gitea/blob/70d2244e49e60e11877f850803d33ef1e3900fa6/docker/root/etc/s6/gitea/setup): ``` [database] DB_TYPE = mysql HOST = database NAME = gitea USER = $DB_USER PASSWD = $DB_PASSWD ``` I didn't make everything variable (like the database), since I just wanted to mask the password/user, and some other small things. My docker-compose.yaml (just relevant parts): ``` environment: - USER_UID=1000 - USER_GID=1000 - DB_USER=${DB_USER} - DB_PASSWD=${DB_PASSWD} volumes: - ./data/etc/templates/app.ini:/etc/templates/app.ini ``` - Make sure that the real app.ini isn't present when starting your docker-container, there is some magic within gitea, which takes the template from /etc/templates/app.ini, replaces all environment variables and saves in gitea/conf/app.ini - I'm backing up just my template file and not the app.ini (since it can be recreated from my template) Passwords are stored in an .env file which isn't backuped either

GiteaMirror commented

2025-11-02 04:23:17 -06:00

@Cyber1000 commented on GitHub (Feb 8, 2020):

Changing title of this issue, since the other 2 points are well working.

An easy import of all local stored repos would be nice on first start (or by triggering it) either from webui or from cli (or both)

@Cyber1000 commented on GitHub (Feb 8, 2020): **Changing title of this issue, since the other 2 points are well working.** An easy import of all local stored repos would be nice on first start (or by triggering it) either from webui or from cli (or both)

GiteaMirror referenced this issue

2025-11-02 12:05:08 -06:00

[PR #2083] [MERGED] Don't ignore gravatar error #16220

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#2073