github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-07-15 11:39:31 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Gitea attempts to include current backup file IN the backup #11777

New Issue
Closed
opened 2025-11-02 09:47:27 -06:00 by GiteaMirror · 4 comments
GiteaMirror commented 2025-11-02 09:47:27 -06:00
Owner
Copy Link

Originally created by @TheFriendlyCoder on GitHub (Oct 4, 2023).

Description

Based on my ad-hoc testing of the gitea dump command, I think the expected behaviour is that the default dump file created by the tool is supposed to be put in the current working folder. One way I found this out was that, while I was testing various options trying to get the dump tool to work (see https://github.com/go-gitea/gitea/issues/27423) I had changed directories into the config folder for gitea before running the dump operation:

cd /var/snap/gitea/common/conf
gitea dump

After waiting for a few minutes for the dump to complete, which was much longer than any of my previous attempts, I started to notice the performance of my system was starting to slow down. Closer examination of the memory, cpu and disk space showed that the dump was consuming tremendous amounts of resources. So I killed the dump operation and looked in the current working folder - there was a zip file there generated by the dump but it was many GB in size ... many times larger than the entire working folder for gitea.

What I think happened in this case is that the dump operation started creating the zip file in the config folder, but then it indexed the contents of the folder, picked up the backup file itself, and then tried to re-compress the file into the zip file, in an infinite recursion sort of way.

If I am correct, this is a rather nasty bug that could lead the system into a crash state fairly easily. It may even been abused as a potential exploit for a DoS type attack. Either way it probably should get fixed.

Gitea Version

Gitea version 1.20.5 built with GNU Make 4.3, go1.21.1 : bindata, sqlite, sqlite_unlock_notify, pam, cert

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

git version 2.34.1

Operating System

Ubuntu 22.04.3 LTS

How are you running Gitea?

From the console

Database

SQLite

Originally created by @TheFriendlyCoder on GitHub (Oct 4, 2023). ### Description Based on my ad-hoc testing of the `gitea dump` command, I think the expected behaviour is that the default dump file created by the tool is supposed to be put in the current working folder. One way I found this out was that, while I was testing various options trying to get the dump tool to work (see https://github.com/go-gitea/gitea/issues/27423) I had changed directories into the config folder for gitea before running the dump operation: ``` cd /var/snap/gitea/common/conf gitea dump ``` After waiting for a few minutes for the dump to complete, which was much longer than any of my previous attempts, I started to notice the performance of my system was starting to slow down. Closer examination of the memory, cpu and disk space showed that the dump was consuming tremendous amounts of resources. So I killed the dump operation and looked in the current working folder - there was a zip file there generated by the dump but it was many GB in size ... many times larger than the entire working folder for gitea. What I think happened in this case is that the dump operation started creating the zip file in the config folder, but then it indexed the contents of the folder, picked up the backup file itself, and then tried to re-compress the file into the zip file, in an infinite recursion sort of way. If I am correct, this is a rather nasty bug that could lead the system into a crash state fairly easily. It may even been abused as a potential exploit for a DoS type attack. Either way it probably should get fixed. ### Gitea Version Gitea version 1.20.5 built with GNU Make 4.3, go1.21.1 : bindata, sqlite, sqlite_unlock_notify, pam, cert ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version git version 2.34.1 ### Operating System Ubuntu 22.04.3 LTS ### How are you running Gitea? From the console ### Database SQLite
GiteaMirror added the type/bug label 2025-11-02 09:47:27 -06:00
GiteaMirror commented 2025-11-02 09:47:28 -06:00
Author
Owner
Copy Link

@lng2020 commented on GitHub (Oct 5, 2023):

The problem is when using snap package, the /var/snap/gitea/common is the CUSTOM_PATH. So when the gitea dumper scan the CUSTOM_PATH folder, it includes the gitea-dumpxxxx.zip file(itself) and somehow fails into infinite recursion.
I think this is where the error occurs(f2b2608a86/cmd/dump.go (L481C12-L481C12))
Probably add the check like the code above will solve the problem.( haven't test it yet)

@lng2020 commented on GitHub (Oct 5, 2023): The problem is when using `snap` package, the `/var/snap/gitea/common` is the CUSTOM_PATH. So when the gitea dumper scan the CUSTOM_PATH folder, it includes the `gitea-dumpxxxx.zip` file(itself) and somehow fails into infinite recursion. I think this is where the error occurs(https://github.com/go-gitea/gitea/blob/f2b2608a86b78f50de5d7a62b07040da1fbcc4ef/cmd/dump.go#L481C12-L481C12) Probably add the check like the [code above](https://github.com/go-gitea/gitea/blob/f2b2608a86b78f50de5d7a62b07040da1fbcc4ef/cmd/dump.go#L458) will solve the problem.( haven't test it yet)
GiteaMirror commented 2025-11-02 09:47:29 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Oct 5, 2023):

There is also a design bug. There are many issues reporting that the dumped "zip" would be included into the dump data again.

@wxiaoguang commented on GitHub (Oct 5, 2023): There is also a design bug. There are many issues reporting that the dumped "zip" would be included into the dump data again.
GiteaMirror commented 2025-11-02 09:47:29 -06:00
Author
Owner
Copy Link

@Xulunix commented on GitHub (Oct 17, 2023):

This also affects Gitea running inside kubernetes.
Depending on your exact configuration, the only location to store larger amounts of data inside a pod are the volumes mounted to it, which in case of Gitea is only the data volume itself.
With gitea trying to backup the backup results in this:

gitea-677689f796-8pzzb:/var/lib/gitea$ gitea dump -c /data/gitea/conf/app.ini -f /data/dump_20231017.zip
2023/10/17 11:50:54 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled
2023/10/17 11:50:54 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled
2023/10/17 11:50:54 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/10/17 11:50:54 ...es/setting/mailer.go:237:loadMailerFrom() [I] Mail Service Enabled
2023/10/17 11:50:54 ...es/setting/mailer.go:259:loadNotifyMailFrom() [I] Notify Mail Service Enabled
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/attachments
2023/10/17 11:50:54 ...s/storage/storage.go:166:initAvatars() [I] Initialising Avatar storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/avatars
2023/10/17 11:50:54 ...s/storage/storage.go:192:initRepoAvatars() [I] Initialising Repository Avatar storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-avatars
2023/10/17 11:50:54 ...s/storage/storage.go:186:initLFS() [I] Initialising LFS storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/lfs/data
2023/10/17 11:50:54 ...s/storage/storage.go:198:initRepoArchives() [I] Initialising Repository Archive storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-archive
2023/10/17 11:50:54 ...s/storage/storage.go:208:initPackages() [I] Initialising Packages storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/packages
2023/10/17 11:50:54 ...s/storage/storage.go:219:initActions() [I] Initialising Actions storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_log
2023/10/17 11:50:54 ...s/storage/storage.go:223:initActions() [I] Initialising ActionsArtifacts storage with type: local
2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_artifacts
2023/10/17 11:50:54 cmd/dump.go:258:runDump() [I] Dumping local repositories... /data/git/gitea-repositories
2023/10/17 11:53:15 cmd/dump.go:299:runDump() [I] Dumping database...
2023/10/17 11:53:28 cmd/dump.go:311:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini
2023/10/17 11:53:28 cmd/dump.go:327:runDump() [I] Custom dir /data/gitea is inside data dir /data, skipped
2023/10/17 11:53:28 cmd/dump.go:339:runDump() [I] Packing data directory.../data
Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device
2023/10/17 11:55:24 cmd/dump.go:166:fatal() [F] Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device```
@Xulunix commented on GitHub (Oct 17, 2023): This also affects Gitea running inside kubernetes. Depending on your exact configuration, the only location to store larger amounts of data inside a pod are the volumes mounted to it, which in case of Gitea is only the data volume itself. With gitea trying to backup the backup results in this: ``` gitea-677689f796-8pzzb:/var/lib/gitea$ gitea dump -c /data/gitea/conf/app.ini -f /data/dump_20231017.zip 2023/10/17 11:50:54 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled 2023/10/17 11:50:54 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled 2023/10/17 11:50:54 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled 2023/10/17 11:50:54 ...es/setting/mailer.go:237:loadMailerFrom() [I] Mail Service Enabled 2023/10/17 11:50:54 ...es/setting/mailer.go:259:loadNotifyMailFrom() [I] Notify Mail Service Enabled 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/attachments 2023/10/17 11:50:54 ...s/storage/storage.go:166:initAvatars() [I] Initialising Avatar storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/avatars 2023/10/17 11:50:54 ...s/storage/storage.go:192:initRepoAvatars() [I] Initialising Repository Avatar storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-avatars 2023/10/17 11:50:54 ...s/storage/storage.go:186:initLFS() [I] Initialising LFS storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/lfs/data 2023/10/17 11:50:54 ...s/storage/storage.go:198:initRepoArchives() [I] Initialising Repository Archive storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/repo-archive 2023/10/17 11:50:54 ...s/storage/storage.go:208:initPackages() [I] Initialising Packages storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/packages 2023/10/17 11:50:54 ...s/storage/storage.go:219:initActions() [I] Initialising Actions storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_log 2023/10/17 11:50:54 ...s/storage/storage.go:223:initActions() [I] Initialising ActionsArtifacts storage with type: local 2023/10/17 11:50:54 ...les/storage/local.go:33:NewLocalStorage() [I] Creating new Local Storage at /data/actions_artifacts 2023/10/17 11:50:54 cmd/dump.go:258:runDump() [I] Dumping local repositories... /data/git/gitea-repositories 2023/10/17 11:53:15 cmd/dump.go:299:runDump() [I] Dumping database... 2023/10/17 11:53:28 cmd/dump.go:311:runDump() [I] Adding custom configuration file from /data/gitea/conf/app.ini 2023/10/17 11:53:28 cmd/dump.go:327:runDump() [I] Custom dir /data/gitea is inside data dir /data, skipped 2023/10/17 11:53:28 cmd/dump.go:339:runDump() [I] Packing data directory.../data Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device 2023/10/17 11:55:24 cmd/dump.go:166:fatal() [F] Failed to include data directory: data/dump_20231017.zip: copying contents: write /data/dump_20231017.zip: no space left on device```
GiteaMirror commented 2025-11-02 09:47:29 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Apr 3, 2024):

It's likely the dump output file is in the Gitea's data directory, then it packs itself again and again.

Some cases could be fixed by #30240 , while there might be some still simliar cases.

So you could use --file option to set the output file to a non-Gitea directory

@wxiaoguang commented on GitHub (Apr 3, 2024): It's likely the dump output file is in the Gitea's data directory, then it packs itself again and again. Some cases could be fixed by #30240 , while there might be some still simliar cases. So you could use `--file` option to set the output file to a non-Gitea directory
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#11777
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?