github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-13 19:37:47 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Feature request: Use zstandard compression instead of zip for gitea dump? #6658

New Issue
Closed
opened 2025-11-02 07:02:50 -06:00 by GiteaMirror · 8 comments
GiteaMirror commented 2025-11-02 07:02:50 -06:00
Owner
Copy Link

Originally created by @symgryph on GitHub (Jan 10, 2021).

gitea dump is cool, but would be way cooler if we could use zstandard for compression instead of zip.....

Originally created by @symgryph on GitHub (Jan 10, 2021). gitea dump is cool, but would be way cooler if we could use zstandard for compression instead of zip.....
GiteaMirror added the type/proposaltype/enhancement labels 2025-11-02 07:02:50 -06:00
GiteaMirror commented 2025-11-02 07:02:51 -06:00
Author
Owner
Copy Link

@dsseng commented on GitHub (Apr 1, 2021):

I can try myself on this. Found a good CGO-free implementation: https://pkg.go.dev/github.com/klauspost/compress/zstd . Does anyone mind depending on this? Another option is using system archiver on *nix.

@dsseng commented on GitHub (Apr 1, 2021): I can try myself on this. Found a good CGO-free implementation: https://pkg.go.dev/github.com/klauspost/compress/zstd . Does anyone mind depending on this? Another option is using system archiver on *nix.
GiteaMirror commented 2025-11-02 07:02:51 -06:00
Author
Owner
Copy Link

@proailurus commented on GitHub (Nov 22, 2021):

I just made a backup of a large instance and would welcome such a feature.

Zstandard would already be faster, but when using it multithreaded (zstdmt), it could go even faster.
Single-threaded zip/gzip only wastes time these days when we have faster methods readily available.

@proailurus commented on GitHub (Nov 22, 2021): I just made a backup of a large instance and would welcome such a feature. Zstandard would already be faster, but when using it multithreaded (zstdmt), it could go even faster. Single-threaded zip/gzip only wastes time these days when we have faster methods readily available.
GiteaMirror commented 2025-11-02 07:02:51 -06:00
Author
Owner
Copy Link

@tepozoa commented on GitHub (Jul 24, 2022):

(24 Jul 2022) The package already being used, https://github.com/mholt/archiver/v3, (now) supports Zstd. I had a look to dump.go and I think it might just now be a matter of making sure the import is up to date with the latest stable version and adding the tar.zstd extension to the outputTypeEnum?

https://pkg.go.dev/github.com/mholt/archiver/v3#Zstd

@tepozoa commented on GitHub (Jul 24, 2022): (24 Jul 2022) The package already being used, https://github.com/mholt/archiver/v3, (now) supports Zstd. I had a look to `dump.go` and I think it might just now be a matter of making sure the import is up to date with the latest stable version and adding the `tar.zstd` extension to the [outputTypeEnum](https://github.com/go-gitea/gitea/blob/main/cmd/dump.go#L94)? https://pkg.go.dev/github.com/mholt/archiver/v3#Zstd
GiteaMirror commented 2025-11-02 07:02:52 -06:00
Author
Owner
Copy Link

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Aug 10, 2022):

is it just me or are the resulting archives twice the size of .zip?

I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size.

perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd...

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Aug 10, 2022): is it just me or are the resulting archives twice the size of `.zip`? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (`-3` out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd...
GiteaMirror commented 2025-11-02 07:02:52 -06:00
Author
Owner
Copy Link

@symgryph commented on GitHub (Oct 11, 2022):

You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable.

Sincerely

Thomas Munn

From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf @.>
Sent: Wednesday, August 10, 2022 7:24:07 AM
To: go-gitea/gitea @.>
Cc: Thomas @.>; Author @.>
Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290)

is it just me or are the resulting archives twice the size of .zip?

I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size.

perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd...

—
Reply to this email directly, view it on GitHubhttps://github.com/go-gitea/gitea/issues/14290#issuecomment-1210538389, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ.
You are receiving this because you authored the thread.Message ID: @.***>

@symgryph commented on GitHub (Oct 11, 2022): You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable. Sincerely Thomas Munn ________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf ***@***.***> Sent: Wednesday, August 10, 2022 7:24:07 AM To: go-gitea/gitea ***@***.***> Cc: Thomas ***@***.***>; Author ***@***.***> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290) is it just me or are the resulting archives twice the size of .zip? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd... — Reply to this email directly, view it on GitHub<https://github.com/go-gitea/gitea/issues/14290#issuecomment-1210538389>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>
GiteaMirror commented 2025-11-02 07:02:52 -06:00
Author
Owner
Copy Link

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Oct 11, 2022):

You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable. Sincerely Thomas Munn
…
________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf @.> Sent: Wednesday, August 10, 2022 7:24:07 AM To: go-gitea/gitea @.> Cc: Thomas @.>; Author @.> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290) is it just me or are the resulting archives twice the size of .zip? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd... — Reply to this email directly, view it on GitHub<#14290 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ. You are receiving this because you authored the thread.Message ID: @.***>

thanks for the reply, did you mean somehow using ZSTD_CLEVEL in the process? could you by any chance provide an example command (e.g. gitea dump -c <config> --type <whattype-you-mentioned-tar>?
I know you can do tar <opts> --zstd or even tar <opts> -I"zstd <zstdopts>", but I don't see how that could be used with gitea directly.

or did you mean that you were able to make gitea create a tar and then zstd-compress it?

apologies, it just wasn't clear to me from your answer. @symgryph

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Oct 11, 2022): > You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable. Sincerely Thomas Munn > […](#) > ________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf ***@***.***> Sent: Wednesday, August 10, 2022 7:24:07 AM To: go-gitea/gitea ***@***.***> Cc: Thomas ***@***.***>; Author ***@***.***> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290) is it just me or are the resulting archives twice the size of .zip? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd... — Reply to this email directly, view it on GitHub<[#14290 (comment)](https://github.com/go-gitea/gitea/issues/14290#issuecomment-1210538389)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ>. You are receiving this because you authored the thread.Message ID: ***@***.***> thanks for the reply, did you mean somehow using `ZSTD_CLEVEL` in the process? could you by any chance provide an example command (e.g. `gitea dump -c <config> --type <whattype-you-mentioned-tar>`? I know you can do `tar <opts> --zstd` or even `tar <opts> -I"zstd <zstdopts>"`, but I don't see how that could be used with gitea directly. or did you mean that you were able to make gitea create a tar and then `zstd`-compress it? apologies, it just wasn't clear to me from your answer. @symgryph
GiteaMirror commented 2025-11-02 07:02:52 -06:00
Author
Owner
Copy Link

@symgryph commented on GitHub (Oct 11, 2022):

Fish format set -gx ZSTD_CLEVEL 19
run your command. The set command is fish shell specific, as I ain't doin no bash!

Sincerely

Thomas Munn

From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf @.>
Sent: Tuesday, October 11, 2022 1:26:36 PM
To: go-gitea/gitea @.>
Cc: Thomas @.>; Mention @.>
Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290)

You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable. Sincerely Thomas Munn
…
________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf @.> Sent: Wednesday, August 10, 2022 7:24:07 AM To: go-gitea/gitea @.> Cc: Thomas @.>; Author @.> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290https://github.com/go-gitea/gitea/issues/14290) is it just me or are the resulting archives twice the size of .zip? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd... — Reply to this email directly, view it on GitHub<#14290 (comment)https://github.com/go-gitea/gitea/issues/14290#issuecomment-1210538389>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ. You are receiving this because you authored the thread.Message ID: @.***>

thanks for the reply, did you mean somehow using ZSTD_CLEVEL in the process? could you by any chance provide an example command (e.g. gitea dump -c --type ?
I know you can do tar --zstd or even tar -I"zstd ", but I don't see how that could be used with gitea directly.

or did you mean that you were able to make gitea create a tar and then zstd-compress it?

apologies, it just wasn't clear to me from your answer. @symgryphhttps://github.com/symgryph

—
Reply to this email directly, view it on GitHubhttps://github.com/go-gitea/gitea/issues/14290#issuecomment-1275038240, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCSE3XMQWDSYSSPMLK67FTWCWPMZANCNFSM4V4DKZAQ.
You are receiving this because you were mentioned.Message ID: @.***>

@symgryph commented on GitHub (Oct 11, 2022): Fish format set -gx ZSTD_CLEVEL 19 run your command. The set command is fish shell specific, as I ain't doin no bash! Sincerely Thomas Munn ________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf ***@***.***> Sent: Tuesday, October 11, 2022 1:26:36 PM To: go-gitea/gitea ***@***.***> Cc: Thomas ***@***.***>; Mention ***@***.***> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290) You can actually specify the level for z standard to use via a environment variable. Look in the reference manuals. I was able to do this with the tar version via a environment variable. Sincerely Thomas Munn … ________________________________ From: wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf @.> Sent: Wednesday, August 10, 2022 7:24:07 AM To: go-gitea/gitea @.> Cc: Thomas @.>; Author @.> Subject: Re: [go-gitea/gitea] Feature request: Use zstandard compression instead of zip for gitea dump? (#14290<https://github.com/go-gitea/gitea/issues/14290>) is it just me or are the resulting archives twice the size of .zip? I'm a fan of zstd, I am aware of what it can do but if this Gitea setting is using zstd's default compression level (-3 out of 1-19), that might explain the archive size. perhaps exposing some knobs in the future could cater to users wishing to tweak at least the compression levels of zstd... — Reply to this email directly, view it on GitHub<#14290 (comment)<https://github.com/go-gitea/gitea/issues/14290#issuecomment-1210538389>>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABCSE3RUPF4NBHRKIZ225ELVYOGNPANCNFSM4V4DKZAQ. You are receiving this because you authored the thread.Message ID: @.***> thanks for the reply, did you mean somehow using ZSTD_CLEVEL in the process? could you by any chance provide an example command (e.g. gitea dump -c <config> --type <whattype-you-mentioned-tar>? I know you can do tar <opts> --zstd or even tar <opts> -I"zstd <zstdopts>", but I don't see how that could be used with gitea directly. or did you mean that you were able to make gitea create a tar and then zstd-compress it? apologies, it just wasn't clear to me from your answer. @symgryph<https://github.com/symgryph> — Reply to this email directly, view it on GitHub<https://github.com/go-gitea/gitea/issues/14290#issuecomment-1275038240>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABCSE3XMQWDSYSSPMLK67FTWCWPMZANCNFSM4V4DKZAQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>
GiteaMirror commented 2025-11-02 07:02:53 -06:00
Author
Owner
Copy Link

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Oct 14, 2022):

Fish format set -gx ZSTD_CLEVEL 19
run your command. The set command is fish shell specific, as I ain't doin no bash!

Sincerely

Thomas Munn

should be sth like export in bash, or running ENVVAR <command> directly.. will try it out, thanks.

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf commented on GitHub (Oct 14, 2022): > Fish format set -gx ZSTD_CLEVEL 19 > run your command. The set command is fish shell specific, as I ain't doin no bash! > > Sincerely > > Thomas Munn should be sth like `export` in bash, or running `ENVVAR <command>` directly.. will try it out, thanks.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label type/enhancement type/proposal
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#6658
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?