github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-16 13:13:23 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Gitea leaves large uploads in /tmp #8902

New Issue
Closed
opened 2025-11-02 08:22:36 -06:00 by GiteaMirror · 9 comments
GiteaMirror commented 2025-11-02 08:22:36 -06:00
Owner
Copy Link

Originally created by @s-hamann on GitHub (May 3, 2022).

Description

We use Gitea with Drone. One of our CD jobs builds a large binary and uploads it to Gitea as a release (using Drone's Gitea-Release Plugin). This jobs runs nightly.
Apparently, Gitea stores the upload in /tmp and does not remove it from there. Therefore, the server runs out of available disk space after a while.

As a workaround, we manually restart Gitea every now and then. Due to Systemd's PrivateTmp this also wipes Giteas /tmp directory.

Note: We have

[repository.upload]
TEMP_PATH = data/tmp/uploads

So, I'm not sure, why uploads are stored in /tmp at all.

After one nightly job the file from the CD job is in Gitea's /tmp:

-rw------- 1 git git 221663149  3. Mai 02:23 multipart-940622690

To debug the issue, I experimented with uploading attachments to releases via the GUI and the API. In the browser, I'm limited to 20 MB file size and these files did not remain in /tmp.
When uploading the same 20 MB file with curl, it also did not remain in /tmp.
However, I could reproduce the issue by uploading a 30 MB file with curl.
So, the issue seems to be related to file size.
I can't tell, if only "large" files get written to /tmp or all files get written there, but only "small" files are deleted after the upload.

I did not investigate, if this issue affects only attachments to releases, or also other kinds of attachments.

It is unclear to me, what exactly the bug is, but I'm fairly certain it is a bug and not a configuration issue on our side.
I think Gitea should probably remove temporary uploads from /tmp or store them in TEMP_PATH instead of /tmp.

If you need any more information or want me to test something, please ask.

We'll update to the latest Gitea release by the end of the week. I'll report back if that changes anything.

Gitea Version

1.16.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.30.2

Operating System

Debian GNU/Linux 11.2

How are you running Gitea?

Gitea binary is from dl.gitea.io and started via a (custom) Systemd unit:

[Unit]
Description=Gitea (Git with a cup of tea)
After=network.target

[Service]
Type=simple
User=git
Group=git
WorkingDirectory=/var/lib/gitea
ExecStart=/opt/gitea/gitea web
Restart=always
Environment=USER=git HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea GITEA_CUSTOM=/opt/gitea/custom MACARON_ENV=production
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true

[Install]
WantedBy=multi-user.target

Database

PostgreSQL

Originally created by @s-hamann on GitHub (May 3, 2022). ### Description We use Gitea with Drone. One of our CD jobs builds a large binary and uploads it to Gitea as a release (using [Drone's Gitea-Release Plugin](https://github.com/drone-plugins/drone-gitea-release)). This jobs runs nightly. Apparently, Gitea stores the upload in `/tmp` and does not remove it from there. Therefore, the server runs out of available disk space after a while. As a workaround, we manually restart Gitea every now and then. Due to Systemd's `PrivateTmp` this also wipes Giteas `/tmp` directory. Note: We have ``` [repository.upload] TEMP_PATH = data/tmp/uploads ``` So, I'm not sure, why uploads are stored in `/tmp` at all. After one nightly job the file from the CD job is in Gitea's `/tmp`: ``` -rw------- 1 git git 221663149 3. Mai 02:23 multipart-940622690 ``` To debug the issue, I experimented with uploading attachments to releases via the GUI and the API. In the browser, I'm limited to 20 MB file size and these files did not remain in `/tmp`. When uploading the same 20 MB file with `curl`, it also did not remain in `/tmp`. However, I could reproduce the issue by uploading a 30 MB file with `curl`. So, the issue seems to be related to file size. I can't tell, if only "large" files get written to `/tmp` or all files get written there, but only "small" files are deleted after the upload. I did not investigate, if this issue affects only attachments to releases, or also other kinds of attachments. It is unclear to me, what exactly the bug is, but I'm fairly certain it is a bug and not a configuration issue on our side. I think Gitea should probably remove temporary uploads from `/tmp` or store them in `TEMP_PATH` instead of `/tmp`. If you need any more information or want me to test something, please ask. We'll update to the latest Gitea release by the end of the week. I'll report back if that changes anything. ### Gitea Version 1.16.1 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version 2.30.2 ### Operating System Debian GNU/Linux 11.2 ### How are you running Gitea? Gitea binary is from dl.gitea.io and started via a (custom) Systemd unit: ``` [Unit] Description=Gitea (Git with a cup of tea) After=network.target [Service] Type=simple User=git Group=git WorkingDirectory=/var/lib/gitea ExecStart=/opt/gitea/gitea web Restart=always Environment=USER=git HOME=/var/lib/gitea GITEA_WORK_DIR=/var/lib/gitea GITEA_CUSTOM=/opt/gitea/custom MACARON_ENV=production CapabilityBoundingSet=CAP_NET_BIND_SERVICE AmbientCapabilities=CAP_NET_BIND_SERVICE ProtectSystem=full ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true [Install] WantedBy=multi-user.target ``` ### Database PostgreSQL
GiteaMirror added the issue/confirmedtype/bug labels 2025-11-02 08:22:36 -06:00
GiteaMirror commented 2025-11-02 08:22:37 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 3, 2022):

I haven't looked into the problem. Now I can tell some information:

  1. The answer to the size is 32M
  2. It's caused by Golang's mime/multipart/formdata.go, if a form is too large(32M), it will store the file into a temp file by
    file, err := os.CreateTemp("", "multipart-")
  3. I have no idea why the temp file is not deleted, if anyone has time to take a look and has clues, please share or fix.
@wxiaoguang commented on GitHub (May 3, 2022): I haven't looked into the problem. Now I can tell some information: 1. The answer to the size is `32M` 2. It's caused by Golang's `mime/multipart/formdata.go`, if a form is too large(32M), it will store the file into a temp file by `file, err := os.CreateTemp("", "multipart-")` 3. I have no idea why the temp file is not deleted, if anyone has time to take a look and has clues, please share or fix.
GiteaMirror commented 2025-11-02 08:22:37 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 3, 2022):

I have a feeling that maybe this problem is upstream related ....

Or .... Gitea does something wrong ....

@wxiaoguang commented on GitHub (May 3, 2022): I have a feeling that maybe this problem is upstream related .... * https://github.com/golang/go/issues/20253 Or .... Gitea does something wrong ....
GiteaMirror commented 2025-11-02 08:22:37 -06:00
Author
Owner
Copy Link

@jolheiser commented on GitHub (May 3, 2022):

As noted in that ticket, perhaps we aren't using https://pkg.go.dev/mime/multipart#Form.RemoveAll
As a hint to anyone who looks

@jolheiser commented on GitHub (May 3, 2022): As noted in that ticket, perhaps we aren't using https://pkg.go.dev/mime/multipart#Form.RemoveAll As a hint to anyone who looks
GiteaMirror commented 2025-11-02 08:22:38 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 3, 2022):

It should be called by Golang's http framework automatically. Not sure why it doesnt' work correctly.

in server.go

func (w *response) finishRequest() {
	.....
	if w.req.MultipartForm != nil {
		w.req.MultipartForm.RemoveAll()
	}
}
@wxiaoguang commented on GitHub (May 3, 2022): It should be called by Golang's http framework automatically. Not sure why it doesnt' work correctly. in `server.go` ```go func (w *response) finishRequest() { ..... if w.req.MultipartForm != nil { w.req.MultipartForm.RemoveAll() } } ```
GiteaMirror commented 2025-11-02 08:22:38 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 3, 2022):

I think I find the possible root case.

Gitea's ctx.Req is a clone of Golang http framework's request, so the ctx.Req.MultipartForm is not managed by Golang.

Just a guess.

@wxiaoguang commented on GitHub (May 3, 2022): I think I find the possible root case. Gitea's `ctx.Req` is a clone of Golang http framework's `request`, so the `ctx.Req.MultipartForm` is not managed by Golang. Just a guess.
GiteaMirror commented 2025-11-02 08:22:38 -06:00
Author
Owner
Copy Link

@s-hamann commented on GitHub (May 4, 2022):

1. The answer to the size is `32M`

Just to be precise: I could reproduce the issue with a file of exactly 31457280 bytes, which is less than either 32 MB or 32 MiB. Not sure if this is really relevant here.

@s-hamann commented on GitHub (May 4, 2022): > 1. The answer to the size is `32M` Just to be precise: I could reproduce the issue with a file of exactly 31457280 bytes, which is less than either 32 MB or 32 MiB. Not sure if this is really relevant here.
GiteaMirror commented 2025-11-02 08:22:38 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 4, 2022):

Could you show your curl command? I could try to fix it if I can confirm the problem.

@wxiaoguang commented on GitHub (May 4, 2022): Could you show your `curl` command? I could try to fix it if I can confirm the problem.
GiteaMirror commented 2025-11-02 08:22:38 -06:00
Author
Owner
Copy Link

@s-hamann commented on GitHub (May 4, 2022):

Could you show your curl command? I could try to fix it if I can confirm the problem.

I had Swagger generate the curl command. This is the result (with some placeholders):

curl -X 'POST' 'https://$domain/api/v1/repos/$user/$repo/releases/$id/assets?token=$api_token' -H 'accept: application/json' -H 'Content-Type: multipart/form-data' -F 'attachment=@$file'

The release I uploaded to was manually created using Gitea's GUI.

And here's how I generated my test file:

dd if=/dev/zero of=$file bs=1024 count=30
@s-hamann commented on GitHub (May 4, 2022): > Could you show your `curl` command? I could try to fix it if I can confirm the problem. I had Swagger generate the `curl` command. This is the result (with some placeholders): ```sh curl -X 'POST' 'https://$domain/api/v1/repos/$user/$repo/releases/$id/assets?token=$api_token' -H 'accept: application/json' -H 'Content-Type: multipart/form-data' -F 'attachment=@$file' ``` The release I uploaded to was manually created using Gitea's GUI. And here's how I generated my test file: ```sh dd if=/dev/zero of=$file bs=1024 count=30 ```
GiteaMirror commented 2025-11-02 08:22:39 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (May 4, 2022):

OK, everything is clear.

My fix works as expected. No more temp files in /tmp after the patch.

The answer to 32M is: Gitea overwrites Golang's maxMemory by:

if err := ctx.Req.ParseMultipartForm(setting.Attachment.MaxSize << 20); err != nil && !strings.Contains(err.Error(), "EOF")

So if your file exceeds setting.Attachment.MaxSize << 20 (in app.ini), then the post body will be buffered on to disk.

@wxiaoguang commented on GitHub (May 4, 2022): OK, everything is clear. My fix works as expected. No more temp files in `/tmp` after the patch. The answer to `32M` is: Gitea overwrites Golang's `maxMemory` by: ```go if err := ctx.Req.ParseMultipartForm(setting.Attachment.MaxSize << 20); err != nil && !strings.Contains(err.Error(), "EOF") ``` So if your file exceeds `setting.Attachment.MaxSize << 20` (in app.ini), then the post body will be buffered on to disk.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label issue/confirmed type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#8902
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?