Package registry: large packages download with size of 0B #9317

@wxiaoguang commented on GitHub (Aug 2, 2022):

Are you using any reverse-proxy, or can you try to download the file from Gitea HTTP port directly? The content-length header doesn't seem to be generated by Gitea.

@wxiaoguang commented on GitHub (Aug 2, 2022): Are you using any reverse-proxy, or can you try to download the file from Gitea HTTP port directly? The content-length header doesn't seem to be generated by Gitea.

GiteaMirror commented

@dorianim commented on GitHub (Aug 2, 2022):

Thanks for your reply! Unfortunately, this does not seem to be the reason.
My instance is indeed running behind a trafik reverse proxy, but when curling it directly, the content length header still exists and is 0:

@dorianim commented on GitHub (Aug 2, 2022): Thanks for your reply! Unfortunately, this does not seem to be the reason. My instance is indeed running behind a trafik reverse proxy, but when curling it directly, the content length header still exists and is 0: <details> ![image](https://user-images.githubusercontent.com/30153207/182305400-807af3be-eb2d-4864-911f-39247832ee99.png) </details>

GiteaMirror commented

@wxiaoguang commented on GitHub (Aug 2, 2022):

I guess the problem is that the uploaded file was empty already. There is a bug when uploading packages.

When a large package is uploaded, the pfci.Data is already EOF before contentStore.Save, so there will an empty file saved in the store.

@KN4CK3R

If I add a pfci.Data.(*packages_module.HashedBuffer).Seek(0, io.SeekStart) before if err := contentStore.Save(packages_module.BlobHash256Key(pb.HashSHA256), pfci.Data, pfci.Data.Size()); err != nil { in addFileToPackageVersion, then the problem is resolved.

@wxiaoguang commented on GitHub (Aug 2, 2022): I guess the problem is that the uploaded file was empty already. There is a bug when uploading packages. When a large package is uploaded, the `pfci.Data` is already EOF before `contentStore.Save`, so there will an empty file saved in the store. @KN4CK3R If I add a `pfci.Data.(*packages_module.HashedBuffer).Seek(0, io.SeekStart)` before `if err := contentStore.Save(packages_module.BlobHash256Key(pb.HashSHA256), pfci.Data, pfci.Data.Size()); err != nil {` in `addFileToPackageVersion`, then the problem is resolved.

GiteaMirror commented

@KN4CK3R commented on GitHub (Aug 2, 2022):

That should not be EOF. My old test upload does not have a problem. I will have a look.

Edit: The error occurs only with larger files which are temporarly stored on the filesystem, so my FileBackedBuffer may be broken.

@KN4CK3R commented on GitHub (Aug 2, 2022): That should not be EOF. My old test upload does not have a problem. I will have a look. Edit: The error occurs only with larger files which are temporarly stored on the filesystem, so my `FileBackedBuffer` may be broken.

GiteaMirror commented

@KN4CK3R commented on GitHub (Aug 2, 2022):

Fixed with #20622

@KN4CK3R commented on GitHub (Aug 2, 2022): Fixed with #20622

GiteaMirror commented

@dorianim commented on GitHub (Aug 2, 2022):

Thanks for the quick fix!
In which version will this be released?

@dorianim commented on GitHub (Aug 2, 2022): Thanks for the quick fix! In which version will this be released?

GiteaMirror commented

@wxiaoguang commented on GitHub (Aug 2, 2022):

Now it on 1.18-dev

There will be a backport for 1.17.1

@wxiaoguang commented on GitHub (Aug 2, 2022): Now it on 1.18-dev There will be a backport for 1.17.1

GiteaMirror commented

@KN4CK3R commented on GitHub (Aug 2, 2022):

Backport in #20635

@KN4CK3R commented on GitHub (Aug 2, 2022): Backport in #20635

GiteaMirror commented

@f403 commented on GitHub (Oct 11, 2022):

I fear, this is not solved, or there is another bug.
When uploading very big files (in my case 1.4G) they do not appear on disk, and downloading results in a 0B size.
Without any warnings during upload/download.

Gitea 1.17.2 (docker)
Steps to reproduce:

$ ll
total 1,4G
drwxrwxr-x  2 someuser someuser 4,0K Okt 11 14:47 .
drwxrwxrwt 41 root     root      12K Okt 11 14:50 ..
-rw-rw-r--  1 someuser someuser 1,4G Aug  9 18:48 ubuntu-22.04.1-live-server-amd64.iso

$ curl --user user:tockentockentockentockentockentockentock --upload-file ubuntu-22.04.1-live-server-amd64.iso https://gitea.host/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso

$ wget -O downloaded.iso https://gitea.host/user/-/packages/generic/ubuntu-live-server/22.04.1/files/30
--2022-10-11 14:57:57--  https://gitea.host/user/-/packages/generic/ubuntu-live-server/22.04.1/files/30
Resolving gitea.host (gitea.host)... 198.51.100.1
Connecting to gitea.host (gitea.host)|198.51.100.1|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [application/octet-stream]
Saving to: ‘downloaded.iso’

downloaded.iso                                           [ <=>                                                                                                                  ]       0  --.-KB/s    in 0s      

2022-10-11 14:57:57 (0,00 B/s) - ‘downloaded.iso’ saved [0/0]

$ ll
total 1,4G
drwxrwxr-x  2 someuser someuser 4,0K Okt 11 14:57 .
drwxrwxrwt 41 root     root      12K Okt 11 14:57 ..
-rw-rw-r--  1 someuser someuser    0 Okt 11 14:57 downloaded.iso
-rw-rw-r--  1 someuser someuser 1,4G Aug  9 18:48 ubuntu-22.04.1-live-server-amd64.iso

And the logs at this time:

gitea     | 2022/10/11 12:53:36 [63456750] router: started   PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322      
gitea     | 2022/10/11 12:53:36 ...rvices/auth/basic.go:67:Verify() [T] [63456750] Basic Authorization: Attempting login for: user                                                                                   
gitea     | 2022/10/11 12:53:36 ...rvices/auth/basic.go:90:Verify() [T] [63456750] Basic Authorization: Valid AccessToken for user[0]                                                                              
gitea     | 2022/10/11 12:53:39 [633ef465-25] router: slow      PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322, elapsed 3888.4ms @ generic/gener
ic.go:67(generic.UploadPackage)                                                                                                                                                                                    
gitea     | 2022/10/11 12:53:50 ...packages/packages.go:112:createPackageAndVersion() [T] [63456750] Creating package: 3, 3, generic, ubuntu-live-server, 22.04.1, map[], map[], false                             
gitea     | 2022/10/11 12:53:51 ...packages/packages.go:226:addFileToPackageVersion() [T] [63456750] Adding package file: 24, ubuntu-22.04.1-live-server-amd64.iso
gitea     | 2022/10/11 12:53:52 [63456750] router: completed PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322, 201 Created in 16835.7ms @ generic/
generic.go:67(generic.UploadPackage)

gitea     | 2022/10/11 12:57:58 [63456856] router: started   GET /user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 for 172.19.0.3:37902
gitea     | 2022/10/11 12:57:58 [63456856] router: completed GET /user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 for 172.19.0.3:37902, 0  in 43.2ms @ user/package.go:370(user.DownloadPackageFile)

And there no new files on disk:

$ touch --date "2022-10-10" /tmp/start
$ find /data/gitea/gitea/packages/ -type f -newer /tmp/start
$

@f403 commented on GitHub (Oct 11, 2022): I fear, this is not solved, or there is another bug. When uploading very big files (in my case 1.4G) they do not appear on disk, and downloading results in a 0B size. Without any warnings during upload/download. Gitea 1.17.2 (docker) Steps to reproduce: ```bash $ ll total 1,4G drwxrwxr-x 2 someuser someuser 4,0K Okt 11 14:47 . drwxrwxrwt 41 root root 12K Okt 11 14:50 .. -rw-rw-r-- 1 someuser someuser 1,4G Aug 9 18:48 ubuntu-22.04.1-live-server-amd64.iso $ curl --user user:tockentockentockentockentockentockentock --upload-file ubuntu-22.04.1-live-server-amd64.iso https://gitea.host/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso $ wget -O downloaded.iso https://gitea.host/user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 --2022-10-11 14:57:57-- https://gitea.host/user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 Resolving gitea.host (gitea.host)... 198.51.100.1 Connecting to gitea.host (gitea.host)|198.51.100.1|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 0 [application/octet-stream] Saving to: ‘downloaded.iso’ downloaded.iso [ <=> ] 0 --.-KB/s in 0s 2022-10-11 14:57:57 (0,00 B/s) - ‘downloaded.iso’ saved [0/0] $ ll total 1,4G drwxrwxr-x 2 someuser someuser 4,0K Okt 11 14:57 . drwxrwxrwt 41 root root 12K Okt 11 14:57 .. -rw-rw-r-- 1 someuser someuser 0 Okt 11 14:57 downloaded.iso -rw-rw-r-- 1 someuser someuser 1,4G Aug 9 18:48 ubuntu-22.04.1-live-server-amd64.iso ``` And the logs at this time: ```text gitea | 2022/10/11 12:53:36 [63456750] router: started PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322 gitea | 2022/10/11 12:53:36 ...rvices/auth/basic.go:67:Verify() [T] [63456750] Basic Authorization: Attempting login for: user gitea | 2022/10/11 12:53:36 ...rvices/auth/basic.go:90:Verify() [T] [63456750] Basic Authorization: Valid AccessToken for user[0] gitea | 2022/10/11 12:53:39 [633ef465-25] router: slow PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322, elapsed 3888.4ms @ generic/gener ic.go:67(generic.UploadPackage) gitea | 2022/10/11 12:53:50 ...packages/packages.go:112:createPackageAndVersion() [T] [63456750] Creating package: 3, 3, generic, ubuntu-live-server, 22.04.1, map[], map[], false gitea | 2022/10/11 12:53:51 ...packages/packages.go:226:addFileToPackageVersion() [T] [63456750] Adding package file: 24, ubuntu-22.04.1-live-server-amd64.iso gitea | 2022/10/11 12:53:52 [63456750] router: completed PUT /api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso for 172.19.0.3:55322, 201 Created in 16835.7ms @ generic/ generic.go:67(generic.UploadPackage) ``` ```text gitea | 2022/10/11 12:57:58 [63456856] router: started GET /user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 for 172.19.0.3:37902 gitea | 2022/10/11 12:57:58 [63456856] router: completed GET /user/-/packages/generic/ubuntu-live-server/22.04.1/files/30 for 172.19.0.3:37902, 0 in 43.2ms @ user/package.go:370(user.DownloadPackageFile) ``` And there no new files on disk: ```bash $ touch --date "2022-10-10" /tmp/start $ find /data/gitea/gitea/packages/ -type f -newer /tmp/start $ ```

GiteaMirror commented

@KN4CK3R commented on GitHub (Oct 11, 2022):

Do you use a reverse proxy?

Tested it with a 1.5gb file and it worked without problems.

@KN4CK3R commented on GitHub (Oct 11, 2022): Do you use a reverse proxy? Tested it with a 1.5gb file and it worked without problems.

GiteaMirror commented

@f403 commented on GitHub (Oct 11, 2022):

Yes, but same happens when uploading without proxy.

$ curl --user user:tockentockentockentockentockentockentock --upload-file ubuntu-22.04.1-live-server-amd64.iso http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso
$ wget -O downloaded.iso http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso
--2022-10-11 14:03:51--  http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso
Connecting to 172.19.0.4:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [application/octet-stream]
Saving to: ‘downloaded.iso’

downloaded.iso                                           [ <=>                                                                                                                  ]       0  --.-KB/s    in 0s      

2022-10-11 14:03:51 (0.00 B/s) - ‘downloaded.iso’ saved [0/0]

$ ll
total 1.4G
drwxrwxr-x  2 someuser someuser 4.0K Oct 11 14:03 ./
drwxrwxrwt 15 root     root     4.0K Oct 11 13:53 ../
-rw-rw-r--  1 someuser someuser    0 Oct 11 14:03 downloaded.iso
-rw-rw-r--  1 someuser someuser 1.4G Aug  9 16:48 ubuntu-22.04.1-live-server-amd64.iso

@f403 commented on GitHub (Oct 11, 2022): Yes, but same happens when uploading without proxy. ```bash $ curl --user user:tockentockentockentockentockentockentock --upload-file ubuntu-22.04.1-live-server-amd64.iso http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso $ wget -O downloaded.iso http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso --2022-10-11 14:03:51-- http://172.19.0.4:3000/api/packages/user/generic/ubuntu-live-server/22.04.1/ubuntu-22.04.1-live-server-amd64.iso Connecting to 172.19.0.4:3000... connected. HTTP request sent, awaiting response... 200 OK Length: 0 [application/octet-stream] Saving to: ‘downloaded.iso’ downloaded.iso [ <=> ] 0 --.-KB/s in 0s 2022-10-11 14:03:51 (0.00 B/s) - ‘downloaded.iso’ saved [0/0] $ ll total 1.4G drwxrwxr-x 2 someuser someuser 4.0K Oct 11 14:03 ./ drwxrwxrwt 15 root root 4.0K Oct 11 13:53 ../ -rw-rw-r-- 1 someuser someuser 0 Oct 11 14:03 downloaded.iso -rw-rw-r-- 1 someuser someuser 1.4G Aug 9 16:48 ubuntu-22.04.1-live-server-amd64.iso ```

GiteaMirror commented

@f403 commented on GitHub (Oct 11, 2022):

Hmm...
It worked in a fresh container on my laptop.

Can you, please, give me a hint, where to look for the uploaded files (temp path, disk path for a certain package file)?

@f403 commented on GitHub (Oct 11, 2022): Hmm... It worked in a fresh container on my laptop. Can you, please, give me a hint, where to look for the uploaded files (temp path, disk path for a certain package file)?

GiteaMirror commented