Timeout when pushing big docker images with docker #13240

Closed
opened 2025-11-02 10:35:53 -06:00 by GiteaMirror · 4 comments
Owner

Originally created by @CPlusPlus17 on GitHub (Jul 4, 2024).

Description

When pushing a big docker images the command is aborted after 60s.

2024/07/04 15:39:30 ...eb/routing/logger.go:68:func1() [W] router: slow      PATCH /v2/xxx/dotnet/build/blobs/uploads/gfjt0i63h5jxsjle8jgvzbyk5 for 10.244.2.206:57674, elapsed 3517.0ms @ packages/api.go:700(packages.ContainerRoutes.func2.2)
2024/07/04 15:40:27 ...ntainer/container.go:90:apiError() [E] unexpected EOF
2024/07/04 15:40:27 ...eb/routing/logger.go:102:func1() [I] router: completed PATCH /v2/xxx/dotnet/build/blobs/uploads/gfjt0i63h5jxsjle8jgvzbyk5 for 10.244.2.206:57674, 500 Internal Server Error in 60197.8ms @ packages/api.go:700(packages.ContainerRoutes.func2.2)

Traefik log:

10.244.0.0 - - [04/Jul/2024:15:27:53 +0000] "PATCH /v2/xxx/dotnet/build/blobs/uploads/heuof3pvgcekqjxiftepaqqdk HTTP/1.1" 499 21 "-" "-" 223 "websecure-gitea-prd-gitea-ingress-services-xxx-xxx-admin-ch-v2@kubernetes" "http://10.244.4.21:3000" 60000ms

Traefik has a timeout of 600s before aborting the request.

Gitea config:

gitea:
  config:
    server:
      ROOT_URL: https://xxx/gitea-prd/
      PER_WRITE_TIMEOUT: -1

I'm checkout the documentation but couldn't find any which is defaulted to 60s and has impact for this command. I'm missing something? I report this as bug because there seems to be no documentation for this issue to be resolved and set a higher timeout.

Gitea Version

1.22.0

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

linux

How are you running Gitea?

Kubernetes with traefik ingress

Database

MSSQL

Originally created by @CPlusPlus17 on GitHub (Jul 4, 2024). ### Description When pushing a big docker images the command is aborted after 60s. ``` 2024/07/04 15:39:30 ...eb/routing/logger.go:68:func1() [W] router: slow PATCH /v2/xxx/dotnet/build/blobs/uploads/gfjt0i63h5jxsjle8jgvzbyk5 for 10.244.2.206:57674, elapsed 3517.0ms @ packages/api.go:700(packages.ContainerRoutes.func2.2) 2024/07/04 15:40:27 ...ntainer/container.go:90:apiError() [E] unexpected EOF 2024/07/04 15:40:27 ...eb/routing/logger.go:102:func1() [I] router: completed PATCH /v2/xxx/dotnet/build/blobs/uploads/gfjt0i63h5jxsjle8jgvzbyk5 for 10.244.2.206:57674, 500 Internal Server Error in 60197.8ms @ packages/api.go:700(packages.ContainerRoutes.func2.2) ``` Traefik log: ```10.244.0.0 - - [04/Jul/2024:15:27:53 +0000] "PATCH /v2/xxx/dotnet/build/blobs/uploads/heuof3pvgcekqjxiftepaqqdk HTTP/1.1" 499 21 "-" "-" 223 "websecure-gitea-prd-gitea-ingress-services-xxx-xxx-admin-ch-v2@kubernetes" "http://10.244.4.21:3000" 60000ms``` Traefik has a timeout of 600s before aborting the request. Gitea config: ``` gitea: config: server: ROOT_URL: https://xxx/gitea-prd/ PER_WRITE_TIMEOUT: -1 ``` I'm checkout the documentation but couldn't find any which is defaulted to 60s and has impact for this command. I'm missing something? I report this as bug because there seems to be no documentation for this issue to be resolved and set a higher timeout. ### Gitea Version 1.22.0 ### Can you reproduce the bug on the Gitea demo site? Yes ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System linux ### How are you running Gitea? Kubernetes with traefik ingress ### Database MSSQL
GiteaMirror added the issue/needs-feedbacktype/bug labels 2025-11-02 10:35:53 -06:00
Author
Owner

@techknowlogick commented on GitHub (Jul 4, 2024):

Can you update to 1.22.1 (has just been tagged, and the blog post hasn't even been written yet), as it has some improvements around this. As well, are you using minio (or s3) to store packages?

@techknowlogick commented on GitHub (Jul 4, 2024): Can you update to 1.22.1 (has just been tagged, and the blog post hasn't even been written yet), as it has some improvements around this. As well, are you using minio (or s3) to store packages?
Author
Owner

@CPlusPlus17 commented on GitHub (Jul 5, 2024):

I updated to 1.22.1, but the problem is still the same. We'are using selfhosted minio as storage. (Clean logs)

2024/07/05 06:39:24 ...eb/routing/logger.go:68:func1() [W] router: slow      PATCH /v2/xxx/dotnet/build/blobs/uploads/ht5ptpwgw1drkjmxh0gcpxwox for 10.244.2.206:56532, elapsed 3123.9ms @ packages/api.go:700(packages.ContainerRoutes.func2.2)
2024/07/05 06:40:21 ...ntainer/container.go:90:apiError() [E] unexpected EOF
2024/07/05 06:40:21 ...eb/routing/logger.go:102:func1() [I] router: completed PATCH /v2/xxx/dotnet/build/blobs/uploads/ht5ptpwgw1drkjmxh0gcpxwox for 10.244.2.206:56532, 500 Internal Server Error in 60199.3ms @ packages/api.go:700(packages.ContainerRoutes.func2.2)
@CPlusPlus17 commented on GitHub (Jul 5, 2024): I updated to 1.22.1, but the problem is still the same. We'are using selfhosted minio as storage. (Clean logs) ``` 2024/07/05 06:39:24 ...eb/routing/logger.go:68:func1() [W] router: slow PATCH /v2/xxx/dotnet/build/blobs/uploads/ht5ptpwgw1drkjmxh0gcpxwox for 10.244.2.206:56532, elapsed 3123.9ms @ packages/api.go:700(packages.ContainerRoutes.func2.2) 2024/07/05 06:40:21 ...ntainer/container.go:90:apiError() [E] unexpected EOF 2024/07/05 06:40:21 ...eb/routing/logger.go:102:func1() [I] router: completed PATCH /v2/xxx/dotnet/build/blobs/uploads/ht5ptpwgw1drkjmxh0gcpxwox for 10.244.2.206:56532, 500 Internal Server Error in 60199.3ms @ packages/api.go:700(packages.ContainerRoutes.func2.2) ```
Author
Owner

@CPlusPlus17 commented on GitHub (Jul 12, 2024):

I was able to solve this, it was an infra issue. We had very slow nfs write performances, which caused EOF in the storage class configuration while reaching a timeout. Sorry for bothering.

@CPlusPlus17 commented on GitHub (Jul 12, 2024): I was able to solve this, it was an infra issue. We had very slow nfs write performances, which caused EOF in the storage class configuration while reaching a timeout. Sorry for bothering.
Author
Owner

@techknowlogick commented on GitHub (Jul 12, 2024):

@CPlusPlus17 no worries. Glad to hear you were able to find what it was.

@techknowlogick commented on GitHub (Jul 12, 2024): @CPlusPlus17 no worries. Glad to hear you were able to find what it was.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#13240