github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-11 17:46:29 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

Error 500 when pushing a docker multiarch image #13885

New Issue
Closed
opened 2025-11-02 10:56:08 -06:00 by GiteaMirror · 9 comments
GiteaMirror commented 2025-11-02 10:56:08 -06:00
Owner
Copy Link

Originally created by @Skydust2B on GitHub (Dec 27, 2024).

Description

Ever since I upgraded to gitea 1.23.0-rc0, I am not able to push to the docker registry multi-arch images, with simple image also randomly failing to push.
Everything was working fine before I upgraded the docker image when the gitea instance suddenly started giving me "package version already exists" errors.

I have joined logs of me running the command on demo.gitea.com and my own server. Let me know if you need more infos.

Gitea Version

1.23.0-rc0

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

https://gist.github.com/Skydust2B/cb365129eec75cac93b1d927d26e5ff4

FROM public.ecr.aws/docker/library/node:22-alpine3.21

RUN apk add --no-cache bash git curl jq

# Vulnerability fixes
RUN npm install -g npm@latest

ENTRYPOINT ["/bin/bash"]


$ docker build . -t <serverurl>/skydust/multiarchtest:latest --platform linux/amd64,linux/arm64 --push

...

ERROR: failed commit on ref "manifest-sha256:1201f32e3f851ed5eb8e8f9e8df297e7c6b6aa64c849fe4de75a2f95872fcc34": unexpected status from PUT request to https://<serverurl>//v2/skydust/multiarchtest/manifests/latest: 500 Internal Server Error

...

2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:ea542b86a0afcd40ccb12aa90fce8a8cf7d8b9798e8ae5ec545087bb
fa70600a for 10.0.0.118:48888, 200 OK in 113.4ms @ container/container.go:501(container.HeadBlob)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:274663461bd74df9f025b6e29c963c0583cb98dbb8641dd609119431
fe17d4a4 for [fd00::46]:35924, 200 OK in 125.7ms @ container/container.go:501(container.HeadBlob)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:796da77e13076b9cd79d52eb61ff3b36816a68ed04bc7f701702679d
c2909442 for 10.0.0.118:48878, 200 OK in 122.9ms @ container/container.go:501(container.HeadBlob)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:35996, 200 OK in 6.2ms @ container/co
ntainer.go:639(container.HeadManifest)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36008, 200 OK in 6.5ms @ container/co
ntainer.go:639(container.HeadManifest)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48916, 200 OK in 27.2ms @ container/c
ontainer.go:639(container.HeadManifest)
2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48920, 200 OK in 40.5ms @ container/c
ontainer.go:639(container.HeadManifest)
2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists
2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists
2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists
2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists
2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36024, 500 Internal Server Error in 52
.7ms @ container/container.go:554(container.UploadManifest)
2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists
2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48928, 500 Internal Server Error in 76
.4ms @ container/container.go:554(container.UploadManifest)
2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists
2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48930, 500 Internal Server Error in 41
.3ms @ container/container.go:554(container.UploadManifest)
2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36010, 201 Created in 90.0ms @ contain
er/container.go:554(container.UploadManifest)

Git Version

2.45.2

Operating System

Docker

How are you running Gitea?

I'm running gitea inside kubernetes using the official gitea/gitea:1.23.0-rc0 container.

Database

PostgreSQL

Originally created by @Skydust2B on GitHub (Dec 27, 2024). ### Description Ever since I upgraded to gitea 1.23.0-rc0, I am not able to push to the docker registry multi-arch images, with simple image also randomly failing to push. Everything was working fine before I upgraded the docker image when the gitea instance suddenly started giving me "package version already exists" errors. I have joined logs of me running the command on demo.gitea.com and my own server. Let me know if you need more infos. ### Gitea Version 1.23.0-rc0 ### Can you reproduce the bug on the Gitea demo site? Yes ### Log Gist https://gist.github.com/Skydust2B/cb365129eec75cac93b1d927d26e5ff4 ``` FROM public.ecr.aws/docker/library/node:22-alpine3.21 RUN apk add --no-cache bash git curl jq # Vulnerability fixes RUN npm install -g npm@latest ENTRYPOINT ["/bin/bash"] ``` ``` $ docker build . -t <serverurl>/skydust/multiarchtest:latest --platform linux/amd64,linux/arm64 --push ... ERROR: failed commit on ref "manifest-sha256:1201f32e3f851ed5eb8e8f9e8df297e7c6b6aa64c849fe4de75a2f95872fcc34": unexpected status from PUT request to https://<serverurl>//v2/skydust/multiarchtest/manifests/latest: 500 Internal Server Error ... 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:ea542b86a0afcd40ccb12aa90fce8a8cf7d8b9798e8ae5ec545087bb fa70600a for 10.0.0.118:48888, 200 OK in 113.4ms @ container/container.go:501(container.HeadBlob) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:274663461bd74df9f025b6e29c963c0583cb98dbb8641dd609119431 fe17d4a4 for [fd00::46]:35924, 200 OK in 125.7ms @ container/container.go:501(container.HeadBlob) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/blobs/sha256:796da77e13076b9cd79d52eb61ff3b36816a68ed04bc7f701702679d c2909442 for 10.0.0.118:48878, 200 OK in 122.9ms @ container/container.go:501(container.HeadBlob) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:35996, 200 OK in 6.2ms @ container/co ntainer.go:639(container.HeadManifest) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36008, 200 OK in 6.5ms @ container/co ntainer.go:639(container.HeadManifest) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48916, 200 OK in 27.2ms @ container/c ontainer.go:639(container.HeadManifest) 2024/12/27 20:49:47 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48920, 200 OK in 40.5ms @ container/c ontainer.go:639(container.HeadManifest) 2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists 2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists 2024/12/27 20:49:48 ...ontainer/manifest.go:364:createPackageAndVersion() [E] Error inserting package: package version already exists 2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists 2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36024, 500 Internal Server Error in 52 .7ms @ container/container.go:554(container.UploadManifest) 2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists 2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48928, 500 Internal Server Error in 76 .4ms @ container/container.go:554(container.UploadManifest) 2024/12/27 20:49:48 ...ntainer/container.go:92:apiError() [E] package version already exists 2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for 10.0.0.118:48930, 500 Internal Server Error in 41 .3ms @ container/container.go:554(container.UploadManifest) 2024/12/27 20:49:48 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/skydust/multiarchtest/manifests/latest for [fd00::46]:36010, 201 Created in 90.0ms @ contain er/container.go:554(container.UploadManifest) ``` ### Git Version 2.45.2 ### Operating System Docker ### How are you running Gitea? I'm running gitea inside kubernetes using the official gitea/gitea:1.23.0-rc0 container. ### Database PostgreSQL
GiteaMirror added the topic/packagestype/bugissue/needs-feedback labels 2025-11-02 10:56:08 -06:00
GiteaMirror commented 2025-11-02 10:56:10 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Dec 30, 2024):

Some related:

But not completely fixed

@KN4CK3R

@wxiaoguang commented on GitHub (Dec 30, 2024): Some related: * #32022 * #31860 * #21862 But not completely fixed @KN4CK3R
GiteaMirror commented 2025-11-02 10:56:10 -06:00
Author
Owner
Copy Link

@hiifong commented on GitHub (Dec 30, 2024):

I recommend you use docker buildx to build multi-architecture images

@hiifong commented on GitHub (Dec 30, 2024): I recommend you use docker buildx to build multi-architecture images
GiteaMirror commented 2025-11-02 10:56:10 -06:00
Author
Owner
Copy Link

@Skydust2B commented on GitHub (Dec 30, 2024):

I recommend you use docker buildx to build multi-architecture images

I should have mentioned, I'm using docker-desktop on a mac and it's using buildkit by default. My CI has the same errors and directly uses docker buildx build.

@Skydust2B commented on GitHub (Dec 30, 2024): > I recommend you use docker buildx to build multi-architecture images I should have mentioned, I'm using docker-desktop on a mac and it's using buildkit by default. My CI has the same errors and directly uses `docker buildx build`.
GiteaMirror commented 2025-11-02 10:56:10 -06:00
Author
Owner
Copy Link

@hiifong commented on GitHub (Jan 5, 2025):

Since I can't reproduce this issue locally, it would be great if you could use Wireshark to capture the push network request and provide more detailed logs (trace level).

@hiifong commented on GitHub (Jan 5, 2025): Since I can't reproduce this issue locally, it would be great if you could use Wireshark to capture the push network request and provide more detailed logs (trace level).
GiteaMirror commented 2025-11-02 10:56:10 -06:00
Author
Owner
Copy Link

@Skydust2B commented on GitHub (Jan 5, 2025):

I tried to capture everything with wireshark, but I'm not sure how useful it will be considering I wasn't able to decrypt the tls packets.
Sorry if this is kind of messy, but here is what I did:
I went on a windows machine, installed docker desktop, set SSLKEYLOGFILE to a usable path. I then enabled containerd, restarted docker desktop, and ran the same command as above for demo.gitea.com (which seemed to have some hiccups with the login token).
I made wireshark follow the path to my log file, disabled my ipv6, and captured all packets going to the ipv4 of demo.gitea.com while building and pushing. (EDIT: Added a better file below, removed the link for clarity)

Because you couldn't reproduce it, I've tried using my mobile network on my mac and pushed again, only to fail to the same error.

EDIT:
I used a proxy, tried a docker push of the same built image, and injected the secrets into the dump file:
capture.pcapng.zip

@Skydust2B commented on GitHub (Jan 5, 2025): I tried to capture everything with wireshark, but I'm not sure how useful it will be considering I wasn't able to decrypt the tls packets. Sorry if this is kind of messy, but here is what I did: I went on a windows machine, installed docker desktop, set SSLKEYLOGFILE to a usable path. I then enabled containerd, restarted docker desktop, and ran the same command as above for demo.gitea.com (which seemed to have some hiccups with the login token). I made wireshark follow the path to my log file, disabled my ipv6, and captured all packets going to the ipv4 of demo.gitea.com while building and pushing. (EDIT: Added a better file below, removed the link for clarity) Because you couldn't reproduce it, I've tried using my mobile network on my mac and pushed again, only to fail to the same error. **EDIT:** I used a proxy, tried a docker push of the same built image, and injected the secrets into the dump file: [capture.pcapng.zip](https://github.com/user-attachments/files/18312310/capture.pcapng.zip)
GiteaMirror commented 2025-11-02 10:56:11 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Jan 6, 2025):

I see the problem, will fix it

@wxiaoguang commented on GitHub (Jan 6, 2025): I see the problem, will fix it
GiteaMirror commented 2025-11-02 10:56:11 -06:00
Author
Owner
Copy Link

@wxiaoguang commented on GitHub (Jan 6, 2025):

could you try 1.23-nightly? I think it should have been fixed there

@wxiaoguang commented on GitHub (Jan 6, 2025): could you try 1.23-nightly? I think it should have been fixed there * https://dl.gitea.com/gitea/1.23-nightly/ * https://hub.docker.com/r/gitea/gitea/tags?name=1.23-nightly
GiteaMirror commented 2025-11-02 10:56:11 -06:00
Author
Owner
Copy Link

@hiifong commented on GitHub (Jan 6, 2025):

image
It works well for me here.

@hiifong commented on GitHub (Jan 6, 2025): ![image](https://github.com/user-attachments/assets/c3bd3cb0-b587-4280-a48f-7b5585533fa1) It works well for me here.
GiteaMirror commented 2025-11-02 10:56:11 -06:00
Author
Owner
Copy Link

@Skydust2B commented on GitHub (Jan 6, 2025):

I do not have any issues using the last nightly 👍Thank you for the speedy resolution

@Skydust2B commented on GitHub (Jan 6, 2025): I do not have any issues using the last nightly 👍Thank you for the speedy resolution
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label issue/needs-feedback topic/packages type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#13885
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?