mirror of
https://github.com/go-gitea/gitea.git
synced 2026-03-12 02:24:21 -05:00
Registry: Error response from daemon: missing signature key after upgrade to 1.21.0 #12046
Closed
opened 2025-11-02 09:55:59 -06:00 by GiteaMirror
·
24 comments
No Branch/Tag Specified
main
release/v1.25
release/v1.24
release/v1.23
release/v1.22
release/v1.21
release/v1.20
release/v1.19
release/v1.18
release/v1.17
release/v1.16
release/v1.15
release/v1.14
release/v1.13
release/v1.12
release/v1.11
release/v1.10
release/v1.9
release/v1.8
v1.25.3
v1.25.2
v1.25.1
v1.25.0
v1.24.7
v1.25.0-rc0
v1.26.0-dev
v1.24.6
v1.24.5
v1.24.4
v1.24.3
v1.24.2
v1.24.1
v1.24.0
v1.23.8
v1.24.0-rc0
v1.25.0-dev
v1.23.7
v1.23.6
v1.23.5
v1.23.4
v1.23.3
v1.23.2
v1.23.1
v1.23.0
v1.23.0-rc0
v1.24.0-dev
v1.22.6
v1.22.5
v1.22.4
v1.22.3
v1.22.2
v1.22.1
v1.22.0
v1.23.0-dev
v1.22.0-rc1
v1.21.11
v1.22.0-rc0
v1.21.10
v1.21.9
v1.21.8
v1.21.7
v1.21.6
v1.21.5
v1.21.4
v1.21.3
v1.21.2
v1.20.6
v1.21.1
v1.21.0
v1.21.0-rc2
v1.21.0-rc1
v1.20.5
v1.22.0-dev
v1.21.0-rc0
v1.20.4
v1.20.3
v1.20.2
v1.20.1
v1.20.0
v1.19.4
v1.21.0-dev
v1.20.0-rc2
v1.20.0-rc1
v1.20.0-rc0
v1.19.3
v1.19.2
v1.19.1
v1.19.0
v1.19.0-rc1
v1.20.0-dev
v1.19.0-rc0
v1.18.5
v1.18.4
v1.18.3
v1.18.2
v1.18.1
v1.18.0
v1.17.4
v1.18.0-rc1
v1.19.0-dev
v1.18.0-rc0
v1.17.3
v1.17.2
v1.17.1
v1.17.0
v1.17.0-rc2
v1.16.9
v1.17.0-rc1
v1.18.0-dev
v1.16.8
v1.16.7
v1.16.6
v1.16.5
v1.16.4
v1.16.3
v1.16.2
v1.16.1
v1.16.0
v1.15.11
v1.17.0-dev
v1.16.0-rc1
v1.15.10
v1.15.9
v1.15.8
v1.15.7
v1.15.6
v1.15.5
v1.15.4
v1.15.3
v1.15.2
v1.15.1
v1.14.7
v1.15.0
v1.15.0-rc3
v1.14.6
v1.15.0-rc2
v1.14.5
v1.16.0-dev
v1.15.0-rc1
v1.14.4
v1.14.3
v1.14.2
v1.14.1
v1.14.0
v1.13.7
v1.14.0-rc2
v1.13.6
v1.13.5
v1.14.0-rc1
v1.15.0-dev
v1.13.4
v1.13.3
v1.13.2
v1.13.1
v1.13.0
v1.12.6
v1.13.0-rc2
v1.14.0-dev
v1.13.0-rc1
v1.12.5
v1.12.4
v1.12.3
v1.12.2
v1.12.1
v1.11.8
v1.12.0
v1.11.7
v1.12.0-rc2
v1.11.6
v1.12.0-rc1
v1.13.0-dev
v1.11.5
v1.11.4
v1.11.3
v1.10.6
v1.12.0-dev
v1.11.2
v1.10.5
v1.11.1
v1.10.4
v1.11.0
v1.11.0-rc2
v1.10.3
v1.11.0-rc1
v1.10.2
v1.10.1
v1.10.0
v1.9.6
v1.9.5
v1.10.0-rc2
v1.11.0-dev
v1.10.0-rc1
v1.9.4
v1.9.3
v1.9.2
v1.9.1
v1.9.0
v1.9.0-rc2
v1.10.0-dev
v1.9.0-rc1
v1.8.3
v1.8.2
v1.8.1
v1.8.0
v1.8.0-rc3
v1.7.6
v1.8.0-rc2
v1.7.5
v1.8.0-rc1
v1.9.0-dev
v1.7.4
v1.7.3
v1.7.2
v1.7.1
v1.7.0
v1.7.0-rc3
v1.6.4
v1.7.0-rc2
v1.6.3
v1.7.0-rc1
v1.7.0-dev
v1.6.2
v1.6.1
v1.6.0
v1.6.0-rc2
v1.5.3
v1.6.0-rc1
v1.6.0-dev
v1.5.2
v1.5.1
v1.5.0
v1.5.0-rc2
v1.5.0-rc1
v1.5.0-dev
v1.4.3
v1.4.2
v1.4.1
v1.4.0
v1.4.0-rc3
v1.4.0-rc2
v1.3.3
v1.4.0-rc1
v1.3.2
v1.3.1
v1.3.0
v1.3.0-rc2
v1.3.0-rc1
v1.2.3
v1.2.2
v1.2.1
v1.2.0
v1.2.0-rc3
v1.2.0-rc2
v1.1.4
v1.2.0-rc1
v1.1.3
v1.1.2
v1.1.1
v1.1.0
v1.0.2
v1.0.1
v1.0.0
v0.9.99
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
Mirrored from GitHub Pull Request
Milestone
No items
No Milestone
Projects
Clear projects
No project
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/gitea#12046
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @RogerSik on GitHub (Nov 19, 2023).
Description
Since the upgrade to 1.21.0 the docker image build succeed but the docker pull fails with
I suspect this is because of 1.21.0 because before the upgrade I din't have this problem. I can't
Test build locally with docker client
I builded the image now on three different ways
I deleted the :development image and rebuild it the same error happens. When using an unused tag example test1 same error.
When downloading a ready image like alpine:latest
Gitea Version
1.21.0
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
1.21.0
Operating System
Ubuntu 22.04 / K3s
How are you running Gitea?
Gitea with the official docker container running on K3S.
Database
PostgreSQL
@KN4CK3R commented on GitHub (Nov 20, 2023):
What is a
signature keyin the docker context? There wasn't a significant change on the package code for 1.21. Could you show the router log for the failing docker command?@RogerSik commented on GitHub (Nov 21, 2023):
I'm also curious. Specially because push is working from Gitea CI or local docker client, but not the pull. On the Demo Site it is working fine.
What is the router log? How do i get them?
I created a test pipeline which can also be visited: https://gitea.sikorski.cloud/RogerSik/registry-test/actions/runs/1/jobs/0
Somehow it needs to be the client? Because now i got the same error but i wasn't fast enough to assign this image to this repository. So this repository was / is empty and the pull error was the message.
Checking now my reverse proxy traefik. I get following message when using the pull command
Will troubleshooting forward in this direction.
@EternalDeiwos commented on GitHub (Nov 24, 2023):
@KN4CK3R I am experiencing the same problem with a very similar environment setup as Roger; please see my logs below:
Logs
In this case, I am attempting to run a CI pipeline for
example-org/example-repothat pulls a base image fromother-org/image-name.@RogerSik commented on GitHub (Nov 25, 2023):
I did now exclued traefik and connected directly with Gitea. "Interesting" part is that the problem is still there.
So something Gitea relevated but with my configuration because of the try.gitea.io it did work.
When executing
appears in the container log.
But I was before authenticiated with docker login (push was also successfull). @EternalDeiwos has the same message.
@evanreichard commented on GitHub (Nov 25, 2023):
I think this is related at least in some regard, as it started happening after upgrading to 1.21.0 as well.
Previously I was using my users credentials. After generating an application specific token, I was able to login with that token and all was good again.
Not sure if it's relevant, but my account uses 2FA.
Edit: Yes, it was relevant:
https://github.com/go-gitea/gitea/issues/27819
@EternalDeiwos commented on GitHub (Nov 26, 2023):
What I find odd here is I am getting the expected 307 that redirects to my S3 storage containing the image layers… and the error message is complaining about a
missing signature key. I don’t think this is authentication related but rather a server-side problem.That said, I’ll play with the auth and see if I can at least rule it out as related.
@RogerSik commented on GitHub (Nov 26, 2023):
@evanreichard I also have 2FA enabled but i was using the application token. For testing I disabled 2FA and re-logged in with the normal user passwort. Sadly same error message:
The strange thing (from begin of this issue) is that uploads seems to working (no errors) but the pull is failing.
@RogerSik commented on GitHub (Nov 26, 2023):
Good clue. Because of that I tried now Gitea Packages with local storages instead of the currently setup with Minio.
Pull is now working. :D :/
@EternalDeiwos commented on GitHub (Nov 27, 2023):
I've tested with fresh admin-level credentials and I am pretty sure it is not auth related. Given local is working for Roger, I'd say that probably indicates something to do with #25543.
Edit: also no change for 1.21.1.
@EternalDeiwos commented on GitHub (Nov 27, 2023):
Further testing; after disabling
SERVE_DIRECTfor packages I am now able to use the registry again as normal.@RogerSik commented on GitHub (Nov 27, 2023):
Can confirm here the same. 🙌
@KN4CK3R commented on GitHub (Nov 28, 2023):
Thanks for testing. Maybe a newer docker client version is more strict with the HTTP headers of the content. With
SERVE_DIRECTthe content response looks like thisBut the response from
url-where-the-blob-isdoes not contain the docker headers.@KN4CK3R commented on GitHub (Nov 28, 2023):
May still be wrong because you can't add arbitrary headers to the response:
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
@KN4CK3R commented on GitHub (Nov 28, 2023):
Looks like the error message is from here:
aabc10ec26/jsonsign.go (L23-L25)But that is unrelated to the serve direct setting...?
The code is only used in Docker schema1 manifests which were removed in 08/23:
0742b56677 (diff-f65d932ec1e10721d44ebd79b23a64cc70599bb6f24db45b48a9f275c9332147L146)What Docker version do you use? Does some setting enforces the usage of schema1 (which is not supported in Gitea)?
@EternalDeiwos commented on GitHub (Nov 28, 2023):
I'm running Docker CE
24.0.6locally.Not as far as I have explicitly configured or am aware of...
@jessielw commented on GitHub (Dec 11, 2023):
I've changed nothing other than the update to 1.21. I can push to packages but cannot pull. I tried to re login to docker and it's getting
Error response from daemon: ............ connection refusedeven though it's the same setup as before.Tried to downgrade but the databases are not compatible. Any work arounds until a fix is out?
Note: out of curiosity I pulled down the
nightlydocker build to see if the fix was on that, but alas it was not. Same issue and now with the database upgrade I'm stuck on thenightlyrelease haha@lunny commented on GitHub (Dec 12, 2023):
Maybe you need to check whether your
ROOT_URLis correct.@jessielw commented on GitHub (Dec 12, 2023):
I'll double-check it in the morning, but it was for the last several months with no configuration changes. Only the last update broke pulling from package registries for me.
@lunny commented on GitHub (Dec 12, 2023):
Maybe you have a different problem as this one? If that, you can create a new issue with more description.
@jessielw commented on GitHub (Dec 12, 2023):
It's possible, actually, although it seems like it's similar.
I'll produce the issue again tomorrow and post a more detailed bug in a separate issue.
@EternalDeiwos commented on GitHub (Dec 12, 2023):
@KN4CK3R I have just seen the needs feedback tag, looking for anything specific?
@wytzevanderploeg commented on GitHub (Feb 23, 2024):
I had this problem, it turned out I had a rather old version of docker-ce running.
Docker version:
Docker version 17.09.1-ce, build 19e2cf6After upgrading to the latest version (Docker version 25.0.3, build 4debf41) I no longer had any problems pulling/building the image.
@itzaname commented on GitHub (Jul 4, 2024):
I'm seeing this using Amazon S3 with
SERVE_DIRECTenabled. OnceSERVE_DIRECTis disabled things work as normal. Gitea1.22.0.Podman:
Docker:
@meruiden commented on GitHub (Jul 19, 2024):
same here, minio over blackblaze b2, SERVE_DIRECT=false fixes it