Builtin Container Registry: Manifests 404 Not Found #13397

Closed
opened 2025-11-02 10:41:06 -06:00 by GiteaMirror · 19 comments
Owner

Originally created by @fekle on GitHub (Aug 18, 2024).

Description

Hi there!

I'm in the process of migrating from GitLab to Gitea, and so far I'm very happy with Gitea - migration was smooth, and things like Actions work great!

Now I wanted to also use Gitea as my Docker Registry to host my container images, but I'm stuck there. Pushing image layers works fine, but the final manifest push fails with 404. I think this issue is related to existing issues such as #31802 and #31644, but I didn't want to spam those issues.

First of all - to make sure that this is not related to my migrated groups and repositories - I created a new Organization called "test" and a project called "test" in my Gitea instance. I also enabled debug logging.

Then, when I'm trying to push a minimal image from my computer, I get this:

> cat Dockerfile
FROM debian:12
CMD bash

> docker build -t GITEA_HOST/test/test:test .
[+] Building 1.1s (5/5) FINISHED
 => [internal] load build definition from Dockerfile
[...]
 => => naming to GITEA_HOST/test/test:test
 => => unpacking to GITEA_HOST/test/test:test

> docker push GITEA_HOST/test/test:test
The push refers to repository [GITEA_HOST/test/test]
7a351543ef85: Pushed
7b24851aa36d: Pushed
failed commit on ref "index-sha256:[...]": unexpected status from PUT request to https://GITEA_HOST/v2/test/test/manifests/test: 404 Not Found

The same thing happens with an Actions workflow and BuildKit.

On the Gitea server side, here are the startup logs:

Server listening on :: port 22.
Server listening on 0.0.0.0 port 22.
2024/08/18 11:33:09 cmd/web.go:242:runWeb() [I] Starting Gitea on PID: 19
2024/08/18 11:33:09 cmd/web.go:111:showWebStartupMessage() [I] Gitea version: 1.22.1 built with GNU Make 4.4.1, go1.22.5 : bindata, timetzdata, sqlite, sqlite_unlock_notify
[...]
2024/08/18 11:33:09 cmd/web.go:304:listen() [I] Listen: http://0.0.0.0:3000
2024/08/18 11:33:09 cmd/web.go:308:listen() [I] AppURL(ROOT_URL): https://GITEA_HOST/
2024/08/18 11:33:09 cmd/web.go:311:listen() [I] LFS server enabled
2024/08/18 11:33:09 ...s/graceful/server.go:50:NewServer() [I] Starting new Web server: tcp:0.0.0.0:3000 on PID: 19
2024/08/18 11:33:09 ...s/graceful/server.go:63:func1() [D] Starting server on tcp:0.0.0.0:3000 (PID: 19)

And this is the output when pushing aforementioned image:

2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.0ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/token for REDACTED:0, 404 Not Found in 0.0ms @ container/container.go:172(container.AuthenticateNotImplemented)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed GET /v2/token?scope=%2A&scope=repository%3Atest%2Ftest%3Apull%2Cpush&service=container_registry for REDACTED:0, 200 OK in 35.2ms @ container/container.go:149(container.Authenticate)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 1.6ms @ container/container.go:486(container.HeadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 1.7ms @ container/container.go:486(container.HeadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 2.1ms @ container/container.go:486(container.HeadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 2.1ms @ container/container.go:486(container.HeadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.6ms @ container/container.go:218(container.InitiateUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.7ms @ container/container.go:218(container.InitiateUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.9ms @ container/container.go:218(container.InitiateUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.9ms @ container/container.go:218(container.InitiateUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/jtzjr5ghfg6qfr6heaqmg0mlh?digest=sha256%3A[...] for REDACTED:0, 201 Created in 2.1ms @ container/container.go:373(container.EndUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/49vudymsvznarmnzzeaumpg7g?digest=sha256%3A[...] for REDACTED:0, 201 Created in 3.4ms @ container/container.go:373(container.EndUploadBlob)
2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/ada4izfsmp6vozywuvtpwuq3t?digest=sha256%3A[...] for REDACTED:0, 201 Created in 4.1ms @ container/container.go:373(container.EndUploadBlob)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/dnvgyyksiuqixcv3ajfxygxfz?digest=sha256%3A[...] for REDACTED:0, 201 Created in 496.1ms @ container/container.go:373(container.EndUploadBlob)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.9ms @ container/container.go:624(container.HeadManifest)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.9ms @ container/container.go:624(container.HeadManifest)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 201 Created in 3.3ms @ container/container.go:539(container.UploadManifest)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 201 Created in 4.9ms @ container/container.go:539(container.UploadManifest)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 200 OK in 0.5ms @ container/container.go:624(container.HeadManifest)
2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.6ms @ container/container.go:539(container.UploadManifest)

Strangely, I'm seeing 401 Unauthorized errors there; which I'm not seeing in the Docker client.
I am logged into the registry via docker login with my administrator user. I also tried creating a new user, which I added to the aforementioned test group, but the same issue persisted.
I also made sure that things like filesystem permissions as well as the reverse proxy configuration are correct.

For some context, here Is my app.ini

APP_NAME = REDACTED
RUN_MODE = prod
RUN_USER = git
WORK_PATH = /data/gitea

[repository]
ROOT = /data/git/repositories

[repository.local]
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo

[repository.upload]
TEMP_PATH = /data/gitea/uploads

[server]
APP_DATA_PATH = /data/gitea
DOMAIN = GITEA_HOST
SSH_DOMAIN = GITEA_HOST
HTTP_PORT = 3000
ROOT_URL = https://GITEA_HOST/
DISABLE_SSH = false
SSH_PORT = 222
SSH_LISTEN_PORT = 22
LFS_START_SERVER = true
LFS_JWT_SECRET = REDACTED
OFFLINE_MODE = true

[database]
PATH = /data/gitea/gitea.db
DB_TYPE = sqlite3
HOST = localhost:3306
NAME = gitea
USER = root
PASSWD =
LOG_SQL = false
SCHEMA =
SSL_MODE = disable
SQLITE_JOURNAL_MODE = WAL

[indexer]
REPO_INDEXER_ENABLED = true
REPO_INDEXER_PATH = /data/gitea/indexers/repos.bleve
ISSUE_INDEXER_ENABLED = true
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

[session]
PROVIDER_CONFIG = /data/gitea/sessions
PROVIDER = file

[picture]
AVATAR_UPLOAD_PATH = /data/gitea/avatars
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars

[attachment]
PATH = /data/gitea/attachments

[log]
MODE = console
LEVEL = debug
ROOT_PATH = /data/gitea/log

[security]
INSTALL_LOCK = true
SECRET_KEY =
REVERSE_PROXY_LIMIT = 1
REVERSE_PROXY_TRUSTED_PROXIES = *
INTERNAL_TOKEN = REDACTED
PASSWORD_HASH_ALGO = argon2

[service]
DISABLE_REGISTRATION = true
REQUIRE_SIGNIN_VIEW = true
REGISTER_EMAIL_CONFIRM = false
ENABLE_NOTIFY_MAIL = false
ALLOW_ONLY_EXTERNAL_REGISTRATION = false
ENABLE_CAPTCHA = false
DEFAULT_KEEP_EMAIL_PRIVATE = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING = true
NO_REPLY_ADDRESS = noreply.localhost

[lfs]
PATH = /data/git/lfs

[mailer]
ENABLED = false

[openid]
ENABLE_OPENID_SIGNIN = false
ENABLE_OPENID_SIGNUP = false

[cron.update_checker]
ENABLED = false

[repository.pull-request]
DEFAULT_MERGE_STYLE = merge

[repository.signing]
DEFAULT_TRUST_MODEL = committer

[oauth2]
JWT_SECRET = REDACTED

[actions]
ENABLED = true

[packages]
ENABLED = true

The reverse proxy configuration:

GITEA_HOST {
	reverse_proxy http://gitea:3000
}

And the docker compose config for Gitea:

name: gitea

services:
  gitea:
    container_name: gitea
    image: gitea/gitea:1
    pull_policy: always
    restart: always
    init: true
    hostname: "GITEA_HOST"
    networks:
      - gitea
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: "4G"
    environment:
      USER_UID: "3000"
      USER_GID: "3000"
    ports:
      - "222:22"
    volumes:
      - "REDACTED:/data:rw"

networks:
  gitea:
    name: gitea
    external: true

I'm kind of at a loss here and would be very grateful if anyone could point me in the right direction :-)

Gitea Version

1.22.1

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.45.2

Operating System

Docker image: gitea/gitea:1; Host: Debian 12.6 6.1.0-23-cloud-amd64

How are you running Gitea?

Hosting myself, on Docker 27.0.3 behind Caddy v2.8.4.

Database

SQLite

Originally created by @fekle on GitHub (Aug 18, 2024). ### Description Hi there! I'm in the process of migrating from GitLab to Gitea, and so far I'm very happy with Gitea - migration was smooth, and things like Actions work great! Now I wanted to also use Gitea as my Docker Registry to host my container images, but I'm stuck there. Pushing image layers works fine, but the final manifest push fails with 404. I think this issue is related to existing issues such as #31802 and #31644, but I didn't want to spam those issues. First of all - to make sure that this is not related to my migrated groups and repositories - I created a new Organization called "test" and a project called "test" in my Gitea instance. I also enabled debug logging. Then, when I'm trying to push a minimal image from my computer, I get this: ```shell > cat Dockerfile FROM debian:12 CMD bash > docker build -t GITEA_HOST/test/test:test . [+] Building 1.1s (5/5) FINISHED => [internal] load build definition from Dockerfile [...] => => naming to GITEA_HOST/test/test:test => => unpacking to GITEA_HOST/test/test:test > docker push GITEA_HOST/test/test:test The push refers to repository [GITEA_HOST/test/test] 7a351543ef85: Pushed 7b24851aa36d: Pushed failed commit on ref "index-sha256:[...]": unexpected status from PUT request to https://GITEA_HOST/v2/test/test/manifests/test: 404 Not Found ``` The same thing happens with an Actions workflow and BuildKit. On the Gitea server side, here are the startup logs: ```shell Server listening on :: port 22. Server listening on 0.0.0.0 port 22. 2024/08/18 11:33:09 cmd/web.go:242:runWeb() [I] Starting Gitea on PID: 19 2024/08/18 11:33:09 cmd/web.go:111:showWebStartupMessage() [I] Gitea version: 1.22.1 built with GNU Make 4.4.1, go1.22.5 : bindata, timetzdata, sqlite, sqlite_unlock_notify [...] 2024/08/18 11:33:09 cmd/web.go:304:listen() [I] Listen: http://0.0.0.0:3000 2024/08/18 11:33:09 cmd/web.go:308:listen() [I] AppURL(ROOT_URL): https://GITEA_HOST/ 2024/08/18 11:33:09 cmd/web.go:311:listen() [I] LFS server enabled 2024/08/18 11:33:09 ...s/graceful/server.go:50:NewServer() [I] Starting new Web server: tcp:0.0.0.0:3000 on PID: 19 2024/08/18 11:33:09 ...s/graceful/server.go:63:func1() [D] Starting server on tcp:0.0.0.0:3000 (PID: 19) ``` And this is the output when pushing aforementioned image: ```shell 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 401 Unauthorized in 0.0ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/token for REDACTED:0, 404 Not Found in 0.0ms @ container/container.go:172(container.AuthenticateNotImplemented) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed GET /v2/token?scope=%2A&scope=repository%3Atest%2Ftest%3Apull%2Cpush&service=container_registry for REDACTED:0, 200 OK in 35.2ms @ container/container.go:149(container.Authenticate) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 1.6ms @ container/container.go:486(container.HeadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 1.7ms @ container/container.go:486(container.HeadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 2.1ms @ container/container.go:486(container.HeadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDACTED:0, 404 Not Found in 2.1ms @ container/container.go:486(container.HeadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.6ms @ container/container.go:218(container.InitiateUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.7ms @ container/container.go:218(container.InitiateUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.9ms @ container/container.go:218(container.InitiateUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/test/test/blobs/uploads/ for REDACTED:0, 202 Accepted in 0.9ms @ container/container.go:218(container.InitiateUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/jtzjr5ghfg6qfr6heaqmg0mlh?digest=sha256%3A[...] for REDACTED:0, 201 Created in 2.1ms @ container/container.go:373(container.EndUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/49vudymsvznarmnzzeaumpg7g?digest=sha256%3A[...] for REDACTED:0, 201 Created in 3.4ms @ container/container.go:373(container.EndUploadBlob) 2024/08/18 11:41:32 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/ada4izfsmp6vozywuvtpwuq3t?digest=sha256%3A[...] for REDACTED:0, 201 Created in 4.1ms @ container/container.go:373(container.EndUploadBlob) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/blobs/uploads/dnvgyyksiuqixcv3ajfxygxfz?digest=sha256%3A[...] for REDACTED:0, 201 Created in 496.1ms @ container/container.go:373(container.EndUploadBlob) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.9ms @ container/container.go:624(container.HeadManifest) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.9ms @ container/container.go:624(container.HeadManifest) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 201 Created in 3.3ms @ container/container.go:539(container.UploadManifest) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 201 Created in 4.9ms @ container/container.go:539(container.UploadManifest) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test for REDACTED:0, 200 OK in 0.5ms @ container/container.go:624(container.HeadManifest) 2024/08/18 11:41:33 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test for REDACTED:0, 404 Not Found in 0.6ms @ container/container.go:539(container.UploadManifest) ``` Strangely, I'm seeing `401 Unauthorized` errors there; which I'm not seeing in the Docker client. I am logged into the registry via `docker login` with my administrator user. I also tried creating a new user, which I added to the aforementioned `test` group, but the same issue persisted. I also made sure that things like filesystem permissions as well as the reverse proxy configuration are correct. For some context, here Is my `app.ini` ```ini APP_NAME = REDACTED RUN_MODE = prod RUN_USER = git WORK_PATH = /data/gitea [repository] ROOT = /data/git/repositories [repository.local] LOCAL_COPY_PATH = /data/gitea/tmp/local-repo [repository.upload] TEMP_PATH = /data/gitea/uploads [server] APP_DATA_PATH = /data/gitea DOMAIN = GITEA_HOST SSH_DOMAIN = GITEA_HOST HTTP_PORT = 3000 ROOT_URL = https://GITEA_HOST/ DISABLE_SSH = false SSH_PORT = 222 SSH_LISTEN_PORT = 22 LFS_START_SERVER = true LFS_JWT_SECRET = REDACTED OFFLINE_MODE = true [database] PATH = /data/gitea/gitea.db DB_TYPE = sqlite3 HOST = localhost:3306 NAME = gitea USER = root PASSWD = LOG_SQL = false SCHEMA = SSL_MODE = disable SQLITE_JOURNAL_MODE = WAL [indexer] REPO_INDEXER_ENABLED = true REPO_INDEXER_PATH = /data/gitea/indexers/repos.bleve ISSUE_INDEXER_ENABLED = true ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve [session] PROVIDER_CONFIG = /data/gitea/sessions PROVIDER = file [picture] AVATAR_UPLOAD_PATH = /data/gitea/avatars REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars [attachment] PATH = /data/gitea/attachments [log] MODE = console LEVEL = debug ROOT_PATH = /data/gitea/log [security] INSTALL_LOCK = true SECRET_KEY = REVERSE_PROXY_LIMIT = 1 REVERSE_PROXY_TRUSTED_PROXIES = * INTERNAL_TOKEN = REDACTED PASSWORD_HASH_ALGO = argon2 [service] DISABLE_REGISTRATION = true REQUIRE_SIGNIN_VIEW = true REGISTER_EMAIL_CONFIRM = false ENABLE_NOTIFY_MAIL = false ALLOW_ONLY_EXTERNAL_REGISTRATION = false ENABLE_CAPTCHA = false DEFAULT_KEEP_EMAIL_PRIVATE = false DEFAULT_ALLOW_CREATE_ORGANIZATION = true DEFAULT_ENABLE_TIMETRACKING = true NO_REPLY_ADDRESS = noreply.localhost [lfs] PATH = /data/git/lfs [mailer] ENABLED = false [openid] ENABLE_OPENID_SIGNIN = false ENABLE_OPENID_SIGNUP = false [cron.update_checker] ENABLED = false [repository.pull-request] DEFAULT_MERGE_STYLE = merge [repository.signing] DEFAULT_TRUST_MODEL = committer [oauth2] JWT_SECRET = REDACTED [actions] ENABLED = true [packages] ENABLED = true ``` The reverse proxy configuration: ```caddyfile GITEA_HOST { reverse_proxy http://gitea:3000 } ``` And the `docker compose` config for Gitea: ```yaml name: gitea services: gitea: container_name: gitea image: gitea/gitea:1 pull_policy: always restart: always init: true hostname: "GITEA_HOST" networks: - gitea deploy: resources: limits: cpus: "4" memory: "4G" environment: USER_UID: "3000" USER_GID: "3000" ports: - "222:22" volumes: - "REDACTED:/data:rw" networks: gitea: name: gitea external: true ``` I'm kind of at a loss here and would be very grateful if anyone could point me in the right direction :-) ### Gitea Version 1.22.1 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version 2.45.2 ### Operating System Docker image: `gitea/gitea:1`; Host: `Debian 12.6 6.1.0-23-cloud-amd64` ### How are you running Gitea? Hosting myself, on `Docker 27.0.3` behind `Caddy v2.8.4`. ### Database SQLite
GiteaMirror added the topic/packagestype/bugissue/workaround labels 2025-11-02 10:41:06 -06:00
Author
Owner

@ChristopherHX commented on GitHub (Aug 18, 2024):

Please try docker build -t GITEA_HOST/test/test:test . --provenance=false, then push.

Based on my research this could have to do with build provenance aka https://docs.docker.com/build/attestations/slsa-provenance/.

My docker setup is old and has no containerd image store, therefore is my system probably unaffected as I can push. For me this seems to be turned off by default due to that reason and trying to enable it tells me that my setup cannot attest.

docker build -t GITEA_HOST/test/test:test . --attest type=provenance,mode=max
ERROR: Attestation is not supported for the docker driver.
Switch to a different driver, or turn on the containerd image store, and try again.
Learn more at https://docs.docker.com/go/attestations/

I would try to debug this next week with provenance enabled for a dind container or similar.

maybe I'm on a wrong path here.

@ChristopherHX commented on GitHub (Aug 18, 2024): Please try `docker build -t GITEA_HOST/test/test:test . --provenance=false`, then push. Based on my research this could have to do with build provenance aka <https://docs.docker.com/build/attestations/slsa-provenance/>. My docker setup is old and has no containerd image store, therefore is my system probably unaffected as I can push. For me this seems to be turned off by default due to that reason and trying to enable it tells me that my setup cannot attest. ``` docker build -t GITEA_HOST/test/test:test . --attest type=provenance,mode=max ERROR: Attestation is not supported for the docker driver. Switch to a different driver, or turn on the containerd image store, and try again. Learn more at https://docs.docker.com/go/attestations/ ``` I would try to debug this next week with provenance enabled for a dind container or similar. maybe I'm on a wrong path here.
Author
Owner

@fekle commented on GitHub (Aug 18, 2024):

Thanks for your swift reply!

I just tried building and pushing without provenance, that seems to work, with plain docker at least - thanks!
Pushing with Buildkit via earthly unfortunately still fails, see below.

> docker build --provenance=false -t GITEA_HOST/test/test:test-without-provenance .                                                               
[+] Building 0.5s (5/5) FINISHED                                                                                                          
 => [internal] load build definition from Dockerfile                                                                                                         
[...]                                                                                 
 => => unpacking to GITEA_HOST/test/test:test-without-provenance                                                                                       

> docker push GITEA_HOST/test/test:test-without-provenance                                                                                       
The push refers to repository [GITEA_HOST/test/test]
7b24851aa36d: Layer already exists
test-without-provenance: digest: sha256:[...] size: 503

(fun fact: at first the size: 503 confused me, thinking that now i'm getting an error 503 ...)

Gitea log:

2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 401 Unauthorized in 0.0ms @ container/container.go:126(container.ReqContainerAccess)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/token for REDCACTED:0, 404 Not Found in 0.0ms @ container/container.go:172(container.AuthenticateNotImplemented)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed GET /v2/token?scope=%2A&scope=repository%3Atest%2Ftest%3Apull%2Cpush&service=container_registry for REDCACTED:0, 200 OK in 57.1ms @ container/container.go:149(container.Authenticate)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 200 OK in 1.3ms @ container/container.go:486(container.HeadBlob)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 200 OK in 1.2ms @ container/container.go:486(container.HeadBlob)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test-without-provenance for REDCACTED:0, 404 Not Found in 0.5ms @ container/container.go:624(container.HeadManifest)
2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test-without-provenance for REDCACTED:0, 201 Created in 3.6ms @ container/container.go:539(container.UploadManifest)

Note that the first HEAD requests still results in 401, do you know what's up with that?
That's also what is now breaking my BuildKit/Earthly build:

unexpected status from HEAD request to https://GITEA_HOST/v2/test/test/blobs/sha256:[...]: 401 Unauthorized

Also, there is still a 404 error for that manifest endpoint, but the docker client is not complaining anymore - strange.

Please let me know if i can be of any help with debugging this issue further :)

@fekle commented on GitHub (Aug 18, 2024): Thanks for your swift reply! I just tried building and pushing without provenance, that seems to work, with plain docker at least - thanks! Pushing with Buildkit via earthly unfortunately still fails, see below. ```shell > docker build --provenance=false -t GITEA_HOST/test/test:test-without-provenance . [+] Building 0.5s (5/5) FINISHED => [internal] load build definition from Dockerfile [...] => => unpacking to GITEA_HOST/test/test:test-without-provenance > docker push GITEA_HOST/test/test:test-without-provenance The push refers to repository [GITEA_HOST/test/test] 7b24851aa36d: Layer already exists test-without-provenance: digest: sha256:[...] size: 503 ``` > (fun fact: at first the `size: 503` confused me, thinking that now i'm getting an error 503 ...) Gitea log: ```shell 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 401 Unauthorized in 0.1ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 401 Unauthorized in 0.0ms @ container/container.go:126(container.ReqContainerAccess) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed POST /v2/token for REDCACTED:0, 404 Not Found in 0.0ms @ container/container.go:172(container.AuthenticateNotImplemented) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed GET /v2/token?scope=%2A&scope=repository%3Atest%2Ftest%3Apull%2Cpush&service=container_registry for REDCACTED:0, 200 OK in 57.1ms @ container/container.go:149(container.Authenticate) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 200 OK in 1.3ms @ container/container.go:486(container.HeadBlob) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/blobs/sha256:[...] for REDCACTED:0, 200 OK in 1.2ms @ container/container.go:486(container.HeadBlob) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed HEAD /v2/test/test/manifests/test-without-provenance for REDCACTED:0, 404 Not Found in 0.5ms @ container/container.go:624(container.HeadManifest) 2024/08/18 14:35:13 ...eb/routing/logger.go:102:func1() [I] router: completed PUT /v2/test/test/manifests/test-without-provenance for REDCACTED:0, 201 Created in 3.6ms @ container/container.go:539(container.UploadManifest) ``` Note that the first `HEAD` requests still results in 401, do you know what's up with that? That's also what is now breaking my BuildKit/Earthly build: ```shell unexpected status from HEAD request to https://GITEA_HOST/v2/test/test/blobs/sha256:[...]: 401 Unauthorized ``` Also, there is still a 404 error for that manifest endpoint, but the docker client is not complaining anymore - strange. Please let me know if i can be of any help with debugging this issue further :)
Author
Owner

@fekle commented on GitHub (Aug 18, 2024):

I had a look at where the 401 is coming from, container/container.go:126; thats:

	if ctx.Doer == nil || (setting.Service.RequireSignInView && ctx.Doer.IsGhost()) {
		apiUnauthorizedError(ctx)
	}

As a quick test, i set REQUIRE_SIGNIN_VIEW to false in my app.ini, restarted Gitea and retried, but i'm still getting the same error with Buildkit - so that's not it. Given that this issue seems to come from ctx.Doer == nil and occurs at the HEAD request, maybe Buildkit is not sending any credentials with that particular request?

@fekle commented on GitHub (Aug 18, 2024): I had a look at where the 401 is coming from, `container/container.go:126`; thats: ```go if ctx.Doer == nil || (setting.Service.RequireSignInView && ctx.Doer.IsGhost()) { apiUnauthorizedError(ctx) } ``` As a quick test, i set `REQUIRE_SIGNIN_VIEW` to `false` in my `app.ini`, restarted Gitea and retried, but i'm still getting the same error with Buildkit - so that's not it. Given that this issue seems to come from `ctx.Doer == nil` and occurs at the `HEAD` request, maybe Buildkit is not sending any credentials with that particular request?
Author
Owner

@ChristopherHX commented on GitHub (Aug 18, 2024):

what is now breaking my BuildKit/Earthly build

Are those still have provenance on? I mean it's not unlikely that provenance is not supported yet by the Gitea Container Registry.

See
https://docs.docker.com/build/building/variables/#buildx_no_default_attestations for an env variable.

Also, there is still a 404 error for that manifest endpoint, but the docker client is not complaining anymore - strange.

Pure logical, isn't the 404 here just telling docker that the tag doesn't exist yet
Then docker creates the tag with PUT that returns created. Now I would expect doing this a second time wouldn't result in a 404 before the put

BTW yes I used buildkit to push my image yesterday, no error (what I tested is in discord, between your post and your reply to me)

My Log
::group::GitHub Actions runtime token ACs
: write
::endgroup::
::group::Docker info
[command]/usr/bin/docker version
Client:
 Version:           26.1.4-1
 API version:       1.45
 Go version:        go1.21.11
 Git commit:        5650f9b10226d75e8e9a490a31cc3e5b846e0034
 Built:             Wed Jun  5 10:47:13 UTC 2024
 OS/Arch:           linux/arm64
 Context:           default
Server: Docker Engine - Community
 Engine:
  Version:          26.1.3
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.10
  Git commit:       8e96db1
  Built:            Thu May 16 08:39:57 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
[command]/usr/bin/docker info
Client:
 Version:    26.1.4-1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.15.1-1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.28.1-1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
Server:
 Containers: 42
  Running: 9
  Paused: 0
  Stopped: 33
 Images: 311
 Server Version: 26.1.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-1055-raspi
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 7.624GiB
 Name: ubuntu
 ID: 2bbda9e8-6738-44c4-8348-144dfa28c073
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 97
  Goroutines: 111
  System Time: 2024-08-17T21:56:03.805054279Z
  EventsListeners: 0
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
::endgroup::
::group::Proxy configuration
No proxy configuration found
::endgroup::
::group::Buildx version
[command]/usr/bin/docker buildx version
github.com/docker/buildx 0.15.1-1 1c1dbb2e4cb5363110f42102744a08d034c2300d
::endgroup::
::group::Builder info
{
  "nodes": [
    {
      "name": "default",
      "endpoint": "default",
      "status": "running",
      "buildkit": "v0.13.2",
      "platforms": "linux/arm64,linux/arm/v7,linux/arm/v6",
      "features": {
        "Automatically load images to the Docker Engine image store": true,
        "Cache export": true,
        "Docker exporter": true,
        "Multi-platform build": true,
        "OCI exporter": true
      },
      "labels": {
        "org.mobyproject.buildkit.worker.moby.host-gateway-ip": "172.17.0.1"
      }
    }
  ],
  "name": "default",
  "driver": "docker"
}
::endgroup::
[command]/usr/bin/docker buildx build --iidfile /tmp/docker-actions-toolkit-IqcWDl/build-iidfile-d9dbbc13a2.txt --tag domain/christopher/test-image:develop --metadata-file /tmp/docker-actions-toolkit-IqcWDl/build-metadata-6189bffe81.json --push .
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 55B done
#1 DONE 0.0s
#2 [internal] load metadata for docker.io/library/ubuntu:latest
#2 DONE 0.0s
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s
#4 [1/1] FROM docker.io/library/ubuntu:latest
#4 CACHED
#5 exporting to image
#5 exporting layers done
#5 writing image sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be done
#5 naming to domain/christopher/test-image:develop done
#5 DONE 0.0s
#6 resolving provenance for metadata file
#6 DONE 0.0s
#7 pushing domain/christopher/test-image:develop with docker
#7 pushing layer a510ae0f066c
#7 pushing layer a510ae0f066c 4.33MB / 69.25MB 1.1s
#7 pushing layer a510ae0f066c 8.61MB / 69.25MB 1.8s
#7 pushing layer a510ae0f066c 12.49MB / 69.25MB 2.5s
#7 pushing layer a510ae0f066c 16.87MB / 69.25MB 3.2s
#7 pushing layer a510ae0f066c 20.64MB / 69.25MB 3.8s
#7 pushing layer a510ae0f066c 24.43MB / 69.25MB 4.3s
#7 pushing layer a510ae0f066c 28.81MB / 69.25MB 5.1s
#7 pushing layer a510ae0f066c 32.69MB / 69.25MB 5.8s
#7 pushing layer a510ae0f066c 36.51MB / 69.25MB 6.6s
#7 pushing layer a510ae0f066c 40.37MB / 69.25MB 7.3s
#7 pushing layer a510ae0f066c 44.72MB / 69.25MB 8.0s
#7 pushing layer a510ae0f066c 48.56MB / 69.25MB 8.7s
#7 pushing layer a510ae0f066c 52.41MB / 69.25MB 9.4s
#7 pushing layer a510ae0f066c 56.67MB / 69.25MB 10.2s
#7 pushing layer a510ae0f066c 60.38MB / 69.25MB 11.6s
#7 pushing layer a510ae0f066c 64.10MB / 69.25MB 12.2s
#7 pushing layer a510ae0f066c 67.84MB / 69.25MB 13.1s
#7 pushing layer a510ae0f066c 71.59MB / 69.25MB 13.8s
#7 pushing layer a510ae0f066c 16.3s done
#7 DONE 16.7s
::group::ImageID
sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be
::endgroup::
::group::Digest
sha256:c77a24d4a20eef7c039be023f4c82bfd4c00a2bf82dd318785cd97312835aa67
::endgroup::
::group::Metadata
{
  "buildx.build.provenance": {
    "buildType": "https://mobyproject.org/buildkit@v1",
    "materials": [
      {
        "uri": "pkg:docker/ubuntu@latest?platform=linux%2Farm64",
        "digest": {
          "sha256": "a510ae0f066c65d73f523d5821fe048a6858e53360ecb8d39b7b931162f11479"
        }
      }
    ],
    "invocation": {
      "configSource": {
        "entryPoint": "Dockerfile"
      },
      "parameters": {
        "frontend": "dockerfile.v0",
        "locals": [
          {
            "name": "context"
          },
          {
            "name": "dockerfile"
          }
        ]
      },
      "environment": {
        "platform": "linux/arm64"
      }
    }
  },
  "buildx.build.ref": "default/default/ayj5ct2nam1a1pvtqze1rmy32",
  "containerimage.config.digest": "sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be",
  "containerimage.digest": "sha256:c77a24d4a20eef7c039be023f4c82bfd4c00a2bf82dd318785cd97312835aa67",
  "image.name": "domain/christopher/test-image:develop"
}
::endgroup::
::group::Reference
default/default/ayj5ct2nam1a1pvtqze1rmy32
::endgroup::
::group::Check build summary support
Build summary is not yet supported on GHES
::endgroup::
@ChristopherHX commented on GitHub (Aug 18, 2024): > what is now breaking my BuildKit/Earthly build Are those still have provenance on? I mean it's not unlikely that provenance is not supported yet by the Gitea Container Registry. See https://docs.docker.com/build/building/variables/#buildx_no_default_attestations for an env variable. > Also, there is still a 404 error for that manifest endpoint, but the docker client is not complaining anymore - strange. Pure logical, isn't the 404 here just telling docker that the tag doesn't exist yet Then docker creates the tag with PUT that returns created. Now I would expect doing this a second time wouldn't result in a 404 before the put BTW yes I used buildkit to push my image yesterday, no error (what I tested is in discord, between your post and your reply to me) <Details> <Summary>My Log</Summary> ``` ::group::GitHub Actions runtime token ACs : write ::endgroup:: ::group::Docker info [command]/usr/bin/docker version Client: Version: 26.1.4-1 API version: 1.45 Go version: go1.21.11 Git commit: 5650f9b10226d75e8e9a490a31cc3e5b846e0034 Built: Wed Jun 5 10:47:13 UTC 2024 OS/Arch: linux/arm64 Context: default Server: Docker Engine - Community Engine: Version: 26.1.3 API version: 1.45 (minimum version 1.24) Go version: go1.21.10 Git commit: 8e96db1 Built: Thu May 16 08:39:57 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.6.31 GitCommit: e377cd56a71523140ca6ae87e30244719194a521 runc: Version: 1.1.12 GitCommit: v1.1.12-0-g51d5e94 docker-init: Version: 0.19.0 GitCommit: de40ad0 [command]/usr/bin/docker info Client: Version: 26.1.4-1 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: 0.15.1-1 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: 2.28.1-1 Path: /usr/libexec/docker/cli-plugins/docker-compose Server: Containers: 42 Running: 9 Paused: 0 Stopped: 33 Images: 311 Server Version: 26.1.3 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Using metacopy: false Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: systemd Cgroup Version: 2 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 runc Default Runtime: runc Init Binary: docker-init containerd version: e377cd56a71523140ca6ae87e30244719194a521 runc version: v1.1.12-0-g51d5e94 init version: de40ad0 Security Options: apparmor seccomp Profile: builtin cgroupns Kernel Version: 5.15.0-1055-raspi Operating System: Ubuntu 22.04.4 LTS OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 7.624GiB Name: ubuntu ID: 2bbda9e8-6738-44c4-8348-144dfa28c073 Docker Root Dir: /var/lib/docker Debug Mode: true File Descriptors: 97 Goroutines: 111 System Time: 2024-08-17T21:56:03.805054279Z EventsListeners: 0 Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false ::endgroup:: ::group::Proxy configuration No proxy configuration found ::endgroup:: ::group::Buildx version [command]/usr/bin/docker buildx version github.com/docker/buildx 0.15.1-1 1c1dbb2e4cb5363110f42102744a08d034c2300d ::endgroup:: ::group::Builder info { "nodes": [ { "name": "default", "endpoint": "default", "status": "running", "buildkit": "v0.13.2", "platforms": "linux/arm64,linux/arm/v7,linux/arm/v6", "features": { "Automatically load images to the Docker Engine image store": true, "Cache export": true, "Docker exporter": true, "Multi-platform build": true, "OCI exporter": true }, "labels": { "org.mobyproject.buildkit.worker.moby.host-gateway-ip": "172.17.0.1" } } ], "name": "default", "driver": "docker" } ::endgroup:: [command]/usr/bin/docker buildx build --iidfile /tmp/docker-actions-toolkit-IqcWDl/build-iidfile-d9dbbc13a2.txt --tag domain/christopher/test-image:develop --metadata-file /tmp/docker-actions-toolkit-IqcWDl/build-metadata-6189bffe81.json --push . #0 building with "default" instance using docker driver #1 [internal] load build definition from Dockerfile #1 transferring dockerfile: 55B done #1 DONE 0.0s #2 [internal] load metadata for docker.io/library/ubuntu:latest #2 DONE 0.0s #3 [internal] load .dockerignore #3 transferring context: 2B done #3 DONE 0.0s #4 [1/1] FROM docker.io/library/ubuntu:latest #4 CACHED #5 exporting to image #5 exporting layers done #5 writing image sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be done #5 naming to domain/christopher/test-image:develop done #5 DONE 0.0s #6 resolving provenance for metadata file #6 DONE 0.0s #7 pushing domain/christopher/test-image:develop with docker #7 pushing layer a510ae0f066c #7 pushing layer a510ae0f066c 4.33MB / 69.25MB 1.1s #7 pushing layer a510ae0f066c 8.61MB / 69.25MB 1.8s #7 pushing layer a510ae0f066c 12.49MB / 69.25MB 2.5s #7 pushing layer a510ae0f066c 16.87MB / 69.25MB 3.2s #7 pushing layer a510ae0f066c 20.64MB / 69.25MB 3.8s #7 pushing layer a510ae0f066c 24.43MB / 69.25MB 4.3s #7 pushing layer a510ae0f066c 28.81MB / 69.25MB 5.1s #7 pushing layer a510ae0f066c 32.69MB / 69.25MB 5.8s #7 pushing layer a510ae0f066c 36.51MB / 69.25MB 6.6s #7 pushing layer a510ae0f066c 40.37MB / 69.25MB 7.3s #7 pushing layer a510ae0f066c 44.72MB / 69.25MB 8.0s #7 pushing layer a510ae0f066c 48.56MB / 69.25MB 8.7s #7 pushing layer a510ae0f066c 52.41MB / 69.25MB 9.4s #7 pushing layer a510ae0f066c 56.67MB / 69.25MB 10.2s #7 pushing layer a510ae0f066c 60.38MB / 69.25MB 11.6s #7 pushing layer a510ae0f066c 64.10MB / 69.25MB 12.2s #7 pushing layer a510ae0f066c 67.84MB / 69.25MB 13.1s #7 pushing layer a510ae0f066c 71.59MB / 69.25MB 13.8s #7 pushing layer a510ae0f066c 16.3s done #7 DONE 16.7s ::group::ImageID sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be ::endgroup:: ::group::Digest sha256:c77a24d4a20eef7c039be023f4c82bfd4c00a2bf82dd318785cd97312835aa67 ::endgroup:: ::group::Metadata { "buildx.build.provenance": { "buildType": "https://mobyproject.org/buildkit@v1", "materials": [ { "uri": "pkg:docker/ubuntu@latest?platform=linux%2Farm64", "digest": { "sha256": "a510ae0f066c65d73f523d5821fe048a6858e53360ecb8d39b7b931162f11479" } } ], "invocation": { "configSource": { "entryPoint": "Dockerfile" }, "parameters": { "frontend": "dockerfile.v0", "locals": [ { "name": "context" }, { "name": "dockerfile" } ] }, "environment": { "platform": "linux/arm64" } } }, "buildx.build.ref": "default/default/ayj5ct2nam1a1pvtqze1rmy32", "containerimage.config.digest": "sha256:d8398bc503e6b06179ed7c2416aa3a837c3d9ba2093f4d6a5521d991f33767be", "containerimage.digest": "sha256:c77a24d4a20eef7c039be023f4c82bfd4c00a2bf82dd318785cd97312835aa67", "image.name": "domain/christopher/test-image:develop" } ::endgroup:: ::group::Reference default/default/ayj5ct2nam1a1pvtqze1rmy32 ::endgroup:: ::group::Check build summary support Build summary is not yet supported on GHES ::endgroup:: ``` </Details>
Author
Owner

@ChristopherHX commented on GitHub (Aug 18, 2024):

Ok I'm out, as I cannot reproduce the issue on my system, provenance push didn't fail for me.

I have created an dind container with containerd image cache and it doesn't fail.

The flag REQUIRE_SIGNIN_VIEW isn't enabled for me, because my ports are not exposed to the www and my reverse proxy configuration is probably totally different.

Maybe someone else can have a look

@ChristopherHX commented on GitHub (Aug 18, 2024): Ok I'm out, as I cannot reproduce the issue on my system, provenance push didn't fail for me. I have created an dind container with containerd image cache and it doesn't fail. The flag REQUIRE_SIGNIN_VIEW isn't enabled for me, because my ports are not exposed to the www and my reverse proxy configuration is probably totally different. Maybe someone else can have a look
Author
Owner

@fekle commented on GitHub (Aug 18, 2024):

Are those still have provenance on? I mean it's not unlikely that provenance is not supported yet by the Gitea Container Registry.

Yes, i used BUILDX_NO_DEFAULT_ATTESTATIONS=1 but could well be that buildkit itself is not picking up that argument, as i'm using a buildkit in my CI; I will have to debug this further. Kinda annoying that BuildKit does not handle this gracefully and just assumes that the registry supports this feature.
But the 401 in my log output above is from a normal docker push with the --provenance=false command, which seems to ignore the 401 when BuildKit doesn't. Given that there provenance is disabled for sure, that seems to be a different issue.

Pure logical, isn't the 404 here just telling docker that the tag doesn't exist yet
Then docker creates the tag with PUT that returns created. Now I would expect doing this a second time wouldn't result in a 404 before the put

While that would indeed make sense, I don't think that's the case as that still occurs when the image and all layers already exist, and the error in the logs is container.AuthenticateNotImplemented. Not sure what that is about, but it could be related to the 401s before.

@fekle commented on GitHub (Aug 18, 2024): > Are those still have provenance on? I mean it's not unlikely that provenance is not supported yet by the Gitea Container Registry. Yes, i used `BUILDX_NO_DEFAULT_ATTESTATIONS=1` but could well be that buildkit itself is not picking up that argument, as i'm using a buildkit in my CI; I will have to debug this further. Kinda annoying that BuildKit does not handle this gracefully and just assumes that the registry supports this feature. But the 401 in my log output above is from a `normal` docker push with the `--provenance=false` command, which seems to ignore the 401 when BuildKit doesn't. Given that there provenance is disabled for sure, that seems to be a different issue. > Pure logical, isn't the 404 here just telling docker that the tag doesn't exist yet > Then docker creates the tag with PUT that returns created. Now I would expect doing this a second time wouldn't result in a 404 before the put While that would indeed make sense, I don't think that's the case as that still occurs when the image and all layers already exist, and the error in the logs is `container.AuthenticateNotImplemented`. Not sure what that is about, but it could be related to the 401s before.
Author
Owner

@fekle commented on GitHub (Aug 18, 2024):

Ok I'm out, as I cannot reproduce the issue on my system, provenance push didn't fail for me.

I have created an dind container with containerd image cache and it doesn't fail.

The flag REQUIRE_SIGNIN_VIEW isn't enabled for me, because my ports are not exposed to the www and my reverse proxy configuration is probably totally different.

Maybe someone else can have a look

That's interesting that it doesnt fail for you, so maybe it's something with my configuration. But thank you for your help!

@fekle commented on GitHub (Aug 18, 2024): > Ok I'm out, as I cannot reproduce the issue on my system, provenance push didn't fail for me. > > I have created an dind container with containerd image cache and it doesn't fail. > > The flag REQUIRE_SIGNIN_VIEW isn't enabled for me, because my ports are not exposed to the www and my reverse proxy configuration is probably totally different. > > Maybe someone else can have a look That's interesting that it doesnt fail for you, so maybe it's something with my configuration. But thank you for your help!
Author
Owner

@fekle commented on GitHub (Aug 19, 2024):

Allright, I got pushing in Actions to work.

I was using

- name: login to container registry
  uses: docker/login-action@v2
  with:
    registry: GITEA_HOST
    username: ${{ gitea.actor }}
    password: ${{ gitea.token }}

to log in to the container registry.

While the login succeeds, pushes fail. When I create a personal access token with access to packages and then use my username as username as well as that token as password, I can push.

Now, if I want to keep that more secure, I can put that token into a secret - however, given that I can only define secrets per repository and not globally, I'll have to copy that secret definition to every repository that I'm using CI with.

Is there a better way to do this? Researching on Google and so on leads to scattered forum posts from various versions, so I can't seem to find a definitive answer. Coming from GitLab CI, there were 3 auto-generated variables in CI: CI_REGISTRY, CI_REGISTRY_USER, and CI_REGISTRY_PASSWORD. That made pushing to the built-in registry a breeze; it would be nice if Gitea had something similar - I would think that pushing to the built-in registry is a common action in CI.

I'd be happy to write some documentation on that and open a PR once I know the "idiomatic" way to do all of this :)

@fekle commented on GitHub (Aug 19, 2024): Allright, I got pushing in Actions to work. I was using ```yaml - name: login to container registry uses: docker/login-action@v2 with: registry: GITEA_HOST username: ${{ gitea.actor }} password: ${{ gitea.token }} ``` to log in to the container registry. While the login succeeds, pushes fail. When I create a personal access token with access to packages and then use my username as `username` as well as that token as `password`, I can push. Now, if I want to keep that more secure, I can put that token into a secret - however, given that I can only define secrets per repository and not globally, I'll have to copy that secret definition to every repository that I'm using CI with. Is there a better way to do this? Researching on Google and so on leads to scattered forum posts from various versions, so I can't seem to find a definitive answer. Coming from GitLab CI, there were 3 auto-generated variables in CI: `CI_REGISTRY`, `CI_REGISTRY_USER`, and `CI_REGISTRY_PASSWORD`. That made pushing to the built-in registry a breeze; it would be nice if Gitea had something similar - I would think that pushing to the built-in registry is a common action in CI. I'd be happy to write some documentation on that and open a PR once I know the "idiomatic" way to do all of this :)
Author
Owner

@shaikzoheb-oi commented on GitHub (Sep 6, 2024):

@fekle In my case, the organisation is "oi" and project is "test". It's a similar use-case like you, where I am trying to use gitea as container registry along with git repo. In my scenario, I've deployed gitea locally and somehow it's failing.

% docker build -t 127.0.0.1:3000/oi/test .
[+] Building 2.1s (5/5) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 61B 0.0s
=> [internal] load metadata for docker.io/library/debian:12 2.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/1] FROM docker.io/library/debian:12@sha256:b8084b1a576c5504a031936e1132574f4ce1d6cc7130bbcc25a28f074539ae6b 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:0c21f32bd1037ae97c3535bb9c7f15f7ca0f829e5809161f607531c20764b61e 0.0s
=> => naming to 127.0.0.1:3000/oi/test 0.0s

% docker push 127.0.0.1:3000/oi/test:test
The push refers to repository [127.0.0.1:3000/oi/test]
Get "http://127.0.0.1:3000/v2/": dial tcp 127.0.0.1:3000: connect: connection refused

telnet and netcat is working:

nc -zv 127.0.0.1 3000
Connection to 127.0.0.1 port 3000 [tcp/hbci] succeeded!

telnet 127.0.0.1 3000
Trying 127.0.0.1...
Connected to localhost.

@shaikzoheb-oi commented on GitHub (Sep 6, 2024): @fekle In my case, the organisation is "oi" and project is "test". It's a similar use-case like you, where I am trying to use gitea as container registry along with git repo. In my scenario, I've deployed gitea locally and somehow it's failing. % docker build -t 127.0.0.1:3000/oi/test . [+] Building 2.1s (5/5) FINISHED docker:desktop-linux => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 61B 0.0s => [internal] load metadata for docker.io/library/debian:12 2.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => CACHED [1/1] FROM docker.io/library/debian:12@sha256:b8084b1a576c5504a031936e1132574f4ce1d6cc7130bbcc25a28f074539ae6b 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => writing image sha256:0c21f32bd1037ae97c3535bb9c7f15f7ca0f829e5809161f607531c20764b61e 0.0s => => naming to 127.0.0.1:3000/oi/test 0.0s % docker push 127.0.0.1:3000/oi/test:test The push refers to repository [127.0.0.1:3000/oi/test] Get "http://127.0.0.1:3000/v2/": dial tcp 127.0.0.1:3000: connect: connection refused telnet and netcat is working: nc -zv 127.0.0.1 3000 Connection to 127.0.0.1 port 3000 [tcp/hbci] succeeded! telnet 127.0.0.1 3000 Trying 127.0.0.1... Connected to localhost.
Author
Owner

@jcjveraa commented on GitHub (Oct 23, 2024):

Exact same issue here. Running the latest official Gitea image via docker. The main git functionality has been working fine for months. The comments above seem to suggest creating a PAT can work, but that didn't do it for me. I created a new PAT, and tried a docker push. The PAT was used according to the Gitea UI, but still the same output as others see.

Then tried a docker build --provenance=false image build, which pushed fine.

I had expected this to work somewhat more 'out of the box' as a basic push doesn't seem special.

For reference, output of the failed push below. Again: with --provenance=false worked fine. Replaced strings with descriptions for privacy.

$ docker --version
Docker version 27.3.1, build ce12230

$ docker push [host]/[organization]/[repo]/[image-name]:latest
The push refers to repository [[host]/[organization]/[repo]/[image-name]]
73b263454379: Pushed 
d88d75d6bbae: Layer already exists 
a480a496ba95: Layer already exists 
a5072a025aa2: Layer already exists 
ae8522101371: Layer already exists 
88900f0244bb: Layer already exists 
5cfa0738ac2f: Layer already exists 
2f779fda9095: Layer already exists 
24f2c78e0691: Layer already exists 
failed commit on ref "index-sha256:07761bba483df9b7123227be97a913b01bb62bee3533214537d694ec3902ec08": unexpected status from PUT request to https://[host]/v2/[organization]/[repo]/[image-name]/manifests/latest: 404 Not Found
@jcjveraa commented on GitHub (Oct 23, 2024): Exact same issue here. Running the latest official Gitea image via docker. The main git functionality has been working fine for months. The comments above seem to suggest creating a PAT can work, but that didn't do it for me. I created a new PAT, and tried a `docker push`. The PAT was used according to the Gitea UI, but still the same output as others see. Then tried a `docker build --provenance=false` image build, which pushed fine. I had expected this to work somewhat more 'out of the box' as a basic push doesn't seem special. For reference, output of the failed push below. Again: with `--provenance=false` worked fine. _Replaced strings with descriptions for privacy._ ``` $ docker --version Docker version 27.3.1, build ce12230 $ docker push [host]/[organization]/[repo]/[image-name]:latest The push refers to repository [[host]/[organization]/[repo]/[image-name]] 73b263454379: Pushed d88d75d6bbae: Layer already exists a480a496ba95: Layer already exists a5072a025aa2: Layer already exists ae8522101371: Layer already exists 88900f0244bb: Layer already exists 5cfa0738ac2f: Layer already exists 2f779fda9095: Layer already exists 24f2c78e0691: Layer already exists failed commit on ref "index-sha256:07761bba483df9b7123227be97a913b01bb62bee3533214537d694ec3902ec08": unexpected status from PUT request to https://[host]/v2/[organization]/[repo]/[image-name]/manifests/latest: 404 Not Found ```
Author
Owner

@lunny commented on GitHub (Oct 23, 2024):

What's your PAT scope?

@lunny commented on GitHub (Oct 23, 2024): What's your PAT scope?
Author
Owner

@jcjveraa commented on GitHub (Oct 24, 2024):

Good point - (non public) packages read & write, I failed to consider a
wider scope but on the flip side thinking about it now I would expect that
to be enough.

Op do 24 okt 2024 om 00:18 schreef Lunny Xiao @.***>

What's your PAT scope?


Reply to this email directly, view it on GitHub
https://github.com/go-gitea/gitea/issues/31861#issuecomment-2433570979,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA6CPHKPBZT6ZYDUTJ4F5L3Z5AOBZAVCNFSM6AAAAABMWJQFWWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZTGU3TAOJXHE
.
You are receiving this because you commented.Message ID:
@.***>

@jcjveraa commented on GitHub (Oct 24, 2024): Good point - (non public) packages read & write, I failed to consider a wider scope but on the flip side thinking about it now I would expect that to be enough. Op do 24 okt 2024 om 00:18 schreef Lunny Xiao ***@***.***> > What's your PAT scope? > > — > Reply to this email directly, view it on GitHub > <https://github.com/go-gitea/gitea/issues/31861#issuecomment-2433570979>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AA6CPHKPBZT6ZYDUTJ4F5L3Z5AOBZAVCNFSM6AAAAABMWJQFWWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZTGU3TAOJXHE> > . > You are receiving this because you commented.Message ID: > ***@***.***> >
Author
Owner

@sbrown61 commented on GitHub (Nov 13, 2024):

I'm having a similar issue with podman push. As far as I know, there is no equivalent for --provenance=false. Podman version is 5.2.5 and gitea is 1.22.3 (via docker hub):

Getting image source signatures
Copying blob 49cadc25f7bb [--------------------------------------] 8.0b / 1.4MiB | 1.9 MiB/s
Copying blob edeb1fd6bf66 [--------------------------------------] 8.0b / 4.1MiB | 1.7 MiB/s
Copying blob 8cbe4b54fa88 [--------------------------------------] 8.0b / 80.1MiB | 1.1 MiB/s
Copying blob fa3e1fa14e04 [--------------------------------------] 8.0b / 341.5KiB | 2.4 MiB/s
Copying blob 7ddf8331309f [--------------------------------------] 8.0b / 3.5KiB | 4.0 MiB/s
Copying blob dcd40c14ad70 [--------------------------------------] 8.0b / 9.1MiB | 1.1 MiB/s
Copying blob 9b03e049cc12 [--------------------------------------] 8.0b / 157.3MiB | 1.2 MiB/s
Error: writing blob: initiating layer upload to /v2/[USERNAME]/blobs/uploads/ in [HOST]:[PORT]: StatusCode: 404, ""
@sbrown61 commented on GitHub (Nov 13, 2024): I'm having a similar issue with podman push. As far as I know, there is no equivalent for `--provenance=false`. Podman version is 5.2.5 and gitea is 1.22.3 (via docker hub): ``` Getting image source signatures Copying blob 49cadc25f7bb [--------------------------------------] 8.0b / 1.4MiB | 1.9 MiB/s Copying blob edeb1fd6bf66 [--------------------------------------] 8.0b / 4.1MiB | 1.7 MiB/s Copying blob 8cbe4b54fa88 [--------------------------------------] 8.0b / 80.1MiB | 1.1 MiB/s Copying blob fa3e1fa14e04 [--------------------------------------] 8.0b / 341.5KiB | 2.4 MiB/s Copying blob 7ddf8331309f [--------------------------------------] 8.0b / 3.5KiB | 4.0 MiB/s Copying blob dcd40c14ad70 [--------------------------------------] 8.0b / 9.1MiB | 1.1 MiB/s Copying blob 9b03e049cc12 [--------------------------------------] 8.0b / 157.3MiB | 1.2 MiB/s Error: writing blob: initiating layer upload to /v2/[USERNAME]/blobs/uploads/ in [HOST]:[PORT]: StatusCode: 404, "" ```
Author
Owner

@jymusic0663 commented on GitHub (Nov 23, 2024):

My version is Gitea v1.22.3, previously I used Docker build - t text: latest . Directly building an AMD64 image can successfully push it up.
But after enabling multi platform build, Docker buildx build -- platform Linux/amd64, Linux/arm64- t test: latest ., the image layer can be uploaded, but the final step is also
failed commit on ref "index-sha256:8a4a910b69...": unexpected status from PUT request to http://192.168.1.6:8080/v2/my/test/manifests/latest: 404 Not Found

@jymusic0663 commented on GitHub (Nov 23, 2024): My version is Gitea v1.22.3, previously I used `Docker build - t text: latest .` Directly building an AMD64 image can successfully push it up. But after enabling multi platform build, `Docker buildx build -- platform Linux/amd64, Linux/arm64- t test: latest .`, the image layer can be uploaded, but the final step is also `failed commit on ref "index-sha256:8a4a910b69...": unexpected status from PUT request to http://192.168.1.6:8080/v2/my/test/manifests/latest: 404 Not Found`
Author
Owner

@dresber commented on GitHub (Mar 7, 2025):

I have the same issue in v1.23.5. The solution with adding the --provenance=false did help me so far.

@dresber commented on GitHub (Mar 7, 2025): I have the same issue in v1.23.5. The solution with adding the `--provenance=false` did help me so far.
Author
Owner

@mbretter commented on GitHub (Mar 26, 2025):

Same issue here, provenance flag does not help.
While the push works from linux, it fails from macos, same project, same docker versions (28.0.1).

@mbretter commented on GitHub (Mar 26, 2025): Same issue here, provenance flag does not help. While the push works from linux, it fails from macos, same project, same docker versions (28.0.1).
Author
Owner

@mbretter commented on GitHub (Mar 26, 2025):

For some reason under macos it did not remember the login, after doing another login before the push, it succeeds, weird.

@mbretter commented on GitHub (Mar 26, 2025): For some reason under macos it did not remember the login, after doing another login before the push, it succeeds, weird.
Author
Owner

@R-HNF commented on GitHub (Apr 27, 2025):

I'm experiencing the same issue with image push failures on Docker Desktop for Mac. However, after looking at the following lines, it occurred to me to try specifying only one platform.

533b8b2d3d/routers/api/packages/container/manifest.go (L243-L245)

0376c09fc2/routers/api/packages/container/manifest.go (L34-L40)

Specifying only one platform allowed me to push successfully.

@R-HNF commented on GitHub (Apr 27, 2025): I'm experiencing the same issue with image push failures on Docker Desktop for Mac. However, after looking at the following lines, it occurred to me to try specifying only one platform. https://github.com/go-gitea/gitea/blob/533b8b2d3d98f2f95b3162eebe8cbf1063f90ec1/routers/api/packages/container/manifest.go#L243-L245 https://github.com/go-gitea/gitea/blob/0376c09fc27d848dbac471b629c83fd7a231852b/routers/api/packages/container/manifest.go#L34-L40 Specifying only one platform allowed me to push successfully.
Author
Owner

@wxiaoguang commented on GitHub (Jul 1, 2025):

I think the problem should have been fixed on main branch, feel free to try the nightly build.

If there is still any problem, feel free to reopen and/or provide some reproducible steps.

Thank you all.

@wxiaoguang commented on GitHub (Jul 1, 2025): I think the problem should have been fixed on main branch, feel free to try the nightly build. * https://dl.gitea.com/gitea/main-nightly/ * https://hub.docker.com/r/gitea/gitea/tags?name=nightly If there is still any problem, feel free to reopen and/or provide some reproducible steps. Thank you all.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#13397