Error 1062 (23000): Duplicate entry when using 'docker buildx imagetools create' in Actions #12583

Closed
opened 2025-11-02 10:14:56 -06:00 by GiteaMirror · 1 comment
Owner

Originally created by @MrMeeb on GitHub (Mar 3, 2024).

Description

I have built an action that builds a docker container on multiple nodes before merging the images together. The approach is basically a carbon copy of the approach outlined here: https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners

When creating the image by merging the digests of each individual image, the action fails with this error:

{"tags":["git.domain.com/mrmeeb/example-container:69.69.76","git.domain.com/mrmeeb/example-container:latest"],"labels":{"org.opencontainers.image.created":"2024-03-03T16:00:29.349Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","org.opencontainers.image.source":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.title":"example-container","org.opencontainers.image.url":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.version":"69.69.76"},"annotations":["manifest:org.opencontainers.image.created=2024-03-03T16:00:29.349Z","manifest:org.opencontainers.image.description=","manifest:org.opencontainers.image.licenses=","manifest:org.opencontainers.image.revision=573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","manifest:org.opencontainers.image.source=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.title=example-container","manifest:org.opencontainers.image.url=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.version=69.69.76"]}
-t git.domain.com/mrmeeb/example-container:69.69.76 -t git.domain.com/mrmeeb/example-container:latest git.domain.com/mrmeeb/example-container@sha256:a54e0f8cd6e76d7fac0b361c18d4d9ea733f367d6f891adaccd57b6dc76c366b git.domain.com/mrmeeb/example-container@sha256:3679da30f127e3250442da966e932e81c425b61d133d91c70030d6a89100efbc
#1 [internal] pushing git.domain.com/mrmeeb/example-container:latest
#1 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest
#1 ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error
#2 [internal] pushing git.domain.com/mrmeeb/example-container:69.69.76
#2 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:69.69.76
#2 DONE 0.1s
------
 > [internal] pushing git.domain.com/mrmeeb/example-container:latest:
0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest
------
ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error

Checking Gitea logs, I see this:

gitea  | 2024/03/03 16:00:30 ...ontainer/manifest.go:452:createManifestBlob() [E] Error inserting package blob: Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5'
gitea  | 2024/03/03 16:00:30 ...ntainer/container.go:89:apiError() [E] Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5'

Looking at the result, I have a tagged container under 69.69.72 (excuse the random testing version), but latest was never created.

Full action for thoroughness:

name: Build-Release-Image
on:
  push:
    tags:
      - '*'

env:
  CONTAINER_NAME: git.domain.com/mrmeeb/example-container
  TEA_VERSION: 0.9.2

jobs:

  Build-Image:
    runs-on: [ubuntu-docker-latest, "${{ matrix.platform }}"]
    strategy:
      fail-fast: false
      matrix:
        platform:
          - linux/amd64
          - linux/arm64
    steps:
      - name: Prepare
        run: |
          platform=${{ matrix.platform }}
          echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
      - name: Checkout
        uses: actions/checkout@v2
      - name: Prepare tags
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.CONTAINER_NAME }}
          tags: |
            type=pep440,pattern={{version}}
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Login to Gitea Container Registry
        uses: docker/login-action@v3
        with:
          registry: git.domain.com
          username: ${{ env.GITHUB_ACTOR }}
          password: ${{ secrets.GTCR_TOKEN }}
      - name: Build and push by digest
        uses: docker/build-push-action@v5
        id: build
        with:
          context: ./app
          platforms: ${{ matrix.platform }}
          labels: ${{ steps.meta.outputs.labels }}
          outputs: type=image,name=${{ env.CONTAINER_NAME }},push-by-digest=true,name-canonical=true,push=true
      - name: Export digest
        run: |
          mkdir -p /tmp/digests
          digest="${{ steps.build.outputs.digest }}"
          touch "/tmp/digests/${digest#sha256:}"
      - name: Upload digest
        uses: actions/upload-artifact@v3
        with:
          name: digests-${{ env.PLATFORM_PAIR }}
          path: /tmp/digests/*
          if-no-files-found: error
          retention-days: 1
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Merge-Images:
    runs-on: ubuntu-docker-latest
    needs: [Build-Image]
    steps:
      - name: Download digests
        uses: actions/download-artifact@v3
        with:
          path: /tmp/digests
          pattern: digests-*
          merge-multiple: true
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Prepare Docker metadata
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ env.CONTAINER_NAME }}
      - name: Login to Gitea Container Registry
        uses: docker/login-action@v3
        with:
          registry: git.domain.com
          username: ${{ env.GITHUB_ACTOR }}
          password: ${{ secrets.GTCR_TOKEN }}
      - name: Create manifest list and push
        working-directory: /tmp/digests
        run: |
          echo $DOCKER_METADATA_OUTPUT_JSON
          echo $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
            $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2))
          docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
            $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2))
      - name: Inspect image
        run: |
          docker buildx imagetools inspect ${{ env.CONTAINER_NAME }}:${{ steps.meta.outputs.version }}          
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Create-Release:
    runs-on: [ubuntu-latest, linux/amd64]
    needs: [Merge-Images]
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Get tag
        run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
      - name: Prepare tea
        run: |
          # Download tea from Gitea release page
          echo "Downloading Tea v${{ env.TEA_VERSION }}" && \
          wget -q -O tea https://gitea.com/gitea/tea/releases/download/v${{ env.TEA_VERSION }}/tea-${{ env.TEA_VERSION }}-linux-amd64 && \
          echo "Downloaded Tea" && \
          chmod +x tea && \
          # Login to Gitea
          echo "Logging in to Gitea using Tea" && \
          ./tea login add --name Gitea --url https://git.domain.com --token ${{ secrets.GITHUB_TOKEN }} && \
          echo "Done"
      - name: Make release
        run: |
          echo "Creating release" && \
          ./tea release create --login "Gitea" --repo ${{ env.GITHUB_REPOSITORY }} --tag ${{ env.RELEASE_VERSION }} -t ${{ env.RELEASE_VERSION }} && \
          echo "Done"
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: failure()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

  Notify:
    runs-on: ubuntu-latest
    needs: [Build-Image, Merge-Images, Create-Release]
    steps:
      - name: Notify
        uses: rjstone/discord-webhook-notify@v1
        if: always()
        with:
          severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }}
          details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}!
          webhookUrl: ${{ secrets.DISCORD_WEBHOOK }}
          username: Gitea
          avatarUrl: ${{ vars.RUNNER_ICON_URL }}

Gitea Version

1.21.6

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

Gitea 1.21.6 running in Docker on an Ubuntu host, with Minio storage for packages only.

Database

MySQL/MariaDB

Originally created by @MrMeeb on GitHub (Mar 3, 2024). ### Description I have built an action that builds a docker container on multiple nodes before merging the images together. The approach is basically a carbon copy of the approach outlined here: https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners When creating the image by merging the digests of each individual image, the action fails with this error: ``` {"tags":["git.domain.com/mrmeeb/example-container:69.69.76","git.domain.com/mrmeeb/example-container:latest"],"labels":{"org.opencontainers.image.created":"2024-03-03T16:00:29.349Z","org.opencontainers.image.description":"","org.opencontainers.image.licenses":"","org.opencontainers.image.revision":"573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","org.opencontainers.image.source":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.title":"example-container","org.opencontainers.image.url":"https://git.domain.com/mrmeeb/example-container","org.opencontainers.image.version":"69.69.76"},"annotations":["manifest:org.opencontainers.image.created=2024-03-03T16:00:29.349Z","manifest:org.opencontainers.image.description=","manifest:org.opencontainers.image.licenses=","manifest:org.opencontainers.image.revision=573fab4bdd6d5ab3dbaa39aea99140b24bbf2bd6","manifest:org.opencontainers.image.source=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.title=example-container","manifest:org.opencontainers.image.url=https://git.domain.com/mrmeeb/example-container","manifest:org.opencontainers.image.version=69.69.76"]} -t git.domain.com/mrmeeb/example-container:69.69.76 -t git.domain.com/mrmeeb/example-container:latest git.domain.com/mrmeeb/example-container@sha256:a54e0f8cd6e76d7fac0b361c18d4d9ea733f367d6f891adaccd57b6dc76c366b git.domain.com/mrmeeb/example-container@sha256:3679da30f127e3250442da966e932e81c425b61d133d91c70030d6a89100efbc #1 [internal] pushing git.domain.com/mrmeeb/example-container:latest #1 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest #1 ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error #2 [internal] pushing git.domain.com/mrmeeb/example-container:69.69.76 #2 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:69.69.76 #2 DONE 0.1s ------ > [internal] pushing git.domain.com/mrmeeb/example-container:latest: 0.000 pushing sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66 to git.domain.com/mrmeeb/example-container:latest ------ ERROR: failed commit on ref "index-sha256:5825d79f54f393aa01787354b551bd788a4ece6c288c6ddb7394b2c73ad3fd66": unexpected status from PUT request to https://git.domain.com/v2/mrmeeb/example-container/manifests/latest: 500 Internal Server Error ``` Checking Gitea logs, I see this: ``` gitea | 2024/03/03 16:00:30 ...ontainer/manifest.go:452:createManifestBlob() [E] Error inserting package blob: Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5' gitea | 2024/03/03 16:00:30 ...ntainer/container.go:89:apiError() [E] Error 1062 (23000): Duplicate entry 'f8292f6774aeb373fb48f8f8967bafc3' for key 'package_blob.UQE_package_blob_md5' ``` Looking at the result, I have a tagged container under `69.69.72` (excuse the random testing version), but `latest` was never created. Full action for thoroughness: ```yaml name: Build-Release-Image on: push: tags: - '*' env: CONTAINER_NAME: git.domain.com/mrmeeb/example-container TEA_VERSION: 0.9.2 jobs: Build-Image: runs-on: [ubuntu-docker-latest, "${{ matrix.platform }}"] strategy: fail-fast: false matrix: platform: - linux/amd64 - linux/arm64 steps: - name: Prepare run: | platform=${{ matrix.platform }} echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV - name: Checkout uses: actions/checkout@v2 - name: Prepare tags id: meta uses: docker/metadata-action@v5 with: images: ${{ env.CONTAINER_NAME }} tags: | type=pep440,pattern={{version}} - name: Set up QEMU uses: docker/setup-qemu-action@v3 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Login to Gitea Container Registry uses: docker/login-action@v3 with: registry: git.domain.com username: ${{ env.GITHUB_ACTOR }} password: ${{ secrets.GTCR_TOKEN }} - name: Build and push by digest uses: docker/build-push-action@v5 id: build with: context: ./app platforms: ${{ matrix.platform }} labels: ${{ steps.meta.outputs.labels }} outputs: type=image,name=${{ env.CONTAINER_NAME }},push-by-digest=true,name-canonical=true,push=true - name: Export digest run: | mkdir -p /tmp/digests digest="${{ steps.build.outputs.digest }}" touch "/tmp/digests/${digest#sha256:}" - name: Upload digest uses: actions/upload-artifact@v3 with: name: digests-${{ env.PLATFORM_PAIR }} path: /tmp/digests/* if-no-files-found: error retention-days: 1 - name: Notify uses: rjstone/discord-webhook-notify@v1 if: failure() with: severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }} details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}! webhookUrl: ${{ secrets.DISCORD_WEBHOOK }} username: Gitea avatarUrl: ${{ vars.RUNNER_ICON_URL }} Merge-Images: runs-on: ubuntu-docker-latest needs: [Build-Image] steps: - name: Download digests uses: actions/download-artifact@v3 with: path: /tmp/digests pattern: digests-* merge-multiple: true - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Prepare Docker metadata id: meta uses: docker/metadata-action@v5 with: images: ${{ env.CONTAINER_NAME }} - name: Login to Gitea Container Registry uses: docker/login-action@v3 with: registry: git.domain.com username: ${{ env.GITHUB_ACTOR }} password: ${{ secrets.GTCR_TOKEN }} - name: Create manifest list and push working-directory: /tmp/digests run: | echo $DOCKER_METADATA_OUTPUT_JSON echo $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2)) docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ $(printf '${{ env.CONTAINER_NAME }}@sha256:%s ' $(ls -p */* | cut -d / -f 2)) - name: Inspect image run: | docker buildx imagetools inspect ${{ env.CONTAINER_NAME }}:${{ steps.meta.outputs.version }} - name: Notify uses: rjstone/discord-webhook-notify@v1 if: failure() with: severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }} details: Build ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}! webhookUrl: ${{ secrets.DISCORD_WEBHOOK }} username: Gitea avatarUrl: ${{ vars.RUNNER_ICON_URL }} Create-Release: runs-on: [ubuntu-latest, linux/amd64] needs: [Merge-Images] steps: - name: Checkout uses: actions/checkout@v2 - name: Get tag run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV - name: Prepare tea run: | # Download tea from Gitea release page echo "Downloading Tea v${{ env.TEA_VERSION }}" && \ wget -q -O tea https://gitea.com/gitea/tea/releases/download/v${{ env.TEA_VERSION }}/tea-${{ env.TEA_VERSION }}-linux-amd64 && \ echo "Downloaded Tea" && \ chmod +x tea && \ # Login to Gitea echo "Logging in to Gitea using Tea" && \ ./tea login add --name Gitea --url https://git.domain.com --token ${{ secrets.GITHUB_TOKEN }} && \ echo "Done" - name: Make release run: | echo "Creating release" && \ ./tea release create --login "Gitea" --repo ${{ env.GITHUB_REPOSITORY }} --tag ${{ env.RELEASE_VERSION }} -t ${{ env.RELEASE_VERSION }} && \ echo "Done" - name: Notify uses: rjstone/discord-webhook-notify@v1 if: failure() with: severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }} details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}! webhookUrl: ${{ secrets.DISCORD_WEBHOOK }} username: Gitea avatarUrl: ${{ vars.RUNNER_ICON_URL }} Notify: runs-on: ubuntu-latest needs: [Build-Image, Merge-Images, Create-Release] steps: - name: Notify uses: rjstone/discord-webhook-notify@v1 if: always() with: severity: ${{ job.status == 'success' && 'info' || (job.status == 'cancelled' && 'warn' || 'error') }} details: Release ${{ job.status == 'success' && 'succeeded' || (job.status == 'cancelled' && 'cancelled' || 'failed') }}! webhookUrl: ${{ secrets.DISCORD_WEBHOOK }} username: Gitea avatarUrl: ${{ vars.RUNNER_ICON_URL }} ``` ### Gitea Version 1.21.6 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System _No response_ ### How are you running Gitea? Gitea 1.21.6 running in Docker on an Ubuntu host, with Minio storage for packages only. ### Database MySQL/MariaDB
GiteaMirror added the topic/packagestype/bug labels 2025-11-02 10:14:56 -06:00
Author
Owner

@lunny commented on GitHub (Sep 12, 2024):

This should be closed by #32022

@lunny commented on GitHub (Sep 12, 2024): This should be closed by #32022
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#12583