github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6413] Ollama Docker Rocm facing constant issues #4030

New Issue
Closed
opened 2026-04-12 14:54:44 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 14:54:44 -05:00
Owner
Copy Link

Originally created by @harshb20 on GitHub (Aug 19, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6413

What is the issue?

So although I have a consistent working Rocm install in Fedora 40- gfx1030 target, also I already have llama.cpp working perfectly in Rocm and Vulkan targets and even Ollama in CPU only mode, but have been consistently getting docker: layers from manifest don't match image configuration. error when trying to run ollama:rocm image in docker. For reference normal ollama/ollama in Docker works fine on CPU only. This issue is almost universal and is existent in any other ollama:x.y.x-rocm tag too.

$ docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm
Unable to find image 'ollama/ollama:rocm' locally
rocm: Pulling from ollama/ollama
2d473b07cdd5: Pull complete 
b3800af90b90: Pull complete 
35d951731348: Pull complete 
93191fa4e1f8: Pull complete 
ec6b9c70f06a: Pull complete 
4767b6298c18: Pull complete 
bd707b33a96a: Pull complete 
10645e0d079c: Pull complete 
ba086e88d2e3: Pull complete 
f35b80e1d7b6: Pull complete 
13bfe13f7f56: Pull complete 
efc528f475fe: Pull complete 
1db71b1d7c67: Pull complete 
c630fdf8b6e2: Pull complete 
3c9d729a222e: Pull complete 
docker: layers from manifest don't match image configuration.
See 'docker run --help'.

OS

Linux, Docker

GPU

AMD

CPU

AMD

Ollama version

0.3.6

Originally created by @harshb20 on GitHub (Aug 19, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6413 ### What is the issue? So although I have a consistent working Rocm install in Fedora 40- gfx1030 target, also I already have llama.cpp working perfectly in Rocm and Vulkan targets and even Ollama in CPU only mode, but have been consistently getting `docker: layers from manifest don't match image configuration.` error when trying to run `ollama:rocm` image in docker. For reference normal `ollama/ollama ` in Docker works fine on CPU only. This issue is almost universal and is existent in any other `ollama:x.y.x-rocm` tag too. ``` $ docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm Unable to find image 'ollama/ollama:rocm' locally rocm: Pulling from ollama/ollama 2d473b07cdd5: Pull complete b3800af90b90: Pull complete 35d951731348: Pull complete 93191fa4e1f8: Pull complete ec6b9c70f06a: Pull complete 4767b6298c18: Pull complete bd707b33a96a: Pull complete 10645e0d079c: Pull complete ba086e88d2e3: Pull complete f35b80e1d7b6: Pull complete 13bfe13f7f56: Pull complete efc528f475fe: Pull complete 1db71b1d7c67: Pull complete c630fdf8b6e2: Pull complete 3c9d729a222e: Pull complete docker: layers from manifest don't match image configuration. See 'docker run --help'. ``` ### OS Linux, Docker ### GPU AMD ### CPU AMD ### Ollama version 0.3.6
GiteaMirror added the bug label 2026-04-12 14:54:44 -05:00
GiteaMirror commented 2026-04-12 14:54:46 -05:00
Author
Owner
Copy Link

@AllenTzul commented on GitHub (Aug 21, 2024):

I am having the exact same issue on Fedora 40. For me the issue started after 0.1.42-rocm.

Command output:

docker pull ollama/ollama:0.3.6-rocm
0.3.6-rocm: Pulling from ollama/ollama
2d473b07cdd5: Pull complete 
b3800af90b90: Pull complete 
35d951731348: Pull complete 
93191fa4e1f8: Pull complete 
ec6b9c70f06a: Pull complete 
4767b6298c18: Pull complete 
bd707b33a96a: Pull complete 
10645e0d079c: Pull complete 
ba086e88d2e3: Pull complete 
f35b80e1d7b6: Pull complete 
13bfe13f7f56: Pull complete 
efc528f475fe: Pull complete 
1db71b1d7c67: Pull complete 
4177e5cd44fe: Pull complete 
f1fe20d85e24: Pull complete 
layers from manifest don't match image configuration

Docker info:

Client: Docker Engine - Community
 Version:           27.1.2
 API version:       1.46
 Go version:        go1.21.13
 Git commit:        d01f264
 Built:             Mon Aug 12 11:52:28 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.1.2
  API version:      1.46 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       f9522e5
  Built:            Mon Aug 12 11:50:46 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
<!-- gh-comment-id:2302441301 --> @AllenTzul commented on GitHub (Aug 21, 2024): I am having the exact same issue on Fedora 40. For me the issue started after **0.1.42-rocm**. **Command output:** ``` docker pull ollama/ollama:0.3.6-rocm 0.3.6-rocm: Pulling from ollama/ollama 2d473b07cdd5: Pull complete b3800af90b90: Pull complete 35d951731348: Pull complete 93191fa4e1f8: Pull complete ec6b9c70f06a: Pull complete 4767b6298c18: Pull complete bd707b33a96a: Pull complete 10645e0d079c: Pull complete ba086e88d2e3: Pull complete f35b80e1d7b6: Pull complete 13bfe13f7f56: Pull complete efc528f475fe: Pull complete 1db71b1d7c67: Pull complete 4177e5cd44fe: Pull complete f1fe20d85e24: Pull complete layers from manifest don't match image configuration ``` **Docker info:** ``` Client: Docker Engine - Community Version: 27.1.2 API version: 1.46 Go version: go1.21.13 Git commit: d01f264 Built: Mon Aug 12 11:52:28 2024 OS/Arch: linux/amd64 Context: default Server: Docker Engine - Community Engine: Version: 27.1.2 API version: 1.46 (minimum version 1.24) Go version: go1.21.13 Git commit: f9522e5 Built: Mon Aug 12 11:50:46 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.7.20 GitCommit: 8fc6bcff51318944179630522a095cc9dbf9f353 runc: Version: 1.1.13 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0 ```
GiteaMirror commented 2026-04-12 14:54:47 -05:00
Author
Owner
Copy Link

@talliansaotome commented on GitHub (Aug 21, 2024):

#Having basically the same issue in podman:

Trying to pull docker.io/ollama/ollama:rocm...
Getting image source signatures
Copying blob 4767b6298c18 skipped: already exists  
Copying blob 2d473b07cdd5 skipped: already exists  
Copying blob bd707b33a96a skipped: already exists  
Copying blob b3800af90b90 skipped: already exists  
Copying blob 93191fa4e1f8 skipped: already exists  
Copying blob ec6b9c70f06a skipped: already exists  
Copying blob 10645e0d079c skipped: already exists  
Copying blob f35b80e1d7b6 skipped: already exists  
Copying blob ba086e88d2e3 skipped: already exists  
Copying blob 13bfe13f7f56 skipped: already exists  
Copying blob efc528f475fe skipped: already exists  
Copying blob 3c9d729a222e done   | 
Copying blob 1db71b1d7c67 done   | 
Copying blob c630fdf8b6e2 done   | 
Copying blob 35d951731348 skipped: already exists  
Error: writing blob: storing blob to file "/var/tmp/container_images_storage1414294462/2": pigz: skipping: <stdin>: corrupted -- crc32 mismatch: exit status 1

The command being run is podman pull ollama/ollama:rocm

Client:       Podman Engine
Version:      5.2.1
API Version:  5.2.1
Go Version:   go1.22.5
Built:        Tue Aug 13 18:00:00 2024
OS/Arch:      linux/amd64
<!-- gh-comment-id:2302666764 --> @talliansaotome commented on GitHub (Aug 21, 2024): #Having basically the same issue in podman: ```✔ docker.io/ollama/ollama:rocm Trying to pull docker.io/ollama/ollama:rocm... Getting image source signatures Copying blob 4767b6298c18 skipped: already exists Copying blob 2d473b07cdd5 skipped: already exists Copying blob bd707b33a96a skipped: already exists Copying blob b3800af90b90 skipped: already exists Copying blob 93191fa4e1f8 skipped: already exists Copying blob ec6b9c70f06a skipped: already exists Copying blob 10645e0d079c skipped: already exists Copying blob f35b80e1d7b6 skipped: already exists Copying blob ba086e88d2e3 skipped: already exists Copying blob 13bfe13f7f56 skipped: already exists Copying blob efc528f475fe skipped: already exists Copying blob 3c9d729a222e done | Copying blob 1db71b1d7c67 done | Copying blob c630fdf8b6e2 done | Copying blob 35d951731348 skipped: already exists Error: writing blob: storing blob to file "/var/tmp/container_images_storage1414294462/2": pigz: skipping: <stdin>: corrupted -- crc32 mismatch: exit status 1 ``` The command being run is `podman pull ollama/ollama:rocm` ```$ podman version Client: Podman Engine Version: 5.2.1 API Version: 5.2.1 Go Version: go1.22.5 Built: Tue Aug 13 18:00:00 2024 OS/Arch: linux/amd64 ```
GiteaMirror commented 2026-04-12 14:54:47 -05:00
Author
Owner
Copy Link

@Jafner commented on GitHub (Aug 24, 2024):

I looped over each version backward from 3.7.0-rc6-rocm and found that the most recent version that pulled correctly was 0.2.5-rocm.

Looks like a bunch of changes came through between 0.2.5 and 0.2.6: diff. But the only change I can see that relates specifically to the Docker image build process is the upgrade from rocm version 6.1.1 to 6.1.2.

This is about as far as my knowledge allows me to investigate this issue.

<!-- gh-comment-id:2308579428 --> @Jafner commented on GitHub (Aug 24, 2024): I looped over each version backward from 3.7.0-rc6-rocm and found that the most recent version that pulled correctly was 0.2.5-rocm. Looks like a bunch of changes came through between 0.2.5 and 0.2.6: [diff](https://github.com/ollama/ollama/compare/v0.2.5...v0.2.6). But the only change I can see that relates specifically to the Docker image build process is the upgrade from rocm version 6.1.1 to 6.1.2. This is about as far as my knowledge allows me to investigate this issue.
GiteaMirror commented 2026-04-12 14:54:48 -05:00
Author
Owner
Copy Link

@arecatsreal commented on GitHub (Aug 25, 2024):

There is a problem with the fedora pigz package, If you just uninstall it and pull then docker image, It should work without problem. As well as this issue seems to be a duplicate of #5979.

<!-- gh-comment-id:2308705403 --> @arecatsreal commented on GitHub (Aug 25, 2024): There is a problem with the fedora pigz package, If you just uninstall it and pull then docker image, It should work without problem. As well as this issue seems to be a duplicate of #5979.
GiteaMirror commented 2026-04-12 14:54:49 -05:00
Author
Owner
Copy Link

@harshb20 commented on GitHub (Aug 26, 2024):

There is a problem with the fedora pigz package, If you just uninstall it and pull then docker image, It should work without problem. As well as this issue seems to be a duplicate of #5979.

Hey, it was indeed the problem, i now have it working correctly.

<!-- gh-comment-id:2310340123 --> @harshb20 commented on GitHub (Aug 26, 2024): > There is a problem with the fedora pigz package, If you just uninstall it and pull then docker image, It should work without problem. As well as this issue seems to be a duplicate of #5979. Hey, it was indeed the problem, i now have it working correctly.
GiteaMirror commented 2026-04-12 14:54:50 -05:00
Author
Owner
Copy Link

@talliansaotome commented on GitHub (Aug 26, 2024):

I wish I had the option of uninstalling pigz, but I am on an immutable distro.

As a note, I was able to pull ollama:latest, just not :rocm. I also wasn't able to pull it on an Alma Linux 9 machine.

<!-- gh-comment-id:2311251701 --> @talliansaotome commented on GitHub (Aug 26, 2024): I wish I had the option of uninstalling pigz, but I am on an immutable distro. As a note, I *was* able to pull ollama:latest, just not :rocm. I also wasn't able to pull it on an Alma Linux 9 machine.
GiteaMirror commented 2026-04-12 14:54:51 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Sep 3, 2024):

Let's track this under #5979

@talliansaotome I don't know if this will work, but you could try working around this by pulling on another system where you can uninstall pigz, then docker save and docker load between the systems.

<!-- gh-comment-id:2327632707 --> @dhiltgen commented on GitHub (Sep 3, 2024): Let's track this under #5979 @talliansaotome I don't know if this will work, but you could try working around this by pulling on another system where you can uninstall pigz, then `docker save` and `docker load` between the systems.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4030
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?