github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3563] Cannot import command-r-plus gguf #64237

New Issue
Closed
opened 2026-05-03 16:42:31 -05:00 by GiteaMirror · 15 comments
GiteaMirror commented 2026-05-03 16:42:31 -05:00
Owner
Copy Link

Originally created by @jason-c-kwan on GitHub (Apr 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3563

What is the issue?

Just cloned ollama earlier today after the merging of PR#6491 in llama.cpp, so it should be able to deal with command-r-plus. There is already some quants of command-r-plus on ollama, but I wanted to import the full range for testing.

Using the GGUFs from dranger003/c4ai-command-r-plus-iMat.GGUF, ./ollama create fails with the following:

transferring model data 
creating model layer 
Error: invalid file magic

Note: I tested with ggml-c4ai-command-r-plus-104b-iq1_s.gguf and gave up after.

Using the GGUFs from pmysl/c4ai-command-r-plus-GGUF, ./ollama create seems to run fine but then ./ollama run fails:

Error: exception done_getting_tensors: wrong number of tensors; expected 642, got 514

What did you expect to see?

For ./ollama create, it should report success. For ./ollama run, I should get an interactive prompt and of course intelligible output from the model.

Steps to reproduce

I cloned Ollama

Installed go 1.22 and cuda 12

(I am using Linux mint)

go generate ./...

go build .

Then attempted to make a Modelfile and did ./ollama create and ./ollama run.

Are there any recent changes that introduced the issue?

No response

OS

Linux

Architecture

amd64

Platform

No response

Ollama version

0.1.30

GPU

Nvidia

GPU info

2 x 3090, 1 x 3060 TI, 1 x Quadro M6000

CPU

Intel

Other software

No response

Originally created by @jason-c-kwan on GitHub (Apr 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3563 ### What is the issue? Just cloned ollama earlier today after the merging of PR#6491 in llama.cpp, so it should be able to deal with command-r-plus. There is already some quants of command-r-plus on ollama, but I wanted to import the full range for testing. Using the GGUFs from dranger003/c4ai-command-r-plus-iMat.GGUF, `./ollama create` fails with the following: ```bash transferring model data creating model layer Error: invalid file magic ``` Note: I tested with ggml-c4ai-command-r-plus-104b-iq1_s.gguf and gave up after. Using the GGUFs from pmysl/c4ai-command-r-plus-GGUF, `./ollama create` seems to run fine but then `./ollama run` fails: ```bash Error: exception done_getting_tensors: wrong number of tensors; expected 642, got 514 ``` ### What did you expect to see? For `./ollama create`, it should report `success`. For `./ollama run`, I should get an interactive prompt and of course intelligible output from the model. ### Steps to reproduce I cloned Ollama Installed go 1.22 and cuda 12 (I am using Linux mint) `go generate ./...` `go build .` Then attempted to make a Modelfile and did `./ollama create` and `./ollama run`. ### Are there any recent changes that introduced the issue? _No response_ ### OS Linux ### Architecture amd64 ### Platform _No response_ ### Ollama version 0.1.30 ### GPU Nvidia ### GPU info 2 x 3090, 1 x 3060 TI, 1 x Quadro M6000 ### CPU Intel ### Other software _No response_
GiteaMirror added the bug label 2026-05-03 16:42:31 -05:00
GiteaMirror commented 2026-05-03 16:42:36 -05:00
Author
Owner
Copy Link

@sammcj commented on GitHub (Apr 9, 2024):

I've had problems with Ollama and IQ quants before, I'm not sure Ollama works with them but it would be nice to have these available.

<!-- gh-comment-id:2046154990 --> @sammcj commented on GitHub (Apr 9, 2024): I've had problems with Ollama and IQ quants before, I'm not sure Ollama works with them but it would be nice to have these available.
GiteaMirror commented 2026-05-03 16:42:37 -05:00
Author
Owner
Copy Link

@jason-c-kwan commented on GitHub (Apr 9, 2024):

I've had problems with Ollama and IQ quants before, I'm not sure Ollama works with them but it would be nice to have these available.

Problem is I couldn't get the non iq quant to work either, although perhaps I am doing something wrong. I know you have been able to do it so any insight would be appreciated.

<!-- gh-comment-id:2046162116 --> @jason-c-kwan commented on GitHub (Apr 9, 2024): > I've had problems with Ollama and IQ quants before, I'm not sure Ollama works with them but it would be nice to have these available. Problem is I couldn't get the non iq quant to work either, although perhaps I am doing something wrong. I know you have been able to do it so any insight would be appreciated.
GiteaMirror commented 2026-05-03 16:42:38 -05:00
Author
Owner
Copy Link

@zhaopengme commented on GitHub (Apr 10, 2024):

+1

<!-- gh-comment-id:2046425679 --> @zhaopengme commented on GitHub (Apr 10, 2024): +1
GiteaMirror commented 2026-05-03 16:42:39 -05:00
Author
Owner
Copy Link

@chigkim commented on GitHub (Apr 10, 2024):

I downloaded q3_k_m part 1 and part 2 and merged them with gguf_split --merge.
It worked fine with the latest llama.cpp.
I tried to import to Ollama and run it, I get the same message about wrong number of tensors like OP.
Hopefully it gets sorted out soon.

<!-- gh-comment-id:2046640974 --> @chigkim commented on GitHub (Apr 10, 2024): I downloaded q3_k_m part 1 and part 2 and merged them with gguf_split --merge. It worked fine with the latest llama.cpp. I tried to import to Ollama and run it, I get the same message about wrong number of tensors like OP. Hopefully it gets sorted out soon.
GiteaMirror commented 2026-05-03 16:42:40 -05:00
Author
Owner
Copy Link

@sammcj commented on GitHub (Apr 10, 2024):

Make sure you built Ollama from the latest source.

If it helps this is my personal build script, feel free to modify etc... https://github.com/sammcj/scripts/blob/master/build_ollama.sh

(Note: Run with PATCH_OLLAMA=false ./build_ollama.sh if you don't want my build tweaks)

<!-- gh-comment-id:2046698014 --> @sammcj commented on GitHub (Apr 10, 2024): Make sure you built Ollama from the latest source. If it helps this is my personal build script, feel free to modify etc... https://github.com/sammcj/scripts/blob/master/build_ollama.sh (Note: Run with `PATCH_OLLAMA=false ./build_ollama.sh` if you don't want my build tweaks)
GiteaMirror commented 2026-05-03 16:42:41 -05:00
Author
Owner
Copy Link

@taozhiyuai commented on GitHub (Apr 10, 2024):

can import, but error when run it

https://github.com/ollama/ollama/issues/3577

<!-- gh-comment-id:2047523918 --> @taozhiyuai commented on GitHub (Apr 10, 2024): can import, but error when run it https://github.com/ollama/ollama/issues/3577
GiteaMirror commented 2026-05-03 16:42:42 -05:00
Author
Owner
Copy Link

@jason-c-kwan commented on GitHub (Apr 10, 2024):

I tried a few more things without success. I confirmed that my local built ollama cannot pull and run sammcj/cohereforai_c4ai-command-r-plus. I also tried to import a non-iq gguf from the dranger0003 version of the model on Hugging Face. I also tried another version of command-r-plus that seems to have been uploaded to the ollama hub (jmorgan/command-r-plus). It seems that in all cases I get the same error when running:

Error: exception done_getting_tensors: wrong number of tensors; expected 642, got 514

So the first error I wrote about above seems to be due to the iq quant versions as @sammcj mentioned. The other error seems to be quite consistent. @sammcj I did try your build script, but it seemed to have a lot of parts specific to macs. I made sure to clone the latest llama.cpp into llm/ and built the normal way again. Got the same error. I guess those users with Macs might be able to use their script successfully?

<!-- gh-comment-id:2048263407 --> @jason-c-kwan commented on GitHub (Apr 10, 2024): I tried a few more things without success. I confirmed that my local built ollama cannot pull and run [sammcj/cohereforai_c4ai-command-r-plus](https://ollama.com/sammcj/cohereforai_c4ai-command-r-plus). I also tried to import a non-iq gguf from the dranger0003 version of the model on Hugging Face. I also tried another version of command-r-plus that seems to have been uploaded to the ollama hub (jmorgan/command-r-plus). It seems that in all cases I get the same error when running: ``` Error: exception done_getting_tensors: wrong number of tensors; expected 642, got 514 ``` So the first error I wrote about above seems to be due to the iq quant versions as @sammcj mentioned. The other error seems to be quite consistent. @sammcj I did try your build script, but it seemed to have a lot of parts specific to macs. I made sure to clone the latest llama.cpp into llm/ and built the normal way again. Got the same error. I guess those users with Macs might be able to use their script successfully?
GiteaMirror commented 2026-05-03 16:42:43 -05:00
Author
Owner
Copy Link

@atgreen commented on GitHub (Apr 11, 2024):

I was able to reproduce this with v0.1.31, but it works now with the ollama v0.1.32 prerelease.

<!-- gh-comment-id:2048679886 --> @atgreen commented on GitHub (Apr 11, 2024): I was able to reproduce this with v0.1.31, but it works now with the ollama v0.1.32 prerelease.
GiteaMirror commented 2026-05-03 16:42:44 -05:00
Author
Owner
Copy Link

@DirtyKnightForVi commented on GitHub (Apr 11, 2024):

I downloaded q3_k_m part 1 and part 2 and merged them with gguf_split --merge. It worked fine with the latest llama.cpp. I tried to import to Ollama and run it, I get the same message about wrong number of tensors like OP. Hopefully it gets sorted out soon.

hello, I'm kinda stuck on how to merge these files. Any tips you could throw my way?

<!-- gh-comment-id:2049006118 --> @DirtyKnightForVi commented on GitHub (Apr 11, 2024): > I downloaded q3_k_m part 1 and part 2 and merged them with gguf_split --merge. It worked fine with the latest llama.cpp. I tried to import to Ollama and run it, I get the same message about wrong number of tensors like OP. Hopefully it gets sorted out soon. hello, I'm kinda stuck on how to merge these files. Any tips you could throw my way?
GiteaMirror commented 2026-05-03 16:42:45 -05:00
Author
Owner
Copy Link

@chigkim commented on GitHub (Apr 11, 2024):

I can also confirm 0.3.2-rc1 works!
@DirtyKnightForVi you need to use gguf-split from llama.cpp.
https://github.com/ggerganov/llama.cpp

<!-- gh-comment-id:2049996641 --> @chigkim commented on GitHub (Apr 11, 2024): I can also confirm 0.3.2-rc1 works! @DirtyKnightForVi you need to use gguf-split from llama.cpp. https://github.com/ggerganov/llama.cpp
GiteaMirror commented 2026-05-03 16:42:46 -05:00
Author
Owner
Copy Link

@jason-c-kwan commented on GitHub (Apr 11, 2024):

Yes, thanks @atgreen ! Can confirm it works. Will close this issue.

<!-- gh-comment-id:2050000518 --> @jason-c-kwan commented on GitHub (Apr 11, 2024): Yes, thanks @atgreen ! Can confirm it works. Will close this issue.
GiteaMirror commented 2026-05-03 16:42:47 -05:00
Author
Owner
Copy Link

@ehartford commented on GitHub (Apr 11, 2024):

how to update to v0.1.32?

$ ollama --version
ollama version is 0.1.31
<!-- gh-comment-id:2050507733 --> @ehartford commented on GitHub (Apr 11, 2024): how to update to v0.1.32? ``` $ ollama --version ollama version is 0.1.31 ```
GiteaMirror commented 2026-05-03 16:42:48 -05:00
Author
Owner
Copy Link

@dector commented on GitHub (Apr 11, 2024):

how to update to v0.1.32?

I guess you can try RC1 here: https://github.com/ollama/ollama/releases/tag/v0.1.32-rc1

<!-- gh-comment-id:2050588606 --> @dector commented on GitHub (Apr 11, 2024): > how to update to v0.1.32? I guess you can try RC1 here: https://github.com/ollama/ollama/releases/tag/v0.1.32-rc1
GiteaMirror commented 2026-05-03 16:42:49 -05:00
Author
Owner
Copy Link

@DirtyKnightForVi commented on GitHub (Apr 12, 2024):

I can also confirm 0.3.2-rc1 works! @DirtyKnightForVi you need to use gguf-split from llama.cpp. https://github.com/ggerganov/llama.cpp

Thank you so much ! Actually, I upgraded ollama to the latest preview version, but I might have messed up the way I used the downloaded model files. Instead of using gguf_split --merge, I just went with a simple cat. It seems like when ollama loads the model files (using Modelfile locally), it prefers loading them as whole rather than splited.

<!-- gh-comment-id:2050783369 --> @DirtyKnightForVi commented on GitHub (Apr 12, 2024): > I can also confirm 0.3.2-rc1 works! @DirtyKnightForVi you need to use gguf-split from llama.cpp. https://github.com/ggerganov/llama.cpp Thank you so much ! Actually, I upgraded ollama to the latest preview version, but I might have messed up the way I used the downloaded model files. Instead of using `gguf_split --merge`, I just went with a simple `cat`. It seems like when ollama loads the model files (using Modelfile locally), it prefers loading them as whole rather than splited.
GiteaMirror commented 2026-05-03 16:42:50 -05:00
Author
Owner
Copy Link

@DirtyKnightForVi commented on GitHub (Apr 12, 2024):

Yes, thanks @atgreen ! Can confirm it works. Will close this issue.

Did you already merge the model files before editing Modelfile? For the files that have been split, I'm not sure if 'FROM' should be followed by '0001of0002.gguf', or if you need to specify the complete merged model file?

<!-- gh-comment-id:2050795342 --> @DirtyKnightForVi commented on GitHub (Apr 12, 2024): > Yes, thanks @atgreen ! Can confirm it works. Will close this issue. Did you already merge the model files before editing Modelfile? For the files that have been split, I'm not sure if 'FROM' should be followed by '0001of0002.gguf', or if you need to specify the complete merged model file?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#64237
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?