github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-13 17:34:40 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #4393] Ollama panics when encountering a GGUF with no vocab #80426

New Issue
Closed
opened 2026-05-09 08:54:13 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-09 08:54:13 -05:00
Owner
Copy Link
Copy Source

Originally created by @laik on GitHub (May 13, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4393

Originally assigned to: @mxyng on GitHub.

ref https://github.com/ollama/ollama/blob/main/docs/import.md convert pytorch model

The model has been imported, but an error occurred during query loading.
image

What is the issue?

2024/05/13 17:12:11 routes.go:1006: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]"
time=2024-05-13T17:12:11.553+08:00 level=INFO source=images.go:704 msg="total blobs: 13"
time=2024-05-13T17:12:11.553+08:00 level=INFO source=images.go:711 msg="total unused blobs removed: 0"
time=2024-05-13T17:12:11.554+08:00 level=INFO source=routes.go:1052 msg="Listening on [::]:11434 (version 0.1.37)"
time=2024-05-13T17:12:11.642+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama2629975841/runners
time=2024-05-13T17:12:16.254+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v60002 cpu cpu_avx]"
time=2024-05-13T17:12:16.447+08:00 level=INFO source=types.go:71 msg="inference compute" id=GPU-360b2b54-a06a-dcbf-7cd9-fa43aeb7f41b library=cuda compute=7.5 driver=12.2 name="NVIDIA GeForce RTX 2080 Ti" total="21.7 GiB" available="21.5 GiB"
[GIN] 2024/05/13 - 17:12:33 | 200 | 41.434µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/05/13 - 17:12:33 | 200 | 564.532µs | 127.0.0.1 | POST "/api/show"
[GIN] 2024/05/13 - 17:12:33 | 200 | 334.999µs | 127.0.0.1 | POST "/api/show"
panic: interface conversion: interface {} is nil, not []interface {}

goroutine 13 [running]:
github.com/ollama/ollama/llm.GGML.GraphSize({{0x1207e888?, 0xc00071a300?}, {0x1207e838?, 0xc00034f770?}}, 0x800, 0x200)
github.com/ollama/ollama/llm/ggml.go:273 +0x908
github.com/ollama/ollama/llm.EstimateGPULayers({0xc0004b8360?, 0xc0006218e8?, 0x524e64?}, 0xc0000370a0, {0x0, 0x0, 0xc000621920?}, {{0x0, 0x800, 0x200, ...}, ...})
github.com/ollama/ollama/llm/memory.go:59 +0x3dd
github.com/ollama/ollama/llm.PredictServerFit({0xc000621d18?, 0xc000621a30?, 0x801d84?}, 0xc0000370a0, {0x37eb04000?, _, _}, {0x0, 0x0, 0x0}, ...)
github.com/ollama/ollama/llm/memory.go:19 +0xbd
github.com/ollama/ollama/server.pickBestFitGPUs(0xc0004c41c0, 0xc0000370a0, {0xc0004b8090?, 0xfffffffffffffffc?, 0x1269986?})
github.com/ollama/ollama/server/sched.go:559 +0x3b3
github.com/ollama/ollama/server.(*Scheduler).processPending(0xc000711220, {0x12082400, 0xc0007111d0})
github.com/ollama/ollama/server/sched.go:153 +0x99a
github.com/ollama/ollama/server.(*Scheduler).Run.func1()
github.com/ollama/ollama/server/sched.go:91 +0x1f
created by github.com/ollama/ollama/server.(*Scheduler).Run in goroutine 1
github.com/ollama/ollama/server/sched.go:90 +0xb4

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

No response

Originally created by @laik on GitHub (May 13, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4393 Originally assigned to: @mxyng on GitHub. ### ref https://github.com/ollama/ollama/blob/main/docs/import.md convert pytorch model The model has been imported, but an error occurred during query loading. <img width="496" alt="image" src="https://github.com/ollama/ollama/assets/11013552/7dd63eba-1474-439c-8b9b-f51a0d520520"> ### What is the issue? 2024/05/13 17:12:11 routes.go:1006: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" time=2024-05-13T17:12:11.553+08:00 level=INFO source=images.go:704 msg="total blobs: 13" time=2024-05-13T17:12:11.553+08:00 level=INFO source=images.go:711 msg="total unused blobs removed: 0" time=2024-05-13T17:12:11.554+08:00 level=INFO source=routes.go:1052 msg="Listening on [::]:11434 (version 0.1.37)" time=2024-05-13T17:12:11.642+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama2629975841/runners time=2024-05-13T17:12:16.254+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v60002 cpu cpu_avx]" time=2024-05-13T17:12:16.447+08:00 level=INFO source=types.go:71 msg="inference compute" id=GPU-360b2b54-a06a-dcbf-7cd9-fa43aeb7f41b library=cuda compute=7.5 driver=12.2 name="NVIDIA GeForce RTX 2080 Ti" total="21.7 GiB" available="21.5 GiB" [GIN] 2024/05/13 - 17:12:33 | 200 | 41.434µs | 127.0.0.1 | HEAD "/" [GIN] 2024/05/13 - 17:12:33 | 200 | 564.532µs | 127.0.0.1 | POST "/api/show" [GIN] 2024/05/13 - 17:12:33 | 200 | 334.999µs | 127.0.0.1 | POST "/api/show" panic: interface conversion: interface {} is nil, not []interface {} goroutine 13 [running]: github.com/ollama/ollama/llm.GGML.GraphSize({{0x1207e888?, 0xc00071a300?}, {0x1207e838?, 0xc00034f770?}}, 0x800, 0x200) github.com/ollama/ollama/llm/ggml.go:273 +0x908 github.com/ollama/ollama/llm.EstimateGPULayers({0xc0004b8360?, 0xc0006218e8?, 0x524e64?}, 0xc0000370a0, {0x0, 0x0, 0xc000621920?}, {{0x0, 0x800, 0x200, ...}, ...}) github.com/ollama/ollama/llm/memory.go:59 +0x3dd github.com/ollama/ollama/llm.PredictServerFit({0xc000621d18?, 0xc000621a30?, 0x801d84?}, 0xc0000370a0, {0x37eb04000?, _, _}, {0x0, 0x0, 0x0}, ...) github.com/ollama/ollama/llm/memory.go:19 +0xbd github.com/ollama/ollama/server.pickBestFitGPUs(0xc0004c41c0, 0xc0000370a0, {0xc0004b8090?, 0xfffffffffffffffc?, 0x1269986?}) github.com/ollama/ollama/server/sched.go:559 +0x3b3 github.com/ollama/ollama/server.(*Scheduler).processPending(0xc000711220, {0x12082400, 0xc0007111d0}) github.com/ollama/ollama/server/sched.go:153 +0x99a github.com/ollama/ollama/server.(*Scheduler).Run.func1() github.com/ollama/ollama/server/sched.go:91 +0x1f created by github.com/ollama/ollama/server.(*Scheduler).Run in goroutine 1 github.com/ollama/ollama/server/sched.go:90 +0xb4 ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version _No response_
GiteaMirror added the createbug labels 2026-05-09 08:54:13 -05:00
GiteaMirror commented 2026-05-09 08:54:16 -05:00
Author
Owner
Copy Link
Copy Source

@pdevine commented on GitHub (May 14, 2024):

It looks like convert.py is creating a gguf which doesn't have the tokens KV. In llm.GraphSize() we're panicking when trying to get the vocab.

vocab := uint64(len(llm.KV()["tokenizer.ggml.tokens"].([]any)))

@laik the convert.py script is broken and not correctly creating the GGUF file. Ollama should handle this and tell you that the GGUF file is invalid.

<!-- gh-comment-id:2109411777 --> @pdevine commented on GitHub (May 14, 2024): It looks like `convert.py` is creating a gguf which doesn't have the tokens KV. In `llm.GraphSize()` we're panicking when trying to get the vocab. > vocab := uint64(len(llm.KV()["tokenizer.ggml.tokens"].([]any))) @laik the `convert.py` script is broken and not correctly creating the GGUF file. Ollama should handle this and tell you that the GGUF file is invalid.
GiteaMirror commented 2026-05-09 08:54:17 -05:00
Author
Owner
Copy Link
Copy Source

@laik commented on GitHub (May 17, 2024):

It looks like convert.py is creating a gguf which doesn't have the tokens KV. In llm.GraphSize() we're panicking when trying to get the vocab.

vocab := uint64(len(llm.KV()["tokenizer.ggml.tokens"].([]any)))

@laik the convert.py script is broken and not correctly creating the GGUF file. Ollama should handle this and tell you that the GGUF file is invalid.

thanks, that's the problem, I've solved it

<!-- gh-comment-id:2116474078 --> @laik commented on GitHub (May 17, 2024): > It looks like `convert.py` is creating a gguf which doesn't have the tokens KV. In `llm.GraphSize()` we're panicking when trying to get the vocab. > > > vocab := uint64(len(llm.KV()["tokenizer.ggml.tokens"].([]any))) > > @laik the `convert.py` script is broken and not correctly creating the GGUF file. Ollama should handle this and tell you that the GGUF file is invalid. thanks, that's the problem, I've solved it
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels bug create
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#80426