[GH-ISSUE #11047] llama runner process has terminated: exit status 2 #69347

Closed
opened 2026-05-04 17:52:50 -05:00 by GiteaMirror · 2 comments
Owner

Originally created by @fishfl on GitHub (Jun 11, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11047

What is the issue?

I follow these instructions in CLI:

docker pull ollama/ollama:0.9.0
docker run -it --entrypoint /bin/sh ollama/ollama:latest
ollama serve &
ollama run gemma3

And got the error:
msg="error loading llama server" error="llama runner process has terminated: exit status 2"

And I tried the other versions(0.7.0 / 0.9.1) and the other models, got the same error.

Relevant log output

Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
Your new public key is:

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAWLs45MrvYntAZlkPIn8+/BaGe02MIaaqCVklA+kZPA

time=2025-06-11T10:56:09.245Z level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2025-06-11T10:56:09.246Z level=INFO source=images.go:463 msg="total blobs: 0"
time=2025-06-11T10:56:09.246Z level=INFO source=images.go:470 msg="total unused blobs removed: 0"
time=2025-06-11T10:56:09.246Z level=INFO source=routes.go:1258 msg="Listening on [::]:11434 (version 0.7.0)"
time=2025-06-11T10:56:09.246Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2025-06-11T10:56:09.252Z level=INFO source=gpu.go:377 msg="no compatible GPUs were discovered"
time=2025-06-11T10:56:09.252Z level=INFO source=types.go:130 msg="inference compute" id=0 library=cpu variant="" compute="" driver=0.0 name="" total="31.3 GiB" available="30.3 GiB"
[GIN] 2025/06/11 - 10:56:57 | 200 |      93.216µs |       127.0.0.1 | HEAD     "/"
[GIN] 2025/06/11 - 10:56:57 | 404 |     346.738µs |       127.0.0.1 | POST     "/api/show"
time=2025-06-11T10:56:59.158Z level=INFO source=download.go:177 msg="downloading aeda25e63ebd in 16 208 MB part(s)"
time=2025-06-11T11:00:10.130Z level=INFO source=download.go:177 msg="downloading e0a42594d802 in 1 358 B part(s)"
time=2025-06-11T11:00:11.746Z level=INFO source=download.go:177 msg="downloading dd084c7d92a3 in 1 8.4 KB part(s)"
time=2025-06-11T11:00:13.361Z level=INFO source=download.go:177 msg="downloading 3116c5225075 in 1 77 B part(s)"
time=2025-06-11T11:00:15.019Z level=INFO source=download.go:177 msg="downloading b6ae5839783f in 1 489 B part(s)"
[GIN] 2025/06/11 - 11:00:33 | 200 |         3m35s |       127.0.0.1 | POST     "/api/pull"
[GIN] 2025/06/11 - 11:00:33 | 200 |  142.680498ms |       127.0.0.1 | POST     "/api/show"
time=2025-06-11T11:00:33.379Z level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="30.2 GiB" free_swap="0 B"
time=2025-06-11T11:00:33.381Z level=INFO source=server.go:168 msg=offload library=cpu layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[30.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.6 GiB" memory.required.partial="0 B" memory.required.kv="450.0 MiB" memory.required.allocations="[5.6 GiB]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
time=2025-06-11T11:00:33.532Z level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 8192 --batch-size 512 --threads 8 --no-mmap --parallel 2 --port 36941"
time=2025-06-11T11:00:33.533Z level=INFO source=sched.go:472 msg="loaded runners" count=1
time=2025-06-11T11:00:33.533Z level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
time=2025-06-11T11:00:33.533Z level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding"
time=2025-06-11T11:00:33.558Z level=INFO source=runner.go:836 msg="starting ollama engine"
time=2025-06-11T11:00:33.559Z level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:36941"
time=2025-06-11T11:00:33.698Z level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36
SIGILL: illegal instruction
PC=0x7f7b553f17c8 m=5 sigcode=2
signal arrived during cgo execution
instruction bytes: 0xc4 0xc1 0x7a 0x10 0x54 0x5d 0x0 0xc5 0xea 0x59 0xd 0x49 0x26 0x7 0x0 0x48

goroutine 9 gp=0xc000102e00 m=5 mp=0xc000100008 [syscall]:
runtime.cgocall(0x55f96826bb00, 0xc000046fc8)
        runtime/cgocall.go:167 +0x4b fp=0xc000046fa0 sp=0xc000046f68 pc=0x55f9675afecb
github.com/ollama/ollama/ml/backend/ggml/ggml/src._Cfunc_ggml_backend_load_all_from_path(0x7f7b58000ca0)
        _cgo_gotypes.go:195 +0x3e fp=0xc000046fc8 sp=0xc000046fa0 pc=0x55f9679484de
github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.func1.1({0xc00003e044, 0xf})
        github.com/ollama/ollama/ml/backend/ggml/ggml/src/ggml.go:97 +0xf5 fp=0xc000047060 sp=0xc000046fc8 pc=0x55f967947f75
github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.func1()
        github.com/ollama/ollama/ml/backend/ggml/ggml/src/ggml.go:98 +0x526 fp=0xc0000472f0 sp=0xc000047060 pc=0x55f967947dc6
github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.OnceFunc.func2()
        sync/oncefunc.go:27 +0x62 fp=0xc000047338 sp=0xc0000472f0 pc=0x55f9679477c2
sync.(*Once).doSlow(0x55f968751b40?, 0x0?)
        sync/once.go:78 +0xab fp=0xc000047390 sp=0xc000047338 pc=0x55f9675c4e0b
sync.(*Once).Do(0x0?, 0x55f968751b40?)
        sync/once.go:69 +0x19 fp=0xc0000473b0 sp=0xc000047390 pc=0x55f9675c4d39
github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.OnceFunc.func3()
        sync/oncefunc.go:32 +0x2d fp=0xc0000473e0 sp=0xc0000473b0 pc=0x55f96794772d
github.com/ollama/ollama/ml/backend/ggml.devices()
        github.com/ollama/ollama/ml/backend/ggml/ggml.go:37 +0x1e fp=0xc000047420 sp=0xc0000473e0 pc=0x55f9679b353e
github.com/ollama/ollama/ml/backend/ggml.New({0x55f9688f4b10, 0xc00061c640}, 0xc00012e008, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, 0x0}, ...})
        github.com/ollama/ollama/ml/backend/ggml/ggml.go:89 +0x4df fp=0xc000047d00 sp=0xc000047420 pc=0x55f9679b3abf
github.com/ollama/ollama/ml.NewBackend({0x55f9688f4b10, 0xc00061c640}, 0xc00012e008, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, 0x0}, ...})
        github.com/ollama/ollama/ml/backend.go:87 +0x9a fp=0xc000047d68 sp=0xc000047d00 pc=0x55f9679a6ffa
github.com/ollama/ollama/model.New({0x55f9688f4b10, 0xc00061c640}, {0x7ffe722c9dc3?, 0x0?}, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, ...}, ...})
        github.com/ollama/ollama/model/model.go:107 +0xe6 fp=0xc000047ea8 sp=0xc000047d68 pc=0x55f9679e4a86
github.com/ollama/ollama/runner/ollamarunner.(*Server).loadModel(0xc0000fdd40, {0x55f9688f4b10?, 0xc00061c640?}, {0x7ffe722c9dc3?, 0x0?}, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, ...}, ...}, ...)
        github.com/ollama/ollama/runner/ollamarunner/runner.go:775 +0x93 fp=0xc000047f20 sp=0xc000047ea8 pc=0x55f967a8cef3
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
        github.com/ollama/ollama/runner/ollamarunner/runner.go:872 +0xbd fp=0xc000047fe0 sp=0xc000047f20 pc=0x55f967a8e3bd
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc000047fe8 sp=0xc000047fe0 pc=0x55f9675ba901
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
        github.com/ollama/ollama/runner/ollamarunner/runner.go:872 +0xa2b

goroutine 1 gp=0xc000002380 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000135638 sp=0xc000135618 pc=0x55f9675b31ce
runtime.netpollblock(0x55f9675b0f53?, 0x6754cae6?, 0xf9?)
        runtime/netpoll.go:575 +0xf7 fp=0xc000135670 sp=0xc000135638 pc=0x55f967577fb7
internal/poll.runtime_pollWait(0x7f7b6e1c5eb0, 0x72)
        runtime/netpoll.go:351 +0x85 fp=0xc000135690 sp=0xc000135670 pc=0x55f9675b23e5
internal/poll.(*pollDesc).wait(0xc000507a80?, 0x55f9687bca20?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001356b8 sp=0xc000135690 pc=0x55f967639827
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000507a80)
        internal/poll/fd_unix.go:620 +0x295 fp=0xc000135760 sp=0xc0001356b8 pc=0x55f96763ebf5
net.(*netFD).accept(0xc000507a80)
        net/fd_unix.go:172 +0x29 fp=0xc000135818 sp=0xc000135760 pc=0x55f9676b1069
net.(*TCPListener).accept(0xc000405f00)
        net/tcpsock_posix.go:159 +0x1b fp=0xc000135868 sp=0xc000135818 pc=0x55f9676c6a1b
net.(*TCPListener).Accept(0xc000405f00)
        net/tcpsock.go:380 +0x30 fp=0xc000135898 sp=0xc000135868 pc=0x55f9676c58d0
net/http.(*onceCloseListener).Accept(0x55f9688f4aa0?)
        <autogenerated>:1 +0x24 fp=0xc0001358b0 sp=0xc000135898 pc=0x55f9678dd024
net/http.(*Server).Serve(0xc000035700, {0x55f9688f2718, 0xc000405f00})
        net/http/server.go:3424 +0x30c fp=0xc0001359e0 sp=0xc0001358b0 pc=0x55f9678b48ec
github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034130, 0xd, 0xd})
        github.com/ollama/ollama/runner/ollamarunner/runner.go:900 +0xe69 fp=0xc000135d08 sp=0xc0001359e0 pc=0x55f967a8e029
github.com/ollama/ollama/runner.Execute({0xc000034110?, 0x0?, 0x0?})
        github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000135d30 sp=0xc000135d08 pc=0x55f967a8e949
github.com/ollama/ollama/cmd.NewCLI.func2(0xc000035400?, {0x55f96844706e?, 0x4?, 0x55f968447072?})
        github.com/ollama/ollama/cmd/cmd.go:1387 +0x45 fp=0xc000135d58 sp=0xc000135d30 pc=0x55f9681ea685
github.com/spf13/cobra.(*Command).execute(0xc0004b8f08, {0xc0005167e0, 0xe, 0xe})
        github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000135e78 sp=0xc000135d58 pc=0x55f96772a6bc
github.com/spf13/cobra.(*Command).ExecuteC(0xc00052f508)
        github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000135f30 sp=0xc000135e78 pc=0x55f96772af05
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
        github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000135f50 sp=0xc000135f30 pc=0x55f9681eb00d
runtime.main()
        runtime/proc.go:283 +0x29d fp=0xc000135fe0 sp=0xc000135f50 pc=0x55f96757f5bd
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc000135fe8 sp=0xc000135fe0 pc=0x55f9675ba901

goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000070fa8 sp=0xc000070f88 pc=0x55f9675b31ce
runtime.goparkunlock(...)
        runtime/proc.go:441
runtime.forcegchelper()
        runtime/proc.go:348 +0xb8 fp=0xc000070fe0 sp=0xc000070fa8 pc=0x55f96757f8f8
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc000070fe8 sp=0xc000070fe0 pc=0x55f9675ba901
created by runtime.init.7 in goroutine 1
        runtime/proc.go:336 +0x1a

goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000071780 sp=0xc000071760 pc=0x55f9675b31ce
runtime.goparkunlock(...)
        runtime/proc.go:441
runtime.bgsweep(0xc00009c000)
        runtime/mgcsweep.go:316 +0xdf fp=0xc0000717c8 sp=0xc000071780 pc=0x55f96756a11f
runtime.gcenable.gowrap1()
        runtime/mgc.go:204 +0x25 fp=0xc0000717e0 sp=0xc0000717c8 pc=0x55f96755e505
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc0000717e8 sp=0xc0000717e0 pc=0x55f9675ba901
created by runtime.gcenable in goroutine 1
        runtime/mgc.go:204 +0x66

goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x55f968602328?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000071f78 sp=0xc000071f58 pc=0x55f9675b31ce
runtime.goparkunlock(...)
        runtime/proc.go:441
runtime.(*scavengerState).park(0x55f96917b5c0)
        runtime/mgcscavenge.go:425 +0x49 fp=0xc000071fa8 sp=0xc000071f78 pc=0x55f967567b69
runtime.bgscavenge(0xc00009c000)
        runtime/mgcscavenge.go:658 +0x59 fp=0xc000071fc8 sp=0xc000071fa8 pc=0x55f9675680f9
runtime.gcenable.gowrap2()
        runtime/mgc.go:205 +0x25 fp=0xc000071fe0 sp=0xc000071fc8 pc=0x55f96755e4a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x55f9675ba901
created by runtime.gcenable in goroutine 1
        runtime/mgc.go:205 +0xa5

goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]:
runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000070688?)
        runtime/proc.go:435 +0xce fp=0xc000070630 sp=0xc000070610 pc=0x55f9675b31ce
runtime.runfinq()
        runtime/mfinal.go:196 +0x107 fp=0xc0000707e0 sp=0xc000070630 pc=0x55f96755d4c7
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc0000707e8 sp=0xc0000707e0 pc=0x55f9675ba901
created by runtime.createfing in goroutine 1
        runtime/mfinal.go:166 +0x3d

goroutine 6 gp=0xc0001d08c0 m=nil [chan receive]:
runtime.gopark(0xc000223680?, 0xc000224048?, 0x60?, 0x27?, 0x55f967697da8?)
        runtime/proc.go:435 +0xce fp=0xc000072718 sp=0xc0000726f8 pc=0x55f9675b31ce
runtime.chanrecv(0xc0000a8310, 0x0, 0x1)
        runtime/chan.go:664 +0x445 fp=0xc000072790 sp=0xc000072718 pc=0x55f96754f6c5
runtime.chanrecv1(0x0?, 0x0?)
        runtime/chan.go:506 +0x12 fp=0xc0000727b8 sp=0xc000072790 pc=0x55f96754f252
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
        runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
        runtime/mgc.go:1799 +0x2f fp=0xc0000727e0 sp=0xc0000727b8 pc=0x55f9675616af
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x55f9675ba901
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
        runtime/mgc.go:1794 +0x85

goroutine 7 gp=0xc0001d0e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000072f38 sp=0xc000072f18 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc000072fc8 sp=0xc000072f38 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc000072fe0 sp=0xc000072fc8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00006c738 sp=0xc00006c718 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00006c7c8 sp=0xc00006c738 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00006c7e0 sp=0xc00006c7c8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00006c7e8 sp=0xc00006c7e0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]:
runtime.gopark(0x17aeca47c6d72d?, 0x3?, 0x6c?, 0xd?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00011a738 sp=0xc00011a718 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00011a7c8 sp=0xc00011a738 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00011a7e0 sp=0xc00011a7c8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00011a7e8 sp=0xc00011a7e0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 8 gp=0xc0001d0fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x17aeca4763ae93?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc000073738 sp=0xc000073718 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc0000737c8 sp=0xc000073738 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc0000737e0 sp=0xc0000737c8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 19 gp=0xc0005041c0 m=nil [GC worker (idle)]:
runtime.gopark(0x17aeca4763b8f2?, 0x3?, 0x33?, 0x29?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00006cf38 sp=0xc00006cf18 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00006cfc8 sp=0xc00006cf38 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00006cfe0 sp=0xc00006cfc8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00006cfe8 sp=0xc00006cfe0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 20 gp=0xc000504380 m=nil [GC worker (idle)]:
runtime.gopark(0x17aeca4763aa13?, 0x0?, 0x0?, 0x0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00006d738 sp=0xc00006d718 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00006d7c8 sp=0xc00006d738 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00006d7e0 sp=0xc00006d7c8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00006d7e8 sp=0xc00006d7e0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 21 gp=0xc000504540 m=nil [GC worker (idle)]:
runtime.gopark(0x17aeca47b58963?, 0x3?, 0xc9?, 0xa7?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00006df38 sp=0xc00006df18 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00006dfc8 sp=0xc00006df38 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00006dfe0 sp=0xc00006dfc8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 22 gp=0xc000504700 m=nil [GC worker (idle)]:
runtime.gopark(0x55f969229e00?, 0x1?, 0x27?, 0xfb?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00006e738 sp=0xc00006e718 pc=0x55f9675b31ce
runtime.gcBgMarkWorker(0xc0000a98f0)
        runtime/mgc.go:1423 +0xe9 fp=0xc00006e7c8 sp=0xc00006e738 pc=0x55f9675609c9
runtime.gcBgMarkStartWorkers.gowrap1()
        runtime/mgc.go:1339 +0x25 fp=0xc00006e7e0 sp=0xc00006e7c8 pc=0x55f9675608a5
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00006e7e8 sp=0xc00006e7e0 pc=0x55f9675ba901
created by runtime.gcBgMarkStartWorkers in goroutine 1
        runtime/mgc.go:1339 +0x105

goroutine 10 gp=0xc000102fc0 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x0?, 0x0?, 0x60?, 0xa0?, 0x0?)
        runtime/proc.go:435 +0xce fp=0xc00011ced0 sp=0xc00011ceb0 pc=0x55f9675b31ce
runtime.goparkunlock(...)
        runtime/proc.go:441
runtime.semacquire1(0xc0000fdd48, 0x0, 0x1, 0x0, 0x18)
        runtime/sema.go:188 +0x229 fp=0xc00011cf38 sp=0xc00011ced0 pc=0x55f967592b89
sync.runtime_SemacquireWaitGroup(0x0?)
        runtime/sema.go:110 +0x25 fp=0xc00011cf70 sp=0xc00011cf38 pc=0x55f9675b4be5
sync.(*WaitGroup).Wait(0x0?)
        sync/waitgroup.go:118 +0x48 fp=0xc00011cf98 sp=0xc00011cf70 pc=0x55f9675c6248
github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0000fdd40, {0x55f9688f4b10, 0xc00061c640})
        github.com/ollama/ollama/runner/ollamarunner/runner.go:344 +0x25 fp=0xc00011cfb8 sp=0xc00011cf98 pc=0x55f967a89645
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2()
        github.com/ollama/ollama/runner/ollamarunner/runner.go:876 +0x28 fp=0xc00011cfe0 sp=0xc00011cfb8 pc=0x55f967a8e2c8
runtime.goexit({})
        runtime/asm_amd64.s:1700 +0x1 fp=0xc00011cfe8 sp=0xc00011cfe0 pc=0x55f9675ba901
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
        github.com/ollama/ollama/runner/ollamarunner/runner.go:876 +0xad5

rax    0x6101304cacd
rbx    0x0
rcx    0x0
rdx    0x4f3cd
rdi    0x7ffe72383000
rsi    0x61012ffd700
rbp    0x7f7b5547bc80
rsp    0x7f7b6d17aa80
r8     0x2f46ab17db0e9c
r9     0x7f7b58008df0
r10    0x7ffe72382080
r11    0x7ffe72382090
r12    0x7f7b5549bc80
r13    0x7f7b5554de60
r14    0x7f7b55461ef0
r15    0x7f7b6d17ab10
rip    0x7f7b553f17c8
rflags 0x10246
cs     0x33
fs     0x0
gs     0x0
time=2025-06-11T11:00:33.784Z level=ERROR source=sched.go:478 msg="error loading llama server" error="llama runner process has terminated: exit status 2"
[GIN] 2025/06/11 - 11:00:33 | 500 |  617.800054ms |       127.0.0.1 | POST     "/api/generate"

OS

Linux

GPU

No response

CPU

Intel

Ollama version

0.7.0 / 0.9.0 / 0.9.1

Originally created by @fishfl on GitHub (Jun 11, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11047 ### What is the issue? I follow these instructions in CLI: docker pull ollama/ollama:0.9.0 docker run -it --entrypoint /bin/sh ollama/ollama:latest ollama serve & ollama run gemma3 And got the error: msg="error loading llama server" error="llama runner process has terminated: exit status 2" And I tried the other versions(0.7.0 / 0.9.1) and the other models, got the same error. ### Relevant log output ```shell Couldn't find '/root/.ollama/id_ed25519'. Generating new private key. Your new public key is: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAWLs45MrvYntAZlkPIn8+/BaGe02MIaaqCVklA+kZPA time=2025-06-11T10:56:09.245Z level=INFO source=routes.go:1205 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" time=2025-06-11T10:56:09.246Z level=INFO source=images.go:463 msg="total blobs: 0" time=2025-06-11T10:56:09.246Z level=INFO source=images.go:470 msg="total unused blobs removed: 0" time=2025-06-11T10:56:09.246Z level=INFO source=routes.go:1258 msg="Listening on [::]:11434 (version 0.7.0)" time=2025-06-11T10:56:09.246Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs" time=2025-06-11T10:56:09.252Z level=INFO source=gpu.go:377 msg="no compatible GPUs were discovered" time=2025-06-11T10:56:09.252Z level=INFO source=types.go:130 msg="inference compute" id=0 library=cpu variant="" compute="" driver=0.0 name="" total="31.3 GiB" available="30.3 GiB" [GIN] 2025/06/11 - 10:56:57 | 200 | 93.216µs | 127.0.0.1 | HEAD "/" [GIN] 2025/06/11 - 10:56:57 | 404 | 346.738µs | 127.0.0.1 | POST "/api/show" time=2025-06-11T10:56:59.158Z level=INFO source=download.go:177 msg="downloading aeda25e63ebd in 16 208 MB part(s)" time=2025-06-11T11:00:10.130Z level=INFO source=download.go:177 msg="downloading e0a42594d802 in 1 358 B part(s)" time=2025-06-11T11:00:11.746Z level=INFO source=download.go:177 msg="downloading dd084c7d92a3 in 1 8.4 KB part(s)" time=2025-06-11T11:00:13.361Z level=INFO source=download.go:177 msg="downloading 3116c5225075 in 1 77 B part(s)" time=2025-06-11T11:00:15.019Z level=INFO source=download.go:177 msg="downloading b6ae5839783f in 1 489 B part(s)" [GIN] 2025/06/11 - 11:00:33 | 200 | 3m35s | 127.0.0.1 | POST "/api/pull" [GIN] 2025/06/11 - 11:00:33 | 200 | 142.680498ms | 127.0.0.1 | POST "/api/show" time=2025-06-11T11:00:33.379Z level=INFO source=server.go:135 msg="system memory" total="31.3 GiB" free="30.2 GiB" free_swap="0 B" time=2025-06-11T11:00:33.381Z level=INFO source=server.go:168 msg=offload library=cpu layers.requested=-1 layers.model=35 layers.offload=0 layers.split="" memory.available="[30.2 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.6 GiB" memory.required.partial="0 B" memory.required.kv="450.0 MiB" memory.required.allocations="[5.6 GiB]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" time=2025-06-11T11:00:33.532Z level=INFO source=server.go:431 msg="starting llama server" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-aeda25e63ebd698fab8638ffb778e68bed908b960d39d0becc650fa981609d25 --ctx-size 8192 --batch-size 512 --threads 8 --no-mmap --parallel 2 --port 36941" time=2025-06-11T11:00:33.533Z level=INFO source=sched.go:472 msg="loaded runners" count=1 time=2025-06-11T11:00:33.533Z level=INFO source=server.go:591 msg="waiting for llama runner to start responding" time=2025-06-11T11:00:33.533Z level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding" time=2025-06-11T11:00:33.558Z level=INFO source=runner.go:836 msg="starting ollama engine" time=2025-06-11T11:00:33.559Z level=INFO source=runner.go:899 msg="Server listening on 127.0.0.1:36941" time=2025-06-11T11:00:33.698Z level=INFO source=ggml.go:73 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=36 SIGILL: illegal instruction PC=0x7f7b553f17c8 m=5 sigcode=2 signal arrived during cgo execution instruction bytes: 0xc4 0xc1 0x7a 0x10 0x54 0x5d 0x0 0xc5 0xea 0x59 0xd 0x49 0x26 0x7 0x0 0x48 goroutine 9 gp=0xc000102e00 m=5 mp=0xc000100008 [syscall]: runtime.cgocall(0x55f96826bb00, 0xc000046fc8) runtime/cgocall.go:167 +0x4b fp=0xc000046fa0 sp=0xc000046f68 pc=0x55f9675afecb github.com/ollama/ollama/ml/backend/ggml/ggml/src._Cfunc_ggml_backend_load_all_from_path(0x7f7b58000ca0) _cgo_gotypes.go:195 +0x3e fp=0xc000046fc8 sp=0xc000046fa0 pc=0x55f9679484de github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.func1.1({0xc00003e044, 0xf}) github.com/ollama/ollama/ml/backend/ggml/ggml/src/ggml.go:97 +0xf5 fp=0xc000047060 sp=0xc000046fc8 pc=0x55f967947f75 github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.func1() github.com/ollama/ollama/ml/backend/ggml/ggml/src/ggml.go:98 +0x526 fp=0xc0000472f0 sp=0xc000047060 pc=0x55f967947dc6 github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.OnceFunc.func2() sync/oncefunc.go:27 +0x62 fp=0xc000047338 sp=0xc0000472f0 pc=0x55f9679477c2 sync.(*Once).doSlow(0x55f968751b40?, 0x0?) sync/once.go:78 +0xab fp=0xc000047390 sp=0xc000047338 pc=0x55f9675c4e0b sync.(*Once).Do(0x0?, 0x55f968751b40?) sync/once.go:69 +0x19 fp=0xc0000473b0 sp=0xc000047390 pc=0x55f9675c4d39 github.com/ollama/ollama/ml/backend/ggml/ggml/src.init.OnceFunc.func3() sync/oncefunc.go:32 +0x2d fp=0xc0000473e0 sp=0xc0000473b0 pc=0x55f96794772d github.com/ollama/ollama/ml/backend/ggml.devices() github.com/ollama/ollama/ml/backend/ggml/ggml.go:37 +0x1e fp=0xc000047420 sp=0xc0000473e0 pc=0x55f9679b353e github.com/ollama/ollama/ml/backend/ggml.New({0x55f9688f4b10, 0xc00061c640}, 0xc00012e008, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}) github.com/ollama/ollama/ml/backend/ggml/ggml.go:89 +0x4df fp=0xc000047d00 sp=0xc000047420 pc=0x55f9679b3abf github.com/ollama/ollama/ml.NewBackend({0x55f9688f4b10, 0xc00061c640}, 0xc00012e008, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}) github.com/ollama/ollama/ml/backend.go:87 +0x9a fp=0xc000047d68 sp=0xc000047d00 pc=0x55f9679a6ffa github.com/ollama/ollama/model.New({0x55f9688f4b10, 0xc00061c640}, {0x7ffe722c9dc3?, 0x0?}, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, 0x0, ...}, ...}) github.com/ollama/ollama/model/model.go:107 +0xe6 fp=0xc000047ea8 sp=0xc000047d68 pc=0x55f9679e4a86 github.com/ollama/ollama/runner/ollamarunner.(*Server).loadModel(0xc0000fdd40, {0x55f9688f4b10?, 0xc00061c640?}, {0x7ffe722c9dc3?, 0x0?}, {0xc0001144a0, 0x8, 0x0, 0x0, {0x0, ...}, ...}, ...) github.com/ollama/ollama/runner/ollamarunner/runner.go:775 +0x93 fp=0xc000047f20 sp=0xc000047ea8 pc=0x55f967a8cef3 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() github.com/ollama/ollama/runner/ollamarunner/runner.go:872 +0xbd fp=0xc000047fe0 sp=0xc000047f20 pc=0x55f967a8e3bd runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000047fe8 sp=0xc000047fe0 pc=0x55f9675ba901 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 github.com/ollama/ollama/runner/ollamarunner/runner.go:872 +0xa2b goroutine 1 gp=0xc000002380 m=nil [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000135638 sp=0xc000135618 pc=0x55f9675b31ce runtime.netpollblock(0x55f9675b0f53?, 0x6754cae6?, 0xf9?) runtime/netpoll.go:575 +0xf7 fp=0xc000135670 sp=0xc000135638 pc=0x55f967577fb7 internal/poll.runtime_pollWait(0x7f7b6e1c5eb0, 0x72) runtime/netpoll.go:351 +0x85 fp=0xc000135690 sp=0xc000135670 pc=0x55f9675b23e5 internal/poll.(*pollDesc).wait(0xc000507a80?, 0x55f9687bca20?, 0x0) internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001356b8 sp=0xc000135690 pc=0x55f967639827 internal/poll.(*pollDesc).waitRead(...) internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc000507a80) internal/poll/fd_unix.go:620 +0x295 fp=0xc000135760 sp=0xc0001356b8 pc=0x55f96763ebf5 net.(*netFD).accept(0xc000507a80) net/fd_unix.go:172 +0x29 fp=0xc000135818 sp=0xc000135760 pc=0x55f9676b1069 net.(*TCPListener).accept(0xc000405f00) net/tcpsock_posix.go:159 +0x1b fp=0xc000135868 sp=0xc000135818 pc=0x55f9676c6a1b net.(*TCPListener).Accept(0xc000405f00) net/tcpsock.go:380 +0x30 fp=0xc000135898 sp=0xc000135868 pc=0x55f9676c58d0 net/http.(*onceCloseListener).Accept(0x55f9688f4aa0?) <autogenerated>:1 +0x24 fp=0xc0001358b0 sp=0xc000135898 pc=0x55f9678dd024 net/http.(*Server).Serve(0xc000035700, {0x55f9688f2718, 0xc000405f00}) net/http/server.go:3424 +0x30c fp=0xc0001359e0 sp=0xc0001358b0 pc=0x55f9678b48ec github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034130, 0xd, 0xd}) github.com/ollama/ollama/runner/ollamarunner/runner.go:900 +0xe69 fp=0xc000135d08 sp=0xc0001359e0 pc=0x55f967a8e029 github.com/ollama/ollama/runner.Execute({0xc000034110?, 0x0?, 0x0?}) github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000135d30 sp=0xc000135d08 pc=0x55f967a8e949 github.com/ollama/ollama/cmd.NewCLI.func2(0xc000035400?, {0x55f96844706e?, 0x4?, 0x55f968447072?}) github.com/ollama/ollama/cmd/cmd.go:1387 +0x45 fp=0xc000135d58 sp=0xc000135d30 pc=0x55f9681ea685 github.com/spf13/cobra.(*Command).execute(0xc0004b8f08, {0xc0005167e0, 0xe, 0xe}) github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000135e78 sp=0xc000135d58 pc=0x55f96772a6bc github.com/spf13/cobra.(*Command).ExecuteC(0xc00052f508) github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000135f30 sp=0xc000135e78 pc=0x55f96772af05 github.com/spf13/cobra.(*Command).Execute(...) github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) github.com/spf13/cobra@v1.7.0/command.go:985 main.main() github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000135f50 sp=0xc000135f30 pc=0x55f9681eb00d runtime.main() runtime/proc.go:283 +0x29d fp=0xc000135fe0 sp=0xc000135f50 pc=0x55f96757f5bd runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000135fe8 sp=0xc000135fe0 pc=0x55f9675ba901 goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000070fa8 sp=0xc000070f88 pc=0x55f9675b31ce runtime.goparkunlock(...) runtime/proc.go:441 runtime.forcegchelper() runtime/proc.go:348 +0xb8 fp=0xc000070fe0 sp=0xc000070fa8 pc=0x55f96757f8f8 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000070fe8 sp=0xc000070fe0 pc=0x55f9675ba901 created by runtime.init.7 in goroutine 1 runtime/proc.go:336 +0x1a goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000071780 sp=0xc000071760 pc=0x55f9675b31ce runtime.goparkunlock(...) runtime/proc.go:441 runtime.bgsweep(0xc00009c000) runtime/mgcsweep.go:316 +0xdf fp=0xc0000717c8 sp=0xc000071780 pc=0x55f96756a11f runtime.gcenable.gowrap1() runtime/mgc.go:204 +0x25 fp=0xc0000717e0 sp=0xc0000717c8 pc=0x55f96755e505 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000717e8 sp=0xc0000717e0 pc=0x55f9675ba901 created by runtime.gcenable in goroutine 1 runtime/mgc.go:204 +0x66 goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]: runtime.gopark(0x10000?, 0x55f968602328?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000071f78 sp=0xc000071f58 pc=0x55f9675b31ce runtime.goparkunlock(...) runtime/proc.go:441 runtime.(*scavengerState).park(0x55f96917b5c0) runtime/mgcscavenge.go:425 +0x49 fp=0xc000071fa8 sp=0xc000071f78 pc=0x55f967567b69 runtime.bgscavenge(0xc00009c000) runtime/mgcscavenge.go:658 +0x59 fp=0xc000071fc8 sp=0xc000071fa8 pc=0x55f9675680f9 runtime.gcenable.gowrap2() runtime/mgc.go:205 +0x25 fp=0xc000071fe0 sp=0xc000071fc8 pc=0x55f96755e4a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x55f9675ba901 created by runtime.gcenable in goroutine 1 runtime/mgc.go:205 +0xa5 goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000070688?) runtime/proc.go:435 +0xce fp=0xc000070630 sp=0xc000070610 pc=0x55f9675b31ce runtime.runfinq() runtime/mfinal.go:196 +0x107 fp=0xc0000707e0 sp=0xc000070630 pc=0x55f96755d4c7 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000707e8 sp=0xc0000707e0 pc=0x55f9675ba901 created by runtime.createfing in goroutine 1 runtime/mfinal.go:166 +0x3d goroutine 6 gp=0xc0001d08c0 m=nil [chan receive]: runtime.gopark(0xc000223680?, 0xc000224048?, 0x60?, 0x27?, 0x55f967697da8?) runtime/proc.go:435 +0xce fp=0xc000072718 sp=0xc0000726f8 pc=0x55f9675b31ce runtime.chanrecv(0xc0000a8310, 0x0, 0x1) runtime/chan.go:664 +0x445 fp=0xc000072790 sp=0xc000072718 pc=0x55f96754f6c5 runtime.chanrecv1(0x0?, 0x0?) runtime/chan.go:506 +0x12 fp=0xc0000727b8 sp=0xc000072790 pc=0x55f96754f252 runtime.unique_runtime_registerUniqueMapCleanup.func2(...) runtime/mgc.go:1796 runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() runtime/mgc.go:1799 +0x2f fp=0xc0000727e0 sp=0xc0000727b8 pc=0x55f9675616af runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x55f9675ba901 created by unique.runtime_registerUniqueMapCleanup in goroutine 1 runtime/mgc.go:1794 +0x85 goroutine 7 gp=0xc0001d0e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000072f38 sp=0xc000072f18 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc000072fc8 sp=0xc000072f38 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000072fe0 sp=0xc000072fc8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00006c738 sp=0xc00006c718 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00006c7c8 sp=0xc00006c738 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00006c7e0 sp=0xc00006c7c8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00006c7e8 sp=0xc00006c7e0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]: runtime.gopark(0x17aeca47c6d72d?, 0x3?, 0x6c?, 0xd?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00011a738 sp=0xc00011a718 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00011a7c8 sp=0xc00011a738 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00011a7e0 sp=0xc00011a7c8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00011a7e8 sp=0xc00011a7e0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 8 gp=0xc0001d0fc0 m=nil [GC worker (idle)]: runtime.gopark(0x17aeca4763ae93?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000073738 sp=0xc000073718 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc0000737c8 sp=0xc000073738 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000737e0 sp=0xc0000737c8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 19 gp=0xc0005041c0 m=nil [GC worker (idle)]: runtime.gopark(0x17aeca4763b8f2?, 0x3?, 0x33?, 0x29?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00006cf38 sp=0xc00006cf18 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00006cfc8 sp=0xc00006cf38 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00006cfe0 sp=0xc00006cfc8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00006cfe8 sp=0xc00006cfe0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 20 gp=0xc000504380 m=nil [GC worker (idle)]: runtime.gopark(0x17aeca4763aa13?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00006d738 sp=0xc00006d718 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00006d7c8 sp=0xc00006d738 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00006d7e0 sp=0xc00006d7c8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00006d7e8 sp=0xc00006d7e0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 21 gp=0xc000504540 m=nil [GC worker (idle)]: runtime.gopark(0x17aeca47b58963?, 0x3?, 0xc9?, 0xa7?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00006df38 sp=0xc00006df18 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00006dfc8 sp=0xc00006df38 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00006dfe0 sp=0xc00006dfc8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 22 gp=0xc000504700 m=nil [GC worker (idle)]: runtime.gopark(0x55f969229e00?, 0x1?, 0x27?, 0xfb?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00006e738 sp=0xc00006e718 pc=0x55f9675b31ce runtime.gcBgMarkWorker(0xc0000a98f0) runtime/mgc.go:1423 +0xe9 fp=0xc00006e7c8 sp=0xc00006e738 pc=0x55f9675609c9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00006e7e0 sp=0xc00006e7c8 pc=0x55f9675608a5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00006e7e8 sp=0xc00006e7e0 pc=0x55f9675ba901 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 10 gp=0xc000102fc0 m=nil [sync.WaitGroup.Wait]: runtime.gopark(0x0?, 0x0?, 0x60?, 0xa0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00011ced0 sp=0xc00011ceb0 pc=0x55f9675b31ce runtime.goparkunlock(...) runtime/proc.go:441 runtime.semacquire1(0xc0000fdd48, 0x0, 0x1, 0x0, 0x18) runtime/sema.go:188 +0x229 fp=0xc00011cf38 sp=0xc00011ced0 pc=0x55f967592b89 sync.runtime_SemacquireWaitGroup(0x0?) runtime/sema.go:110 +0x25 fp=0xc00011cf70 sp=0xc00011cf38 pc=0x55f9675b4be5 sync.(*WaitGroup).Wait(0x0?) sync/waitgroup.go:118 +0x48 fp=0xc00011cf98 sp=0xc00011cf70 pc=0x55f9675c6248 github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0000fdd40, {0x55f9688f4b10, 0xc00061c640}) github.com/ollama/ollama/runner/ollamarunner/runner.go:344 +0x25 fp=0xc00011cfb8 sp=0xc00011cf98 pc=0x55f967a89645 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2() github.com/ollama/ollama/runner/ollamarunner/runner.go:876 +0x28 fp=0xc00011cfe0 sp=0xc00011cfb8 pc=0x55f967a8e2c8 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00011cfe8 sp=0xc00011cfe0 pc=0x55f9675ba901 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 github.com/ollama/ollama/runner/ollamarunner/runner.go:876 +0xad5 rax 0x6101304cacd rbx 0x0 rcx 0x0 rdx 0x4f3cd rdi 0x7ffe72383000 rsi 0x61012ffd700 rbp 0x7f7b5547bc80 rsp 0x7f7b6d17aa80 r8 0x2f46ab17db0e9c r9 0x7f7b58008df0 r10 0x7ffe72382080 r11 0x7ffe72382090 r12 0x7f7b5549bc80 r13 0x7f7b5554de60 r14 0x7f7b55461ef0 r15 0x7f7b6d17ab10 rip 0x7f7b553f17c8 rflags 0x10246 cs 0x33 fs 0x0 gs 0x0 time=2025-06-11T11:00:33.784Z level=ERROR source=sched.go:478 msg="error loading llama server" error="llama runner process has terminated: exit status 2" [GIN] 2025/06/11 - 11:00:33 | 500 | 617.800054ms | 127.0.0.1 | POST "/api/generate" ``` ### OS Linux ### GPU _No response_ ### CPU Intel ### Ollama version 0.7.0 / 0.9.0 / 0.9.1
GiteaMirror added the bug label 2026-05-04 17:52:50 -05:00
Author
Owner

@rick-github commented on GitHub (Jun 11, 2025):

The runner is executing an instruction not supported by your CPU:

0000000000000000 <.data>:
   0:	c4 c1 7a 10 54 5d 00 	vmovss 0x0(%r13,%rbx,2),%xmm2
   7:	c5 ea 59 0d 49 26 07 	vmulss 0x72649(%rip),%xmm2,%xmm1        # 0x72658
   e:	00 
   f:	48                   	rex.W

If you are running ollama in a proxmox VE, you need to set AVX flags: https://pve.proxmox.com/pve-docs/cpu-models.conf.5.html#_example_file

If it's not a virtual CPU, you can try deleting backends from /usr/lib/ollama. Note that when the docker container is restarted the backends will be restored, so if you go this route you should create a custom image:

FROM ollama/ollama
RUN for i in sandybridge haswell skylakex icelake alderlake ; do rm /usr/lib/ollama/libggml-cpu-$i.so ; done
$ docker build -f Dockerfile -t ollama:noavx .

Otherwise, you can build a custom version of the server by removing unsupported instruction sets as shown here.

<!-- gh-comment-id:2963218328 --> @rick-github commented on GitHub (Jun 11, 2025): The runner is executing an instruction not supported by your CPU: ``` 0000000000000000 <.data>: 0: c4 c1 7a 10 54 5d 00 vmovss 0x0(%r13,%rbx,2),%xmm2 7: c5 ea 59 0d 49 26 07 vmulss 0x72649(%rip),%xmm2,%xmm1 # 0x72658 e: 00 f: 48 rex.W ``` If you are running ollama in a proxmox VE, you need to set AVX flags: https://pve.proxmox.com/pve-docs/cpu-models.conf.5.html#_example_file If it's not a virtual CPU, you can try deleting backends from `/usr/lib/ollama`. Note that when the docker container is restarted the backends will be restored, so if you go this route you should create a custom image: ```dockerfile FROM ollama/ollama RUN for i in sandybridge haswell skylakex icelake alderlake ; do rm /usr/lib/ollama/libggml-cpu-$i.so ; done ``` ```console $ docker build -f Dockerfile -t ollama:noavx . ``` Otherwise, you can build a custom version of the server by removing unsupported instruction sets as shown [here](https://github.com/ollama/ollama/issues/10918#issuecomment-2923590731).
Author
Owner

@fishfl commented on GitHub (Jun 12, 2025):

Awesome! It's very useful!
Thank you very much, it works.

<!-- gh-comment-id:2965275762 --> @fishfl commented on GitHub (Jun 12, 2025): Awesome! It's very useful! Thank you very much, it works.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#69347