github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10991] ollama crash running Gemma3:27b #53759

New Issue
Closed
opened 2026-04-29 04:42:14 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-29 04:42:14 -05:00
Owner
Copy Link

Originally created by @mario-grgic on GitHub (Jun 6, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10991

What is the issue?

ollama crashes when running gemma3:27b. I did git bisect and it looks like the behaviour was introduced in the following commit:

aaa7818000c42a82fc030212c35ef83f9799efd7 is the first bad commit
commit aaa7818000c42a82fc030212c35ef83f9799efd7
Author: Jesse Gross <jesse@ollama.com>
Date:   Thu Apr 24 11:48:49 2025 -0700

    ggml: Export GPU UUIDs
    
    This enables matching up devices and information reported by the backend
    with system management libraries such as nvml to get accurate free
    memory reporting.

 llama/patches/0017-ggml-Export-GPU-UUIDs.patch   | 102 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 ml/backend.go                                    |   8 +++++
 ml/backend/ggml/ggml.go                          |   6 ++++
 ml/backend/ggml/ggml/include/ggml-backend.h      |   1 +
 ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu  |  33 +++++++++++++++++++++
 ml/backend/ggml/ggml/src/ggml-metal/ggml-metal.m |   1 +
 6 files changed, 151 insertions(+)
 create mode 100644 llama/patches/0017-ggml-Export-GPU-UUIDs.patch

tag 0.9.0 works fine.

Here is the crash log:

$ ./ollama serve
time=2025-06-05T21:10:01.164-04:00 level=INFO source=routes.go:1242 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/mariogrgic/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:true OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]"
time=2025-06-05T21:10:01.165-04:00 level=INFO source=images.go:479 msg="total blobs: 16"
time=2025-06-05T21:10:01.165-04:00 level=INFO source=images.go:486 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func4 (5 handlers)
[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers)
[GIN-debug] GET    /api/ps                   --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embed                --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] POST   /v1/completions           --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers)
[GIN-debug] POST   /v1/embeddings            --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models                --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers)
[GIN-debug] GET    /v1/models/:model         --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers)
time=2025-06-05T21:10:01.166-04:00 level=INFO source=routes.go:1295 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2025-06-05T21:10:01.195-04:00 level=INFO source=types.go:130 msg="inference compute" id=0 library=metal variant="" compute="" driver=0.0 name="" total="96.0 GiB" available="96.0 GiB"
[GIN] 2025/06/05 - 21:10:06 | 200 |     222.334µs |       127.0.0.1 | HEAD     "/"
[GIN] 2025/06/05 - 21:10:06 | 200 |   49.765208ms |       127.0.0.1 | POST     "/api/show"
time=2025-06-05T21:10:06.725-04:00 level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/Users/mariogrgic/.ollama/models/blobs/sha256-e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68 gpu=0 parallel=2 available=103079215104 required="20.1 GiB"
time=2025-06-05T21:10:06.725-04:00 level=INFO source=server.go:135 msg="system memory" total="128.0 GiB" free="117.4 GiB" free_swap="0 B"
time=2025-06-05T21:10:06.726-04:00 level=INFO source=server.go:168 msg=offload library=metal layers.requested=-1 layers.model=63 layers.offload=63 layers.split="" memory.available="[96.0 GiB]" memory.gpu_overhead="0 B" memory.required.full="20.1 GiB" memory.required.partial="20.1 GiB" memory.required.kv="1.6 GiB" memory.required.allocations="[20.1 GiB]" memory.weights.total="15.4 GiB" memory.weights.repeating="14.3 GiB" memory.weights.nonrepeating="1.1 GiB" memory.graph.full="565.0 MiB" memory.graph.partial="565.0 MiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
time=2025-06-05T21:10:06.756-04:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/Volumes/DATA/dev/ollama/ollama.git/ollama runner --ollama-engine --model /Users/mariogrgic/.ollama/models/blobs/sha256-e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68 --ctx-size 8192 --batch-size 512 --n-gpu-layers 63 --threads 12 --parallel 2 --port 49300"
time=2025-06-05T21:10:06.757-04:00 level=INFO source=sched.go:483 msg="loaded runners" count=1
time=2025-06-05T21:10:06.757-04:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding"
time=2025-06-05T21:10:06.757-04:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding"
time=2025-06-05T21:10:06.765-04:00 level=INFO source=runner.go:925 msg="starting ollama engine"
time=2025-06-05T21:10:06.766-04:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:49300"
time=2025-06-05T21:10:06.793-04:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1247 num_key_values=37
time=2025-06-05T21:10:06.793-04:00 level=INFO source=ggml.go:104 msg=system Metal.0.EMBED_LIBRARY=1 CPU.0.ARM_FMA=1 CPU.0.FP16_VA=1 CPU.0.DOTPROD=1 CPU.0.LLAMAFILE=1 CPU.0.ACCELERATE=1 compiler=cgo(clang)
unexpected fault address 0x2000000000
fatal error: fault
[signal SIGBUS: bus error code=0x1 addr=0x2000000000 pc=0x100edea94]

goroutine 14 gp=0x14000103a40 m=8 mp=0x14000600008 [running]:
runtime.throw({0x101c05000?, 0x0?})
	/usr/local/go/src/runtime/panic.go:1101 +0x38 fp=0x1400004d100 sp=0x1400004d0d0 pc=0x100f4c788
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:922 +0x170 fp=0x1400004d160 sp=0x1400004d100 pc=0x100f4ebf0
indexbytebody()
	/usr/local/go/src/internal/bytealg/indexbyte_arm64.s:74 +0x64 fp=0x1400004d170 sp=0x1400004d170 pc=0x100edea94
runtime.findnull(0x1400004d208?)
	/usr/local/go/src/runtime/string.go:577 +0x78 fp=0x1400004d1d0 sp=0x1400004d170 pc=0x100f35ba8
runtime.gostring(0x2000000000)
	/usr/local/go/src/runtime/string.go:363 +0x20 fp=0x1400004d210 sp=0x1400004d1d0 pc=0x100f4fa60
github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...)
	_cgo_gotypes.go:300
github.com/ollama/ollama/ml/backend/ggml.New({0x16ef26ca2, 0x6e}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0})
	/Volumes/DATA/dev/ollama/ollama.git/ml/backend/ggml/ggml.go:141 +0xaa4 fp=0x1400004dc00 sp=0x1400004d210 pc=0x1013180d4
github.com/ollama/ollama/ml.NewBackend({0x16ef26ca2, 0x6e}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0})
	/Volumes/DATA/dev/ollama/ollama.git/ml/backend.go:209 +0x90 fp=0x1400004dc60 sp=0x1400004dc00 pc=0x10130dc90
github.com/ollama/ollama/model.New({0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0})
	/Volumes/DATA/dev/ollama/ollama.git/model/model.go:102 +0x5c fp=0x1400004dd60 sp=0x1400004dc60 pc=0x10132464c
github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0x140004b9b00, {0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0}, ...)
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:841 +0x80 fp=0x1400004ddc0 sp=0x1400004dd60 pc=0x10139d3a0
github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0x140004b9b00, {0x102076dd0, 0x14000390410}, {0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, ...}, ...}, ...)
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:878 +0xa8 fp=0x1400004df20 sp=0x1400004ddc0 pc=0x10139d6b8
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:959 +0x88 fp=0x1400004dfd0 sp=0x1400004df20 pc=0x10139e738
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400004dfd0 sp=0x1400004dfd0 pc=0x100f54c64
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:959 +0x840

goroutine 1 gp=0x140000021c0 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x100f625b0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140001335e0 sp=0x140001335c0 pc=0x100f4c8a8
runtime.netpollblock(0x140004b5678?, 0xfd1170?, 0x1?)
	/usr/local/go/src/runtime/netpoll.go:575 +0x158 fp=0x14000133620 sp=0x140001335e0 pc=0x100f12728
internal/poll.runtime_pollWait(0x149b16f30, 0x72)
	/usr/local/go/src/runtime/netpoll.go:351 +0xa0 fp=0x14000133650 sp=0x14000133620 pc=0x100f4ba60
internal/poll.(*pollDesc).wait(0x14000055780?, 0x101f3fb40?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000133680 sp=0x14000133650 pc=0x100fcc988
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0x14000055780)
	/usr/local/go/src/internal/poll/fd_unix.go:620 +0x24c fp=0x14000133730 sp=0x14000133680 pc=0x100fd125c
net.(*netFD).accept(0x14000055780)
	/usr/local/go/src/net/fd_unix.go:172 +0x28 fp=0x140001337f0 sp=0x14000133730 pc=0x1010403c8
net.(*TCPListener).accept(0x1400043b100)
	/usr/local/go/src/net/tcpsock_posix.go:159 +0x24 fp=0x14000133840 sp=0x140001337f0 pc=0x101054624
net.(*TCPListener).Accept(0x1400043b100)
	/usr/local/go/src/net/tcpsock.go:380 +0x2c fp=0x14000133880 sp=0x14000133840 pc=0x10105360c
net/http.(*onceCloseListener).Accept(0x102076d60?)
	<autogenerated>:1 +0x30 fp=0x140001338a0 sp=0x14000133880 pc=0x10122eb40
net/http.(*Server).Serve(0x1400011f100, {0x102074978, 0x1400043b100})
	/usr/local/go/src/net/http/server.go:3424 +0x290 fp=0x140001339d0 sp=0x140001338a0 pc=0x1012081e0
github.com/ollama/ollama/runner/ollamarunner.Execute({0x14000000270, 0xe, 0xf})
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:984 +0xb78 fp=0x14000133ce0 sp=0x140001339d0 pc=0x10139e408
github.com/ollama/ollama/runner.Execute({0x14000000250?, 0x0?, 0x0?})
	/Volumes/DATA/dev/ollama/ollama.git/runner/runner.go:20 +0x120 fp=0x14000133d10 sp=0x14000133ce0 pc=0x10139ecb0
github.com/ollama/ollama/cmd.NewCLI.func2(0x1400011ef00?, {0x101c03c5e?, 0x4?, 0x101c03c62?})
	/Volumes/DATA/dev/ollama/ollama.git/cmd/cmd.go:1529 +0x54 fp=0x14000133d40 sp=0x14000133d10 pc=0x1019e9984
github.com/spf13/cobra.(*Command).execute(0x140004e6f08, {0x14000434f00, 0xf, 0xf})
	/Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x648 fp=0x14000133e60 sp=0x14000133d40 pc=0x1010ae968
github.com/spf13/cobra.(*Command).ExecuteC(0x140004cc908)
	/Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x320 fp=0x14000133f20 sp=0x14000133e60 pc=0x1010af0b0
github.com/spf13/cobra.(*Command).Execute(...)
	/Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	/Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	/Volumes/DATA/dev/ollama/ollama.git/main.go:12 +0x54 fp=0x14000133f40 sp=0x14000133f20 pc=0x1019ea464
runtime.main()
	/usr/local/go/src/runtime/proc.go:283 +0x284 fp=0x14000133fd0 sp=0x14000133f40 pc=0x100f19244
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000133fd0 sp=0x14000133fd0 pc=0x100f54c64

goroutine 2 gp=0x14000002c40 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000084f90 sp=0x14000084f70 pc=0x100f4c8a8
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:441
runtime.forcegchelper()
	/usr/local/go/src/runtime/proc.go:348 +0xb8 fp=0x14000084fd0 sp=0x14000084f90 pc=0x100f19598
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000084fd0 sp=0x14000084fd0 pc=0x100f54c64
created by runtime.init.7 in goroutine 1
	/usr/local/go/src/runtime/proc.go:336 +0x24

goroutine 3 gp=0x14000003180 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000085760 sp=0x14000085740 pc=0x100f4c8a8
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:441
runtime.bgsweep(0x140000b0000)
	/usr/local/go/src/runtime/mgcsweep.go:316 +0x108 fp=0x140000857b0 sp=0x14000085760 pc=0x100f046c8
runtime.gcenable.gowrap1()
	/usr/local/go/src/runtime/mgc.go:204 +0x28 fp=0x140000857d0 sp=0x140000857b0 pc=0x100ef84c8
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000857d0 sp=0x140000857d0 pc=0x100f54c64
created by runtime.gcenable in goroutine 1
	/usr/local/go/src/runtime/mgc.go:204 +0x6c

goroutine 4 gp=0x14000003340 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x101db4f78?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000085f60 sp=0x14000085f40 pc=0x100f4c8a8
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:441
runtime.(*scavengerState).park(0x102917f40)
	/usr/local/go/src/runtime/mgcscavenge.go:425 +0x5c fp=0x14000085f90 sp=0x14000085f60 pc=0x100f0215c
runtime.bgscavenge(0x140000b0000)
	/usr/local/go/src/runtime/mgcscavenge.go:658 +0xac fp=0x14000085fb0 sp=0x14000085f90 pc=0x100f026fc
runtime.gcenable.gowrap2()
	/usr/local/go/src/runtime/mgc.go:205 +0x28 fp=0x14000085fd0 sp=0x14000085fb0 pc=0x100ef8468
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000085fd0 sp=0x14000085fd0 pc=0x100f54c64
created by runtime.gcenable in goroutine 1
	/usr/local/go/src/runtime/mgc.go:205 +0xac

goroutine 5 gp=0x14000003c00 m=nil [finalizer wait]:
runtime.gopark(0x18000845c8?, 0x1000000000000?, 0xf8?, 0x45?, 0x1012314ac?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000084590 sp=0x14000084570 pc=0x100f4c8a8
runtime.runfinq()
	/usr/local/go/src/runtime/mfinal.go:196 +0x108 fp=0x140000847d0 sp=0x14000084590 pc=0x100ef74c8
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000847d0 sp=0x140000847d0 pc=0x100f54c64
created by runtime.createfing in goroutine 1
	/usr/local/go/src/runtime/mfinal.go:166 +0x80

goroutine 6 gp=0x140001ee700 m=nil [chan receive]:
runtime.gopark(0x14000233680?, 0x1400000e1b0?, 0x48?, 0x67?, 0x101014698?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140000866f0 sp=0x140000866d0 pc=0x100f4c8a8
runtime.chanrecv(0x140000be310, 0x0, 0x1)
	/usr/local/go/src/runtime/chan.go:664 +0x42c fp=0x14000086770 sp=0x140000866f0 pc=0x100ee9f7c
runtime.chanrecv1(0x0?, 0x0?)
	/usr/local/go/src/runtime/chan.go:506 +0x14 fp=0x140000867a0 sp=0x14000086770 pc=0x100ee9b14
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
	/usr/local/go/src/runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1799 +0x3c fp=0x140000867d0 sp=0x140000867a0 pc=0x100efb6ec
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000867d0 sp=0x140000867d0 pc=0x100f54c64
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1794 +0x78

goroutine 7 gp=0x140001eee00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000086f10 sp=0x14000086ef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000086fb0 sp=0x14000086f10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000086fd0 sp=0x14000086fb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000086fd0 sp=0x14000086fd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 18 gp=0x14000504000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000080710 sp=0x140000806f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000807b0 sp=0x14000080710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000807d0 sp=0x140000807b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000807d0 sp=0x140000807d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 34 gp=0x14000102380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011a710 sp=0x1400011a6f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011a7b0 sp=0x1400011a710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011a7d0 sp=0x1400011a7b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011a7d0 sp=0x1400011a7d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 35 gp=0x14000102540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011af10 sp=0x1400011aef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011afb0 sp=0x1400011af10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011afd0 sp=0x1400011afb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011afd0 sp=0x1400011afd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 8 gp=0x140001eefc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000087710 sp=0x140000876f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000877b0 sp=0x14000087710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000877d0 sp=0x140000877b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000877d0 sp=0x140000877d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 19 gp=0x140005041c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000080f10 sp=0x14000080ef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000080fb0 sp=0x14000080f10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000080fd0 sp=0x14000080fb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000080fd0 sp=0x14000080fd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 20 gp=0x14000504380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000081710 sp=0x140000816f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000817b0 sp=0x14000081710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000817d0 sp=0x140000817b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000817d0 sp=0x140000817d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 36 gp=0x14000102700 m=nil [GC worker (idle)]:
runtime.gopark(0x26313f85800?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011b710 sp=0x1400011b6f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011b7b0 sp=0x1400011b710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011b7d0 sp=0x1400011b7b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011b7d0 sp=0x1400011b7d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 37 gp=0x140001028c0 m=nil [GC worker (idle)]:
runtime.gopark(0x102948780?, 0x1?, 0xc?, 0xa8?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011bf10 sp=0x1400011bef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011bfb0 sp=0x1400011bf10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011bfd0 sp=0x1400011bfb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011bfd0 sp=0x1400011bfd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 38 gp=0x14000102a80 m=nil [GC worker (idle)]:
runtime.gopark(0x26313daa359?, 0x0?, 0x0?, 0x0?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011c710 sp=0x1400011c6f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011c7b0 sp=0x1400011c710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011c7d0 sp=0x1400011c7b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011c7d0 sp=0x1400011c7d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 39 gp=0x14000102c40 m=nil [GC worker (idle)]:
runtime.gopark(0x26313da80ae?, 0x3?, 0x5e?, 0x1b?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011cf10 sp=0x1400011cef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011cfb0 sp=0x1400011cf10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011cfd0 sp=0x1400011cfb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011cfd0 sp=0x1400011cfd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 9 gp=0x140001ef180 m=nil [GC worker (idle)]:
runtime.gopark(0x26313f87e6a?, 0x1?, 0x18?, 0x5f?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000087f10 sp=0x14000087ef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000087fb0 sp=0x14000087f10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000087fd0 sp=0x14000087fb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000087fd0 sp=0x14000087fd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 10 gp=0x140001ef340 m=nil [GC worker (idle)]:
runtime.gopark(0x26313da7de9?, 0x3?, 0x53?, 0x8?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000116710 sp=0x140001166f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140001167b0 sp=0x14000116710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140001167d0 sp=0x140001167b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001167d0 sp=0x140001167d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 11 gp=0x140001ef500 m=nil [GC worker (idle)]:
runtime.gopark(0x26313da7643?, 0x1?, 0x12?, 0xe3?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000116f10 sp=0x14000116ef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000116fb0 sp=0x14000116f10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000116fd0 sp=0x14000116fb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000116fd0 sp=0x14000116fd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 12 gp=0x140001ef6c0 m=nil [GC worker (idle)]:
runtime.gopark(0x26313da7a2b?, 0x1?, 0x6f?, 0xa9?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000117710 sp=0x140001176f0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140001177b0 sp=0x14000117710 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140001177d0 sp=0x140001177b0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001177d0 sp=0x140001177d0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 13 gp=0x140001ef880 m=nil [GC worker (idle)]:
runtime.gopark(0x26313da7b4f?, 0x1?, 0xc3?, 0xb?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000117f10 sp=0x14000117ef0 pc=0x100f4c8a8
runtime.gcBgMarkWorker(0x140000bf730)
	/usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000117fb0 sp=0x14000117f10 pc=0x100efa95c
runtime.gcBgMarkStartWorkers.gowrap1()
	/usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000117fd0 sp=0x14000117fb0 pc=0x100efa848
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000117fd0 sp=0x14000117fd0 pc=0x100f54c64
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/usr/local/go/src/runtime/mgc.go:1339 +0x140

goroutine 15 gp=0x14000103c00 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x1029266a0?, 0x0?, 0x60?, 0x20?, 0x0?)
	/usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140001196d0 sp=0x140001196b0 pc=0x100f4c8a8
runtime.goparkunlock(...)
	/usr/local/go/src/runtime/proc.go:441
runtime.semacquire1(0x140004b9b08, 0x0, 0x1, 0x0, 0x18)
	/usr/local/go/src/runtime/sema.go:188 +0x204 fp=0x14000119720 sp=0x140001196d0 pc=0x100f2d724
sync.runtime_SemacquireWaitGroup(0x0?)
	/usr/local/go/src/runtime/sema.go:110 +0x2c fp=0x14000119760 sp=0x14000119720 pc=0x100f4e31c
sync.(*WaitGroup).Wait(0x140004b9b00)
	/usr/local/go/src/sync/waitgroup.go:118 +0x70 fp=0x14000119780 sp=0x14000119760 pc=0x100f60540
github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0x140004b9b00, {0x102076dd0, 0x14000390410})
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:355 +0x2c fp=0x140001197a0 sp=0x14000119780 pc=0x101399e9c
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2()
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:960 +0x30 fp=0x140001197d0 sp=0x140001197a0 pc=0x10139e670
runtime.goexit({})
	/usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001197d0 sp=0x140001197d0 pc=0x100f54c64
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	/Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:960 +0x898
time=2025-06-05T21:10:06.799-04:00 level=ERROR source=server.go:457 msg="llama runner terminated" error="exit status 2"
time=2025-06-05T21:10:07.007-04:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault"
[GIN] 2025/06/05 - 21:10:07 | 500 |   335.97975ms |       127.0.0.1 | POST     "/api/generate"

Relevant log output

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

No response

Originally created by @mario-grgic on GitHub (Jun 6, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10991 ### What is the issue? ollama crashes when running gemma3:27b. I did git bisect and it looks like the behaviour was introduced in the following commit: ``` aaa7818000c42a82fc030212c35ef83f9799efd7 is the first bad commit commit aaa7818000c42a82fc030212c35ef83f9799efd7 Author: Jesse Gross <jesse@ollama.com> Date: Thu Apr 24 11:48:49 2025 -0700 ggml: Export GPU UUIDs This enables matching up devices and information reported by the backend with system management libraries such as nvml to get accurate free memory reporting. llama/patches/0017-ggml-Export-GPU-UUIDs.patch | 102 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ml/backend.go | 8 +++++ ml/backend/ggml/ggml.go | 6 ++++ ml/backend/ggml/ggml/include/ggml-backend.h | 1 + ml/backend/ggml/ggml/src/ggml-cuda/ggml-cuda.cu | 33 +++++++++++++++++++++ ml/backend/ggml/ggml/src/ggml-metal/ggml-metal.m | 1 + 6 files changed, 151 insertions(+) create mode 100644 llama/patches/0017-ggml-Export-GPU-UUIDs.patch ``` tag 0.9.0 works fine. Here is the crash log: ``` $ ./ollama serve time=2025-06-05T21:10:01.164-04:00 level=INFO source=routes.go:1242 msg="server config" env="map[HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/Users/mariogrgic/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:true OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false http_proxy: https_proxy: no_proxy:]" time=2025-06-05T21:10:01.165-04:00 level=INFO source=images.go:479 msg="total blobs: 16" time=2025-06-05T21:10:01.165-04:00 level=INFO source=images.go:486 msg="total unused blobs removed: 0" [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func4 (5 handlers) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushHandler-fm (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyHandler-fm (5 handlers) [GIN-debug] GET /api/ps --> github.com/ollama/ollama/server.(*Server).PsHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embed --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] POST /v1/completions --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (6 handlers) [GIN-debug] POST /v1/embeddings --> github.com/ollama/ollama/server.(*Server).EmbedHandler-fm (6 handlers) [GIN-debug] GET /v1/models --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (6 handlers) [GIN-debug] GET /v1/models/:model --> github.com/ollama/ollama/server.(*Server).ShowHandler-fm (6 handlers) time=2025-06-05T21:10:01.166-04:00 level=INFO source=routes.go:1295 msg="Listening on 127.0.0.1:11434 (version 0.0.0)" time=2025-06-05T21:10:01.195-04:00 level=INFO source=types.go:130 msg="inference compute" id=0 library=metal variant="" compute="" driver=0.0 name="" total="96.0 GiB" available="96.0 GiB" [GIN] 2025/06/05 - 21:10:06 | 200 | 222.334µs | 127.0.0.1 | HEAD "/" [GIN] 2025/06/05 - 21:10:06 | 200 | 49.765208ms | 127.0.0.1 | POST "/api/show" time=2025-06-05T21:10:06.725-04:00 level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/Users/mariogrgic/.ollama/models/blobs/sha256-e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68 gpu=0 parallel=2 available=103079215104 required="20.1 GiB" time=2025-06-05T21:10:06.725-04:00 level=INFO source=server.go:135 msg="system memory" total="128.0 GiB" free="117.4 GiB" free_swap="0 B" time=2025-06-05T21:10:06.726-04:00 level=INFO source=server.go:168 msg=offload library=metal layers.requested=-1 layers.model=63 layers.offload=63 layers.split="" memory.available="[96.0 GiB]" memory.gpu_overhead="0 B" memory.required.full="20.1 GiB" memory.required.partial="20.1 GiB" memory.required.kv="1.6 GiB" memory.required.allocations="[20.1 GiB]" memory.weights.total="15.4 GiB" memory.weights.repeating="14.3 GiB" memory.weights.nonrepeating="1.1 GiB" memory.graph.full="565.0 MiB" memory.graph.partial="565.0 MiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" time=2025-06-05T21:10:06.756-04:00 level=INFO source=server.go:431 msg="starting llama server" cmd="/Volumes/DATA/dev/ollama/ollama.git/ollama runner --ollama-engine --model /Users/mariogrgic/.ollama/models/blobs/sha256-e796792eba26c4d3b04b0ac5adb01a453dd9ec2dfd83b6c59cbf6fe5f30b0f68 --ctx-size 8192 --batch-size 512 --n-gpu-layers 63 --threads 12 --parallel 2 --port 49300" time=2025-06-05T21:10:06.757-04:00 level=INFO source=sched.go:483 msg="loaded runners" count=1 time=2025-06-05T21:10:06.757-04:00 level=INFO source=server.go:591 msg="waiting for llama runner to start responding" time=2025-06-05T21:10:06.757-04:00 level=INFO source=server.go:625 msg="waiting for server to become available" status="llm server not responding" time=2025-06-05T21:10:06.765-04:00 level=INFO source=runner.go:925 msg="starting ollama engine" time=2025-06-05T21:10:06.766-04:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:49300" time=2025-06-05T21:10:06.793-04:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1247 num_key_values=37 time=2025-06-05T21:10:06.793-04:00 level=INFO source=ggml.go:104 msg=system Metal.0.EMBED_LIBRARY=1 CPU.0.ARM_FMA=1 CPU.0.FP16_VA=1 CPU.0.DOTPROD=1 CPU.0.LLAMAFILE=1 CPU.0.ACCELERATE=1 compiler=cgo(clang) unexpected fault address 0x2000000000 fatal error: fault [signal SIGBUS: bus error code=0x1 addr=0x2000000000 pc=0x100edea94] goroutine 14 gp=0x14000103a40 m=8 mp=0x14000600008 [running]: runtime.throw({0x101c05000?, 0x0?}) /usr/local/go/src/runtime/panic.go:1101 +0x38 fp=0x1400004d100 sp=0x1400004d0d0 pc=0x100f4c788 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:922 +0x170 fp=0x1400004d160 sp=0x1400004d100 pc=0x100f4ebf0 indexbytebody() /usr/local/go/src/internal/bytealg/indexbyte_arm64.s:74 +0x64 fp=0x1400004d170 sp=0x1400004d170 pc=0x100edea94 runtime.findnull(0x1400004d208?) /usr/local/go/src/runtime/string.go:577 +0x78 fp=0x1400004d1d0 sp=0x1400004d170 pc=0x100f35ba8 runtime.gostring(0x2000000000) /usr/local/go/src/runtime/string.go:363 +0x20 fp=0x1400004d210 sp=0x1400004d1d0 pc=0x100f4fa60 github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...) _cgo_gotypes.go:300 github.com/ollama/ollama/ml/backend/ggml.New({0x16ef26ca2, 0x6e}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0}) /Volumes/DATA/dev/ollama/ollama.git/ml/backend/ggml/ggml.go:141 +0xaa4 fp=0x1400004dc00 sp=0x1400004d210 pc=0x1013180d4 github.com/ollama/ollama/ml.NewBackend({0x16ef26ca2, 0x6e}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0}) /Volumes/DATA/dev/ollama/ollama.git/ml/backend.go:209 +0x90 fp=0x1400004dc60 sp=0x1400004dc00 pc=0x10130dc90 github.com/ollama/ollama/model.New({0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0}) /Volumes/DATA/dev/ollama/ollama.git/model/model.go:102 +0x5c fp=0x1400004dd60 sp=0x1400004dc60 pc=0x10132464c github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0x140004b9b00, {0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, 0x0}, 0x0}, ...) /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:841 +0x80 fp=0x1400004ddc0 sp=0x1400004dd60 pc=0x10139d3a0 github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0x140004b9b00, {0x102076dd0, 0x14000390410}, {0x16ef26ca2?, 0x0?}, {0xc, 0x0, 0x3f, {0x0, 0x0, ...}, ...}, ...) /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:878 +0xa8 fp=0x1400004df20 sp=0x1400004ddc0 pc=0x10139d6b8 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:959 +0x88 fp=0x1400004dfd0 sp=0x1400004df20 pc=0x10139e738 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400004dfd0 sp=0x1400004dfd0 pc=0x100f54c64 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:959 +0x840 goroutine 1 gp=0x140000021c0 m=nil [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x100f625b0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140001335e0 sp=0x140001335c0 pc=0x100f4c8a8 runtime.netpollblock(0x140004b5678?, 0xfd1170?, 0x1?) /usr/local/go/src/runtime/netpoll.go:575 +0x158 fp=0x14000133620 sp=0x140001335e0 pc=0x100f12728 internal/poll.runtime_pollWait(0x149b16f30, 0x72) /usr/local/go/src/runtime/netpoll.go:351 +0xa0 fp=0x14000133650 sp=0x14000133620 pc=0x100f4ba60 internal/poll.(*pollDesc).wait(0x14000055780?, 0x101f3fb40?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x14000133680 sp=0x14000133650 pc=0x100fcc988 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0x14000055780) /usr/local/go/src/internal/poll/fd_unix.go:620 +0x24c fp=0x14000133730 sp=0x14000133680 pc=0x100fd125c net.(*netFD).accept(0x14000055780) /usr/local/go/src/net/fd_unix.go:172 +0x28 fp=0x140001337f0 sp=0x14000133730 pc=0x1010403c8 net.(*TCPListener).accept(0x1400043b100) /usr/local/go/src/net/tcpsock_posix.go:159 +0x24 fp=0x14000133840 sp=0x140001337f0 pc=0x101054624 net.(*TCPListener).Accept(0x1400043b100) /usr/local/go/src/net/tcpsock.go:380 +0x2c fp=0x14000133880 sp=0x14000133840 pc=0x10105360c net/http.(*onceCloseListener).Accept(0x102076d60?) <autogenerated>:1 +0x30 fp=0x140001338a0 sp=0x14000133880 pc=0x10122eb40 net/http.(*Server).Serve(0x1400011f100, {0x102074978, 0x1400043b100}) /usr/local/go/src/net/http/server.go:3424 +0x290 fp=0x140001339d0 sp=0x140001338a0 pc=0x1012081e0 github.com/ollama/ollama/runner/ollamarunner.Execute({0x14000000270, 0xe, 0xf}) /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:984 +0xb78 fp=0x14000133ce0 sp=0x140001339d0 pc=0x10139e408 github.com/ollama/ollama/runner.Execute({0x14000000250?, 0x0?, 0x0?}) /Volumes/DATA/dev/ollama/ollama.git/runner/runner.go:20 +0x120 fp=0x14000133d10 sp=0x14000133ce0 pc=0x10139ecb0 github.com/ollama/ollama/cmd.NewCLI.func2(0x1400011ef00?, {0x101c03c5e?, 0x4?, 0x101c03c62?}) /Volumes/DATA/dev/ollama/ollama.git/cmd/cmd.go:1529 +0x54 fp=0x14000133d40 sp=0x14000133d10 pc=0x1019e9984 github.com/spf13/cobra.(*Command).execute(0x140004e6f08, {0x14000434f00, 0xf, 0xf}) /Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x648 fp=0x14000133e60 sp=0x14000133d40 pc=0x1010ae968 github.com/spf13/cobra.(*Command).ExecuteC(0x140004cc908) /Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x320 fp=0x14000133f20 sp=0x14000133e60 pc=0x1010af0b0 github.com/spf13/cobra.(*Command).Execute(...) /Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) /Volumes/DATA/dev/ollama/ollama.git/gopath/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 main.main() /Volumes/DATA/dev/ollama/ollama.git/main.go:12 +0x54 fp=0x14000133f40 sp=0x14000133f20 pc=0x1019ea464 runtime.main() /usr/local/go/src/runtime/proc.go:283 +0x284 fp=0x14000133fd0 sp=0x14000133f40 pc=0x100f19244 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000133fd0 sp=0x14000133fd0 pc=0x100f54c64 goroutine 2 gp=0x14000002c40 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000084f90 sp=0x14000084f70 pc=0x100f4c8a8 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:441 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:348 +0xb8 fp=0x14000084fd0 sp=0x14000084f90 pc=0x100f19598 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000084fd0 sp=0x14000084fd0 pc=0x100f54c64 created by runtime.init.7 in goroutine 1 /usr/local/go/src/runtime/proc.go:336 +0x24 goroutine 3 gp=0x14000003180 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000085760 sp=0x14000085740 pc=0x100f4c8a8 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:441 runtime.bgsweep(0x140000b0000) /usr/local/go/src/runtime/mgcsweep.go:316 +0x108 fp=0x140000857b0 sp=0x14000085760 pc=0x100f046c8 runtime.gcenable.gowrap1() /usr/local/go/src/runtime/mgc.go:204 +0x28 fp=0x140000857d0 sp=0x140000857b0 pc=0x100ef84c8 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000857d0 sp=0x140000857d0 pc=0x100f54c64 created by runtime.gcenable in goroutine 1 /usr/local/go/src/runtime/mgc.go:204 +0x6c goroutine 4 gp=0x14000003340 m=nil [GC scavenge wait]: runtime.gopark(0x10000?, 0x101db4f78?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000085f60 sp=0x14000085f40 pc=0x100f4c8a8 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:441 runtime.(*scavengerState).park(0x102917f40) /usr/local/go/src/runtime/mgcscavenge.go:425 +0x5c fp=0x14000085f90 sp=0x14000085f60 pc=0x100f0215c runtime.bgscavenge(0x140000b0000) /usr/local/go/src/runtime/mgcscavenge.go:658 +0xac fp=0x14000085fb0 sp=0x14000085f90 pc=0x100f026fc runtime.gcenable.gowrap2() /usr/local/go/src/runtime/mgc.go:205 +0x28 fp=0x14000085fd0 sp=0x14000085fb0 pc=0x100ef8468 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000085fd0 sp=0x14000085fd0 pc=0x100f54c64 created by runtime.gcenable in goroutine 1 /usr/local/go/src/runtime/mgc.go:205 +0xac goroutine 5 gp=0x14000003c00 m=nil [finalizer wait]: runtime.gopark(0x18000845c8?, 0x1000000000000?, 0xf8?, 0x45?, 0x1012314ac?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000084590 sp=0x14000084570 pc=0x100f4c8a8 runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:196 +0x108 fp=0x140000847d0 sp=0x14000084590 pc=0x100ef74c8 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000847d0 sp=0x140000847d0 pc=0x100f54c64 created by runtime.createfing in goroutine 1 /usr/local/go/src/runtime/mfinal.go:166 +0x80 goroutine 6 gp=0x140001ee700 m=nil [chan receive]: runtime.gopark(0x14000233680?, 0x1400000e1b0?, 0x48?, 0x67?, 0x101014698?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140000866f0 sp=0x140000866d0 pc=0x100f4c8a8 runtime.chanrecv(0x140000be310, 0x0, 0x1) /usr/local/go/src/runtime/chan.go:664 +0x42c fp=0x14000086770 sp=0x140000866f0 pc=0x100ee9f7c runtime.chanrecv1(0x0?, 0x0?) /usr/local/go/src/runtime/chan.go:506 +0x14 fp=0x140000867a0 sp=0x14000086770 pc=0x100ee9b14 runtime.unique_runtime_registerUniqueMapCleanup.func2(...) /usr/local/go/src/runtime/mgc.go:1796 runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() /usr/local/go/src/runtime/mgc.go:1799 +0x3c fp=0x140000867d0 sp=0x140000867a0 pc=0x100efb6ec runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000867d0 sp=0x140000867d0 pc=0x100f54c64 created by unique.runtime_registerUniqueMapCleanup in goroutine 1 /usr/local/go/src/runtime/mgc.go:1794 +0x78 goroutine 7 gp=0x140001eee00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000086f10 sp=0x14000086ef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000086fb0 sp=0x14000086f10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000086fd0 sp=0x14000086fb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000086fd0 sp=0x14000086fd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 18 gp=0x14000504000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000080710 sp=0x140000806f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000807b0 sp=0x14000080710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000807d0 sp=0x140000807b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000807d0 sp=0x140000807d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 34 gp=0x14000102380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011a710 sp=0x1400011a6f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011a7b0 sp=0x1400011a710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011a7d0 sp=0x1400011a7b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011a7d0 sp=0x1400011a7d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 35 gp=0x14000102540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011af10 sp=0x1400011aef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011afb0 sp=0x1400011af10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011afd0 sp=0x1400011afb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011afd0 sp=0x1400011afd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 8 gp=0x140001eefc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000087710 sp=0x140000876f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000877b0 sp=0x14000087710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000877d0 sp=0x140000877b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000877d0 sp=0x140000877d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 19 gp=0x140005041c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000080f10 sp=0x14000080ef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000080fb0 sp=0x14000080f10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000080fd0 sp=0x14000080fb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000080fd0 sp=0x14000080fd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 20 gp=0x14000504380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000081710 sp=0x140000816f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140000817b0 sp=0x14000081710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140000817d0 sp=0x140000817b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000817d0 sp=0x140000817d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 36 gp=0x14000102700 m=nil [GC worker (idle)]: runtime.gopark(0x26313f85800?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011b710 sp=0x1400011b6f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011b7b0 sp=0x1400011b710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011b7d0 sp=0x1400011b7b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011b7d0 sp=0x1400011b7d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 37 gp=0x140001028c0 m=nil [GC worker (idle)]: runtime.gopark(0x102948780?, 0x1?, 0xc?, 0xa8?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011bf10 sp=0x1400011bef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011bfb0 sp=0x1400011bf10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011bfd0 sp=0x1400011bfb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011bfd0 sp=0x1400011bfd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 38 gp=0x14000102a80 m=nil [GC worker (idle)]: runtime.gopark(0x26313daa359?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011c710 sp=0x1400011c6f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011c7b0 sp=0x1400011c710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011c7d0 sp=0x1400011c7b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011c7d0 sp=0x1400011c7d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 39 gp=0x14000102c40 m=nil [GC worker (idle)]: runtime.gopark(0x26313da80ae?, 0x3?, 0x5e?, 0x1b?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x1400011cf10 sp=0x1400011cef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x1400011cfb0 sp=0x1400011cf10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x1400011cfd0 sp=0x1400011cfb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x1400011cfd0 sp=0x1400011cfd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 9 gp=0x140001ef180 m=nil [GC worker (idle)]: runtime.gopark(0x26313f87e6a?, 0x1?, 0x18?, 0x5f?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000087f10 sp=0x14000087ef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000087fb0 sp=0x14000087f10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000087fd0 sp=0x14000087fb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000087fd0 sp=0x14000087fd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 10 gp=0x140001ef340 m=nil [GC worker (idle)]: runtime.gopark(0x26313da7de9?, 0x3?, 0x53?, 0x8?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000116710 sp=0x140001166f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140001167b0 sp=0x14000116710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140001167d0 sp=0x140001167b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001167d0 sp=0x140001167d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 11 gp=0x140001ef500 m=nil [GC worker (idle)]: runtime.gopark(0x26313da7643?, 0x1?, 0x12?, 0xe3?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000116f10 sp=0x14000116ef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000116fb0 sp=0x14000116f10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000116fd0 sp=0x14000116fb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000116fd0 sp=0x14000116fd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 12 gp=0x140001ef6c0 m=nil [GC worker (idle)]: runtime.gopark(0x26313da7a2b?, 0x1?, 0x6f?, 0xa9?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000117710 sp=0x140001176f0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x140001177b0 sp=0x14000117710 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x140001177d0 sp=0x140001177b0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001177d0 sp=0x140001177d0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 13 gp=0x140001ef880 m=nil [GC worker (idle)]: runtime.gopark(0x26313da7b4f?, 0x1?, 0xc3?, 0xb?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x14000117f10 sp=0x14000117ef0 pc=0x100f4c8a8 runtime.gcBgMarkWorker(0x140000bf730) /usr/local/go/src/runtime/mgc.go:1423 +0xdc fp=0x14000117fb0 sp=0x14000117f10 pc=0x100efa95c runtime.gcBgMarkStartWorkers.gowrap1() /usr/local/go/src/runtime/mgc.go:1339 +0x28 fp=0x14000117fd0 sp=0x14000117fb0 pc=0x100efa848 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000117fd0 sp=0x14000117fd0 pc=0x100f54c64 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1339 +0x140 goroutine 15 gp=0x14000103c00 m=nil [sync.WaitGroup.Wait]: runtime.gopark(0x1029266a0?, 0x0?, 0x60?, 0x20?, 0x0?) /usr/local/go/src/runtime/proc.go:435 +0xc8 fp=0x140001196d0 sp=0x140001196b0 pc=0x100f4c8a8 runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:441 runtime.semacquire1(0x140004b9b08, 0x0, 0x1, 0x0, 0x18) /usr/local/go/src/runtime/sema.go:188 +0x204 fp=0x14000119720 sp=0x140001196d0 pc=0x100f2d724 sync.runtime_SemacquireWaitGroup(0x0?) /usr/local/go/src/runtime/sema.go:110 +0x2c fp=0x14000119760 sp=0x14000119720 pc=0x100f4e31c sync.(*WaitGroup).Wait(0x140004b9b00) /usr/local/go/src/sync/waitgroup.go:118 +0x70 fp=0x14000119780 sp=0x14000119760 pc=0x100f60540 github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0x140004b9b00, {0x102076dd0, 0x14000390410}) /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:355 +0x2c fp=0x140001197a0 sp=0x14000119780 pc=0x101399e9c github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2() /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:960 +0x30 fp=0x140001197d0 sp=0x140001197a0 pc=0x10139e670 runtime.goexit({}) /usr/local/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001197d0 sp=0x140001197d0 pc=0x100f54c64 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 /Volumes/DATA/dev/ollama/ollama.git/runner/ollamarunner/runner.go:960 +0x898 time=2025-06-05T21:10:06.799-04:00 level=ERROR source=server.go:457 msg="llama runner terminated" error="exit status 2" time=2025-06-05T21:10:07.007-04:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault" [GIN] 2025/06/05 - 21:10:07 | 500 | 335.97975ms | 127.0.0.1 | POST "/api/generate" ``` ### Relevant log output ```shell ``` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-29 04:42:14 -05:00
GiteaMirror commented 2026-04-29 04:42:18 -05:00
Author
Owner
Copy Link

@mario-grgic commented on GitHub (Jun 6, 2025):

Turns out I am building with go 1.24.2. Once I upgraded golang to 1.24.4, there is no crash.

<!-- gh-comment-id:2947657098 --> @mario-grgic commented on GitHub (Jun 6, 2025): Turns out I am building with go 1.24.2. Once I upgraded golang to 1.24.4, there is no crash.
GiteaMirror commented 2026-04-29 04:42:18 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Jun 6, 2025):

@mario-grgic Thanks for the issue. Some changes don't break the go build cache properly and so a go clean -cache is required. Upgrading Go must have fixed this.

<!-- gh-comment-id:2947956058 --> @jmorganca commented on GitHub (Jun 6, 2025): @mario-grgic Thanks for the issue. Some changes don't break the go build cache properly and so a `go clean -cache` is required. Upgrading Go must have fixed this.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#53759
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?