[GH-ISSUE #11211] Segmentation fault when running gemma3n #7384

New Issue

GiteaMirror · 2026-04-12T19:27:49-05:00

GiteaMirror commented

2026-04-12 19:27:49 -05:00

Originally created by @YoMamasNerd on GitHub (Jun 26, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11211

What is the issue?

When running gemma3n ollama crashes with a seg fault.

Relevant log output

Jun 26 23:08:53 openwebui ollama[167]: [GIN] 2025/06/26 - 23:08:53 | 200 |      19.547µs |       127.0.0.1 | HEAD     "/"
Jun 26 23:08:53 openwebui ollama[167]: [GIN] 2025/06/26 - 23:08:53 | 200 |    2.462291ms |       127.0.0.1 | GET      "/api/tags"
Jun 26 23:09:11 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:11 | 200 |      19.808µs |       127.0.0.1 | HEAD     "/"
Jun 26 23:09:11 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:11 | 200 |  103.064489ms |       127.0.0.1 | POST     "/api/show"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.605+02:00 level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 gpu=GPU-9a848503-fb33-6ada-ed50-8570e46db60c parallel=2 available=12384731136 required="5.7 GiB"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.760+02:00 level=INFO source=server.go:135 msg="system memory" total="21.5 GiB" free="20.9 GiB" free_swap="512.0 MiB"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.760+02:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=36 layers.offload=36 layers.split="" memory.available="[11.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.7 GiB" memory.required.partial="5.7 GiB" memory.required.kv="560.0 MiB" memory.required.allocations="[5.7 GiB]" memory.weights.total="2.6 GiB" memory.weights.repeating="2.2 GiB" memory.weights.nonrepeating="420.4 MiB" memory.graph.full="2.0 GiB" memory.graph.partial="3.7 GiB"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=server.go:438 msg="starting llama server" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 --ctx-size 8192 --batch-size 512 --n-gpu-layers 36 --threads 3 --parallel 2 --port 33503"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=sched.go:483 msg="loaded runners" count=1
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.813+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.822+02:00 level=INFO source=runner.go:925 msg="starting ollama engine"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.827+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:33503"
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.879+02:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3n file_type=Q4_K_M name="" description="" num_tensors=847 num_key_values=40
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: found 1 CUDA devices:
Jun 26 23:09:11 openwebui ollama[167]:   Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yesJun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CUDA backend from /usr/lib/ollama/libggml-cuda.so
Jun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-haswell.so
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: found 1 CUDA devices:
Jun 26 23:09:11 openwebui ollama[167]:   Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yesJun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so
Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.957+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
Jun 26 23:09:12 openwebui ollama[167]: unexpected fault address 0x2e2300000
Jun 26 23:09:12 openwebui ollama[167]: fatal error: fault
Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]
Jun 26 23:09:12 openwebui ollama[167]: goroutine 10 gp=0xc000582700 m=5 mp=0xc000100008 [running]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.throw({0x65273431c479?, 0xc000582700?})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/panic.go:1096 +0x4a fp=0xc001c27110 sp=0xc001c270e0 pc=0x652733473c2a
Jun 26 23:09:12 openwebui ollama[167]: runtime.sigpanic()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/signal_unix.go:939 +0x26c fp=0xc001c27170 sp=0xc001c27110 pc=0x6527334760ac
Jun 26 23:09:12 openwebui ollama[167]: indexbytebody()
Jun 26 23:09:12 openwebui ollama[167]:         internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc001c27178
sp=0xc001c27170 pc=0x652733403780
Jun 26 23:09:12 openwebui ollama[167]: runtime.findnull(0xc001c271f8?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/string.go:577 +0x79 fp=0xc001c271d0 sp=0xc001c27178 pc=0x65273345b8b9
Jun 26 23:09:12 openwebui ollama[167]: runtime.gostring(0x2e2300000)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/string.go:363 +0x1c fp=0xc001c27208 sp=0xc001c271d0 pc=0x652733476f1c
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...)
Jun 26 23:09:12 openwebui ollama[167]:         _cgo_gotypes.go:311
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend/ggml.New({0x7ffff6922d63, 0x62},
{0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1336
fp=0xc001c27c18 sp=0xc001c27208 pc=0x6527338b0376
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml.NewBackend({0x7ffff6922d63, 0x62}, {0x3,
0x0, 0x24, {0x0, 0x0, 0x0}, 0x0})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc001c27c70 sp=0xc001c27c18 pc=0x6527338a1e11
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/model.New({0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/model/model.go:102 +0x8f fp=0xc001c27d68 sp=0xc001c27c70 pc=0x6527338bfa2f
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc00061f7a0, {0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0}, ...)
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc001c27dc8 sp=0xc001c27d68 pc=0x652733961b6d
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc00061f7a0, {0x6527347d4880, 0xc000129d60}, {0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, ...}, ...}, ...)
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc001c27f20 sp=0xc001c27dc8 pc=0x652733961ed8
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc001c27fe0 sp=0xc001c27f20 pc=0x652733963307
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc001c27fe8 sp=0xc001c27fe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11
Jun 26 23:09:12 openwebui ollama[167]: goroutine 1 gp=0xc000002380 m=nil [IO wait]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000049650 sp=0xc000049630 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.netpollblock(0x652733471ad3?, 0x3340cb46?, 0x27?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/netpoll.go:575 +0xf7 fp=0xc000049688 sp=0xc000049650
pc=0x652733438837
Jun 26 23:09:12 openwebui ollama[167]: internal/poll.runtime_pollWait(0x751b8ec58eb0, 0x72)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/netpoll.go:351 +0x85 fp=0xc0000496a8 sp=0xc000049688
pc=0x652733472f65
Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*pollDesc).wait(0xc000623680?, 0x380016?, 0x0)
Jun 26 23:09:12 openwebui ollama[167]:         internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000496d0 sp=0xc0000496a8 pc=0x6527334fa3a7
Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*pollDesc).waitRead(...)
Jun 26 23:09:12 openwebui ollama[167]:         internal/poll/fd_poll_runtime.go:89
Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*FD).Accept(0xc000623680)
Jun 26 23:09:12 openwebui ollama[167]:         internal/poll/fd_unix.go:620 +0x295 fp=0xc000049778 sp=0xc0000496d0 pc=0x6527334ff775
Jun 26 23:09:12 openwebui ollama[167]: net.(*netFD).accept(0xc000623680)
Jun 26 23:09:12 openwebui ollama[167]:         net/fd_unix.go:172 +0x29 fp=0xc000049830 sp=0xc000049778 pc=0x652733571c89
Jun 26 23:09:12 openwebui ollama[167]: net.(*TCPListener).accept(0xc0002dc100)
Jun 26 23:09:12 openwebui ollama[167]:         net/tcpsock_posix.go:159 +0x1b fp=0xc000049880 sp=0xc000049830 pc=0x65273358763b
Jun 26 23:09:12 openwebui ollama[167]: net.(*TCPListener).Accept(0xc0002dc100)
Jun 26 23:09:12 openwebui ollama[167]:         net/tcpsock.go:380 +0x30 fp=0xc0000498b0 sp=0xc000049880 pc=0x6527335864f0
Jun 26 23:09:12 openwebui ollama[167]: net/http.(*onceCloseListener).Accept(0x6527347d4810?)
Jun 26 23:09:12 openwebui ollama[167]:         <autogenerated>:1 +0x24 fp=0xc0000498c8 sp=0xc0000498b0 pc=0x65273379dc44
Jun 26 23:09:12 openwebui ollama[167]: net/http.(*Server).Serve(0xc0001f0600, {0x6527347d2408, 0xc0002dc100})
Jun 26 23:09:12 openwebui ollama[167]:         net/http/server.go:3424 +0x30c fp=0xc0000499f8 sp=0xc0000498c8 pc=0x65273377550c
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034150, 0xe, 0xf})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc000049d08 sp=0xc0000499f8 pc=0x652733962f69
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner.Execute({0xc000034130?, 0x0?, 0x0?})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000049d30 sp=0xc000049d08 pc=0x652733963869
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/cmd.NewCLI.func2(0xc0001f1400?, {0x65273431b075?, 0x4?, 0x65273431b079?})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc000049d58 sp=0xc000049d30 pc=0x6527340c0645
Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).execute(0xc0004eaf08, {0xc00040aa50, 0xf, 0xf})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000049e78 sp=0xc000049d58 pc=0x6527335eb2dc
Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).ExecuteC(0xc0004c8f08)
Jun 26 23:09:12 openwebui ollama[167]:         github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000049f30 sp=0xc000049e78 pc=0x6527335ebb25
Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).Execute(...)
Jun 26 23:09:12 openwebui ollama[167]:         github.com/spf13/cobra@v1.7.0/command.go:992
Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).ExecuteContext(...)
Jun 26 23:09:12 openwebui ollama[167]:         github.com/spf13/cobra@v1.7.0/command.go:985
Jun 26 23:09:12 openwebui ollama[167]: main.main()
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000049f50 sp=0xc000049f30 pc=0x6527340c10cd
Jun 26 23:09:12 openwebui ollama[167]: runtime.main()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:283 +0x29d fp=0xc000049fe0 sp=0xc000049f50 pc=0x65273343febd
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000049fe8 sp=0xc000049fe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000072fa8 sp=0xc000072f88 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:441
Jun 26 23:09:12 openwebui ollama[167]: runtime.forcegchelper()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:348 +0xb8 fp=0xc000072fe0 sp=0xc000072fa8 pc=0x6527334401f8
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.init.7 in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:336 +0x1a
Jun 26 23:09:12 openwebui ollama[167]: goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000073780 sp=0xc000073760 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:441
Jun 26 23:09:12 openwebui ollama[167]: runtime.bgsweep(0xc00007e000)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgcsweep.go:316 +0xdf fp=0xc0000737c8 sp=0xc000073780 pc=0x65273342a99f
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcenable.gowrap1()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:204 +0x25 fp=0xc0000737e0 sp=0xc0000737c8 pc=0x65273341ed85
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcenable in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:204 +0x66
Jun 26 23:09:12 openwebui ollama[167]: goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x10000?, 0x6527344dac38?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000073f78 sp=0xc000073f58 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:441
Jun 26 23:09:12 openwebui ollama[167]: runtime.(*scavengerState).park(0x6527350628e0)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgcscavenge.go:425 +0x49 fp=0xc000073fa8 sp=0xc000073f78 pc=0x6527334283e9
Jun 26 23:09:12 openwebui ollama[167]: runtime.bgscavenge(0xc00007e000)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgcscavenge.go:658 +0x59 fp=0xc000073fc8 sp=0xc000073fa8 pc=0x652733428979
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcenable.gowrap2()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:205 +0x25 fp=0xc000073fe0 sp=0xc000073fc8 pc=0x65273341ed25
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000073fe8 sp=0xc000073fe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcenable in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:205 +0xa5
Jun 26 23:09:12 openwebui ollama[167]: goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000072688?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000072630 sp=0xc000072610 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.runfinq()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mfinal.go:196 +0x107 fp=0xc0000727e0 sp=0xc000072630
pc=0x65273341dd47
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.createfing in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mfinal.go:166 +0x3d
Jun 26 23:09:12 openwebui ollama[167]: goroutine 6 gp=0xc0001ce8c0 m=nil [chan receive]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0xc00022b5e0?, 0xc000590018?, 0x60?, 0x47?, 0x6527335589c8?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000074718 sp=0xc0000746f8 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.chanrecv(0xc000042380, 0x0, 0x1)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/chan.go:664 +0x445 fp=0xc000074790 sp=0xc000074718 pc=0x65273340f725
Jun 26 23:09:12 openwebui ollama[167]: runtime.chanrecv1(0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/chan.go:506 +0x12 fp=0xc0000747b8 sp=0xc000074790 pc=0x65273340f2b2
Jun 26 23:09:12 openwebui ollama[167]: runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1796
Jun 26 23:09:12 openwebui ollama[167]: runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1799 +0x2f fp=0xc0000747e0 sp=0xc0000747b8 pc=0x652733421f2f
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0000747e8 sp=0xc0000747e0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by unique.runtime_registerUniqueMapCleanup in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1794 +0x85
Jun 26 23:09:12 openwebui ollama[167]: goroutine 7 gp=0xc0001cefc0 m=nil [GC worker (idle)]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc000074f38 sp=0xc000074f18 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkWorker(0xc000043960)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1423 +0xe9 fp=0xc000074fc8 sp=0xc000074f38 pc=0x652733421249
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkStartWorkers.gowrap1()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1339 +0x25 fp=0xc000074fe0 sp=0xc000074fc8 pc=0x652733421125
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcBgMarkStartWorkers in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1339 +0x105
Jun 26 23:09:12 openwebui ollama[167]: goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x4a79dbff8039f?, 0x0?, 0x0?, 0x0?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkWorker(0xc000043960)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1423 +0xe9 fp=0xc00011afc8 sp=0xc00011af38 pc=0x652733421249
Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkStartWorkers.gowrap1()
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1339 +0x25 fp=0xc00011afe0 sp=0xc00011afc8 pc=0x652733421125
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00011afe8 sp=0xc00011afe0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcBgMarkStartWorkers in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         runtime/mgc.go:1339 +0x105
Jun 26 23:09:12 openwebui ollama[167]: goroutine 11 gp=0xc0005828c0 m=nil [sync.WaitGroup.Wait]:
Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0xc0?, 0x40?, 0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:435 +0xce fp=0xc00011d6d0 sp=0xc00011d6b0 pc=0x652733473d4e
Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/proc.go:441
Jun 26 23:09:12 openwebui ollama[167]: runtime.semacquire1(0xc00061f7a8, 0x0, 0x1, 0x0, 0x18)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/sema.go:188 +0x229 fp=0xc00011d738 sp=0xc00011d6d0 pc=0x652733453489
Jun 26 23:09:12 openwebui ollama[167]: sync.runtime_SemacquireWaitGroup(0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         runtime/sema.go:110 +0x25 fp=0xc00011d770 sp=0xc00011d738 pc=0x652733475765
Jun 26 23:09:12 openwebui ollama[167]: sync.(*WaitGroup).Wait(0x0?)
Jun 26 23:09:12 openwebui ollama[167]:         sync/waitgroup.go:118 +0x48 fp=0xc00011d798 sp=0xc00011d770 pc=0x652733486dc8
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc00061f7a0, {0x6527347d4880, 0xc000129d60})
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc00011d7b8 sp=0xc00011d798 pc=0x65273395da85
Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2()
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc00011d7e0 sp=0xc00011d7b8 pc=0x652733963208
Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({})
Jun 26 23:09:12 openwebui ollama[167]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00011d7e8 sp=0xc00011d7e0 pc=0x65273347b481
Jun 26 23:09:12 openwebui ollama[167]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
Jun 26 23:09:12 openwebui ollama[167]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74
Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.092+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error"
Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.124+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2"
Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.342+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault"
Jun 26 23:09:12 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:12 | 500 |  1.065972031s |       127.0.0.1 | POST     "/api/generate"
Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.496+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.153974104 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6
Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.746+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.403432821 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6
Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.996+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.653711673 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.9.3

Originally created by @YoMamasNerd on GitHub (Jun 26, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11211 ### What is the issue? When running gemma3n ollama crashes with a seg fault. ### Relevant log output ```shell Jun 26 23:08:53 openwebui ollama[167]: [GIN] 2025/06/26 - 23:08:53 | 200 | 19.547µs | 127.0.0.1 | HEAD "/" Jun 26 23:08:53 openwebui ollama[167]: [GIN] 2025/06/26 - 23:08:53 | 200 | 2.462291ms | 127.0.0.1 | GET "/api/tags" Jun 26 23:09:11 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:11 | 200 | 19.808µs | 127.0.0.1 | HEAD "/" Jun 26 23:09:11 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:11 | 200 | 103.064489ms | 127.0.0.1 | POST "/api/show" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.605+02:00 level=INFO source=sched.go:788 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 gpu=GPU-9a848503-fb33-6ada-ed50-8570e46db60c parallel=2 available=12384731136 required="5.7 GiB" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.760+02:00 level=INFO source=server.go:135 msg="system memory" total="21.5 GiB" free="20.9 GiB" free_swap="512.0 MiB" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.760+02:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=36 layers.offload=36 layers.split="" memory.available="[11.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.7 GiB" memory.required.partial="5.7 GiB" memory.required.kv="560.0 MiB" memory.required.allocations="[5.7 GiB]" memory.weights.total="2.6 GiB" memory.weights.repeating="2.2 GiB" memory.weights.nonrepeating="420.4 MiB" memory.graph.full="2.0 GiB" memory.graph.partial="3.7 GiB" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=server.go:438 msg="starting llama server" cmd="/usr/bin/ollama runner --ollama-engine --model /root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 --ctx-size 8192 --batch-size 512 --n-gpu-layers 36 --threads 3 --parallel 2 --port 33503" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=sched.go:483 msg="loaded runners" count=1 Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.812+02:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.813+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.822+02:00 level=INFO source=runner.go:925 msg="starting ollama engine" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.827+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:33503" Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.879+02:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3n file_type=Q4_K_M name="" description="" num_tensors=847 num_key_values=40 Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: found 1 CUDA devices: Jun 26 23:09:11 openwebui ollama[167]: Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yesJun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CUDA backend from /usr/lib/ollama/libggml-cuda.so Jun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CPU backend from /usr/lib/ollama/libggml-cpu-haswell.so Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Jun 26 23:09:11 openwebui ollama[167]: ggml_cuda_init: found 1 CUDA devices: Jun 26 23:09:11 openwebui ollama[167]: Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yesJun 26 23:09:11 openwebui ollama[167]: load_backend: loaded CUDA backend from /usr/lib/ollama/cuda_v12/libggml-cuda.so Jun 26 23:09:11 openwebui ollama[167]: time=2025-06-26T23:09:11.957+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) Jun 26 23:09:12 openwebui ollama[167]: unexpected fault address 0x2e2300000 Jun 26 23:09:12 openwebui ollama[167]: fatal error: fault Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780] Jun 26 23:09:12 openwebui ollama[167]: goroutine 10 gp=0xc000582700 m=5 mp=0xc000100008 [running]: Jun 26 23:09:12 openwebui ollama[167]: runtime.throw({0x65273431c479?, 0xc000582700?}) Jun 26 23:09:12 openwebui ollama[167]: runtime/panic.go:1096 +0x4a fp=0xc001c27110 sp=0xc001c270e0 pc=0x652733473c2a Jun 26 23:09:12 openwebui ollama[167]: runtime.sigpanic() Jun 26 23:09:12 openwebui ollama[167]: runtime/signal_unix.go:939 +0x26c fp=0xc001c27170 sp=0xc001c27110 pc=0x6527334760ac Jun 26 23:09:12 openwebui ollama[167]: indexbytebody() Jun 26 23:09:12 openwebui ollama[167]: internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc001c27178 sp=0xc001c27170 pc=0x652733403780 Jun 26 23:09:12 openwebui ollama[167]: runtime.findnull(0xc001c271f8?) Jun 26 23:09:12 openwebui ollama[167]: runtime/string.go:577 +0x79 fp=0xc001c271d0 sp=0xc001c27178 pc=0x65273345b8b9 Jun 26 23:09:12 openwebui ollama[167]: runtime.gostring(0x2e2300000) Jun 26 23:09:12 openwebui ollama[167]: runtime/string.go:363 +0x1c fp=0xc001c27208 sp=0xc001c271d0 pc=0x652733476f1c Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...) Jun 26 23:09:12 openwebui ollama[167]: _cgo_gotypes.go:311 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend/ggml.New({0x7ffff6922d63, 0x62}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1336 fp=0xc001c27c18 sp=0xc001c27208 pc=0x6527338b0376 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml.NewBackend({0x7ffff6922d63, 0x62}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc001c27c70 sp=0xc001c27c18 pc=0x6527338a1e11 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/model.New({0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/model/model.go:102 +0x8f fp=0xc001c27d68 sp=0xc001c27c70 pc=0x6527338bfa2f Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc00061f7a0, {0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, 0x0}, 0x0}, ...) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc001c27dc8 sp=0xc001c27d68 pc=0x652733961b6d Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc00061f7a0, {0x6527347d4880, 0xc000129d60}, {0x7ffff6922d63?, 0x0?}, {0x3, 0x0, 0x24, {0x0, 0x0, ...}, ...}, ...) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc001c27f20 sp=0xc001c27dc8 pc=0x652733961ed8 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc001c27fe0 sp=0xc001c27f20 pc=0x652733963307 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc001c27fe8 sp=0xc001c27fe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11 Jun 26 23:09:12 openwebui ollama[167]: goroutine 1 gp=0xc000002380 m=nil [IO wait]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000049650 sp=0xc000049630 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.netpollblock(0x652733471ad3?, 0x3340cb46?, 0x27?) Jun 26 23:09:12 openwebui ollama[167]: runtime/netpoll.go:575 +0xf7 fp=0xc000049688 sp=0xc000049650 pc=0x652733438837 Jun 26 23:09:12 openwebui ollama[167]: internal/poll.runtime_pollWait(0x751b8ec58eb0, 0x72) Jun 26 23:09:12 openwebui ollama[167]: runtime/netpoll.go:351 +0x85 fp=0xc0000496a8 sp=0xc000049688 pc=0x652733472f65 Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*pollDesc).wait(0xc000623680?, 0x380016?, 0x0) Jun 26 23:09:12 openwebui ollama[167]: internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0000496d0 sp=0xc0000496a8 pc=0x6527334fa3a7 Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*pollDesc).waitRead(...) Jun 26 23:09:12 openwebui ollama[167]: internal/poll/fd_poll_runtime.go:89 Jun 26 23:09:12 openwebui ollama[167]: internal/poll.(*FD).Accept(0xc000623680) Jun 26 23:09:12 openwebui ollama[167]: internal/poll/fd_unix.go:620 +0x295 fp=0xc000049778 sp=0xc0000496d0 pc=0x6527334ff775 Jun 26 23:09:12 openwebui ollama[167]: net.(*netFD).accept(0xc000623680) Jun 26 23:09:12 openwebui ollama[167]: net/fd_unix.go:172 +0x29 fp=0xc000049830 sp=0xc000049778 pc=0x652733571c89 Jun 26 23:09:12 openwebui ollama[167]: net.(*TCPListener).accept(0xc0002dc100) Jun 26 23:09:12 openwebui ollama[167]: net/tcpsock_posix.go:159 +0x1b fp=0xc000049880 sp=0xc000049830 pc=0x65273358763b Jun 26 23:09:12 openwebui ollama[167]: net.(*TCPListener).Accept(0xc0002dc100) Jun 26 23:09:12 openwebui ollama[167]: net/tcpsock.go:380 +0x30 fp=0xc0000498b0 sp=0xc000049880 pc=0x6527335864f0 Jun 26 23:09:12 openwebui ollama[167]: net/http.(*onceCloseListener).Accept(0x6527347d4810?) Jun 26 23:09:12 openwebui ollama[167]: <autogenerated>:1 +0x24 fp=0xc0000498c8 sp=0xc0000498b0 pc=0x65273379dc44 Jun 26 23:09:12 openwebui ollama[167]: net/http.(*Server).Serve(0xc0001f0600, {0x6527347d2408, 0xc0002dc100}) Jun 26 23:09:12 openwebui ollama[167]: net/http/server.go:3424 +0x30c fp=0xc0000499f8 sp=0xc0000498c8 pc=0x65273377550c Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034150, 0xe, 0xf}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc000049d08 sp=0xc0000499f8 pc=0x652733962f69 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner.Execute({0xc000034130?, 0x0?, 0x0?}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000049d30 sp=0xc000049d08 pc=0x652733963869 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/cmd.NewCLI.func2(0xc0001f1400?, {0x65273431b075?, 0x4?, 0x65273431b079?}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc000049d58 sp=0xc000049d30 pc=0x6527340c0645 Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).execute(0xc0004eaf08, {0xc00040aa50, 0xf, 0xf}) Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000049e78 sp=0xc000049d58 pc=0x6527335eb2dc Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).ExecuteC(0xc0004c8f08) Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000049f30 sp=0xc000049e78 pc=0x6527335ebb25 Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).Execute(...) Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra@v1.7.0/command.go:992 Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra.(*Command).ExecuteContext(...) Jun 26 23:09:12 openwebui ollama[167]: github.com/spf13/cobra@v1.7.0/command.go:985 Jun 26 23:09:12 openwebui ollama[167]: main.main() Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000049f50 sp=0xc000049f30 pc=0x6527340c10cd Jun 26 23:09:12 openwebui ollama[167]: runtime.main() Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:283 +0x29d fp=0xc000049fe0 sp=0xc000049f50 pc=0x65273343febd Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000049fe8 sp=0xc000049fe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000072fa8 sp=0xc000072f88 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:441 Jun 26 23:09:12 openwebui ollama[167]: runtime.forcegchelper() Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:348 +0xb8 fp=0xc000072fe0 sp=0xc000072fa8 pc=0x6527334401f8 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000072fe8 sp=0xc000072fe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.init.7 in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:336 +0x1a Jun 26 23:09:12 openwebui ollama[167]: goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000073780 sp=0xc000073760 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:441 Jun 26 23:09:12 openwebui ollama[167]: runtime.bgsweep(0xc00007e000) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgcsweep.go:316 +0xdf fp=0xc0000737c8 sp=0xc000073780 pc=0x65273342a99f Jun 26 23:09:12 openwebui ollama[167]: runtime.gcenable.gowrap1() Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:204 +0x25 fp=0xc0000737e0 sp=0xc0000737c8 pc=0x65273341ed85 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0000737e8 sp=0xc0000737e0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcenable in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:204 +0x66 Jun 26 23:09:12 openwebui ollama[167]: goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x10000?, 0x6527344dac38?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000073f78 sp=0xc000073f58 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:441 Jun 26 23:09:12 openwebui ollama[167]: runtime.(*scavengerState).park(0x6527350628e0) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgcscavenge.go:425 +0x49 fp=0xc000073fa8 sp=0xc000073f78 pc=0x6527334283e9 Jun 26 23:09:12 openwebui ollama[167]: runtime.bgscavenge(0xc00007e000) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgcscavenge.go:658 +0x59 fp=0xc000073fc8 sp=0xc000073fa8 pc=0x652733428979 Jun 26 23:09:12 openwebui ollama[167]: runtime.gcenable.gowrap2() Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:205 +0x25 fp=0xc000073fe0 sp=0xc000073fc8 pc=0x65273341ed25 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000073fe8 sp=0xc000073fe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcenable in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:205 +0xa5 Jun 26 23:09:12 openwebui ollama[167]: goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000072688?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000072630 sp=0xc000072610 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.runfinq() Jun 26 23:09:12 openwebui ollama[167]: runtime/mfinal.go:196 +0x107 fp=0xc0000727e0 sp=0xc000072630 pc=0x65273341dd47 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0000727e8 sp=0xc0000727e0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.createfing in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mfinal.go:166 +0x3d Jun 26 23:09:12 openwebui ollama[167]: goroutine 6 gp=0xc0001ce8c0 m=nil [chan receive]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0xc00022b5e0?, 0xc000590018?, 0x60?, 0x47?, 0x6527335589c8?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000074718 sp=0xc0000746f8 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.chanrecv(0xc000042380, 0x0, 0x1) Jun 26 23:09:12 openwebui ollama[167]: runtime/chan.go:664 +0x445 fp=0xc000074790 sp=0xc000074718 pc=0x65273340f725 Jun 26 23:09:12 openwebui ollama[167]: runtime.chanrecv1(0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/chan.go:506 +0x12 fp=0xc0000747b8 sp=0xc000074790 pc=0x65273340f2b2 Jun 26 23:09:12 openwebui ollama[167]: runtime.unique_runtime_registerUniqueMapCleanup.func2(...) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1796 Jun 26 23:09:12 openwebui ollama[167]: runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1799 +0x2f fp=0xc0000747e0 sp=0xc0000747b8 pc=0x652733421f2f Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0000747e8 sp=0xc0000747e0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by unique.runtime_registerUniqueMapCleanup in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1794 +0x85 Jun 26 23:09:12 openwebui ollama[167]: goroutine 7 gp=0xc0001cefc0 m=nil [GC worker (idle)]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc000074f38 sp=0xc000074f18 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkWorker(0xc000043960) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1423 +0xe9 fp=0xc000074fc8 sp=0xc000074f38 pc=0x652733421249 Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkStartWorkers.gowrap1() Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1339 +0x25 fp=0xc000074fe0 sp=0xc000074fc8 pc=0x652733421125 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000074fe8 sp=0xc000074fe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcBgMarkStartWorkers in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1339 +0x105 Jun 26 23:09:12 openwebui ollama[167]: goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x4a79dbff8039f?, 0x0?, 0x0?, 0x0?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkWorker(0xc000043960) Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1423 +0xe9 fp=0xc00011afc8 sp=0xc00011af38 pc=0x652733421249 Jun 26 23:09:12 openwebui ollama[167]: runtime.gcBgMarkStartWorkers.gowrap1() Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1339 +0x25 fp=0xc00011afe0 sp=0xc00011afc8 pc=0x652733421125 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00011afe8 sp=0xc00011afe0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by runtime.gcBgMarkStartWorkers in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: runtime/mgc.go:1339 +0x105 Jun 26 23:09:12 openwebui ollama[167]: goroutine 11 gp=0xc0005828c0 m=nil [sync.WaitGroup.Wait]: Jun 26 23:09:12 openwebui ollama[167]: runtime.gopark(0x0?, 0x0?, 0xc0?, 0x40?, 0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:435 +0xce fp=0xc00011d6d0 sp=0xc00011d6b0 pc=0x652733473d4e Jun 26 23:09:12 openwebui ollama[167]: runtime.goparkunlock(...) Jun 26 23:09:12 openwebui ollama[167]: runtime/proc.go:441 Jun 26 23:09:12 openwebui ollama[167]: runtime.semacquire1(0xc00061f7a8, 0x0, 0x1, 0x0, 0x18) Jun 26 23:09:12 openwebui ollama[167]: runtime/sema.go:188 +0x229 fp=0xc00011d738 sp=0xc00011d6d0 pc=0x652733453489 Jun 26 23:09:12 openwebui ollama[167]: sync.runtime_SemacquireWaitGroup(0x0?) Jun 26 23:09:12 openwebui ollama[167]: runtime/sema.go:110 +0x25 fp=0xc00011d770 sp=0xc00011d738 pc=0x652733475765 Jun 26 23:09:12 openwebui ollama[167]: sync.(*WaitGroup).Wait(0x0?) Jun 26 23:09:12 openwebui ollama[167]: sync/waitgroup.go:118 +0x48 fp=0xc00011d798 sp=0xc00011d770 pc=0x652733486dc8 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc00061f7a0, {0x6527347d4880, 0xc000129d60}) Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc00011d7b8 sp=0xc00011d798 pc=0x65273395da85 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2() Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc00011d7e0 sp=0xc00011d7b8 pc=0x652733963208 Jun 26 23:09:12 openwebui ollama[167]: runtime.goexit({}) Jun 26 23:09:12 openwebui ollama[167]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00011d7e8 sp=0xc00011d7e0 pc=0x65273347b481 Jun 26 23:09:12 openwebui ollama[167]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 Jun 26 23:09:12 openwebui ollama[167]: github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74 Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.092+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error" Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.124+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2" Jun 26 23:09:12 openwebui ollama[167]: time=2025-06-26T23:09:12.342+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault" Jun 26 23:09:12 openwebui ollama[167]: [GIN] 2025/06/26 - 23:09:12 | 500 | 1.065972031s | 127.0.0.1 | POST "/api/generate" Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.496+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.153974104 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.746+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.403432821 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 Jun 26 23:09:17 openwebui ollama[167]: time=2025-06-26T23:09:17.996+02:00 level=WARN source=sched.go:687 msg="gpu VRAM usage didn't recover within timeout" seconds=5.653711673 runner.size="5.7 GiB" runner.vram="5.7 GiB" runner.parallel=2 runner.pid=6312 runner.model=/root/.ollama/models/blobs/sha256-38e8dcc30df4eb0e29eaf5c74ba6ce3f2cd66badad50768fc14362acfb8b8cb6 ``` ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.9.3

GiteaMirror added the bug label 2026-04-12 19:27:49 -05:00

GiteaMirror closed this issue

2026-04-12 19:27:51 -05:00

GiteaMirror commented

2026-04-12 19:27:52 -05:00

@LFd3v commented on GitHub (Jun 26, 2025):

Same problem here. Actually I get "Error: llama runner process has terminated: error:fault" when I try to run any Gemma3 models now, while it worked fine before v0.9.3.

Linux iGPU + Nvidia 3050 6GB.

@LFd3v commented on GitHub (Jun 26, 2025): Same problem here. Actually I get "Error: llama runner process has terminated: error:fault" when I try to run *any* Gemma3 models now, while it worked fine before v0.9.3. Linux iGPU + Nvidia 3050 6GB.

GiteaMirror commented

2026-04-12 19:27:52 -05:00

@illusdolphin commented on GitHub (Jun 26, 2025):

The same issue with mistral-small3.1:latest and mistral-small3.2:latest, gemma3:12b and gemma3:27b:
POST /api/chat
{ "model": "mistral-small3.1:latest", "messages": [ { "role": "user", "content": "hello" } ], "stream": false }

Result is: 
{
    "error": "llama runner process has terminated: error:fault"
}

5070ti, windows 11
It used to work before on 0.9x, so looks like regression of 0.9.3

update: it was a suggestion in #11209 to install https://aka.ms/vs/17/release/vc_redist.x64.exe, tried to install, but:
[7564:68A0][2025-06-27T00:15:39]e000: Error 0x80070666: Cannot install a product when a newer version is installed.
Looks like already newer version installed and it doesn't help

@illusdolphin commented on GitHub (Jun 26, 2025): The same issue with mistral-small3.1:latest and mistral-small3.2:latest, gemma3:12b and gemma3:27b: `POST /api/chat` `{ "model": "mistral-small3.1:latest", "messages": [ { "role": "user", "content": "hello" } ], "stream": false }` ``` Result is: { "error": "llama runner process has terminated: error:fault" } ``` 5070ti, windows 11 It used to work before on 0.9x, so looks like regression of 0.9.3 update: it was a suggestion in #11209 to install https://aka.ms/vs/17/release/vc_redist.x64.exe, tried to install, but: `[7564:68A0][2025-06-27T00:15:39]e000: Error 0x80070666: Cannot install a product when a newer version is installed.` Looks like already newer version installed and it doesn't help

GiteaMirror commented

2026-04-12 19:27:53 -05:00

@Bzucko commented on GitHub (Jun 27, 2025):

Same issue here.
I am using all-in-one package from https://www.techpowerup.com/download/visual-c-redistributable-runtime-package-all-in-one/ and even if I install https://aka.ms/vs/17/release/vc_redist.x64.exe on top of that, which is 14.44.35208.0 , after the restart error persist "Error: llama runner process has terminated: error:fault", when running(loading from disk does not even start) any gemma3 model.

@Bzucko commented on GitHub (Jun 27, 2025): Same issue here. I am using all-in-one package from https://www.techpowerup.com/download/visual-c-redistributable-runtime-package-all-in-one/ and even if I install https://aka.ms/vs/17/release/vc_redist.x64.exe on top of that, which is 14.44.35208.0 , after the restart error persist "Error: llama runner process has terminated: error:fault", when running(loading from disk does not even start) any gemma3 model.

GiteaMirror commented

2026-04-12 19:27:54 -05:00

@baiwei666 commented on GitHub (Jun 27, 2025):

Same issue

@baiwei666 commented on GitHub (Jun 27, 2025): Same issue

GiteaMirror commented

2026-04-12 19:27:55 -05:00

@MartinsCode commented on GitHub (Jun 27, 2025):

Same issue here, same log file.

Linux, AMD Ryzen 5, Nvidia 3060 (12 GB VRAM)

Models working with 0.9.2:

Mistral-small:22b, Mistral-small:24b
Mistral-small3.2:24b
gemma3:12b
phi4:latest

Models working with 0.9.3/0.9.4-rc0:

Mistral-small:22b, Mistral-Small:24b
phi4:latest

Models not working with 0.9.3/0.9.4-rc0:

Mistral-small3.2:24b (which is a very little bit bigger than Mistral-small:24b and has according to API another "general.architecture")
gemma3:12b
gemma3n:e4b

@MartinsCode commented on GitHub (Jun 27, 2025): Same issue here, same log file. Linux, AMD Ryzen 5, Nvidia 3060 (12 GB VRAM) Models working with 0.9.2: - Mistral-small:22b, Mistral-small:24b - Mistral-small3.2:24b - gemma3:12b - phi4:latest Models working with 0.9.3/0.9.4-rc0: - Mistral-small:22b, Mistral-Small:24b - phi4:latest Models not working with 0.9.3/0.9.4-rc0: - Mistral-small3.2:24b (which is a very little bit bigger than Mistral-small:24b and has according to API another "general.architecture") - gemma3:12b - gemma3n:e4b

GiteaMirror commented

2026-04-12 19:27:56 -05:00

@LFd3v commented on GitHub (Jun 27, 2025):

More info: the segfault occurs only when ollama tries to load the model using the GPU, it works fine if the whole model is loaded in RAM (ollama ps shows 100% CPU).

segfault log

jun 27 11:42:36 ollama[232241]: load_backend: loaded CUDA backend from /home/user/ollama/lib/ollama/cuda_v12/libggml-cuda.so
jun 27 11:42:36 ollama[232241]: time=2025-06-27T11:42:36.739-03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
jun 27 11:42:36 ollama[232241]: unexpected fault address 0x15e7f0000
jun 27 11:42:36 ollama[232241]: fatal error: fault
jun 27 11:42:36 ollama[232241]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x15e7f0000 pc=0x55c4f93b1780]
jun 27 11:42:36 ollama[232241]: goroutine 8 gp=0xc0004a0e00 m=9 mp=0xc000496008 [running]:

@LFd3v commented on GitHub (Jun 27, 2025): More info: the segfault occurs only when `ollama` tries to load the model using the GPU, it works fine if the whole model is loaded in RAM (`ollama ps` shows 100% CPU). <details><summary>segfault log</summary> jun 27 11:42:36 ollama[232241]: load_backend: loaded CUDA backend from /home/user/ollama/lib/ollama/cuda_v12/libggml-cuda.so jun 27 11:42:36 ollama[232241]: time=2025-06-27T11:42:36.739-03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) jun 27 11:42:36 ollama[232241]: unexpected fault address 0x15e7f0000 jun 27 11:42:36 ollama[232241]: fatal error: fault jun 27 11:42:36 ollama[232241]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x15e7f0000 pc=0x55c4f93b1780] jun 27 11:42:36 ollama[232241]: goroutine 8 gp=0xc0004a0e00 m=9 mp=0xc000496008 [running]: </details>

GiteaMirror commented

2026-04-12 19:27:56 -05:00

@jmorganca commented on GitHub (Jun 27, 2025):

@LFd3v do you have the remaining logs after that? Sorry about the issue

@jmorganca commented on GitHub (Jun 27, 2025): @LFd3v do you have the remaining logs after that? Sorry about the issue

GiteaMirror commented

2026-04-12 19:27:57 -05:00

@LFd3v commented on GitHub (Jun 27, 2025):

@jmorganca Sure, I am glad to help.

I have restarted Ollama and then tried to run ollama run gemma3:4b --verbose which worked fine before.

Details

jun 27 12:51:21 systemd[1959]: Stopping Ollama Service...
jun 27 12:51:21 systemd[1959]: Stopped Ollama Service.
jun 27 12:51:21 systemd[1959]: ollama.service: Consumed 1h 35min 25.887s CPU time, 11G memory peak, 769.4M memory swap peak.
jun 27 12:51:21 systemd[1959]: Started Ollama Service.
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.325-03:00 level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:q8_0 OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:2 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/user/ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[moz-extension://* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.334-03:00 level=INFO source=images.go:476 msg="total blobs: 251"
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.339-03:00 level=INFO source=images.go:483 msg="total unused blobs removed: 0"
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.341-03:00 level=INFO source=routes.go:1288 msg="Listening on [::]:11434 (version 0.9.3)"
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.341-03:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.599-03:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c105e882-a73e-b413-9d0c-4a78b0c40f3c library=cuda variant=v12 compute=8.6 driver=12.9 name="NVIDIA GeForce RTX 3050 6GB Laptop GPU" total="5.7 GiB" available="5.5 GiB"
jun 27 12:51:38 ollama[1148060]: [GIN] 2025/06/27 - 12:51:38 | 200 |      28.688µs |       127.0.0.1 | HEAD     "/"
jun 27 12:51:38 ollama[1148060]: [GIN] 2025/06/27 - 12:51:38 | 200 |   87.555524ms |       127.0.0.1 | POST     "/api/show"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.330-03:00 level=INFO source=server.go:135 msg="system memory" total="15.3 GiB" free="9.9 GiB" free_swap="8.9 GiB"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.447-03:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=34 layers.split="" memory.available="[5.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.7 GiB" memory.required.partial="3.4 GiB" memory.required.kv="127.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.447-03:00 level=INFO source=server.go:218 msg="enabling flash attention"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.497-03:00 level=INFO source=server.go:438 msg="starting llama server" cmd="/user/ollama/bin/ollama runner --ollama-engine --model /user/ollama/models/blobs/sha256-377655e65351a68cddfbd69b7c8dc60c1890466254628c3e494661a52c2c5ada --ctx-size 4096 --batch-size 512 --n-gpu-layers 34 --threads 4 --flash-attn --kv-cache-type q8_0 --parallel 1 --port 41237"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=sched.go:483 msg="loaded runners" count=1
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.508-03:00 level=INFO source=runner.go:925 msg="starting ollama engine"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.509-03:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:41237"
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.556-03:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=35
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: found 1 CUDA devices:
jun 27 12:51:39 ollama[1148060]:   Device 0: NVIDIA GeForce RTX 3050 6GB Laptop GPU, compute capability 8.6, VMM: yes
jun 27 12:51:39 ollama[1148060]: load_backend: loaded CUDA backend from /user/ollama/lib/ollama/libggml-cuda.so
jun 27 12:51:39 ollama[1148060]: load_backend: loaded CPU backend from /user/ollama/lib/ollama/libggml-cpu-alderlake.so
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: found 1 CUDA devices:
jun 27 12:51:39 ollama[1148060]:   Device 0: NVIDIA GeForce RTX 3050 6GB Laptop GPU, compute capability 8.6, VMM: yes
jun 27 12:51:39 ollama[1148060]: load_backend: loaded CUDA backend from /user/ollama/lib/ollama/cuda_v12/libggml-cuda.so
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.676-03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.749-03:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model"
jun 27 12:51:39 ollama[1148060]: unexpected fault address 0x1617f0000
jun 27 12:51:39 ollama[1148060]: fatal error: fault
jun 27 12:51:39 ollama[1148060]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x1617f0000 pc=0x5653a2459780]
jun 27 12:51:39 ollama[1148060]: goroutine 16 gp=0xc000102a80 m=3 mp=0xc000083008 [running]:
jun 27 12:51:39 ollama[1148060]: runtime.throw({0x5653a3372479?, 0xc000102a80?})
jun 27 12:51:39 ollama[1148060]:         runtime/panic.go:1096 +0x4a fp=0xc000053110 sp=0xc0000530e0 pc=0x5653a24c9c2a
jun 27 12:51:39 ollama[1148060]: runtime.sigpanic()
jun 27 12:51:39 ollama[1148060]:         runtime/signal_unix.go:939 +0x26c fp=0xc000053170 sp=0xc000053110 pc=0x5653a24cc0ac
jun 27 12:51:39 ollama[1148060]: indexbytebody()
jun 27 12:51:39 ollama[1148060]:         internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc000053178 sp=0xc000053170 pc=0x5653a2459780
jun 27 12:51:39 ollama[1148060]: runtime.findnull(0xc0000531f8?)
jun 27 12:51:39 ollama[1148060]:         runtime/string.go:577 +0x79 fp=0xc0000531d0 sp=0xc000053178 pc=0x5653a24b18b9
jun 27 12:51:39 ollama[1148060]: runtime.gostring(0x1617f0000)
jun 27 12:51:39 ollama[1148060]:         runtime/string.go:363 +0x1c fp=0xc000053208 sp=0xc0000531d0 pc=0x5653a24ccf1c
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...)
jun 27 12:51:39 ollama[1148060]:         _cgo_gotypes.go:311
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend/ggml.New({0x7ffe1cb5cdd5, 0x6f}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1336 fp=0xc000053c18 sp=0xc000053208 pc=0x5653a2906376
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml.NewBackend({0x7ffe1cb5cdd5, 0x6f}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc000053c70 sp=0xc000053c18 pc=0x5653a28f7e11
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/model.New({0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/model/model.go:102 +0x8f fp=0xc000053d68 sp=0xc000053c70 pc=0x5653a2915a2f
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc00063d9e0, {0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1}, ...)
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc000053dc8 sp=0xc000053d68 pc=0x5653a29b7b6d
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc00063d9e0, {0x5653a382a880, 0xc0005afae0}, {0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, ...}, ...}, ...)
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc000053f20 sp=0xc000053dc8 pc=0x5653a29b7ed8
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc000053fe0 sp=0xc000053f20 pc=0x5653a29b9307
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000053fe8 sp=0xc000053fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11
jun 27 12:51:39 ollama[1148060]: goroutine 1 gp=0xc000002380 m=nil [IO wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000121650 sp=0xc000121630 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.netpollblock(0xc0001216a0?, 0xa2462b46?, 0x53?)
jun 27 12:51:39 ollama[1148060]:         runtime/netpoll.go:575 +0xf7 fp=0xc000121688 sp=0xc000121650 pc=0x5653a248e837
jun 27 12:51:39 ollama[1148060]: internal/poll.runtime_pollWait(0x7fbff94c6eb0, 0x72)
jun 27 12:51:39 ollama[1148060]:         runtime/netpoll.go:351 +0x85 fp=0xc0001216a8 sp=0xc000121688 pc=0x5653a24c8f65
jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).wait(0xc00048dd80?, 0x900000036?, 0x0)
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001216d0 sp=0xc0001216a8 pc=0x5653a25503a7
jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).waitRead(...)
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_poll_runtime.go:89
jun 27 12:51:39 ollama[1148060]: internal/poll.(*FD).Accept(0xc00048dd80)
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_unix.go:620 +0x295 fp=0xc000121778 sp=0xc0001216d0 pc=0x5653a2555775
jun 27 12:51:39 ollama[1148060]: net.(*netFD).accept(0xc00048dd80)
jun 27 12:51:39 ollama[1148060]:         net/fd_unix.go:172 +0x29 fp=0xc000121830 sp=0xc000121778 pc=0x5653a25c7c89
jun 27 12:51:39 ollama[1148060]: net.(*TCPListener).accept(0xc00042c000)
jun 27 12:51:39 ollama[1148060]:         net/tcpsock_posix.go:159 +0x1b fp=0xc000121880 sp=0xc000121830 pc=0x5653a25dd63b
jun 27 12:51:39 ollama[1148060]: net.(*TCPListener).Accept(0xc00042c000)
jun 27 12:51:39 ollama[1148060]:         net/tcpsock.go:380 +0x30 fp=0xc0001218b0 sp=0xc000121880 pc=0x5653a25dc4f0
jun 27 12:51:39 ollama[1148060]: net/http.(*onceCloseListener).Accept(0xc0004e43f0?)
jun 27 12:51:39 ollama[1148060]:         <autogenerated>:1 +0x24 fp=0xc0001218c8 sp=0xc0001218b0 pc=0x5653a27f3c44
jun 27 12:51:39 ollama[1148060]: net/http.(*Server).Serve(0xc0002d2000, {0x5653a3828408, 0xc00042c000})
jun 27 12:51:39 ollama[1148060]:         net/http/server.go:3424 +0x30c fp=0xc0001219f8 sp=0xc0001218c8 pc=0x5653a27cb50c
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034170, 0x11, 0x11})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc000121d08 sp=0xc0001219f8 pc=0x5653a29b8f69
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner.Execute({0xc000034150?, 0x0?, 0x0?})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000121d30 sp=0xc000121d08 pc=0x5653a29b9869
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/cmd.NewCLI.func2(0xc0001f7100?, {0x5653a3371075?, 0x4?, 0x5653a3371079?})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc000121d58 sp=0xc000121d30 pc=0x5653a3116645
jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).execute(0xc0004af508, {0xc00063d7a0, 0x12, 0x12})
jun 27 12:51:39 ollama[1148060]:         github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000121e78 sp=0xc000121d58 pc=0x5653a26412dc
jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).ExecuteC(0xc000644c08)
jun 27 12:51:39 ollama[1148060]:         github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000121f30 sp=0xc000121e78 pc=0x5653a2641b25
jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).Execute(...)
jun 27 12:51:39 ollama[1148060]:         github.com/spf13/cobra@v1.7.0/command.go:992
jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).ExecuteContext(...)
jun 27 12:51:39 ollama[1148060]:         github.com/spf13/cobra@v1.7.0/command.go:985
jun 27 12:51:39 ollama[1148060]: main.main()
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000121f50 sp=0xc000121f30 pc=0x5653a31170cd
jun 27 12:51:39 ollama[1148060]: runtime.main()
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:283 +0x29d fp=0xc000121fe0 sp=0xc000121f50 pc=0x5653a2495ebd
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000121fe8 sp=0xc000121fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007cfa8 sp=0xc00007cf88 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:441
jun 27 12:51:39 ollama[1148060]: runtime.forcegchelper()
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:348 +0xb8 fp=0xc00007cfe0 sp=0xc00007cfa8 pc=0x5653a24961f8
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007cfe8 sp=0xc00007cfe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.init.7 in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:336 +0x1a
jun 27 12:51:39 ollama[1148060]: goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007d780 sp=0xc00007d760 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:441
jun 27 12:51:39 ollama[1148060]: runtime.bgsweep(0xc000046080)
jun 27 12:51:39 ollama[1148060]:         runtime/mgcsweep.go:316 +0xdf fp=0xc00007d7c8 sp=0xc00007d780 pc=0x5653a248099f
jun 27 12:51:39 ollama[1148060]: runtime.gcenable.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:204 +0x25 fp=0xc00007d7e0 sp=0xc00007d7c8 pc=0x5653a2474d85
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007d7e8 sp=0xc00007d7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcenable in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:204 +0x66
jun 27 12:51:39 ollama[1148060]: goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x10000?, 0x5653a3530c38?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007df78 sp=0xc00007df58 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:441
jun 27 12:51:39 ollama[1148060]: runtime.(*scavengerState).park(0x5653a40b88e0)
jun 27 12:51:39 ollama[1148060]:         runtime/mgcscavenge.go:425 +0x49 fp=0xc00007dfa8 sp=0xc00007df78 pc=0x5653a247e3e9
jun 27 12:51:39 ollama[1148060]: runtime.bgscavenge(0xc000046080)
jun 27 12:51:39 ollama[1148060]:         runtime/mgcscavenge.go:658 +0x59 fp=0xc00007dfc8 sp=0xc00007dfa8 pc=0x5653a247e979
jun 27 12:51:39 ollama[1148060]: runtime.gcenable.gowrap2()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:205 +0x25 fp=0xc00007dfe0 sp=0xc00007dfc8 pc=0x5653a2474d25
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007dfe8 sp=0xc00007dfe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcenable in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:205 +0xa5
jun 27 12:51:39 ollama[1148060]: goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc00007c688?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007c630 sp=0xc00007c610 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.runfinq()
jun 27 12:51:39 ollama[1148060]:         runtime/mfinal.go:196 +0x107 fp=0xc00007c7e0 sp=0xc00007c630 pc=0x5653a2473d47
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007c7e8 sp=0xc00007c7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.createfing in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mfinal.go:166 +0x3d
jun 27 12:51:39 ollama[1148060]: goroutine 6 gp=0xc0001d48c0 m=nil [chan receive]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0xc000233540?, 0xc000590018?, 0x60?, 0xe7?, 0x5653a25ae9c8?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007e718 sp=0xc00007e6f8 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.chanrecv(0xc000048460, 0x0, 0x1)
jun 27 12:51:39 ollama[1148060]:         runtime/chan.go:664 +0x445 fp=0xc00007e790 sp=0xc00007e718 pc=0x5653a2465725
jun 27 12:51:39 ollama[1148060]: runtime.chanrecv1(0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/chan.go:506 +0x12 fp=0xc00007e7b8 sp=0xc00007e790 pc=0x5653a24652b2
jun 27 12:51:39 ollama[1148060]: runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1796
jun 27 12:51:39 ollama[1148060]: runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1799 +0x2f fp=0xc00007e7e0 sp=0xc00007e7b8 pc=0x5653a2477f2f
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007e7e8 sp=0xc00007e7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by unique.runtime_registerUniqueMapCleanup in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1794 +0x85
jun 27 12:51:39 ollama[1148060]: goroutine 7 gp=0xc0001d4c40 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007ef38 sp=0xc00007ef18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00007efc8 sp=0xc00007ef38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00007efe0 sp=0xc00007efc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007efe8 sp=0xc00007efe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 8 gp=0xc0001d4e00 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007f738 sp=0xc00007f718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00007f7c8 sp=0xc00007f738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00007f7e0 sp=0xc00007f7c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007f7e8 sp=0xc00007f7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 9 gp=0xc0001d4fc0 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007ff38 sp=0xc00007ff18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00007ffc8 sp=0xc00007ff38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00007ffe0 sp=0xc00007ffc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007ffe8 sp=0xc00007ffe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 10 gp=0xc0001d5180 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000078738 sp=0xc000078718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc0000787c8 sp=0xc000078738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc0000787e0 sp=0xc0000787c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0000787e8 sp=0xc0000787e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 11 gp=0xc0001d5340 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636cc5706a?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000078f38 sp=0xc000078f18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc000078fc8 sp=0xc000078f38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc000078fe0 sp=0xc000078fc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000078fe8 sp=0xc000078fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a4167120?, 0x1?, 0x6b?, 0xd5?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00050a738 sp=0xc00050a718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00050a7c8 sp=0xc00050a738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00050a7e0 sp=0xc00050a7c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9caf3a?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000506738 sp=0xc000506718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc0005067c8 sp=0xc000506738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc0005067e0 sp=0xc0005067c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 35 gp=0xc000102540 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c3a07?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000506f38 sp=0xc000506f18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc000506fc8 sp=0xc000506f38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc000506fe0 sp=0xc000506fc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 12 gp=0xc0001d5500 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636cc62da8?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000079738 sp=0xc000079718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc0000797c8 sp=0xc000079738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc0000797e0 sp=0xc0000797c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc0000797e8 sp=0xc0000797e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 13 gp=0xc0001d56c0 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a4167120?, 0x1?, 0xc9?, 0xf8?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000079f38 sp=0xc000079f18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc000079fc8 sp=0xc000079f38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc000079fe0 sp=0xc000079fc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 14 gp=0xc0001d5880 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c36bb?, 0x0?, 0x0?, 0x0?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007a738 sp=0xc00007a718 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00007a7c8 sp=0xc00007a738 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00007a7e0 sp=0xc00007a7c8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007a7e8 sp=0xc00007a7e0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 15 gp=0xc0001d5a40 m=nil [GC worker (idle)]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c5df5?, 0x1?, 0x33?, 0xb6?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00007af38 sp=0xc00007af18 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40)
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1423 +0xe9 fp=0xc00007afc8 sp=0xc00007af38 pc=0x5653a2477249
jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1()
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x25 fp=0xc00007afe0 sp=0xc00007afc8 pc=0x5653a2477125
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00007afe8 sp=0xc00007afe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1
jun 27 12:51:39 ollama[1148060]:         runtime/mgc.go:1339 +0x105
jun 27 12:51:39 ollama[1148060]: goroutine 50 gp=0xc000102c40 m=nil [sync.WaitGroup.Wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x20?, 0x81?, 0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc00050bed0 sp=0xc00050beb0 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:441
jun 27 12:51:39 ollama[1148060]: runtime.semacquire1(0xc00063d9e8, 0x0, 0x1, 0x0, 0x18)
jun 27 12:51:39 ollama[1148060]:         runtime/sema.go:188 +0x229 fp=0xc00050bf38 sp=0xc00050bed0 pc=0x5653a24a9489
jun 27 12:51:39 ollama[1148060]: sync.runtime_SemacquireWaitGroup(0x0?)
jun 27 12:51:39 ollama[1148060]:         runtime/sema.go:110 +0x25 fp=0xc00050bf70 sp=0xc00050bf38 pc=0x5653a24cb765
jun 27 12:51:39 ollama[1148060]: sync.(*WaitGroup).Wait(0x0?)
jun 27 12:51:39 ollama[1148060]:         sync/waitgroup.go:118 +0x48 fp=0xc00050bf98 sp=0xc00050bf70 pc=0x5653a24dcdc8
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc00063d9e0, {0x5653a382a880, 0xc0005afae0})
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc00050bfb8 sp=0xc00050bf98 pc=0x5653a29b3a85
jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2()
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc00050bfe0 sp=0xc00050bfb8 pc=0x5653a29b9208
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
jun 27 12:51:39 ollama[1148060]:         github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74
jun 27 12:51:39 ollama[1148060]: goroutine 51 gp=0xc000103180 m=nil [IO wait]:
jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a25539a5?, 0xc00048c280?, 0x40?, 0x3a?, 0xb?)
jun 27 12:51:39 ollama[1148060]:         runtime/proc.go:435 +0xce fp=0xc000123948 sp=0xc000123928 pc=0x5653a24c9d4e
jun 27 12:51:39 ollama[1148060]: runtime.netpollblock(0x5653a24ed0b8?, 0xa2462b46?, 0x53?)
jun 27 12:51:39 ollama[1148060]:         runtime/netpoll.go:575 +0xf7 fp=0xc000123980 sp=0xc000123948 pc=0x5653a248e837
jun 27 12:51:39 ollama[1148060]: internal/poll.runtime_pollWait(0x7fbff94c6d98, 0x72)
jun 27 12:51:39 ollama[1148060]:         runtime/netpoll.go:351 +0x85 fp=0xc0001239a0 sp=0xc000123980 pc=0x5653a24c8f65
jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).wait(0xc00048c280?, 0xc001776000?, 0x0)
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001239c8 sp=0xc0001239a0 pc=0x5653a25503a7
jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).waitRead(...)
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_poll_runtime.go:89
jun 27 12:51:39 ollama[1148060]: internal/poll.(*FD).Read(0xc00048c280, {0xc001776000, 0x1000, 0x1000})
jun 27 12:51:39 ollama[1148060]:         internal/poll/fd_unix.go:165 +0x27a fp=0xc000123a60 sp=0xc0001239c8 pc=0x5653a255169a
jun 27 12:51:39 ollama[1148060]: net.(*netFD).Read(0xc00048c280, {0xc001776000?, 0xc000123ad0?, 0x5653a2550865?})
jun 27 12:51:39 ollama[1148060]:         net/fd_posix.go:55 +0x25 fp=0xc000123aa8 sp=0xc000123a60 pc=0x5653a25c5ce5
jun 27 12:51:39 ollama[1148060]: net.(*conn).Read(0xc0005a6010, {0xc001776000?, 0x0?, 0x0?})
jun 27 12:51:39 ollama[1148060]:         net/net.go:194 +0x45 fp=0xc000123af0 sp=0xc000123aa8 pc=0x5653a25d40a5
jun 27 12:51:39 ollama[1148060]: net/http.(*connReader).Read(0xc000226d50, {0xc001776000, 0x1000, 0x1000})
jun 27 12:51:39 ollama[1148060]:         net/http/server.go:798 +0x159 fp=0xc000123b40 sp=0xc000123af0 pc=0x5653a27c03b9
jun 27 12:51:39 ollama[1148060]: bufio.(*Reader).fill(0xc000110060)
jun 27 12:51:39 ollama[1148060]:         bufio/bufio.go:113 +0x103 fp=0xc000123b78 sp=0xc000123b40 pc=0x5653a25eb843
jun 27 12:51:39 ollama[1148060]: bufio.(*Reader).Peek(0xc000110060, 0x4)
jun 27 12:51:39 ollama[1148060]:         bufio/bufio.go:152 +0x53 fp=0xc000123b98 sp=0xc000123b78 pc=0x5653a25eb973
jun 27 12:51:39 ollama[1148060]: net/http.(*conn).serve(0xc0004e43f0, {0x5653a382a848, 0xc00036a660})
jun 27 12:51:39 ollama[1148060]:         net/http/server.go:2137 +0x785 fp=0xc000123fb8 sp=0xc000123b98 pc=0x5653a27c61a5
jun 27 12:51:39 ollama[1148060]: net/http.(*Server).Serve.gowrap3()
jun 27 12:51:39 ollama[1148060]:         net/http/server.go:3454 +0x28 fp=0xc000123fe0 sp=0xc000123fb8 pc=0x5653a27cb908
jun 27 12:51:39 ollama[1148060]: runtime.goexit({})
jun 27 12:51:39 ollama[1148060]:         runtime/asm_amd64.s:1700 +0x1 fp=0xc000123fe8 sp=0xc000123fe0 pc=0x5653a24d1481
jun 27 12:51:39 ollama[1148060]: created by net/http.(*Server).Serve in goroutine 1
jun 27 12:51:39 ollama[1148060]:         net/http/server.go:3454 +0x485
jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.799-03:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2"
jun 27 12:51:40 ollama[1148060]: time=2025-06-27T12:51:39.999-03:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault"

PS: I tried the same without these enviroment variables just to be sure they are not affecting anything, but got the same error:

Environment="OLLAMA_NUM_PARALLEL=1"
Environment="OLLAMA_MAX_LOADED_MODELS=2"
Environment="OLLAMA_FLASH_ATTENTION=1"
Environment="OLLAMA_KV_CACHE_TYPE=q8_0"
Environment="GGML_CUDA_ENABLE_UNIFIED_MEMORY=1"

Please feel free to ask for more info or tests. And thanks to all the Ollama dev team.

@LFd3v commented on GitHub (Jun 27, 2025): @jmorganca Sure, I am glad to help. I have restarted Ollama and then tried to run `ollama run gemma3:4b --verbose` which worked fine before. <details><summary>Details</summary> ``` jun 27 12:51:21 systemd[1959]: Stopping Ollama Service... jun 27 12:51:21 systemd[1959]: Stopped Ollama Service. jun 27 12:51:21 systemd[1959]: ollama.service: Consumed 1h 35min 25.887s CPU time, 11G memory peak, 769.4M memory swap peak. jun 27 12:51:21 systemd[1959]: Started Ollama Service. jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.325-03:00 level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE:q8_0 OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:2 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/user/ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[moz-extension://* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.334-03:00 level=INFO source=images.go:476 msg="total blobs: 251" jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.339-03:00 level=INFO source=images.go:483 msg="total unused blobs removed: 0" jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.341-03:00 level=INFO source=routes.go:1288 msg="Listening on [::]:11434 (version 0.9.3)" jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.341-03:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs" jun 27 12:51:21 ollama[1148060]: time=2025-06-27T12:51:21.599-03:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-c105e882-a73e-b413-9d0c-4a78b0c40f3c library=cuda variant=v12 compute=8.6 driver=12.9 name="NVIDIA GeForce RTX 3050 6GB Laptop GPU" total="5.7 GiB" available="5.5 GiB" jun 27 12:51:38 ollama[1148060]: [GIN] 2025/06/27 - 12:51:38 | 200 | 28.688µs | 127.0.0.1 | HEAD "/" jun 27 12:51:38 ollama[1148060]: [GIN] 2025/06/27 - 12:51:38 | 200 | 87.555524ms | 127.0.0.1 | POST "/api/show" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.330-03:00 level=INFO source=server.go:135 msg="system memory" total="15.3 GiB" free="9.9 GiB" free_swap="8.9 GiB" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.447-03:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=35 layers.offload=34 layers.split="" memory.available="[5.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.7 GiB" memory.required.partial="3.4 GiB" memory.required.kv="127.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="2.3 GiB" memory.weights.repeating="1.8 GiB" memory.weights.nonrepeating="525.0 MiB" memory.graph.full="517.0 MiB" memory.graph.partial="1.0 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.447-03:00 level=INFO source=server.go:218 msg="enabling flash attention" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.497-03:00 level=INFO source=server.go:438 msg="starting llama server" cmd="/user/ollama/bin/ollama runner --ollama-engine --model /user/ollama/models/blobs/sha256-377655e65351a68cddfbd69b7c8dc60c1890466254628c3e494661a52c2c5ada --ctx-size 4096 --batch-size 512 --n-gpu-layers 34 --threads 4 --flash-attn --kv-cache-type q8_0 --parallel 1 --port 41237" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=sched.go:483 msg="loaded runners" count=1 jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.498-03:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server not responding" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.508-03:00 level=INFO source=runner.go:925 msg="starting ollama engine" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.509-03:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:41237" jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.556-03:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=883 num_key_values=35 jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: found 1 CUDA devices: jun 27 12:51:39 ollama[1148060]: Device 0: NVIDIA GeForce RTX 3050 6GB Laptop GPU, compute capability 8.6, VMM: yes jun 27 12:51:39 ollama[1148060]: load_backend: loaded CUDA backend from /user/ollama/lib/ollama/libggml-cuda.so jun 27 12:51:39 ollama[1148060]: load_backend: loaded CPU backend from /user/ollama/lib/ollama/libggml-cpu-alderlake.so jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no jun 27 12:51:39 ollama[1148060]: ggml_cuda_init: found 1 CUDA devices: jun 27 12:51:39 ollama[1148060]: Device 0: NVIDIA GeForce RTX 3050 6GB Laptop GPU, compute capability 8.6, VMM: yes jun 27 12:51:39 ollama[1148060]: load_backend: loaded CUDA backend from /user/ollama/lib/ollama/cuda_v12/libggml-cuda.so jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.676-03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.749-03:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model" jun 27 12:51:39 ollama[1148060]: unexpected fault address 0x1617f0000 jun 27 12:51:39 ollama[1148060]: fatal error: fault jun 27 12:51:39 ollama[1148060]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x1617f0000 pc=0x5653a2459780] jun 27 12:51:39 ollama[1148060]: goroutine 16 gp=0xc000102a80 m=3 mp=0xc000083008 [running]: jun 27 12:51:39 ollama[1148060]: runtime.throw({0x5653a3372479?, 0xc000102a80?}) jun 27 12:51:39 ollama[1148060]: runtime/panic.go:1096 +0x4a fp=0xc000053110 sp=0xc0000530e0 pc=0x5653a24c9c2a jun 27 12:51:39 ollama[1148060]: runtime.sigpanic() jun 27 12:51:39 ollama[1148060]: runtime/signal_unix.go:939 +0x26c fp=0xc000053170 sp=0xc000053110 pc=0x5653a24cc0ac jun 27 12:51:39 ollama[1148060]: indexbytebody() jun 27 12:51:39 ollama[1148060]: internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc000053178 sp=0xc000053170 pc=0x5653a2459780 jun 27 12:51:39 ollama[1148060]: runtime.findnull(0xc0000531f8?) jun 27 12:51:39 ollama[1148060]: runtime/string.go:577 +0x79 fp=0xc0000531d0 sp=0xc000053178 pc=0x5653a24b18b9 jun 27 12:51:39 ollama[1148060]: runtime.gostring(0x1617f0000) jun 27 12:51:39 ollama[1148060]: runtime/string.go:363 +0x1c fp=0xc000053208 sp=0xc0000531d0 pc=0x5653a24ccf1c jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...) jun 27 12:51:39 ollama[1148060]: _cgo_gotypes.go:311 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend/ggml.New({0x7ffe1cb5cdd5, 0x6f}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1336 fp=0xc000053c18 sp=0xc000053208 pc=0x5653a2906376 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml.NewBackend({0x7ffe1cb5cdd5, 0x6f}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc000053c70 sp=0xc000053c18 pc=0x5653a28f7e11 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/model.New({0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/model/model.go:102 +0x8f fp=0xc000053d68 sp=0xc000053c70 pc=0x5653a2915a2f jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc00063d9e0, {0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, 0x0}, 0x1}, ...) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc000053dc8 sp=0xc000053d68 pc=0x5653a29b7b6d jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc00063d9e0, {0x5653a382a880, 0xc0005afae0}, {0x7ffe1cb5cdd5?, 0x0?}, {0x4, 0x0, 0x22, {0x0, 0x0, ...}, ...}, ...) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc000053f20 sp=0xc000053dc8 pc=0x5653a29b7ed8 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc000053fe0 sp=0xc000053f20 pc=0x5653a29b9307 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000053fe8 sp=0xc000053fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11 jun 27 12:51:39 ollama[1148060]: goroutine 1 gp=0xc000002380 m=nil [IO wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000121650 sp=0xc000121630 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.netpollblock(0xc0001216a0?, 0xa2462b46?, 0x53?) jun 27 12:51:39 ollama[1148060]: runtime/netpoll.go:575 +0xf7 fp=0xc000121688 sp=0xc000121650 pc=0x5653a248e837 jun 27 12:51:39 ollama[1148060]: internal/poll.runtime_pollWait(0x7fbff94c6eb0, 0x72) jun 27 12:51:39 ollama[1148060]: runtime/netpoll.go:351 +0x85 fp=0xc0001216a8 sp=0xc000121688 pc=0x5653a24c8f65 jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).wait(0xc00048dd80?, 0x900000036?, 0x0) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001216d0 sp=0xc0001216a8 pc=0x5653a25503a7 jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).waitRead(...) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_poll_runtime.go:89 jun 27 12:51:39 ollama[1148060]: internal/poll.(*FD).Accept(0xc00048dd80) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_unix.go:620 +0x295 fp=0xc000121778 sp=0xc0001216d0 pc=0x5653a2555775 jun 27 12:51:39 ollama[1148060]: net.(*netFD).accept(0xc00048dd80) jun 27 12:51:39 ollama[1148060]: net/fd_unix.go:172 +0x29 fp=0xc000121830 sp=0xc000121778 pc=0x5653a25c7c89 jun 27 12:51:39 ollama[1148060]: net.(*TCPListener).accept(0xc00042c000) jun 27 12:51:39 ollama[1148060]: net/tcpsock_posix.go:159 +0x1b fp=0xc000121880 sp=0xc000121830 pc=0x5653a25dd63b jun 27 12:51:39 ollama[1148060]: net.(*TCPListener).Accept(0xc00042c000) jun 27 12:51:39 ollama[1148060]: net/tcpsock.go:380 +0x30 fp=0xc0001218b0 sp=0xc000121880 pc=0x5653a25dc4f0 jun 27 12:51:39 ollama[1148060]: net/http.(*onceCloseListener).Accept(0xc0004e43f0?) jun 27 12:51:39 ollama[1148060]: <autogenerated>:1 +0x24 fp=0xc0001218c8 sp=0xc0001218b0 pc=0x5653a27f3c44 jun 27 12:51:39 ollama[1148060]: net/http.(*Server).Serve(0xc0002d2000, {0x5653a3828408, 0xc00042c000}) jun 27 12:51:39 ollama[1148060]: net/http/server.go:3424 +0x30c fp=0xc0001219f8 sp=0xc0001218c8 pc=0x5653a27cb50c jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute({0xc000034170, 0x11, 0x11}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc000121d08 sp=0xc0001219f8 pc=0x5653a29b8f69 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner.Execute({0xc000034150?, 0x0?, 0x0?}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc000121d30 sp=0xc000121d08 pc=0x5653a29b9869 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/cmd.NewCLI.func2(0xc0001f7100?, {0x5653a3371075?, 0x4?, 0x5653a3371079?}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc000121d58 sp=0xc000121d30 pc=0x5653a3116645 jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).execute(0xc0004af508, {0xc00063d7a0, 0x12, 0x12}) jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000121e78 sp=0xc000121d58 pc=0x5653a26412dc jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).ExecuteC(0xc000644c08) jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000121f30 sp=0xc000121e78 pc=0x5653a2641b25 jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).Execute(...) jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra@v1.7.0/command.go:992 jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra.(*Command).ExecuteContext(...) jun 27 12:51:39 ollama[1148060]: github.com/spf13/cobra@v1.7.0/command.go:985 jun 27 12:51:39 ollama[1148060]: main.main() jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000121f50 sp=0xc000121f30 pc=0x5653a31170cd jun 27 12:51:39 ollama[1148060]: runtime.main() jun 27 12:51:39 ollama[1148060]: runtime/proc.go:283 +0x29d fp=0xc000121fe0 sp=0xc000121f50 pc=0x5653a2495ebd jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000121fe8 sp=0xc000121fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007cfa8 sp=0xc00007cf88 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:441 jun 27 12:51:39 ollama[1148060]: runtime.forcegchelper() jun 27 12:51:39 ollama[1148060]: runtime/proc.go:348 +0xb8 fp=0xc00007cfe0 sp=0xc00007cfa8 pc=0x5653a24961f8 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007cfe8 sp=0xc00007cfe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.init.7 in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/proc.go:336 +0x1a jun 27 12:51:39 ollama[1148060]: goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007d780 sp=0xc00007d760 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:441 jun 27 12:51:39 ollama[1148060]: runtime.bgsweep(0xc000046080) jun 27 12:51:39 ollama[1148060]: runtime/mgcsweep.go:316 +0xdf fp=0xc00007d7c8 sp=0xc00007d780 pc=0x5653a248099f jun 27 12:51:39 ollama[1148060]: runtime.gcenable.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:204 +0x25 fp=0xc00007d7e0 sp=0xc00007d7c8 pc=0x5653a2474d85 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007d7e8 sp=0xc00007d7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcenable in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:204 +0x66 jun 27 12:51:39 ollama[1148060]: goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x10000?, 0x5653a3530c38?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007df78 sp=0xc00007df58 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:441 jun 27 12:51:39 ollama[1148060]: runtime.(*scavengerState).park(0x5653a40b88e0) jun 27 12:51:39 ollama[1148060]: runtime/mgcscavenge.go:425 +0x49 fp=0xc00007dfa8 sp=0xc00007df78 pc=0x5653a247e3e9 jun 27 12:51:39 ollama[1148060]: runtime.bgscavenge(0xc000046080) jun 27 12:51:39 ollama[1148060]: runtime/mgcscavenge.go:658 +0x59 fp=0xc00007dfc8 sp=0xc00007dfa8 pc=0x5653a247e979 jun 27 12:51:39 ollama[1148060]: runtime.gcenable.gowrap2() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:205 +0x25 fp=0xc00007dfe0 sp=0xc00007dfc8 pc=0x5653a2474d25 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007dfe8 sp=0xc00007dfe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcenable in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:205 +0xa5 jun 27 12:51:39 ollama[1148060]: goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc00007c688?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007c630 sp=0xc00007c610 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.runfinq() jun 27 12:51:39 ollama[1148060]: runtime/mfinal.go:196 +0x107 fp=0xc00007c7e0 sp=0xc00007c630 pc=0x5653a2473d47 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007c7e8 sp=0xc00007c7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.createfing in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mfinal.go:166 +0x3d jun 27 12:51:39 ollama[1148060]: goroutine 6 gp=0xc0001d48c0 m=nil [chan receive]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0xc000233540?, 0xc000590018?, 0x60?, 0xe7?, 0x5653a25ae9c8?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007e718 sp=0xc00007e6f8 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.chanrecv(0xc000048460, 0x0, 0x1) jun 27 12:51:39 ollama[1148060]: runtime/chan.go:664 +0x445 fp=0xc00007e790 sp=0xc00007e718 pc=0x5653a2465725 jun 27 12:51:39 ollama[1148060]: runtime.chanrecv1(0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/chan.go:506 +0x12 fp=0xc00007e7b8 sp=0xc00007e790 pc=0x5653a24652b2 jun 27 12:51:39 ollama[1148060]: runtime.unique_runtime_registerUniqueMapCleanup.func2(...) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1796 jun 27 12:51:39 ollama[1148060]: runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1799 +0x2f fp=0xc00007e7e0 sp=0xc00007e7b8 pc=0x5653a2477f2f jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007e7e8 sp=0xc00007e7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by unique.runtime_registerUniqueMapCleanup in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1794 +0x85 jun 27 12:51:39 ollama[1148060]: goroutine 7 gp=0xc0001d4c40 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007ef38 sp=0xc00007ef18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00007efc8 sp=0xc00007ef38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00007efe0 sp=0xc00007efc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007efe8 sp=0xc00007efe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 8 gp=0xc0001d4e00 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007f738 sp=0xc00007f718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00007f7c8 sp=0xc00007f738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00007f7e0 sp=0xc00007f7c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007f7e8 sp=0xc00007f7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 9 gp=0xc0001d4fc0 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007ff38 sp=0xc00007ff18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00007ffc8 sp=0xc00007ff38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00007ffe0 sp=0xc00007ffc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007ffe8 sp=0xc00007ffe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 10 gp=0xc0001d5180 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000078738 sp=0xc000078718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc0000787c8 sp=0xc000078738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc0000787e0 sp=0xc0000787c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0000787e8 sp=0xc0000787e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 11 gp=0xc0001d5340 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636cc5706a?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000078f38 sp=0xc000078f18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc000078fc8 sp=0xc000078f38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc000078fe0 sp=0xc000078fc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000078fe8 sp=0xc000078fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 18 gp=0xc000504000 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a4167120?, 0x1?, 0x6b?, 0xd5?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00050a738 sp=0xc00050a718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00050a7c8 sp=0xc00050a738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00050a7e0 sp=0xc00050a7c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9caf3a?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000506738 sp=0xc000506718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc0005067c8 sp=0xc000506738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc0005067e0 sp=0xc0005067c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 35 gp=0xc000102540 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c3a07?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000506f38 sp=0xc000506f18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc000506fc8 sp=0xc000506f38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc000506fe0 sp=0xc000506fc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 12 gp=0xc0001d5500 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636cc62da8?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000079738 sp=0xc000079718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc0000797c8 sp=0xc000079738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc0000797e0 sp=0xc0000797c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc0000797e8 sp=0xc0000797e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 13 gp=0xc0001d56c0 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a4167120?, 0x1?, 0xc9?, 0xf8?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000079f38 sp=0xc000079f18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc000079fc8 sp=0xc000079f38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc000079fe0 sp=0xc000079fc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000079fe8 sp=0xc000079fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 14 gp=0xc0001d5880 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c36bb?, 0x0?, 0x0?, 0x0?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007a738 sp=0xc00007a718 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00007a7c8 sp=0xc00007a738 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00007a7e0 sp=0xc00007a7c8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007a7e8 sp=0xc00007a7e0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 15 gp=0xc0001d5a40 m=nil [GC worker (idle)]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x2e636c9c5df5?, 0x1?, 0x33?, 0xb6?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00007af38 sp=0xc00007af18 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkWorker(0xc000049a40) jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1423 +0xe9 fp=0xc00007afc8 sp=0xc00007af38 pc=0x5653a2477249 jun 27 12:51:39 ollama[1148060]: runtime.gcBgMarkStartWorkers.gowrap1() jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x25 fp=0xc00007afe0 sp=0xc00007afc8 pc=0x5653a2477125 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00007afe8 sp=0xc00007afe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by runtime.gcBgMarkStartWorkers in goroutine 1 jun 27 12:51:39 ollama[1148060]: runtime/mgc.go:1339 +0x105 jun 27 12:51:39 ollama[1148060]: goroutine 50 gp=0xc000102c40 m=nil [sync.WaitGroup.Wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x0?, 0x0?, 0x20?, 0x81?, 0x0?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc00050bed0 sp=0xc00050beb0 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.goparkunlock(...) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:441 jun 27 12:51:39 ollama[1148060]: runtime.semacquire1(0xc00063d9e8, 0x0, 0x1, 0x0, 0x18) jun 27 12:51:39 ollama[1148060]: runtime/sema.go:188 +0x229 fp=0xc00050bf38 sp=0xc00050bed0 pc=0x5653a24a9489 jun 27 12:51:39 ollama[1148060]: sync.runtime_SemacquireWaitGroup(0x0?) jun 27 12:51:39 ollama[1148060]: runtime/sema.go:110 +0x25 fp=0xc00050bf70 sp=0xc00050bf38 pc=0x5653a24cb765 jun 27 12:51:39 ollama[1148060]: sync.(*WaitGroup).Wait(0x0?) jun 27 12:51:39 ollama[1148060]: sync/waitgroup.go:118 +0x48 fp=0xc00050bf98 sp=0xc00050bf70 pc=0x5653a24dcdc8 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc00063d9e0, {0x5653a382a880, 0xc0005afae0}) jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc00050bfb8 sp=0xc00050bf98 pc=0x5653a29b3a85 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2() jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc00050bfe0 sp=0xc00050bfb8 pc=0x5653a29b9208 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 jun 27 12:51:39 ollama[1148060]: github.com/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74 jun 27 12:51:39 ollama[1148060]: goroutine 51 gp=0xc000103180 m=nil [IO wait]: jun 27 12:51:39 ollama[1148060]: runtime.gopark(0x5653a25539a5?, 0xc00048c280?, 0x40?, 0x3a?, 0xb?) jun 27 12:51:39 ollama[1148060]: runtime/proc.go:435 +0xce fp=0xc000123948 sp=0xc000123928 pc=0x5653a24c9d4e jun 27 12:51:39 ollama[1148060]: runtime.netpollblock(0x5653a24ed0b8?, 0xa2462b46?, 0x53?) jun 27 12:51:39 ollama[1148060]: runtime/netpoll.go:575 +0xf7 fp=0xc000123980 sp=0xc000123948 pc=0x5653a248e837 jun 27 12:51:39 ollama[1148060]: internal/poll.runtime_pollWait(0x7fbff94c6d98, 0x72) jun 27 12:51:39 ollama[1148060]: runtime/netpoll.go:351 +0x85 fp=0xc0001239a0 sp=0xc000123980 pc=0x5653a24c8f65 jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).wait(0xc00048c280?, 0xc001776000?, 0x0) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0001239c8 sp=0xc0001239a0 pc=0x5653a25503a7 jun 27 12:51:39 ollama[1148060]: internal/poll.(*pollDesc).waitRead(...) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_poll_runtime.go:89 jun 27 12:51:39 ollama[1148060]: internal/poll.(*FD).Read(0xc00048c280, {0xc001776000, 0x1000, 0x1000}) jun 27 12:51:39 ollama[1148060]: internal/poll/fd_unix.go:165 +0x27a fp=0xc000123a60 sp=0xc0001239c8 pc=0x5653a255169a jun 27 12:51:39 ollama[1148060]: net.(*netFD).Read(0xc00048c280, {0xc001776000?, 0xc000123ad0?, 0x5653a2550865?}) jun 27 12:51:39 ollama[1148060]: net/fd_posix.go:55 +0x25 fp=0xc000123aa8 sp=0xc000123a60 pc=0x5653a25c5ce5 jun 27 12:51:39 ollama[1148060]: net.(*conn).Read(0xc0005a6010, {0xc001776000?, 0x0?, 0x0?}) jun 27 12:51:39 ollama[1148060]: net/net.go:194 +0x45 fp=0xc000123af0 sp=0xc000123aa8 pc=0x5653a25d40a5 jun 27 12:51:39 ollama[1148060]: net/http.(*connReader).Read(0xc000226d50, {0xc001776000, 0x1000, 0x1000}) jun 27 12:51:39 ollama[1148060]: net/http/server.go:798 +0x159 fp=0xc000123b40 sp=0xc000123af0 pc=0x5653a27c03b9 jun 27 12:51:39 ollama[1148060]: bufio.(*Reader).fill(0xc000110060) jun 27 12:51:39 ollama[1148060]: bufio/bufio.go:113 +0x103 fp=0xc000123b78 sp=0xc000123b40 pc=0x5653a25eb843 jun 27 12:51:39 ollama[1148060]: bufio.(*Reader).Peek(0xc000110060, 0x4) jun 27 12:51:39 ollama[1148060]: bufio/bufio.go:152 +0x53 fp=0xc000123b98 sp=0xc000123b78 pc=0x5653a25eb973 jun 27 12:51:39 ollama[1148060]: net/http.(*conn).serve(0xc0004e43f0, {0x5653a382a848, 0xc00036a660}) jun 27 12:51:39 ollama[1148060]: net/http/server.go:2137 +0x785 fp=0xc000123fb8 sp=0xc000123b98 pc=0x5653a27c61a5 jun 27 12:51:39 ollama[1148060]: net/http.(*Server).Serve.gowrap3() jun 27 12:51:39 ollama[1148060]: net/http/server.go:3454 +0x28 fp=0xc000123fe0 sp=0xc000123fb8 pc=0x5653a27cb908 jun 27 12:51:39 ollama[1148060]: runtime.goexit({}) jun 27 12:51:39 ollama[1148060]: runtime/asm_amd64.s:1700 +0x1 fp=0xc000123fe8 sp=0xc000123fe0 pc=0x5653a24d1481 jun 27 12:51:39 ollama[1148060]: created by net/http.(*Server).Serve in goroutine 1 jun 27 12:51:39 ollama[1148060]: net/http/server.go:3454 +0x485 jun 27 12:51:39 ollama[1148060]: time=2025-06-27T12:51:39.799-03:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2" jun 27 12:51:40 ollama[1148060]: time=2025-06-27T12:51:39.999-03:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault" ``` </details> PS: I tried the same without these enviroment variables just to be sure they are not affecting anything, but got the same error: ``` Environment="OLLAMA_NUM_PARALLEL=1" Environment="OLLAMA_MAX_LOADED_MODELS=2" Environment="OLLAMA_FLASH_ATTENTION=1" Environment="OLLAMA_KV_CACHE_TYPE=q8_0" Environment="GGML_CUDA_ENABLE_UNIFIED_MEMORY=1" ``` Please feel free to ask for more info or tests. And thanks to all the Ollama dev team.

GiteaMirror commented

2026-04-12 19:27:58 -05:00

@dhiltgen commented on GitHub (Jun 27, 2025):

We're trying to isolate if this is linux specific, or includes windows. Could anyone who thinks they hit the same crash on Windows share their server logs showing the crash so we can see if it lines up?

@dhiltgen commented on GitHub (Jun 27, 2025): We're trying to isolate if this is linux specific, or includes windows. Could anyone who thinks they hit the same crash on Windows share their server logs showing the crash so we can see if it lines up?

GiteaMirror commented

2026-04-12 19:27:59 -05:00

@Bzucko commented on GitHub (Jun 27, 2025):

@dhiltgen server log from windows

server.log

@Bzucko commented on GitHub (Jun 27, 2025): @dhiltgen server log from windows [server.log](https://github.com/user-attachments/files/20953991/server.log)

GiteaMirror commented

2026-04-12 19:28:00 -05:00

@YoMamasNerd commented on GitHub (Jun 27, 2025):

We're trying to isolate if this is linux specific, or includes windows. Could anyone who thinks they hit the same crash on Windows share their server logs showing the crash so we can see if it lines up?

I would spin up a lxc with ssh access for you if this helps you figure out the issue.

@YoMamasNerd commented on GitHub (Jun 27, 2025): > We're trying to isolate if this is linux specific, or includes windows. Could anyone who thinks they hit the same crash on Windows share their server logs showing the crash so we can see if it lines up? I would spin up a lxc with ssh access for you if this helps you figure out the issue.

GiteaMirror commented

2026-04-12 19:28:02 -05:00

@jessegross commented on GitHub (Jun 27, 2025):

@YoMamasNerd Yes, that would be awesome if you could do that.

@jessegross commented on GitHub (Jun 27, 2025): @YoMamasNerd Yes, that would be awesome if you could do that.

GiteaMirror commented

2026-04-12 19:28:03 -05:00

@YoMamasNerd commented on GitHub (Jun 27, 2025):

@jessegross
Are you jesse_ollama on discord?

@YoMamasNerd commented on GitHub (Jun 27, 2025): @jessegross Are you jesse_ollama on discord?

GiteaMirror commented

2026-04-12 19:28:03 -05:00

@jessegross commented on GitHub (Jun 27, 2025):

@YoMamasNerd Yes

@jessegross commented on GitHub (Jun 27, 2025): @YoMamasNerd Yes

GiteaMirror commented

2026-04-12 19:28:05 -05:00

@CRCODE22 commented on GitHub (Jun 27, 2025):

Since the latest Ollama update models no longer load on GPU I am getting the same llama runner process has terminated: error:fault I have tried the gemma3n:e2b-it-q8_0 and qwen2.5vl:7b-q8_0. The qwen2.5vl:7b-q8_0 used to work on older versions of Ollama. I have Windows 11 and 16GB of VRAM

@CRCODE22 commented on GitHub (Jun 27, 2025): Since the latest Ollama update models no longer load on GPU I am getting the same llama runner process has terminated: error:fault I have tried the gemma3n:e2b-it-q8_0 and qwen2.5vl:7b-q8_0. The qwen2.5vl:7b-q8_0 used to work on older versions of Ollama. I have Windows 11 and 16GB of VRAM

GiteaMirror commented

2026-04-12 19:28:06 -05:00

@CRCODE22 commented on GitHub (Jun 27, 2025):

time=2025-06-27T23:21:28.012+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:56803"
time=2025-06-27T23:21:28.060+02:00 level=INFO source=ggml.go:92 msg="" architecture=qwen25vl file_type=Q8_0 name="" description="" num_tensors=858 num_key_values=36
time=2025-06-27T23:21:28.196+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model"
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes
load_backend: loaded CUDA backend from C:\Users\CRCODE22\AppData\Local\Programs\Ollama\lib\ollama\ggml-cuda.dll
load_backend: loaded CPU backend from C:\Users\CRCODE22\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-haswell.dll
time=2025-06-27T23:21:28.229+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2025-06-27T23:21:28.440+02:00 level=INFO source=ggml.go:359 msg="model weights" buffer=CPU size="552.2 MiB"
time=2025-06-27T23:21:28.440+02:00 level=INFO source=ggml.go:359 msg="model weights" buffer=CUDA0 size="8.2 GiB"
time=2025-06-27T23:21:28.785+02:00 level=INFO source=ggml.go:648 msg="compute graph" backend=CUDA0 buffer_type=CUDA0 size="1.7 GiB"
time=2025-06-27T23:21:28.785+02:00 level=INFO source=ggml.go:648 msg="compute graph" backend=CPU buffer_type=CPU size="0 B"
ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed
time=2025-06-27T23:21:28.964+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error"
time=2025-06-27T23:21:29.030+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 0xc0000409"
time=2025-06-27T23:21:29.214+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed"
[GIN] 2025/06/27 - 23:21:29 | 500 | 3.6056702s | 127.0.0.1 | POST "/api/chat"
time=2025-06-27T23:22:56.832+02:00 level=INFO source=download.go:177 msg="downloading e62aabdaf381 in 1 414 B part(s)"
[GIN] 2025/06/27 - 23:24:47 | 200 | 4m43s | 127.0.0.1 | POST "/api/pull"

@CRCODE22 commented on GitHub (Jun 27, 2025): time=2025-06-27T23:21:28.012+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:56803" time=2025-06-27T23:21:28.060+02:00 level=INFO source=ggml.go:92 msg="" architecture=qwen25vl file_type=Q8_0 name="" description="" num_tensors=858 num_key_values=36 time=2025-06-27T23:21:28.196+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model" ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4060 Ti, compute capability 8.9, VMM: yes load_backend: loaded CUDA backend from C:\Users\CRCODE22\AppData\Local\Programs\Ollama\lib\ollama\ggml-cuda.dll load_backend: loaded CPU backend from C:\Users\CRCODE22\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-haswell.dll time=2025-06-27T23:21:28.229+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2025-06-27T23:21:28.440+02:00 level=INFO source=ggml.go:359 msg="model weights" buffer=CPU size="552.2 MiB" time=2025-06-27T23:21:28.440+02:00 level=INFO source=ggml.go:359 msg="model weights" buffer=CUDA0 size="8.2 GiB" time=2025-06-27T23:21:28.785+02:00 level=INFO source=ggml.go:648 msg="compute graph" backend=CUDA0 buffer_type=CUDA0 size="1.7 GiB" time=2025-06-27T23:21:28.785+02:00 level=INFO source=ggml.go:648 msg="compute graph" backend=CPU buffer_type=CPU size="0 B" ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed time=2025-06-27T23:21:28.964+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error" time=2025-06-27T23:21:29.030+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 0xc0000409" time=2025-06-27T23:21:29.214+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed" [GIN] 2025/06/27 - 23:21:29 | 500 | 3.6056702s | 127.0.0.1 | POST "/api/chat" time=2025-06-27T23:22:56.832+02:00 level=INFO source=download.go:177 msg="downloading e62aabdaf381 in 1 414 B part(s)" [GIN] 2025/06/27 - 23:24:47 | 200 | 4m43s | 127.0.0.1 | POST "/api/pull"

GiteaMirror commented

2026-04-12 19:28:07 -05:00

@jessegross commented on GitHub (Jun 27, 2025):

@CRCODE22 That does not look related, please file a separate bug.

@jessegross commented on GitHub (Jun 27, 2025): @CRCODE22 That does not look related, please file a separate bug.

GiteaMirror commented

2026-04-12 19:28:08 -05:00

@CRCODE22 commented on GitHub (Jun 27, 2025):

@CRCODE22 That does not look related, please file a separate bug.

How is this "time=2025-06-27T23:21:29.030+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 0xc0000409" Not related since multiple users have mentioned it in this issue?

@CRCODE22 commented on GitHub (Jun 27, 2025): > [@CRCODE22](https://github.com/CRCODE22) That does not look related, please file a separate bug. How is this "time=2025-06-27T23:21:29.030+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 0xc0000409" Not related since multiple users have mentioned it in this issue?

GiteaMirror commented

2026-04-12 19:28:09 -05:00

@CRCODE22 commented on GitHub (Jun 27, 2025):

@jessegross see? Need more examples?

@CRCODE22 commented on GitHub (Jun 27, 2025): ![Image](https://github.com/user-attachments/assets/2e86ca83-6327-423a-8968-a23d5e675d8c) @jessegross see? Need more examples?

GiteaMirror commented

2026-04-12 19:28:10 -05:00

@jessegross commented on GitHub (Jun 27, 2025):

It just means that it crashed. Crashes can have multiple causes. The actual trigger:

Original:
Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]

Yours:
ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed

@jessegross commented on GitHub (Jun 27, 2025): It just means that it crashed. Crashes can have multiple causes. The actual trigger: Original: `Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]` Yours: `ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed`

GiteaMirror commented

2026-04-12 19:28:11 -05:00

@CRCODE22 commented on GitHub (Jun 27, 2025):

It just means that it crashed. Crashes can have multiple causes. The actual trigger:

Original: Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]

Yours: ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed

The original is from openwebui log I posted directly from the ollama server log it is the same problem. The error is also shown in OpenWebUI as I also was using both when the error occured.

@CRCODE22 commented on GitHub (Jun 27, 2025): > It just means that it crashed. Crashes can have multiple causes. The actual trigger: > > Original: `Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]` > > Yours: `ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed` The original is from openwebui log I posted directly from the ollama server log it is the same problem. The error is also shown in OpenWebUI as I also was using both when the error occured.

GiteaMirror commented

2026-04-12 19:28:12 -05:00

@mfranzon commented on GitHub (Jun 27, 2025):

unexpected fault address 0x3dda30000
fatal error: fault
...
[signal SIGSEGV: segmentation violation code=0x1 addr=0x3dda30000 pc=0x557d91572780]

This error here with all gemma models. Linux (ubuntu) + 2xTesla V100-16GB

@mfranzon commented on GitHub (Jun 27, 2025): ``` unexpected fault address 0x3dda30000 fatal error: fault ... [signal SIGSEGV: segmentation violation code=0x1 addr=0x3dda30000 pc=0x557d91572780] ``` This error here with all `gemma` models. Linux (ubuntu) + 2xTesla V100-16GB

GiteaMirror commented

2026-04-12 19:28:12 -05:00

@dhiltgen commented on GitHub (Jun 27, 2025):

It looks like the root cause of this failure may be users who are manually installing Ollama using either the tar files on linux, or zip file on windows, and overlay the new version directly on top of the old version without cleaning up the old version first. The install.sh (linux) and OllamaSetup.exe handle this cleanup automatically, but when manually installing, you'll have to make sure the new location is clean and files from the old version aren't lingering. The failure occurs because we moved some libraries around in the recent release, and the old versions are still there at the old path and getting picked up as well but aren't binary compatible.

This requirement for manual installs is described here:

@dhiltgen commented on GitHub (Jun 27, 2025): It looks like the root cause of this failure may be users who are manually installing Ollama using either the tar files on linux, or zip file on windows, and overlay the new version directly on top of the old version without cleaning up the old version first. The install.sh (linux) and OllamaSetup.exe handle this cleanup automatically, but when manually installing, you'll have to make sure the new location is clean and files from the old version aren't lingering. The failure occurs because we moved some libraries around in the recent release, and the old versions are still there at the old path and getting picked up as well but aren't binary compatible. This requirement for manual installs is described here: - https://github.com/ollama/ollama/blob/main/docs/linux.md#manual-install - https://github.com/ollama/ollama/blob/main/docs/windows.md#standalone-cli

GiteaMirror commented

2026-04-12 19:28:13 -05:00

@YoMamasNerd commented on GitHub (Jun 27, 2025):

It just means that it crashed. Crashes can have multiple causes. The actual trigger:

Original: Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]

Yours: ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed

The original is from openwebui log I posted directly from the ollama server log it is the same problem. The error is also shown in OpenWebUI as I also was using both when the error occured.

@CRCODE22
openwebui is just the name of the server. The log is directly from ollama

@YoMamasNerd commented on GitHub (Jun 27, 2025): > > It just means that it crashed. Crashes can have multiple causes. The actual trigger: > > > > Original: `Jun 26 23:09:12 openwebui ollama[167]: [signal SIGSEGV: segmentation violation code=0x2 addr=0x2e2300000 pc=0x652733403780]` > > > > Yours: `ggml.c:1554: GGML_ASSERT(view_src == NULL || data_size == 0 || data_size + view_offs <= ggml_nbytes(view_src)) failed` > > The original is from openwebui log I posted directly from the ollama server log it is the same problem. The error is also shown in OpenWebUI as I also was using both when the error occured. @CRCODE22 openwebui is just the name of the server. The log is directly from ollama

GiteaMirror commented

2026-04-12 19:28:13 -05:00

@dhiltgen commented on GitHub (Jun 27, 2025):

We've repro'd the failure by installing 0.9.2 and then extracting the 0.9.3 tar file on top, so I think we're fairly confident that's the root cause at this point.

Folks who are hitting this, please try 0.9.3 again, and before extracting the tgz/zip, make sure the target directory is cleaned up.

@dhiltgen commented on GitHub (Jun 27, 2025): We've repro'd the failure by installing 0.9.2 and then extracting the 0.9.3 tar file on top, so I think we're fairly confident that's the root cause at this point. Folks who are hitting this, please try 0.9.3 again, and before extracting the tgz/zip, make sure the target directory is cleaned up.

GiteaMirror commented

2026-04-12 19:28:14 -05:00

@mfranzon commented on GitHub (Jun 27, 2025):

We've repro'd the failure by installing 0.9.2 and then extracting the 0.9.3 tar file on top, so I think we're fairly confident that's the root cause at this point.

Folks who are hitting this, please try 0.9.3 again, and before extracting the tgz/zip, make sure the target directory is cleaned up.

The full cleanup solve the issue for me. Thanks

@mfranzon commented on GitHub (Jun 27, 2025): > We've repro'd the failure by installing 0.9.2 and then extracting the 0.9.3 tar file on top, so I think we're fairly confident that's the root cause at this point. > > Folks who are hitting this, please try 0.9.3 again, and before extracting the tgz/zip, make sure the target directory is cleaned up. The full cleanup solve the issue for me. Thanks

GiteaMirror commented

2026-04-12 19:28:15 -05:00

@CRCODE22 commented on GitHub (Jun 28, 2025):

I have not installed ollama manually I used the setup or the update feature within ollama when it notifies me there is an update.

@CRCODE22 commented on GitHub (Jun 28, 2025): I have not installed ollama manually I used the setup or the update feature within ollama when it notifies me there is an update.

GiteaMirror commented

2026-04-12 19:28:15 -05:00

@johny-mnemonic commented on GitHub (Jun 28, 2025):

@CRCODE22 same here. I also installed using standard installer. But I can confirm that complete uninstall and then install of the latest version fixed the crashing for me.

@johny-mnemonic commented on GitHub (Jun 28, 2025): @CRCODE22 same here. I also installed using standard installer. But I can confirm that complete uninstall and then install of the latest version fixed the crashing for me.

GiteaMirror commented

2026-04-12 19:28:15 -05:00

@jclauzel commented on GitHub (Jun 29, 2025):

+1 same here trying to run llama4:maverick uninstall and reinstall fixed the issue. Thank you!

@jclauzel commented on GitHub (Jun 29, 2025): +1 same here trying to run llama4:maverick uninstall and reinstall fixed the issue. Thank you!

GiteaMirror commented

2026-04-12 19:28:17 -05:00

@serge-medvedev commented on GitHub (Jul 4, 2025):

I run Ollama in Docker and gemma3 is not working with the latest (0.9.5) version.

@serge-medvedev commented on GitHub (Jul 4, 2025): I run Ollama in Docker and gemma3 is not working with the latest (0.9.5) version.

GiteaMirror commented

2026-04-12 19:28:18 -05:00

@LFd3v commented on GitHub (Jul 4, 2025):

I run Ollama in Docker and gemma3 is not working with the latest (0.9.5) version.

What OS and how Ollama was installed? This issue was closed already, the problem was the installation keeping old files (specially in the Ollama lib directory) and not a bug in Ollama code that runs models. Try to remove the old installation, or at least the old Ollama lib folder and install again. Here this fixed the problem with v0.9.3 and Gemma3 models still run "fine" (the split between VRAM and RAM is bad, but I guess this happens due to the model architecture and how Ollama run some vision models, unfortunately) with v0.9.5.

As you are using Docker, you may want to check #11220, which is also closed now. Or simply try a normal installation, manual or not.

@LFd3v commented on GitHub (Jul 4, 2025): > I run Ollama in Docker and gemma3 is not working with the latest (0.9.5) version. What OS and how Ollama was installed? This issue was closed already, the problem was the installation keeping old files (specially in the Ollama lib directory) and _not_ a bug in Ollama code that runs models. Try to remove the old installation, or at least the old Ollama lib folder and install again. Here this fixed the problem with v0.9.3 and Gemma3 models still run "fine" (the split between VRAM and RAM is bad, but I guess this happens due to the model architecture and how Ollama run some vision models, unfortunately) with v0.9.5. As you are using Docker, you may want to check #11220, which is also closed now. Or simply try a normal installation, manual or not.

GiteaMirror commented

2026-04-12 19:28:18 -05:00

@soda-pop-ice-cream commented on GitHub (Jul 30, 2025):

sudo rm -rf /usr/lib/ollama/
sudo tar -C /usr -xzf ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz

Solved this issue for me. I have Ryzen 7 5700X & RX 9070 XT

(This doesn't remove models since they are stored in /usr/share/)

@soda-pop-ice-cream commented on GitHub (Jul 30, 2025): ``` sudo rm -rf /usr/lib/ollama/ sudo tar -C /usr -xzf ollama-linux-amd64.tgz sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz ``` Solved this issue for me. I have Ryzen 7 5700X & RX 9070 XT (This doesn't remove models since they are stored in /usr/share/)

GiteaMirror referenced this issue

2026-04-12 23:58:24 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #12405

GiteaMirror referenced this issue

2026-04-16 06:10:41 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #17676

GiteaMirror referenced this issue

2026-04-19 16:40:27 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #22945

GiteaMirror referenced this issue

2026-04-22 10:03:36 -05:00

[GH-ISSUE #7362] Llama3.2-vision image processing not implemented for /generate #30438

GiteaMirror referenced this issue

2026-04-22 10:04:44 -05:00

[GH-ISSUE #7388] Llama3.2-vision - fails to process png files #30456

GiteaMirror referenced this issue

2026-04-22 22:57:07 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #38278

GiteaMirror referenced this issue

2026-04-24 23:15:35 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #43653

GiteaMirror referenced this issue

2026-04-28 18:53:54 -05:00

[GH-ISSUE #7362] Llama3.2-vision image processing not implemented for /generate #51189

GiteaMirror referenced this issue

2026-04-28 18:55:10 -05:00

[GH-ISSUE #7388] Llama3.2-vision - fails to process png files #51207

GiteaMirror referenced this issue

2026-04-29 13:58:48 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #59102

GiteaMirror referenced this issue

2026-05-04 08:00:44 -05:00

[GH-ISSUE #7362] Llama3.2-vision image processing not implemented for /generate #66734

GiteaMirror referenced this issue

2026-05-04 08:05:31 -05:00

[GH-ISSUE #7388] Llama3.2-vision - fails to process png files #66752

GiteaMirror referenced this issue

2026-05-05 06:57:05 -05:00

[PR #7384] [MERGED] add mllama image processing to the generate handler #74699

Sign in to join this conversation.

Branches Tags

main

dhiltgen/ci

hoyyeva/editor-config-repair

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

hoyyeva/launch-backup-ux

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-mlx-decode-checkpoints

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

brucemacd/download-before-remove

parth/update-claude-docs

parth-anthropic-reference-images-path

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#7384