[GH-ISSUE #11598] ollama run gemma3:12b error #7656

Closed
opened 2026-04-12 19:44:50 -05:00 by GiteaMirror · 2 comments
Owner

Originally created by @x2611 on GitHub (Jul 31, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11598

What is the issue?

Gemma3:12b ran fine with v0.9.2 and earlier, on both of my PC's. Nothing changed on either PC.
Now Ollama fails to run it with:

Windows Terminal:
Error: llama runner process has terminated: error:fault

Chatbox:
API Error: Error from Ollama: Error: API Error: Status Code 500, {"error":{"message":"llama runner process has terminated: error:fault","type":"api_error","param":null,"code":null}}

Deepseek-r1:8b still runs fine on both machines:
i5-10600KF, 16GB, GTX1070 8GB, Win10
i7-6700K, 32GB, GTX1080Ti 11Gb, Win10

Using Gemma3:12b since it became available. I downgraded Ollama to v0.9.2 to test it and Gemma ran fine again:

https://postimg.cc/TL67WTNc
https://postimg.cc/BL4pnMtY

Ollama updates itself unfortunately so the bug returns. Automatic updates should be turn off-able.

Relevant log output

time=2025-07-31T09:05:09.629+02:00 level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:E:\\Programs\\ai\\ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]"
time=2025-07-31T09:05:09.631+02:00 level=INFO source=images.go:476 msg="total blobs: 10"
time=2025-07-31T09:05:09.631+02:00 level=INFO source=images.go:483 msg="total unused blobs removed: 0"
time=2025-07-31T09:05:09.632+02:00 level=INFO source=routes.go:1288 msg="Listening on 127.0.0.1:11434 (version 0.9.6)"
time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu_windows.go:167 msg=packages count=1
time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=6 efficiency=0 threads=12
time=2025-07-31T09:05:09.746+02:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-76fa6921-4f55-c156-f1d9-8f257f3fba18 library=cuda variant=v12 compute=6.1 driver=12.7 name="NVIDIA GeForce GTX 1070" total="8.0 GiB" available="7.0 GiB"
[GIN] 2025/07/31 - 09:05:10 | 200 |            0s |       127.0.0.1 | HEAD     "/"
[GIN] 2025/07/31 - 09:05:10 | 200 |    122.0154ms |       127.0.0.1 | POST     "/api/show"
time=2025-07-31T09:05:10.368+02:00 level=INFO source=server.go:135 msg="system memory" total="15.9 GiB" free="9.6 GiB" free_swap="8.2 GiB"
time=2025-07-31T09:05:10.369+02:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=49 layers.offload=27 layers.split="" memory.available="[5.7 GiB]" memory.gpu_overhead="0 B" memory.required.full="11.2 GiB" memory.required.partial="5.7 GiB" memory.required.kv="736.0 MiB" memory.required.allocations="[5.7 GiB]" memory.weights.total="6.8 GiB" memory.weights.repeating="6.0 GiB" memory.weights.nonrepeating="787.5 MiB" memory.graph.full="519.5 MiB" memory.graph.partial="1.3 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB"
time=2025-07-31T09:05:10.445+02:00 level=INFO source=server.go:438 msg="starting llama server" cmd="E:\\Programs\\ai\\ollama\\ollama.exe runner --ollama-engine --model E:\\Programs\\ai\\ollama\\models\\blobs\\sha256-e8ad13eff07a78d89926e9e8b882317d082ef5bf9768ad7b50fcdbbcd63748de --ctx-size 4096 --batch-size 512 --n-gpu-layers 27 --threads 6 --no-mmap --parallel 1 --port 55114"
time=2025-07-31T09:05:10.446+02:00 level=INFO source=sched.go:483 msg="loaded runners" count=1
time=2025-07-31T09:05:10.446+02:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding"
time=2025-07-31T09:05:10.447+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error"
time=2025-07-31T09:05:10.484+02:00 level=INFO source=runner.go:925 msg="starting ollama engine"
time=2025-07-31T09:05:10.489+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:55114"
time=2025-07-31T09:05:10.549+02:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1065 num_key_values=37
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1, VMM: yes
load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\ggml-cuda.dll
load_backend: loaded CPU backend from E:\Programs\ai\ollama\lib\ollama\ggml-cpu-haswell.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1, VMM: yes
load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\cuda_v12\ggml-cuda.dll
time=2025-07-31T09:05:10.648+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2025-07-31T09:05:10.698+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model"
unexpected fault address 0x1c3100000
fatal error: fault
[signal 0xc0000005 code=0x0 addr=0x1c3100000 pc=0x7ff676547180]

goroutine 10 gp=0xc000484c40 m=7 mp=0xc000510008 [running]:
runtime.throw({0x7ff6777977d7?, 0xc000056508?})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/panic.go:1096 +0x4d fp=0xc00004b0d0 sp=0xc00004b0a0 pc=0x7ff6765b61ed
runtime.sigpanic()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/signal_windows.go:414 +0xd0 fp=0xc00004b118 sp=0xc00004b0d0 pc=0x7ff676598f10
indexbytebody()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc00004b120 sp=0xc00004b118 pc=0x7ff676547180
runtime.findnull(0xc00004b1a0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/string.go:577 +0x79 fp=0xc00004b178 sp=0xc00004b120 pc=0x7ff67659e039
runtime.gostring(0x1c3100000)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/string.go:363 +0x1c fp=0xc00004b1b0 sp=0xc00004b178 pc=0x7ff6765b8d5c
github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...)
	_cgo_gotypes.go:299
github.com/ollama/ollama/ml/backend/ggml.New({0xc00003e150, 0x6a}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0})
	C:/a/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1359 fp=0xc00004bc18 sp=0xc00004b1b0 pc=0x7ff676a08499
github.com/ollama/ollama/ml.NewBackend({0xc00003e150, 0x6a}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0})
	C:/a/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc00004bc70 sp=0xc00004bc18 pc=0x7ff6769f9091
github.com/ollama/ollama/model.New({0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0})
	C:/a/ollama/ollama/model/model.go:102 +0x8f fp=0xc00004bd68 sp=0xc00004bc70 pc=0x7ff676a17f6f
github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc0005ab7a0, {0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0}, ...)
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc00004bdc8 sp=0xc00004bd68 pc=0x7ff676ab9fed
github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc0005ab7a0, {0x7ff677973510, 0xc0003499a0}, {0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, ...}, ...}, ...)
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc00004bf20 sp=0xc00004bdc8 pc=0x7ff676aba358
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc00004bfe0 sp=0xc00004bf20 pc=0x7ff676abb787
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00004bfe8 sp=0xc00004bfe0 pc=0x7ff6765bdb01
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11

goroutine 1 gp=0xc0000021c0 m=nil [IO wait]:
runtime.gopark(0x7ff6765bf300?, 0x7ff6782c5500?, 0x20?, 0x20?, 0xc0004820cc?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00179b508 sp=0xc00179b4e8 pc=0x7ff6765b630e
runtime.netpollblock(0x278?, 0x76550446?, 0xf6?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:575 +0xf7 fp=0xc00179b540 sp=0xc00179b508 pc=0x7ff67657bf37
internal/poll.runtime_pollWait(0x1caefbd4e00, 0x72)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:351 +0x85 fp=0xc00179b560 sp=0xc00179b540 pc=0x7ff6765b54a5
internal/poll.(*pollDesc).wait(0x7ff67664a293?, 0x7ff676562776?, 0x0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00179b588 sp=0xc00179b560 pc=0x7ff67664b887
internal/poll.execIO(0xc000482020, 0xc00179b630)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:177 +0x105 fp=0xc00179b600 sp=0xc00179b588 pc=0x7ff67664cce5
internal/poll.(*FD).acceptOne(0xc000482008, 0x2f0, {0xc0000e80f0?, 0x7ff67656ae05?, 0x7ff67833a6e0?}, 0x25b0aaa?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:946 +0x65 fp=0xc00179b660 sp=0xc00179b600 pc=0x7ff676651265
internal/poll.(*FD).Accept(0xc000482008, 0xc00179b810)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:980 +0x1b6 fp=0xc00179b718 sp=0xc00179b660 pc=0x7ff676651596
net.(*netFD).accept(0xc000482008)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/fd_windows.go:182 +0x4b fp=0xc00179b830 sp=0xc00179b718 pc=0x7ff6766c20ab
net.(*TCPListener).accept(0xc00008a000)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/tcpsock_posix.go:159 +0x1b fp=0xc00179b880 sp=0xc00179b830 pc=0x7ff6766d80fb
net.(*TCPListener).Accept(0xc00008a000)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/tcpsock.go:380 +0x30 fp=0xc00179b8b0 sp=0xc00179b880 pc=0x7ff6766d6eb0
net/http.(*onceCloseListener).Accept(0xc0000d1ef0?)
	<autogenerated>:1 +0x24 fp=0xc00179b8c8 sp=0xc00179b8b0 pc=0x7ff6768f02a4
net/http.(*Server).Serve(0xc0005c2000, {0x7ff677970f20, 0xc00008a000})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3424 +0x30c fp=0xc00179b9f8 sp=0xc00179b8c8 pc=0x7ff6768c7b6c
github.com/ollama/ollama/runner/ollamarunner.Execute({0xc0000a8030, 0xf, 0x1d})
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc00179bd08 sp=0xc00179b9f8 pc=0x7ff676abb3e9
github.com/ollama/ollama/runner.Execute({0xc0000a8010?, 0x0?, 0x0?})
	C:/a/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc00179bd30 sp=0xc00179bd08 pc=0x7ff676abbce9
github.com/ollama/ollama/cmd.NewCLI.func2(0xc0005c2f00?, {0x7ff677796384?, 0x4?, 0x7ff677796388?})
	C:/a/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc00179bd58 sp=0xc00179bd30 pc=0x7ff67721a765
github.com/spf13/cobra.(*Command).execute(0xc00020cf08, {0xc0005c3100, 0x10, 0x10})
	C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc00179be78 sp=0xc00179bd58 pc=0x7ff67673cb7c
github.com/spf13/cobra.(*Command).ExecuteC(0xc000466f08)
	C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc00179bf30 sp=0xc00179be78 pc=0x7ff67673d3c5
github.com/spf13/cobra.(*Command).Execute(...)
	C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	C:/a/ollama/ollama/main.go:12 +0x4d fp=0xc00179bf50 sp=0xc00179bf30 pc=0x7ff67721b1ed
runtime.main()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:283 +0x27d fp=0xc00179bfe0 sp=0xc00179bf50 pc=0x7ff676584f1d
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00179bfe8 sp=0xc00179bfe0 pc=0x7ff6765bdb01

goroutine 2 gp=0xc0000028c0 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000071fa8 sp=0xc000071f88 pc=0x7ff6765b630e
runtime.goparkunlock(...)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441
runtime.forcegchelper()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:348 +0xb8 fp=0xc000071fe0 sp=0xc000071fa8 pc=0x7ff676585238
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x7ff6765bdb01
created by runtime.init.7 in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:336 +0x1a

goroutine 3 gp=0xc000002c40 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000073f80 sp=0xc000073f60 pc=0x7ff6765b630e
runtime.goparkunlock(...)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441
runtime.bgsweep(0xc000080000)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcsweep.go:316 +0xdf fp=0xc000073fc8 sp=0xc000073f80 pc=0x7ff67656dfff
runtime.gcenable.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:204 +0x25 fp=0xc000073fe0 sp=0xc000073fc8 pc=0x7ff6765623c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000073fe8 sp=0xc000073fe0 pc=0x7ff6765bdb01
created by runtime.gcenable in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:204 +0x66

goroutine 4 gp=0xc000002e00 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x7ff67795e160?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000087f78 sp=0xc000087f58 pc=0x7ff6765b630e
runtime.goparkunlock(...)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441
runtime.(*scavengerState).park(0x7ff6782ebc00)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000087fa8 sp=0xc000087f78 pc=0x7ff67656ba49
runtime.bgscavenge(0xc000080000)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000087fc8 sp=0xc000087fa8 pc=0x7ff67656bfd9
runtime.gcenable.gowrap2()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:205 +0x25 fp=0xc000087fe0 sp=0xc000087fc8 pc=0x7ff676562365
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000087fe8 sp=0xc000087fe0 pc=0x7ff6765bdb01
created by runtime.gcenable in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:205 +0xa5

goroutine 5 gp=0xc000003340 m=nil [finalizer wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000089e30 sp=0xc000089e10 pc=0x7ff6765b630e
runtime.runfinq()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mfinal.go:196 +0x107 fp=0xc000089fe0 sp=0xc000089e30 pc=0x7ff676561347
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000089fe8 sp=0xc000089fe0 pc=0x7ff6765bdb01
created by runtime.createfing in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mfinal.go:166 +0x3d

goroutine 6 gp=0xc000003dc0 m=nil [chan receive]:
runtime.gopark(0xc00016f860?, 0xc000588018?, 0x60?, 0x5f?, 0x7ff6766ab0e8?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000075f18 sp=0xc000075ef8 pc=0x7ff6765b630e
runtime.chanrecv(0xc00003e3f0, 0x0, 0x1)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/chan.go:664 +0x445 fp=0xc000075f90 sp=0xc000075f18 pc=0x7ff676552d85
runtime.chanrecv1(0x7ff676585080?, 0xc000075f76?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/chan.go:506 +0x12 fp=0xc000075fb8 sp=0xc000075f90 pc=0x7ff676552912
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1799 +0x2f fp=0xc000075fe0 sp=0xc000075fb8 pc=0x7ff6765655ef
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0x7ff6765bdb01
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1794 +0x85

goroutine 7 gp=0xc0003e8380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000083f38 sp=0xc000083f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000083fc8 sp=0xc000083f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 18 gp=0xc0002081c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000213f38 sp=0xc000213f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000213fc8 sp=0xc000213f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000213fe0 sp=0xc000213fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000213fe8 sp=0xc000213fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 19 gp=0xc000208380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000215f38 sp=0xc000215f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000215fc8 sp=0xc000215f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000215fe0 sp=0xc000215fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000215fe8 sp=0xc000215fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 20 gp=0xc000208540 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d9da674?, 0x1?, 0x4c?, 0x40?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00020ff38 sp=0xc00020ff18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00020ffc8 sp=0xc00020ff38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00020ffe0 sp=0xc00020ffc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00020ffe8 sp=0xc00020ffe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 21 gp=0xc000208700 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d9da674?, 0x3?, 0x4c?, 0x40?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000211f38 sp=0xc000211f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000211fc8 sp=0xc000211f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000211fe0 sp=0xc000211fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000211fe8 sp=0xc000211fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 34 gp=0xc000484000 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d9da674?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00048bf38 sp=0xc00048bf18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00048bfc8 sp=0xc00048bf38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00048bfe0 sp=0xc00048bfc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00048bfe8 sp=0xc00048bfe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 8 gp=0xc0003e8540 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d85c07c?, 0x3?, 0x4c?, 0x40?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000085f38 sp=0xc000085f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000085fc8 sp=0xc000085f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 22 gp=0xc0002088c0 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d85c07c?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000487f38 sp=0xc000487f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000487fc8 sp=0xc000487f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000487fe0 sp=0xc000487fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 35 gp=0xc0004841c0 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d768030?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00048df38 sp=0xc00048df18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00048dfc8 sp=0xc00048df38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00048dfe0 sp=0xc00048dfc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 9 gp=0xc0003e8700 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d9da674?, 0x3?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000505f38 sp=0xc000505f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000505fc8 sp=0xc000505f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000505fe0 sp=0xc000505fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 23 gp=0xc000208a80 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d768030?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000489f38 sp=0xc000489f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000489fc8 sp=0xc000489f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000489fe0 sp=0xc000489fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 36 gp=0xc000484380 m=nil [GC worker (idle)]:
runtime.gopark(0xe5b71d768030?, 0x0?, 0x0?, 0x0?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000501f38 sp=0xc000501f18 pc=0x7ff6765b630e
runtime.gcBgMarkWorker(0xc00003f9d0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000501fc8 sp=0xc000501f38 pc=0x7ff6765648e9
runtime.gcBgMarkStartWorkers.gowrap1()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000501fe0 sp=0xc000501fc8 pc=0x7ff6765647c5
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000501fe8 sp=0xc000501fe0 pc=0x7ff6765bdb01
created by runtime.gcBgMarkStartWorkers in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105

goroutine 11 gp=0xc000484e00 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x0?, 0x0?, 0xc0?, 0xde?, 0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000507ed0 sp=0xc000507eb0 pc=0x7ff6765b630e
runtime.goparkunlock(...)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441
runtime.semacquire1(0xc0005ab7a8, 0x0, 0x1, 0x0, 0x18)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/sema.go:188 +0x22f fp=0xc000507f38 sp=0xc000507ed0 pc=0x7ff67659764f
sync.runtime_SemacquireWaitGroup(0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/sema.go:110 +0x25 fp=0xc000507f70 sp=0xc000507f38 pc=0x7ff6765b79e5
sync.(*WaitGroup).Wait(0x0?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/sync/waitgroup.go:118 +0x48 fp=0xc000507f98 sp=0xc000507f70 pc=0x7ff6765cb9c8
github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0005ab7a0, {0x7ff677973510, 0xc0003499a0})
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc000507fb8 sp=0xc000507f98 pc=0x7ff676ab5f05
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2()
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc000507fe0 sp=0xc000507fb8 pc=0x7ff676abb688
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000507fe8 sp=0xc000507fe0 pc=0x7ff6765bdb01
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	C:/a/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74

goroutine 50 gp=0xc000208fc0 m=nil [IO wait]:
runtime.gopark(0x0?, 0xc0004822a0?, 0x48?, 0x23?, 0xc00048234c?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc0017978c8 sp=0xc0017978a8 pc=0x7ff6765b630e
runtime.netpollblock(0x2ac?, 0x76550446?, 0xf6?)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:575 +0xf7 fp=0xc001797900 sp=0xc0017978c8 pc=0x7ff67657bf37
internal/poll.runtime_pollWait(0x1caefbd4ce8, 0x72)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:351 +0x85 fp=0xc001797920 sp=0xc001797900 pc=0x7ff6765b54a5
internal/poll.(*pollDesc).wait(0x7ff67677bb57?, 0xc001797970?, 0x0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc001797948 sp=0xc001797920 pc=0x7ff67664b887
internal/poll.execIO(0xc0004822a0, 0x7ff67780b1a0)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:177 +0x105 fp=0xc0017979c0 sp=0xc001797948 pc=0x7ff67664cce5
internal/poll.(*FD).Read(0xc000482288, {0xc001c0a000, 0x1000, 0x1000})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:438 +0x29b fp=0xc001797a60 sp=0xc0017979c0 pc=0x7ff67664d9bb
net.(*netFD).Read(0xc000482288, {0xc001c0a000?, 0xc001797ad0?, 0x7ff67664bd45?})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/fd_posix.go:55 +0x25 fp=0xc001797aa8 sp=0xc001797a60 pc=0x7ff6766c01c5
net.(*conn).Read(0xc000078000, {0xc001c0a000?, 0x0?, 0x0?})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/net.go:194 +0x45 fp=0xc001797af0 sp=0xc001797aa8 pc=0x7ff6766cf6a5
net/http.(*connReader).Read(0xc000330450, {0xc001c0a000, 0x1000, 0x1000})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:798 +0x159 fp=0xc001797b40 sp=0xc001797af0 pc=0x7ff6768bca19
bufio.(*Reader).fill(0xc0005e0120)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/bufio/bufio.go:113 +0x103 fp=0xc001797b78 sp=0xc001797b40 pc=0x7ff6766e5ee3
bufio.(*Reader).Peek(0xc0005e0120, 0x4)
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/bufio/bufio.go:152 +0x53 fp=0xc001797b98 sp=0xc001797b78 pc=0x7ff6766e6013
net/http.(*conn).serve(0xc0000d1ef0, {0x7ff6779734d8, 0xc0002b6270})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:2137 +0x785 fp=0xc001797fb8 sp=0xc001797b98 pc=0x7ff6768c2805
net/http.(*Server).Serve.gowrap3()
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3454 +0x28 fp=0xc001797fe0 sp=0xc001797fb8 pc=0x7ff6768c7f68
runtime.goexit({})
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc001797fe8 sp=0xc001797fe0 pc=0x7ff6765bdb01
created by net/http.(*Server).Serve in goroutine 1
	C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3454 +0x485
time=2025-07-31T09:05:10.772+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2"
time=2025-07-31T09:05:10.949+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault"
[GIN] 2025/07/31 - 09:05:10 | 500 |    816.9657ms |       127.0.0.1 | POST     "/api/generate"

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @x2611 on GitHub (Jul 31, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11598 ### What is the issue? Gemma3:12b ran fine with v0.9.2 and earlier, on both of my PC's. Nothing changed on either PC. Now Ollama fails to run it with: Windows Terminal: Error: llama runner process has terminated: error:fault Chatbox: API Error: Error from Ollama: Error: API Error: Status Code 500, {"error":{"message":"llama runner process has terminated: error:fault","type":"api_error","param":null,"code":null}} Deepseek-r1:8b still runs fine on both machines: i5-10600KF, 16GB, GTX1070 8GB, Win10 i7-6700K, 32GB, GTX1080Ti 11Gb, Win10 Using Gemma3:12b since it became available. I downgraded Ollama to v0.9.2 to test it and Gemma ran fine again: https://postimg.cc/TL67WTNc https://postimg.cc/BL4pnMtY Ollama updates itself unfortunately so the bug returns. Automatic updates should be turn off-able. ### Relevant log output ```shell time=2025-07-31T09:05:09.629+02:00 level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:E:\\Programs\\ai\\ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]" time=2025-07-31T09:05:09.631+02:00 level=INFO source=images.go:476 msg="total blobs: 10" time=2025-07-31T09:05:09.631+02:00 level=INFO source=images.go:483 msg="total unused blobs removed: 0" time=2025-07-31T09:05:09.632+02:00 level=INFO source=routes.go:1288 msg="Listening on 127.0.0.1:11434 (version 0.9.6)" time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu.go:217 msg="looking for compatible GPUs" time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu_windows.go:167 msg=packages count=1 time=2025-07-31T09:05:09.632+02:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=6 efficiency=0 threads=12 time=2025-07-31T09:05:09.746+02:00 level=INFO source=types.go:130 msg="inference compute" id=GPU-76fa6921-4f55-c156-f1d9-8f257f3fba18 library=cuda variant=v12 compute=6.1 driver=12.7 name="NVIDIA GeForce GTX 1070" total="8.0 GiB" available="7.0 GiB" [GIN] 2025/07/31 - 09:05:10 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2025/07/31 - 09:05:10 | 200 | 122.0154ms | 127.0.0.1 | POST "/api/show" time=2025-07-31T09:05:10.368+02:00 level=INFO source=server.go:135 msg="system memory" total="15.9 GiB" free="9.6 GiB" free_swap="8.2 GiB" time=2025-07-31T09:05:10.369+02:00 level=INFO source=server.go:175 msg=offload library=cuda layers.requested=-1 layers.model=49 layers.offload=27 layers.split="" memory.available="[5.7 GiB]" memory.gpu_overhead="0 B" memory.required.full="11.2 GiB" memory.required.partial="5.7 GiB" memory.required.kv="736.0 MiB" memory.required.allocations="[5.7 GiB]" memory.weights.total="6.8 GiB" memory.weights.repeating="6.0 GiB" memory.weights.nonrepeating="787.5 MiB" memory.graph.full="519.5 MiB" memory.graph.partial="1.3 GiB" projector.weights="795.9 MiB" projector.graph="1.0 GiB" time=2025-07-31T09:05:10.445+02:00 level=INFO source=server.go:438 msg="starting llama server" cmd="E:\\Programs\\ai\\ollama\\ollama.exe runner --ollama-engine --model E:\\Programs\\ai\\ollama\\models\\blobs\\sha256-e8ad13eff07a78d89926e9e8b882317d082ef5bf9768ad7b50fcdbbcd63748de --ctx-size 4096 --batch-size 512 --n-gpu-layers 27 --threads 6 --no-mmap --parallel 1 --port 55114" time=2025-07-31T09:05:10.446+02:00 level=INFO source=sched.go:483 msg="loaded runners" count=1 time=2025-07-31T09:05:10.446+02:00 level=INFO source=server.go:598 msg="waiting for llama runner to start responding" time=2025-07-31T09:05:10.447+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server error" time=2025-07-31T09:05:10.484+02:00 level=INFO source=runner.go:925 msg="starting ollama engine" time=2025-07-31T09:05:10.489+02:00 level=INFO source=runner.go:983 msg="Server listening on 127.0.0.1:55114" time=2025-07-31T09:05:10.549+02:00 level=INFO source=ggml.go:92 msg="" architecture=gemma3 file_type=Q4_K_M name="" description="" num_tensors=1065 num_key_values=37 ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1, VMM: yes load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\ggml-cuda.dll load_backend: loaded CPU backend from E:\Programs\ai\ollama\lib\ollama\ggml-cpu-haswell.dll ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce GTX 1070, compute capability 6.1, VMM: yes load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\cuda_v12\ggml-cuda.dll time=2025-07-31T09:05:10.648+02:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2025-07-31T09:05:10.698+02:00 level=INFO source=server.go:632 msg="waiting for server to become available" status="llm server loading model" unexpected fault address 0x1c3100000 fatal error: fault [signal 0xc0000005 code=0x0 addr=0x1c3100000 pc=0x7ff676547180] goroutine 10 gp=0xc000484c40 m=7 mp=0xc000510008 [running]: runtime.throw({0x7ff6777977d7?, 0xc000056508?}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/panic.go:1096 +0x4d fp=0xc00004b0d0 sp=0xc00004b0a0 pc=0x7ff6765b61ed runtime.sigpanic() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/signal_windows.go:414 +0xd0 fp=0xc00004b118 sp=0xc00004b0d0 pc=0x7ff676598f10 indexbytebody() C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/bytealg/indexbyte_amd64.s:131 +0xe0 fp=0xc00004b120 sp=0xc00004b118 pc=0x7ff676547180 runtime.findnull(0xc00004b1a0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/string.go:577 +0x79 fp=0xc00004b178 sp=0xc00004b120 pc=0x7ff67659e039 runtime.gostring(0x1c3100000) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/string.go:363 +0x1c fp=0xc00004b1b0 sp=0xc00004b178 pc=0x7ff6765b8d5c github.com/ollama/ollama/ml/backend/ggml._Cfunc_GoString(...) _cgo_gotypes.go:299 github.com/ollama/ollama/ml/backend/ggml.New({0xc00003e150, 0x6a}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0}) C:/a/ollama/ollama/ml/backend/ggml/ggml.go:158 +0x1359 fp=0xc00004bc18 sp=0xc00004b1b0 pc=0x7ff676a08499 github.com/ollama/ollama/ml.NewBackend({0xc00003e150, 0x6a}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0}) C:/a/ollama/ollama/ml/backend.go:209 +0xb1 fp=0xc00004bc70 sp=0xc00004bc18 pc=0x7ff6769f9091 github.com/ollama/ollama/model.New({0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0}) C:/a/ollama/ollama/model/model.go:102 +0x8f fp=0xc00004bd68 sp=0xc00004bc70 pc=0x7ff676a17f6f github.com/ollama/ollama/runner/ollamarunner.(*Server).initModel(0xc0005ab7a0, {0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, 0x0}, 0x0}, ...) C:/a/ollama/ollama/runner/ollamarunner/runner.go:841 +0x8d fp=0xc00004bdc8 sp=0xc00004bd68 pc=0x7ff676ab9fed github.com/ollama/ollama/runner/ollamarunner.(*Server).load(0xc0005ab7a0, {0x7ff677973510, 0xc0003499a0}, {0xc00003e150?, 0x0?}, {0x6, 0x0, 0x1b, {0x0, 0x0, ...}, ...}, ...) C:/a/ollama/ollama/runner/ollamarunner/runner.go:878 +0xb8 fp=0xc00004bf20 sp=0xc00004bdc8 pc=0x7ff676aba358 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xc7 fp=0xc00004bfe0 sp=0xc00004bf20 pc=0x7ff676abb787 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00004bfe8 sp=0xc00004bfe0 pc=0x7ff6765bdb01 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 C:/a/ollama/ollama/runner/ollamarunner/runner.go:959 +0xa11 goroutine 1 gp=0xc0000021c0 m=nil [IO wait]: runtime.gopark(0x7ff6765bf300?, 0x7ff6782c5500?, 0x20?, 0x20?, 0xc0004820cc?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00179b508 sp=0xc00179b4e8 pc=0x7ff6765b630e runtime.netpollblock(0x278?, 0x76550446?, 0xf6?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:575 +0xf7 fp=0xc00179b540 sp=0xc00179b508 pc=0x7ff67657bf37 internal/poll.runtime_pollWait(0x1caefbd4e00, 0x72) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:351 +0x85 fp=0xc00179b560 sp=0xc00179b540 pc=0x7ff6765b54a5 internal/poll.(*pollDesc).wait(0x7ff67664a293?, 0x7ff676562776?, 0x0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00179b588 sp=0xc00179b560 pc=0x7ff67664b887 internal/poll.execIO(0xc000482020, 0xc00179b630) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:177 +0x105 fp=0xc00179b600 sp=0xc00179b588 pc=0x7ff67664cce5 internal/poll.(*FD).acceptOne(0xc000482008, 0x2f0, {0xc0000e80f0?, 0x7ff67656ae05?, 0x7ff67833a6e0?}, 0x25b0aaa?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:946 +0x65 fp=0xc00179b660 sp=0xc00179b600 pc=0x7ff676651265 internal/poll.(*FD).Accept(0xc000482008, 0xc00179b810) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:980 +0x1b6 fp=0xc00179b718 sp=0xc00179b660 pc=0x7ff676651596 net.(*netFD).accept(0xc000482008) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/fd_windows.go:182 +0x4b fp=0xc00179b830 sp=0xc00179b718 pc=0x7ff6766c20ab net.(*TCPListener).accept(0xc00008a000) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/tcpsock_posix.go:159 +0x1b fp=0xc00179b880 sp=0xc00179b830 pc=0x7ff6766d80fb net.(*TCPListener).Accept(0xc00008a000) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/tcpsock.go:380 +0x30 fp=0xc00179b8b0 sp=0xc00179b880 pc=0x7ff6766d6eb0 net/http.(*onceCloseListener).Accept(0xc0000d1ef0?) <autogenerated>:1 +0x24 fp=0xc00179b8c8 sp=0xc00179b8b0 pc=0x7ff6768f02a4 net/http.(*Server).Serve(0xc0005c2000, {0x7ff677970f20, 0xc00008a000}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3424 +0x30c fp=0xc00179b9f8 sp=0xc00179b8c8 pc=0x7ff6768c7b6c github.com/ollama/ollama/runner/ollamarunner.Execute({0xc0000a8030, 0xf, 0x1d}) C:/a/ollama/ollama/runner/ollamarunner/runner.go:984 +0xe09 fp=0xc00179bd08 sp=0xc00179b9f8 pc=0x7ff676abb3e9 github.com/ollama/ollama/runner.Execute({0xc0000a8010?, 0x0?, 0x0?}) C:/a/ollama/ollama/runner/runner.go:20 +0xc9 fp=0xc00179bd30 sp=0xc00179bd08 pc=0x7ff676abbce9 github.com/ollama/ollama/cmd.NewCLI.func2(0xc0005c2f00?, {0x7ff677796384?, 0x4?, 0x7ff677796388?}) C:/a/ollama/ollama/cmd/cmd.go:1529 +0x45 fp=0xc00179bd58 sp=0xc00179bd30 pc=0x7ff67721a765 github.com/spf13/cobra.(*Command).execute(0xc00020cf08, {0xc0005c3100, 0x10, 0x10}) C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc00179be78 sp=0xc00179bd58 pc=0x7ff67673cb7c github.com/spf13/cobra.(*Command).ExecuteC(0xc000466f08) C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc00179bf30 sp=0xc00179be78 pc=0x7ff67673d3c5 github.com/spf13/cobra.(*Command).Execute(...) C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) C:/Users/runneradmin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 main.main() C:/a/ollama/ollama/main.go:12 +0x4d fp=0xc00179bf50 sp=0xc00179bf30 pc=0x7ff67721b1ed runtime.main() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:283 +0x27d fp=0xc00179bfe0 sp=0xc00179bf50 pc=0x7ff676584f1d runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00179bfe8 sp=0xc00179bfe0 pc=0x7ff6765bdb01 goroutine 2 gp=0xc0000028c0 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000071fa8 sp=0xc000071f88 pc=0x7ff6765b630e runtime.goparkunlock(...) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441 runtime.forcegchelper() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:348 +0xb8 fp=0xc000071fe0 sp=0xc000071fa8 pc=0x7ff676585238 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000071fe8 sp=0xc000071fe0 pc=0x7ff6765bdb01 created by runtime.init.7 in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:336 +0x1a goroutine 3 gp=0xc000002c40 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000073f80 sp=0xc000073f60 pc=0x7ff6765b630e runtime.goparkunlock(...) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441 runtime.bgsweep(0xc000080000) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcsweep.go:316 +0xdf fp=0xc000073fc8 sp=0xc000073f80 pc=0x7ff67656dfff runtime.gcenable.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:204 +0x25 fp=0xc000073fe0 sp=0xc000073fc8 pc=0x7ff6765623c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000073fe8 sp=0xc000073fe0 pc=0x7ff6765bdb01 created by runtime.gcenable in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:204 +0x66 goroutine 4 gp=0xc000002e00 m=nil [GC scavenge wait]: runtime.gopark(0x10000?, 0x7ff67795e160?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000087f78 sp=0xc000087f58 pc=0x7ff6765b630e runtime.goparkunlock(...) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441 runtime.(*scavengerState).park(0x7ff6782ebc00) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000087fa8 sp=0xc000087f78 pc=0x7ff67656ba49 runtime.bgscavenge(0xc000080000) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000087fc8 sp=0xc000087fa8 pc=0x7ff67656bfd9 runtime.gcenable.gowrap2() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:205 +0x25 fp=0xc000087fe0 sp=0xc000087fc8 pc=0x7ff676562365 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000087fe8 sp=0xc000087fe0 pc=0x7ff6765bdb01 created by runtime.gcenable in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:205 +0xa5 goroutine 5 gp=0xc000003340 m=nil [finalizer wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000089e30 sp=0xc000089e10 pc=0x7ff6765b630e runtime.runfinq() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mfinal.go:196 +0x107 fp=0xc000089fe0 sp=0xc000089e30 pc=0x7ff676561347 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000089fe8 sp=0xc000089fe0 pc=0x7ff6765bdb01 created by runtime.createfing in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mfinal.go:166 +0x3d goroutine 6 gp=0xc000003dc0 m=nil [chan receive]: runtime.gopark(0xc00016f860?, 0xc000588018?, 0x60?, 0x5f?, 0x7ff6766ab0e8?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000075f18 sp=0xc000075ef8 pc=0x7ff6765b630e runtime.chanrecv(0xc00003e3f0, 0x0, 0x1) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/chan.go:664 +0x445 fp=0xc000075f90 sp=0xc000075f18 pc=0x7ff676552d85 runtime.chanrecv1(0x7ff676585080?, 0xc000075f76?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/chan.go:506 +0x12 fp=0xc000075fb8 sp=0xc000075f90 pc=0x7ff676552912 runtime.unique_runtime_registerUniqueMapCleanup.func2(...) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1796 runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1799 +0x2f fp=0xc000075fe0 sp=0xc000075fb8 pc=0x7ff6765655ef runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000075fe8 sp=0xc000075fe0 pc=0x7ff6765bdb01 created by unique.runtime_registerUniqueMapCleanup in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1794 +0x85 goroutine 7 gp=0xc0003e8380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000083f38 sp=0xc000083f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000083fc8 sp=0xc000083f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 18 gp=0xc0002081c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000213f38 sp=0xc000213f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000213fc8 sp=0xc000213f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000213fe0 sp=0xc000213fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000213fe8 sp=0xc000213fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 19 gp=0xc000208380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000215f38 sp=0xc000215f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000215fc8 sp=0xc000215f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000215fe0 sp=0xc000215fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000215fe8 sp=0xc000215fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 20 gp=0xc000208540 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d9da674?, 0x1?, 0x4c?, 0x40?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00020ff38 sp=0xc00020ff18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00020ffc8 sp=0xc00020ff38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00020ffe0 sp=0xc00020ffc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00020ffe8 sp=0xc00020ffe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 21 gp=0xc000208700 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d9da674?, 0x3?, 0x4c?, 0x40?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000211f38 sp=0xc000211f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000211fc8 sp=0xc000211f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000211fe0 sp=0xc000211fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000211fe8 sp=0xc000211fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 34 gp=0xc000484000 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d9da674?, 0x3?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00048bf38 sp=0xc00048bf18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00048bfc8 sp=0xc00048bf38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00048bfe0 sp=0xc00048bfc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00048bfe8 sp=0xc00048bfe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 8 gp=0xc0003e8540 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d85c07c?, 0x3?, 0x4c?, 0x40?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000085f38 sp=0xc000085f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000085fc8 sp=0xc000085f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000085fe0 sp=0xc000085fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000085fe8 sp=0xc000085fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 22 gp=0xc0002088c0 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d85c07c?, 0x3?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000487f38 sp=0xc000487f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000487fc8 sp=0xc000487f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000487fe0 sp=0xc000487fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 35 gp=0xc0004841c0 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d768030?, 0x3?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc00048df38 sp=0xc00048df18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc00048dfc8 sp=0xc00048df38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc00048dfe0 sp=0xc00048dfc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc00048dfe8 sp=0xc00048dfe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 9 gp=0xc0003e8700 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d9da674?, 0x3?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000505f38 sp=0xc000505f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000505fc8 sp=0xc000505f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000505fe0 sp=0xc000505fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 23 gp=0xc000208a80 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d768030?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000489f38 sp=0xc000489f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000489fc8 sp=0xc000489f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000489fe0 sp=0xc000489fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000489fe8 sp=0xc000489fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 36 gp=0xc000484380 m=nil [GC worker (idle)]: runtime.gopark(0xe5b71d768030?, 0x0?, 0x0?, 0x0?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000501f38 sp=0xc000501f18 pc=0x7ff6765b630e runtime.gcBgMarkWorker(0xc00003f9d0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000501fc8 sp=0xc000501f38 pc=0x7ff6765648e9 runtime.gcBgMarkStartWorkers.gowrap1() C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x25 fp=0xc000501fe0 sp=0xc000501fc8 pc=0x7ff6765647c5 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000501fe8 sp=0xc000501fe0 pc=0x7ff6765bdb01 created by runtime.gcBgMarkStartWorkers in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/mgc.go:1339 +0x105 goroutine 11 gp=0xc000484e00 m=nil [sync.WaitGroup.Wait]: runtime.gopark(0x0?, 0x0?, 0xc0?, 0xde?, 0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc000507ed0 sp=0xc000507eb0 pc=0x7ff6765b630e runtime.goparkunlock(...) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:441 runtime.semacquire1(0xc0005ab7a8, 0x0, 0x1, 0x0, 0x18) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/sema.go:188 +0x22f fp=0xc000507f38 sp=0xc000507ed0 pc=0x7ff67659764f sync.runtime_SemacquireWaitGroup(0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/sema.go:110 +0x25 fp=0xc000507f70 sp=0xc000507f38 pc=0x7ff6765b79e5 sync.(*WaitGroup).Wait(0x0?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/sync/waitgroup.go:118 +0x48 fp=0xc000507f98 sp=0xc000507f70 pc=0x7ff6765cb9c8 github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0005ab7a0, {0x7ff677973510, 0xc0003499a0}) C:/a/ollama/ollama/runner/ollamarunner/runner.go:355 +0x25 fp=0xc000507fb8 sp=0xc000507f98 pc=0x7ff676ab5f05 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap2() C:/a/ollama/ollama/runner/ollamarunner/runner.go:960 +0x28 fp=0xc000507fe0 sp=0xc000507fb8 pc=0x7ff676abb688 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000507fe8 sp=0xc000507fe0 pc=0x7ff6765bdb01 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 C:/a/ollama/ollama/runner/ollamarunner/runner.go:960 +0xa74 goroutine 50 gp=0xc000208fc0 m=nil [IO wait]: runtime.gopark(0x0?, 0xc0004822a0?, 0x48?, 0x23?, 0xc00048234c?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/proc.go:435 +0xce fp=0xc0017978c8 sp=0xc0017978a8 pc=0x7ff6765b630e runtime.netpollblock(0x2ac?, 0x76550446?, 0xf6?) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:575 +0xf7 fp=0xc001797900 sp=0xc0017978c8 pc=0x7ff67657bf37 internal/poll.runtime_pollWait(0x1caefbd4ce8, 0x72) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/netpoll.go:351 +0x85 fp=0xc001797920 sp=0xc001797900 pc=0x7ff6765b54a5 internal/poll.(*pollDesc).wait(0x7ff67677bb57?, 0xc001797970?, 0x0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc001797948 sp=0xc001797920 pc=0x7ff67664b887 internal/poll.execIO(0xc0004822a0, 0x7ff67780b1a0) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:177 +0x105 fp=0xc0017979c0 sp=0xc001797948 pc=0x7ff67664cce5 internal/poll.(*FD).Read(0xc000482288, {0xc001c0a000, 0x1000, 0x1000}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/internal/poll/fd_windows.go:438 +0x29b fp=0xc001797a60 sp=0xc0017979c0 pc=0x7ff67664d9bb net.(*netFD).Read(0xc000482288, {0xc001c0a000?, 0xc001797ad0?, 0x7ff67664bd45?}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/fd_posix.go:55 +0x25 fp=0xc001797aa8 sp=0xc001797a60 pc=0x7ff6766c01c5 net.(*conn).Read(0xc000078000, {0xc001c0a000?, 0x0?, 0x0?}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/net.go:194 +0x45 fp=0xc001797af0 sp=0xc001797aa8 pc=0x7ff6766cf6a5 net/http.(*connReader).Read(0xc000330450, {0xc001c0a000, 0x1000, 0x1000}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:798 +0x159 fp=0xc001797b40 sp=0xc001797af0 pc=0x7ff6768bca19 bufio.(*Reader).fill(0xc0005e0120) C:/hostedtoolcache/windows/go/1.24.0/x64/src/bufio/bufio.go:113 +0x103 fp=0xc001797b78 sp=0xc001797b40 pc=0x7ff6766e5ee3 bufio.(*Reader).Peek(0xc0005e0120, 0x4) C:/hostedtoolcache/windows/go/1.24.0/x64/src/bufio/bufio.go:152 +0x53 fp=0xc001797b98 sp=0xc001797b78 pc=0x7ff6766e6013 net/http.(*conn).serve(0xc0000d1ef0, {0x7ff6779734d8, 0xc0002b6270}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:2137 +0x785 fp=0xc001797fb8 sp=0xc001797b98 pc=0x7ff6768c2805 net/http.(*Server).Serve.gowrap3() C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3454 +0x28 fp=0xc001797fe0 sp=0xc001797fb8 pc=0x7ff6768c7f68 runtime.goexit({}) C:/hostedtoolcache/windows/go/1.24.0/x64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc001797fe8 sp=0xc001797fe0 pc=0x7ff6765bdb01 created by net/http.(*Server).Serve in goroutine 1 C:/hostedtoolcache/windows/go/1.24.0/x64/src/net/http/server.go:3454 +0x485 time=2025-07-31T09:05:10.772+02:00 level=ERROR source=server.go:464 msg="llama runner terminated" error="exit status 2" time=2025-07-31T09:05:10.949+02:00 level=ERROR source=sched.go:489 msg="error loading llama server" error="llama runner process has terminated: error:fault" [GIN] 2025/07/31 - 09:05:10 | 500 | 816.9657ms | 127.0.0.1 | POST "/api/generate" ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-12 19:44:50 -05:00
Author
Owner

@rick-github commented on GitHub (Jul 31, 2025):

load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\ggml-cuda.dll
load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\cuda_v12\ggml-cuda.dll

#11211

Remove E:\Programs\ai\ollama\lib\ollama\cuda_v12

<!-- gh-comment-id:3139378290 --> @rick-github commented on GitHub (Jul 31, 2025): ``` load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\ggml-cuda.dll load_backend: loaded CUDA backend from E:\Programs\ai\ollama\lib\ollama\cuda_v12\ggml-cuda.dll ``` #11211 Remove E:\Programs\ai\ollama\lib\ollama\cuda_v12
Author
Owner

@x2611 commented on GitHub (Jul 31, 2025):

Removing E:\Programs\ai\ollama\lib\ollama\cuda_v12 worked.
So did an Ollama clean install. Thx.

<!-- gh-comment-id:3139500105 --> @x2611 commented on GitHub (Jul 31, 2025): Removing E:\Programs\ai\ollama\lib\ollama\cuda_v12 worked. So did an Ollama clean install. Thx.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7656