github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #14921] Strix halo (gfx1511) fails to load rocm iGPU - falls back to CPU #71664

New Issue
Closed
opened 2026-05-05 02:17:43 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-05 02:17:43 -05:00
Owner
Copy Link

Originally created by @nishtahir on GitHub (Mar 18, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/14921

What is the issue?

From what I can tell this is isolated to >= 0.18.0-rocm. I tested this on rocm 7.0.0+ and in every case, I consistently get filtering device which didn't fully initialize before it falls back to CPU.

I'm tested with docker options --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined --group-add=video --group-add=render --ipc=host --privileged --cap-add=SYS_PTRACE

Environment variables:
OLLAMA_DEBUG=2, AMD_LOG_LEVEL=3

System:

OS: Ubuntu 24.04.4 LTS
Arch: x86_64
Kernel: 6.8.0-106-generic
CPU Cores: 32
RAM: 124.5 GB

Downgrading to 0.17.0-rocm seems to work.

Relevant log output

time=2026-03-18T00:43:29.654Z level=INFO source=routes.go:1727 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG-4 OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:<REDACTED>@v1.7.0/command.go:940 +0x85c fp=0xc000521e78 sp=0xc000521d58 pc=0x65141dc85cdc
github.com/spf13/cobra.(*Command).ExecuteC(0xc000552908)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000521f30 sp=0xc000521e78 pc=0x65141dc86525
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000521f50 sp=0xc000521f30 pc=0x65141e9f038d
runtime.main()
	runtime/proc.go:283 +0x29d fp=0xc000521fe0 sp=0xc000521f50 pc=0x65141dad871d
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000521fe8 sp=0xc000521fe0 pc=0x65141db13e61

goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000aafa8 sp=0xc0000aaf88 pc=0x65141db0beee
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.forcegchelper()
	runtime/proc.go:348 +0xb8 fp=0xc0000aafe0 sp=0xc0000aafa8 pc=0x65141dad8a58
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000aafe8 sp=0xc0000aafe0 pc=0x65141db13e61
created by runtime.init.7 in goroutine 1
	runtime/proc.go:336 +0x1a

goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000ab780 sp=0xc0000ab760 pc=0x65141db0beee
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.bgsweep(0xc0000d6000)
	runtime/mgcsweep.go:316 +0xdf fp=0xc0000ab7c8 sp=0xc0000ab780 pc=0x65141dac31ff
runtime.gcenable.gowrap1()
	runtime/mgc.go:204 +0x25 fp=0xc0000ab7e0 sp=0xc0000ab7c8 pc=0x65141dab75e5
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ab7e8 sp=0xc0000ab7e0 pc=0x65141db13e61
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:204 +0x66

goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x65141efdc748?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000abf78 sp=0xc0000abf58 pc=0x65141db0beee
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.(*scavengerState).park(0x65141fde7e00)
	runtime/mgcscavenge.go:425 +0x49 fp=0xc0000abfa8 sp=0xc0000abf78 pc=0x65141dac0c49
runtime.bgscavenge(0xc0000d6000)
	runtime/mgcscavenge.go:658 +0x59 fp=0xc0000abfc8 sp=0xc0000abfa8 pc=0x65141dac11d9
runtime.gcenable.gowrap2()
	runtime/mgc.go:205 +0x25 fp=0xc0000abfe0 sp=0xc0000abfc8 pc=0x65141dab7585
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000abfe8 sp=0xc0000abfe0 pc=0x65141db13e61
created by runtime.gcenable in goroutine 1
	runtime/mgc.go:205 +0xa5

goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]:
runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc0000aa688?)
	runtime/proc.go:435 +0xce fp=0xc0000aa630 sp=0xc0000aa610 pc=0x65141db0beee
runtime.runfinq()
	runtime/mfinal.go:196 +0x107 fp=0xc0000aa7e0 sp=0xc0000aa630 pc=0x65141dab65a7
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000aa7e8 sp=0xc0000aa7e0 pc=0x65141db13e61
created by runtime.createfing in goroutine 1
	runtime/mfinal.go:166 +0x3d

goroutine 6 gp=0xc0002148c0 m=nil [chan receive]:
runtime.gopark(0xc0001fdcc0?, 0xc000011128?, 0x60?, 0xc7?, 0x65141dbf28a8?)
	runtime/proc.go:435 +0xce fp=0xc0000ac718 sp=0xc0000ac6f8 pc=0x65141db0beee
runtime.chanrecv(0xc000040380, 0x0, 0x1)
	runtime/chan.go:664 +0x445 fp=0xc0000ac790 sp=0xc0000ac718 pc=0x65141daa8085
runtime.chanrecv1(0x0?, 0x0?)
	runtime/chan.go:506 +0x12 fp=0xc0000ac7b8 sp=0xc0000ac790 pc=0x65141daa7c12
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
	runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
	runtime/mgc.go:1799 +0x2f fp=0xc0000ac7e0 sp=0xc0000ac7b8 pc=0x65141daba78f
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ac7e8 sp=0xc0000ac7e0 pc=0x65141db13e61
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
	runtime/mgc.go:1794 +0x85

goroutine 7 gp=0xc000214c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000acf38 sp=0xc0000acf18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000acfc8 sp=0xc0000acf38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000acfe0 sp=0xc0000acfc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000acfe8 sp=0xc0000acfe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 8 gp=0xc000214e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000ad738 sp=0xc0000ad718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000ad7c8 sp=0xc0000ad738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000ad7e0 sp=0xc0000ad7c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ad7e8 sp=0xc0000ad7e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 9 gp=0xc000214fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000adf38 sp=0xc0000adf18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000adfc8 sp=0xc0000adf38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000adfe0 sp=0xc0000adfc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000adfe8 sp=0xc0000adfe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 10 gp=0xc000215180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a6738 sp=0xc0000a6718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a67c8 sp=0xc0000a6738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a67e0 sp=0xc0000a67c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a67e8 sp=0xc0000a67e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 11 gp=0xc000215340 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a6f38 sp=0xc0000a6f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a6fc8 sp=0xc0000a6f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a6fe0 sp=0xc0000a6fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a6fe8 sp=0xc0000a6fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 12 gp=0xc000215500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a7738 sp=0xc0000a7718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a77c8 sp=0xc0000a7738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a77e0 sp=0xc0000a77c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a77e8 sp=0xc0000a77e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 18 gp=0xc00031a000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000320738 sp=0xc000320718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0003207c8 sp=0xc000320738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0003207e0 sp=0xc0003207c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0003207e8 sp=0xc0003207e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 19 gp=0xc00031a1c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000320f38 sp=0xc000320f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000320fc8 sp=0xc000320f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000320fe0 sp=0xc000320fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000320fe8 sp=0xc000320fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 20 gp=0xc00031a380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000321738 sp=0xc000321718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0003217c8 sp=0xc000321738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0003217e0 sp=0xc0003217c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0003217e8 sp=0xc0003217e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 21 gp=0xc00031a540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000321f38 sp=0xc000321f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000321fc8 sp=0xc000321f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000321fe0 sp=0xc000321fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000321fe8 sp=0xc000321fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 22 gp=0xc00031a700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000322738 sp=0xc000322718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0003227c8 sp=0xc000322738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0003227e0 sp=0xc0003227c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0003227e8 sp=0xc0003227e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031c738 sp=0xc00031c718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031c7c8 sp=0xc00031c738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031c7e0 sp=0xc00031c7c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031c7e8 sp=0xc00031c7e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 35 gp=0xc000102540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031cf38 sp=0xc00031cf18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031cfc8 sp=0xc00031cf38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031cfe0 sp=0xc00031cfc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031cfe8 sp=0xc00031cfe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 36 gp=0xc000102700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031d738 sp=0xc00031d718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031d7c8 sp=0xc00031d738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031d7e0 sp=0xc00031d7c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031d7e8 sp=0xc00031d7e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 37 gp=0xc0001028c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031df38 sp=0xc00031df18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031dfc8 sp=0xc00031df38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031dfe0 sp=0xc00031dfc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031dfe8 sp=0xc00031dfe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 38 gp=0xc000102a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031e738 sp=0xc00031e718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031e7c8 sp=0xc00031e738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031e7e0 sp=0xc00031e7c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031e7e8 sp=0xc00031e7e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 39 gp=0xc000102c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031ef38 sp=0xc00031ef18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031efc8 sp=0xc00031ef38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031efe0 sp=0xc00031efc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031efe8 sp=0xc00031efe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 40 gp=0xc000102e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031f738 sp=0xc00031f718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031f7c8 sp=0xc00031f738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031f7e0 sp=0xc00031f7c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031f7e8 sp=0xc00031f7e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 41 gp=0xc000102fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc00031ff38 sp=0xc00031ff18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc00031ffc8 sp=0xc00031ff38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc00031ffe0 sp=0xc00031ffc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00031ffe8 sp=0xc00031ffe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 42 gp=0xc000103180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000118738 sp=0xc000118718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0001187c8 sp=0xc000118738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0001187e0 sp=0xc0001187c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 13 gp=0xc0002156c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a7f38 sp=0xc0000a7f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a7fc8 sp=0xc0000a7f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a7fe0 sp=0xc0000a7fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a7fe8 sp=0xc0000a7fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 14 gp=0xc000215880 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a8738 sp=0xc0000a8718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a87c8 sp=0xc0000a8738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a87e0 sp=0xc0000a87c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a87e8 sp=0xc0000a87e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 15 gp=0xc000215a40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000a8f38 sp=0xc0000a8f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0000a8fc8 sp=0xc0000a8f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0000a8fe0 sp=0xc0000a8fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a8fe8 sp=0xc0000a8fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 23 gp=0xc00031a8c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000322f38 sp=0xc000322f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000322fc8 sp=0xc000322f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000322fe0 sp=0xc000322fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000322fe8 sp=0xc000322fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 24 gp=0xc00031aa80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000323738 sp=0xc000323718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0003237c8 sp=0xc000323738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0003237e0 sp=0xc0003237c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0003237e8 sp=0xc0003237e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 25 gp=0xc00031ac40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000323f38 sp=0xc000323f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000323fc8 sp=0xc000323f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000323fe0 sp=0xc000323fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000323fe8 sp=0xc000323fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 26 gp=0xc00031ae00 m=nil [GC worker (idle)]:
runtime.gopark(0xcda9966b25?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000114738 sp=0xc000114718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0001147c8 sp=0xc000114738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0001147e0 sp=0xc0001147c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 27 gp=0xc00031afc0 m=nil [GC worker (idle)]:
runtime.gopark(0xcda9965b0d?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000114f38 sp=0xc000114f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000114fc8 sp=0xc000114f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000114fe0 sp=0xc000114fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000114fe8 sp=0xc000114fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 28 gp=0xc00031b180 m=nil [GC worker (idle)]:
runtime.gopark(0xcda9975372?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000115738 sp=0xc000115718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0001157c8 sp=0xc000115738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0001157e0 sp=0xc0001157c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0001157e8 sp=0xc0001157e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 29 gp=0xc00031b340 m=nil [GC worker (idle)]:
runtime.gopark(0xcda9965c9d?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000115f38 sp=0xc000115f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000115fc8 sp=0xc000115f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000115fe0 sp=0xc000115fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 30 gp=0xc00031b500 m=nil [GC worker (idle)]:
runtime.gopark(0xcda99625ed?, 0x1?, 0x3a?, 0xe9?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000116738 sp=0xc000116718 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc0001167c8 sp=0xc000116738 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc0001167e0 sp=0xc0001167c8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0001167e8 sp=0xc0001167e0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 43 gp=0xc000103340 m=nil [GC worker (idle)]:
runtime.gopark(0xcda9962bf1?, 0x0?, 0x0?, 0x0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc000118f38 sp=0xc000118f18 pc=0x65141db0beee
runtime.gcBgMarkWorker(0xc0000415e0)
	runtime/mgc.go:1423 +0xe9 fp=0xc000118fc8 sp=0xc000118f38 pc=0x65141dab9aa9
runtime.gcBgMarkStartWorkers.gowrap1()
	runtime/mgc.go:1339 +0x25 fp=0xc000118fe0 sp=0xc000118fc8 pc=0x65141dab9985
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc000118fe8 sp=0xc000118fe0 pc=0x65141db13e61
created by runtime.gcBgMarkStartWorkers in goroutine 1
	runtime/mgc.go:1339 +0x105

goroutine 16 gp=0xc000602fc0 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0xe0?, 0x0?)
	runtime/proc.go:435 +0xce fp=0xc0000bda90 sp=0xc0000bda70 pc=0x65141db0beee
runtime.goparkunlock(...)
	runtime/proc.go:441
runtime.semacquire1(0xc000263198, 0x0, 0x1, 0x0, 0x18)
	runtime/sema.go:188 +0x229 fp=0xc0000bdaf8 sp=0xc0000bda90 pc=0x65141daebce9
sync.runtime_SemacquireWaitGroup(0x0?)
	runtime/sema.go:110 +0x25 fp=0xc0000bdb30 sp=0xc0000bdaf8 pc=0x65141db0d825
sync.(*WaitGroup).Wait(0xc000263190?)
	sync/waitgroup.go:118 +0x48 fp=0xc0000bdb58 sp=0xc0000bdb30 pc=0x65141db1f8c8
github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0002630e0, {0x65141f3ac170, 0xc000578820})
	github.com/ollama/ollama/runner/ollamarunner/runner.go:442 +0x45 fp=0xc0000bdfb8 sp=0xc0000bdb58 pc=0x65141e1255c5
github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1()
	github.com/ollama/ollama/runner/ollamarunner/runner.go:1424 +0x28 fp=0xc0000bdfe0 sp=0xc0000bdfb8 pc=0x65141e12ee48
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc0000bdfe8 sp=0xc0000bdfe0 pc=0x65141db13e61
created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1
	github.com/ollama/ollama/runner/ollamarunner/runner.go:1424 +0x4c9

goroutine 51 gp=0xc000603340 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0xb?)
	runtime/proc.go:435 +0xce fp=0xc00011b5d8 sp=0xc00011b5b8 pc=0x65141db0beee
runtime.netpollblock(0x65141db2f798?, 0x1daa54a6?, 0x14?)
	runtime/netpoll.go:575 +0xf7 fp=0xc00011b610 sp=0xc00011b5d8 pc=0x65141dad1097
internal/poll.runtime_pollWait(0x74389520f5b8, 0x72)
	runtime/netpoll.go:351 +0x85 fp=0xc00011b630 sp=0xc00011b610 pc=0x65141db0b105
internal/poll.(*pollDesc).wait(0xc00011ec80?, 0xc000139661?, 0x0)
	internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00011b658 sp=0xc00011b630 pc=0x65141db93487
internal/poll.(*pollDesc).waitRead(...)
	internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc00011ec80, {0xc000139661, 0x1, 0x1})
	internal/poll/fd_unix.go:165 +0x27a fp=0xc00011b6f0 sp=0xc00011b658 pc=0x65141db9477a
net.(*netFD).Read(0xc00011ec80, {0xc000139661?, 0x0?, 0x0?})
	net/fd_posix.go:55 +0x25 fp=0xc00011b738 sp=0xc00011b6f0 pc=0x65141dc09da5
net.(*conn).Read(0xc0000aea90, {0xc000139661?, 0x0?, 0x0?})
	net/net.go:194 +0x45 fp=0xc00011b780 sp=0xc00011b738 pc=0x65141dc18165
net/http.(*connReader).backgroundRead(0xc000139650)
	net/http/server.go:690 +0x37 fp=0xc00011b7c8 sp=0xc00011b780 pc=0x65141de14057
net/http.(*connReader).startBackgroundRead.gowrap2()
	net/http/server.go:686 +0x25 fp=0xc00011b7e0 sp=0xc00011b7c8 pc=0x65141de13f85
runtime.goexit({})
	runtime/asm_amd64.s:1700 +0x1 fp=0xc00011b7e8 sp=0xc00011b7e0 pc=0x65141db13e61
created by net/http.(*connReader).startBackgroundRead in goroutine 50
	net/http/server.go:686 +0xb6

rax    0x743845a31948
rbx    0x74383472d0b0
rcx    0x1
rdx    0x743834892
rdi    0x74383472d0b0
rsi    0x74384c868320
rbp    0x74384c868320
rsp    0x74384c868280
r8     0x7438340008e0
r9     0x7
r10    0x743834892160
r11    0xc2f59f465ad1bfdd
r12    0x0
r13    0x74383472d598
r14    0x7438348c9c50
r15    0x743834892160
rip    0x7438458b7170
rflags 0x10202
cs     0x33
fs     0x0
gs     0x0
time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:462 msg="runner exited" OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" extra_envs="map[GGML_CUDA_INIT:1 ROCR_VISIBLE_DEVICES:0]" code=2
time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:467 msg="runner enumerated devices" OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" devices=[]
time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=74.526671ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" extra_envs="map[GGML_CUDA_INIT:1 ROCR_VISIBLE_DEVICES:0]"
time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=0 libdir=/usr/lib/ollama/rocm pci_id=0000:c5:00.0 library=ROCm
time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:174 msg="supported GPU library combinations before filtering" supported=map[]
time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:183 msg="removing unsupported or overlapping GPU combination" libDir=/usr/lib/ollama/rocm description="Radeon 8060S Graphics" compute=gfx1151 pci_id=0000:c5:00.0
time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=141.02312ms
time=2026-03-18T00:43:29.795Z level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="124.5 GiB" available="124.4 GiB"
time=2026-03-18T00:43:29.795Z level=INFO source=routes.go:1832 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.18.*

Originally created by @nishtahir on GitHub (Mar 18, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/14921 ### What is the issue? From what I can tell this is isolated to >= 0.18.0-rocm. I tested this on rocm 7.0.0+ and in every case, I consistently get `filtering device which didn't fully initialize` before it falls back to CPU. I'm tested with docker options `--device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined --group-add=video --group-add=render --ipc=host --privileged --cap-add=SYS_PTRACE` Environment variables: `OLLAMA_DEBUG=2, AMD_LOG_LEVEL=3` System: ``` OS: Ubuntu 24.04.4 LTS Arch: x86_64 Kernel: 6.8.0-106-generic CPU Cores: 32 RAM: 124.5 GB ``` Downgrading to `0.17.0-rocm` seems to work. ### Relevant log output ```shell time=2026-03-18T00:43:29.654Z level=INFO source=routes.go:1727 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG-4 OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:<REDACTED>@v1.7.0/command.go:940 +0x85c fp=0xc000521e78 sp=0xc000521d58 pc=0x65141dc85cdc github.com/spf13/cobra.(*Command).ExecuteC(0xc000552908) github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000521f30 sp=0xc000521e78 pc=0x65141dc86525 github.com/spf13/cobra.(*Command).Execute(...) github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) github.com/spf13/cobra@v1.7.0/command.go:985 main.main() github.com/ollama/ollama/main.go:12 +0x4d fp=0xc000521f50 sp=0xc000521f30 pc=0x65141e9f038d runtime.main() runtime/proc.go:283 +0x29d fp=0xc000521fe0 sp=0xc000521f50 pc=0x65141dad871d runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000521fe8 sp=0xc000521fe0 pc=0x65141db13e61 goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000aafa8 sp=0xc0000aaf88 pc=0x65141db0beee runtime.goparkunlock(...) runtime/proc.go:441 runtime.forcegchelper() runtime/proc.go:348 +0xb8 fp=0xc0000aafe0 sp=0xc0000aafa8 pc=0x65141dad8a58 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000aafe8 sp=0xc0000aafe0 pc=0x65141db13e61 created by runtime.init.7 in goroutine 1 runtime/proc.go:336 +0x1a goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000ab780 sp=0xc0000ab760 pc=0x65141db0beee runtime.goparkunlock(...) runtime/proc.go:441 runtime.bgsweep(0xc0000d6000) runtime/mgcsweep.go:316 +0xdf fp=0xc0000ab7c8 sp=0xc0000ab780 pc=0x65141dac31ff runtime.gcenable.gowrap1() runtime/mgc.go:204 +0x25 fp=0xc0000ab7e0 sp=0xc0000ab7c8 pc=0x65141dab75e5 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ab7e8 sp=0xc0000ab7e0 pc=0x65141db13e61 created by runtime.gcenable in goroutine 1 runtime/mgc.go:204 +0x66 goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]: runtime.gopark(0x10000?, 0x65141efdc748?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000abf78 sp=0xc0000abf58 pc=0x65141db0beee runtime.goparkunlock(...) runtime/proc.go:441 runtime.(*scavengerState).park(0x65141fde7e00) runtime/mgcscavenge.go:425 +0x49 fp=0xc0000abfa8 sp=0xc0000abf78 pc=0x65141dac0c49 runtime.bgscavenge(0xc0000d6000) runtime/mgcscavenge.go:658 +0x59 fp=0xc0000abfc8 sp=0xc0000abfa8 pc=0x65141dac11d9 runtime.gcenable.gowrap2() runtime/mgc.go:205 +0x25 fp=0xc0000abfe0 sp=0xc0000abfc8 pc=0x65141dab7585 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000abfe8 sp=0xc0000abfe0 pc=0x65141db13e61 created by runtime.gcenable in goroutine 1 runtime/mgc.go:205 +0xa5 goroutine 5 gp=0xc000003dc0 m=nil [finalizer wait]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc0000aa688?) runtime/proc.go:435 +0xce fp=0xc0000aa630 sp=0xc0000aa610 pc=0x65141db0beee runtime.runfinq() runtime/mfinal.go:196 +0x107 fp=0xc0000aa7e0 sp=0xc0000aa630 pc=0x65141dab65a7 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000aa7e8 sp=0xc0000aa7e0 pc=0x65141db13e61 created by runtime.createfing in goroutine 1 runtime/mfinal.go:166 +0x3d goroutine 6 gp=0xc0002148c0 m=nil [chan receive]: runtime.gopark(0xc0001fdcc0?, 0xc000011128?, 0x60?, 0xc7?, 0x65141dbf28a8?) runtime/proc.go:435 +0xce fp=0xc0000ac718 sp=0xc0000ac6f8 pc=0x65141db0beee runtime.chanrecv(0xc000040380, 0x0, 0x1) runtime/chan.go:664 +0x445 fp=0xc0000ac790 sp=0xc0000ac718 pc=0x65141daa8085 runtime.chanrecv1(0x0?, 0x0?) runtime/chan.go:506 +0x12 fp=0xc0000ac7b8 sp=0xc0000ac790 pc=0x65141daa7c12 runtime.unique_runtime_registerUniqueMapCleanup.func2(...) runtime/mgc.go:1796 runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() runtime/mgc.go:1799 +0x2f fp=0xc0000ac7e0 sp=0xc0000ac7b8 pc=0x65141daba78f runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ac7e8 sp=0xc0000ac7e0 pc=0x65141db13e61 created by unique.runtime_registerUniqueMapCleanup in goroutine 1 runtime/mgc.go:1794 +0x85 goroutine 7 gp=0xc000214c40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000acf38 sp=0xc0000acf18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000acfc8 sp=0xc0000acf38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000acfe0 sp=0xc0000acfc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000acfe8 sp=0xc0000acfe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 8 gp=0xc000214e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000ad738 sp=0xc0000ad718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000ad7c8 sp=0xc0000ad738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000ad7e0 sp=0xc0000ad7c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000ad7e8 sp=0xc0000ad7e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 9 gp=0xc000214fc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000adf38 sp=0xc0000adf18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000adfc8 sp=0xc0000adf38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000adfe0 sp=0xc0000adfc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000adfe8 sp=0xc0000adfe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 10 gp=0xc000215180 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a6738 sp=0xc0000a6718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a67c8 sp=0xc0000a6738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a67e0 sp=0xc0000a67c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a67e8 sp=0xc0000a67e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 11 gp=0xc000215340 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a6f38 sp=0xc0000a6f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a6fc8 sp=0xc0000a6f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a6fe0 sp=0xc0000a6fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a6fe8 sp=0xc0000a6fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 12 gp=0xc000215500 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a7738 sp=0xc0000a7718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a77c8 sp=0xc0000a7738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a77e0 sp=0xc0000a77c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a77e8 sp=0xc0000a77e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 18 gp=0xc00031a000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000320738 sp=0xc000320718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0003207c8 sp=0xc000320738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0003207e0 sp=0xc0003207c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0003207e8 sp=0xc0003207e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 19 gp=0xc00031a1c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000320f38 sp=0xc000320f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000320fc8 sp=0xc000320f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000320fe0 sp=0xc000320fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000320fe8 sp=0xc000320fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 20 gp=0xc00031a380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000321738 sp=0xc000321718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0003217c8 sp=0xc000321738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0003217e0 sp=0xc0003217c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0003217e8 sp=0xc0003217e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 21 gp=0xc00031a540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000321f38 sp=0xc000321f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000321fc8 sp=0xc000321f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000321fe0 sp=0xc000321fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000321fe8 sp=0xc000321fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 22 gp=0xc00031a700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000322738 sp=0xc000322718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0003227c8 sp=0xc000322738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0003227e0 sp=0xc0003227c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0003227e8 sp=0xc0003227e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 34 gp=0xc000102380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031c738 sp=0xc00031c718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031c7c8 sp=0xc00031c738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031c7e0 sp=0xc00031c7c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031c7e8 sp=0xc00031c7e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 35 gp=0xc000102540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031cf38 sp=0xc00031cf18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031cfc8 sp=0xc00031cf38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031cfe0 sp=0xc00031cfc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031cfe8 sp=0xc00031cfe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 36 gp=0xc000102700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031d738 sp=0xc00031d718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031d7c8 sp=0xc00031d738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031d7e0 sp=0xc00031d7c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031d7e8 sp=0xc00031d7e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 37 gp=0xc0001028c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031df38 sp=0xc00031df18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031dfc8 sp=0xc00031df38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031dfe0 sp=0xc00031dfc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031dfe8 sp=0xc00031dfe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 38 gp=0xc000102a80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031e738 sp=0xc00031e718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031e7c8 sp=0xc00031e738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031e7e0 sp=0xc00031e7c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031e7e8 sp=0xc00031e7e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 39 gp=0xc000102c40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031ef38 sp=0xc00031ef18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031efc8 sp=0xc00031ef38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031efe0 sp=0xc00031efc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031efe8 sp=0xc00031efe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 40 gp=0xc000102e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031f738 sp=0xc00031f718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031f7c8 sp=0xc00031f738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031f7e0 sp=0xc00031f7c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031f7e8 sp=0xc00031f7e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 41 gp=0xc000102fc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00031ff38 sp=0xc00031ff18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc00031ffc8 sp=0xc00031ff38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00031ffe0 sp=0xc00031ffc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00031ffe8 sp=0xc00031ffe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 42 gp=0xc000103180 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000118738 sp=0xc000118718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0001187c8 sp=0xc000118738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0001187e0 sp=0xc0001187c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 13 gp=0xc0002156c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a7f38 sp=0xc0000a7f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a7fc8 sp=0xc0000a7f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a7fe0 sp=0xc0000a7fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a7fe8 sp=0xc0000a7fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 14 gp=0xc000215880 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a8738 sp=0xc0000a8718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a87c8 sp=0xc0000a8738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a87e0 sp=0xc0000a87c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a87e8 sp=0xc0000a87e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 15 gp=0xc000215a40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000a8f38 sp=0xc0000a8f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0000a8fc8 sp=0xc0000a8f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0000a8fe0 sp=0xc0000a8fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a8fe8 sp=0xc0000a8fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 23 gp=0xc00031a8c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000322f38 sp=0xc000322f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000322fc8 sp=0xc000322f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000322fe0 sp=0xc000322fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000322fe8 sp=0xc000322fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 24 gp=0xc00031aa80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000323738 sp=0xc000323718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0003237c8 sp=0xc000323738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0003237e0 sp=0xc0003237c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0003237e8 sp=0xc0003237e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 25 gp=0xc00031ac40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000323f38 sp=0xc000323f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000323fc8 sp=0xc000323f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000323fe0 sp=0xc000323fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000323fe8 sp=0xc000323fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 26 gp=0xc00031ae00 m=nil [GC worker (idle)]: runtime.gopark(0xcda9966b25?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000114738 sp=0xc000114718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0001147c8 sp=0xc000114738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0001147e0 sp=0xc0001147c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 27 gp=0xc00031afc0 m=nil [GC worker (idle)]: runtime.gopark(0xcda9965b0d?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000114f38 sp=0xc000114f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000114fc8 sp=0xc000114f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000114fe0 sp=0xc000114fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000114fe8 sp=0xc000114fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 28 gp=0xc00031b180 m=nil [GC worker (idle)]: runtime.gopark(0xcda9975372?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000115738 sp=0xc000115718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0001157c8 sp=0xc000115738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0001157e0 sp=0xc0001157c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0001157e8 sp=0xc0001157e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 29 gp=0xc00031b340 m=nil [GC worker (idle)]: runtime.gopark(0xcda9965c9d?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000115f38 sp=0xc000115f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000115fc8 sp=0xc000115f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000115fe0 sp=0xc000115fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 30 gp=0xc00031b500 m=nil [GC worker (idle)]: runtime.gopark(0xcda99625ed?, 0x1?, 0x3a?, 0xe9?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000116738 sp=0xc000116718 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc0001167c8 sp=0xc000116738 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0001167e0 sp=0xc0001167c8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0001167e8 sp=0xc0001167e0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 43 gp=0xc000103340 m=nil [GC worker (idle)]: runtime.gopark(0xcda9962bf1?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000118f38 sp=0xc000118f18 pc=0x65141db0beee runtime.gcBgMarkWorker(0xc0000415e0) runtime/mgc.go:1423 +0xe9 fp=0xc000118fc8 sp=0xc000118f38 pc=0x65141dab9aa9 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000118fe0 sp=0xc000118fc8 pc=0x65141dab9985 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000118fe8 sp=0xc000118fe0 pc=0x65141db13e61 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 16 gp=0xc000602fc0 m=nil [sync.WaitGroup.Wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0xe0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0000bda90 sp=0xc0000bda70 pc=0x65141db0beee runtime.goparkunlock(...) runtime/proc.go:441 runtime.semacquire1(0xc000263198, 0x0, 0x1, 0x0, 0x18) runtime/sema.go:188 +0x229 fp=0xc0000bdaf8 sp=0xc0000bda90 pc=0x65141daebce9 sync.runtime_SemacquireWaitGroup(0x0?) runtime/sema.go:110 +0x25 fp=0xc0000bdb30 sp=0xc0000bdaf8 pc=0x65141db0d825 sync.(*WaitGroup).Wait(0xc000263190?) sync/waitgroup.go:118 +0x48 fp=0xc0000bdb58 sp=0xc0000bdb30 pc=0x65141db1f8c8 github.com/ollama/ollama/runner/ollamarunner.(*Server).run(0xc0002630e0, {0x65141f3ac170, 0xc000578820}) github.com/ollama/ollama/runner/ollamarunner/runner.go:442 +0x45 fp=0xc0000bdfb8 sp=0xc0000bdb58 pc=0x65141e1255c5 github.com/ollama/ollama/runner/ollamarunner.Execute.gowrap1() github.com/ollama/ollama/runner/ollamarunner/runner.go:1424 +0x28 fp=0xc0000bdfe0 sp=0xc0000bdfb8 pc=0x65141e12ee48 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0000bdfe8 sp=0xc0000bdfe0 pc=0x65141db13e61 created by github.com/ollama/ollama/runner/ollamarunner.Execute in goroutine 1 github.com/ollama/ollama/runner/ollamarunner/runner.go:1424 +0x4c9 goroutine 51 gp=0xc000603340 m=nil [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0xb?) runtime/proc.go:435 +0xce fp=0xc00011b5d8 sp=0xc00011b5b8 pc=0x65141db0beee runtime.netpollblock(0x65141db2f798?, 0x1daa54a6?, 0x14?) runtime/netpoll.go:575 +0xf7 fp=0xc00011b610 sp=0xc00011b5d8 pc=0x65141dad1097 internal/poll.runtime_pollWait(0x74389520f5b8, 0x72) runtime/netpoll.go:351 +0x85 fp=0xc00011b630 sp=0xc00011b610 pc=0x65141db0b105 internal/poll.(*pollDesc).wait(0xc00011ec80?, 0xc000139661?, 0x0) internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00011b658 sp=0xc00011b630 pc=0x65141db93487 internal/poll.(*pollDesc).waitRead(...) internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc00011ec80, {0xc000139661, 0x1, 0x1}) internal/poll/fd_unix.go:165 +0x27a fp=0xc00011b6f0 sp=0xc00011b658 pc=0x65141db9477a net.(*netFD).Read(0xc00011ec80, {0xc000139661?, 0x0?, 0x0?}) net/fd_posix.go:55 +0x25 fp=0xc00011b738 sp=0xc00011b6f0 pc=0x65141dc09da5 net.(*conn).Read(0xc0000aea90, {0xc000139661?, 0x0?, 0x0?}) net/net.go:194 +0x45 fp=0xc00011b780 sp=0xc00011b738 pc=0x65141dc18165 net/http.(*connReader).backgroundRead(0xc000139650) net/http/server.go:690 +0x37 fp=0xc00011b7c8 sp=0xc00011b780 pc=0x65141de14057 net/http.(*connReader).startBackgroundRead.gowrap2() net/http/server.go:686 +0x25 fp=0xc00011b7e0 sp=0xc00011b7c8 pc=0x65141de13f85 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00011b7e8 sp=0xc00011b7e0 pc=0x65141db13e61 created by net/http.(*connReader).startBackgroundRead in goroutine 50 net/http/server.go:686 +0xb6 rax 0x743845a31948 rbx 0x74383472d0b0 rcx 0x1 rdx 0x743834892 rdi 0x74383472d0b0 rsi 0x74384c868320 rbp 0x74384c868320 rsp 0x74384c868280 r8 0x7438340008e0 r9 0x7 r10 0x743834892160 r11 0xc2f59f465ad1bfdd r12 0x0 r13 0x74383472d598 r14 0x7438348c9c50 r15 0x743834892160 rip 0x7438458b7170 rflags 0x10202 cs 0x33 fs 0x0 gs 0x0 time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:462 msg="runner exited" OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" extra_envs="map[GGML_CUDA_INIT:1 ROCR_VISIBLE_DEVICES:0]" code=2 time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:467 msg="runner enumerated devices" OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" devices=[] time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:437 msg="bootstrap discovery took" duration=74.526671ms OLLAMA_LIBRARY_PATH="[/usr/lib/ollama /usr/lib/ollama/rocm]" extra_envs="map[GGML_CUDA_INIT:1 ROCR_VISIBLE_DEVICES:0]" time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:153 msg="filtering device which didn't fully initialize" id=0 libdir=/usr/lib/ollama/rocm pci_id=0000:c5:00.0 library=ROCm time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:174 msg="supported GPU library combinations before filtering" supported=map[] time=2026-03-18T00:43:29.795Z level=TRACE source=runner.go:183 msg="removing unsupported or overlapping GPU combination" libDir=/usr/lib/ollama/rocm description="Radeon 8060S Graphics" compute=gfx1151 pci_id=0000:c5:00.0 time=2026-03-18T00:43:29.795Z level=DEBUG source=runner.go:40 msg="GPU bootstrap discovery took" duration=141.02312ms time=2026-03-18T00:43:29.795Z level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="124.5 GiB" available="124.4 GiB" time=2026-03-18T00:43:29.795Z level=INFO source=routes.go:1832 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096 ``` ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.18.*
GiteaMirror added the bug label 2026-05-05 02:17:43 -05:00
GiteaMirror commented 2026-05-05 02:17:47 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 18, 2026):

Too much has been <REDACTED>. At least include the GPU detection.

<!-- gh-comment-id:4079058978 --> @rick-github commented on GitHub (Mar 18, 2026): Too much has been `<REDACTED>`. At least include the GPU detection.
GiteaMirror commented 2026-05-05 02:17:48 -05:00
Author
Owner
Copy Link

@nishtahir commented on GitHub (Mar 18, 2026):

The only redacted content to my knowledge is the port number. That's all the logs that gets printed. Is there something else I should be looking for?

<!-- gh-comment-id:4079067713 --> @nishtahir commented on GitHub (Mar 18, 2026): The only redacted content to my knowledge is the port number. That's all the logs that gets printed. Is there something else I should be looking for?
GiteaMirror commented 2026-05-05 02:17:48 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 18, 2026):

Run this and post the output:

docker run --rm --device=/dev/kfd --device=/dev/dri -e OLLAMA_DEBUG=1 ollama/ollama:0.18.0-rocm
<!-- gh-comment-id:4079074887 --> @rick-github commented on GitHub (Mar 18, 2026): Run this and post the output: ``` docker run --rm --device=/dev/kfd --device=/dev/dri -e OLLAMA_DEBUG=1 ollama/ollama:0.18.0-rocm ```
GiteaMirror commented 2026-05-05 02:17:49 -05:00
Author
Owner
Copy Link

@nishtahir commented on GitHub (Mar 18, 2026):

The issue may have been the kernel version. Downgrading to 6.17 as described here https://github.com/ollama/ollama/issues/13920#issuecomment-3817128040 seems to have fixed my issue

<!-- gh-comment-id:4079358591 --> @nishtahir commented on GitHub (Mar 18, 2026): The issue may have been the kernel version. Downgrading to 6.17 as described here https://github.com/ollama/ollama/issues/13920#issuecomment-3817128040 seems to have fixed my issue
GiteaMirror commented 2026-05-05 02:17:49 -05:00
Author
Owner
Copy Link

@uipoet commented on GitHub (Mar 19, 2026):

In my case with NixOS and latest kernel when rebooting, the ollama service would start up before the unified RAM was initialized. I added a preStart to the existing service that waits until /sys/class/drm/*/device/mem_info_vram_total is above 0. This prevents starting up in cpu mode where I would have to manually restart the ollama service after a reboot.

<!-- gh-comment-id:4094067068 --> @uipoet commented on GitHub (Mar 19, 2026): In my case with NixOS and latest kernel when rebooting, the ollama service would start up before the unified RAM was initialized. I added a preStart to the existing service that waits until `/sys/class/drm/*/device/mem_info_vram_total` is above 0. This prevents starting up in cpu mode where I would have to manually restart the ollama service after a reboot.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#71664
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?