github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10590] Llama 4 Scout runtime error #32728

New Issue
Closed
opened 2026-04-22 14:33:48 -05:00 by GiteaMirror · 20 comments
GiteaMirror commented 2026-04-22 14:33:48 -05:00
Owner
Copy Link

Originally created by @renepeinl on GitHub (May 6, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10590

Originally assigned to: @jmorganca on GitHub.

What is the issue?

I'm running the latest ollama on Ubuntu Linux 22.04 LTS with CUDA 12.2 and Python 3.12 in a docker container (Jupyterhub). I've tested 10 different models from Phi4 to Llama 3.3 70B. All are working fine, including the MoE Qwen 30B. When I try to run Llama 4 Scout, I get the error message printed below, rather sooner than later. My test dataset has 109 questions. Sometimes I get 6-7 responses, sometimes only a single one, before the error appears. I'm using the http API with an ollama client in Python on the same machine.
I've tried it both on an A100 80GB as well as on an H100 80 GB.

Relevant log output

created by net/http.(*connReader).startBackgroundRead in goroutine 133
        net/http/server.go:686 +0xb6

rax    0x0
rbx    0x7fa4baffd640
rcx    0x7fb9fa5a89fc
rdx    0x6
rdi    0xabd
rsi    0xb51
rbp    0xb51
rsp    0x7fa4baffc8b0
r8     0x7fa4baffc980
r9     0x7fa4baffc950
r10    0x8
r11    0x246
r12    0x6
r13    0x16
r14    0x7fb990893888
r15    0x7fa684002f30
rip    0x7fb9fa5a89fc
rflags 0x246
cs     0x33
fs     0x0
gs     0x0
[GIN] 2025/05/06 - 13:17:44 | 500 |  1.396443419s |       127.0.0.1 | POST     "/api/chat"
time=2025-05-06T13:17:44.586Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 2"

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @renepeinl on GitHub (May 6, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10590 Originally assigned to: @jmorganca on GitHub. ### What is the issue? I'm running the latest ollama on Ubuntu Linux 22.04 LTS with CUDA 12.2 and Python 3.12 in a docker container (Jupyterhub). I've tested 10 different models from Phi4 to Llama 3.3 70B. All are working fine, including the MoE Qwen 30B. When I try to run Llama 4 Scout, I get the error message printed below, rather sooner than later. My test dataset has 109 questions. Sometimes I get 6-7 responses, sometimes only a single one, before the error appears. I'm using the http API with an ollama client in Python on the same machine. I've tried it both on an A100 80GB as well as on an H100 80 GB. ### Relevant log output ```shell created by net/http.(*connReader).startBackgroundRead in goroutine 133 net/http/server.go:686 +0xb6 rax 0x0 rbx 0x7fa4baffd640 rcx 0x7fb9fa5a89fc rdx 0x6 rdi 0xabd rsi 0xb51 rbp 0xb51 rsp 0x7fa4baffc8b0 r8 0x7fa4baffc980 r9 0x7fa4baffc950 r10 0x8 r11 0x246 r12 0x6 r13 0x16 r14 0x7fb990893888 r15 0x7fa684002f30 rip 0x7fb9fa5a89fc rflags 0x246 cs 0x33 fs 0x0 gs 0x0 [GIN] 2025/05/06 - 13:17:44 | 500 | 1.396443419s | 127.0.0.1 | POST "/api/chat" time=2025-05-06T13:17:44.586Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 2" ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-22 14:33:48 -05:00
GiteaMirror commented 2026-04-22 14:33:50 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

A full log will make it easier to debug.

<!-- gh-comment-id:2854777857 --> @rick-github commented on GitHub (May 6, 2025): A full log will make it easier to debug.
GiteaMirror commented 2026-04-22 14:33:50 -05:00
Author
Owner
Copy Link

@renepeinl commented on GitHub (May 6, 2025):

The log looks very repetitive and truncated in my console
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 55 gp=0xc0002288c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000220f38 sp=0xc000220f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000220fc8 sp=0xc000220f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000220fe0 sp=0xc000220fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000220fe8 sp=0xc000220fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 56 gp=0xc000228a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000221738 sp=0xc000221718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002217c8 sp=0xc000221738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002217e0 sp=0xc0002217c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002217e8 sp=0xc0002217e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 57 gp=0xc000228c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000221f38 sp=0xc000221f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000221fc8 sp=0xc000221f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000221fe0 sp=0xc000221fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000221fe8 sp=0xc000221fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 58 gp=0xc000228e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022e738 sp=0xc00022e718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022e7c8 sp=0xc00022e738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022e7e0 sp=0xc00022e7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022e7e8 sp=0xc00022e7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 59 gp=0xc000228fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022ef38 sp=0xc00022ef18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022efc8 sp=0xc00022ef38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022efe0 sp=0xc00022efc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022efe8 sp=0xc00022efe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 60 gp=0xc000229180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022f738 sp=0xc00022f718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022f7c8 sp=0xc00022f738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022f7e0 sp=0xc00022f7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022f7e8 sp=0xc00022f7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 61 gp=0xc000229340 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022ff38 sp=0xc00022ff18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022ffc8 sp=0xc00022ff38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022ffe0 sp=0xc00022ffc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022ffe8 sp=0xc00022ffe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 71 gp=0xc000510540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00050af38 sp=0xc00050af18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00050afc8 sp=0xc00050af38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00050afe0 sp=0xc00050afc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 72 gp=0xc000510700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00050b738 sp=0xc00050b718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00050b7c8 sp=0xc00050b738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00050b7e0 sp=0xc00050b7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 73 gp=0xc0005108c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00050bf38 sp=0xc00050bf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00050bfc8 sp=0xc00050bf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00050bfe0 sp=0xc00050bfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 74 gp=0xc000510a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022a738 sp=0xc00022a718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022a7c8 sp=0xc00022a738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022a7e0 sp=0xc00022a7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022a7e8 sp=0xc00022a7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 14 gp=0xc0002e96c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000218738 sp=0xc000218718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002187c8 sp=0xc000218738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002187e0 sp=0xc0002187c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002187e8 sp=0xc0002187e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 15 gp=0xc0002e9880 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000218f38 sp=0xc000218f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000218fc8 sp=0xc000218f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000218fe0 sp=0xc000218fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000218fe8 sp=0xc000218fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 16 gp=0xc0002e9a40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000219738 sp=0xc000219718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002197c8 sp=0xc000219738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002197e0 sp=0xc0002197c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002197e8 sp=0xc0002197e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 82 gp=0xc0002e9c00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000219f38 sp=0xc000219f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000219fc8 sp=0xc000219f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000219fe0 sp=0xc000219fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000219fe8 sp=0xc000219fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 75 gp=0xc000510c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022af38 sp=0xc00022af18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022afc8 sp=0xc00022af38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022afe0 sp=0xc00022afc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022afe8 sp=0xc00022afe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 83 gp=0xc0002e9dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049e738 sp=0xc00049e718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049e7c8 sp=0xc00049e738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049e7e0 sp=0xc00049e7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049e7e8 sp=0xc00049e7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 84 gp=0xc0004a2000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049ef38 sp=0xc00049ef18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049efc8 sp=0xc00049ef38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049efe0 sp=0xc00049efc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049efe8 sp=0xc00049efe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 62 gp=0xc000229500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000230738 sp=0xc000230718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002307c8 sp=0xc000230738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002307e0 sp=0xc0002307c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002307e8 sp=0xc0002307e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 76 gp=0xc000510e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022b738 sp=0xc00022b718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022b7c8 sp=0xc00022b738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022b7e0 sp=0xc00022b7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022b7e8 sp=0xc00022b7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 85 gp=0xc0004a21c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049f738 sp=0xc00049f718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049f7c8 sp=0xc00049f738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049f7e0 sp=0xc00049f7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049f7e8 sp=0xc00049f7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 63 gp=0xc0002296c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000230f38 sp=0xc000230f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000230fc8 sp=0xc000230f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000230fe0 sp=0xc000230fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000230fe8 sp=0xc000230fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 77 gp=0xc000510fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022bf38 sp=0xc00022bf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022bfc8 sp=0xc00022bf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022bfe0 sp=0xc00022bfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022bfe8 sp=0xc00022bfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 86 gp=0xc0004a2380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049ff38 sp=0xc00049ff18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049ffc8 sp=0xc00049ff38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049ffe0 sp=0xc00049ffc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 64 gp=0xc000229880 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000231738 sp=0xc000231718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002317c8 sp=0xc000231738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002317e0 sp=0xc0002317c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002317e8 sp=0xc0002317e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 78 gp=0xc000511180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022c738 sp=0xc00022c718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022c7c8 sp=0xc00022c738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022c7e0 sp=0xc00022c7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022c7e8 sp=0xc00022c7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 87 gp=0xc0004a2540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc0004a0738 sp=0xc0004a0718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0004a07c8 sp=0xc0004a0738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0004a07e0 sp=0xc0004a07c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a07e8 sp=0xc0004a07e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 65 gp=0xc000229a40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000231f38 sp=0xc000231f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000231fc8 sp=0xc000231f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000231fe0 sp=0xc000231fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000231fe8 sp=0xc000231fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 79 gp=0xc000511340 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022cf38 sp=0xc00022cf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022cfc8 sp=0xc00022cf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022cfe0 sp=0xc00022cfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022cfe8 sp=0xc00022cfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 88 gp=0xc0004a2700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc0004a0f38 sp=0xc0004a0f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0004a0fc8 sp=0xc0004a0f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0004a0fe0 sp=0xc0004a0fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a0fe8 sp=0xc0004a0fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 98 gp=0xc000229c00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049a738 sp=0xc00049a718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049a7c8 sp=0xc00049a738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049a7e0 sp=0xc00049a7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049a7e8 sp=0xc00049a7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 80 gp=0xc000511500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022d738 sp=0xc00022d718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022d7c8 sp=0xc00022d738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022d7e0 sp=0xc00022d7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022d7e8 sp=0xc00022d7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 89 gp=0xc0004a28c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc0004a1738 sp=0xc0004a1718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0004a17c8 sp=0xc0004a1738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0004a17e0 sp=0xc0004a17c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a17e8 sp=0xc0004a17e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 99 gp=0xc000229dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049af38 sp=0xc00049af18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049afc8 sp=0xc00049af38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049afe0 sp=0xc00049afc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049afe8 sp=0xc00049afe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 81 gp=0xc0005116c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00022df38 sp=0xc00022df18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00022dfc8 sp=0xc00022df38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00022dfe0 sp=0xc00022dfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00022dfe8 sp=0xc00022dfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 90 gp=0xc0004a2a80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc0004a1f38 sp=0xc0004a1f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0004a1fc8 sp=0xc0004a1f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0004a1fe0 sp=0xc0004a1fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 100 gp=0xc000232000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049b738 sp=0xc00049b718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049b7c8 sp=0xc00049b738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049b7e0 sp=0xc00049b7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049b7e8 sp=0xc00049b7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 114 gp=0xc000511880 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000518738 sp=0xc000518718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005187c8 sp=0xc000518738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005187e0 sp=0xc0005187c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005187e8 sp=0xc0005187e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 91 gp=0xc0004a2c40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000514738 sp=0xc000514718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005147c8 sp=0xc000514738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005147e0 sp=0xc0005147c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005147e8 sp=0xc0005147e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 101 gp=0xc0002321c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049bf38 sp=0xc00049bf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049bfc8 sp=0xc00049bf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049bfe0 sp=0xc00049bfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 102 gp=0xc000232380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049c738 sp=0xc00049c718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049c7c8 sp=0xc00049c738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049c7e0 sp=0xc00049c7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049c7e8 sp=0xc00049c7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 103 gp=0xc000232540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049cf38 sp=0xc00049cf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049cfc8 sp=0xc00049cf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049cfe0 sp=0xc00049cfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049cfe8 sp=0xc00049cfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 104 gp=0xc000232700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049d738 sp=0xc00049d718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049d7c8 sp=0xc00049d738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049d7e0 sp=0xc00049d7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049d7e8 sp=0xc00049d7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 105 gp=0xc0002328c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00049df38 sp=0xc00049df18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00049dfc8 sp=0xc00049df38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00049dfe0 sp=0xc00049dfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00049dfe8 sp=0xc00049dfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 115 gp=0xc000511a40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000518f38 sp=0xc000518f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000518fc8 sp=0xc000518f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000518fe0 sp=0xc000518fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000518fe8 sp=0xc000518fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 92 gp=0xc0004a2e00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000514f38 sp=0xc000514f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000514fc8 sp=0xc000514f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000514fe0 sp=0xc000514fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000514fe8 sp=0xc000514fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 93 gp=0xc0004a2fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000515738 sp=0xc000515718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005157c8 sp=0xc000515738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005157e0 sp=0xc0005157c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005157e8 sp=0xc0005157e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 94 gp=0xc0004a3180 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x4a?, 0x5d?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000515f38 sp=0xc000515f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000515fc8 sp=0xc000515f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000515fe0 sp=0xc000515fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000515fe8 sp=0xc000515fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 95 gp=0xc0004a3340 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x7d?, 0xbe?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000516738 sp=0xc000516718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005167c8 sp=0xc000516738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005167e0 sp=0xc0005167c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005167e8 sp=0xc0005167e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 96 gp=0xc0004a3500 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c0434ea?, 0x1?, 0x9f?, 0xb1?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000516f38 sp=0xc000516f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000516fc8 sp=0xc000516f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000516fe0 sp=0xc000516fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000516fe8 sp=0xc000516fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 97 gp=0xc0004a36c0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03f648?, 0x1?, 0x2c?, 0xb9?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000517738 sp=0xc000517718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005177c8 sp=0xc000517738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005177e0 sp=0xc0005177c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005177e8 sp=0xc0005177e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 130 gp=0xc0004a3880 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x47?, 0x4d?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000517f38 sp=0xc000517f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000517fc8 sp=0xc000517f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000517fe0 sp=0xc000517fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000517fe8 sp=0xc000517fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 116 gp=0xc000511c00 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03c923?, 0x1?, 0x5?, 0xed?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000519738 sp=0xc000519718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0005197c8 sp=0xc000519738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0005197e0 sp=0xc0005197c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0005197e8 sp=0xc0005197e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 117 gp=0xc000511dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0xa0?, 0x5e?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000519f38 sp=0xc000519f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000519fc8 sp=0xc000519f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000519fe0 sp=0xc000519fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000519fe8 sp=0xc000519fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 118 gp=0xc00051c000 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0xed?, 0x49?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00051a738 sp=0xc00051a718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00051a7c8 sp=0xc00051a738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00051a7e0 sp=0xc00051a7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00051a7e8 sp=0xc00051a7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 119 gp=0xc00051c1c0 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x6e?, 0x1e?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00051af38 sp=0xc00051af18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00051afc8 sp=0xc00051af38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00051afe0 sp=0xc00051afc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00051afe8 sp=0xc00051afe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 106 gp=0xc000232a80 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c04234f?, 0x1?, 0xd3?, 0xb0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000238738 sp=0xc000238718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002387c8 sp=0xc000238738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002387e0 sp=0xc0002387c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002387e8 sp=0xc0002387e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 107 gp=0xc000232c40 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bdc1d36?, 0x1?, 0x57?, 0x19?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000238f38 sp=0xc000238f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000238fc8 sp=0xc000238f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000238fe0 sp=0xc000238fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000238fe8 sp=0xc000238fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 108 gp=0xc000232e00 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bdc66ea?, 0x1?, 0xfb?, 0xb?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000239738 sp=0xc000239718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002397c8 sp=0xc000239738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002397e0 sp=0xc0002397c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002397e8 sp=0xc0002397e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 109 gp=0xc000232fc0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03ce10?, 0x1?, 0x1d?, 0x7b?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000239f38 sp=0xc000239f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000239fc8 sp=0xc000239f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000239fe0 sp=0xc000239fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000239fe8 sp=0xc000239fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 110 gp=0xc000233180 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0xee?, 0xc7?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00023a738 sp=0xc00023a718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00023a7c8 sp=0xc00023a738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00023a7e0 sp=0xc00023a7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00023a7e8 sp=0xc00023a7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 111 gp=0xc000233340 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x63?, 0x14?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00023af38 sp=0xc00023af18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00023afc8 sp=0xc00023af38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00023afe0 sp=0xc00023afc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00023afe8 sp=0xc00023afe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 112 gp=0xc000233500 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bdf0e6a?, 0x1?, 0x6e?, 0x36?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00023b738 sp=0xc00023b718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00023b7c8 sp=0xc00023b738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00023b7e0 sp=0xc00023b7c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00023b7e8 sp=0xc00023b7e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 113 gp=0xc0002336c0 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0xb2?, 0xc3?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc00023bf38 sp=0xc00023bf18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc00023bfc8 sp=0xc00023bf38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc00023bfe0 sp=0xc00023bfc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc00023bfe8 sp=0xc00023bfe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 146 gp=0xc000233880 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bde9ae4?, 0x1?, 0x7c?, 0xd0?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000234738 sp=0xc000234718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002347c8 sp=0xc000234738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002347e0 sp=0xc0002347c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002347e8 sp=0xc0002347e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 147 gp=0xc000233a40 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x3?, 0x6e?, 0x53?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000234f38 sp=0xc000234f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000234fc8 sp=0xc000234f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000234fe0 sp=0xc000234fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000234fe8 sp=0xc000234fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 148 gp=0xc000233c00 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x10?, 0xe7?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000235738 sp=0xc000235718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002357c8 sp=0xc000235738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002357e0 sp=0xc0002357c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002357e8 sp=0xc0002357e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 149 gp=0xc000233dc0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bde3d95?, 0x3?, 0x22?, 0x1?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000235f38 sp=0xc000235f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000235fc8 sp=0xc000235f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000235fe0 sp=0xc000235fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000235fe8 sp=0xc000235fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 150 gp=0xc00023c000 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03cffb?, 0x1?, 0x28?, 0x78?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000236738 sp=0xc000236718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002367c8 sp=0xc000236738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002367e0 sp=0xc0002367c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002367e8 sp=0xc0002367e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 151 gp=0xc00023c1c0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c0411f0?, 0x1?, 0x99?, 0xa3?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000236f38 sp=0xc000236f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000236fc8 sp=0xc000236f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000236fe0 sp=0xc000236fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000236fe8 sp=0xc000236fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 152 gp=0xc00023c380 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03a72d?, 0x3?, 0x3b?, 0xad?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000237738 sp=0xc000237718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002377c8 sp=0xc000237738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002377e0 sp=0xc0002377c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002377e8 sp=0xc0002377e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 153 gp=0xc00023c540 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03e041?, 0x1?, 0x42?, 0x8b?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000237f38 sp=0xc000237f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000237fc8 sp=0xc000237f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000237fe0 sp=0xc000237fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000237fe8 sp=0xc000237fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 154 gp=0xc00023c700 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x3?, 0x3c?, 0x44?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000242738 sp=0xc000242718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002427c8 sp=0xc000242738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002427e0 sp=0xc0002427c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002427e8 sp=0xc0002427e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 155 gp=0xc00023c8c0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4bdc4667?, 0x3?, 0x17?, 0x9f?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000242f38 sp=0xc000242f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000242fc8 sp=0xc000242f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000242fe0 sp=0xc000242fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000242fe8 sp=0xc000242fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 156 gp=0xc00023ca80 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c0404fe?, 0x1?, 0xdc?, 0x54?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000243738 sp=0xc000243718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002437c8 sp=0xc000243738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002437e0 sp=0xc0002437c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002437e8 sp=0xc0002437e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 157 gp=0xc00023cc40 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4be1495a?, 0x1?, 0x5?, 0x5b?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000243f38 sp=0xc000243f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000243fc8 sp=0xc000243f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000243fe0 sp=0xc000243fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000243fe8 sp=0xc000243fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 158 gp=0xc00023ce00 m=nil [GC worker (idle)]:
runtime.gopark(0x56221bb67c00?, 0x1?, 0x9c?, 0x3d?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000244738 sp=0xc000244718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002447c8 sp=0xc000244738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002447e0 sp=0xc0002447c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002447e8 sp=0xc0002447e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 159 gp=0xc00023cfc0 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4c03b679?, 0x1?, 0x48?, 0xb5?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000244f38 sp=0xc000244f18 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc000244fc8 sp=0xc000244f38 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc000244fe0 sp=0xc000244fc8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc000244fe8 sp=0xc000244fe0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 160 gp=0xc00023d180 m=nil [GC worker (idle)]:
runtime.gopark(0x10465c4be15b1d?, 0x1?, 0x61?, 0xdd?, 0x0?)
runtime/proc.go:435 +0xce fp=0xc000245738 sp=0xc000245718 pc=0x562219f2374e
runtime.gcBgMarkWorker(0xc000045500)
runtime/mgc.go:1423 +0xe9 fp=0xc0002457c8 sp=0xc000245738 pc=0x562219ed0f49
runtime.gcBgMarkStartWorkers.gowrap1()
runtime/mgc.go:1339 +0x25 fp=0xc0002457e0 sp=0xc0002457c8 pc=0x562219ed0e25
runtime.goexit({})
runtime/asm_amd64.s:1700 +0x1 fp=0xc0002457e8 sp=0xc0002457e0 pc=0x562219f2ae81
created by runtime.gcBgMarkStartWorkers in goroutine 1
runtime/mgc.go:1339 +0x105

goroutine 133 gp=0xc000582a80 m=nil [select]:
runtime.gopark(0xc000053a10?, 0x2?, 0x0?, 0xe6?, 0xc000053874?)
runtime/proc.go:435 +0xce fp=0xc000053680 sp=0xc000053660 pc=0x562219f2374e
runtime.selectgo(0xc000053a10, 0xc000053870, 0x167?, 0x0, 0x4?, 0x1)
runtime/select.go:351 +0x837 fp=0xc0000537b8 sp=0xc000053680 pc=0x562219f02037
github.com/ollama/ollama/runner/ollamarunner.(*Server).completion(0xc00057ec60, {0x56221b240978, 0xc000000000}, 0xc00311a140)
github.com/ollama/ollama/runner/ollamarunner/runner.go:664 +0xb65 fp=0xc000053ac0 sp=0xc0000537b8 pc=0x56221a3edc25
github.com/ollama/ollama/runner/ollamarunner.(*Server).completion-fm({0x56221b240978?, 0xc000000000?}, 0xc000625b40?)
:1 +0x36 fp=0xc000053af0 sp=0xc000053ac0 pc=0x56221a3f04f6
net/http.HandlerFunc.ServeHTTP(0xc00056a780?, {0x56221b240978?, 0xc000000000?}, 0xc000625b60?)
net/http/server.go:2294 +0x29 fp=0xc000053b18 sp=0xc000053af0 pc=0x56221a221d69
net/http.(*ServeMux).ServeHTTP(0x562219ec7f65?, {0x56221b240978, 0xc000000000}, 0xc00311a140)
net/http/server.go:2822 +0x1c4 fp=0xc000053b68 sp=0xc000053b18 pc=0x56221a223c64
net/http.serverHandler.ServeHTTP({0x56221b23d050?}, {0x56221b240978?, 0xc000000000?}, 0x1?)
net/http/server.go:3301 +0x8e fp=0xc000053b98 sp=0xc000053b68 pc=0x56221a2416ee
net/http.(*conn).serve(0xc00026a000, {0x56221b242ae8, 0xc0005c7260})
net/http/server.go:2102 +0x625 fp=0xc000053fb8 sp=0xc000053b98 pc=0x56221a220265
net/http.(*Server).Serve.gowrap3()
net/http/server.go:3454 +0x28 fp=0xc000053fe0 sp=0xc000053fb8 pc=0x56221a225b28
runtime.goexit({})

<!-- gh-comment-id:2854809758 --> @renepeinl commented on GitHub (May 6, 2025): The log looks very repetitive and truncated in my console created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 55 gp=0xc0002288c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000220f38 sp=0xc000220f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000220fc8 sp=0xc000220f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000220fe0 sp=0xc000220fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000220fe8 sp=0xc000220fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 56 gp=0xc000228a80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000221738 sp=0xc000221718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002217c8 sp=0xc000221738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002217e0 sp=0xc0002217c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002217e8 sp=0xc0002217e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 57 gp=0xc000228c40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000221f38 sp=0xc000221f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000221fc8 sp=0xc000221f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000221fe0 sp=0xc000221fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000221fe8 sp=0xc000221fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 58 gp=0xc000228e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022e738 sp=0xc00022e718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022e7c8 sp=0xc00022e738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022e7e0 sp=0xc00022e7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022e7e8 sp=0xc00022e7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 59 gp=0xc000228fc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022ef38 sp=0xc00022ef18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022efc8 sp=0xc00022ef38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022efe0 sp=0xc00022efc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022efe8 sp=0xc00022efe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 60 gp=0xc000229180 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022f738 sp=0xc00022f718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022f7c8 sp=0xc00022f738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022f7e0 sp=0xc00022f7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022f7e8 sp=0xc00022f7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 61 gp=0xc000229340 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022ff38 sp=0xc00022ff18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022ffc8 sp=0xc00022ff38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022ffe0 sp=0xc00022ffc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022ffe8 sp=0xc00022ffe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 71 gp=0xc000510540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00050af38 sp=0xc00050af18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00050afc8 sp=0xc00050af38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00050afe0 sp=0xc00050afc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 72 gp=0xc000510700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00050b738 sp=0xc00050b718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00050b7c8 sp=0xc00050b738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00050b7e0 sp=0xc00050b7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 73 gp=0xc0005108c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00050bf38 sp=0xc00050bf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00050bfc8 sp=0xc00050bf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00050bfe0 sp=0xc00050bfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 74 gp=0xc000510a80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022a738 sp=0xc00022a718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022a7c8 sp=0xc00022a738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022a7e0 sp=0xc00022a7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022a7e8 sp=0xc00022a7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 14 gp=0xc0002e96c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000218738 sp=0xc000218718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002187c8 sp=0xc000218738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002187e0 sp=0xc0002187c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002187e8 sp=0xc0002187e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 15 gp=0xc0002e9880 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000218f38 sp=0xc000218f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000218fc8 sp=0xc000218f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000218fe0 sp=0xc000218fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000218fe8 sp=0xc000218fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 16 gp=0xc0002e9a40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000219738 sp=0xc000219718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002197c8 sp=0xc000219738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002197e0 sp=0xc0002197c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002197e8 sp=0xc0002197e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 82 gp=0xc0002e9c00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000219f38 sp=0xc000219f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000219fc8 sp=0xc000219f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000219fe0 sp=0xc000219fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000219fe8 sp=0xc000219fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 75 gp=0xc000510c40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022af38 sp=0xc00022af18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022afc8 sp=0xc00022af38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022afe0 sp=0xc00022afc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022afe8 sp=0xc00022afe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 83 gp=0xc0002e9dc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049e738 sp=0xc00049e718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049e7c8 sp=0xc00049e738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049e7e0 sp=0xc00049e7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049e7e8 sp=0xc00049e7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 84 gp=0xc0004a2000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049ef38 sp=0xc00049ef18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049efc8 sp=0xc00049ef38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049efe0 sp=0xc00049efc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049efe8 sp=0xc00049efe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 62 gp=0xc000229500 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000230738 sp=0xc000230718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002307c8 sp=0xc000230738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002307e0 sp=0xc0002307c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002307e8 sp=0xc0002307e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 76 gp=0xc000510e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022b738 sp=0xc00022b718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022b7c8 sp=0xc00022b738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022b7e0 sp=0xc00022b7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022b7e8 sp=0xc00022b7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 85 gp=0xc0004a21c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049f738 sp=0xc00049f718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049f7c8 sp=0xc00049f738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049f7e0 sp=0xc00049f7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049f7e8 sp=0xc00049f7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 63 gp=0xc0002296c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000230f38 sp=0xc000230f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000230fc8 sp=0xc000230f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000230fe0 sp=0xc000230fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000230fe8 sp=0xc000230fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 77 gp=0xc000510fc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022bf38 sp=0xc00022bf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022bfc8 sp=0xc00022bf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022bfe0 sp=0xc00022bfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022bfe8 sp=0xc00022bfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 86 gp=0xc0004a2380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049ff38 sp=0xc00049ff18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049ffc8 sp=0xc00049ff38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049ffe0 sp=0xc00049ffc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049ffe8 sp=0xc00049ffe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 64 gp=0xc000229880 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000231738 sp=0xc000231718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002317c8 sp=0xc000231738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002317e0 sp=0xc0002317c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002317e8 sp=0xc0002317e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 78 gp=0xc000511180 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022c738 sp=0xc00022c718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022c7c8 sp=0xc00022c738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022c7e0 sp=0xc00022c7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022c7e8 sp=0xc00022c7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 87 gp=0xc0004a2540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0004a0738 sp=0xc0004a0718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0004a07c8 sp=0xc0004a0738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0004a07e0 sp=0xc0004a07c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a07e8 sp=0xc0004a07e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 65 gp=0xc000229a40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000231f38 sp=0xc000231f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000231fc8 sp=0xc000231f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000231fe0 sp=0xc000231fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000231fe8 sp=0xc000231fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 79 gp=0xc000511340 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022cf38 sp=0xc00022cf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022cfc8 sp=0xc00022cf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022cfe0 sp=0xc00022cfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022cfe8 sp=0xc00022cfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 88 gp=0xc0004a2700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0004a0f38 sp=0xc0004a0f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0004a0fc8 sp=0xc0004a0f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0004a0fe0 sp=0xc0004a0fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a0fe8 sp=0xc0004a0fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 98 gp=0xc000229c00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049a738 sp=0xc00049a718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049a7c8 sp=0xc00049a738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049a7e0 sp=0xc00049a7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049a7e8 sp=0xc00049a7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 80 gp=0xc000511500 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022d738 sp=0xc00022d718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022d7c8 sp=0xc00022d738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022d7e0 sp=0xc00022d7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022d7e8 sp=0xc00022d7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 89 gp=0xc0004a28c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0004a1738 sp=0xc0004a1718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0004a17c8 sp=0xc0004a1738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0004a17e0 sp=0xc0004a17c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a17e8 sp=0xc0004a17e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 99 gp=0xc000229dc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049af38 sp=0xc00049af18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049afc8 sp=0xc00049af38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049afe0 sp=0xc00049afc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049afe8 sp=0xc00049afe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 81 gp=0xc0005116c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00022df38 sp=0xc00022df18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00022dfc8 sp=0xc00022df38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00022dfe0 sp=0xc00022dfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00022dfe8 sp=0xc00022dfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 90 gp=0xc0004a2a80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc0004a1f38 sp=0xc0004a1f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0004a1fc8 sp=0xc0004a1f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0004a1fe0 sp=0xc0004a1fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0004a1fe8 sp=0xc0004a1fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 100 gp=0xc000232000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049b738 sp=0xc00049b718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049b7c8 sp=0xc00049b738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049b7e0 sp=0xc00049b7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049b7e8 sp=0xc00049b7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 114 gp=0xc000511880 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000518738 sp=0xc000518718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005187c8 sp=0xc000518738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005187e0 sp=0xc0005187c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005187e8 sp=0xc0005187e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 91 gp=0xc0004a2c40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000514738 sp=0xc000514718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005147c8 sp=0xc000514738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005147e0 sp=0xc0005147c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005147e8 sp=0xc0005147e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 101 gp=0xc0002321c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049bf38 sp=0xc00049bf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049bfc8 sp=0xc00049bf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049bfe0 sp=0xc00049bfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049bfe8 sp=0xc00049bfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 102 gp=0xc000232380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049c738 sp=0xc00049c718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049c7c8 sp=0xc00049c738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049c7e0 sp=0xc00049c7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049c7e8 sp=0xc00049c7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 103 gp=0xc000232540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049cf38 sp=0xc00049cf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049cfc8 sp=0xc00049cf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049cfe0 sp=0xc00049cfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049cfe8 sp=0xc00049cfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 104 gp=0xc000232700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049d738 sp=0xc00049d718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049d7c8 sp=0xc00049d738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049d7e0 sp=0xc00049d7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049d7e8 sp=0xc00049d7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 105 gp=0xc0002328c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00049df38 sp=0xc00049df18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00049dfc8 sp=0xc00049df38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00049dfe0 sp=0xc00049dfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00049dfe8 sp=0xc00049dfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 115 gp=0xc000511a40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000518f38 sp=0xc000518f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000518fc8 sp=0xc000518f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000518fe0 sp=0xc000518fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000518fe8 sp=0xc000518fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 92 gp=0xc0004a2e00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000514f38 sp=0xc000514f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000514fc8 sp=0xc000514f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000514fe0 sp=0xc000514fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000514fe8 sp=0xc000514fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 93 gp=0xc0004a2fc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000515738 sp=0xc000515718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005157c8 sp=0xc000515738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005157e0 sp=0xc0005157c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005157e8 sp=0xc0005157e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 94 gp=0xc0004a3180 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x4a?, 0x5d?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000515f38 sp=0xc000515f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000515fc8 sp=0xc000515f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000515fe0 sp=0xc000515fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000515fe8 sp=0xc000515fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 95 gp=0xc0004a3340 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x7d?, 0xbe?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000516738 sp=0xc000516718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005167c8 sp=0xc000516738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005167e0 sp=0xc0005167c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005167e8 sp=0xc0005167e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 96 gp=0xc0004a3500 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c0434ea?, 0x1?, 0x9f?, 0xb1?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000516f38 sp=0xc000516f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000516fc8 sp=0xc000516f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000516fe0 sp=0xc000516fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000516fe8 sp=0xc000516fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 97 gp=0xc0004a36c0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03f648?, 0x1?, 0x2c?, 0xb9?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000517738 sp=0xc000517718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005177c8 sp=0xc000517738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005177e0 sp=0xc0005177c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005177e8 sp=0xc0005177e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 130 gp=0xc0004a3880 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x47?, 0x4d?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000517f38 sp=0xc000517f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000517fc8 sp=0xc000517f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000517fe0 sp=0xc000517fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000517fe8 sp=0xc000517fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 116 gp=0xc000511c00 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03c923?, 0x1?, 0x5?, 0xed?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000519738 sp=0xc000519718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0005197c8 sp=0xc000519738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0005197e0 sp=0xc0005197c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0005197e8 sp=0xc0005197e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 117 gp=0xc000511dc0 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0xa0?, 0x5e?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000519f38 sp=0xc000519f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000519fc8 sp=0xc000519f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000519fe0 sp=0xc000519fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000519fe8 sp=0xc000519fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 118 gp=0xc00051c000 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0xed?, 0x49?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00051a738 sp=0xc00051a718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00051a7c8 sp=0xc00051a738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00051a7e0 sp=0xc00051a7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00051a7e8 sp=0xc00051a7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 119 gp=0xc00051c1c0 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x6e?, 0x1e?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00051af38 sp=0xc00051af18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00051afc8 sp=0xc00051af38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00051afe0 sp=0xc00051afc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00051afe8 sp=0xc00051afe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 106 gp=0xc000232a80 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c04234f?, 0x1?, 0xd3?, 0xb0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000238738 sp=0xc000238718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002387c8 sp=0xc000238738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002387e0 sp=0xc0002387c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002387e8 sp=0xc0002387e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 107 gp=0xc000232c40 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bdc1d36?, 0x1?, 0x57?, 0x19?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000238f38 sp=0xc000238f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000238fc8 sp=0xc000238f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000238fe0 sp=0xc000238fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000238fe8 sp=0xc000238fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 108 gp=0xc000232e00 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bdc66ea?, 0x1?, 0xfb?, 0xb?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000239738 sp=0xc000239718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002397c8 sp=0xc000239738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002397e0 sp=0xc0002397c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002397e8 sp=0xc0002397e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 109 gp=0xc000232fc0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03ce10?, 0x1?, 0x1d?, 0x7b?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000239f38 sp=0xc000239f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000239fc8 sp=0xc000239f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000239fe0 sp=0xc000239fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000239fe8 sp=0xc000239fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 110 gp=0xc000233180 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0xee?, 0xc7?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00023a738 sp=0xc00023a718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00023a7c8 sp=0xc00023a738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00023a7e0 sp=0xc00023a7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00023a7e8 sp=0xc00023a7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 111 gp=0xc000233340 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x63?, 0x14?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00023af38 sp=0xc00023af18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00023afc8 sp=0xc00023af38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00023afe0 sp=0xc00023afc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00023afe8 sp=0xc00023afe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 112 gp=0xc000233500 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bdf0e6a?, 0x1?, 0x6e?, 0x36?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00023b738 sp=0xc00023b718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00023b7c8 sp=0xc00023b738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00023b7e0 sp=0xc00023b7c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00023b7e8 sp=0xc00023b7e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 113 gp=0xc0002336c0 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0xb2?, 0xc3?, 0x0?) runtime/proc.go:435 +0xce fp=0xc00023bf38 sp=0xc00023bf18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc00023bfc8 sp=0xc00023bf38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc00023bfe0 sp=0xc00023bfc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc00023bfe8 sp=0xc00023bfe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 146 gp=0xc000233880 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bde9ae4?, 0x1?, 0x7c?, 0xd0?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000234738 sp=0xc000234718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002347c8 sp=0xc000234738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002347e0 sp=0xc0002347c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002347e8 sp=0xc0002347e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 147 gp=0xc000233a40 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x3?, 0x6e?, 0x53?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000234f38 sp=0xc000234f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000234fc8 sp=0xc000234f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000234fe0 sp=0xc000234fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000234fe8 sp=0xc000234fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 148 gp=0xc000233c00 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x10?, 0xe7?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000235738 sp=0xc000235718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002357c8 sp=0xc000235738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002357e0 sp=0xc0002357c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002357e8 sp=0xc0002357e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 149 gp=0xc000233dc0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bde3d95?, 0x3?, 0x22?, 0x1?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000235f38 sp=0xc000235f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000235fc8 sp=0xc000235f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000235fe0 sp=0xc000235fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000235fe8 sp=0xc000235fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 150 gp=0xc00023c000 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03cffb?, 0x1?, 0x28?, 0x78?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000236738 sp=0xc000236718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002367c8 sp=0xc000236738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002367e0 sp=0xc0002367c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002367e8 sp=0xc0002367e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 151 gp=0xc00023c1c0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c0411f0?, 0x1?, 0x99?, 0xa3?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000236f38 sp=0xc000236f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000236fc8 sp=0xc000236f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000236fe0 sp=0xc000236fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000236fe8 sp=0xc000236fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 152 gp=0xc00023c380 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03a72d?, 0x3?, 0x3b?, 0xad?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000237738 sp=0xc000237718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002377c8 sp=0xc000237738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002377e0 sp=0xc0002377c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002377e8 sp=0xc0002377e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 153 gp=0xc00023c540 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03e041?, 0x1?, 0x42?, 0x8b?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000237f38 sp=0xc000237f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000237fc8 sp=0xc000237f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000237fe0 sp=0xc000237fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000237fe8 sp=0xc000237fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 154 gp=0xc00023c700 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x3?, 0x3c?, 0x44?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000242738 sp=0xc000242718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002427c8 sp=0xc000242738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002427e0 sp=0xc0002427c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002427e8 sp=0xc0002427e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 155 gp=0xc00023c8c0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4bdc4667?, 0x3?, 0x17?, 0x9f?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000242f38 sp=0xc000242f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000242fc8 sp=0xc000242f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000242fe0 sp=0xc000242fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000242fe8 sp=0xc000242fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 156 gp=0xc00023ca80 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c0404fe?, 0x1?, 0xdc?, 0x54?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000243738 sp=0xc000243718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002437c8 sp=0xc000243738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002437e0 sp=0xc0002437c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002437e8 sp=0xc0002437e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 157 gp=0xc00023cc40 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4be1495a?, 0x1?, 0x5?, 0x5b?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000243f38 sp=0xc000243f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000243fc8 sp=0xc000243f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000243fe0 sp=0xc000243fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000243fe8 sp=0xc000243fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 158 gp=0xc00023ce00 m=nil [GC worker (idle)]: runtime.gopark(0x56221bb67c00?, 0x1?, 0x9c?, 0x3d?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000244738 sp=0xc000244718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002447c8 sp=0xc000244738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002447e0 sp=0xc0002447c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002447e8 sp=0xc0002447e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 159 gp=0xc00023cfc0 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4c03b679?, 0x1?, 0x48?, 0xb5?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000244f38 sp=0xc000244f18 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc000244fc8 sp=0xc000244f38 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc000244fe0 sp=0xc000244fc8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc000244fe8 sp=0xc000244fe0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 160 gp=0xc00023d180 m=nil [GC worker (idle)]: runtime.gopark(0x10465c4be15b1d?, 0x1?, 0x61?, 0xdd?, 0x0?) runtime/proc.go:435 +0xce fp=0xc000245738 sp=0xc000245718 pc=0x562219f2374e runtime.gcBgMarkWorker(0xc000045500) runtime/mgc.go:1423 +0xe9 fp=0xc0002457c8 sp=0xc000245738 pc=0x562219ed0f49 runtime.gcBgMarkStartWorkers.gowrap1() runtime/mgc.go:1339 +0x25 fp=0xc0002457e0 sp=0xc0002457c8 pc=0x562219ed0e25 runtime.goexit({}) runtime/asm_amd64.s:1700 +0x1 fp=0xc0002457e8 sp=0xc0002457e0 pc=0x562219f2ae81 created by runtime.gcBgMarkStartWorkers in goroutine 1 runtime/mgc.go:1339 +0x105 goroutine 133 gp=0xc000582a80 m=nil [select]: runtime.gopark(0xc000053a10?, 0x2?, 0x0?, 0xe6?, 0xc000053874?) runtime/proc.go:435 +0xce fp=0xc000053680 sp=0xc000053660 pc=0x562219f2374e runtime.selectgo(0xc000053a10, 0xc000053870, 0x167?, 0x0, 0x4?, 0x1) runtime/select.go:351 +0x837 fp=0xc0000537b8 sp=0xc000053680 pc=0x562219f02037 github.com/ollama/ollama/runner/ollamarunner.(*Server).completion(0xc00057ec60, {0x56221b240978, 0xc000000000}, 0xc00311a140) github.com/ollama/ollama/runner/ollamarunner/runner.go:664 +0xb65 fp=0xc000053ac0 sp=0xc0000537b8 pc=0x56221a3edc25 github.com/ollama/ollama/runner/ollamarunner.(*Server).completion-fm({0x56221b240978?, 0xc000000000?}, 0xc000625b40?) <autogenerated>:1 +0x36 fp=0xc000053af0 sp=0xc000053ac0 pc=0x56221a3f04f6 net/http.HandlerFunc.ServeHTTP(0xc00056a780?, {0x56221b240978?, 0xc000000000?}, 0xc000625b60?) net/http/server.go:2294 +0x29 fp=0xc000053b18 sp=0xc000053af0 pc=0x56221a221d69 net/http.(*ServeMux).ServeHTTP(0x562219ec7f65?, {0x56221b240978, 0xc000000000}, 0xc00311a140) net/http/server.go:2822 +0x1c4 fp=0xc000053b68 sp=0xc000053b18 pc=0x56221a223c64 net/http.serverHandler.ServeHTTP({0x56221b23d050?}, {0x56221b240978?, 0xc000000000?}, 0x1?) net/http/server.go:3301 +0x8e fp=0xc000053b98 sp=0xc000053b68 pc=0x56221a2416ee net/http.(*conn).serve(0xc00026a000, {0x56221b242ae8, 0xc0005c7260}) net/http/server.go:2102 +0x625 fp=0xc000053fb8 sp=0xc000053b98 pc=0x56221a220265 net/http.(*Server).Serve.gowrap3() net/http/server.go:3454 +0x28 fp=0xc000053fe0 sp=0xc000053fb8 pc=0x56221a225b28 runtime.goexit({})
GiteaMirror commented 2026-04-22 14:33:51 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

Any context from before the crash?

<!-- gh-comment-id:2854817995 --> @rick-github commented on GitHub (May 6, 2025): Any context from before the crash?
GiteaMirror commented 2026-04-22 14:33:51 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

What version of ollama?

<!-- gh-comment-id:2854823023 --> @rick-github commented on GitHub (May 6, 2025): What version of ollama?
GiteaMirror commented 2026-04-22 14:33:51 -05:00
Author
Owner
Copy Link

@renepeinl commented on GitHub (May 6, 2025):

No additional context. Latest version. I just installed it with the download you provide for Linux, so I guess 0.6.8

<!-- gh-comment-id:2854847385 --> @renepeinl commented on GitHub (May 6, 2025): No additional context. Latest version. I just installed it with the download you provide for Linux, so I guess 0.6.8
GiteaMirror commented 2026-04-22 14:33:52 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

Similar issue on Windows with ollama 0.6.8

ollama run llama4:scout

Who are you ?
Error: POST predict: Post "http://127.0.0.1:62232/completion": read tcp 127.0.0.1:62234->127.0.0.1:62232: wsarecv: An existing connection was forcibly closed by the remote host.

The model seams to load, but at the moment of inference it breaks.
Below is the log :

[GIN] 2025/05/06 - 16:46:54 | 200 | 0s | 127.0.0.1 | HEAD "/"
time=2025-05-06T16:46:57.585Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:57.631Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
[GIN] 2025/05/06 - 16:46:57 | 200 | 2.8987308s | 127.0.0.1 | POST "/api/show"
time=2025-05-06T16:46:57.690Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:57.788Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:57.834Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:57.836Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.839Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.841Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.844Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.848Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.858Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.861Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.863Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.865Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.870Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.872Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.874Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.877Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.880Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.885Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.887Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.889Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.891Z level=INFO source=sched.go:770 msg="new model will fit in available VRAM, loading" model=E:\models\blobs\sha256-9d507a36062c2845dd3bb3e93364e9abc1607118acd8650727a700f72fb126e5 library=cuda parallel=2 required="87.3 GiB"
time=2025-05-06T16:46:57.947Z level=INFO source=server.go:106 msg="system memory" total="760.0 GiB" free="711.8 GiB" free_swap="750.8 GiB"
time=2025-05-06T16:46:57.948Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0
time=2025-05-06T16:46:57.949Z level=INFO source=server.go:139 msg=offload library=cuda layers.requested=-1 layers.model=49 layers.offload=49 layers.split=7,6,6,6,6,6,6,6 memory.available="[22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 21.3 GiB]" memory.gpu_overhead="0 B" memory.required.full="87.3 GiB" memory.required.partial="87.3 GiB" memory.required.kv="1.5 GiB" memory.required.allocations="[13.0 GiB 10.6 GiB 10.6 GiB 10.8 GiB 10.6 GiB 10.6 GiB 10.6 GiB 10.4 GiB]" memory.weights.total="60.6 GiB" memory.weights.repeating="59.8 GiB" memory.weights.nonrepeating="809.3 MiB" memory.graph.full="1.1 GiB" memory.graph.partial="1.1 GiB" projector.weights="1.6 GiB" projector.graph="0 B"
time=2025-05-06T16:46:58.035Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:58.036Z level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.pretokenizer default="[^\r\n\p{L}\p{N}]?[\p{Lu}\p{Lt}\p{Lm}\p{Lo}\p{M}][\p{Ll}\p{Lm}\p{Lo}\p{M}]+(?i:'s|'t|'re|'ve|'m|'ll|'d)?|[^\r\n\p{L}\p{N}]?[\p{Lu}\p{Lt}\p{Lm}\p{Lo}\p{M}]+[\p{Ll}\p{Lm}\p{Lo}\p{M}](?i:'s|'t|'re|'ve|'m|'ll|'d)?|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n/]|\s[\r\n]+|\s+(?!\S)|\s+"
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=3
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.max_upscaling_size default=448
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.rope.freq_scale default=1
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.no_rope_interval default=4
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.temperature_tuning default=true
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.scale default=0.10000000149011612
time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.floor_scale default=8192
time=2025-05-06T16:46:58.040Z level=INFO source=server.go:410 msg="starting llama server" cmd="C:\Users\Sebastien\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --model E:\models\blobs\sha256-9d507a36062c2845dd3bb3e93364e9abc1607118acd8650727a700f72fb126e5 --ctx-size 8192 --batch-size 512 --n-gpu-layers 49 --threads 192 --no-mmap --parallel 2 --tensor-split 7,6,6,6,6,6,6,6 --port 62232"
time=2025-05-06T16:46:58.043Z level=INFO source=sched.go:452 msg="loaded runners" count=1
time=2025-05-06T16:46:58.043Z level=INFO source=server.go:589 msg="waiting for llama runner to start responding"
time=2025-05-06T16:46:58.044Z level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server error"
time=2025-05-06T16:46:58.093Z level=INFO source=runner.go:851 msg="starting ollama engine"
time=2025-05-06T16:46:58.116Z level=INFO source=runner.go:914 msg="Server listening on 127.0.0.1:62232"
time=2025-05-06T16:46:58.198Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
time=2025-05-06T16:46:58.200Z level=WARN source=ggml.go:152 msg="key not found" key=general.name default=""
time=2025-05-06T16:46:58.200Z level=WARN source=ggml.go:152 msg="key not found" key=general.description default=""
time=2025-05-06T16:46:58.200Z level=INFO source=ggml.go:72 msg="" architecture=llama4 file_type=Q4_K_M name="" description="" num_tensors=1182 num_key_values=45
load_backend: loaded CPU backend from C:\Users\Sebastien\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-haswell.dll
time=2025-05-06T16:46:58.296Z level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server loading model"
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 8 CUDA devices:
Device 0: NVIDIA A10G, compute capability 8.6, VMM: no
Device 1: NVIDIA A10G, compute capability 8.6, VMM: no
Device 2: NVIDIA A10G, compute capability 8.6, VMM: no
Device 3: NVIDIA A10G, compute capability 8.6, VMM: no
Device 4: NVIDIA A10G, compute capability 8.6, VMM: no
Device 5: NVIDIA A10G, compute capability 8.6, VMM: no
Device 6: NVIDIA A10G, compute capability 8.6, VMM: no
Device 7: NVIDIA A10G, compute capability 8.6, VMM: no
load_backend: loaded CUDA backend from C:\Users\Sebastien\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll
time=2025-05-06T16:46:59.180Z level=INFO source=ggml.go:103 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 CUDA.2.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.2.USE_GRAPHS=1 CUDA.2.PEER_MAX_BATCH_SIZE=128 CUDA.3.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.3.USE_GRAPHS=1 CUDA.3.PEER_MAX_BATCH_SIZE=128 CUDA.4.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.4.USE_GRAPHS=1 CUDA.4.PEER_MAX_BATCH_SIZE=128 CUDA.5.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.5.USE_GRAPHS=1 CUDA.5.PEER_MAX_BATCH_SIZE=128 CUDA.6.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.6.USE_GRAPHS=1 CUDA.6.PEER_MAX_BATCH_SIZE=128 CUDA.7.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.7.USE_GRAPHS=1 CUDA.7.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA2 size="7.3 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA6 size="7.5 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA7 size="9.1 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA5 size="7.3 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA1 size="7.3 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA3 size="7.3 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA4 size="7.3 GiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CPU size="554.9 MiB"
time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA0 size="9.2 GiB"
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.pretokenizer default="[^\r\n\p{L}\p{N}]?[\p{Lu}\p{Lt}\p{Lm}\p{Lo}\p{M}][\p{Ll}\p{Lm}\p{Lo}\p{M}]+(?i:'s|'t|'re|'ve|'m|'ll|'d)?|[^\r\n\p{L}\p{N}]?[\p{Lu}\p{Lt}\p{Lm}\p{Lo}\p{M}]+[\p{Ll}\p{Lm}\p{Lo}\p{M}](?i:'s|'t|'re|'ve|'m|'ll|'d)?|\p{N}{1,3}| ?[^\s\p{L}\p{N}]+[\r\n/]|\s[\r\n]+|\s+(?!\S)|\s+"
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=3
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.max_upscaling_size default=448
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.rope.freq_scale default=1
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.no_rope_interval default=4
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.temperature_tuning default=true
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.scale default=0.10000000149011612
time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.floor_scale default=8192
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA0 buffer_type=CUDA0 size="828.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA1 buffer_type=CUDA1 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA2 buffer_type=CUDA2 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA3 buffer_type=CUDA3 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA4 buffer_type=CUDA4 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA5 buffer_type=CUDA5 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA6 buffer_type=CUDA6 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA7 buffer_type=CUDA7 size="868.0 MiB"
time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CPU buffer_type=CPU size="10.0 MiB"
time=2025-05-06T16:47:36.427Z level=INFO source=server.go:628 msg="llama runner started in 38.38 seconds"
[GIN] 2025/05/06 - 16:47:36 | 200 | 38.7876623s | 127.0.0.1 | POST "/api/generate"
time=2025-05-06T16:48:16.915Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
CUDA error: an illegal memory access was encountered
current device: 0, in function ggml_cuda_mul_mat_q at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\mmq.cu:145
cudaMemcpyAsync(ids_host.data(), ids->data, ggml_nbytes(ids), cudaMemcpyDeviceToHost, stream)
C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error
[GIN] 2025/05/06 - 16:48:17 | 200 | 898.0709ms | 127.0.0.1 | POST "/api/chat"
time=2025-05-06T16:48:18.090Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 0xc0000409"

<!-- gh-comment-id:2854869446 --> @sebdg commented on GitHub (May 6, 2025): Similar issue on Windows with ollama 0.6.8 ollama run llama4:scout >>> Who are you ? Error: POST predict: Post "http://127.0.0.1:62232/completion": read tcp 127.0.0.1:62234->127.0.0.1:62232: wsarecv: An existing connection was forcibly closed by the remote host. The model seams to load, but at the moment of inference it breaks. Below is the log : [GIN] 2025/05/06 - 16:46:54 | 200 | 0s | 127.0.0.1 | HEAD "/" time=2025-05-06T16:46:57.585Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:57.631Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 [GIN] 2025/05/06 - 16:46:57 | 200 | 2.8987308s | 127.0.0.1 | POST "/api/show" time=2025-05-06T16:46:57.690Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:57.788Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:57.834Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:57.836Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.839Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.841Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.844Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.848Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.858Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.861Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.863Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.865Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.870Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.872Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.874Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.877Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.880Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.885Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.887Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.889Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.891Z level=INFO source=sched.go:770 msg="new model will fit in available VRAM, loading" model=E:\models\blobs\sha256-9d507a36062c2845dd3bb3e93364e9abc1607118acd8650727a700f72fb126e5 library=cuda parallel=2 required="87.3 GiB" time=2025-05-06T16:46:57.947Z level=INFO source=server.go:106 msg="system memory" total="760.0 GiB" free="711.8 GiB" free_swap="750.8 GiB" time=2025-05-06T16:46:57.948Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=0 time=2025-05-06T16:46:57.949Z level=INFO source=server.go:139 msg=offload library=cuda layers.requested=-1 layers.model=49 layers.offload=49 layers.split=7,6,6,6,6,6,6,6 memory.available="[22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 22.0 GiB 21.3 GiB]" memory.gpu_overhead="0 B" memory.required.full="87.3 GiB" memory.required.partial="87.3 GiB" memory.required.kv="1.5 GiB" memory.required.allocations="[13.0 GiB 10.6 GiB 10.6 GiB 10.8 GiB 10.6 GiB 10.6 GiB 10.6 GiB 10.4 GiB]" memory.weights.total="60.6 GiB" memory.weights.repeating="59.8 GiB" memory.weights.nonrepeating="809.3 MiB" memory.graph.full="1.1 GiB" memory.graph.partial="1.1 GiB" projector.weights="1.6 GiB" projector.graph="0 B" time=2025-05-06T16:46:58.035Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:58.036Z level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.pretokenizer default="[^\\r\\n\\p{L}\\p{N}]?[\\p{Lu}\\p{Lt}\\p{Lm}\\p{Lo}\\p{M}]*[\\p{Ll}\\p{Lm}\\p{Lo}\\p{M}]+(?i:'s|'t|'re|'ve|'m|'ll|'d)?|[^\\r\\n\\p{L}\\p{N}]?[\\p{Lu}\\p{Lt}\\p{Lm}\\p{Lo}\\p{M}]+[\\p{Ll}\\p{Lm}\\p{Lo}\\p{M}]*(?i:'s|'t|'re|'ve|'m|'ll|'d)?|\\p{N}{1,3}| ?[^\\s\\p{L}\\p{N}]+[\\r\\n/]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+" time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=3 time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.max_upscaling_size default=448 time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.rope.freq_scale default=1 time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.no_rope_interval default=4 time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.temperature_tuning default=true time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.scale default=0.10000000149011612 time=2025-05-06T16:46:58.037Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.floor_scale default=8192 time=2025-05-06T16:46:58.040Z level=INFO source=server.go:410 msg="starting llama server" cmd="C:\\Users\\Sebastien\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model E:\\models\\blobs\\sha256-9d507a36062c2845dd3bb3e93364e9abc1607118acd8650727a700f72fb126e5 --ctx-size 8192 --batch-size 512 --n-gpu-layers 49 --threads 192 --no-mmap --parallel 2 --tensor-split 7,6,6,6,6,6,6,6 --port 62232" time=2025-05-06T16:46:58.043Z level=INFO source=sched.go:452 msg="loaded runners" count=1 time=2025-05-06T16:46:58.043Z level=INFO source=server.go:589 msg="waiting for llama runner to start responding" time=2025-05-06T16:46:58.044Z level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server error" time=2025-05-06T16:46:58.093Z level=INFO source=runner.go:851 msg="starting ollama engine" time=2025-05-06T16:46:58.116Z level=INFO source=runner.go:914 msg="Server listening on 127.0.0.1:62232" time=2025-05-06T16:46:58.198Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 time=2025-05-06T16:46:58.200Z level=WARN source=ggml.go:152 msg="key not found" key=general.name default="" time=2025-05-06T16:46:58.200Z level=WARN source=ggml.go:152 msg="key not found" key=general.description default="" time=2025-05-06T16:46:58.200Z level=INFO source=ggml.go:72 msg="" architecture=llama4 file_type=Q4_K_M name="" description="" num_tensors=1182 num_key_values=45 load_backend: loaded CPU backend from C:\Users\Sebastien\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-haswell.dll time=2025-05-06T16:46:58.296Z level=INFO source=server.go:623 msg="waiting for server to become available" status="llm server loading model" ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 8 CUDA devices: Device 0: NVIDIA A10G, compute capability 8.6, VMM: no Device 1: NVIDIA A10G, compute capability 8.6, VMM: no Device 2: NVIDIA A10G, compute capability 8.6, VMM: no Device 3: NVIDIA A10G, compute capability 8.6, VMM: no Device 4: NVIDIA A10G, compute capability 8.6, VMM: no Device 5: NVIDIA A10G, compute capability 8.6, VMM: no Device 6: NVIDIA A10G, compute capability 8.6, VMM: no Device 7: NVIDIA A10G, compute capability 8.6, VMM: no load_backend: loaded CUDA backend from C:\Users\Sebastien\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll time=2025-05-06T16:46:59.180Z level=INFO source=ggml.go:103 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 CUDA.2.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.2.USE_GRAPHS=1 CUDA.2.PEER_MAX_BATCH_SIZE=128 CUDA.3.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.3.USE_GRAPHS=1 CUDA.3.PEER_MAX_BATCH_SIZE=128 CUDA.4.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.4.USE_GRAPHS=1 CUDA.4.PEER_MAX_BATCH_SIZE=128 CUDA.5.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.5.USE_GRAPHS=1 CUDA.5.PEER_MAX_BATCH_SIZE=128 CUDA.6.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.6.USE_GRAPHS=1 CUDA.6.PEER_MAX_BATCH_SIZE=128 CUDA.7.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.7.USE_GRAPHS=1 CUDA.7.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA2 size="7.3 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA6 size="7.5 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA7 size="9.1 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA5 size="7.3 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA1 size="7.3 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA3 size="7.3 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA4 size="7.3 GiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CPU size="554.9 MiB" time=2025-05-06T16:46:59.290Z level=INFO source=ggml.go:298 msg="model weights" buffer=CUDA0 size="9.2 GiB" time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=tokenizer.ggml.pretokenizer default="[^\\r\\n\\p{L}\\p{N}]?[\\p{Lu}\\p{Lt}\\p{Lm}\\p{Lo}\\p{M}]*[\\p{Ll}\\p{Lm}\\p{Lo}\\p{M}]+(?i:'s|'t|'re|'ve|'m|'ll|'d)?|[^\\r\\n\\p{L}\\p{N}]?[\\p{Lu}\\p{Lt}\\p{Lm}\\p{Lo}\\p{M}]+[\\p{Ll}\\p{Lm}\\p{Lo}\\p{M}]*(?i:'s|'t|'re|'ve|'m|'ll|'d)?|\\p{N}{1,3}| ?[^\\s\\p{L}\\p{N}]+[\\r\\n/]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+" time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.num_channels default=3 time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.vision.max_upscaling_size default=448 time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.rope.freq_scale default=1 time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.no_rope_interval default=4 time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.temperature_tuning default=true time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.scale default=0.10000000149011612 time=2025-05-06T16:47:36.268Z level=WARN source=ggml.go:152 msg="key not found" key=llama4.attention.floor_scale default=8192 time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA0 buffer_type=CUDA0 size="828.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA1 buffer_type=CUDA1 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA2 buffer_type=CUDA2 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA3 buffer_type=CUDA3 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA4 buffer_type=CUDA4 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA5 buffer_type=CUDA5 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA6 buffer_type=CUDA6 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CUDA7 buffer_type=CUDA7 size="868.0 MiB" time=2025-05-06T16:47:36.383Z level=INFO source=ggml.go:553 msg="compute graph" backend=CPU buffer_type=CPU size="10.0 MiB" time=2025-05-06T16:47:36.427Z level=INFO source=server.go:628 msg="llama runner started in 38.38 seconds" [GIN] 2025/05/06 - 16:47:36 | 200 | 38.7876623s | 127.0.0.1 | POST "/api/generate" time=2025-05-06T16:48:16.915Z level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 CUDA error: an illegal memory access was encountered current device: 0, in function ggml_cuda_mul_mat_q at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\mmq.cu:145 cudaMemcpyAsync(ids_host.data(), ids->data, ggml_nbytes(ids), cudaMemcpyDeviceToHost, stream) C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error [GIN] 2025/05/06 - 16:48:17 | 200 | 898.0709ms | 127.0.0.1 | POST "/api/chat" time=2025-05-06T16:48:18.090Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 0xc0000409"
GiteaMirror commented 2026-04-22 14:33:52 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

Actually relevant part of abovelog seams to be :

CUDA error: an illegal memory access was encountered
current device: 0, in function ggml_cuda_mul_mat_q at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\mmq.cu:145
cudaMemcpyAsync(ids_host.data(), ids->data, ggml_nbytes(ids), cudaMemcpyDeviceToHost, stream)
C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error
[GIN] 2025/05/06 - 16:48:17 | 200 | 898.0709ms | 127.0.0.1 | POST "/api/chat"
time=2025-05-06T16:48:18.090Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 0xc0000409"

I also posted this on the discord server but had no reaction from other users.

<!-- gh-comment-id:2854874415 --> @sebdg commented on GitHub (May 6, 2025): Actually relevant part of abovelog seams to be : CUDA error: an illegal memory access was encountered current device: 0, in function ggml_cuda_mul_mat_q at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\mmq.cu:145 cudaMemcpyAsync(ids_host.data(), ids->data, ggml_nbytes(ids), cudaMemcpyDeviceToHost, stream) C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error [GIN] 2025/05/06 - 16:48:17 | 200 | 898.0709ms | 127.0.0.1 | POST "/api/chat" time=2025-05-06T16:48:18.090Z level=ERROR source=server.go:455 msg="llama runner terminated" error="exit status 0xc0000409" I also posted this on the discord server but had no reaction from other users.
GiteaMirror commented 2026-04-22 14:33:52 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

Did your query have an image attached?

<!-- gh-comment-id:2854906423 --> @rick-github commented on GitHub (May 6, 2025): Did your query have an image attached?
GiteaMirror commented 2026-04-22 14:33:53 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

May be a llama.cpp problem: https://github.com/ggml-org/llama.cpp/issues/13085#issuecomment-2844242697 or https://github.com/ggml-org/llama.cpp/issues/13281

<!-- gh-comment-id:2855111766 --> @rick-github commented on GitHub (May 6, 2025): May be a llama.cpp problem: https://github.com/ggml-org/llama.cpp/issues/13085#issuecomment-2844242697 or https://github.com/ggml-org/llama.cpp/issues/13281
GiteaMirror commented 2026-04-22 14:33:53 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

Did your query have an image attached?

no image, just plain text from the cli and api

<!-- gh-comment-id:2855220125 --> @sebdg commented on GitHub (May 6, 2025): > Did your query have an image attached? no image, just plain text from the cli and api
GiteaMirror commented 2026-04-22 14:33:53 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

May be a llama.cpp problem: ggml-org/llama.cpp#13085 (comment)

That issue seams to be related to mistral-small models, and it's an OOM error the one I have is rather an Invalid access error.

btw. I found this out while running benchmarks on 10+ models mistal-small included as well as others like qwen3, llama3.3, gemma etc etc... the only one actually crashing out was llama4

<!-- gh-comment-id:2855241420 --> @sebdg commented on GitHub (May 6, 2025): > May be a llama.cpp problem: [ggml-org/llama.cpp#13085 (comment)](https://github.com/ggml-org/llama.cpp/issues/13085#issuecomment-2844242697) That issue seams to be related to mistral-small models, and it's an OOM error the one I have is rather an Invalid access error. btw. I found this out while running benchmarks on 10+ models mistal-small included as well as others like qwen3, llama3.3, gemma etc etc... the only one actually crashing out was llama4
GiteaMirror commented 2026-04-22 14:33:54 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 6, 2025):

From the linked comment:

 current device: 1, in function ggml_cuda_mul_mat_q at /home/mostlygeek/llama.cpp/ggml/src/ggml-cuda/mmq.cu:145

      --model /mnt/nvme/models/unsloth/llama-4/UD-Q4_K_XL/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf

These match the information in your post. ollama recently synced to llama.cpp head and so these issues might be a result of a new codebase.

<!-- gh-comment-id:2855278116 --> @rick-github commented on GitHub (May 6, 2025): From the linked comment: ``` current device: 1, in function ggml_cuda_mul_mat_q at /home/mostlygeek/llama.cpp/ggml/src/ggml-cuda/mmq.cu:145 ``` ``` --model /mnt/nvme/models/unsloth/llama-4/UD-Q4_K_XL/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf ``` These match the information in your post. ollama recently [synced](https://github.com/ollama/ollama/commit/8dd12c873d7816f76d5488c8ff879a574dad5e01) to llama.cpp head and so these issues might be a result of a new codebase.
GiteaMirror commented 2026-04-22 14:33:54 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

From the linked comment:

 current device: 1, in function ggml_cuda_mul_mat_q at /home/mostlygeek/llama.cpp/ggml/src/ggml-cuda/mmq.cu:145

      --model /mnt/nvme/models/unsloth/llama-4/UD-Q4_K_XL/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf

These match the information in your post. ollama recently synced to llama.cpp head and so these issues might be a result of a new codebase.

I'll try to downgrade to 0.6.7 later tonight to give it a shot

<!-- gh-comment-id:2855292581 --> @sebdg commented on GitHub (May 6, 2025): > From the linked comment: > > ``` > current device: 1, in function ggml_cuda_mul_mat_q at /home/mostlygeek/llama.cpp/ggml/src/ggml-cuda/mmq.cu:145 > ``` > > ``` > --model /mnt/nvme/models/unsloth/llama-4/UD-Q4_K_XL/Llama-4-Scout-17B-16E-Instruct-UD-Q4_K_XL-00001-of-00002.gguf > ``` > > These match the information in your post. ollama recently [synced](https://github.com/ollama/ollama/commit/8dd12c873d7816f76d5488c8ff879a574dad5e01) to llama.cpp head and so these issues might be a result of a new codebase. I'll try to downgrade to 0.6.7 later tonight to give it a shot
GiteaMirror commented 2026-04-22 14:33:54 -05:00
Author
Owner
Copy Link

@rpeinl commented on GitHub (May 6, 2025):

I can confirm that the error does not occur with version 0.6.7 in my setup.

<!-- gh-comment-id:2855407529 --> @rpeinl commented on GitHub (May 6, 2025): I can confirm that the error does not occur with version 0.6.7 in my setup.
GiteaMirror commented 2026-04-22 14:33:54 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 6, 2025):

I can confirm that the error does not occur with version 0.6.7 in my setup.

same here works on 0.6.7

<!-- gh-comment-id:2855414982 --> @sebdg commented on GitHub (May 6, 2025): > I can confirm that the error does not occur with version 0.6.7 in my setup. same here works on 0.6.7
GiteaMirror commented 2026-04-22 14:33:56 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (May 11, 2025):

Sorry about this all - working on a fix.

<!-- gh-comment-id:2869300947 --> @jmorganca commented on GitHub (May 11, 2025): Sorry about this all - working on a fix.
GiteaMirror commented 2026-04-22 14:33:56 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (May 11, 2025):

A fix is in one of the commits brought in by https://github.com/ollama/ollama/pull/10655. There was an issue in one of the kernels used by the text model, I'm not sure which commit yet but will try to find out.

<!-- gh-comment-id:2869329350 --> @jmorganca commented on GitHub (May 11, 2025): A fix is in one of the commits brought in by https://github.com/ollama/ollama/pull/10655. There was an issue in one of the kernels used by the text model, I'm not sure which commit yet but will try to find out.
GiteaMirror commented 2026-04-22 14:33:56 -05:00
Author
Owner
Copy Link

@sebdg commented on GitHub (May 12, 2025):

A fix is in one of the commits brought in by #10655. There was an issue in one of the kernels used by the text model, I'm not sure which commit yet but will try to find out.

Thank you for your actions jmorgan, will test this as soon as there is (pre-)release

<!-- gh-comment-id:2873756872 --> @sebdg commented on GitHub (May 12, 2025): > A fix is in one of the commits brought in by [#10655](https://github.com/ollama/ollama/pull/10655). There was an issue in one of the kernels used by the text model, I'm not sure which commit yet but will try to find out. Thank you for your actions jmorgan, will test this as soon as there is (pre-)release
GiteaMirror commented 2026-04-22 14:33:57 -05:00
Author
Owner
Copy Link

@klemenska commented on GitHub (May 15, 2025):

same issues after the upgrade to 0.6.8

I am not able to install 0.7.0 rc0

Manual install

Note

If you are upgrading from a prior version, you should remove the old libraries with sudo rm -rf /usr/lib/ollama first.

Download and extract the package:

curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

Start Ollama:

ollama serve

In another terminal, verify that Ollama is running:

ollama -v

ollama version is 0.6.8

Commandline:
user@delta:~$ ollama run llama4

hello
Error: POST predict: Post "http://127.0.0.1:43261/completion": EOF
user@delta:~$

<!-- gh-comment-id:2882658748 --> @klemenska commented on GitHub (May 15, 2025): same issues after the upgrade to 0.6.8 I am not able to install 0.7.0 rc0 Manual install Note If you are upgrading from a prior version, you should remove the old libraries with sudo rm -rf /usr/lib/ollama first. Download and extract the package: curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz sudo tar -C /usr -xzf ollama-linux-amd64.tgz Start Ollama: ollama serve In another terminal, verify that Ollama is running: ollama -v ollama version is 0.6.8 Commandline: user@delta:~$ ollama run llama4 >>> hello Error: POST predict: Post "http://127.0.0.1:43261/completion": EOF user@delta:~$
GiteaMirror commented 2026-04-22 14:33:57 -05:00
Author
Owner
Copy Link

@klemenska commented on GitHub (May 15, 2025):

works with 0.7.0

<!-- gh-comment-id:2885068774 --> @klemenska commented on GitHub (May 15, 2025): works with 0.7.0
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#32728
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?