[GH-ISSUE #9807] Ollama REFUSES to use GFX803 EVEN when detected #6416

Closed
opened 2026-04-12 17:58:04 -05:00 by GiteaMirror · 2 comments
Owner

Originally created by @sanchez314c on GitHub (Mar 17, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9807

Been trying to resolve this for weeks now. Modify GPU.go to Rocm.....8 from 9. Add gfx803 to Cmakelist text and compile with:

cmake -B build -DCMAKE_PREFIX_PATH=/opt/rocm -DAMDGPU_TARGETS=gfx803 -DLLAMA_HIPBLAS=1 -DLLAMA_CUBLAS=0 && cmake --build build -- -j18

At runtime I get all the mess below. I've tried other variants of cmake flags and can also get it to detect the card and open a model but when using it GPU usage is sporadic/random and never pegs much higher than 30%.

Help please.

##########################################
./ollama run tinyllama
[GIN] 2025/03/17 - 02:36:31 | 200 | 75.723µs | 127.0.0.1 | HEAD "/"
[GIN] 2025/03/17 - 02:36:31 | 200 | 13.070198ms | 127.0.0.1 | POST "/api/show"
time=2025-03-17T02:36:31.280-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.vision.block_count default=0
time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.key_length default=64
time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.value_length default=64
time=2025-03-17T02:36:31.281-04:00 level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 gpu=0 parallel=4 available=8071651328 required="1.7 GiB"
time=2025-03-17T02:36:31.281-04:00 level=INFO source=server.go:105 msg="system memory" total="125.5 GiB" free="121.9 GiB" free_swap="8.0 GiB"
time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.vision.block_count default=0
time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.key_length default=64
time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.value_length default=64
time=2025-03-17T02:36:31.281-04:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=23 layers.offload=23 layers.split="" memory.available="[7.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="1.7 GiB" memory.required.partial="1.7 GiB" memory.required.kv="176.0 MiB" memory.required.allocations="[1.7 GiB]" memory.weights.total="520.1 MiB" memory.weights.repeating="520.1 MiB" memory.weights.nonrepeating="51.3 MiB" memory.graph.full="544.0 MiB" memory.graph.partial="546.3 MiB"
llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = TinyLlama
llama_model_loader: - kv 2: llama.context_length u32 = 2048
llama_model_loader: - kv 3: llama.embedding_length u32 = 2048
llama_model_loader: - kv 4: llama.block_count u32 = 22
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64
llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: general.file_type u32 = 2
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["", "", "", "<0x00>", "<...
llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{% if m...
llama_model_loader: - kv 22: general.quantization_version u32 = 2
llama_model_loader: - type f32: 45 tensors
llama_model_loader: - type q4_0: 155 tensors
llama_model_loader: - type q6_K: 1 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_0
print_info: file size = 606.53 MiB (4.63 BPW)
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: special tokens cache size = 3
load: token to piece cache size = 0.1684 MB
print_info: arch = llama
print_info: vocab_only = 1
print_info: model type = ?B
print_info: model params = 1.10 B
print_info: general.name = TinyLlama
print_info: vocab type = SPM
print_info: n_vocab = 32000
print_info: n_merges = 0
print_info: BOS token = 1 ''
print_info: EOS token = 2 '
'
print_info: UNK token = 0 ''
print_info: PAD token = 2 ''
print_info: LF token = 13 '<0x0A>'
print_info: EOG token = 2 ''
print_info: max token length = 48
llama_model_load: vocab only - skipping tensors
time=2025-03-17T02:36:31.324-04:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/home/heathen-admin/ollama-for-amd/ollama runner --model /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 --ctx-size 8192 --batch-size 512 --n-gpu-layers 23 --threads 10 --parallel 4 --port 43171"
time=2025-03-17T02:36:31.325-04:00 level=INFO source=sched.go:450 msg="loaded runners" count=1
time=2025-03-17T02:36:31.325-04:00 level=INFO source=server.go:585 msg="waiting for llama runner to start responding"
time=2025-03-17T02:36:31.325-04:00 level=INFO source=server.go:619 msg="waiting for server to become available" status="llm server error"
time=2025-03-17T02:36:31.335-04:00 level=INFO source=runner.go:931 msg="starting go runner"
⠇ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
Device 0: Radeon RX 580 Series, gfx803 (0x803), VMM: no, Wave Size: 64
load_backend: loaded ROCm backend from /home/heathen-admin/ollama-for-amd/build/lib/ollama/libggml-hip.so
load_backend: loaded CPU backend from /home/heathen-admin/ollama-for-amd/build/lib/ollama/libggml-cpu-skylakex.so
time=2025-03-17T02:36:33.250-04:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.AVX512=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
time=2025-03-17T02:36:33.251-04:00 level=INFO source=runner.go:991 msg="Server listening on 127.0.0.1:43171"
llama_model_load_from_file_impl: using device ROCm0 (Radeon RX 580 Series) - 8120 MiB free
⠋ llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = TinyLlama
llama_model_loader: - kv 2: llama.context_length u32 = 2048
llama_model_loader: - kv 3: llama.embedding_length u32 = 2048
llama_model_loader: - kv 4: llama.block_count u32 = 22
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64
llama_model_loader: - kv 7: llama.attention.head_count u32 = 32
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000
llama_model_loader: - kv 11: general.file_type u32 = 2
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["", "", "", "<0x00>", "<...
llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{% if m...
llama_model_loader: - kv 22: general.quantization_version u32 = 2
llama_model_loader: - type f32: 45 tensors
llama_model_loader: - type q4_0: 155 tensors
llama_model_loader: - type q6_K: 1 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_0
print_info: file size = 606.53 MiB (4.63 BPW)
load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
load: special tokens cache size = 3
load: token to piece cache size = 0.1684 MB
print_info: arch = llama
print_info: vocab_only = 0
print_info: n_ctx_train = 2048
print_info: n_embd = 2048
print_info: n_layer = 22
print_info: n_head = 32
print_info: n_head_kv = 4
print_info: n_rot = 64
print_info: n_swa = 0
print_info: n_embd_head_k = 64
print_info: n_embd_head_v = 64
print_info: n_gqa = 8
print_info: n_embd_k_gqa = 256
print_info: n_embd_v_gqa = 256
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: n_ff = 5632
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 0
print_info: rope scaling = linear
print_info: freq_base_train = 10000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 2048
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 0
print_info: ssm_d_inner = 0
print_info: ssm_d_state = 0
print_info: ssm_dt_rank = 0
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 1B
print_info: model params = 1.10 B
print_info: general.name = TinyLlama
print_info: vocab type = SPM
print_info: n_vocab = 32000
print_info: n_merges = 0
print_info: BOS token = 1 ''
print_info: EOS token = 2 '
'
print_info: UNK token = 0 ''
print_info: PAD token = 2 ''
print_info: LF token = 13 '<0x0A>'
print_info: EOG token = 2 ''
print_info: max token length = 48
load_tensors: loading model tensors, this can take a while... (mmap = true)
time=2025-03-17T02:36:33.333-04:00 level=INFO source=server.go:619 msg="waiting for server to become available" status="llm server loading model"
⠋ load_tensors: offloading 22 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 23/23 layers to GPU
load_tensors: ROCm0 model buffer size = 571.37 MiB
load_tensors: CPU_Mapped model buffer size = 35.16 MiB
SIGSEGV: segmentation violation
PC=0x7f8d52964434 m=5 sigcode=1 addr=0x18
signal arrived during cgo execution

goroutine 23 gp=0xc000002000 m=5 mp=0xc000100008 [syscall]:
runtime.cgocall(0x1105a00, 0xc0000a9c00)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/cgocall.go:167 +0x4b fp=0xc0000a9bd8 sp=0xc0000a9ba0 pc=0x494f4b
github.com/ollama/ollama/llama._Cfunc_llama_model_load_from_file(0x7f8d58000b70, {0x0, 0x17, 0x1, 0x0, 0x0, 0x11051b0, 0xc00053cab8, 0x0, 0x0, ...})
_cgo_gotypes.go:797 +0x47 fp=0xc0000a9c00 sp=0xc0000a9bd8 pc=0x82d167
github.com/ollama/ollama/llama.LoadModelFromFile.func1(...)
/home/heathen-admin/ollama-for-amd/llama/llama.go:240
github.com/ollama/ollama/llama.LoadModelFromFile({0x7ffe17091d2f, 0x70}, {0x17, 0x0, 0x1, 0x0, {0x0, 0x0, 0x0}, 0xc0005481c0, ...})
/home/heathen-admin/ollama-for-amd/llama/llama.go:240 +0x36b fp=0xc0000a9dc8 sp=0xc0000a9c00 pc=0x82fccb
github.com/ollama/ollama/runner/llamarunner.(*Server).loadModel(0xc000476000, {0x17, 0x0, 0x1, 0x0, {0x0, 0x0, 0x0}, 0xc0005481c0, 0x0}, ...)
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:849 +0x9b fp=0xc0000a9f10 sp=0xc0000a9dc8 pc=0x83bd5b
github.com/ollama/ollama/runner/llamarunner.Execute.gowrap1()
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:965 +0xda fp=0xc0000a9fe0 sp=0xc0000a9f10 pc=0x83d53a
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a9fe8 sp=0xc0000a9fe0 pc=0x49f881
created by github.com/ollama/ollama/runner/llamarunner.Execute in goroutine 1
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:965 +0xbd7

goroutine 1 gp=0xc000002380 m=nil [IO wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc0004995f8 sp=0xc0004995d8 pc=0x49822e
runtime.netpollblock(0xc000499648?, 0x431e86?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:575 +0xf7 fp=0xc000499630 sp=0xc0004995f8 pc=0x45d217
internal/poll.runtime_pollWait(0x7f8dba068de0, 0x72)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:351 +0x85 fp=0xc000499650 sp=0xc000499630 pc=0x497445
internal/poll.(*pollDesc).wait(0xc000710180?, 0x900000036?, 0x0)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000499678 sp=0xc000499650 pc=0x51e7a7
internal/poll.(*pollDesc).waitRead(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000710180)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_unix.go:620 +0x295 fp=0xc000499720 sp=0xc000499678 pc=0x523b75
net.(*netFD).accept(0xc000710180)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/fd_unix.go:172 +0x29 fp=0xc0004997d8 sp=0xc000499720 pc=0x596989
net.(*TCPListener).accept(0xc000786380)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/tcpsock_posix.go:159 +0x1b fp=0xc000499828 sp=0xc0004997d8 pc=0x5ac2fb
net.(*TCPListener).Accept(0xc000786380)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/tcpsock.go:380 +0x30 fp=0xc000499858 sp=0xc000499828 pc=0x5ab1b0
net/http.(*onceCloseListener).Accept(0xc0000fa360?)
:1 +0x24 fp=0xc000499870 sp=0xc000499858 pc=0x7c2724
net/http.(*Server).Serve(0xc000488300, {0x16b6df8, 0xc000786380})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3424 +0x30c fp=0xc0004999a0 sp=0xc000499870 pc=0x799fec
github.com/ollama/ollama/runner/llamarunner.Execute({0xc000132020, 0xe, 0xe})
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:992 +0x108a fp=0xc000499d08 sp=0xc0004999a0 pc=0x83d16a
github.com/ollama/ollama/runner.Execute({0xc000132010?, 0x0?, 0x0?})
/home/heathen-admin/ollama-for-amd/runner/runner.go:22 +0xd4 fp=0xc000499d30 sp=0xc000499d08 pc=0x9279d4
github.com/ollama/ollama/cmd.NewCLI.func2(0xc000133500?, {0x14e9c5c?, 0x4?, 0x14e9c60?})
/home/heathen-admin/ollama-for-amd/cmd/cmd.go:1327 +0x45 fp=0xc000499d58 sp=0xc000499d30 pc=0x1097f85
github.com/spf13/cobra.(*Command).execute(0xc0000fcf08, {0xc000557260, 0xe, 0xe})
/home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000499e78 sp=0xc000499d58 pc=0x60ff9c
github.com/spf13/cobra.(*Command).ExecuteC(0xc00079e908)
/home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000499f30 sp=0xc000499e78 pc=0x6107e5
github.com/spf13/cobra.(*Command).Execute(...)
/home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
/home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
/home/heathen-admin/ollama-for-amd/main.go:12 +0x4d fp=0xc000499f50 sp=0xc000499f30 pc=0x10982ed
runtime.main()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:283 +0x28b fp=0xc000499fe0 sp=0xc000499f50 pc=0x4647ab
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x49f881

goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000096fa8 sp=0xc000096f88 pc=0x49822e
runtime.goparkunlock(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441
runtime.forcegchelper()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:348 +0xb3 fp=0xc000096fe0 sp=0xc000096fa8 pc=0x464af3
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000096fe8 sp=0xc000096fe0 pc=0x49f881
created by runtime.init.7 in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:336 +0x1a

goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000097780 sp=0xc000097760 pc=0x49822e
runtime.goparkunlock(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441
runtime.bgsweep(0xc0000c2000)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcsweep.go:316 +0xdf fp=0xc0000977c8 sp=0xc000097780 pc=0x44f39f
runtime.gcenable.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc0000977e0 sp=0xc0000977c8 pc=0x443805
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000977e8 sp=0xc0000977e0 pc=0x49f881
created by runtime.gcenable in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:204 +0x66

goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]:
runtime.gopark(0x10000?, 0x16a4e90?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000097f78 sp=0xc000097f58 pc=0x49822e
runtime.goparkunlock(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441
runtime.(*scavengerState).park(0x1fd5b80)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000097fa8 sp=0xc000097f78 pc=0x44cde9
runtime.bgscavenge(0xc0000c2000)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000097fc8 sp=0xc000097fa8 pc=0x44d379
runtime.gcenable.gowrap2()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:205 +0x25 fp=0xc000097fe0 sp=0xc000097fc8 pc=0x4437a5
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x49f881
created by runtime.gcenable in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:205 +0xa5

goroutine 18 gp=0xc000102700 m=nil [finalizer wait]:
runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000096688?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000096630 sp=0xc000096610 pc=0x49822e
runtime.runfinq()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mfinal.go:196 +0x107 fp=0xc0000967e0 sp=0xc000096630 pc=0x4427c7
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000967e8 sp=0xc0000967e0 pc=0x49f881
created by runtime.createfing in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mfinal.go:166 +0x3d

goroutine 19 gp=0xc000103180 m=nil [chan receive]:
runtime.gopark(0xc00017f860?, 0xc000510018?, 0x60?, 0x27?, 0x57d6c8?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000092718 sp=0xc0000926f8 pc=0x49822e
runtime.chanrecv(0xc000110310, 0x0, 0x1)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/chan.go:664 +0x445 fp=0xc000092790 sp=0xc000092718 pc=0x434a05
runtime.chanrecv1(0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/chan.go:506 +0x12 fp=0xc0000927b8 sp=0xc000092790 pc=0x434592
runtime.unique_runtime_registerUniqueMapCleanup.func2(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1796
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1799 +0x2f fp=0xc0000927e0 sp=0xc0000927b8 pc=0x44694f
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000927e8 sp=0xc0000927e0 pc=0x49f881
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1794 +0x79

goroutine 20 gp=0xc000103500 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000092f38 sp=0xc000092f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000092fc8 sp=0xc000092f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000092fe0 sp=0xc000092fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000092fe8 sp=0xc000092fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 34 gp=0xc0003be000 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000484738 sp=0xc000484718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004847c8 sp=0xc000484738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004847e0 sp=0xc0004847c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004847e8 sp=0xc0004847e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 5 gp=0xc000003a40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000098738 sp=0xc000098718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0000987c8 sp=0xc000098738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0000987e0 sp=0xc0000987c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000987e8 sp=0xc0000987e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 21 gp=0xc0001036c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000093738 sp=0xc000093718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0000937c8 sp=0xc000093738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0000937e0 sp=0xc0000937c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000937e8 sp=0xc0000937e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 35 gp=0xc0003be1c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000484f38 sp=0xc000484f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000484fc8 sp=0xc000484f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000484fe0 sp=0xc000484fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000484fe8 sp=0xc000484fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 36 gp=0xc0003be380 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000485738 sp=0xc000485718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004857c8 sp=0xc000485738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004857e0 sp=0xc0004857c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004857e8 sp=0xc0004857e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 37 gp=0xc0003be540 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000485f38 sp=0xc000485f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000485fc8 sp=0xc000485f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000485fe0 sp=0xc000485fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 38 gp=0xc0003be700 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000486738 sp=0xc000486718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004867c8 sp=0xc000486738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004867e0 sp=0xc0004867c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004867e8 sp=0xc0004867e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 39 gp=0xc0003be8c0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000486f38 sp=0xc000486f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000486fc8 sp=0xc000486f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000486fe0 sp=0xc000486fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000486fe8 sp=0xc000486fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 40 gp=0xc0003bea80 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000487738 sp=0xc000487718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004877c8 sp=0xc000487738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004877e0 sp=0xc0004877c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004877e8 sp=0xc0004877e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 41 gp=0xc0003bec40 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000487f38 sp=0xc000487f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000487fc8 sp=0xc000487f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000487fe0 sp=0xc000487fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 42 gp=0xc0003bee00 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000480738 sp=0xc000480718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004807c8 sp=0xc000480738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004807e0 sp=0xc0004807c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004807e8 sp=0xc0004807e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 43 gp=0xc0003befc0 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000480f38 sp=0xc000480f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000480fc8 sp=0xc000480f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000480fe0 sp=0xc000480fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000480fe8 sp=0xc000480fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 44 gp=0xc0003bf180 m=nil [GC worker (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000481738 sp=0xc000481718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004817c8 sp=0xc000481738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004817e0 sp=0xc0004817c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004817e8 sp=0xc0004817e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 45 gp=0xc0003bf340 m=nil [GC worker (idle)]:
runtime.gopark(0x4b8fb78195?, 0x0?, 0x0?, 0x0?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000481f38 sp=0xc000481f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000481fc8 sp=0xc000481f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000481fe0 sp=0xc000481fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000481fe8 sp=0xc000481fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 46 gp=0xc0003bf500 m=nil [GC worker (idle)]:
runtime.gopark(0x20842e0?, 0x1?, 0x39?, 0xcc?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000482738 sp=0xc000482718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004827c8 sp=0xc000482738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004827e0 sp=0xc0004827c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004827e8 sp=0xc0004827e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 47 gp=0xc0003bf6c0 m=nil [GC worker (idle)]:
runtime.gopark(0x20842e0?, 0x1?, 0xe9?, 0xb1?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000482f38 sp=0xc000482f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000482fc8 sp=0xc000482f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000482fe0 sp=0xc000482fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000482fe8 sp=0xc000482fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 48 gp=0xc0003bf880 m=nil [GC worker (idle)]:
runtime.gopark(0x20842e0?, 0x1?, 0x5d?, 0xd1?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000483738 sp=0xc000483718 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004837c8 sp=0xc000483738 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004837e0 sp=0xc0004837c8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004837e8 sp=0xc0004837e0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 22 gp=0xc000103880 m=nil [GC worker (idle)]:
runtime.gopark(0x4b8fb52ef6?, 0x3?, 0x66?, 0x36?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000093f38 sp=0xc000093f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000093fc8 sp=0xc000093f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000093fe0 sp=0xc000093fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 6 gp=0xc000003c00 m=nil [GC worker (idle)]:
runtime.gopark(0x20842e0?, 0x1?, 0xfd?, 0x5?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000098f38 sp=0xc000098f18 pc=0x49822e
runtime.gcBgMarkWorker(0xc000111570)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000098fc8 sp=0xc000098f38 pc=0x445c69
runtime.gcBgMarkStartWorkers.gowrap1()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000098fe0 sp=0xc000098fc8 pc=0x445b45
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000098fe8 sp=0xc000098fe0 pc=0x49f881
created by runtime.gcBgMarkStartWorkers in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105

goroutine 24 gp=0xc000003dc0 m=nil [sync.WaitGroup.Wait]:
runtime.gopark(0x0?, 0x0?, 0x20?, 0x1?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000094618 sp=0xc0000945f8 pc=0x49822e
runtime.goparkunlock(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441
runtime.semacquire1(0xc000476008, 0x0, 0x1, 0x0, 0x18)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/sema.go:188 +0x21d fp=0xc000094680 sp=0xc000094618 pc=0x477cbd
sync.runtime_SemacquireWaitGroup(0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/sema.go:110 +0x25 fp=0xc0000946b8 sp=0xc000094680 pc=0x499c25
sync.(*WaitGroup).Wait(0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/sync/waitgroup.go:118 +0x48 fp=0xc0000946e0 sp=0xc0000946b8 pc=0x4ab2e8
github.com/ollama/ollama/runner/llamarunner.(*Server).run(0xc000476000, {0x16b90c0, 0xc00078a460})
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:316 +0x47 fp=0xc0000947b8 sp=0xc0000946e0 pc=0x8389e7
github.com/ollama/ollama/runner/llamarunner.Execute.gowrap2()
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:972 +0x28 fp=0xc0000947e0 sp=0xc0000947b8 pc=0x83d428
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000947e8 sp=0xc0000947e0 pc=0x49f881
created by github.com/ollama/ollama/runner/llamarunner.Execute in goroutine 1
/home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:972 +0xcb7

goroutine 7 gp=0xc000502700 m=nil [IO wait]:
runtime.gopark(0x521da5?, 0xc000545a00?, 0x40?, 0x5a?, 0xb?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000235948 sp=0xc000235928 pc=0x49822e
runtime.netpollblock(0x4bb5b8?, 0x431e86?, 0x0?)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:575 +0xf7 fp=0xc000235980 sp=0xc000235948 pc=0x45d217
internal/poll.runtime_pollWait(0x7f8dba068cc8, 0x72)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:351 +0x85 fp=0xc0002359a0 sp=0xc000235980 pc=0x497445
internal/poll.(*pollDesc).wait(0xc000545a00?, 0xc000021000?, 0x0)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0002359c8 sp=0xc0002359a0 pc=0x51e7a7
internal/poll.(*pollDesc).waitRead(...)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000545a00, {0xc000021000, 0x1000, 0x1000})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_unix.go:165 +0x27a fp=0xc000235a60 sp=0xc0002359c8 pc=0x51fa9a
net.(*netFD).Read(0xc000545a00, {0xc000021000?, 0xc000235ad0?, 0x51ec65?})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc000235aa8 sp=0xc000235a60 pc=0x5949e5
net.(*conn).Read(0xc00011c950, {0xc000021000?, 0x0?, 0x0?})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/net.go:194 +0x45 fp=0xc000235af0 sp=0xc000235aa8 pc=0x5a2d85
net/http.(*connReader).Read(0xc00017dd70, {0xc000021000, 0x1000, 0x1000})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:798 +0x159 fp=0xc000235b40 sp=0xc000235af0 pc=0x78ee99
bufio.(*Reader).fill(0xc0000c65a0)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/bufio/bufio.go:113 +0x103 fp=0xc000235b78 sp=0xc000235b40 pc=0x5ba503
bufio.(*Reader).Peek(0xc0000c65a0, 0x4)
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/bufio/bufio.go:152 +0x53 fp=0xc000235b98 sp=0xc000235b78 pc=0x5ba633
net/http.(*conn).serve(0xc0000fa360, {0x16b9088, 0xc000224840})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:2137 +0x785 fp=0xc000235fb8 sp=0xc000235b98 pc=0x794c85
net/http.(*Server).Serve.gowrap3()
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3454 +0x28 fp=0xc000235fe0 sp=0xc000235fb8 pc=0x79a3e8
runtime.goexit({})
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000235fe8 sp=0xc000235fe0 pc=0x49f881
created by net/http.(*Server).Serve in goroutine 1
/home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3454 +0x485

rax 0x7f8d71cfc5e0
rbx 0x7f8d58815a20
rcx 0x3
rdx 0x7f8d580051d0
rdi 0x7f8d52b08f28
rsi 0x3
rbp 0x0
rsp 0x7f8d71cfadc0
r8 0x0
r9 0x0
r10 0x1
r11 0x1
r12 0x7f8d71cfc5d8
r13 0x0
r14 0x18
r15 0x7f8cabe26580
rip 0x7f8d52964434
rflags 0x10202
cs 0x33
fs 0x0
gs 0x0
⠹ time=2025-03-17T02:36:34.588-04:00 level=ERROR source=sched.go:456 msg="error loading llama server" error="llama runner process has terminated: exit status 2"
[GIN] 2025/03/17 - 02:36:34 | 500 | 3.326278253s | 127.0.0.1 | POST "/api/generate"
Error: llama runner process has terminated: exit status 2

Originally created by @sanchez314c on GitHub (Mar 17, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9807 Been trying to resolve this for weeks now. Modify GPU.go to Rocm.....8 from 9. Add gfx803 to Cmakelist text and compile with: cmake -B build -DCMAKE_PREFIX_PATH=/opt/rocm -DAMDGPU_TARGETS=gfx803 -DLLAMA_HIPBLAS=1 -DLLAMA_CUBLAS=0 && cmake --build build -- -j18 At runtime I get all the mess below. I've tried other variants of cmake flags and can also get it to detect the card and open a model but when using it GPU usage is sporadic/random and never pegs much higher than 30%. Help please. ########################################## ./ollama run tinyllama [GIN] 2025/03/17 - 02:36:31 | 200 | 75.723µs | 127.0.0.1 | HEAD "/" [GIN] 2025/03/17 - 02:36:31 | 200 | 13.070198ms | 127.0.0.1 | POST "/api/show" time=2025-03-17T02:36:31.280-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.vision.block_count default=0 time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.key_length default=64 time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.value_length default=64 time=2025-03-17T02:36:31.281-04:00 level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 gpu=0 parallel=4 available=8071651328 required="1.7 GiB" time=2025-03-17T02:36:31.281-04:00 level=INFO source=server.go:105 msg="system memory" total="125.5 GiB" free="121.9 GiB" free_swap="8.0 GiB" time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.vision.block_count default=0 time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.key_length default=64 time=2025-03-17T02:36:31.281-04:00 level=WARN source=ggml.go:149 msg="key not found" key=llama.attention.value_length default=64 time=2025-03-17T02:36:31.281-04:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=23 layers.offload=23 layers.split="" memory.available="[7.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="1.7 GiB" memory.required.partial="1.7 GiB" memory.required.kv="176.0 MiB" memory.required.allocations="[1.7 GiB]" memory.weights.total="520.1 MiB" memory.weights.repeating="520.1 MiB" memory.weights.nonrepeating="51.3 MiB" memory.graph.full="544.0 MiB" memory.graph.partial="546.3 MiB" llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = TinyLlama llama_model_loader: - kv 2: llama.context_length u32 = 2048 llama_model_loader: - kv 3: llama.embedding_length u32 = 2048 llama_model_loader: - kv 4: llama.block_count u32 = 22 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632 llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64 llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4 llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000 llama_model_loader: - kv 11: general.file_type u32 = 2 llama_model_loader: - kv 12: tokenizer.ggml.model str = llama llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n... llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2 llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{% if m... llama_model_loader: - kv 22: general.quantization_version u32 = 2 llama_model_loader: - type f32: 45 tensors llama_model_loader: - type q4_0: 155 tensors llama_model_loader: - type q6_K: 1 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q4_0 print_info: file size = 606.53 MiB (4.63 BPW) load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect load: special tokens cache size = 3 load: token to piece cache size = 0.1684 MB print_info: arch = llama print_info: vocab_only = 1 print_info: model type = ?B print_info: model params = 1.10 B print_info: general.name = TinyLlama print_info: vocab type = SPM print_info: n_vocab = 32000 print_info: n_merges = 0 print_info: BOS token = 1 '<s>' print_info: EOS token = 2 '</s>' print_info: UNK token = 0 '<unk>' print_info: PAD token = 2 '</s>' print_info: LF token = 13 '<0x0A>' print_info: EOG token = 2 '</s>' print_info: max token length = 48 llama_model_load: vocab only - skipping tensors time=2025-03-17T02:36:31.324-04:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/home/heathen-admin/ollama-for-amd/ollama runner --model /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 --ctx-size 8192 --batch-size 512 --n-gpu-layers 23 --threads 10 --parallel 4 --port 43171" time=2025-03-17T02:36:31.325-04:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 time=2025-03-17T02:36:31.325-04:00 level=INFO source=server.go:585 msg="waiting for llama runner to start responding" time=2025-03-17T02:36:31.325-04:00 level=INFO source=server.go:619 msg="waiting for server to become available" status="llm server error" time=2025-03-17T02:36:31.335-04:00 level=INFO source=runner.go:931 msg="starting go runner" ⠇ ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 ROCm devices: Device 0: Radeon RX 580 Series, gfx803 (0x803), VMM: no, Wave Size: 64 load_backend: loaded ROCm backend from /home/heathen-admin/ollama-for-amd/build/lib/ollama/libggml-hip.so load_backend: loaded CPU backend from /home/heathen-admin/ollama-for-amd/build/lib/ollama/libggml-cpu-skylakex.so time=2025-03-17T02:36:33.250-04:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.AVX512=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) time=2025-03-17T02:36:33.251-04:00 level=INFO source=runner.go:991 msg="Server listening on 127.0.0.1:43171" llama_model_load_from_file_impl: using device ROCm0 (Radeon RX 580 Series) - 8120 MiB free ⠋ llama_model_loader: loaded meta data with 23 key-value pairs and 201 tensors from /home/heathen-admin/.ollama/models/blobs/sha256-2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = TinyLlama llama_model_loader: - kv 2: llama.context_length u32 = 2048 llama_model_loader: - kv 3: llama.embedding_length u32 = 2048 llama_model_loader: - kv 4: llama.block_count u32 = 22 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 5632 llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 64 llama_model_loader: - kv 7: llama.attention.head_count u32 = 32 llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 4 llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 10: llama.rope.freq_base f32 = 10000.000000 llama_model_loader: - kv 11: general.file_type u32 = 2 llama_model_loader: - kv 12: tokenizer.ggml.model str = llama llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... llama_model_loader: - kv 14: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n... llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 19: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 2 llama_model_loader: - kv 21: tokenizer.chat_template str = {% for message in messages %}\n{% if m... llama_model_loader: - kv 22: general.quantization_version u32 = 2 llama_model_loader: - type f32: 45 tensors llama_model_loader: - type q4_0: 155 tensors llama_model_loader: - type q6_K: 1 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q4_0 print_info: file size = 606.53 MiB (4.63 BPW) load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect load: special tokens cache size = 3 load: token to piece cache size = 0.1684 MB print_info: arch = llama print_info: vocab_only = 0 print_info: n_ctx_train = 2048 print_info: n_embd = 2048 print_info: n_layer = 22 print_info: n_head = 32 print_info: n_head_kv = 4 print_info: n_rot = 64 print_info: n_swa = 0 print_info: n_embd_head_k = 64 print_info: n_embd_head_v = 64 print_info: n_gqa = 8 print_info: n_embd_k_gqa = 256 print_info: n_embd_v_gqa = 256 print_info: f_norm_eps = 0.0e+00 print_info: f_norm_rms_eps = 1.0e-05 print_info: f_clamp_kqv = 0.0e+00 print_info: f_max_alibi_bias = 0.0e+00 print_info: f_logit_scale = 0.0e+00 print_info: n_ff = 5632 print_info: n_expert = 0 print_info: n_expert_used = 0 print_info: causal attn = 1 print_info: pooling type = 0 print_info: rope type = 0 print_info: rope scaling = linear print_info: freq_base_train = 10000.0 print_info: freq_scale_train = 1 print_info: n_ctx_orig_yarn = 2048 print_info: rope_finetuned = unknown print_info: ssm_d_conv = 0 print_info: ssm_d_inner = 0 print_info: ssm_d_state = 0 print_info: ssm_dt_rank = 0 print_info: ssm_dt_b_c_rms = 0 print_info: model type = 1B print_info: model params = 1.10 B print_info: general.name = TinyLlama print_info: vocab type = SPM print_info: n_vocab = 32000 print_info: n_merges = 0 print_info: BOS token = 1 '<s>' print_info: EOS token = 2 '</s>' print_info: UNK token = 0 '<unk>' print_info: PAD token = 2 '</s>' print_info: LF token = 13 '<0x0A>' print_info: EOG token = 2 '</s>' print_info: max token length = 48 load_tensors: loading model tensors, this can take a while... (mmap = true) time=2025-03-17T02:36:33.333-04:00 level=INFO source=server.go:619 msg="waiting for server to become available" status="llm server loading model" ⠋ load_tensors: offloading 22 repeating layers to GPU load_tensors: offloading output layer to GPU load_tensors: offloaded 23/23 layers to GPU load_tensors: ROCm0 model buffer size = 571.37 MiB load_tensors: CPU_Mapped model buffer size = 35.16 MiB SIGSEGV: segmentation violation PC=0x7f8d52964434 m=5 sigcode=1 addr=0x18 signal arrived during cgo execution goroutine 23 gp=0xc000002000 m=5 mp=0xc000100008 [syscall]: runtime.cgocall(0x1105a00, 0xc0000a9c00) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/cgocall.go:167 +0x4b fp=0xc0000a9bd8 sp=0xc0000a9ba0 pc=0x494f4b github.com/ollama/ollama/llama._Cfunc_llama_model_load_from_file(0x7f8d58000b70, {0x0, 0x17, 0x1, 0x0, 0x0, 0x11051b0, 0xc00053cab8, 0x0, 0x0, ...}) _cgo_gotypes.go:797 +0x47 fp=0xc0000a9c00 sp=0xc0000a9bd8 pc=0x82d167 github.com/ollama/ollama/llama.LoadModelFromFile.func1(...) /home/heathen-admin/ollama-for-amd/llama/llama.go:240 github.com/ollama/ollama/llama.LoadModelFromFile({0x7ffe17091d2f, 0x70}, {0x17, 0x0, 0x1, 0x0, {0x0, 0x0, 0x0}, 0xc0005481c0, ...}) /home/heathen-admin/ollama-for-amd/llama/llama.go:240 +0x36b fp=0xc0000a9dc8 sp=0xc0000a9c00 pc=0x82fccb github.com/ollama/ollama/runner/llamarunner.(*Server).loadModel(0xc000476000, {0x17, 0x0, 0x1, 0x0, {0x0, 0x0, 0x0}, 0xc0005481c0, 0x0}, ...) /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:849 +0x9b fp=0xc0000a9f10 sp=0xc0000a9dc8 pc=0x83bd5b github.com/ollama/ollama/runner/llamarunner.Execute.gowrap1() /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:965 +0xda fp=0xc0000a9fe0 sp=0xc0000a9f10 pc=0x83d53a runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000a9fe8 sp=0xc0000a9fe0 pc=0x49f881 created by github.com/ollama/ollama/runner/llamarunner.Execute in goroutine 1 /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:965 +0xbd7 goroutine 1 gp=0xc000002380 m=nil [IO wait]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc0004995f8 sp=0xc0004995d8 pc=0x49822e runtime.netpollblock(0xc000499648?, 0x431e86?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:575 +0xf7 fp=0xc000499630 sp=0xc0004995f8 pc=0x45d217 internal/poll.runtime_pollWait(0x7f8dba068de0, 0x72) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:351 +0x85 fp=0xc000499650 sp=0xc000499630 pc=0x497445 internal/poll.(*pollDesc).wait(0xc000710180?, 0x900000036?, 0x0) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000499678 sp=0xc000499650 pc=0x51e7a7 internal/poll.(*pollDesc).waitRead(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc000710180) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_unix.go:620 +0x295 fp=0xc000499720 sp=0xc000499678 pc=0x523b75 net.(*netFD).accept(0xc000710180) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/fd_unix.go:172 +0x29 fp=0xc0004997d8 sp=0xc000499720 pc=0x596989 net.(*TCPListener).accept(0xc000786380) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/tcpsock_posix.go:159 +0x1b fp=0xc000499828 sp=0xc0004997d8 pc=0x5ac2fb net.(*TCPListener).Accept(0xc000786380) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/tcpsock.go:380 +0x30 fp=0xc000499858 sp=0xc000499828 pc=0x5ab1b0 net/http.(*onceCloseListener).Accept(0xc0000fa360?) <autogenerated>:1 +0x24 fp=0xc000499870 sp=0xc000499858 pc=0x7c2724 net/http.(*Server).Serve(0xc000488300, {0x16b6df8, 0xc000786380}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3424 +0x30c fp=0xc0004999a0 sp=0xc000499870 pc=0x799fec github.com/ollama/ollama/runner/llamarunner.Execute({0xc000132020, 0xe, 0xe}) /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:992 +0x108a fp=0xc000499d08 sp=0xc0004999a0 pc=0x83d16a github.com/ollama/ollama/runner.Execute({0xc000132010?, 0x0?, 0x0?}) /home/heathen-admin/ollama-for-amd/runner/runner.go:22 +0xd4 fp=0xc000499d30 sp=0xc000499d08 pc=0x9279d4 github.com/ollama/ollama/cmd.NewCLI.func2(0xc000133500?, {0x14e9c5c?, 0x4?, 0x14e9c60?}) /home/heathen-admin/ollama-for-amd/cmd/cmd.go:1327 +0x45 fp=0xc000499d58 sp=0xc000499d30 pc=0x1097f85 github.com/spf13/cobra.(*Command).execute(0xc0000fcf08, {0xc000557260, 0xe, 0xe}) /home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x85c fp=0xc000499e78 sp=0xc000499d58 pc=0x60ff9c github.com/spf13/cobra.(*Command).ExecuteC(0xc00079e908) /home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc000499f30 sp=0xc000499e78 pc=0x6107e5 github.com/spf13/cobra.(*Command).Execute(...) /home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) /home/heathen-admin/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 main.main() /home/heathen-admin/ollama-for-amd/main.go:12 +0x4d fp=0xc000499f50 sp=0xc000499f30 pc=0x10982ed runtime.main() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:283 +0x28b fp=0xc000499fe0 sp=0xc000499f50 pc=0x4647ab runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000499fe8 sp=0xc000499fe0 pc=0x49f881 goroutine 2 gp=0xc000002e00 m=nil [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000096fa8 sp=0xc000096f88 pc=0x49822e runtime.goparkunlock(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441 runtime.forcegchelper() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:348 +0xb3 fp=0xc000096fe0 sp=0xc000096fa8 pc=0x464af3 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000096fe8 sp=0xc000096fe0 pc=0x49f881 created by runtime.init.7 in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:336 +0x1a goroutine 3 gp=0xc000003340 m=nil [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000097780 sp=0xc000097760 pc=0x49822e runtime.goparkunlock(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441 runtime.bgsweep(0xc0000c2000) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcsweep.go:316 +0xdf fp=0xc0000977c8 sp=0xc000097780 pc=0x44f39f runtime.gcenable.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:204 +0x25 fp=0xc0000977e0 sp=0xc0000977c8 pc=0x443805 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000977e8 sp=0xc0000977e0 pc=0x49f881 created by runtime.gcenable in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:204 +0x66 goroutine 4 gp=0xc000003500 m=nil [GC scavenge wait]: runtime.gopark(0x10000?, 0x16a4e90?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000097f78 sp=0xc000097f58 pc=0x49822e runtime.goparkunlock(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441 runtime.(*scavengerState).park(0x1fd5b80) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000097fa8 sp=0xc000097f78 pc=0x44cde9 runtime.bgscavenge(0xc0000c2000) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000097fc8 sp=0xc000097fa8 pc=0x44d379 runtime.gcenable.gowrap2() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:205 +0x25 fp=0xc000097fe0 sp=0xc000097fc8 pc=0x4437a5 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000097fe8 sp=0xc000097fe0 pc=0x49f881 created by runtime.gcenable in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:205 +0xa5 goroutine 18 gp=0xc000102700 m=nil [finalizer wait]: runtime.gopark(0x1b8?, 0xc000002380?, 0x1?, 0x23?, 0xc000096688?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000096630 sp=0xc000096610 pc=0x49822e runtime.runfinq() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mfinal.go:196 +0x107 fp=0xc0000967e0 sp=0xc000096630 pc=0x4427c7 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000967e8 sp=0xc0000967e0 pc=0x49f881 created by runtime.createfing in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mfinal.go:166 +0x3d goroutine 19 gp=0xc000103180 m=nil [chan receive]: runtime.gopark(0xc00017f860?, 0xc000510018?, 0x60?, 0x27?, 0x57d6c8?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000092718 sp=0xc0000926f8 pc=0x49822e runtime.chanrecv(0xc000110310, 0x0, 0x1) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/chan.go:664 +0x445 fp=0xc000092790 sp=0xc000092718 pc=0x434a05 runtime.chanrecv1(0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/chan.go:506 +0x12 fp=0xc0000927b8 sp=0xc000092790 pc=0x434592 runtime.unique_runtime_registerUniqueMapCleanup.func2(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1796 runtime.unique_runtime_registerUniqueMapCleanup.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1799 +0x2f fp=0xc0000927e0 sp=0xc0000927b8 pc=0x44694f runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000927e8 sp=0xc0000927e0 pc=0x49f881 created by unique.runtime_registerUniqueMapCleanup in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1794 +0x79 goroutine 20 gp=0xc000103500 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000092f38 sp=0xc000092f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000092fc8 sp=0xc000092f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000092fe0 sp=0xc000092fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000092fe8 sp=0xc000092fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 34 gp=0xc0003be000 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000484738 sp=0xc000484718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004847c8 sp=0xc000484738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004847e0 sp=0xc0004847c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004847e8 sp=0xc0004847e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 5 gp=0xc000003a40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000098738 sp=0xc000098718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0000987c8 sp=0xc000098738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0000987e0 sp=0xc0000987c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000987e8 sp=0xc0000987e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 21 gp=0xc0001036c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000093738 sp=0xc000093718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0000937c8 sp=0xc000093738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0000937e0 sp=0xc0000937c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000937e8 sp=0xc0000937e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 35 gp=0xc0003be1c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000484f38 sp=0xc000484f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000484fc8 sp=0xc000484f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000484fe0 sp=0xc000484fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000484fe8 sp=0xc000484fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 36 gp=0xc0003be380 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000485738 sp=0xc000485718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004857c8 sp=0xc000485738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004857e0 sp=0xc0004857c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004857e8 sp=0xc0004857e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 37 gp=0xc0003be540 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000485f38 sp=0xc000485f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000485fc8 sp=0xc000485f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000485fe0 sp=0xc000485fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000485fe8 sp=0xc000485fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 38 gp=0xc0003be700 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000486738 sp=0xc000486718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004867c8 sp=0xc000486738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004867e0 sp=0xc0004867c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004867e8 sp=0xc0004867e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 39 gp=0xc0003be8c0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000486f38 sp=0xc000486f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000486fc8 sp=0xc000486f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000486fe0 sp=0xc000486fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000486fe8 sp=0xc000486fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 40 gp=0xc0003bea80 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000487738 sp=0xc000487718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004877c8 sp=0xc000487738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004877e0 sp=0xc0004877c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004877e8 sp=0xc0004877e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 41 gp=0xc0003bec40 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000487f38 sp=0xc000487f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000487fc8 sp=0xc000487f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000487fe0 sp=0xc000487fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000487fe8 sp=0xc000487fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 42 gp=0xc0003bee00 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000480738 sp=0xc000480718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004807c8 sp=0xc000480738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004807e0 sp=0xc0004807c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004807e8 sp=0xc0004807e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 43 gp=0xc0003befc0 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000480f38 sp=0xc000480f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000480fc8 sp=0xc000480f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000480fe0 sp=0xc000480fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000480fe8 sp=0xc000480fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 44 gp=0xc0003bf180 m=nil [GC worker (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000481738 sp=0xc000481718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004817c8 sp=0xc000481738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004817e0 sp=0xc0004817c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004817e8 sp=0xc0004817e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 45 gp=0xc0003bf340 m=nil [GC worker (idle)]: runtime.gopark(0x4b8fb78195?, 0x0?, 0x0?, 0x0?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000481f38 sp=0xc000481f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000481fc8 sp=0xc000481f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000481fe0 sp=0xc000481fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000481fe8 sp=0xc000481fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 46 gp=0xc0003bf500 m=nil [GC worker (idle)]: runtime.gopark(0x20842e0?, 0x1?, 0x39?, 0xcc?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000482738 sp=0xc000482718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004827c8 sp=0xc000482738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004827e0 sp=0xc0004827c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004827e8 sp=0xc0004827e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 47 gp=0xc0003bf6c0 m=nil [GC worker (idle)]: runtime.gopark(0x20842e0?, 0x1?, 0xe9?, 0xb1?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000482f38 sp=0xc000482f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000482fc8 sp=0xc000482f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000482fe0 sp=0xc000482fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000482fe8 sp=0xc000482fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 48 gp=0xc0003bf880 m=nil [GC worker (idle)]: runtime.gopark(0x20842e0?, 0x1?, 0x5d?, 0xd1?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000483738 sp=0xc000483718 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc0004837c8 sp=0xc000483738 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc0004837e0 sp=0xc0004837c8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0004837e8 sp=0xc0004837e0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 22 gp=0xc000103880 m=nil [GC worker (idle)]: runtime.gopark(0x4b8fb52ef6?, 0x3?, 0x66?, 0x36?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000093f38 sp=0xc000093f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000093fc8 sp=0xc000093f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000093fe0 sp=0xc000093fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 6 gp=0xc000003c00 m=nil [GC worker (idle)]: runtime.gopark(0x20842e0?, 0x1?, 0xfd?, 0x5?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000098f38 sp=0xc000098f18 pc=0x49822e runtime.gcBgMarkWorker(0xc000111570) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1423 +0xe9 fp=0xc000098fc8 sp=0xc000098f38 pc=0x445c69 runtime.gcBgMarkStartWorkers.gowrap1() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x25 fp=0xc000098fe0 sp=0xc000098fc8 pc=0x445b45 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000098fe8 sp=0xc000098fe0 pc=0x49f881 created by runtime.gcBgMarkStartWorkers in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/mgc.go:1339 +0x105 goroutine 24 gp=0xc000003dc0 m=nil [sync.WaitGroup.Wait]: runtime.gopark(0x0?, 0x0?, 0x20?, 0x1?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000094618 sp=0xc0000945f8 pc=0x49822e runtime.goparkunlock(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:441 runtime.semacquire1(0xc000476008, 0x0, 0x1, 0x0, 0x18) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/sema.go:188 +0x21d fp=0xc000094680 sp=0xc000094618 pc=0x477cbd sync.runtime_SemacquireWaitGroup(0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/sema.go:110 +0x25 fp=0xc0000946b8 sp=0xc000094680 pc=0x499c25 sync.(*WaitGroup).Wait(0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/sync/waitgroup.go:118 +0x48 fp=0xc0000946e0 sp=0xc0000946b8 pc=0x4ab2e8 github.com/ollama/ollama/runner/llamarunner.(*Server).run(0xc000476000, {0x16b90c0, 0xc00078a460}) /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:316 +0x47 fp=0xc0000947b8 sp=0xc0000946e0 pc=0x8389e7 github.com/ollama/ollama/runner/llamarunner.Execute.gowrap2() /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:972 +0x28 fp=0xc0000947e0 sp=0xc0000947b8 pc=0x83d428 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc0000947e8 sp=0xc0000947e0 pc=0x49f881 created by github.com/ollama/ollama/runner/llamarunner.Execute in goroutine 1 /home/heathen-admin/ollama-for-amd/runner/llamarunner/runner.go:972 +0xcb7 goroutine 7 gp=0xc000502700 m=nil [IO wait]: runtime.gopark(0x521da5?, 0xc000545a00?, 0x40?, 0x5a?, 0xb?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/proc.go:435 +0xce fp=0xc000235948 sp=0xc000235928 pc=0x49822e runtime.netpollblock(0x4bb5b8?, 0x431e86?, 0x0?) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:575 +0xf7 fp=0xc000235980 sp=0xc000235948 pc=0x45d217 internal/poll.runtime_pollWait(0x7f8dba068cc8, 0x72) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/netpoll.go:351 +0x85 fp=0xc0002359a0 sp=0xc000235980 pc=0x497445 internal/poll.(*pollDesc).wait(0xc000545a00?, 0xc000021000?, 0x0) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc0002359c8 sp=0xc0002359a0 pc=0x51e7a7 internal/poll.(*pollDesc).waitRead(...) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc000545a00, {0xc000021000, 0x1000, 0x1000}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/internal/poll/fd_unix.go:165 +0x27a fp=0xc000235a60 sp=0xc0002359c8 pc=0x51fa9a net.(*netFD).Read(0xc000545a00, {0xc000021000?, 0xc000235ad0?, 0x51ec65?}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/fd_posix.go:55 +0x25 fp=0xc000235aa8 sp=0xc000235a60 pc=0x5949e5 net.(*conn).Read(0xc00011c950, {0xc000021000?, 0x0?, 0x0?}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/net.go:194 +0x45 fp=0xc000235af0 sp=0xc000235aa8 pc=0x5a2d85 net/http.(*connReader).Read(0xc00017dd70, {0xc000021000, 0x1000, 0x1000}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:798 +0x159 fp=0xc000235b40 sp=0xc000235af0 pc=0x78ee99 bufio.(*Reader).fill(0xc0000c65a0) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/bufio/bufio.go:113 +0x103 fp=0xc000235b78 sp=0xc000235b40 pc=0x5ba503 bufio.(*Reader).Peek(0xc0000c65a0, 0x4) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/bufio/bufio.go:152 +0x53 fp=0xc000235b98 sp=0xc000235b78 pc=0x5ba633 net/http.(*conn).serve(0xc0000fa360, {0x16b9088, 0xc000224840}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:2137 +0x785 fp=0xc000235fb8 sp=0xc000235b98 pc=0x794c85 net/http.(*Server).Serve.gowrap3() /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3454 +0x28 fp=0xc000235fe0 sp=0xc000235fb8 pc=0x79a3e8 runtime.goexit({}) /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/runtime/asm_amd64.s:1700 +0x1 fp=0xc000235fe8 sp=0xc000235fe0 pc=0x49f881 created by net/http.(*Server).Serve in goroutine 1 /home/heathen-admin/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.0.linux-amd64/src/net/http/server.go:3454 +0x485 rax 0x7f8d71cfc5e0 rbx 0x7f8d58815a20 rcx 0x3 rdx 0x7f8d580051d0 rdi 0x7f8d52b08f28 rsi 0x3 rbp 0x0 rsp 0x7f8d71cfadc0 r8 0x0 r9 0x0 r10 0x1 r11 0x1 r12 0x7f8d71cfc5d8 r13 0x0 r14 0x18 r15 0x7f8cabe26580 rip 0x7f8d52964434 rflags 0x10202 cs 0x33 fs 0x0 gs 0x0 ⠹ time=2025-03-17T02:36:34.588-04:00 level=ERROR source=sched.go:456 msg="error loading llama server" error="llama runner process has terminated: exit status 2" [GIN] 2025/03/17 - 02:36:34 | 500 | 3.326278253s | 127.0.0.1 | POST "/api/generate" Error: llama runner process has terminated: exit status 2
Author
Owner

@sanchez314c commented on GitHub (Mar 17, 2025):

BUMP......

<!-- gh-comment-id:2731023981 --> @sanchez314c commented on GitHub (Mar 17, 2025): BUMP......
Author
Owner

@robertrosenbusch commented on GitHub (Apr 9, 2025):

@sanchez314c : Just a hint...

Caution

Prevent ROCm SegFaults on your Linux Distro

After some feedback/research from Users who are using the Dockercontainer from my GIT in Ollama and PyTorch/ComfyUI, cause the devices /dev/dri and /dev/kfd crashed with SegFaults. Please proofe your used Linux-Kernel Version and switch up or down to a well known working Kernel-Version. Fedora 41, Arch and Debian 13 using (in April 2015) suspicious Linux-Kernel-Versions as default.

Kernel Version 5.19 6.2 6.8 6.9 6.10 6.11 6.12 6.13 6.14
working on ROCm 6.3 for Ollama/PyTorch 🟥 🟥
<!-- gh-comment-id:2788645172 --> @robertrosenbusch commented on GitHub (Apr 9, 2025): @sanchez314c : Just a hint... > [!CAUTION] > #### Prevent ROCm SegFaults on your Linux Distro > After some feedback/research from Users who are using the Dockercontainer from my GIT in [Ollama](https://github.com/robertrosenbusch/gfx803_rocm/issues/8#issue-2919996555) and [PyTorch/ComfyUI](https://github.com/robertrosenbusch/gfx803_rocm/issues/13#issuecomment-2754796999), cause the devices `/dev/dri` and `/dev/kfd` crashed with SegFaults. Please proofe your used Linux-Kernel Version and switch up or down to a well known working Kernel-Version. Fedora 41, Arch and Debian 13 using (in April 2015) suspicious Linux-Kernel-Versions as default. > |Kernel Version|5.19|6.2|6.8|6.9|6.10|6.11|6.12|6.13|6.14| > |--------------|-----|-----|------|-----|------|-----|-----|-----|-----| > |working on ROCm 6.3 for Ollama/PyTorch|✅|✅|✅|✅|✅|✅|🟥|🟥|✅|
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#6416