github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2054] ROCM crash when loading model with integrated GPU #26950

New Issue
Closed
opened 2026-04-22 03:44:44 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-22 03:44:44 -05:00
Owner
Copy Link

Originally created by @Gagootron on GitHub (Jan 18, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2054

Originally assigned to: @dhiltgen on GitHub.

When running version 0.1.20 on my computer the ollama server crashes when loading any model.

Computer Specs:

  • GPU: RX7900XTX
  • CPU: 7800X3D
  • RAM: 32G
  • OS: Arch Linux
  • ROCM Version: 5.7.1
  • Kernel: 6.7.0

Server log output:

2024/01/18 17:15:39 images.go:808: total blobs: 14
2024/01/18 17:15:39 images.go:815: total unused blobs removed: 0
2024/01/18 17:15:39 routes.go:930: Listening on 127.0.0.1:11434 (version 0.1.20)
2024/01/18 17:15:39 shim_ext_server.go:142: Dynamic LLM variants [cuda rocm]
2024/01/18 17:15:39 gpu.go:88: Detecting GPU type
2024/01/18 17:15:39 gpu.go:203: Searching for GPU management library libnvidia-ml.so
2024/01/18 17:15:39 gpu.go:248: Discovered GPU libraries: []
2024/01/18 17:15:39 gpu.go:203: Searching for GPU management library librocm_smi64.so
2024/01/18 17:15:39 gpu.go:248: Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0]
2024/01/18 17:15:39 gpu.go:104: Radeon GPU detected
[GIN] 2024/01/18 - 17:15:40 | 200 |       28.41µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/01/18 - 17:15:40 | 200 |      353.04µs |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/01/18 - 17:15:40 | 200 |      179.68µs |       127.0.0.1 | POST     "/api/show"
2024/01/18 17:15:40 shim_ext_server_linux.go:24: Updating PATH to /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/lib/rustup/bin:/home/user/bin:/home/user/.cargo/bin:/tmp/ollama1188601244/rocm
2024/01/18 17:15:40 shim_ext_server.go:92: Loading Dynamic Shim llm server: /tmp/ollama1188601244/rocm/libext_server.so
2024/01/18 17:15:40 ext_server_common.go:136: Initializing internal llama server
free(): invalid pointer

After that failed I compiled ollama myself (Using commit d5a7353357), there i got a completely different error with the following log after loading a model: crash.txt

I'm not shure if i made an error with my setup, or if this is a bug in ollama. But I got other AIs working like the stable diffusion webui working, so i would think that my ROCM installation works.

Originally created by @Gagootron on GitHub (Jan 18, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2054 Originally assigned to: @dhiltgen on GitHub. When running version 0.1.20 on my computer the ollama server crashes when loading any model. Computer Specs: * GPU: RX7900XTX * CPU: 7800X3D * RAM: 32G * OS: Arch Linux * ROCM Version: 5.7.1 * Kernel: 6.7.0 Server log output: ``` 2024/01/18 17:15:39 images.go:808: total blobs: 14 2024/01/18 17:15:39 images.go:815: total unused blobs removed: 0 2024/01/18 17:15:39 routes.go:930: Listening on 127.0.0.1:11434 (version 0.1.20) 2024/01/18 17:15:39 shim_ext_server.go:142: Dynamic LLM variants [cuda rocm] 2024/01/18 17:15:39 gpu.go:88: Detecting GPU type 2024/01/18 17:15:39 gpu.go:203: Searching for GPU management library libnvidia-ml.so 2024/01/18 17:15:39 gpu.go:248: Discovered GPU libraries: [] 2024/01/18 17:15:39 gpu.go:203: Searching for GPU management library librocm_smi64.so 2024/01/18 17:15:39 gpu.go:248: Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0] 2024/01/18 17:15:39 gpu.go:104: Radeon GPU detected [GIN] 2024/01/18 - 17:15:40 | 200 | 28.41µs | 127.0.0.1 | HEAD "/" [GIN] 2024/01/18 - 17:15:40 | 200 | 353.04µs | 127.0.0.1 | POST "/api/show" [GIN] 2024/01/18 - 17:15:40 | 200 | 179.68µs | 127.0.0.1 | POST "/api/show" 2024/01/18 17:15:40 shim_ext_server_linux.go:24: Updating PATH to /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/lib/rustup/bin:/home/user/bin:/home/user/.cargo/bin:/tmp/ollama1188601244/rocm 2024/01/18 17:15:40 shim_ext_server.go:92: Loading Dynamic Shim llm server: /tmp/ollama1188601244/rocm/libext_server.so 2024/01/18 17:15:40 ext_server_common.go:136: Initializing internal llama server free(): invalid pointer ``` After that failed I compiled ollama myself (Using commit d5a73533574acb02069e74f1d01f6775577391bc), there i got a completely different error with the following log after loading a model: [crash.txt](https://github.com/jmorganca/ollama/files/13979234/fail.txt) I'm not shure if i made an error with my setup, or if this is a bug in ollama. But I got other AIs working like the [stable diffusion webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui) working, so i would think that my ROCM installation works.
GiteaMirror added the bugamd labels 2026-04-22 03:44:44 -05:00
GiteaMirror commented 2026-04-22 03:44:45 -05:00
Author
Owner
Copy Link

@xlmnxp commented on GitHub (Jan 22, 2024):

Same issue but with latest docker image (I built it my self docker build -t ollama/ollama .)

Computer Specs:

GPU: RX 7900 XTX
CPU: R9 7950X
RAM: 64GB
OS: Ubuntu 23.10 (Docker Container)
ROCM Version: 6.0.0
Kernel: 6.5.0
Server log output:

2024/01/22 09:49:51 images.go:810: INFO total blobs: 6
2024/01/22 09:49:51 images.go:817: INFO total unused blobs removed: 0
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /api/pull                 --> github.com/jmorganca/ollama/server.PullModelHandler (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/jmorganca/ollama/server.GenerateHandler (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/jmorganca/ollama/server.ChatHandler (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/jmorganca/ollama/server.EmbeddingHandler (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/jmorganca/ollama/server.CreateModelHandler (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/jmorganca/ollama/server.PushModelHandler (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/jmorganca/ollama/server.CopyModelHandler (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/jmorganca/ollama/server.DeleteModelHandler (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/jmorganca/ollama/server.ShowModelHandler (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/jmorganca/ollama/server.CreateBlobHandler (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/jmorganca/ollama/server.HeadBlobHandler (5 handlers)
[GIN-debug] GET    /                         --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers)
2024/01/22 09:49:51 routes.go:943: INFO Listening on [::]:11434 (version 0.0.0)
2024/01/22 09:49:51 payload_common.go:106: INFO Extracting dynamic libraries...
2024/01/22 09:49:52 payload_common.go:145: INFO Dynamic LLM libraries [rocm_v5 rocm_v6 cuda_v11 cpu cpu_avx cpu_avx2]
2024/01/22 09:49:52 gpu.go:91: INFO Detecting GPU type
2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library libnvidia-ml.so
2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: []
2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library librocm_smi64.so
2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.6.0.60000 /opt/rocm-6.0.0/lib/librocm_smi64.so.6.0.60000]
2024/01/22 09:49:52 gpu.go:106: INFO Radeon GPU detected
2024/01/22 09:50:03 cpu_common.go:11: INFO CPU has AVX2
2024/01/22 09:50:03 dyn_ext_server.go:90: INFO Loading Dynamic llm server: /tmp/ollama2441091586/rocm_v6/libext_server.so
2024/01/22 09:50:03 dyn_ext_server.go:139: INFO Initializing llama server
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 2 ROCm devices:
  Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no
  Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no
llama_model_loader: loaded meta data with 23 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256:2609048d349e7c70196401be59bea7eb89a968d4642e409b0e798b34403b96c8 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = LLaMA v2
llama_model_loader: - kv   2:                       llama.context_length u32              = 4096
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 5120
llama_model_loader: - kv   4:                          llama.block_count u32              = 40
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 13824
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 40
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 40
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                          general.file_type u32              = 2
llama_model_loader: - kv  11:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  15:                      tokenizer.ggml.merges arr[str,61249]   = ["▁ t", "e r", "i n", "▁ a", "e n...
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  19:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  20:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  21:                    tokenizer.chat_template str              = {% if messages[0]['role'] == 'system'...
llama_model_loader: - kv  22:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   81 tensors
llama_model_loader: - type q4_0:  281 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 4096
llm_load_print_meta: n_embd           = 5120
llm_load_print_meta: n_head           = 40
llm_load_print_meta: n_head_kv        = 40
llm_load_print_meta: n_layer          = 40
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 1
llm_load_print_meta: n_embd_k_gqa     = 5120
llm_load_print_meta: n_embd_v_gqa     = 5120
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 13824
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 4096
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 13B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 13.02 B
llm_load_print_meta: model size       = 6.86 GiB (4.53 BPW) 
llm_load_print_meta: general.name     = LLaMA v2
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size       =    0.14 MiB
llm_load_tensors: using ROCm for GPU acceleration
llm_load_tensors: system memory used  =   88.03 MiB
llm_load_tensors: VRAM used           = 6936.01 MiB
llm_load_tensors: offloading 40 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 41/41 layers to GPU
...................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: VRAM kv self = 1600.00 MB
llama_new_context_with_model: KV self size  = 1600.00 MiB, K (f16):  800.00 MiB, V (f16):  800.00 MiB
llama_build_graph: non-view tensors processed: 844/844
llama_new_context_with_model: compute buffer total size = 197.19 MiB
llama_new_context_with_model: VRAM scratch buffer: 194.00 MiB
llama_new_context_with_model: total VRAM used: 8730.01 MiB (model: 6936.01 MiB, context: 1794.00 MiB)
CUDA error: shared object initialization failed
  current device: 0, in function ggml_cuda_op_flatten at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:8688
  hipGetLastError()
GGML_ASSERT: /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:229: !"CUDA error"
loading library /tmp/ollama2441091586/rocm_v6/libext_server.so
No symbol table is loaded.  Use the "file" command.
ptrace: Operation not permitted.
No stack.
The program is not being run.
SIGABRT: abort
PC=0x7fb4b251d387 m=31 sigcode=18446744073709551610
signal arrived during cgo execution

goroutine 66 [syscall]:
runtime.cgocall(0x9b4670, 0xc00055e808)
        /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc00055e7e0 sp=0xc00055e7a8 pc=0x409b0b
github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7fb410000e00, 0x7fb409a545a0, 0x7fb409a54cf0, 0x7fb409a54d80, 0x7fb409a54f30, 0x7fb409a550a0, 0x7fb409a55560, 0x7fb409a55540, 0x7fb409a555f0, 0x7fb409a55ba0, ...}, ...)
        _cgo_gotypes.go:280 +0x45 fp=0xc00055e808 sp=0xc00055e7e0 pc=0x7c2b25
github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xae6f99?, 0x62?)
        /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xef fp=0xc00055e8f8 sp=0xc00055e808 pc=0x7c3fcf
github.com/jmorganca/ollama/llm.newDynExtServer({0xc00002a840, 0x2e}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
        /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xa32 fp=0xc00055eb88 sp=0xc00055e8f8 pc=0x7c3cd2
github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...)
        /go/src/github.com/jmorganca/ollama/llm/llm.go:147 +0x36a fp=0xc00055ed48 sp=0xc00055eb88 pc=0x7c04ea
github.com/jmorganca/ollama/llm.New({0x0?, 0x1000100000100?}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...)
        /go/src/github.com/jmorganca/ollama/llm/llm.go:122 +0x6f9 fp=0xc00055efb8 sp=0xc00055ed48 pc=0x7bff19
github.com/jmorganca/ollama/server.load(0xc000002000?, 0xc000002000, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...)
        /go/src/github.com/jmorganca/ollama/server/routes.go:83 +0x3a5 fp=0xc00055f138 sp=0xc00055efb8 pc=0x9909c5
github.com/jmorganca/ollama/server.ChatHandler(0xc0004a0b00)
        /go/src/github.com/jmorganca/ollama/server/routes.go:1071 +0x828 fp=0xc00055f748 sp=0xc00055f138 pc=0x99b308
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0004a0b00)
        /go/src/github.com/jmorganca/ollama/server/routes.go:883 +0x68 fp=0xc00055f780 sp=0xc00055f748 pc=0x999e48
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0004a0b00)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc00055f7d0 sp=0xc00055f780 pc=0x9756ba
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0004a0b00)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xde fp=0xc00055f980 sp=0xc00055f7d0 pc=0x97485e
github.com/gin-gonic/gin.(*Context).Next(...)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0005824e0, 0xc0004a0b00)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x65b fp=0xc00055fb08 sp=0xc00055f980 pc=0x97391b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0005824e0, {0x10632140?, 0xc000518540}, 0xc0004a0a00)
        /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1dd fp=0xc00055fb48 sp=0xc00055fb08 pc=0x9730dd
net/http.serverHandler.ServeHTTP({0x10630460?}, {0x10632140?, 0xc000518540?}, 0x6?)
        /usr/local/go/src/net/http/server.go:2938 +0x8e fp=0xc00055fb78 sp=0xc00055fb48 pc=0x6ce60e
net/http.(*conn).serve(0xc0001b4240, {0x106337a8, 0xc0001ec840})
        /usr/local/go/src/net/http/server.go:2009 +0x5f4 fp=0xc00055ffb8 sp=0xc00055fb78 pc=0x6ca4f4
net/http.(*Server).Serve.func3()
        /usr/local/go/src/net/http/server.go:3086 +0x28 fp=0xc00055ffe0 sp=0xc00055ffb8 pc=0x6cee28
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00055ffe8 sp=0xc00055ffe0 pc=0x46e0a1
created by net/http.(*Server).Serve in goroutine 1
        /usr/local/go/src/net/http/server.go:3086 +0x5cb

goroutine 1 [IO wait]:
runtime.gopark(0x4808b0?, 0xc00059d848?, 0x98?, 0xd8?, 0x4f69dd?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00059d828 sp=0xc00059d808 pc=0x43e6ae
runtime.netpollblock(0x46c112?, 0x4092a6?, 0x0?)
        /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00059d860 sp=0xc00059d828 pc=0x437137
internal/poll.runtime_pollWait(0x7fb46907be80, 0x72)
        /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00059d880 sp=0xc00059d860 pc=0x4688c5
internal/poll.(*pollDesc).wait(0xc0004a2000?, 0x4?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00059d8a8 sp=0xc00059d880 pc=0x4ef627
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc0004a2000)
        /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00059d950 sp=0xc00059d8a8 pc=0x4f4b0c
net.(*netFD).accept(0xc0004a2000)
        /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc00059da08 sp=0xc00059d950 pc=0x56b609
net.(*TCPListener).accept(0xc0004755a0)
        /usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc00059da30 sp=0xc00059da08 pc=0x58041e
net.(*TCPListener).Accept(0xc0004755a0)
        /usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc00059da60 sp=0xc00059da30 pc=0x57f5d0
net/http.(*onceCloseListener).Accept(0xc0001b4240?)
        <autogenerated>:1 +0x24 fp=0xc00059da78 sp=0xc00059da60 pc=0x6f13a4
net/http.(*Server).Serve(0xc000122000, {0x10631f30, 0xc0004755a0})
        /usr/local/go/src/net/http/server.go:3056 +0x364 fp=0xc00059dba8 sp=0xc00059da78 pc=0x6cea64
github.com/jmorganca/ollama/server.Serve({0x10631f30, 0xc0004755a0})
        /go/src/github.com/jmorganca/ollama/server/routes.go:970 +0x488 fp=0xc00059dc98 sp=0xc00059dba8 pc=0x99a328
github.com/jmorganca/ollama/cmd.RunServer(0xc0004a0400?, {0x10a75780?, 0x4?, 0xacee21?})
        /go/src/github.com/jmorganca/ollama/cmd/cmd.go:690 +0x199 fp=0xc00059dd30 sp=0xc00059dc98 pc=0x9ac719
github.com/spf13/cobra.(*Command).execute(0xc000453800, {0x10a75780, 0x0, 0x0})
        /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x87c fp=0xc00059de68 sp=0xc00059dd30 pc=0x7641dc
github.com/spf13/cobra.(*Command).ExecuteC(0xc000452c00)
        /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc00059df20 sp=0xc00059de68 pc=0x764a05
github.com/spf13/cobra.(*Command).Execute(...)
        /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
        /go/src/github.com/jmorganca/ollama/main.go:11 +0x4d fp=0xc00059df40 sp=0xc00059df20 pc=0x9b378d
runtime.main()
        /usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc00059dfe0 sp=0xc00059df40 pc=0x43e25b
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00059dfe8 sp=0xc00059dfe0 pc=0x46e0a1

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090fa8 sp=0xc000090f88 pc=0x43e6ae
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc000090fe0 sp=0xc000090fa8 pc=0x43e533
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000090fe8 sp=0xc000090fe0 pc=0x46e0a1
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:310 +0x1a

goroutine 3 [GC sweep wait]:
runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091778 sp=0xc000091758 pc=0x43e6ae
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.bgsweep(0x0?)
        /usr/local/go/src/runtime/mgcsweep.go:321 +0xdf fp=0xc0000917c8 sp=0xc000091778 pc=0x42a5ff
runtime.gcenable.func1()
        /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000917e0 sp=0xc0000917c8 pc=0x41f725
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000917e8 sp=0xc0000917e0 pc=0x46e0a1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:200 +0x66

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x3572e7?, 0x7a2aec?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091f70 sp=0xc000091f50 pc=0x43e6ae
runtime.goparkunlock(...)
        /usr/local/go/src/runtime/proc.go:404
runtime.(*scavengerState).park(0x10a45b00)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000091fa0 sp=0xc000091f70 pc=0x427e29
runtime.bgscavenge(0x0?)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000091fc8 sp=0xc000091fa0 pc=0x4283d9
runtime.gcenable.func2()
        /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000091fe0 sp=0xc000091fc8 pc=0x41f6c5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x46e0a1
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:201 +0xa5

goroutine 5 [finalizer wait]:
runtime.gopark(0xac7de0?, 0x10043f801?, 0x0?, 0x0?, 0x446865?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090628 sp=0xc000090608 pc=0x43e6ae
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000907e0 sp=0xc000090628 pc=0x41e7a7
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000907e8 sp=0xc0000907e0 pc=0x46e0a1
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:163 +0x3d

goroutine 6 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a03f?, 0x3?, 0xf0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092750 sp=0xc000092730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000927e0 sp=0xc000092750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000927e8 sp=0xc0000927e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 18 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a053?, 0x3?, 0x94?, 0x60?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008c750 sp=0xc00008c730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008c7e0 sp=0xc00008c750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008c7e8 sp=0xc00008c7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 19 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af81473?, 0x1?, 0x89?, 0x78?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008cf50 sp=0xc00008cf30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008cfe0 sp=0xc00008cf50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008cfe8 sp=0xc00008cfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 34 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af89f80?, 0x3?, 0x86?, 0x77?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508750 sp=0xc000508730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005087e0 sp=0xc000508750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 20 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a0fd?, 0x1?, 0x29?, 0x17?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008d750 sp=0xc00008d730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008d7e0 sp=0xc00008d750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008d7e8 sp=0xc00008d7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 35 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8aab2?, 0x3?, 0x9b?, 0xa5?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508f50 sp=0xc000508f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000508fe0 sp=0xc000508f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 7 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8e277?, 0x3?, 0xc9?, 0x93?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092f50 sp=0xc000092f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000092fe0 sp=0xc000092f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000092fe8 sp=0xc000092fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 36 [GC worker (idle)]:
runtime.gopark(0xc000037228?, 0x1?, 0xb5?, 0xa4?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509750 sp=0xc000509730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005097e0 sp=0xc000509750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 8 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x3?, 0x23?, 0xe5?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093750 sp=0xc000093730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000937e0 sp=0xc000093750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000937e8 sp=0xc0000937e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 9 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af813d3?, 0x3?, 0xfc?, 0x64?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093f50 sp=0xc000093f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000093fe0 sp=0xc000093f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 21 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x3?, 0xbd?, 0x50?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008df50 sp=0xc00008df30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008dfe0 sp=0xc00008df50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008dfe8 sp=0xc00008dfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 22 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8ae9c?, 0x3?, 0x9c?, 0xad?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008e750 sp=0xc00008e730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008e7e0 sp=0xc00008e750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008e7e8 sp=0xc00008e7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 37 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0xee?, 0x2c?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509f50 sp=0xc000509f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000509fe0 sp=0xc000509f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000509fe8 sp=0xc000509fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 23 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8affa?, 0xc00046e4e0?, 0x1a?, 0x14?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ef50 sp=0xc00008ef30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008efe0 sp=0xc00008ef50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008efe8 sp=0xc00008efe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 38 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c527?, 0x3?, 0x5c?, 0x68?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050a750 sp=0xc00050a730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050a7e0 sp=0xc00050a750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 39 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af7e3ba?, 0x3?, 0x53?, 0x3?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050af50 sp=0xc00050af30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050afe0 sp=0xc00050af50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 24 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8ce59?, 0x3?, 0xd0?, 0xa8?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008f750 sp=0xc00008f730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008f7e0 sp=0xc00008f750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008f7e8 sp=0xc00008f7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 10 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0x59?, 0x4c?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504750 sp=0xc000504730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005047e0 sp=0xc000504750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005047e8 sp=0xc0005047e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 25 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c834?, 0x3?, 0x37?, 0x44?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ff50 sp=0xc00008ff30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008ffe0 sp=0xc00008ff50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 26 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8e186?, 0x1?, 0xa5?, 0x89?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118750 sp=0xc000118730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001187e0 sp=0xc000118750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 40 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8c9cf?, 0x1?, 0x9c?, 0xec?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050b750 sp=0xc00050b730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050b7e0 sp=0xc00050b750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 11 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8a175?, 0x3?, 0xa4?, 0x3d?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504f50 sp=0xc000504f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000504fe0 sp=0xc000504f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000504fe8 sp=0xc000504fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 12 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb6a?, 0x3?, 0xd1?, 0xff?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505750 sp=0xc000505730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005057e0 sp=0xc000505750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005057e8 sp=0xc0005057e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 13 [GC worker (idle)]:
runtime.gopark(0x10a774a0?, 0x1?, 0x5d?, 0x34?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505f50 sp=0xc000505f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000505fe0 sp=0xc000505f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 14 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cf90?, 0x3?, 0xd7?, 0x7b?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506750 sp=0xc000506730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005067e0 sp=0xc000506750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 41 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8921e?, 0x3?, 0x63?, 0xf5?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050bf50 sp=0xc00050bf30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050bfe0 sp=0xc00050bf50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 27 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb74?, 0x3?, 0xb6?, 0xb1?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118f50 sp=0xc000118f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000118fe0 sp=0xc000118f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000118fe8 sp=0xc000118fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 42 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cd18?, 0x3?, 0x7a?, 0x70?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114750 sp=0xc000114730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001147e0 sp=0xc000114750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 15 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8750a?, 0x3?, 0x9b?, 0xc3?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506f50 sp=0xc000506f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000506fe0 sp=0xc000506f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 28 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb7e?, 0x3?, 0x67?, 0x79?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119750 sp=0xc000119730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001197e0 sp=0xc000119750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001197e8 sp=0xc0001197e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 16 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8cb42?, 0x1?, 0xdc?, 0xa5?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000507750 sp=0xc000507730 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005077e0 sp=0xc000507750 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005077e8 sp=0xc0005077e0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 29 [GC worker (idle)]:
runtime.gopark(0x2f1fe8af8bd35?, 0x3?, 0x2d?, 0xb8?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119f50 sp=0xc000119f30 pc=0x43e6ae
runtime.gcBgMarkWorker()
        /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000119fe0 sp=0xc000119f50 pc=0x4212a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000119fe8 sp=0xc000119fe0 pc=0x46e0a1
created by runtime.gcBgMarkStartWorkers in goroutine 1
        /usr/local/go/src/runtime/mgc.go:1217 +0x1c

goroutine 30 [select, locked to thread]:
runtime.gopark(0xc000114fa8?, 0x2?, 0x49?, 0xe9?, 0xc000114fa4?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114e38 sp=0xc000114e18 pc=0x43e6ae
runtime.selectgo(0xc000114fa8, 0xc000114fa0, 0x0?, 0x0, 0x0?, 0x1)
        /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000114f58 sp=0xc000114e38 pc=0x44e1e5
runtime.ensureSigM.func1()
        /usr/local/go/src/runtime/signal_unix.go:1014 +0x19f fp=0xc000114fe0 sp=0xc000114f58 pc=0x46521f
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000114fe8 sp=0xc000114fe0 pc=0x46e0a1
created by runtime.ensureSigM in goroutine 1
        /usr/local/go/src/runtime/signal_unix.go:997 +0xc8

goroutine 50 [syscall]:
runtime.notetsleepg(0x0?, 0x0?)
        /usr/local/go/src/runtime/lock_futex.go:236 +0x29 fp=0xc0005947a0 sp=0xc000594768 pc=0x411209
os/signal.signal_recv()
        /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0005947c0 sp=0xc0005947a0 pc=0x46aa69
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0005947e0 sp=0xc0005947c0 pc=0x6f3dd3
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005947e8 sp=0xc0005947e0 pc=0x46e0a1
created by os/signal.Notify.func1.1 in goroutine 1
        /usr/local/go/src/os/signal/signal.go:151 +0x1f

goroutine 51 [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000594f18 sp=0xc000594ef8 pc=0x43e6ae
runtime.chanrecv(0xc00068e840, 0x0, 0x1)
        /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000594f90 sp=0xc000594f18 pc=0x40beed
runtime.chanrecv1(0x0?, 0x0?)
        /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000594fb8 sp=0xc000594f90 pc=0x40baf2
github.com/jmorganca/ollama/server.Serve.func1()
        /go/src/github.com/jmorganca/ollama/server/routes.go:952 +0x25 fp=0xc000594fe0 sp=0xc000594fb8 pc=0x99a3c5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000594fe8 sp=0xc000594fe0 pc=0x46e0a1
created by github.com/jmorganca/ollama/server.Serve in goroutine 1
        /go/src/github.com/jmorganca/ollama/server/routes.go:951 +0x3f6

goroutine 67 [IO wait]:
runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x11?)
        /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000115da0 sp=0xc000115d80 pc=0x43e6ae
runtime.netpollblock(0x47ea18?, 0x4092a6?, 0x0?)
        /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc000115dd8 sp=0xc000115da0 pc=0x437137
internal/poll.runtime_pollWait(0x7fb46907bc90, 0x72)
        /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc000115df8 sp=0xc000115dd8 pc=0x4688c5
internal/poll.(*pollDesc).wait(0xc0001c0600?, 0xc0001eca01?, 0x0)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000115e20 sp=0xc000115df8 pc=0x4ef627
internal/poll.(*pollDesc).waitRead(...)
        /usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc0001c0600, {0xc0001eca01, 0x1, 0x1})
        /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000115eb8 sp=0xc000115e20 pc=0x4f091a
net.(*netFD).Read(0xc0001c0600, {0xc0001eca01?, 0x0?, 0x0?})
        /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc000115f00 sp=0xc000115eb8 pc=0x5695e5
net.(*conn).Read(0xc000690060, {0xc0001eca01?, 0x0?, 0x0?})
        /usr/local/go/src/net/net.go:179 +0x45 fp=0xc000115f48 sp=0xc000115f00 pc=0x577885
net.(*TCPConn).Read(0x0?, {0xc0001eca01?, 0x0?, 0x0?})
        <autogenerated>:1 +0x25 fp=0xc000115f78 sp=0xc000115f48 pc=0x589785
net/http.(*connReader).backgroundRead(0xc0001ec9f0)
        /usr/local/go/src/net/http/server.go:683 +0x37 fp=0xc000115fc8 sp=0xc000115f78 pc=0x6c4377
net/http.(*connReader).startBackgroundRead.func2()
        /usr/local/go/src/net/http/server.go:679 +0x25 fp=0xc000115fe0 sp=0xc000115fc8 pc=0x6c42a5
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0x46e0a1
created by net/http.(*connReader).startBackgroundRead in goroutine 66
        /usr/local/go/src/net/http/server.go:679 +0xba

rax    0x0
rbx    0x7fb409c0950e
rcx    0x7fb4b251d387
rdx    0x6
rdi    0x1
rsi    0x24
rbp    0x21f0
rsp    0x7fb41effc368
r8     0x0
r9     0x1
r10    0x8
r11    0x202
r12    0x7fb4b28af868
r13    0x7fb0f380a1b0
r14    0x7fb409c08c1c
r15    0x7fb409c094b3
rip    0x7fb4b251d387
rflags 0x202
cs     0x33
fs     0x0
gs     0x0
<!-- gh-comment-id:1903689432 --> @xlmnxp commented on GitHub (Jan 22, 2024): Same issue but with latest docker image (I built it my self `docker build -t ollama/ollama .`) Computer Specs: GPU: `RX 7900 XTX` CPU: `R9 7950X` RAM: `64GB` OS: `Ubuntu 23.10 (Docker Container)` ROCM Version: `6.0.0` Kernel: `6.5.0` Server log output: ``` 2024/01/22 09:49:51 images.go:810: INFO total blobs: 6 2024/01/22 09:49:51 images.go:817: INFO total unused blobs removed: 0 [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/jmorganca/ollama/server.PullModelHandler (5 handlers) [GIN-debug] POST /api/generate --> github.com/jmorganca/ollama/server.GenerateHandler (5 handlers) [GIN-debug] POST /api/chat --> github.com/jmorganca/ollama/server.ChatHandler (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/jmorganca/ollama/server.EmbeddingHandler (5 handlers) [GIN-debug] POST /api/create --> github.com/jmorganca/ollama/server.CreateModelHandler (5 handlers) [GIN-debug] POST /api/push --> github.com/jmorganca/ollama/server.PushModelHandler (5 handlers) [GIN-debug] POST /api/copy --> github.com/jmorganca/ollama/server.CopyModelHandler (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/jmorganca/ollama/server.DeleteModelHandler (5 handlers) [GIN-debug] POST /api/show --> github.com/jmorganca/ollama/server.ShowModelHandler (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/jmorganca/ollama/server.CreateBlobHandler (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/jmorganca/ollama/server.HeadBlobHandler (5 handlers) [GIN-debug] GET / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] GET /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers) [GIN-debug] GET /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers) [GIN-debug] HEAD / --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (5 handlers) [GIN-debug] HEAD /api/version --> github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func3 (5 handlers) 2024/01/22 09:49:51 routes.go:943: INFO Listening on [::]:11434 (version 0.0.0) 2024/01/22 09:49:51 payload_common.go:106: INFO Extracting dynamic libraries... 2024/01/22 09:49:52 payload_common.go:145: INFO Dynamic LLM libraries [rocm_v5 rocm_v6 cuda_v11 cpu cpu_avx cpu_avx2] 2024/01/22 09:49:52 gpu.go:91: INFO Detecting GPU type 2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library libnvidia-ml.so 2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: [] 2024/01/22 09:49:52 gpu.go:210: INFO Searching for GPU management library librocm_smi64.so 2024/01/22 09:49:52 gpu.go:256: INFO Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.6.0.60000 /opt/rocm-6.0.0/lib/librocm_smi64.so.6.0.60000] 2024/01/22 09:49:52 gpu.go:106: INFO Radeon GPU detected 2024/01/22 09:50:03 cpu_common.go:11: INFO CPU has AVX2 2024/01/22 09:50:03 dyn_ext_server.go:90: INFO Loading Dynamic llm server: /tmp/ollama2441091586/rocm_v6/libext_server.so 2024/01/22 09:50:03 dyn_ext_server.go:139: INFO Initializing llama server ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes ggml_init_cublas: found 2 ROCm devices: Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no llama_model_loader: loaded meta data with 23 key-value pairs and 363 tensors from /root/.ollama/models/blobs/sha256:2609048d349e7c70196401be59bea7eb89a968d4642e409b0e798b34403b96c8 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = LLaMA v2 llama_model_loader: - kv 2: llama.context_length u32 = 4096 llama_model_loader: - kv 3: llama.embedding_length u32 = 5120 llama_model_loader: - kv 4: llama.block_count u32 = 40 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 13824 llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 7: llama.attention.head_count u32 = 40 llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 40 llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 10: general.file_type u32 = 2 llama_model_loader: - kv 11: tokenizer.ggml.model str = llama llama_model_loader: - kv 12: tokenizer.ggml.tokens arr[str,32000] = ["<unk>", "<s>", "</s>", "<0x00>", "<... llama_model_loader: - kv 13: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000... llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ... llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,61249] = ["▁ t", "e r", "i n", "▁ a", "e n... llama_model_loader: - kv 16: tokenizer.ggml.bos_token_id u32 = 1 llama_model_loader: - kv 17: tokenizer.ggml.eos_token_id u32 = 2 llama_model_loader: - kv 18: tokenizer.ggml.unknown_token_id u32 = 0 llama_model_loader: - kv 19: tokenizer.ggml.add_bos_token bool = true llama_model_loader: - kv 20: tokenizer.ggml.add_eos_token bool = false llama_model_loader: - kv 21: tokenizer.chat_template str = {% if messages[0]['role'] == 'system'... llama_model_loader: - kv 22: general.quantization_version u32 = 2 llama_model_loader: - type f32: 81 tensors llama_model_loader: - type q4_0: 281 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens definition check successful ( 259/32000 ). llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = SPM llm_load_print_meta: n_vocab = 32000 llm_load_print_meta: n_merges = 0 llm_load_print_meta: n_ctx_train = 4096 llm_load_print_meta: n_embd = 5120 llm_load_print_meta: n_head = 40 llm_load_print_meta: n_head_kv = 40 llm_load_print_meta: n_layer = 40 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 1 llm_load_print_meta: n_embd_k_gqa = 5120 llm_load_print_meta: n_embd_v_gqa = 5120 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: n_ff = 13824 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 10000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_yarn_orig_ctx = 4096 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: model type = 13B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 13.02 B llm_load_print_meta: model size = 6.86 GiB (4.53 BPW) llm_load_print_meta: general.name = LLaMA v2 llm_load_print_meta: BOS token = 1 '<s>' llm_load_print_meta: EOS token = 2 '</s>' llm_load_print_meta: UNK token = 0 '<unk>' llm_load_print_meta: LF token = 13 '<0x0A>' llm_load_tensors: ggml ctx size = 0.14 MiB llm_load_tensors: using ROCm for GPU acceleration llm_load_tensors: system memory used = 88.03 MiB llm_load_tensors: VRAM used = 6936.01 MiB llm_load_tensors: offloading 40 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 41/41 layers to GPU ................................................................................................... llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: freq_base = 10000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: VRAM kv self = 1600.00 MB llama_new_context_with_model: KV self size = 1600.00 MiB, K (f16): 800.00 MiB, V (f16): 800.00 MiB llama_build_graph: non-view tensors processed: 844/844 llama_new_context_with_model: compute buffer total size = 197.19 MiB llama_new_context_with_model: VRAM scratch buffer: 194.00 MiB llama_new_context_with_model: total VRAM used: 8730.01 MiB (model: 6936.01 MiB, context: 1794.00 MiB) CUDA error: shared object initialization failed current device: 0, in function ggml_cuda_op_flatten at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:8688 hipGetLastError() GGML_ASSERT: /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:229: !"CUDA error" loading library /tmp/ollama2441091586/rocm_v6/libext_server.so No symbol table is loaded. Use the "file" command. ptrace: Operation not permitted. No stack. The program is not being run. SIGABRT: abort PC=0x7fb4b251d387 m=31 sigcode=18446744073709551610 signal arrived during cgo execution goroutine 66 [syscall]: runtime.cgocall(0x9b4670, 0xc00055e808) /usr/local/go/src/runtime/cgocall.go:157 +0x4b fp=0xc00055e7e0 sp=0xc00055e7a8 pc=0x409b0b github.com/jmorganca/ollama/llm._Cfunc_dyn_llama_server_init({0x7fb410000e00, 0x7fb409a545a0, 0x7fb409a54cf0, 0x7fb409a54d80, 0x7fb409a54f30, 0x7fb409a550a0, 0x7fb409a55560, 0x7fb409a55540, 0x7fb409a555f0, 0x7fb409a55ba0, ...}, ...) _cgo_gotypes.go:280 +0x45 fp=0xc00055e808 sp=0xc00055e7e0 pc=0x7c2b25 github.com/jmorganca/ollama/llm.newDynExtServer.func7(0xae6f99?, 0x62?) /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xef fp=0xc00055e8f8 sp=0xc00055e808 pc=0x7c3fcf github.com/jmorganca/ollama/llm.newDynExtServer({0xc00002a840, 0x2e}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...) /go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:142 +0xa32 fp=0xc00055eb88 sp=0xc00055e8f8 pc=0x7c3cd2 github.com/jmorganca/ollama/llm.newLlmServer({{_, _, _}, {_, _}, {_, _}}, {_, _}, {0x0, ...}, ...) /go/src/github.com/jmorganca/ollama/llm/llm.go:147 +0x36a fp=0xc00055ed48 sp=0xc00055eb88 pc=0x7c04ea github.com/jmorganca/ollama/llm.New({0x0?, 0x1000100000100?}, {0xc0000302a0, _}, {_, _, _}, {0x0, 0x0, 0x0}, ...) /go/src/github.com/jmorganca/ollama/llm/llm.go:122 +0x6f9 fp=0xc00055efb8 sp=0xc00055ed48 pc=0x7bff19 github.com/jmorganca/ollama/server.load(0xc000002000?, 0xc000002000, {{0x0, 0x800, 0x200, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x1, ...}, ...}, ...) /go/src/github.com/jmorganca/ollama/server/routes.go:83 +0x3a5 fp=0xc00055f138 sp=0xc00055efb8 pc=0x9909c5 github.com/jmorganca/ollama/server.ChatHandler(0xc0004a0b00) /go/src/github.com/jmorganca/ollama/server/routes.go:1071 +0x828 fp=0xc00055f748 sp=0xc00055f138 pc=0x99b308 github.com/gin-gonic/gin.(*Context).Next(...) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0xc0004a0b00) /go/src/github.com/jmorganca/ollama/server/routes.go:883 +0x68 fp=0xc00055f780 sp=0xc00055f748 pc=0x999e48 github.com/gin-gonic/gin.(*Context).Next(...) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc0004a0b00) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x7a fp=0xc00055f7d0 sp=0xc00055f780 pc=0x9756ba github.com/gin-gonic/gin.(*Context).Next(...) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.LoggerWithConfig.func1(0xc0004a0b00) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xde fp=0xc00055f980 sp=0xc00055f7d0 pc=0x97485e github.com/gin-gonic/gin.(*Context).Next(...) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc0005824e0, 0xc0004a0b00) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x65b fp=0xc00055fb08 sp=0xc00055f980 pc=0x97391b github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc0005824e0, {0x10632140?, 0xc000518540}, 0xc0004a0a00) /root/go/pkg/mod/github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1dd fp=0xc00055fb48 sp=0xc00055fb08 pc=0x9730dd net/http.serverHandler.ServeHTTP({0x10630460?}, {0x10632140?, 0xc000518540?}, 0x6?) /usr/local/go/src/net/http/server.go:2938 +0x8e fp=0xc00055fb78 sp=0xc00055fb48 pc=0x6ce60e net/http.(*conn).serve(0xc0001b4240, {0x106337a8, 0xc0001ec840}) /usr/local/go/src/net/http/server.go:2009 +0x5f4 fp=0xc00055ffb8 sp=0xc00055fb78 pc=0x6ca4f4 net/http.(*Server).Serve.func3() /usr/local/go/src/net/http/server.go:3086 +0x28 fp=0xc00055ffe0 sp=0xc00055ffb8 pc=0x6cee28 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00055ffe8 sp=0xc00055ffe0 pc=0x46e0a1 created by net/http.(*Server).Serve in goroutine 1 /usr/local/go/src/net/http/server.go:3086 +0x5cb goroutine 1 [IO wait]: runtime.gopark(0x4808b0?, 0xc00059d848?, 0x98?, 0xd8?, 0x4f69dd?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00059d828 sp=0xc00059d808 pc=0x43e6ae runtime.netpollblock(0x46c112?, 0x4092a6?, 0x0?) /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc00059d860 sp=0xc00059d828 pc=0x437137 internal/poll.runtime_pollWait(0x7fb46907be80, 0x72) /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc00059d880 sp=0xc00059d860 pc=0x4688c5 internal/poll.(*pollDesc).wait(0xc0004a2000?, 0x4?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc00059d8a8 sp=0xc00059d880 pc=0x4ef627 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Accept(0xc0004a2000) /usr/local/go/src/internal/poll/fd_unix.go:611 +0x2ac fp=0xc00059d950 sp=0xc00059d8a8 pc=0x4f4b0c net.(*netFD).accept(0xc0004a2000) /usr/local/go/src/net/fd_unix.go:172 +0x29 fp=0xc00059da08 sp=0xc00059d950 pc=0x56b609 net.(*TCPListener).accept(0xc0004755a0) /usr/local/go/src/net/tcpsock_posix.go:152 +0x1e fp=0xc00059da30 sp=0xc00059da08 pc=0x58041e net.(*TCPListener).Accept(0xc0004755a0) /usr/local/go/src/net/tcpsock.go:315 +0x30 fp=0xc00059da60 sp=0xc00059da30 pc=0x57f5d0 net/http.(*onceCloseListener).Accept(0xc0001b4240?) <autogenerated>:1 +0x24 fp=0xc00059da78 sp=0xc00059da60 pc=0x6f13a4 net/http.(*Server).Serve(0xc000122000, {0x10631f30, 0xc0004755a0}) /usr/local/go/src/net/http/server.go:3056 +0x364 fp=0xc00059dba8 sp=0xc00059da78 pc=0x6cea64 github.com/jmorganca/ollama/server.Serve({0x10631f30, 0xc0004755a0}) /go/src/github.com/jmorganca/ollama/server/routes.go:970 +0x488 fp=0xc00059dc98 sp=0xc00059dba8 pc=0x99a328 github.com/jmorganca/ollama/cmd.RunServer(0xc0004a0400?, {0x10a75780?, 0x4?, 0xacee21?}) /go/src/github.com/jmorganca/ollama/cmd/cmd.go:690 +0x199 fp=0xc00059dd30 sp=0xc00059dc98 pc=0x9ac719 github.com/spf13/cobra.(*Command).execute(0xc000453800, {0x10a75780, 0x0, 0x0}) /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:940 +0x87c fp=0xc00059de68 sp=0xc00059dd30 pc=0x7641dc github.com/spf13/cobra.(*Command).ExecuteC(0xc000452c00) /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 fp=0xc00059df20 sp=0xc00059de68 pc=0x764a05 github.com/spf13/cobra.(*Command).Execute(...) /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) /root/go/pkg/mod/github.com/spf13/cobra@v1.7.0/command.go:985 main.main() /go/src/github.com/jmorganca/ollama/main.go:11 +0x4d fp=0xc00059df40 sp=0xc00059df20 pc=0x9b378d runtime.main() /usr/local/go/src/runtime/proc.go:267 +0x2bb fp=0xc00059dfe0 sp=0xc00059df40 pc=0x43e25b runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00059dfe8 sp=0xc00059dfe0 pc=0x46e0a1 goroutine 2 [force gc (idle)]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090fa8 sp=0xc000090f88 pc=0x43e6ae runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:404 runtime.forcegchelper() /usr/local/go/src/runtime/proc.go:322 +0xb3 fp=0xc000090fe0 sp=0xc000090fa8 pc=0x43e533 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000090fe8 sp=0xc000090fe0 pc=0x46e0a1 created by runtime.init.6 in goroutine 1 /usr/local/go/src/runtime/proc.go:310 +0x1a goroutine 3 [GC sweep wait]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091778 sp=0xc000091758 pc=0x43e6ae runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:404 runtime.bgsweep(0x0?) /usr/local/go/src/runtime/mgcsweep.go:321 +0xdf fp=0xc0000917c8 sp=0xc000091778 pc=0x42a5ff runtime.gcenable.func1() /usr/local/go/src/runtime/mgc.go:200 +0x25 fp=0xc0000917e0 sp=0xc0000917c8 pc=0x41f725 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000917e8 sp=0xc0000917e0 pc=0x46e0a1 created by runtime.gcenable in goroutine 1 /usr/local/go/src/runtime/mgc.go:200 +0x66 goroutine 4 [GC scavenge wait]: runtime.gopark(0x3572e7?, 0x7a2aec?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000091f70 sp=0xc000091f50 pc=0x43e6ae runtime.goparkunlock(...) /usr/local/go/src/runtime/proc.go:404 runtime.(*scavengerState).park(0x10a45b00) /usr/local/go/src/runtime/mgcscavenge.go:425 +0x49 fp=0xc000091fa0 sp=0xc000091f70 pc=0x427e29 runtime.bgscavenge(0x0?) /usr/local/go/src/runtime/mgcscavenge.go:658 +0x59 fp=0xc000091fc8 sp=0xc000091fa0 pc=0x4283d9 runtime.gcenable.func2() /usr/local/go/src/runtime/mgc.go:201 +0x25 fp=0xc000091fe0 sp=0xc000091fc8 pc=0x41f6c5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000091fe8 sp=0xc000091fe0 pc=0x46e0a1 created by runtime.gcenable in goroutine 1 /usr/local/go/src/runtime/mgc.go:201 +0xa5 goroutine 5 [finalizer wait]: runtime.gopark(0xac7de0?, 0x10043f801?, 0x0?, 0x0?, 0x446865?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000090628 sp=0xc000090608 pc=0x43e6ae runtime.runfinq() /usr/local/go/src/runtime/mfinal.go:193 +0x107 fp=0xc0000907e0 sp=0xc000090628 pc=0x41e7a7 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000907e8 sp=0xc0000907e0 pc=0x46e0a1 created by runtime.createfing in goroutine 1 /usr/local/go/src/runtime/mfinal.go:163 +0x3d goroutine 6 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8a03f?, 0x3?, 0xf0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092750 sp=0xc000092730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000927e0 sp=0xc000092750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000927e8 sp=0xc0000927e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 18 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8a053?, 0x3?, 0x94?, 0x60?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008c750 sp=0xc00008c730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008c7e0 sp=0xc00008c750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008c7e8 sp=0xc00008c7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 19 [GC worker (idle)]: runtime.gopark(0x2f1fe8af81473?, 0x1?, 0x89?, 0x78?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008cf50 sp=0xc00008cf30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008cfe0 sp=0xc00008cf50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008cfe8 sp=0xc00008cfe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 34 [GC worker (idle)]: runtime.gopark(0x2f1fe8af89f80?, 0x3?, 0x86?, 0x77?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508750 sp=0xc000508730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005087e0 sp=0xc000508750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005087e8 sp=0xc0005087e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 20 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8a0fd?, 0x1?, 0x29?, 0x17?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008d750 sp=0xc00008d730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008d7e0 sp=0xc00008d750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008d7e8 sp=0xc00008d7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 35 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8aab2?, 0x3?, 0x9b?, 0xa5?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000508f50 sp=0xc000508f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000508fe0 sp=0xc000508f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000508fe8 sp=0xc000508fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 7 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8e277?, 0x3?, 0xc9?, 0x93?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000092f50 sp=0xc000092f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000092fe0 sp=0xc000092f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000092fe8 sp=0xc000092fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 36 [GC worker (idle)]: runtime.gopark(0xc000037228?, 0x1?, 0xb5?, 0xa4?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509750 sp=0xc000509730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005097e0 sp=0xc000509750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005097e8 sp=0xc0005097e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 8 [GC worker (idle)]: runtime.gopark(0x10a774a0?, 0x3?, 0x23?, 0xe5?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093750 sp=0xc000093730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0000937e0 sp=0xc000093750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0000937e8 sp=0xc0000937e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 9 [GC worker (idle)]: runtime.gopark(0x2f1fe8af813d3?, 0x3?, 0xfc?, 0x64?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000093f50 sp=0xc000093f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000093fe0 sp=0xc000093f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000093fe8 sp=0xc000093fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 21 [GC worker (idle)]: runtime.gopark(0x10a774a0?, 0x3?, 0xbd?, 0x50?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008df50 sp=0xc00008df30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008dfe0 sp=0xc00008df50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008dfe8 sp=0xc00008dfe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 22 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8ae9c?, 0x3?, 0x9c?, 0xad?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008e750 sp=0xc00008e730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008e7e0 sp=0xc00008e750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008e7e8 sp=0xc00008e7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 37 [GC worker (idle)]: runtime.gopark(0x10a774a0?, 0x1?, 0xee?, 0x2c?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000509f50 sp=0xc000509f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000509fe0 sp=0xc000509f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000509fe8 sp=0xc000509fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 23 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8affa?, 0xc00046e4e0?, 0x1a?, 0x14?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ef50 sp=0xc00008ef30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008efe0 sp=0xc00008ef50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008efe8 sp=0xc00008efe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 38 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8c527?, 0x3?, 0x5c?, 0x68?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050a750 sp=0xc00050a730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050a7e0 sp=0xc00050a750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050a7e8 sp=0xc00050a7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 39 [GC worker (idle)]: runtime.gopark(0x2f1fe8af7e3ba?, 0x3?, 0x53?, 0x3?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050af50 sp=0xc00050af30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050afe0 sp=0xc00050af50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050afe8 sp=0xc00050afe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 24 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8ce59?, 0x3?, 0xd0?, 0xa8?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008f750 sp=0xc00008f730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008f7e0 sp=0xc00008f750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008f7e8 sp=0xc00008f7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 10 [GC worker (idle)]: runtime.gopark(0x10a774a0?, 0x1?, 0x59?, 0x4c?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504750 sp=0xc000504730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005047e0 sp=0xc000504750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005047e8 sp=0xc0005047e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 25 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8c834?, 0x3?, 0x37?, 0x44?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00008ff50 sp=0xc00008ff30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00008ffe0 sp=0xc00008ff50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00008ffe8 sp=0xc00008ffe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 26 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8e186?, 0x1?, 0xa5?, 0x89?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118750 sp=0xc000118730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001187e0 sp=0xc000118750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001187e8 sp=0xc0001187e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 40 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8c9cf?, 0x1?, 0x9c?, 0xec?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050b750 sp=0xc00050b730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050b7e0 sp=0xc00050b750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050b7e8 sp=0xc00050b7e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 11 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8a175?, 0x3?, 0xa4?, 0x3d?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000504f50 sp=0xc000504f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000504fe0 sp=0xc000504f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000504fe8 sp=0xc000504fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 12 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cb6a?, 0x3?, 0xd1?, 0xff?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505750 sp=0xc000505730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005057e0 sp=0xc000505750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005057e8 sp=0xc0005057e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 13 [GC worker (idle)]: runtime.gopark(0x10a774a0?, 0x1?, 0x5d?, 0x34?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000505f50 sp=0xc000505f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000505fe0 sp=0xc000505f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000505fe8 sp=0xc000505fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 14 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cf90?, 0x3?, 0xd7?, 0x7b?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506750 sp=0xc000506730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005067e0 sp=0xc000506750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005067e8 sp=0xc0005067e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 41 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8921e?, 0x3?, 0x63?, 0xf5?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc00050bf50 sp=0xc00050bf30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc00050bfe0 sp=0xc00050bf50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc00050bfe8 sp=0xc00050bfe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 27 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cb74?, 0x3?, 0xb6?, 0xb1?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000118f50 sp=0xc000118f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000118fe0 sp=0xc000118f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000118fe8 sp=0xc000118fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 42 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cd18?, 0x3?, 0x7a?, 0x70?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114750 sp=0xc000114730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001147e0 sp=0xc000114750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001147e8 sp=0xc0001147e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 15 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8750a?, 0x3?, 0x9b?, 0xc3?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000506f50 sp=0xc000506f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000506fe0 sp=0xc000506f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000506fe8 sp=0xc000506fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 28 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cb7e?, 0x3?, 0x67?, 0x79?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119750 sp=0xc000119730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0001197e0 sp=0xc000119750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0001197e8 sp=0xc0001197e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 16 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8cb42?, 0x1?, 0xdc?, 0xa5?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000507750 sp=0xc000507730 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc0005077e0 sp=0xc000507750 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005077e8 sp=0xc0005077e0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 29 [GC worker (idle)]: runtime.gopark(0x2f1fe8af8bd35?, 0x3?, 0x2d?, 0xb8?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000119f50 sp=0xc000119f30 pc=0x43e6ae runtime.gcBgMarkWorker() /usr/local/go/src/runtime/mgc.go:1293 +0xe5 fp=0xc000119fe0 sp=0xc000119f50 pc=0x4212a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000119fe8 sp=0xc000119fe0 pc=0x46e0a1 created by runtime.gcBgMarkStartWorkers in goroutine 1 /usr/local/go/src/runtime/mgc.go:1217 +0x1c goroutine 30 [select, locked to thread]: runtime.gopark(0xc000114fa8?, 0x2?, 0x49?, 0xe9?, 0xc000114fa4?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000114e38 sp=0xc000114e18 pc=0x43e6ae runtime.selectgo(0xc000114fa8, 0xc000114fa0, 0x0?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:327 +0x725 fp=0xc000114f58 sp=0xc000114e38 pc=0x44e1e5 runtime.ensureSigM.func1() /usr/local/go/src/runtime/signal_unix.go:1014 +0x19f fp=0xc000114fe0 sp=0xc000114f58 pc=0x46521f runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000114fe8 sp=0xc000114fe0 pc=0x46e0a1 created by runtime.ensureSigM in goroutine 1 /usr/local/go/src/runtime/signal_unix.go:997 +0xc8 goroutine 50 [syscall]: runtime.notetsleepg(0x0?, 0x0?) /usr/local/go/src/runtime/lock_futex.go:236 +0x29 fp=0xc0005947a0 sp=0xc000594768 pc=0x411209 os/signal.signal_recv() /usr/local/go/src/runtime/sigqueue.go:152 +0x29 fp=0xc0005947c0 sp=0xc0005947a0 pc=0x46aa69 os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:23 +0x13 fp=0xc0005947e0 sp=0xc0005947c0 pc=0x6f3dd3 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc0005947e8 sp=0xc0005947e0 pc=0x46e0a1 created by os/signal.Notify.func1.1 in goroutine 1 /usr/local/go/src/os/signal/signal.go:151 +0x1f goroutine 51 [chan receive]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000594f18 sp=0xc000594ef8 pc=0x43e6ae runtime.chanrecv(0xc00068e840, 0x0, 0x1) /usr/local/go/src/runtime/chan.go:583 +0x3cd fp=0xc000594f90 sp=0xc000594f18 pc=0x40beed runtime.chanrecv1(0x0?, 0x0?) /usr/local/go/src/runtime/chan.go:442 +0x12 fp=0xc000594fb8 sp=0xc000594f90 pc=0x40baf2 github.com/jmorganca/ollama/server.Serve.func1() /go/src/github.com/jmorganca/ollama/server/routes.go:952 +0x25 fp=0xc000594fe0 sp=0xc000594fb8 pc=0x99a3c5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000594fe8 sp=0xc000594fe0 pc=0x46e0a1 created by github.com/jmorganca/ollama/server.Serve in goroutine 1 /go/src/github.com/jmorganca/ollama/server/routes.go:951 +0x3f6 goroutine 67 [IO wait]: runtime.gopark(0x0?, 0xb?, 0x0?, 0x0?, 0x11?) /usr/local/go/src/runtime/proc.go:398 +0xce fp=0xc000115da0 sp=0xc000115d80 pc=0x43e6ae runtime.netpollblock(0x47ea18?, 0x4092a6?, 0x0?) /usr/local/go/src/runtime/netpoll.go:564 +0xf7 fp=0xc000115dd8 sp=0xc000115da0 pc=0x437137 internal/poll.runtime_pollWait(0x7fb46907bc90, 0x72) /usr/local/go/src/runtime/netpoll.go:343 +0x85 fp=0xc000115df8 sp=0xc000115dd8 pc=0x4688c5 internal/poll.(*pollDesc).wait(0xc0001c0600?, 0xc0001eca01?, 0x0) /usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x27 fp=0xc000115e20 sp=0xc000115df8 pc=0x4ef627 internal/poll.(*pollDesc).waitRead(...) /usr/local/go/src/internal/poll/fd_poll_runtime.go:89 internal/poll.(*FD).Read(0xc0001c0600, {0xc0001eca01, 0x1, 0x1}) /usr/local/go/src/internal/poll/fd_unix.go:164 +0x27a fp=0xc000115eb8 sp=0xc000115e20 pc=0x4f091a net.(*netFD).Read(0xc0001c0600, {0xc0001eca01?, 0x0?, 0x0?}) /usr/local/go/src/net/fd_posix.go:55 +0x25 fp=0xc000115f00 sp=0xc000115eb8 pc=0x5695e5 net.(*conn).Read(0xc000690060, {0xc0001eca01?, 0x0?, 0x0?}) /usr/local/go/src/net/net.go:179 +0x45 fp=0xc000115f48 sp=0xc000115f00 pc=0x577885 net.(*TCPConn).Read(0x0?, {0xc0001eca01?, 0x0?, 0x0?}) <autogenerated>:1 +0x25 fp=0xc000115f78 sp=0xc000115f48 pc=0x589785 net/http.(*connReader).backgroundRead(0xc0001ec9f0) /usr/local/go/src/net/http/server.go:683 +0x37 fp=0xc000115fc8 sp=0xc000115f78 pc=0x6c4377 net/http.(*connReader).startBackgroundRead.func2() /usr/local/go/src/net/http/server.go:679 +0x25 fp=0xc000115fe0 sp=0xc000115fc8 pc=0x6c42a5 runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:1650 +0x1 fp=0xc000115fe8 sp=0xc000115fe0 pc=0x46e0a1 created by net/http.(*connReader).startBackgroundRead in goroutine 66 /usr/local/go/src/net/http/server.go:679 +0xba rax 0x0 rbx 0x7fb409c0950e rcx 0x7fb4b251d387 rdx 0x6 rdi 0x1 rsi 0x24 rbp 0x21f0 rsp 0x7fb41effc368 r8 0x0 r9 0x1 r10 0x8 r11 0x202 r12 0x7fb4b28af868 r13 0x7fb0f380a1b0 r14 0x7fb409c08c1c r15 0x7fb409c094b3 rip 0x7fb4b251d387 rflags 0x202 cs 0x33 fs 0x0 gs 0x0 ```
GiteaMirror commented 2026-04-22 03:44:45 -05:00
Author
Owner
Copy Link

@xlmnxp commented on GitHub (Jan 22, 2024):

I tried 2 models (mistral and llama2:13b) and both of them failed at|

CUDA error: shared object initialization failed
  current device: 0, in function ggml_cuda_op_flatten at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:8688
  hipGetLastError()
GGML_ASSERT: /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:229: !"CUDA error"
<!-- gh-comment-id:1903709589 --> @xlmnxp commented on GitHub (Jan 22, 2024): I tried 2 models (`mistral` and `llama2:13b`) and both of them failed at| ``` CUDA error: shared object initialization failed current device: 0, in function ggml_cuda_op_flatten at /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:8688 hipGetLastError() GGML_ASSERT: /go/src/github.com/jmorganca/ollama/llm/llama.cpp/ggml-cuda.cu:229: !"CUDA error" ```
GiteaMirror commented 2026-04-22 03:44:46 -05:00
Author
Owner
Copy Link

@Gagootron commented on GitHub (Jan 22, 2024):

I figured it out! It is the iGPU of the CPU! If if force disable it in the bios ollama works as expected. I suppose there must be some way of preventing ollama from using the iGPU?

<!-- gh-comment-id:1904266351 --> @Gagootron commented on GitHub (Jan 22, 2024): I figured it out! It is the iGPU of the CPU! If if force disable it in the bios ollama works as expected. I suppose there must be some way of preventing ollama from using the iGPU?
GiteaMirror commented 2026-04-22 03:44:47 -05:00
Author
Owner
Copy Link

@Gagootron commented on GitHub (Jan 22, 2024):

ROCM can be set to a specific GPU (or multiple of GPUs for that matter) with the ROCR_VISIBLE_DEVICES environment variable.
For example if the log shows:

ggml_init_cublas: found 2 ROCm devices:
  Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no
  Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no

you can set ROCR_VISIBLE_DEVICES="0" and only the RX7900XTX will be used. If you want multiple GPUs you can separate the Device numbers with commas, like ROCR_VISIBLE_DEVICES="1,2,7"

See https://rocm.docs.amd.com/en/latest/conceptual/gpu-isolation.html for more info

I consider this problem solved, but I think it should be possible for ollama to figure this out by itself somehow...

<!-- gh-comment-id:1904299043 --> @Gagootron commented on GitHub (Jan 22, 2024): ROCM can be set to a specific GPU (or multiple of GPUs for that matter) with the ROCR_VISIBLE_DEVICES environment variable. For example if the log shows: ``` ggml_init_cublas: found 2 ROCm devices: Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no ``` you can set `ROCR_VISIBLE_DEVICES="0"` and only the RX7900XTX will be used. If you want multiple GPUs you can separate the Device numbers with commas, like `ROCR_VISIBLE_DEVICES="1,2,7"` See https://rocm.docs.amd.com/en/latest/conceptual/gpu-isolation.html for more info I consider this problem solved, but I think it should be possible for ollama to figure this out by itself somehow...
GiteaMirror commented 2026-04-22 03:44:47 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jan 22, 2024):

Happy to hear you found a workaround @Gagootron. We'd definitely like to improve the UX around this so Ollama "just works" on this type of setup without requiring users to figure out flags to override default broken behavior.

<!-- gh-comment-id:1904426392 --> @dhiltgen commented on GitHub (Jan 22, 2024): Happy to hear you found a workaround @Gagootron. We'd definitely like to improve the UX around this so Ollama "just works" on this type of setup without requiring users to figure out flags to override default broken behavior.
GiteaMirror commented 2026-04-22 03:44:48 -05:00
Author
Owner
Copy Link

@xlmnxp commented on GitHub (Jan 22, 2024):

@Gagootron that solve the issue for me!
thank you

<!-- gh-comment-id:1904958840 --> @xlmnxp commented on GitHub (Jan 22, 2024): @Gagootron that solve the issue for me! thank you
GiteaMirror commented 2026-04-22 03:44:49 -05:00
Author
Owner
Copy Link

@meminens commented on GitHub (Jan 30, 2024):

ROCM can be set to a specific GPU (or multiple of GPUs for that matter) with the ROCR_VISIBLE_DEVICES environment variable. For example if the log shows:

ggml_init_cublas: found 2 ROCm devices:
  Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no
  Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no

you can set ROCR_VISIBLE_DEVICES="0" and only the RX7900XTX will be used. If you want multiple GPUs you can separate the Device numbers with commas, like ROCR_VISIBLE_DEVICES="1,2,7"

See https://rocm.docs.amd.com/en/latest/conceptual/gpu-isolation.html for more info

I consider this problem solved, but I think it should be possible for ollama to figure this out by itself somehow...

I've run export ROCR_VISIBLE_DEVICES=0 in the command line and restarted. But ollama is still using the integrated GPU. I've restarted the daemon and ollama. Can you help me set up ollama so that it uses the external GPU (AMD 7900 xtx)? I am on Arch Linux.

<!-- gh-comment-id:1917530285 --> @meminens commented on GitHub (Jan 30, 2024): > ROCM can be set to a specific GPU (or multiple of GPUs for that matter) with the ROCR_VISIBLE_DEVICES environment variable. For example if the log shows: > > ``` > ggml_init_cublas: found 2 ROCm devices: > Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no > Device 1: AMD Radeon Graphics, compute capability 10.3, VMM: no > ``` > > you can set `ROCR_VISIBLE_DEVICES="0"` and only the RX7900XTX will be used. If you want multiple GPUs you can separate the Device numbers with commas, like `ROCR_VISIBLE_DEVICES="1,2,7"` > > See https://rocm.docs.amd.com/en/latest/conceptual/gpu-isolation.html for more info > > I consider this problem solved, but I think it should be possible for ollama to figure this out by itself somehow... I've run `export ROCR_VISIBLE_DEVICES=0` in the command line and restarted. But ollama is still using the integrated GPU. I've restarted the daemon and ollama. Can you help me set up ollama so that it uses the external GPU (AMD 7900 xtx)? I am on Arch Linux.
GiteaMirror commented 2026-04-22 03:44:50 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jan 31, 2024):

@misaligar can you share a log of what the server is doing with debug enabled so we can see why?

% OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve
...
<!-- gh-comment-id:1919505629 --> @dhiltgen commented on GitHub (Jan 31, 2024): @misaligar can you share a log of what the server is doing with debug enabled so we can see why? ``` % OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve ... ```
GiteaMirror commented 2026-04-22 03:44:51 -05:00
Author
Owner
Copy Link

@meminens commented on GitHub (Jan 31, 2024):

@misaligar can you share a log of what the server is doing with debug enabled so we can see why?

% OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve
...

Hope this helps!

misal@arch:~$ OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve
time=2024-01-31T13:25:01.376-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/server/routes.go:926 msg="Debug logging enabled"
time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/images.go:857 msg="total blobs: 5"
time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/images.go:864 msg="total unused blobs removed: 0"
time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/routes.go:950 msg="Listening on 127.0.0.1:11434 (version 0.1.22)"
time=2024-01-31T13:25:01.377-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:106 msg="Extracting dynamic libraries..."
time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:145 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v5 cpu rocm_v6 cpu_avx]"
time=2024-01-31T13:25:02.754-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:146 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:94 msg="Detecting GPU type"
time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:236 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-01-31T13:25:02.754-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:254 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /home/misal/libnvidia-ml.so*]"
time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:282 msg="Discovered GPU libraries: []"
time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:236 msg="Searching for GPU management library librocm_smi64.so"
time=2024-01-31T13:25:02.758-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:254 msg="gpu management search paths: [/opt/rocm*/lib*/librocm_smi64.so* /home/misal/librocm_smi64.so*]"
time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:282 msg="Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0]"
wiring rocm management library functions in /opt/rocm/lib/librocm_smi64.so.5.0
dlsym: rsmi_init
dlsym: rsmi_shut_down
dlsym: rsmi_dev_memory_total_get
dlsym: rsmi_dev_memory_usage_get
dlsym: rsmi_version_get
dlsym: rsmi_num_monitor_devices
dlsym: rsmi_dev_id_get
dlsym: rsmi_dev_name_get
dlsym: rsmi_dev_brand_get
dlsym: rsmi_dev_vendor_name_get
dlsym: rsmi_dev_vram_vendor_get
dlsym: rsmi_dev_serial_number_get
dlsym: rsmi_dev_subsystem_name_get
dlsym: rsmi_dev_vbios_version_get
time=2024-01-31T13:25:02.760-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:109 msg="Radeon GPU detected"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX]
[0] ROCm vbios version: 113-31XFSHBS1-L02
[0] ROCm totalMem 25753026560
[0] ROCm usedMem 1080086528
time=2024-01-31T13:25:02.761-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:225 msg="rocm detected 1 devices with 21176M available memory"
[GIN] 2024/01/31 - 13:25:09 | 200 |      24.346µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/01/31 - 13:25:09 | 200 |     417.489µs |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/01/31 - 13:25:09 | 200 |     415.786µs |       127.0.0.1 | POST     "/api/show"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX]
[0] ROCm vbios version: 113-31XFSHBS1-L02
[0] ROCm totalMem 25753026560
[0] ROCm usedMem 1075900416
time=2024-01-31T13:25:09.854-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:225 msg="rocm detected 1 devices with 21180M available memory"
discovered 1 ROCm GPU Devices
[0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX]
[0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI]
[0] ROCm VRAM vendor: samsung
rsmi_dev_serial_number_get failed: 2
[0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX]
[0] ROCm vbios version: 113-31XFSHBS1-L02
[0] ROCm totalMem 25753026560
[0] ROCm usedMem 1075900416
time=2024-01-31T13:25:09.855-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/cpu_common.go:11 msg="CPU has AVX2"
loading library /tmp/ollama679134691/rocm_v5/libext_server.so
time=2024-01-31T13:25:09.882-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama679134691/rocm_v5/libext_server.so"
time=2024-01-31T13:25:09.882-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:145 msg="Initializing llama server"
[1706725509] system info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | 
[1706725509] Performing pre-initialization of GPU
free(): invalid pointer
Aborted (core dumped)
<!-- gh-comment-id:1919690678 --> @meminens commented on GitHub (Jan 31, 2024): > @misaligar can you share a log of what the server is doing with debug enabled so we can see why? > > ``` > % OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve > ... > ``` Hope this helps! ``` misal@arch:~$ OLLAMA_DEBUG=1 ROCR_VISIBLE_DEVICES=0 ollama serve time=2024-01-31T13:25:01.376-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/server/routes.go:926 msg="Debug logging enabled" time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/images.go:857 msg="total blobs: 5" time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/images.go:864 msg="total unused blobs removed: 0" time=2024-01-31T13:25:01.376-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/server/routes.go:950 msg="Listening on 127.0.0.1:11434 (version 0.1.22)" time=2024-01-31T13:25:01.377-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:106 msg="Extracting dynamic libraries..." time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:145 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v5 cpu rocm_v6 cpu_avx]" time=2024-01-31T13:25:02.754-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/llm/payload_common.go:146 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:94 msg="Detecting GPU type" time=2024-01-31T13:25:02.754-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:236 msg="Searching for GPU management library libnvidia-ml.so" time=2024-01-31T13:25:02.754-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:254 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /home/misal/libnvidia-ml.so*]" time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:282 msg="Discovered GPU libraries: []" time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:236 msg="Searching for GPU management library librocm_smi64.so" time=2024-01-31T13:25:02.758-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:254 msg="gpu management search paths: [/opt/rocm*/lib*/librocm_smi64.so* /home/misal/librocm_smi64.so*]" time=2024-01-31T13:25:02.758-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:282 msg="Discovered GPU libraries: [/opt/rocm/lib/librocm_smi64.so.5.0]" wiring rocm management library functions in /opt/rocm/lib/librocm_smi64.so.5.0 dlsym: rsmi_init dlsym: rsmi_shut_down dlsym: rsmi_dev_memory_total_get dlsym: rsmi_dev_memory_usage_get dlsym: rsmi_version_get dlsym: rsmi_num_monitor_devices dlsym: rsmi_dev_id_get dlsym: rsmi_dev_name_get dlsym: rsmi_dev_brand_get dlsym: rsmi_dev_vendor_name_get dlsym: rsmi_dev_vram_vendor_get dlsym: rsmi_dev_serial_number_get dlsym: rsmi_dev_subsystem_name_get dlsym: rsmi_dev_vbios_version_get time=2024-01-31T13:25:02.760-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:109 msg="Radeon GPU detected" discovered 1 ROCm GPU Devices [0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI] [0] ROCm VRAM vendor: samsung rsmi_dev_serial_number_get failed: 2 [0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX] [0] ROCm vbios version: 113-31XFSHBS1-L02 [0] ROCm totalMem 25753026560 [0] ROCm usedMem 1080086528 time=2024-01-31T13:25:02.761-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:225 msg="rocm detected 1 devices with 21176M available memory" [GIN] 2024/01/31 - 13:25:09 | 200 | 24.346µs | 127.0.0.1 | HEAD "/" [GIN] 2024/01/31 - 13:25:09 | 200 | 417.489µs | 127.0.0.1 | POST "/api/show" [GIN] 2024/01/31 - 13:25:09 | 200 | 415.786µs | 127.0.0.1 | POST "/api/show" discovered 1 ROCm GPU Devices [0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI] [0] ROCm VRAM vendor: samsung rsmi_dev_serial_number_get failed: 2 [0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX] [0] ROCm vbios version: 113-31XFSHBS1-L02 [0] ROCm totalMem 25753026560 [0] ROCm usedMem 1075900416 time=2024-01-31T13:25:09.854-05:00 level=DEBUG source=/go/src/github.com/jmorganca/ollama/gpu/gpu.go:225 msg="rocm detected 1 devices with 21180M available memory" discovered 1 ROCm GPU Devices [0] ROCm device name: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm brand: Navi 31 [Radeon RX 7900 XT/7900 XTX] [0] ROCm vendor: Advanced Micro Devices, Inc. [AMD/ATI] [0] ROCm VRAM vendor: samsung rsmi_dev_serial_number_get failed: 2 [0] ROCm subsystem name: RX-79XMERCB9 [SPEEDSTER MERC 310 RX 7900 XTX] [0] ROCm vbios version: 113-31XFSHBS1-L02 [0] ROCm totalMem 25753026560 [0] ROCm usedMem 1075900416 time=2024-01-31T13:25:09.855-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/gpu/cpu_common.go:11 msg="CPU has AVX2" loading library /tmp/ollama679134691/rocm_v5/libext_server.so time=2024-01-31T13:25:09.882-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama679134691/rocm_v5/libext_server.so" time=2024-01-31T13:25:09.882-05:00 level=INFO source=/go/src/github.com/jmorganca/ollama/llm/dyn_ext_server.go:145 msg="Initializing llama server" [1706725509] system info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | [1706725509] Performing pre-initialization of GPU free(): invalid pointer Aborted (core dumped) ```
GiteaMirror commented 2026-04-22 03:44:54 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jan 31, 2024):

@misaligar this looks unrelated to integrated GPUs. You appear to have hit #2165

<!-- gh-comment-id:1919978933 --> @dhiltgen commented on GitHub (Jan 31, 2024): @misaligar this looks unrelated to integrated GPUs. You appear to have hit #2165
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label amd bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#26950
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?