github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5573] ggml_cuda_init: failed to initialize CUDA: system has unsupported display driver / cuda driver combination #3484

New Issue
Closed
opened 2026-04-12 14:10:29 -05:00 by GiteaMirror · 18 comments
GiteaMirror commented 2026-04-12 14:10:29 -05:00
Owner
Copy Link

Originally created by @skinnynpale on GitHub (Jul 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5573

What is the issue?

time=2024-07-09T13:56:46.484Z level=INFO source=sched.go:738 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa gpu=GPU-72b1bc75-c26b-1c04-f9cd-ff1942a73215 parallel=4 available=24748556288 required="6.2 GiB"
time=2024-07-09T13:56:46.484Z level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[23.0 GiB]" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB"
time=2024-07-09T13:56:46.485Z level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama368855320/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --parallel 4 --port 39141"
time=2024-07-09T13:56:46.485Z level=INFO source=sched.go:474 msg="loaded runners" count=1
time=2024-07-09T13:56:46.485Z level=INFO source=server.go:563 msg="waiting for llama runner to start responding"
time=2024-07-09T13:56:46.486Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error"
INFO [main] build info | build=1 commit="a8db2a9" tid="139659088556032" timestamp=1720533406
INFO [main] system info | n_threads=32 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139659088556032" timestamp=1720533406 total_threads=64
INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="63" port="39141" tid="139659088556032" timestamp=1720533406
llama_model_loader: loaded meta data with 22 key-value pairs and 291 tensors from /root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = Meta-Llama-3-8B-Instruct
llama_model_loader: - kv   2:                          llama.block_count u32              = 32
llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
llama_model_loader: - kv   4:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   7:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   8:                       llama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                          general.file_type u32              = 2
llama_model_loader: - kv  11:                           llama.vocab_size u32              = 128256
llama_model_loader: - kv  12:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  14:                         tokenizer.ggml.pre str              = llama-bpe
llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  18:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 128009
llama_model_loader: - kv  20:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
llama_model_loader: - kv  21:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q4_0:  225 tensors
llama_model_loader: - type q6_K:    1 tensors
time=2024-07-09T13:56:46.737Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model"
llm_load_vocab: special tokens cache size = 256
llm_load_vocab: token to piece cache size = 0.8000 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 128256
llm_load_print_meta: n_merges         = 280147
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 8192
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 500000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn  = 8192
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 8B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 8.03 B
llm_load_print_meta: model size       = 4.33 GiB (4.64 BPW)
llm_load_print_meta: general.name     = Meta-Llama-3-8B-Instruct
llm_load_print_meta: BOS token        = 128000 '<|begin_of_text|>'
llm_load_print_meta: EOS token        = 128009 '<|eot_id|>'
llm_load_print_meta: LF token         = 128 'Ä'
llm_load_print_meta: EOT token        = 128009 '<|eot_id|>'
llm_load_print_meta: max token length = 256
ggml_cuda_init: failed to initialize CUDA: system has unsupported display driver / cuda driver combination
llm_load_tensors: ggml ctx size =    0.14 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors:        CPU buffer size =  4437.80 MiB
llama_new_context_with_model: n_ctx      = 8192
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 500000.0
llama_new_context_with_model: freq_scale = 1
ggml_cuda_host_malloc: failed to allocate 1024.00 MiB of pinned memory: system has unsupported display driver / cuda driver combination
llama_kv_cache_init:        CPU KV buffer size =  1024.00 MiB
llama_new_context_with_model: KV self size  = 1024.00 MiB, K (f16):  512.00 MiB, V (f16):  512.00 MiB
ggml_cuda_host_malloc: failed to allocate 2.02 MiB of pinned memory: system has unsupported display driver / cuda driver combination
llama_new_context_with_model:        CPU  output buffer size =     2.02 MiB
ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination
llama_new_context_with_model:  CUDA_Host compute buffer size =   560.01 MiB
llama_new_context_with_model: graph nodes  = 1030
llama_new_context_with_model: graph splits = 1
INFO [main] model loaded | tid="139659088556032" timestamp=1720533410
time=2024-07-09T13:56:50.631Z level=INFO source=server.go:609 msg="llama runner started in 4.15 seconds"
[GIN] 2024/07/09 - 13:56:50 | 200 |  4.429415046s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/07/09 - 13:57:41 | 200 |  8.412510746s |       127.0.0.1 | POST     "/api/chat"


nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0
nvidia-smi
Tue Jul  9 13:52:27 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:A2:00.0 Off |                  Off |
| 30%   34C    P8             23W /  450W |       1MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+


+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.2.1

Originally created by @skinnynpale on GitHub (Jul 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5573 ### What is the issue? ```bash time=2024-07-09T13:56:46.484Z level=INFO source=sched.go:738 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa gpu=GPU-72b1bc75-c26b-1c04-f9cd-ff1942a73215 parallel=4 available=24748556288 required="6.2 GiB" time=2024-07-09T13:56:46.484Z level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=33 layers.offload=33 layers.split="" memory.available="[23.0 GiB]" memory.required.full="6.2 GiB" memory.required.partial="6.2 GiB" memory.required.kv="1.0 GiB" memory.required.allocations="[6.2 GiB]" memory.weights.total="4.7 GiB" memory.weights.repeating="4.3 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="560.0 MiB" memory.graph.partial="677.5 MiB" time=2024-07-09T13:56:46.485Z level=INFO source=server.go:375 msg="starting llama server" cmd="/tmp/ollama368855320/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --parallel 4 --port 39141" time=2024-07-09T13:56:46.485Z level=INFO source=sched.go:474 msg="loaded runners" count=1 time=2024-07-09T13:56:46.485Z level=INFO source=server.go:563 msg="waiting for llama runner to start responding" time=2024-07-09T13:56:46.486Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error" INFO [main] build info | build=1 commit="a8db2a9" tid="139659088556032" timestamp=1720533406 INFO [main] system info | n_threads=32 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139659088556032" timestamp=1720533406 total_threads=64 INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="63" port="39141" tid="139659088556032" timestamp=1720533406 llama_model_loader: loaded meta data with 22 key-value pairs and 291 tensors from /root/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53868f220ff6e389f6f8ef87a01d77c96807de94ca2aa (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.name str = Meta-Llama-3-8B-Instruct llama_model_loader: - kv 2: llama.block_count u32 = 32 llama_model_loader: - kv 3: llama.context_length u32 = 8192 llama_model_loader: - kv 4: llama.embedding_length u32 = 4096 llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 llama_model_loader: - kv 6: llama.attention.head_count u32 = 32 llama_model_loader: - kv 7: llama.attention.head_count_kv u32 = 8 llama_model_loader: - kv 8: llama.rope.freq_base f32 = 500000.000000 llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 10: general.file_type u32 = 2 llama_model_loader: - kv 11: llama.vocab_size u32 = 128256 llama_model_loader: - kv 12: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 14: tokenizer.ggml.pre str = llama-bpe llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 128000 llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 128009 llama_model_loader: - kv 20: tokenizer.chat_template str = {% set loop_messages = messages %}{% ... llama_model_loader: - kv 21: general.quantization_version u32 = 2 llama_model_loader: - type f32: 65 tensors llama_model_loader: - type q4_0: 225 tensors llama_model_loader: - type q6_K: 1 tensors time=2024-07-09T13:56:46.737Z level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" llm_load_vocab: special tokens cache size = 256 llm_load_vocab: token to piece cache size = 0.8000 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 128256 llm_load_print_meta: n_merges = 280147 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 8192 llm_load_print_meta: n_embd = 4096 llm_load_print_meta: n_layer = 32 llm_load_print_meta: n_head = 32 llm_load_print_meta: n_head_kv = 8 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 4 llm_load_print_meta: n_embd_k_gqa = 1024 llm_load_print_meta: n_embd_v_gqa = 1024 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 14336 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 500000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 8192 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: model type = 8B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 8.03 B llm_load_print_meta: model size = 4.33 GiB (4.64 BPW) llm_load_print_meta: general.name = Meta-Llama-3-8B-Instruct llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>' llm_load_print_meta: EOS token = 128009 '<|eot_id|>' llm_load_print_meta: LF token = 128 'Ä' llm_load_print_meta: EOT token = 128009 '<|eot_id|>' llm_load_print_meta: max token length = 256 ggml_cuda_init: failed to initialize CUDA: system has unsupported display driver / cuda driver combination llm_load_tensors: ggml ctx size = 0.14 MiB llm_load_tensors: offloading 32 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 33/33 layers to GPU llm_load_tensors: CPU buffer size = 4437.80 MiB llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 500000.0 llama_new_context_with_model: freq_scale = 1 ggml_cuda_host_malloc: failed to allocate 1024.00 MiB of pinned memory: system has unsupported display driver / cuda driver combination llama_kv_cache_init: CPU KV buffer size = 1024.00 MiB llama_new_context_with_model: KV self size = 1024.00 MiB, K (f16): 512.00 MiB, V (f16): 512.00 MiB ggml_cuda_host_malloc: failed to allocate 2.02 MiB of pinned memory: system has unsupported display driver / cuda driver combination llama_new_context_with_model: CPU output buffer size = 2.02 MiB ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination llama_new_context_with_model: CUDA_Host compute buffer size = 560.01 MiB llama_new_context_with_model: graph nodes = 1030 llama_new_context_with_model: graph splits = 1 INFO [main] model loaded | tid="139659088556032" timestamp=1720533410 time=2024-07-09T13:56:50.631Z level=INFO source=server.go:609 msg="llama runner started in 4.15 seconds" [GIN] 2024/07/09 - 13:56:50 | 200 | 4.429415046s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/07/09 - 13:57:41 | 200 | 8.412510746s | 127.0.0.1 | POST "/api/chat" ``` ```bash nvcc --version nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Fri_Jan__6_16:45:21_PST_2023 Cuda compilation tools, release 12.0, V12.0.140 Build cuda_12.0.r12.0/compiler.32267302_0 nvidia-smi Tue Jul 9 13:52:27 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4090 On | 00000000:A2:00.0 Off | Off | | 30% 34C P8 23W / 450W | 1MiB / 24564MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ ``` ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.2.1
GiteaMirror added the gpunvidiabug labels 2026-04-12 14:10:29 -05:00
GiteaMirror commented 2026-04-12 14:10:32 -05:00
Author
Owner
Copy Link

@iflamed commented on GitHub (Jul 9, 2024):

Same problem.

<!-- gh-comment-id:2218215032 --> @iflamed commented on GitHub (Jul 9, 2024): Same problem.
GiteaMirror commented 2026-04-12 14:10:33 -05:00
Author
Owner
Copy Link

@skinnynpale commented on GitHub (Jul 9, 2024):

I switch CPU from AMD EPYC 7282 16-Core Processor to Xeon® E5-2680 v4 ... and all WORKS

<!-- gh-comment-id:2218363703 --> @skinnynpale commented on GitHub (Jul 9, 2024): I switch CPU from AMD EPYC 7282 16-Core Processor to **Xeon® E5-2680 v4** ... and all WORKS
GiteaMirror commented 2026-04-12 14:10:34 -05:00
Author
Owner
Copy Link

@chigkim commented on GitHub (Jul 9, 2024):

Same problem on runpod.io instance with rtx-3090. I install lshw.

<!-- gh-comment-id:2219029518 --> @chigkim commented on GitHub (Jul 9, 2024): Same problem on runpod.io instance with rtx-3090. I install lshw.
GiteaMirror commented 2026-04-12 14:10:34 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Jul 10, 2024):

Hi folks sorry about this. Are you running in Docker by chance?

<!-- gh-comment-id:2219098684 --> @jmorganca commented on GitHub (Jul 10, 2024): Hi folks sorry about this. Are you running in Docker by chance?
GiteaMirror commented 2026-04-12 14:10:35 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jul 10, 2024):

Can you check to see which version of the cuda driver library is installed? Not sure what Distro/package manager you're using but something like

% dpkg -l | grep "libnvidia-compute-"

If the version doesn't match the driver (550 from the nvidia-smi output above), that could explain the error. The driver and cuda driver library must match exactly, and the package dependencies should handle this, but it seems the 555 driver packaging has created some hiccups.

<!-- gh-comment-id:2219100519 --> @dhiltgen commented on GitHub (Jul 10, 2024): Can you check to see which version of the cuda driver library is installed? Not sure what Distro/package manager you're using but something like ``` % dpkg -l | grep "libnvidia-compute-" ``` If the version doesn't match the driver (550 from the nvidia-smi output above), that could explain the error. The driver and cuda driver library must match exactly, and the package dependencies should handle this, but it seems the 555 driver packaging has created some hiccups.
GiteaMirror commented 2026-04-12 14:10:35 -05:00
Author
Owner
Copy Link

@iflamed commented on GitHub (Jul 10, 2024):

Hi folks sorry about this. Are you running in Docker by chance?

Yes, I run ollama in the docker.

12 vCPU Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz
Miniconda  conda3
Python  3.8(ubuntu20.04)
Cuda  11.8
RTX 4090(24GB) * 1
<!-- gh-comment-id:2219327868 --> @iflamed commented on GitHub (Jul 10, 2024): > Hi folks sorry about this. Are you running in Docker by chance? Yes, I run ollama in the docker. ``` 12 vCPU Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz Miniconda conda3 Python 3.8(ubuntu20.04) Cuda 11.8 RTX 4090(24GB) * 1 ```
GiteaMirror commented 2026-04-12 14:10:36 -05:00
Author
Owner
Copy Link

@iflamed commented on GitHub (Jul 10, 2024):

dpkg -l | grep "libnvidia-compute-"

This command output nothing in my docker.

I got some information from nvidia-smi:

NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2
<!-- gh-comment-id:2219330669 --> @iflamed commented on GitHub (Jul 10, 2024): > dpkg -l | grep "libnvidia-compute-" This command output nothing in my docker. I got some information from `nvidia-smi`: ``` NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 ```
GiteaMirror commented 2026-04-12 14:10:37 -05:00
Author
Owner
Copy Link

@chigkim commented on GitHub (Jul 10, 2024):

Also running dpkg -l | grep "libnvidia-compute-" outputs nothing.

# nvidia-smi
Wed Jul 10 02:17:17 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        On  |   00000000:19:00.0 Off |                  N/A |
|  0%   45C    P8             30W /  350W |       1MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
<!-- gh-comment-id:2219357740 --> @chigkim commented on GitHub (Jul 10, 2024): Also running `dpkg -l | grep "libnvidia-compute-"` outputs nothing. ```bash # nvidia-smi Wed Jul 10 02:17:17 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3090 On | 00000000:19:00.0 Off | N/A | | 0% 45C P8 30W / 350W | 1MiB / 24576MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+ ```
GiteaMirror commented 2026-04-12 14:10:37 -05:00
Author
Owner
Copy Link

@adamfilli commented on GitHub (Jul 10, 2024):

Also having this issue on Runpod:

0|ollama  | ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination

nvidia-smi
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |

I have another host from a few days ago that is working still:

nvidia-smi
| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2
<!-- gh-comment-id:2219364855 --> @adamfilli commented on GitHub (Jul 10, 2024): Also having this issue on Runpod: ``` 0|ollama | ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination ``` ``` nvidia-smi | NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 | ``` I have another host from a few days ago that is working still: ``` nvidia-smi | NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 ```
GiteaMirror commented 2026-04-12 14:10:37 -05:00
Author
Owner
Copy Link

@skinnynpale commented on GitHub (Jul 10, 2024):

Hi folks sorry about this. Are you running in Docker by chance?

Yes, i'm working inside docker container.

<!-- gh-comment-id:2219734309 --> @skinnynpale commented on GitHub (Jul 10, 2024): > Hi folks sorry about this. Are you running in Docker by chance? Yes, i'm working inside docker container.
GiteaMirror commented 2026-04-12 14:10:38 -05:00
Author
Owner
Copy Link

@Farah-Boun commented on GitHub (Jul 10, 2024):

I got the same issue on Kaggle on both T4 and P100 GPUs

<!-- gh-comment-id:2219812195 --> @Farah-Boun commented on GitHub (Jul 10, 2024): I got the same issue on Kaggle on both T4 and P100 GPUs
GiteaMirror commented 2026-04-12 14:10:38 -05:00
Author
Owner
Copy Link

@box9527 commented on GitHub (Jul 10, 2024):

Hi folks sorry about this. Are you running in Docker by chance?

I am suffering this issue with docker too.

<!-- gh-comment-id:2219913456 --> @box9527 commented on GitHub (Jul 10, 2024): > Hi folks sorry about this. Are you running in Docker by chance? I am suffering this issue with docker too.
GiteaMirror commented 2026-04-12 14:10:40 -05:00
Author
Owner
Copy Link

@jianghuawang commented on GitHub (Jul 10, 2024):

I meet the same problem, but I solved. it. Just make sure the version of CUDA is compatible with your CUDA compiler.
run nvcc --version to check the version of CUDA compiler.
Next follows the guide on CUDA-toolkit-archive to download the compatible CUDA Toolkit.

<!-- gh-comment-id:2219994232 --> @jianghuawang commented on GitHub (Jul 10, 2024): I meet the same problem, but I solved. it. Just make sure the version of CUDA is compatible with your CUDA compiler. run `nvcc --version` to check the version of CUDA compiler. Next follows the guide on [CUDA-toolkit-archive](https://developer.nvidia.com/cuda-toolkit-archive) to download the compatible CUDA Toolkit.
GiteaMirror commented 2026-04-12 14:10:41 -05:00
Author
Owner
Copy Link

@ckj18 commented on GitHub (Jul 10, 2024):

Hello everyone, I had the same error. It worked well with Docker that I installed earlier, but when I created a new container and ran the latest version of Ollama, it didn't use the GPU. So, I tested it with the previous version, and it worked fine. Now, let's try using the following command:
curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.48#' | sh

<!-- gh-comment-id:2220053828 --> @ckj18 commented on GitHub (Jul 10, 2024): Hello everyone, I had the same error. It worked well with Docker that I installed earlier, but when I created a new container and ran the latest version of Ollama, it didn't use the GPU. So, I tested it with the previous version, and it worked fine. Now, let's try using the following command: `curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.48#' | sh`
GiteaMirror commented 2026-04-12 14:10:42 -05:00
Author
Owner
Copy Link

@adamfilli commented on GitHub (Jul 10, 2024):

curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.48#' | sh

this resolved the issue for me, thanks @ckj18 !

<!-- gh-comment-id:2221210026 --> @adamfilli commented on GitHub (Jul 10, 2024): ``` curl -fsSL https://ollama.com/install.sh | sed 's#https://ollama.com/download#https://github.com/jmorganca/ollama/releases/download/v0.1.48#' | sh ``` this resolved the issue for me, thanks @ckj18 !
GiteaMirror commented 2026-04-12 14:10:42 -05:00
Author
Owner
Copy Link

@duktig666 commented on GitHub (Jul 11, 2024):

CUDA compiler

To solve the problem, reinstall the CUDA compiler and ensure that the cuda version is the same as the CUDA compiler version. I tried runpod 3090 to reinstall the compiler and adjust the GPU.

<!-- gh-comment-id:2221899329 --> @duktig666 commented on GitHub (Jul 11, 2024): > CUDA compiler To solve the problem, reinstall the CUDA compiler and ensure that the cuda version is the same as the CUDA compiler version. I tried runpod 3090 to reinstall the compiler and adjust the GPU.
GiteaMirror commented 2026-04-12 14:10:44 -05:00
Author
Owner
Copy Link

@duktig666 commented on GitHub (Jul 11, 2024):

Also having this issue on Runpod:

0|ollama  | ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination

nvidia-smi
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |

I have another host from a few days ago that is working still:

nvidia-smi
| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-4

https://developer.nvidia.com/cuda-12-4-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_network

This will solve the problem

<!-- gh-comment-id:2221900150 --> @duktig666 commented on GitHub (Jul 11, 2024): > Also having this issue on Runpod: > > ``` > 0|ollama | ggml_cuda_host_malloc: failed to allocate 560.01 MiB of pinned memory: system has unsupported display driver / cuda driver combination > ``` > > ``` > nvidia-smi > | NVIDIA-SMI 535.129.03 Driver Version: 535.129.03 CUDA Version: 12.2 | > ``` > > I have another host from a few days ago that is working still: > > ``` > nvidia-smi > | NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2 > ``` wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb sudo dpkg -i cuda-keyring_1.1-1_all.deb sudo apt-get update sudo apt-get -y install cuda-toolkit-12-4 https://developer.nvidia.com/cuda-12-4-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=Ubuntu&target_version=22.04&target_type=deb_network This will solve the problem
GiteaMirror commented 2026-04-12 14:10:44 -05:00
Author
Owner
Copy Link

@kanishkaditya commented on GitHub (Aug 19, 2025):

I am still facing this issue pls help, When i do nvidia-smi the driver version is 550.144.03 and cuda version is written as 12.4, apart from this dpkg -l | grep "libnvidia-compute-" outputs nothing and nvcc --version outputs release 11.8, v11.8.89

<!-- gh-comment-id:3201645586 --> @kanishkaditya commented on GitHub (Aug 19, 2025): I am still facing this issue pls help, When i do nvidia-smi the driver version is 550.144.03 and cuda version is written as 12.4, apart from this dpkg -l | grep "libnvidia-compute-" outputs nothing and nvcc --version outputs release 11.8, v11.8.89
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug gpu nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3484
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?