github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #5239] Mutli-GPU asymmetric VRAM with smaller first causes ordering bug and incorrect tensor split - cudaMalloc failed: out of memory #49790

New Issue
Closed
opened 2026-04-28 12:55:59 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-28 12:55:59 -05:00
Owner
Copy Link

Originally created by @chrisoutwright on GitHub (Jun 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5239

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

After going to 0.1.45 from 0.1.43 version I get out of memory, I did try as well
Set-ItemProperty -Path 'HKCU:\Environment' -Name 'OLLAMA_SCHED_SPREAD' -Value 1
and
Set-ItemProperty -Path 'HKCU:\Environment' -Name 'CUDA_VISIBLE_DEVICES' -Value "0,1"

But still it is happening.

llm_load_print_meta: model ftype      = Q6_K
llm_load_print_meta: model params     = 22.25 B
llm_load_print_meta: model size       = 17.00 GiB (6.56 BPW)
llm_load_print_meta: general.name     = Codestral-22B-v0.1
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 781 '<0x0A>'
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 2 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes
  Device 1: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
llm_load_tensors: ggml ctx size =    0.77 MiB
time=2024-06-23T16:45:00.347+02:00 level=INFO source=server.go:589 msg="waiting for server to become available" status="llm server loading model"
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 11597.72 MiB on device 0: cudaMalloc failed: out of memory
llama_model_load: error loading model: unable to allocate backend buffer
llama_load_model_from_file: exception loading model
time=2024-06-23T16:45:01.388+02:00 level=INFO source=server.go:589 msg="waiting for server to become available" status="llm server error"
time=2024-06-23T16:45:01.652+02:00 level=ERROR source=sched.go:388 msg="error loading llama server" error="llama runner process has terminated: exit status 0xc0000409 cudaMalloc failed: out of memory"
[GIN] 2024/06/23 - 16:45:01 | 500 |    1.8154377s |             ::1 | POST     "/api/chat"

What could be the issue? I thought GPU splitting would work out of the box now?

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.1.45

Originally created by @chrisoutwright on GitHub (Jun 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5239 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? After going to 0.1.45 from 0.1.43 version I get out of memory, I did try as well Set-ItemProperty -Path 'HKCU:\Environment' -Name 'OLLAMA_SCHED_SPREAD' -Value 1 and Set-ItemProperty -Path 'HKCU:\Environment' -Name 'CUDA_VISIBLE_DEVICES' -Value "0,1" But still it is happening. ``` llm_load_print_meta: model ftype = Q6_K llm_load_print_meta: model params = 22.25 B llm_load_print_meta: model size = 17.00 GiB (6.56 BPW) llm_load_print_meta: general.name = Codestral-22B-v0.1 llm_load_print_meta: BOS token = 1 '<s>' llm_load_print_meta: EOS token = 2 '</s>' llm_load_print_meta: UNK token = 0 '<unk>' llm_load_print_meta: LF token = 781 '<0x0A>' ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes ggml_cuda_init: found 2 CUDA devices: Device 0: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes Device 1: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes llm_load_tensors: ggml ctx size = 0.77 MiB time=2024-06-23T16:45:00.347+02:00 level=INFO source=server.go:589 msg="waiting for server to become available" status="llm server loading model" ggml_backend_cuda_buffer_type_alloc_buffer: allocating 11597.72 MiB on device 0: cudaMalloc failed: out of memory llama_model_load: error loading model: unable to allocate backend buffer llama_load_model_from_file: exception loading model time=2024-06-23T16:45:01.388+02:00 level=INFO source=server.go:589 msg="waiting for server to become available" status="llm server error" time=2024-06-23T16:45:01.652+02:00 level=ERROR source=sched.go:388 msg="error loading llama server" error="llama runner process has terminated: exit status 0xc0000409 cudaMalloc failed: out of memory" [GIN] 2024/06/23 - 16:45:01 | 500 | 1.8154377s | ::1 | POST "/api/chat" ``` What could be the issue? I thought GPU splitting would work out of the box now? ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.1.45
GiteaMirror added the needs more infobugnvidiawindows labels 2026-04-28 12:56:01 -05:00
GiteaMirror commented 2026-04-28 12:56:10 -05:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (Jun 23, 2024):

This is what I get in 0.1.43:

llm_load_vocab: token to piece cache size = 0.3368 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32768
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 32768
llm_load_print_meta: n_embd           = 6144
llm_load_print_meta: n_head           = 48
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 56
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 6
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 16384
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 32768
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = ?B
llm_load_print_meta: model ftype      = Q6_K
llm_load_print_meta: model params     = 22.25 B
llm_load_print_meta: model size       = 17.00 GiB (6.56 BPW)
llm_load_print_meta: general.name     = Codestral-22B-v0.1
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 781 '<0x0A>'
llm_load_print_meta: PRE token        = 32007 '材'
llm_load_print_meta: SUF token        = 32008 'ホ'
llm_load_print_meta: MID token        = 32009 '張'
llm_load_print_meta: EOT token        = 32010 '洞'
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:   no
ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes
ggml_cuda_init: found 2 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
  Device 1: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes
time=2024-06-23T16:56:47.829+02:00 level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server loading model"
llm_load_tensors: ggml ctx size =    0.77 MiB
llm_load_tensors: offloading 56 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 57/57 layers to GPU
llm_load_tensors:        CPU buffer size =   157.50 MiB
llm_load_tensors:      CUDA0 buffer size = 12208.12 MiB
llm_load_tensors:      CUDA1 buffer size =  5040.77 MiB
llama_new_context_with_model: n_ctx      = 20000
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      CUDA0 KV buffer size =  3125.00 MiB
llama_kv_cache_init:      CUDA1 KV buffer size =  1250.00 MiB
llama_new_context_with_model: KV self size  = 4375.00 MiB, K (f16): 2187.50 MiB, V (f16): 2187.50 MiB
llama_new_context_with_model:  CUDA_Host  output buffer size =     0.15 MiB
llama_new_context_with_model: pipeline parallelism enabled (n_copies=4)
llama_new_context_with_model:      CUDA0 compute buffer size =  2127.26 MiB
llama_new_context_with_model:      CUDA1 compute buffer size =  2127.27 MiB
llama_new_context_with_model:  CUDA_Host compute buffer size =   168.27 MiB
llama_new_context_with_model: graph nodes  = 1798
llama_new_context_with_model: graph splits = 3
INFO [wmain] model loaded | tid="14480" timestamp=1719154614
time=2024-06-23T16:56:54.713+02:00 level=INFO source=server.go:572 msg="llama runner started in 8.67 seconds"
[GIN] 2024/06/23 - 16:56:56 | 200 |   10.8633922s |             ::1 | POST     "/api/chat"
[GIN] 2024/06/23 - 16:56:57 | 200 |    688.4637ms |             ::1 | POST     "/v1/chat/completions"
<!-- gh-comment-id:2185022335 --> @chrisoutwright commented on GitHub (Jun 23, 2024): This is what I get in 0.1.43: ``` llm_load_vocab: token to piece cache size = 0.3368 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = SPM llm_load_print_meta: n_vocab = 32768 llm_load_print_meta: n_merges = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 6144 llm_load_print_meta: n_head = 48 llm_load_print_meta: n_head_kv = 8 llm_load_print_meta: n_layer = 56 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 6 llm_load_print_meta: n_embd_k_gqa = 1024 llm_load_print_meta: n_embd_v_gqa = 1024 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 16384 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_yarn_orig_ctx = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: model type = ?B llm_load_print_meta: model ftype = Q6_K llm_load_print_meta: model params = 22.25 B llm_load_print_meta: model size = 17.00 GiB (6.56 BPW) llm_load_print_meta: general.name = Codestral-22B-v0.1 llm_load_print_meta: BOS token = 1 '<s>' llm_load_print_meta: EOS token = 2 '</s>' llm_load_print_meta: UNK token = 0 '<unk>' llm_load_print_meta: LF token = 781 '<0x0A>' llm_load_print_meta: PRE token = 32007 '材' llm_load_print_meta: SUF token = 32008 'ホ' llm_load_print_meta: MID token = 32009 '張' llm_load_print_meta: EOT token = 32010 '洞' ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: CUDA_USE_TENSOR_CORES: yes ggml_cuda_init: found 2 CUDA devices: Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes Device 1: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes time=2024-06-23T16:56:47.829+02:00 level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server loading model" llm_load_tensors: ggml ctx size = 0.77 MiB llm_load_tensors: offloading 56 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 57/57 layers to GPU llm_load_tensors: CPU buffer size = 157.50 MiB llm_load_tensors: CUDA0 buffer size = 12208.12 MiB llm_load_tensors: CUDA1 buffer size = 5040.77 MiB llama_new_context_with_model: n_ctx = 20000 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 1000000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: CUDA0 KV buffer size = 3125.00 MiB llama_kv_cache_init: CUDA1 KV buffer size = 1250.00 MiB llama_new_context_with_model: KV self size = 4375.00 MiB, K (f16): 2187.50 MiB, V (f16): 2187.50 MiB llama_new_context_with_model: CUDA_Host output buffer size = 0.15 MiB llama_new_context_with_model: pipeline parallelism enabled (n_copies=4) llama_new_context_with_model: CUDA0 compute buffer size = 2127.26 MiB llama_new_context_with_model: CUDA1 compute buffer size = 2127.27 MiB llama_new_context_with_model: CUDA_Host compute buffer size = 168.27 MiB llama_new_context_with_model: graph nodes = 1798 llama_new_context_with_model: graph splits = 3 INFO [wmain] model loaded | tid="14480" timestamp=1719154614 time=2024-06-23T16:56:54.713+02:00 level=INFO source=server.go:572 msg="llama runner started in 8.67 seconds" [GIN] 2024/06/23 - 16:56:56 | 200 | 10.8633922s | ::1 | POST "/api/chat" [GIN] 2024/06/23 - 16:56:57 | 200 | 688.4637ms | ::1 | POST "/v1/chat/completions" ```
GiteaMirror commented 2026-04-28 12:56:12 -05:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (Jun 23, 2024):

0.1.45_output_ollama_serve.log

0.1.43_output_ollama_serve.log

<!-- gh-comment-id:2185041301 --> @chrisoutwright commented on GitHub (Jun 23, 2024): [0.1.45_output_ollama_serve.log](https://github.com/user-attachments/files/15945164/0.1.45_output_ollama_serve.log) [0.1.43_output_ollama_serve.log](https://github.com/user-attachments/files/15945168/0.1.43_output_ollama_serve.log)
GiteaMirror commented 2026-04-28 12:56:16 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 23, 2024):

Can you try again setting CUDA_VISIBLE_DEVICES to the GUIDs and pick the larger GPU first? I think there's a logic error in here and we're assuming the bigger GPU is first, but device 0 is your smaller GPU, so we're trying to put too many layers on that one, and too few on the bigger GPU.

excerpt from the log

time=2024-06-23T17:19:21.260+02:00 level=INFO source=types.go:98 msg="inference compute" id=GPU-971b407f-ae20-75ed-99c8-42c696057b0e library=cuda compute=8.9 driver=12.3 name="NVIDIA GeForce RTX 4090" total="24.0 GiB" available="22.5 GiB"
time=2024-06-23T17:19:21.260+02:00 level=INFO source=types.go:98 msg="inference compute" id=GPU-ae839d73-bcac-72a4-3ae2-6167ecc83e89 library=cuda compute=7.5 driver=12.3 name="NVIDIA GeForce RTX 2080 Ti" total="11.0 GiB" available="9.9 GiB"
time=2024-06-23T17:19:31.259+02:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=57 layers.offload=57 layers.split=42,15 memory.available="[23.6 GiB 10.0 GiB]" memory.required.full="31.9 GiB" memory.required.partial="31.9 GiB" memory.required.kv="4.3 GiB" memory.required.allocations="[22.1 GiB 9.8 GiB]" memory.weights.total="25.9 GiB" memory.weights.repeating="25.7 GiB" memory.weights.nonrepeating="204.0 MiB" memory.graph.full="2.0 GiB" memory.graph.partial="2.0 GiB"
time=2024-06-23T17:19:31.264+02:00 level=INFO source=server.go:359 msg="starting llama server" cmd="C:\\Users\\Chris\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\cuda_v11.3\\ollama_llama_server.exe --model D:\\Ollama\\models\\blobs\\sha256-90038122ded15fb535d2f1aee888d33916de5d83c5f66c1b107a3c757a79c326 --ctx-size 20000 --batch-size 512 --embedding --log-disable --n-gpu-layers 57 --no-mmap --parallel 1 --tensor-split 42,15 --tensor-split 42,15 --port 51212"
...
ggml_cuda_init: found 2 CUDA devices:
  Device 0: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes
  Device 1: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes
<!-- gh-comment-id:2185313247 --> @dhiltgen commented on GitHub (Jun 23, 2024): Can you try again setting CUDA_VISIBLE_DEVICES to the GUIDs and pick the larger GPU first? I think there's a logic error in here and we're assuming the bigger GPU is first, but device 0 is your smaller GPU, so we're trying to put too many layers on that one, and too few on the bigger GPU. excerpt from the log ``` time=2024-06-23T17:19:21.260+02:00 level=INFO source=types.go:98 msg="inference compute" id=GPU-971b407f-ae20-75ed-99c8-42c696057b0e library=cuda compute=8.9 driver=12.3 name="NVIDIA GeForce RTX 4090" total="24.0 GiB" available="22.5 GiB" time=2024-06-23T17:19:21.260+02:00 level=INFO source=types.go:98 msg="inference compute" id=GPU-ae839d73-bcac-72a4-3ae2-6167ecc83e89 library=cuda compute=7.5 driver=12.3 name="NVIDIA GeForce RTX 2080 Ti" total="11.0 GiB" available="9.9 GiB" time=2024-06-23T17:19:31.259+02:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=57 layers.offload=57 layers.split=42,15 memory.available="[23.6 GiB 10.0 GiB]" memory.required.full="31.9 GiB" memory.required.partial="31.9 GiB" memory.required.kv="4.3 GiB" memory.required.allocations="[22.1 GiB 9.8 GiB]" memory.weights.total="25.9 GiB" memory.weights.repeating="25.7 GiB" memory.weights.nonrepeating="204.0 MiB" memory.graph.full="2.0 GiB" memory.graph.partial="2.0 GiB" time=2024-06-23T17:19:31.264+02:00 level=INFO source=server.go:359 msg="starting llama server" cmd="C:\\Users\\Chris\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\cuda_v11.3\\ollama_llama_server.exe --model D:\\Ollama\\models\\blobs\\sha256-90038122ded15fb535d2f1aee888d33916de5d83c5f66c1b107a3c757a79c326 --ctx-size 20000 --batch-size 512 --embedding --log-disable --n-gpu-layers 57 --no-mmap --parallel 1 --tensor-split 42,15 --tensor-split 42,15 --port 51212" ... ggml_cuda_init: found 2 CUDA devices: Device 0: NVIDIA GeForce RTX 2080 Ti, compute capability 7.5, VMM: yes Device 1: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes ``` ```
GiteaMirror commented 2026-04-28 12:56:20 -05:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (Jun 23, 2024):

Yes @dhiltgen, by switching the order in CUDA_VISIBLE_DEVICES, it now works. Since the 4090 is that big, it only fits in the second PCIe slot without hitting anything, I would have not thought for that to play a role in the offloading.

<!-- gh-comment-id:2185336154 --> @chrisoutwright commented on GitHub (Jun 23, 2024): Yes @dhiltgen, by switching the order in CUDA_VISIBLE_DEVICES, it now works. Since the 4090 is that big, it only fits in the second PCIe slot without hitting anything, I would have not thought for that to play a role in the offloading.
GiteaMirror commented 2026-04-28 12:56:22 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 23, 2024):

If you omit the CUDA_VISIBLE_DEVICES and let the default algorithm run, do we get it right, or are we still favoring PCI slot IDs and messing up the order? (if we just use slots by default and not size, that's a bug I'll fix)

<!-- gh-comment-id:2185339234 --> @dhiltgen commented on GitHub (Jun 23, 2024): If you omit the CUDA_VISIBLE_DEVICES and let the default algorithm run, do we get it right, or are we still favoring PCI slot IDs and messing up the order? (if we just use slots by default and not size, that's a bug I'll fix)
GiteaMirror commented 2026-04-28 12:56:25 -05:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (Jun 25, 2024):

If you omit the CUDA_VISIBLE_DEVICES and let the default algorithm run, do we get it right, or are we still favoring PCI slot IDs and messing up the order? (if we just use slots by default and not size, that's a bug I'll fix)

I tried without it and it would still favor the first for finding out params for the splitting decision (in that case the wrong one), which resulted in a wrong one (out of memory)

<!-- gh-comment-id:2188356065 --> @chrisoutwright commented on GitHub (Jun 25, 2024): > If you omit the CUDA_VISIBLE_DEVICES and let the default algorithm run, do we get it right, or are we still favoring PCI slot IDs and messing up the order? (if we just use slots by default and not size, that's a bug I'll fix) I tried without it and it would still favor the first for finding out params for the splitting decision (in that case the wrong one), which resulted in a wrong one (out of memory)
GiteaMirror commented 2026-04-28 12:56:30 -05:00
Author
Owner
Copy Link

@sksonic commented on GitHub (Jul 5, 2024):

Might be related: https://github.com/ollama/ollama/issues/5476

<!-- gh-comment-id:2210190498 --> @sksonic commented on GitHub (Jul 5, 2024): Might be related: https://github.com/ollama/ollama/issues/5476
GiteaMirror commented 2026-04-28 12:56:34 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Oct 22, 2024):

Is this still a problem on the latest releases?

If you do get ordering incorrect, can you run with OLLAMA_DEBUG=1 set so we can see what is passed to the subprocess? It should sort by free space, and use the GUID now, so I believe this should be resolved.

<!-- gh-comment-id:2430510777 --> @dhiltgen commented on GitHub (Oct 22, 2024): Is this still a problem on the latest releases? If you do get ordering incorrect, can you run with OLLAMA_DEBUG=1 set so we can see what is passed to the subprocess? It should sort by free space, and use the GUID now, so I believe this should be resolved.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info nvidia windows
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#49790
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?