github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #14433] qwen3.5-27B works but qwen3.5-35B qwen3.5-122B failed in 0.17.1-rc2 #35131

New Issue
Closed
opened 2026-04-22 19:24:29 -05:00 by GiteaMirror · 12 comments
GiteaMirror commented 2026-04-22 19:24:29 -05:00
Owner
Copy Link

Originally created by @houzhuanghao on GitHub (Feb 26, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/14433

What is the issue?

unsing RTX4090 24GB, 128G RAM

qwen3.5-27B_q4_K_M was success run in 0.17.1-rc2 ;
but when pull the newly version of qwen3.5-35B and qwen3.5:122b-a10b, the ollama failed.

Error: 500 Internal Server Error: model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details

instresting imformations: cmd run: ollama run qwen3.5:122b-a10b (or qwen3.5:35b), when input the question words<29 chinese characters, it runs. when input question words>30 chinese characters, it failed.

Relevant log output

[GIN] 2026/02/26 - 12:19:37 | 200 |            0s |       127.0.0.1 | GET      "/api/version"
[GIN] 2026/02/26 - 12:19:37 | 200 |       2.191ms |       127.0.0.1 | GET      "/api/tags"
[GIN] 2026/02/26 - 12:19:38 | 401 |    402.2283ms |       127.0.0.1 | POST     "/api/me"
[GIN] 2026/02/26 - 12:19:50 | 200 |    1.5592533s |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/02/26 - 12:19:50 | 200 |    105.5966ms |       127.0.0.1 | POST     "/api/show"
time=2026-02-26T12:19:50.297+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 49392"
time=2026-02-26T12:19:53.295+08:00 level=INFO source=runner.go:464 msg="failure during GPU discovery" OLLAMA_LIBRARY_PATH="[C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\lib\\ollama C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cuda_v12]" extra_envs=map[] error="failed to finish discovery before timeout"
time=2026-02-26T12:19:53.296+08:00 level=WARN source=runner.go:356 msg="unable to refresh free memory, using old values"
time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:148 msg=packages count=1
time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:164 msg="efficiency cores detected" maxEfficiencyClass=1
time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=24 efficiency=16 threads=32
time=2026-02-26T12:19:53.371+08:00 level=INFO source=server.go:247 msg="enabling flash attention"
time=2026-02-26T12:19:53.372+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model G:\\models\\blobs\\sha256-2abd0d805943fa113f934d1ae4f2d5a749b5d4fe2a0a9c64b645c1df15868da7 --port 49399"
time=2026-02-26T12:19:53.374+08:00 level=INFO source=sched.go:491 msg="system memory" total="127.8 GiB" free="116.5 GiB" free_swap="122.7 GiB"
time=2026-02-26T12:19:53.374+08:00 level=INFO source=sched.go:498 msg="gpu memory" id=GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 library=CUDA available="22.2 GiB" free="22.6 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-02-26T12:19:53.374+08:00 level=INFO source=server.go:757 msg="loading model" "model layers"=41 requested=-1
time=2026-02-26T12:19:53.396+08:00 level=INFO source=runner.go:1411 msg="starting ollama engine"
time=2026-02-26T12:19:53.399+08:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:49399"
time=2026-02-26T12:19:53.406+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:41[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:41(0..40)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T12:19:53.435+08:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35moe file_type=Q4_K_M name="" description="" num_tensors=1959 num_key_values=57
load_backend: loaded CPU backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, ID: GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88
load_backend: loaded CUDA backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll
time=2026-02-26T12:19:57.863+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2026-02-26T12:19:58.366+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T12:19:58.655+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:482 msg="offloading 39 repeating layers to GPU"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:486 msg="offloading output layer to CPU"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:494 msg="offloaded 39/41 layers to GPU"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="19.8 GiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="2.4 GiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="1.6 GiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:256 msg="kv cache" device=CPU size="52.3 MiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="625.0 MiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="630.8 MiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:272 msg="total memory" size="25.1 GiB"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=sched.go:566 msg="loaded runners" count=1
time=2026-02-26T12:19:59.311+08:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-02-26T12:19:59.311+08:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-02-26T12:20:05.319+08:00 level=INFO source=server.go:1388 msg="llama runner started in 11.95 seconds"
CUDA error: invalid argument
  current device: 0, in function ggml_cuda_cpy at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\cpy.cu:438
  cudaMemcpyAsyncReserve(src1_ddc, src0_ddc, ggml_nbytes(src0), cudaMemcpyDeviceToDevice, main_stream)
C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:94: CUDA error
time=2026-02-26T12:20:05.645+08:00 level=ERROR source=server.go:1610 msg="post predict" error="Post \"http://127.0.0.1:49399/completion\": read tcp 127.0.0.1:49403->127.0.0.1:49399: wsarecv: An existing connection was forcibly closed by the remote host."
[GIN] 2026/02/26 - 12:20:05 | 500 |   15.4406179s |       127.0.0.1 | POST     "/api/chat"
time=2026-02-26T12:20:05.749+08:00 level=ERROR source=server.go:304 msg="llama runner terminated" error="exit status 1"
[GIN] 2026/02/26 - 12:20:07 | 200 |       2.354ms |       127.0.0.1 | GET      "/api/tags"

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.17.1-rc2

Originally created by @houzhuanghao on GitHub (Feb 26, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/14433 ### What is the issue? unsing RTX4090 24GB, 128G RAM qwen3.5-27B_q4_K_M was success run in 0.17.1-rc2 ; but when pull the newly version of qwen3.5-35B and qwen3.5:122b-a10b, the ollama failed. Error: 500 Internal Server Error: model runner has unexpectedly stopped, this may be due to resource limitations or an internal error, check ollama server logs for details **instresting imformations: cmd run: ollama run qwen3.5:122b-a10b (or qwen3.5:35b), when input the question words<29 chinese characters, it runs. when input question words>30 chinese characters, it failed.** ### Relevant log output ```shell [GIN] 2026/02/26 - 12:19:37 | 200 | 0s | 127.0.0.1 | GET "/api/version" [GIN] 2026/02/26 - 12:19:37 | 200 | 2.191ms | 127.0.0.1 | GET "/api/tags" [GIN] 2026/02/26 - 12:19:38 | 401 | 402.2283ms | 127.0.0.1 | POST "/api/me" [GIN] 2026/02/26 - 12:19:50 | 200 | 1.5592533s | 127.0.0.1 | POST "/api/show" [GIN] 2026/02/26 - 12:19:50 | 200 | 105.5966ms | 127.0.0.1 | POST "/api/show" time=2026-02-26T12:19:50.297+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 49392" time=2026-02-26T12:19:53.295+08:00 level=INFO source=runner.go:464 msg="failure during GPU discovery" OLLAMA_LIBRARY_PATH="[C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\lib\\ollama C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cuda_v12]" extra_envs=map[] error="failed to finish discovery before timeout" time=2026-02-26T12:19:53.296+08:00 level=WARN source=runner.go:356 msg="unable to refresh free memory, using old values" time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:148 msg=packages count=1 time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:164 msg="efficiency cores detected" maxEfficiencyClass=1 time=2026-02-26T12:19:53.296+08:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=24 efficiency=16 threads=32 time=2026-02-26T12:19:53.371+08:00 level=INFO source=server.go:247 msg="enabling flash attention" time=2026-02-26T12:19:53.372+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model G:\\models\\blobs\\sha256-2abd0d805943fa113f934d1ae4f2d5a749b5d4fe2a0a9c64b645c1df15868da7 --port 49399" time=2026-02-26T12:19:53.374+08:00 level=INFO source=sched.go:491 msg="system memory" total="127.8 GiB" free="116.5 GiB" free_swap="122.7 GiB" time=2026-02-26T12:19:53.374+08:00 level=INFO source=sched.go:498 msg="gpu memory" id=GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 library=CUDA available="22.2 GiB" free="22.6 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-02-26T12:19:53.374+08:00 level=INFO source=server.go:757 msg="loading model" "model layers"=41 requested=-1 time=2026-02-26T12:19:53.396+08:00 level=INFO source=runner.go:1411 msg="starting ollama engine" time=2026-02-26T12:19:53.399+08:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:49399" time=2026-02-26T12:19:53.406+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:41[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:41(0..40)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T12:19:53.435+08:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35moe file_type=Q4_K_M name="" description="" num_tensors=1959 num_key_values=57 load_backend: loaded CPU backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, ID: GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 load_backend: loaded CUDA backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll time=2026-02-26T12:19:57.863+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2026-02-26T12:19:58.366+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T12:19:58.655+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T12:19:59.311+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:482 msg="offloading 39 repeating layers to GPU" time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:486 msg="offloading output layer to CPU" time=2026-02-26T12:19:59.311+08:00 level=INFO source=ggml.go:494 msg="offloaded 39/41 layers to GPU" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="19.8 GiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="2.4 GiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="1.6 GiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:256 msg="kv cache" device=CPU size="52.3 MiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="625.0 MiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="630.8 MiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=device.go:272 msg="total memory" size="25.1 GiB" time=2026-02-26T12:19:59.311+08:00 level=INFO source=sched.go:566 msg="loaded runners" count=1 time=2026-02-26T12:19:59.311+08:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-02-26T12:19:59.311+08:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-02-26T12:20:05.319+08:00 level=INFO source=server.go:1388 msg="llama runner started in 11.95 seconds" CUDA error: invalid argument current device: 0, in function ggml_cuda_cpy at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\cpy.cu:438 cudaMemcpyAsyncReserve(src1_ddc, src0_ddc, ggml_nbytes(src0), cudaMemcpyDeviceToDevice, main_stream) C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:94: CUDA error time=2026-02-26T12:20:05.645+08:00 level=ERROR source=server.go:1610 msg="post predict" error="Post \"http://127.0.0.1:49399/completion\": read tcp 127.0.0.1:49403->127.0.0.1:49399: wsarecv: An existing connection was forcibly closed by the remote host." [GIN] 2026/02/26 - 12:20:05 | 500 | 15.4406179s | 127.0.0.1 | POST "/api/chat" time=2026-02-26T12:20:05.749+08:00 level=ERROR source=server.go:304 msg="llama runner terminated" error="exit status 1" [GIN] 2026/02/26 - 12:20:07 | 200 | 2.354ms | 127.0.0.1 | GET "/api/tags" ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.17.1-rc2
GiteaMirror added the bug label 2026-04-22 19:24:29 -05:00
GiteaMirror commented 2026-04-22 19:24:31 -05:00
Author
Owner
Copy Link

@JortVlaming commented on GitHub (Feb 26, 2026):

Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory.

Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further

<!-- gh-comment-id:3965678439 --> @JortVlaming commented on GitHub (Feb 26, 2026): Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory. Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further
GiteaMirror commented 2026-04-22 19:24:32 -05:00
Author
Owner
Copy Link

@gemlincong-dotcom commented on GitHub (Feb 26, 2026):

请问下你这个27B Q4KM的在哪下载的呢，unsloth的我下载了运行不了啊, 提示如下：
Error: 500 Internal Server Error: unable to load model: C:\Users\Waver.ollama\models\blobs\sha256-728960e4dda52d4f2af5bee09b2cbe86addfa93220fe9324bfac9dc727605c17

<!-- gh-comment-id:3965754967 --> @gemlincong-dotcom commented on GitHub (Feb 26, 2026): 请问下你这个27B Q4KM的在哪下载的呢，unsloth的我下载了运行不了啊, 提示如下： Error: 500 Internal Server Error: unable to load model: C:\Users\Waver\.ollama\models\blobs\sha256-728960e4dda52d4f2af5bee09b2cbe86addfa93220fe9324bfac9dc727605c17
GiteaMirror commented 2026-04-22 19:24:32 -05:00
Author
Owner
Copy Link

@houzhuanghao commented on GitHub (Feb 26, 2026):

请问下你这个27B Q4KM的在哪下载的呢，unsloth的我下载了运行不了啊, 提示如下： Error: 500 Internal Server Error: unable to load model: C:\Users\Waver.ollama\models\blobs\sha256-728960e4dda52d4f2af5bee09b2cbe86addfa93220fe9324bfac9dc727605c17

ollama官网就有q4km的版本。https://ollama.com/library/qwen3.5:27b-q4_K_M
unsloth提供的q4km的我也安装过，运行不了。

<!-- gh-comment-id:3966014580 --> @houzhuanghao commented on GitHub (Feb 26, 2026): > 请问下你这个27B Q4KM的在哪下载的呢，unsloth的我下载了运行不了啊, 提示如下： Error: 500 Internal Server Error: unable to load model: C:\Users\Waver.ollama\models\blobs\sha256-728960e4dda52d4f2af5bee09b2cbe86addfa93220fe9324bfac9dc727605c17 ollama官网就有q4km的版本。https://ollama.com/library/qwen3.5:27b-q4_K_M unsloth提供的q4km的我也安装过，运行不了。
GiteaMirror commented 2026-04-22 19:24:33 -05:00
Author
Owner
Copy Link

@houzhuanghao commented on GitHub (Feb 26, 2026):

Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory.

Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further

How to offload layer using ollama? I just used the command "ollama run +model".
I tryed different version:
the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b.
the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b.

<!-- gh-comment-id:3966037598 --> @houzhuanghao commented on GitHub (Feb 26, 2026): > Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory. > > Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further How to offload layer using ollama? I just used the command "ollama run +model". I tryed different version: the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b. the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b.
GiteaMirror commented 2026-04-22 19:24:35 -05:00
Author
Owner
Copy Link

@JortVlaming commented on GitHub (Feb 26, 2026):

Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory.
Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further

How to offload layer using ollama? I just used the command "ollama run +model". I tryed different version: the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b. the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b.

If you want it for just current session:

set OLLAMA_NUM_GPU_LAYERS=35
ollama run modelname

If you want it permanent:
Open Windows Start
Search Environment Variables
Add a new user variable:
Name: OLLAMA_NUM_GPU_LAYERS
Value: 35
Restart terminal after.

<!-- gh-comment-id:3966371795 --> @JortVlaming commented on GitHub (Feb 26, 2026): > > Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory. > > Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further > > How to offload layer using ollama? I just used the command "ollama run +model". I tryed different version: the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b. the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b. If you want it for just current session: ```ps1 set OLLAMA_NUM_GPU_LAYERS=35 ollama run modelname ``` If you want it permanent: Open Windows Start Search Environment Variables Add a new user variable: Name: OLLAMA_NUM_GPU_LAYERS Value: 35 Restart terminal after.
GiteaMirror commented 2026-04-22 19:24:37 -05:00
Author
Owner
Copy Link

@houzhuanghao commented on GitHub (Feb 26, 2026):

Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory.
Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further

How to offload layer using ollama? I just used the command "ollama run +model". I tryed different version: the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b. the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b.

If you want it for just current session:

set OLLAMA_NUM_GPU_LAYERS=35
ollama run modelname
If you want it permanent: Open Windows Start Search Environment Variables Add a new user variable: Name: OLLAMA_NUM_GPU_LAYERS Value: 35 Restart terminal after.

I try set OLLAMA_NUM_GPU_LAYERS=35 or set OLLAMA_NUM_GPU_LAYERS=20，the qwen3.5-35b remain failed.

<!-- gh-comment-id:3966403651 --> @houzhuanghao commented on GitHub (Feb 26, 2026): > > > Im not very experienced with this but have you tried decreasing the amount of layers you offload to GPU? The crash is occuring due to your GPU running out of memory. > > > Try decreasing layer offload to 34-35 (which is still pretty close to the edge) and if that doesnt work try lowering it even further > > > > > > How to offload layer using ollama? I just used the command "ollama run +model". I tryed different version: the 0.17.1-rc1 can run qwen3.5-35b，but fail in run qwen3.5-27b. the 0.17.1-rc2 can run qwen3.5-27b，but fail in run qwen3.5-35b. > > If you want it for just current session: > > set OLLAMA_NUM_GPU_LAYERS=35 > ollama run modelname > If you want it permanent: Open Windows Start Search Environment Variables Add a new user variable: Name: OLLAMA_NUM_GPU_LAYERS Value: 35 Restart terminal after. I try set OLLAMA_NUM_GPU_LAYERS=35 or set OLLAMA_NUM_GPU_LAYERS=20，the qwen3.5-35b remain failed.
GiteaMirror commented 2026-04-22 19:24:39 -05:00
Author
Owner
Copy Link

@JortVlaming commented on GitHub (Feb 26, 2026):

logs?

<!-- gh-comment-id:3966409239 --> @JortVlaming commented on GitHub (Feb 26, 2026): logs?
GiteaMirror commented 2026-04-22 19:24:40 -05:00
Author
Owner
Copy Link

@houzhuanghao commented on GitHub (Feb 26, 2026):

logs?
set OLLAMA_NUM_GPU_LAYERS=20
ollama run qwen3.5:35b

[GIN] 2026/02/26 - 20:47:22 | 200 | 2.0355ms | 127.0.0.1 | GET "/api/tags"
[GIN] 2026/02/26 - 20:47:42 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2026/02/26 - 20:47:42 | 200 | 238.5673ms | 127.0.0.1 | POST "/api/show"
[GIN] 2026/02/26 - 20:47:42 | 200 | 208.1896ms | 127.0.0.1 | POST "/api/show"
time=2026-02-26T20:47:42.868+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\Users\ms327\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --port 61267"
time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:148 msg=packages count=1
time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:164 msg="efficiency cores detected" maxEfficiencyClass=1
time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=24 efficiency=16 threads=32
time=2026-02-26T20:47:43.291+08:00 level=INFO source=server.go:247 msg="enabling flash attention"
time=2026-02-26T20:47:43.292+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\Users\ms327\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --model G:\models\blobs\sha256-2abd0d805943fa113f934d1ae4f2d5a749b5d4fe2a0a9c64b645c1df15868da7 --port 61273"
time=2026-02-26T20:47:43.296+08:00 level=INFO source=sched.go:491 msg="system memory" total="127.8 GiB" free="115.6 GiB" free_swap="122.1 GiB"
time=2026-02-26T20:47:43.296+08:00 level=INFO source=sched.go:498 msg="gpu memory" id=GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 library=CUDA available="22.4 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-02-26T20:47:43.296+08:00 level=INFO source=server.go:757 msg="loading model" "model layers"=41 requested=-1
time=2026-02-26T20:47:43.330+08:00 level=INFO source=runner.go:1411 msg="starting ollama engine"
time=2026-02-26T20:47:43.341+08:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:61273"
time=2026-02-26T20:47:43.348+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:41[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:41(0..40)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T20:47:43.423+08:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35moe file_type=Q4_K_M name="" description="" num_tensors=1959 num_key_values=57
load_backend: loaded CPU backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, ID: GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88
load_backend: loaded CUDA backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll
time=2026-02-26T20:47:43.562+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2026-02-26T20:47:44.331+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T20:47:44.872+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T20:47:45.604+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:482 msg="offloading 39 repeating layers to GPU"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:486 msg="offloading output layer to CPU"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:494 msg="offloaded 39/41 layers to GPU"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="19.8 GiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="2.4 GiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="1.6 GiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:256 msg="kv cache" device=CPU size="52.3 MiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="625.0 MiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="630.8 MiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:272 msg="total memory" size="25.1 GiB"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=sched.go:566 msg="loaded runners" count=1
time=2026-02-26T20:47:45.605+08:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-02-26T20:47:45.605+08:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-02-26T20:47:51.723+08:00 level=INFO source=server.go:1388 msg="llama runner started in 8.43 seconds"
[GIN] 2026/02/26 - 20:47:51 | 200 | 9.1339599s | 127.0.0.1 | POST "/api/generate"
[GIN] 2026/02/26 - 20:47:53 | 200 | 1.1502ms | 127.0.0.1 | GET "/api/tags"
CUDA error: invalid argument
current device: 0, in function ggml_cuda_cpy at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\cpy.cu:438
cudaMemcpyAsyncReserve(src1_ddc, src0_ddc, ggml_nbytes(src0), cudaMemcpyDeviceToDevice, main_stream)
C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:94: CUDA error
time=2026-02-26T20:47:59.042+08:00 level=ERROR source=server.go:1610 msg="post predict" error="Post "http://127.0.0.1:61273/completion": read tcp 127.0.0.1:61277->127.0.0.1:61273: wsarecv: An existing connection was forcibly closed by the remote host."
[GIN] 2026/02/26 - 20:47:59 | 500 | 545.3312ms | 127.0.0.1 | POST "/api/chat"
time=2026-02-26T20:47:59.152+08:00 level=ERROR source=server.go:304 msg="llama runner terminated" error="exit status 1"

<!-- gh-comment-id:3966421778 --> @houzhuanghao commented on GitHub (Feb 26, 2026): > logs? set OLLAMA_NUM_GPU_LAYERS=20 ollama run qwen3.5:35b [GIN] 2026/02/26 - 20:47:22 | 200 | 2.0355ms | 127.0.0.1 | GET "/api/tags" [GIN] 2026/02/26 - 20:47:42 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2026/02/26 - 20:47:42 | 200 | 238.5673ms | 127.0.0.1 | POST "/api/show" [GIN] 2026/02/26 - 20:47:42 | 200 | 208.1896ms | 127.0.0.1 | POST "/api/show" time=2026-02-26T20:47:42.868+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 61267" time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:148 msg=packages count=1 time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:164 msg="efficiency cores detected" maxEfficiencyClass=1 time=2026-02-26T20:47:43.133+08:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=24 efficiency=16 threads=32 time=2026-02-26T20:47:43.291+08:00 level=INFO source=server.go:247 msg="enabling flash attention" time=2026-02-26T20:47:43.292+08:00 level=INFO source=server.go:431 msg="starting runner" cmd="C:\\Users\\ms327\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model G:\\models\\blobs\\sha256-2abd0d805943fa113f934d1ae4f2d5a749b5d4fe2a0a9c64b645c1df15868da7 --port 61273" time=2026-02-26T20:47:43.296+08:00 level=INFO source=sched.go:491 msg="system memory" total="127.8 GiB" free="115.6 GiB" free_swap="122.1 GiB" time=2026-02-26T20:47:43.296+08:00 level=INFO source=sched.go:498 msg="gpu memory" id=GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 library=CUDA available="22.4 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-02-26T20:47:43.296+08:00 level=INFO source=server.go:757 msg="loading model" "model layers"=41 requested=-1 time=2026-02-26T20:47:43.330+08:00 level=INFO source=runner.go:1411 msg="starting ollama engine" time=2026-02-26T20:47:43.341+08:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:61273" time=2026-02-26T20:47:43.348+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:41[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:41(0..40)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T20:47:43.423+08:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35moe file_type=Q4_K_M name="" description="" num_tensors=1959 num_key_values=57 load_backend: loaded CPU backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-alderlake.dll ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 4090, compute capability 8.9, VMM: yes, ID: GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 load_backend: loaded CUDA backend from C:\Users\ms327\AppData\Local\Programs\Ollama\lib\ollama\cuda_v12\ggml-cuda.dll time=2026-02-26T20:47:43.562+08:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX_VNNI=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,520,600,610,700,750,800,860,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2026-02-26T20:47:44.331+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T20:47:44.872+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T20:47:45.604+08:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:8192 KvCacheType: NumThreads:8 GPULayers:39[ID:GPU-06cffc9e-715e-6099-d5d9-fce41b2a7c88 Layers:39(1..39)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:482 msg="offloading 39 repeating layers to GPU" time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:486 msg="offloading output layer to CPU" time=2026-02-26T20:47:45.605+08:00 level=INFO source=ggml.go:494 msg="offloaded 39/41 layers to GPU" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="19.8 GiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="2.4 GiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="1.6 GiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:256 msg="kv cache" device=CPU size="52.3 MiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="625.0 MiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="630.8 MiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=device.go:272 msg="total memory" size="25.1 GiB" time=2026-02-26T20:47:45.605+08:00 level=INFO source=sched.go:566 msg="loaded runners" count=1 time=2026-02-26T20:47:45.605+08:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-02-26T20:47:45.605+08:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-02-26T20:47:51.723+08:00 level=INFO source=server.go:1388 msg="llama runner started in 8.43 seconds" [GIN] 2026/02/26 - 20:47:51 | 200 | 9.1339599s | 127.0.0.1 | POST "/api/generate" [GIN] 2026/02/26 - 20:47:53 | 200 | 1.1502ms | 127.0.0.1 | GET "/api/tags" CUDA error: invalid argument current device: 0, in function ggml_cuda_cpy at C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\cpy.cu:438 cudaMemcpyAsyncReserve(src1_ddc, src0_ddc, ggml_nbytes(src0), cudaMemcpyDeviceToDevice, main_stream) C:\a\ollama\ollama\ml\backend\ggml\ggml\src\ggml-cuda\ggml-cuda.cu:94: CUDA error time=2026-02-26T20:47:59.042+08:00 level=ERROR source=server.go:1610 msg="post predict" error="Post \"http://127.0.0.1:61273/completion\": read tcp 127.0.0.1:61277->127.0.0.1:61273: wsarecv: An existing connection was forcibly closed by the remote host." [GIN] 2026/02/26 - 20:47:59 | 500 | 545.3312ms | 127.0.0.1 | POST "/api/chat" time=2026-02-26T20:47:59.152+08:00 level=ERROR source=server.go:304 msg="llama runner terminated" error="exit status 1"
GiteaMirror commented 2026-04-22 19:24:41 -05:00
Author
Owner
Copy Link

@JortVlaming commented on GitHub (Feb 26, 2026):

Looks like the same error. Unfortunately I dont know another solution so youll have to wait for someone more qualified to respond

<!-- gh-comment-id:3966443376 --> @JortVlaming commented on GitHub (Feb 26, 2026): Looks like the same error. Unfortunately I dont know another solution so youll have to wait for someone more qualified to respond
GiteaMirror commented 2026-04-22 19:24:41 -05:00
Author
Owner
Copy Link

@kliom commented on GitHub (Mar 8, 2026):

My device has 11GB of graphics memory and 16GB of RAM. When the context length exceeds 64KB, this error occurs. You might also consider reducing the context length.

<!-- gh-comment-id:4018283139 --> @kliom commented on GitHub (Mar 8, 2026): My device has 11GB of graphics memory and 16GB of RAM. When the context length exceeds 64KB, this error occurs. You might also consider reducing the context length.
GiteaMirror commented 2026-04-22 19:24:42 -05:00
Author
Owner
Copy Link

@LukaszRT commented on GitHub (Mar 24, 2026):

Same BUG
time=2026-03-24T19:35:20.424+01:00 level=INFO source=server.go:1568 msg="aborting completion request due to client closing the connection"
PS C:\Windows\system32> $env:OLLAMA_CONTEXT_LENGTH="16384"
PS C:\Windows\system32> $env:OLLAMA_GPU_LAYERS="999"
PS C:\Windows\system32> $env:OLLAMA_MAX_QUEUE="2"
PS C:\Windows\system32> $env:OLLAMA_HOST="0.0.0.0:11434"
PS C:\Windows\system32> $env:OLLAMA_INTEL_GPU="0"
PS C:\Windows\system32> $env:OLLAMA_LLM_LIBRARY="cuda_v13"
PS C:\Windows\system32> $env:OLLAMA_MAX_LOADED_MODELS="1"
PS C:\Windows\system32> $env:OLLAMA_MAX_VRAM="24GB"
PS C:\Windows\system32> $env:OLLAMA_NUM_GPU="1"
PS C:\Windows\system32> $env:OLLAMA_NUM_PARALLEL="1"
PS C:\Windows\system32> $env:OLLAMA_NUM_THREADS="4"
PS C:\Windows\system32> $env:OLLAMA_GPU_OVERHEAD="0"
PS C:\Windows\system32>
PS C:\Windows\system32> ollama serve
time=2026-03-24T19:37:32.115+01:00 level=INFO source=routes.go:1727 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:16384 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY:cuda_v13 OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:2 OLLAMA_MODELS:C:\Users\LukaszRT\.ollama\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES:]"
time=2026-03-24T19:37:32.142+01:00 level=INFO source=routes.go:1729 msg="Ollama cloud disabled: true"
time=2026-03-24T19:37:32.145+01:00 level=INFO source=images.go:477 msg="total blobs: 4"
time=2026-03-24T19:37:32.145+01:00 level=INFO source=images.go:484 msg="total unused blobs removed: 0"
time=2026-03-24T19:37:32.145+01:00 level=INFO source=routes.go:1782 msg="Listening on [::]:11434 (version 0.18.2)"
time=2026-03-24T19:37:32.150+01:00 level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-03-24T19:37:32.172+01:00 level=INFO source=server.go:430 msg="starting runner"
#########
here ist the hanging Problem after STRG+C:
#########
cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --port 64238"
time=2026-03-24T19:37:32.543+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --port 64260"
time=2026-03-24T19:37:32.870+01:00 level=INFO source=types.go:42 msg="inference compute" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA GeForce RTX 3090" libdirs=ollama,cuda_v13 driver=13.2 pci_id=0000:01:00.0 type=discrete total="24.0 GiB" available="22.8 GiB"
time=2026-03-24T19:37:32.870+01:00 level=INFO source=routes.go:1832 msg="vram-based default context" total_vram="24.0 GiB" default_num_ctx=32768
time=2026-03-24T19:41:05.049+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --port 64291"
[GIN] 2026/03/24 - 19:50:26 | 503 | 291.9794ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses"
[GIN] 2026/03/24 - 19:55:20 | 503 | 254.2572ms | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 19:55:27 | 503 | 261.2948ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses"
[GIN] 2026/03/24 - 19:58:11 | 503 | 305.5755ms | 127.0.0.1 | POST "/api/generate"
[GIN] 2026/03/24 - 19:59:38 | 200 | 2.7791ms | 127.0.0.1 | GET "/api/tags"
[GIN] 2026/03/24 - 20:00:22 | 503 | 276.0651ms | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:00:28 | 503 | 272.7774ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses"
[GIN] 2026/03/24 - 20:00:30 | 200 | 549.7µs | 127.0.0.1 | HEAD "/"
time=2026-03-24T20:01:14.243+01:00 level=INFO source=runner.go:464 msg="failure during GPU discovery" OLLAMA_LIBRARY_PATH="[C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13]" extra_envs=map[] error="failed to finish discovery before timeout"
time=2026-03-24T20:01:14.244+01:00 level=WARN source=runner.go:356 msg="unable to refresh free memory, using old values"
time=2026-03-24T20:01:14.245+01:00 level=INFO source=cpu_windows.go:148 msg=packages count=1
time=2026-03-24T20:01:14.245+01:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=4 efficiency=0 threads=8
time=2026-03-24T20:01:14.451+01:00 level=INFO source=server.go:246 msg="enabling flash attention"
time=2026-03-24T20:01:14.457+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --model C:\Users\LukaszRT\.ollama\models\blobs\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 54864"
time=2026-03-24T20:01:14.461+01:00 level=INFO source=sched.go:484 msg="system memory" total="15.9 GiB" free="11.9 GiB" free_swap="35.4 GiB"
time=2026-03-24T20:01:14.461+01:00 level=INFO source=sched.go:491 msg="gpu memory" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 library=CUDA available="22.4 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-03-24T20:01:14.461+01:00 level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1
time=2026-03-24T20:01:14.462+01:00 level=ERROR source=server.go:1205 msg="do load request" error="Post "http://127.0.0.1:54864/load": context canceled"
time=2026-03-24T20:01:14.462+01:00 level=INFO source=sched.go:511 msg="Load failed" model=C:\Users\LukaszRT.ollama\models\blobs\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 error="context canceled"
[GIN] 2026/03/24 - 20:01:14 | 499 | 20m9s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
time=2026-03-24T20:01:16.551+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --port 54865"
time=2026-03-24T20:01:16.928+01:00 level=INFO source=cpu_windows.go:148 msg=packages count=1
time=2026-03-24T20:01:16.928+01:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=4 efficiency=0 threads=8
time=2026-03-24T20:01:17.119+01:00 level=INFO source=server.go:246 msg="enabling flash attention"
time=2026-03-24T20:01:17.119+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\Users\LukaszRT\AppData\Local\Programs\Ollama\ollama.exe runner --ollama-engine --model C:\Users\LukaszRT\.ollama\models\blobs\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 54891"
time=2026-03-24T20:01:17.121+01:00 level=INFO source=sched.go:484 msg="system memory" total="15.9 GiB" free="11.6 GiB" free_swap="33.9 GiB"
time=2026-03-24T20:01:17.121+01:00 level=INFO source=sched.go:491 msg="gpu memory" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 library=CUDA available="22.3 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B"
time=2026-03-24T20:01:17.121+01:00 level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1
time=2026-03-24T20:01:17.314+01:00 level=INFO source=runner.go:1411 msg="starting ollama engine"
time=2026-03-24T20:01:17.342+01:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:54891"
time=2026-03-24T20:01:17.344+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-24T20:01:17.413+01:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53
load_backend: loaded CPU backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-sandybridge.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, ID: GPU-c1adeb78-275d-685f-cafa-6946de43bfb7
load_backend: loaded CUDA backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll
time=2026-03-24T20:01:17.512+01:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=750,800,860,870,890,900,1000,1030,1100,1200,1210 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2026-03-24T20:01:18.831+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:489 msg="offloading output layer to GPU"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="15.5 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="710.2 MiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="4.7 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="1.1 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="168.0 MiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:272 msg="total memory" size="22.2 GiB"
time=2026-03-24T20:01:20.409+01:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm

Same problem. The runner hangs; only after pressing Ctrl+C does the log file, which is then displayed full in 1second, suddenly fill up. As you can see, Ollama falls back on CPU resources. Cause: Runner hangs.

The problem is a hung runner and a stuck process. From this point on, the bug occurs where the protocol hangs and, invisibly in the background, opens a CPU instance. This results in a CPU fallback. Bug: Hang runner, then protocol hang!

Win11 RTX3090TI

This is what's happening to you right now (the bug sequence):

The blind flight: Ollama is trying to "discover" (detect) the GPU. Due to your PCIe 3.0 connection or a driver hang (HAGS!), the GPU isn't responding in time.

The protocol jam: The runner starts in the background (invisibly), but the standard output stream (the protocol you see) is blocked by the system. Windows is holding the text output in RAM because the process is stuck in "I/O wait."

The "dam break" (Ctrl+C): When you press Ctrl+C, Windows sends an interrupt signal. This briefly breaks the kernel's blocking state. At this moment, the system "spits out" the entire accumulated protocol in one millisecond.

The CPU fallback: Because the GPU discovery process timed out, Ollama says: "Okay, the GPU is probably broken or not responding, I'll use the CPU now." That's why your processor is overheating, even though you have a 3090 Ti in your computer.

bug in the communication between the Ollama server and its runner.

What you're seeing (the "dam break" after Ctrl+C) isn't a hardware defect, but a failure of inter-process communication (IPC) under Windows.

Why it's a software bug:

The deadlock in the pipe buffer: Ollama (the server) starts the runner (ollama_llama_server.exe) and opens a "pipe" (a channel) to read the log data. If the runner encounters a problem (e.g., a brief stutter in the graphics driver), it fills the buffer of this pipe with error messages or status updates. If the server doesn't "pull" this data quickly enough, the runner completely freezes because it can't get rid of its logs.

The deadlock in the pipe buffer: Ollama (the server) starts the runner (ollama_llama_server.exe) and opens a "pipe" (a channel) to read the log data. The Ctrl+C effect: By pressing Ctrl+C, you send a signal to the process tree. This breaks the pipe's blockage state. Only then does the accumulated data flow through in one go. This is a classic sign of poor buffering in the software.

The incorrect fallback: Instead of cleanly reporting the error ("GPU not responding in time"), the runner chokes. The server then thinks: "Oh, the runner isn't responding on port 54891, so I'll just try again—this time on the CPU, just to be safe." This is a poor error-handling logic in Ollama's code.

so what happen after STRGC

time=2026-03-24T20:01:17.413+01:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53
load_backend: loaded CPU backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-sandybridge.dll
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, ID: GPU-c1adeb78-275d-685f-cafa-6946de43bfb7
load_backend: loaded CUDA backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll
time=2026-03-24T20:01:17.512+01:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=750,800,860,870,890,900,1000,1030,1100,1200,1210 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang)
time=2026-03-24T20:01:18.831+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:489 msg="offloading output layer to GPU"
time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="15.5 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="710.2 MiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="4.7 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="1.1 GiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="168.0 MiB"
time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:272 msg="total memory" size="22.2 GiB"
time=2026-03-24T20:01:20.409+01:00 level=INFO source=sched.go:561 msg="loaded runners" count=1
time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model"
time=2026-03-24T20:01:52.956+01:00 level=INFO source=server.go:1388 msg="llama runner started in 35.84 seconds"
[GIN] 2026/03/24 - 20:02:06 | 200 | 50.6421895s | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:16 | 200 | 1m0s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:22 | 200 | 3.9144743s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:27 | 200 | 3.9046539s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:34 | 200 | 5.0256313s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:41 | 200 | 4.2370656s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:49 | 200 | 6.1240108s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:02:54 | 200 | 2.1847587s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:10:28 | 200 | 9.9641267s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:10:30 | 200 | 2.3000894s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:17:27 | 200 | 15.3798629s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:17:34 | 200 | 7.5285074s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:17:39 | 200 | 4.1610616s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:17:47 | 200 | 6.8888154s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:17:56 | 200 | 8.3210607s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:18:03 | 200 | 7.0737789s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:18:11 | 200 | 6.2583758s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:18:18 | 200 | 6.7728291s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:18:26 | 200 | 7.1462118s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"
[GIN] 2026/03/24 - 20:18:54 | 200 | 27.8891987s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses"

This clearly shows that it is an ollama bug.

Debugging Steps for Developers

Provide them with this list of steps to reproduce or further analyze the bug:

Pipe Buffer Monitoring: They should check if the runner's stdout/stderr pipe buffer overflows under Windows (Win32 pipes) when GPU discovery takes longer than the DiscoveryTimeout.

Race Condition Check: Analyze the startup sequencer in gpu/gpu_windows.go. If discovery times out, the runner thread appears to enter an infinite wait state instead of terminating cleanly.

Increase Verbosity: They should provide the command for a debug build:

Ideally, the process should be terminated after being queried, and only then should a clean retry be performed. However, the server is starting a new process that then uses CPU resources and is also invisible in the log.

<!-- gh-comment-id:4120730478 --> @LukaszRT commented on GitHub (Mar 24, 2026): Same BUG time=2026-03-24T19:35:20.424+01:00 level=INFO source=server.go:1568 msg="aborting completion request due to client closing the connection" PS C:\Windows\system32> $env:OLLAMA_CONTEXT_LENGTH="16384" PS C:\Windows\system32> $env:OLLAMA_GPU_LAYERS="999" PS C:\Windows\system32> $env:OLLAMA_MAX_QUEUE="2" PS C:\Windows\system32> $env:OLLAMA_HOST="0.0.0.0:11434" PS C:\Windows\system32> $env:OLLAMA_INTEL_GPU="0" PS C:\Windows\system32> $env:OLLAMA_LLM_LIBRARY="cuda_v13" PS C:\Windows\system32> $env:OLLAMA_MAX_LOADED_MODELS="1" PS C:\Windows\system32> $env:OLLAMA_MAX_VRAM="24GB" PS C:\Windows\system32> $env:OLLAMA_NUM_GPU="1" PS C:\Windows\system32> $env:OLLAMA_NUM_PARALLEL="1" PS C:\Windows\system32> $env:OLLAMA_NUM_THREADS="4" PS C:\Windows\system32> $env:OLLAMA_GPU_OVERHEAD="0" PS C:\Windows\system32> PS C:\Windows\system32> ollama serve time=2026-03-24T19:37:32.115+01:00 level=INFO source=routes.go:1727 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:16384 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_KEEP_ALIVE:2562047h47m16.854775807s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY:cuda_v13 OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:2 OLLAMA_MODELS:C:\\Users\\LukaszRT\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:true OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES:]" time=2026-03-24T19:37:32.142+01:00 level=INFO source=routes.go:1729 msg="Ollama cloud disabled: true" time=2026-03-24T19:37:32.145+01:00 level=INFO source=images.go:477 msg="total blobs: 4" time=2026-03-24T19:37:32.145+01:00 level=INFO source=images.go:484 msg="total unused blobs removed: 0" time=2026-03-24T19:37:32.145+01:00 level=INFO source=routes.go:1782 msg="Listening on [::]:11434 (version 0.18.2)" time=2026-03-24T19:37:32.150+01:00 level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-03-24T19:37:32.172+01:00 level=INFO source=server.go:430 msg="starting runner" ######### here ist the hanging Problem after STRG+C: ######### cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 64238" time=2026-03-24T19:37:32.543+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 64260" time=2026-03-24T19:37:32.870+01:00 level=INFO source=types.go:42 msg="inference compute" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 filter_id="" library=CUDA compute=8.6 name=CUDA0 description="NVIDIA GeForce RTX 3090" libdirs=ollama,cuda_v13 driver=13.2 pci_id=0000:01:00.0 type=discrete total="24.0 GiB" available="22.8 GiB" time=2026-03-24T19:37:32.870+01:00 level=INFO source=routes.go:1832 msg="vram-based default context" total_vram="24.0 GiB" default_num_ctx=32768 time=2026-03-24T19:41:05.049+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 64291" [GIN] 2026/03/24 - 19:50:26 | 503 | 291.9794ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses" [GIN] 2026/03/24 - 19:55:20 | 503 | 254.2572ms | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 19:55:27 | 503 | 261.2948ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses" [GIN] 2026/03/24 - 19:58:11 | 503 | 305.5755ms | 127.0.0.1 | POST "/api/generate" [GIN] 2026/03/24 - 19:59:38 | 200 | 2.7791ms | 127.0.0.1 | GET "/api/tags" [GIN] 2026/03/24 - 20:00:22 | 503 | 276.0651ms | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:00:28 | 503 | 272.7774ms | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses" [GIN] 2026/03/24 - 20:00:30 | 200 | 549.7µs | 127.0.0.1 | HEAD "/" time=2026-03-24T20:01:14.243+01:00 level=INFO source=runner.go:464 msg="failure during GPU discovery" OLLAMA_LIBRARY_PATH="[C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\lib\\ollama C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cuda_v13]" extra_envs=map[] error="failed to finish discovery before timeout" time=2026-03-24T20:01:14.244+01:00 level=WARN source=runner.go:356 msg="unable to refresh free memory, using old values" time=2026-03-24T20:01:14.245+01:00 level=INFO source=cpu_windows.go:148 msg=packages count=1 time=2026-03-24T20:01:14.245+01:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=4 efficiency=0 threads=8 time=2026-03-24T20:01:14.451+01:00 level=INFO source=server.go:246 msg="enabling flash attention" time=2026-03-24T20:01:14.457+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model C:\\Users\\LukaszRT\\.ollama\\models\\blobs\\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 54864" time=2026-03-24T20:01:14.461+01:00 level=INFO source=sched.go:484 msg="system memory" total="15.9 GiB" free="11.9 GiB" free_swap="35.4 GiB" time=2026-03-24T20:01:14.461+01:00 level=INFO source=sched.go:491 msg="gpu memory" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 library=CUDA available="22.4 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-03-24T20:01:14.461+01:00 level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1 time=2026-03-24T20:01:14.462+01:00 level=ERROR source=server.go:1205 msg="do load request" error="Post \"http://127.0.0.1:54864/load\": context canceled" time=2026-03-24T20:01:14.462+01:00 level=INFO source=sched.go:511 msg="Load failed" model=C:\Users\LukaszRT\.ollama\models\blobs\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 error="context canceled" [GIN] 2026/03/24 - 20:01:14 | 499 | 20m9s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" time=2026-03-24T20:01:16.551+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --port 54865" time=2026-03-24T20:01:16.928+01:00 level=INFO source=cpu_windows.go:148 msg=packages count=1 time=2026-03-24T20:01:16.928+01:00 level=INFO source=cpu_windows.go:195 msg="" package=0 cores=4 efficiency=0 threads=8 time=2026-03-24T20:01:17.119+01:00 level=INFO source=server.go:246 msg="enabling flash attention" time=2026-03-24T20:01:17.119+01:00 level=INFO source=server.go:430 msg="starting runner" cmd="C:\\Users\\LukaszRT\\AppData\\Local\\Programs\\Ollama\\ollama.exe runner --ollama-engine --model C:\\Users\\LukaszRT\\.ollama\\models\\blobs\\sha256-d4b8b4f4c350f5d322dc8235175eeae02d32c6f3fd70bdb9ea481e3abb7d7fc4 --port 54891" time=2026-03-24T20:01:17.121+01:00 level=INFO source=sched.go:484 msg="system memory" total="15.9 GiB" free="11.6 GiB" free_swap="33.9 GiB" time=2026-03-24T20:01:17.121+01:00 level=INFO source=sched.go:491 msg="gpu memory" id=GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 library=CUDA available="22.3 GiB" free="22.8 GiB" minimum="457.0 MiB" overhead="0 B" time=2026-03-24T20:01:17.121+01:00 level=INFO source=server.go:757 msg="loading model" "model layers"=65 requested=-1 time=2026-03-24T20:01:17.314+01:00 level=INFO source=runner.go:1411 msg="starting ollama engine" time=2026-03-24T20:01:17.342+01:00 level=INFO source=runner.go:1446 msg="Server listening on 127.0.0.1:54891" time=2026-03-24T20:01:17.344+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:fit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-24T20:01:17.413+01:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53 load_backend: loaded CPU backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-sandybridge.dll ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, ID: GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 load_backend: loaded CUDA backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll time=2026-03-24T20:01:17.512+01:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=750,800,860,870,890,900,1000,1030,1100,1200,1210 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2026-03-24T20:01:18.831+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-24T20:01:20.407+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:489 msg="offloading output layer to GPU" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="15.5 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="710.2 MiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="4.7 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="1.1 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="168.0 MiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:272 msg="total memory" size="22.2 GiB" time=2026-03-24T20:01:20.409+01:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm Same problem. The runner hangs; only after pressing Ctrl+C does the log file, which is then displayed full in 1second, suddenly fill up. As you can see, Ollama falls back on CPU resources. Cause: Runner hangs. The problem is a hung runner and a stuck process. From this point on, the bug occurs where the protocol hangs and, invisibly in the background, opens a CPU instance. This results in a CPU fallback. Bug: Hang runner, then protocol hang! Win11 RTX3090TI This is what's happening to you right now (the bug sequence): The blind flight: Ollama is trying to "discover" (detect) the GPU. Due to your PCIe 3.0 connection or a driver hang (HAGS!), the GPU isn't responding in time. The protocol jam: The runner starts in the background (invisibly), but the standard output stream (the protocol you see) is blocked by the system. Windows is holding the text output in RAM because the process is stuck in "I/O wait." The "dam break" (Ctrl+C): When you press Ctrl+C, Windows sends an interrupt signal. This briefly breaks the kernel's blocking state. At this moment, the system "spits out" the entire accumulated protocol in one millisecond. The CPU fallback: Because the GPU discovery process timed out, Ollama says: "Okay, the GPU is probably broken or not responding, I'll use the CPU now." That's why your processor is overheating, even though you have a 3090 Ti in your computer. bug in the communication between the Ollama server and its runner. What you're seeing (the "dam break" after Ctrl+C) isn't a hardware defect, but a failure of inter-process communication (IPC) under Windows. Why it's a software bug: The deadlock in the pipe buffer: Ollama (the server) starts the runner (ollama_llama_server.exe) and opens a "pipe" (a channel) to read the log data. If the runner encounters a problem (e.g., a brief stutter in the graphics driver), it fills the buffer of this pipe with error messages or status updates. If the server doesn't "pull" this data quickly enough, the runner completely freezes because it can't get rid of its logs. The deadlock in the pipe buffer: Ollama (the server) starts the runner (ollama_llama_server.exe) and opens a "pipe" (a channel) to read the log data. The Ctrl+C effect: By pressing Ctrl+C, you send a signal to the process tree. This breaks the pipe's blockage state. Only then does the accumulated data flow through in one go. This is a classic sign of poor buffering in the software. The incorrect fallback: Instead of cleanly reporting the error ("GPU not responding in time"), the runner chokes. The server then thinks: "Oh, the runner isn't responding on port 54891, so I'll just try again—this time on the CPU, just to be safe." This is a poor error-handling logic in Ollama's code. so what happen after STRGC time=2026-03-24T20:01:17.413+01:00 level=INFO source=ggml.go:136 msg="" architecture=qwen35 file_type=Q4_K_M name="" description="" num_tensors=1307 num_key_values=53 load_backend: loaded CPU backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\ggml-cpu-sandybridge.dll ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes, ID: GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 load_backend: loaded CUDA backend from C:\Users\LukaszRT\AppData\Local\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll time=2026-03-24T20:01:17.512+01:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=750,800,860,870,890,900,1000,1030,1100,1200,1210 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(clang) time=2026-03-24T20:01:18.831+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-24T20:01:20.407+01:00 level=INFO source=runner.go:1284 msg=load request="{Operation:commit LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:16384 KvCacheType: NumThreads:4 GPULayers:65[ID:GPU-c1adeb78-275d-685f-cafa-6946de43bfb7 Layers:65(0..64)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:482 msg="offloading 64 repeating layers to GPU" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:489 msg="offloading output layer to GPU" time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:240 msg="model weights" device=CUDA0 size="15.5 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:245 msg="model weights" device=CPU size="710.2 MiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:251 msg="kv cache" device=CUDA0 size="4.7 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:262 msg="compute graph" device=CUDA0 size="1.1 GiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:267 msg="compute graph" device=CPU size="168.0 MiB" time=2026-03-24T20:01:20.408+01:00 level=INFO source=device.go:272 msg="total memory" size="22.2 GiB" time=2026-03-24T20:01:20.409+01:00 level=INFO source=sched.go:561 msg="loaded runners" count=1 time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding" time=2026-03-24T20:01:20.409+01:00 level=INFO source=server.go:1384 msg="waiting for server to become available" status="llm server loading model" time=2026-03-24T20:01:52.956+01:00 level=INFO source=server.go:1388 msg="llama runner started in 35.84 seconds" [GIN] 2026/03/24 - 20:02:06 | 200 | 50.6421895s | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:16 | 200 | 1m0s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:22 | 200 | 3.9144743s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:27 | 200 | 3.9046539s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:34 | 200 | 5.0256313s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:41 | 200 | 4.2370656s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:49 | 200 | 6.1240108s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:02:54 | 200 | 2.1847587s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:10:28 | 200 | 9.9641267s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:10:30 | 200 | 2.3000894s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:17:27 | 200 | 15.3798629s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:17:34 | 200 | 7.5285074s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:17:39 | 200 | 4.1610616s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:17:47 | 200 | 6.8888154s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:17:56 | 200 | 8.3210607s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:18:03 | 200 | 7.0737789s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:18:11 | 200 | 6.2583758s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:18:18 | 200 | 6.7728291s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:18:26 | 200 | 7.1462118s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" [GIN] 2026/03/24 - 20:18:54 | 200 | 27.8891987s | fd47:c2ae:c976:0:1a51:781f:e95a:e43f | POST "/v1/responses" This clearly shows that it is an ollama bug. Debugging Steps for Developers Provide them with this list of steps to reproduce or further analyze the bug: Pipe Buffer Monitoring: They should check if the runner's stdout/stderr pipe buffer overflows under Windows (Win32 pipes) when GPU discovery takes longer than the DiscoveryTimeout. Race Condition Check: Analyze the startup sequencer in gpu/gpu_windows.go. If discovery times out, the runner thread appears to enter an infinite wait state instead of terminating cleanly. Increase Verbosity: They should provide the command for a debug build: Ideally, the process should be terminated after being queried, and only then should a clean retry be performed. However, the server is starting a new process that then uses CPU resources and is also invisible in the log.
GiteaMirror commented 2026-04-22 19:24:42 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 24, 2026):

Same BUG

Different issue, there is no mention of cudaMemcpyAsyncReserve in the supplied log.

As you can see, Ollama falls back on CPU resources.

It does not:

time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU"

Why it's a software bug:

There are no bugs shown in this log. The model loads and responds to queries:

[GIN] 2026/03/24 - 20:02:06 | 200 | 50.6421895s | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses"

If you have a problem, open a new issue, describe what the issue is, and include a full server log. Hint: don't use an LLM to diagnose the problem, that's just wasting time and money.

<!-- gh-comment-id:4121047808 --> @rick-github commented on GitHub (Mar 24, 2026): > Same BUG Different issue, there is no mention of `cudaMemcpyAsyncReserve` in the supplied log. > As you can see, Ollama falls back on CPU resources. It does not: ``` time=2026-03-24T20:01:20.407+01:00 level=INFO source=ggml.go:494 msg="offloaded 65/65 layers to GPU" ``` > Why it's a software bug: There are no bugs shown in this log. The model loads and responds to queries: ``` [GIN] 2026/03/24 - 20:02:06 | 200 | 50.6421895s | 2001:16b8:908e:8700:4e45:4470:ff1f:a11b | POST "/v1/responses" ``` If you have a problem, open a new issue, describe what the issue is, and include a full server log. Hint: don't use an LLM to diagnose the problem, that's just wasting time and money.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#35131
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?