github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #7609] Problem with some responses and unloading / feature request more detailed processing #4855

New Issue
Closed
opened 2026-04-12 15:51:15 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-12 15:51:15 -05:00
Owner
Copy Link

Originally created by @AncientMystic on GitHub (Nov 11, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7609

Ollama seems to have an issue hanging and not unloading at times.

When running a model (especially larger models) sometimes it will just hang and not respond, it acts like it is processing and uses cpu etc as if it is, sometimes when it hits the timeout it simply unloads and keeps saying it is processing in open-webui and other times it just won't respond and keeps using cpu (ive seen it go as long as 45 min) when i try to stop it, typically it just keeps running away, sometimes even when i completely close ollama and restart it, it will reload the same hanging model. It becomes rather persistent and typically the only solution is to open task manager and end the runner process, then it will stop or jump to the next task if one has been sent.

From reading over the logs it seems completely normal just reads like it started a normal process and thats it, it just stops in the middle of processing somewhere.

Also for the feature request portion, is there any way to add more verbose process information for responses detailing what is currently happening in the process so we can physically see when it becomes stuck or is working on a certain part of processing the response instead of just guessing if it is working or hanging and hoping for the best.

Originally created by @AncientMystic on GitHub (Nov 11, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7609 Ollama seems to have an issue hanging and not unloading at times. When running a model (especially larger models) sometimes it will just hang and not respond, it acts like it is processing and uses cpu etc as if it is, sometimes when it hits the timeout it simply unloads and keeps saying it is processing in open-webui and other times it just won't respond and keeps using cpu (ive seen it go as long as 45 min) when i try to stop it, typically it just keeps running away, sometimes even when i completely close ollama and restart it, it will reload the same hanging model. It becomes rather persistent and typically the only solution is to open task manager and end the runner process, then it will stop or jump to the next task if one has been sent. From reading over the logs it seems completely normal just reads like it started a normal process and thats it, it just stops in the middle of processing somewhere. Also for the feature request portion, is there any way to add more verbose process information for responses detailing what is currently happening in the process so we can physically see when it becomes stuck or is working on a certain part of processing the response instead of just guessing if it is working or hanging and hoping for the best.
GiteaMirror added the needs more info label 2026-04-12 15:51:15 -05:00
GiteaMirror commented 2026-04-12 15:51:16 -05:00
Author
Owner
Copy Link

@1zilc commented on GitHub (Nov 11, 2024):

Same with me👀

<!-- gh-comment-id:2467411106 --> @1zilc commented on GitHub (Nov 11, 2024): Same with me👀
GiteaMirror commented 2026-04-12 15:51:16 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 11, 2024):

Server logs may help in debugging. Examples of prompts, the models that show this behaviour, information about the system, etc. will also help.

<!-- gh-comment-id:2467741259 --> @rick-github commented on GitHub (Nov 11, 2024): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may help in debugging. Examples of prompts, the models that show this behaviour, information about the system, etc. will also help.
GiteaMirror commented 2026-04-12 15:51:17 -05:00
Author
Owner
Copy Link

@JTHesse commented on GitHub (Nov 11, 2024):

This might be related to #7573

<!-- gh-comment-id:2467755157 --> @JTHesse commented on GitHub (Nov 11, 2024): This might be related to #7573
GiteaMirror commented 2026-04-12 15:51:17 -05:00
Author
Owner
Copy Link

@AncientMystic commented on GitHub (Nov 11, 2024):

here are system details and a log from the latest time it has occured. the prompts themselves do not seem to matter, the larger the model and more layers on the CPU, the more likely it is to occur.

(i suspect it is possibly related to flash attention as it is mixing CPU / GPU runners and GPU is set to flash attention and CPU runner is not capable of it, but i could be completely wrong, it is not like i have a lot to go on here.)

EDIT: just disabled flash attention and it replied instead of hanging.
this may be related in a weird way to #7584 since ollama forces cpu/gpu hybrid use instead of nvidia fallback memory pure GPU use, ollama is trying to load a model with half flash attention support half without flash attention support, while trying to force flash attention.
so it seems it would be required to purely use GPU and make full use of fallback memory to force flash attention. (with models that will not fit into vram at least)

server:

Server Setup: Host OS: Proxmox 8.2.7

Hardware:
CPU: i7-7820X
RAM: 96GB DDR4 2133mhz
GPU 1: GTX 1060 3GB
GPU 2: Intel Arc A310 4GB
GPU 3: Tesla P4 8GB (used for ollama)
MAIN Drive: 1TB WD Blue SN550 NVME
2ND Drive: 1TB WD Blue SA510
3RD/4TH: Drive: 10TB HGST Sata enterprise HDD
5TH: 12TB HGST Sata Enterprise HDD

Guest OS:
Windows 11 24H2:
CPU: full core allocation to use 100% of host cpu
Ram: 60GB
vGPU: 8GB from tesla p4

Ollama in guest vm, compiled to use avx512 and k/v quantization as well as a few other tweaks/fixes. ( why i also test it on my laptop without any of these things to ensure none of them are the root issue)
open-webui running in Docker within an LXC on proxmox host

Server.log from server workstation when it has just happened again:

server.log time=2024-11-11T20:54:01.786Z level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=GPU-c707ca87-9ffc-11ef-acd4-9c4a84a45058 library=cuda total="8.0 GiB" available="3.6 GiB"
time=2024-11-11T20:54:01.786Z level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=oneapi total="3.9 GiB" available="3.7 GiB"
[GIN] 2024/11/11 - 20:54:06 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:54:06 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
time=2024-11-11T20:54:06.876Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.0880466 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2
time=2024-11-11T20:54:07.126Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.3384254 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2
time=2024-11-11T20:54:07.292Z level=INFO source=server.go:106 msg="system memory" total="54.0 GiB" free="47.8 GiB" free_swap="63.2 GiB"
time=2024-11-11T20:54:07.293Z level=INFO source=memory.go:354 msg="offload to cuda" layers.requested=33 layers.model=41 layers.offload=8 layers.split="" memory.available="[6.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="20.8 GiB" memory.required.partial="6.3 GiB" memory.required.kv="5.0 GiB" memory.required.allocations="[6.3 GiB]" memory.weights.total="16.5 GiB" memory.weights.repeating="15.2 GiB" memory.weights.nonrepeating="1.3 GiB" memory.graph.full="1.1 GiB" memory.graph.partial="2.1 GiB"
time=2024-11-11T20:54:07.295Z level=INFO source=server.go:300 msg="Enabling flash attention"
time=2024-11-11T20:54:07.298Z level=INFO source=server.go:467 msg="starting llama server" cmd="C:\\Users\\VMZ\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\runners\\cuda_v12\\ollama_llama_server.exe --model C:\\Users\\VMZ\\.ollama\\models\\blobs\\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 33 --threads 16 --flash-attn --cache-type-k q8_0 --cache-type-v q8_0 --no-mmap --parallel 1 --port 55066"
time=2024-11-11T20:54:07.300Z level=INFO source=sched.go:449 msg="loaded runners" count=1
time=2024-11-11T20:54:07.300Z level=INFO source=server.go:646 msg="waiting for llama runner to start responding"
time=2024-11-11T20:54:07.301Z level=INFO source=server.go:680 msg="waiting for server to become available" status="llm server error"
time=2024-11-11T20:54:07.376Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.5879818 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2
[GIN] 2024/11/11 - 20:54:07 | 200 | 0s | 127.0.0.1 | HEAD "/"
time=2024-11-11T20:54:07.446Z level=INFO source=runner.go:845 msg="starting go runner"
time=2024-11-11T20:54:07.446Z level=INFO source=runner.go:846 msg=system info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | cgo(gcc)" threads=16
[GIN] 2024/11/11 - 20:54:07 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
time=2024-11-11T20:54:07.447Z level=INFO source=.:0 msg="Server listening on 127.0.0.1:55066"
time=2024-11-11T20:54:07.553Z level=INFO source=server.go:680 msg="waiting for server to become available" status="llm server loading model"
llama_model_loader: loaded meta data with 32 key-value pairs and 322 tensors from C:\Users\VMZ\.ollama\models\blobs\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = command-r
llama_model_loader: - kv 1: general.name str = aya-23-35B
llama_model_loader: - kv 2: command-r.block_count u32 = 40
llama_model_loader: - kv 3: command-r.context_length u32 = 8192
llama_model_loader: - kv 4: command-r.embedding_length u32 = 8192
llama_model_loader: - kv 5: command-r.feed_forward_length u32 = 22528
llama_model_loader: - kv 6: command-r.attention.head_count u32 = 64
llama_model_loader: - kv 7: command-r.attention.head_count_kv u32 = 64
llama_model_loader: - kv 8: command-r.rope.freq_base f32 = 8000000.000000
llama_model_loader: - kv 9: command-r.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 10: general.file_type u32 = 23
llama_model_loader: - kv 11: command-r.logit_scale f32 = 0.062500
llama_model_loader: - kv 12: command-r.rope.scaling.type str = none
llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 14: tokenizer.ggml.pre str = command-r
llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,256000] = ["", "", "", "", ...
llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,256000] = [3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, ...
llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,253333] = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ a...
llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 5
llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 255001
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 23: tokenizer.chat_template.tool_use str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 24: tokenizer.chat_template.rag str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 25: tokenizer.chat_templates arr[str,2] = ["rag", "tool_use"]
llama_model_loader: - kv 26: tokenizer.chat_template str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 27: general.quantization_version u32 = 2
llama_model_loader: - kv 28: quantize.imatrix.file str = aya-23-35B-IMat-GGUF/imatrix.dat
llama_model_loader: - kv 29: quantize.imatrix.dataset str = aya-23-35B-IMat-GGUF/imatrix.dataset
llama_model_loader: - kv 30: quantize.imatrix.entries_count i32 = 280
llama_model_loader: - kv 31: quantize.imatrix.chunks_count i32 = 194
llama_model_loader: - type f32: 41 tensors
llama_model_loader: - type q5_K: 1 tensors
llama_model_loader: - type iq3_xxs: 160 tensors
llama_model_loader: - type iq3_s: 40 tensors
llama_model_loader: - type iq2_s: 80 tensors
[GIN] 2024/11/11 - 20:54:08 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:54:08 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
llm_load_vocab: special tokens cache size = 1008
llm_load_vocab: token to piece cache size = 1.8528 MB
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = command-r
llm_load_print_meta: vocab type = BPE
llm_load_print_meta: n_vocab = 256000
llm_load_print_meta: n_merges = 253333
llm_load_print_meta: vocab_only = 0
llm_load_print_meta: n_ctx_train = 8192
llm_load_print_meta: n_embd = 8192
llm_load_print_meta: n_layer = 40
llm_load_print_meta: n_head = 64
llm_load_print_meta: n_head_kv = 64
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_swa = 0
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 8192
llm_load_print_meta: n_embd_v_gqa = 8192
llm_load_print_meta: f_norm_eps = 1.0e-05
llm_load_print_meta: f_norm_rms_eps = 0.0e+00
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale = 6.2e-02
llm_load_print_meta: n_ff = 22528
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: causal attn = 1
llm_load_print_meta: pooling type = 0
llm_load_print_meta: rope type = 0
llm_load_print_meta: rope scaling = none
llm_load_print_meta: freq_base_train = 8000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn = 8192
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: ssm_d_conv = 0
llm_load_print_meta: ssm_d_inner = 0
llm_load_print_meta: ssm_d_state = 0
llm_load_print_meta: ssm_dt_rank = 0
llm_load_print_meta: ssm_dt_b_c_rms = 0
llm_load_print_meta: model type = 35B
llm_load_print_meta: model ftype = IQ3_XXS - 3.0625 bpw
llm_load_print_meta: model params = 34.98 B
llm_load_print_meta: model size = 12.87 GiB (3.16 BPW)
llm_load_print_meta: general.name = aya-23-35B
llm_load_print_meta: BOS token = 5 ''
llm_load_print_meta: EOS token = 255001 '<|END_OF_TURN_TOKEN|>'
llm_load_print_meta: PAD token = 0 ''
llm_load_print_meta: LF token = 136 'Ä'
llm_load_print_meta: EOG token = 255001 '<|END_OF_TURN_TOKEN|>'
llm_load_print_meta: max token length = 1024
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: GRID P40-8A, compute capability 6.1, VMM: no
llm_load_tensors: ggml ctx size = 0.31 MiB
[GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:54:09 | 200 | 544.8µs | 127.0.0.1 | GET "/api/ps"
llm_load_tensors: offloading 33 repeating layers to GPU
llm_load_tensors: offloaded 33/41 layers to GPU
llm_load_tensors: CUDA_Host buffer size = 4816.12 MiB
llm_load_tensors: CUDA0 buffer size = 9740.16 MiB
llama_new_context_with_model: n_ctx = 8192
llama_new_context_with_model: n_batch = 512
llama_new_context_with_model: n_ubatch = 512
llama_new_context_with_model: flash_attn = 1
llama_new_context_with_model: freq_base = 8000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init: CUDA_Host KV buffer size = 952.00 MiB
llama_kv_cache_init: CUDA0 KV buffer size = 4488.00 MiB
llama_new_context_with_model: KV self size = 5440.00 MiB, K (q8_0): 2720.00 MiB, V (q8_0): 2720.00 MiB
llama_new_context_with_model: CUDA_Host output buffer size = 1.01 MiB
llama_new_context_with_model: CUDA0 compute buffer size = 1891.00 MiB
llama_new_context_with_model: CUDA_Host compute buffer size = 48.01 MiB
llama_new_context_with_model: graph nodes = 1049
llama_new_context_with_model: graph splits = 74
time=2024-11-11T20:54:22.101Z level=INFO source=server.go:685 msg="llama runner started in 14.80 seconds"
llama_model_loader: loaded meta data with 32 key-value pairs and 322 tensors from C:\Users\VMZ\.ollama\models\blobs\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = command-r
llama_model_loader: - kv 1: general.name str = aya-23-35B
llama_model_loader: - kv 2: command-r.block_count u32 = 40
llama_model_loader: - kv 3: command-r.context_length u32 = 8192
llama_model_loader: - kv 4: command-r.embedding_length u32 = 8192
llama_model_loader: - kv 5: command-r.feed_forward_length u32 = 22528
llama_model_loader: - kv 6: command-r.attention.head_count u32 = 64
llama_model_loader: - kv 7: command-r.attention.head_count_kv u32 = 64
llama_model_loader: - kv 8: command-r.rope.freq_base f32 = 8000000.000000
llama_model_loader: - kv 9: command-r.attention.layer_norm_epsilon f32 = 0.000010
llama_model_loader: - kv 10: general.file_type u32 = 23
llama_model_loader: - kv 11: command-r.logit_scale f32 = 0.062500
llama_model_loader: - kv 12: command-r.rope.scaling.type str = none
llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 14: tokenizer.ggml.pre str = command-r
llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,256000] = ["", "", "", "", ...
llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,256000] = [3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, ...
llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,253333] = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ a...
llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 5
llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 255001
llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - kv 23: tokenizer.chat_template.tool_use str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 24: tokenizer.chat_template.rag str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 25: tokenizer.chat_templates arr[str,2] = ["rag", "tool_use"]
llama_model_loader: - kv 26: tokenizer.chat_template str = {{ bos_token }}{% if messages[0]['rol...
llama_model_loader: - kv 27: general.quantization_version u32 = 2
llama_model_loader: - kv 28: quantize.imatrix.file str = aya-23-35B-IMat-GGUF/imatrix.dat
llama_model_loader: - kv 29: quantize.imatrix.dataset str = aya-23-35B-IMat-GGUF/imatrix.dataset
llama_model_loader: - kv 30: quantize.imatrix.entries_count i32 = 280
llama_model_loader: - kv 31: quantize.imatrix.chunks_count i32 = 194
llama_model_loader: - type f32: 41 tensors
llama_model_loader: - type q5_K: 1 tensors
llama_model_loader: - type iq3_xxs: 160 tensors
llama_model_loader: - type iq3_s: 40 tensors
llama_model_loader: - type iq2_s: 80 tensors
llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
llm_load_vocab: special tokens cache size = 1008
llm_load_vocab: token to piece cache size = 1.8528 MB
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = command-r
llm_load_print_meta: vocab type = BPE
llm_load_print_meta: n_vocab = 256000
llm_load_print_meta: n_merges = 253333
llm_load_print_meta: vocab_only = 1
llm_load_print_meta: model type = ?B
llm_load_print_meta: model ftype = all F32
llm_load_print_meta: model params = 34.98 B
llm_load_print_meta: model size = 12.87 GiB (3.16 BPW)
llm_load_print_meta: general.name = aya-23-35B
llm_load_print_meta: BOS token = 5 ''
llm_load_print_meta: EOS token = 255001 '<|END_OF_TURN_TOKEN|>'
llm_load_print_meta: PAD token = 0 ''
llm_load_print_meta: LF token = 136 'Ä'
llm_load_print_meta: EOG token = 255001 '<|END_OF_TURN_TOKEN|>'
llm_load_print_meta: max token length = 1024
llama_model_load: vocab only - skipping tensors
[GIN] 2024/11/11 - 20:55:42 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:55:42 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:56:56 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:56:56 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:56:57 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:56:57 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:56:58 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:56:58 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:56:59 | 200 | 0s | 127.0.0.1 | HEAD "/"
[GIN] 2024/11/11 - 20:56:59 | 200 | 0s | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/11/11 - 20:57:42 | 200 | 75.2673ms | 10.0.0.220 | GET "/api/tags"
[GIN] 2024/11/11 - 20:57:43 | 200 | 57.2µs | 10.0.0.220 | GET "/api/version"
Laptop: Hardware:
OS: Windows 10 22H2
CPU: i7-8750H
RAM: 32GB DDR4 2133MHZ
MAIN Drive: WD BLACK 1TB SN850X NVME
2ND Drive: Micron 256GB NVME
3RD Drive: Seagate BarraCuda 2TB
Software: fresh install of the latest ollama
Docker running open-webui
<!-- gh-comment-id:2469024511 --> @AncientMystic commented on GitHub (Nov 11, 2024): here are system details and a log from the latest time it has occured. the prompts themselves do not seem to matter, the larger the model and more layers on the CPU, the more likely it is to occur. (i suspect it is possibly related to flash attention as it is mixing CPU / GPU runners and GPU is set to flash attention and CPU runner is not capable of it, but i could be completely wrong, it is not like i have a lot to go on here.) EDIT: just disabled flash attention and it replied instead of hanging. this may be related in a weird way to #7584 since ollama forces cpu/gpu hybrid use instead of nvidia fallback memory pure GPU use, ollama is trying to load a model with half flash attention support half without flash attention support, while trying to force flash attention. so it seems it would be required to purely use GPU and make full use of fallback memory to force flash attention. (with models that will not fit into vram at least) server: <details> <summary> Server Setup: </summary> Host OS: Proxmox 8.2.7<br/> <br/> Hardware: <br/> CPU: i7-7820X<br/> RAM: 96GB DDR4 2133mhz<br/> GPU 1: GTX 1060 3GB <br/> GPU 2: Intel Arc A310 4GB <br/> GPU 3: Tesla P4 8GB (used for ollama) <br/> MAIN Drive: 1TB WD Blue SN550 NVME <br/> 2ND Drive: 1TB WD Blue SA510<br/> 3RD/4TH: Drive: 10TB HGST Sata enterprise HDD<br/> 5TH: 12TB HGST Sata Enterprise HDD <br/> <br/> Guest OS: <br/> Windows 11 24H2: <br/> CPU: full core allocation to use 100% of host cpu <br/> Ram: 60GB <br/> vGPU: 8GB from tesla p4 <br/> <br/> Ollama in guest vm, compiled to use avx512 and k/v quantization as well as a few other tweaks/fixes. ( why i also test it on my laptop without any of these things to ensure none of them are the root issue) <br/> open-webui running in Docker within an LXC on proxmox host <br/> </details> Server.log from server workstation when it has just happened again: <details> <summary> server.log </summary> time=2024-11-11T20:54:01.786Z level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=GPU-c707ca87-9ffc-11ef-acd4-9c4a84a45058 library=cuda total="8.0 GiB" available="3.6 GiB"<br/> time=2024-11-11T20:54:01.786Z level=INFO source=sched.go:507 msg="updated VRAM based on existing loaded models" gpu=0 library=oneapi total="3.9 GiB" available="3.7 GiB"<br/> [GIN] 2024/11/11 - 20:54:06 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:54:06 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> time=2024-11-11T20:54:06.876Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.0880466 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2<br/> time=2024-11-11T20:54:07.126Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.3384254 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2<br/> time=2024-11-11T20:54:07.292Z level=INFO source=server.go:106 msg="system memory" total="54.0 GiB" free="47.8 GiB" free_swap="63.2 GiB"<br/> time=2024-11-11T20:54:07.293Z level=INFO source=memory.go:354 msg="offload to cuda" layers.requested=33 layers.model=41 layers.offload=8 layers.split="" memory.available="[6.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="20.8 GiB" memory.required.partial="6.3 GiB" memory.required.kv="5.0 GiB" memory.required.allocations="[6.3 GiB]" memory.weights.total="16.5 GiB" memory.weights.repeating="15.2 GiB" memory.weights.nonrepeating="1.3 GiB" memory.graph.full="1.1 GiB" memory.graph.partial="2.1 GiB"<br/> time=2024-11-11T20:54:07.295Z level=INFO source=server.go:300 msg="Enabling flash attention"<br/> time=2024-11-11T20:54:07.298Z level=INFO source=server.go:467 msg="starting llama server" cmd="C:\\Users\\VMZ\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\runners\\cuda_v12\\ollama_llama_server.exe --model C:\\Users\\VMZ\\.ollama\\models\\blobs\\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 --ctx-size 8192 --batch-size 512 --embedding --n-gpu-layers 33 --threads 16 --flash-attn --cache-type-k q8_0 --cache-type-v q8_0 --no-mmap --parallel 1 --port 55066"<br/> time=2024-11-11T20:54:07.300Z level=INFO source=sched.go:449 msg="loaded runners" count=1<br/> time=2024-11-11T20:54:07.300Z level=INFO source=server.go:646 msg="waiting for llama runner to start responding"<br/> time=2024-11-11T20:54:07.301Z level=INFO source=server.go:680 msg="waiting for server to become available" status="llm server error"<br/> time=2024-11-11T20:54:07.376Z level=WARN source=sched.go:646 msg="gpu VRAM usage didn't recover within timeout" seconds=5.5879818 model=C:\Users\VMZ\.ollama\models\blobs\sha256-e2c23eddd5f577b82ba3714b19c4350edbf1f4edfb7c5a4bc941ebc608b43bc2<br/> [GIN] 2024/11/11 - 20:54:07 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> time=2024-11-11T20:54:07.446Z level=INFO source=runner.go:845 msg="starting go runner"<br/> time=2024-11-11T20:54:07.446Z level=INFO source=runner.go:846 msg=system info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 1 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | cgo(gcc)" threads=16<br/> [GIN] 2024/11/11 - 20:54:07 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> time=2024-11-11T20:54:07.447Z level=INFO source=.:0 msg="Server listening on 127.0.0.1:55066"<br/> time=2024-11-11T20:54:07.553Z level=INFO source=server.go:680 msg="waiting for server to become available" status="llm server loading model"<br/> llama_model_loader: loaded meta data with 32 key-value pairs and 322 tensors from C:\Users\VMZ\.ollama\models\blobs\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 (version GGUF V3 (latest))<br/> llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.<br/> llama_model_loader: - kv 0: general.architecture str = command-r<br/> llama_model_loader: - kv 1: general.name str = aya-23-35B<br/> llama_model_loader: - kv 2: command-r.block_count u32 = 40<br/> llama_model_loader: - kv 3: command-r.context_length u32 = 8192<br/> llama_model_loader: - kv 4: command-r.embedding_length u32 = 8192<br/> llama_model_loader: - kv 5: command-r.feed_forward_length u32 = 22528<br/> llama_model_loader: - kv 6: command-r.attention.head_count u32 = 64<br/> llama_model_loader: - kv 7: command-r.attention.head_count_kv u32 = 64<br/> llama_model_loader: - kv 8: command-r.rope.freq_base f32 = 8000000.000000<br/> llama_model_loader: - kv 9: command-r.attention.layer_norm_epsilon f32 = 0.000010<br/> llama_model_loader: - kv 10: general.file_type u32 = 23<br/> llama_model_loader: - kv 11: command-r.logit_scale f32 = 0.062500<br/> llama_model_loader: - kv 12: command-r.rope.scaling.type str = none<br/> llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2<br/> llama_model_loader: - kv 14: tokenizer.ggml.pre str = command-r<br/> llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,256000] = ["<PAD>", "<UNK>", "<CLS>", "<SEP>", ...<br/> llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,256000] = [3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, ...<br/> llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,253333] = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ a...<br/> llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 5<br/> llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 255001<br/> llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0<br/> llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true<br/> llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false<br/> llama_model_loader: - kv 23: tokenizer.chat_template.tool_use str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 24: tokenizer.chat_template.rag str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 25: tokenizer.chat_templates arr[str,2] = ["rag", "tool_use"]<br/> llama_model_loader: - kv 26: tokenizer.chat_template str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 27: general.quantization_version u32 = 2<br/> llama_model_loader: - kv 28: quantize.imatrix.file str = aya-23-35B-IMat-GGUF/imatrix.dat<br/> llama_model_loader: - kv 29: quantize.imatrix.dataset str = aya-23-35B-IMat-GGUF/imatrix.dataset<br/> llama_model_loader: - kv 30: quantize.imatrix.entries_count i32 = 280<br/> llama_model_loader: - kv 31: quantize.imatrix.chunks_count i32 = 194<br/> llama_model_loader: - type f32: 41 tensors<br/> llama_model_loader: - type q5_K: 1 tensors<br/> llama_model_loader: - type iq3_xxs: 160 tensors<br/> llama_model_loader: - type iq3_s: 40 tensors<br/> llama_model_loader: - type iq2_s: 80 tensors<br/> [GIN] 2024/11/11 - 20:54:08 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:54:08 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect<br/> llm_load_vocab: special tokens cache size = 1008<br/> llm_load_vocab: token to piece cache size = 1.8528 MB<br/> llm_load_print_meta: format = GGUF V3 (latest)<br/> llm_load_print_meta: arch = command-r<br/> llm_load_print_meta: vocab type = BPE<br/> llm_load_print_meta: n_vocab = 256000<br/> llm_load_print_meta: n_merges = 253333<br/> llm_load_print_meta: vocab_only = 0<br/> llm_load_print_meta: n_ctx_train = 8192<br/> llm_load_print_meta: n_embd = 8192<br/> llm_load_print_meta: n_layer = 40<br/> llm_load_print_meta: n_head = 64<br/> llm_load_print_meta: n_head_kv = 64<br/> llm_load_print_meta: n_rot = 128<br/> llm_load_print_meta: n_swa = 0<br/> llm_load_print_meta: n_embd_head_k = 128<br/> llm_load_print_meta: n_embd_head_v = 128<br/> llm_load_print_meta: n_gqa = 1<br/> llm_load_print_meta: n_embd_k_gqa = 8192<br/> llm_load_print_meta: n_embd_v_gqa = 8192<br/> llm_load_print_meta: f_norm_eps = 1.0e-05<br/> llm_load_print_meta: f_norm_rms_eps = 0.0e+00<br/> llm_load_print_meta: f_clamp_kqv = 0.0e+00<br/> llm_load_print_meta: f_max_alibi_bias = 0.0e+00<br/> llm_load_print_meta: f_logit_scale = 6.2e-02<br/> llm_load_print_meta: n_ff = 22528<br/> llm_load_print_meta: n_expert = 0<br/> llm_load_print_meta: n_expert_used = 0<br/> llm_load_print_meta: causal attn = 1<br/> llm_load_print_meta: pooling type = 0<br/> llm_load_print_meta: rope type = 0<br/> llm_load_print_meta: rope scaling = none<br/> llm_load_print_meta: freq_base_train = 8000000.0<br/> llm_load_print_meta: freq_scale_train = 1<br/> llm_load_print_meta: n_ctx_orig_yarn = 8192<br/> llm_load_print_meta: rope_finetuned = unknown<br/> llm_load_print_meta: ssm_d_conv = 0<br/> llm_load_print_meta: ssm_d_inner = 0<br/> llm_load_print_meta: ssm_d_state = 0<br/> llm_load_print_meta: ssm_dt_rank = 0<br/> llm_load_print_meta: ssm_dt_b_c_rms = 0<br/> llm_load_print_meta: model type = 35B<br/> llm_load_print_meta: model ftype = IQ3_XXS - 3.0625 bpw<br/> llm_load_print_meta: model params = 34.98 B<br/> llm_load_print_meta: model size = 12.87 GiB (3.16 BPW) <br/> llm_load_print_meta: general.name = aya-23-35B<br/> llm_load_print_meta: BOS token = 5 '<BOS_TOKEN>'<br/> llm_load_print_meta: EOS token = 255001 '<|END_OF_TURN_TOKEN|>'<br/> llm_load_print_meta: PAD token = 0 '<PAD>'<br/> llm_load_print_meta: LF token = 136 'Ä'<br/> llm_load_print_meta: EOG token = 255001 '<|END_OF_TURN_TOKEN|>'<br/> llm_load_print_meta: max token length = 1024<br/> ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no<br/> ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no<br/> ggml_cuda_init: found 1 CUDA devices:<br/> Device 0: GRID P40-8A, compute capability 6.1, VMM: no<br/> llm_load_tensors: ggml ctx size = 0.31 MiB<br/> [GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:54:09 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:54:09 | 200 | 544.8µs | 127.0.0.1 | GET "/api/ps"<br/> llm_load_tensors: offloading 33 repeating layers to GPU<br/> llm_load_tensors: offloaded 33/41 layers to GPU<br/> llm_load_tensors: CUDA_Host buffer size = 4816.12 MiB<br/> llm_load_tensors: CUDA0 buffer size = 9740.16 MiB<br/> llama_new_context_with_model: n_ctx = 8192<br/> llama_new_context_with_model: n_batch = 512<br/> llama_new_context_with_model: n_ubatch = 512<br/> llama_new_context_with_model: flash_attn = 1<br/> llama_new_context_with_model: freq_base = 8000000.0<br/> llama_new_context_with_model: freq_scale = 1<br/> llama_kv_cache_init: CUDA_Host KV buffer size = 952.00 MiB<br/> llama_kv_cache_init: CUDA0 KV buffer size = 4488.00 MiB<br/> llama_new_context_with_model: KV self size = 5440.00 MiB, K (q8_0): 2720.00 MiB, V (q8_0): 2720.00 MiB<br/> llama_new_context_with_model: CUDA_Host output buffer size = 1.01 MiB<br/> llama_new_context_with_model: CUDA0 compute buffer size = 1891.00 MiB<br/> llama_new_context_with_model: CUDA_Host compute buffer size = 48.01 MiB<br/> llama_new_context_with_model: graph nodes = 1049<br/> llama_new_context_with_model: graph splits = 74<br/> time=2024-11-11T20:54:22.101Z level=INFO source=server.go:685 msg="llama runner started in 14.80 seconds"<br/> llama_model_loader: loaded meta data with 32 key-value pairs and 322 tensors from C:\Users\VMZ\.ollama\models\blobs\sha256-690e6eead2a62f841df521aa8eda2320db8f7592b1e206a1e77ad30c84f8fad5 (version GGUF V3 (latest))<br/> llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.<br/> llama_model_loader: - kv 0: general.architecture str = command-r<br/> llama_model_loader: - kv 1: general.name str = aya-23-35B<br/> llama_model_loader: - kv 2: command-r.block_count u32 = 40<br/> llama_model_loader: - kv 3: command-r.context_length u32 = 8192<br/> llama_model_loader: - kv 4: command-r.embedding_length u32 = 8192<br/> llama_model_loader: - kv 5: command-r.feed_forward_length u32 = 22528<br/> llama_model_loader: - kv 6: command-r.attention.head_count u32 = 64<br/> llama_model_loader: - kv 7: command-r.attention.head_count_kv u32 = 64<br/> llama_model_loader: - kv 8: command-r.rope.freq_base f32 = 8000000.000000<br/> llama_model_loader: - kv 9: command-r.attention.layer_norm_epsilon f32 = 0.000010<br/> llama_model_loader: - kv 10: general.file_type u32 = 23<br/> llama_model_loader: - kv 11: command-r.logit_scale f32 = 0.062500<br/> llama_model_loader: - kv 12: command-r.rope.scaling.type str = none<br/> llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2<br/> llama_model_loader: - kv 14: tokenizer.ggml.pre str = command-r<br/> llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,256000] = ["<PAD>", "<UNK>", "<CLS>", "<SEP>", ...<br/> llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,256000] = [3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, ...<br/> llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,253333] = ["Ġ Ġ", "Ġ t", "e r", "i n", "Ġ a...<br/> llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 5<br/> llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 255001<br/> llama_model_loader: - kv 20: tokenizer.ggml.padding_token_id u32 = 0<br/> llama_model_loader: - kv 21: tokenizer.ggml.add_bos_token bool = true<br/> llama_model_loader: - kv 22: tokenizer.ggml.add_eos_token bool = false<br/> llama_model_loader: - kv 23: tokenizer.chat_template.tool_use str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 24: tokenizer.chat_template.rag str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 25: tokenizer.chat_templates arr[str,2] = ["rag", "tool_use"]<br/> llama_model_loader: - kv 26: tokenizer.chat_template str = {{ bos_token }}{% if messages[0]['rol...<br/> llama_model_loader: - kv 27: general.quantization_version u32 = 2<br/> llama_model_loader: - kv 28: quantize.imatrix.file str = aya-23-35B-IMat-GGUF/imatrix.dat<br/> llama_model_loader: - kv 29: quantize.imatrix.dataset str = aya-23-35B-IMat-GGUF/imatrix.dataset<br/> llama_model_loader: - kv 30: quantize.imatrix.entries_count i32 = 280<br/> llama_model_loader: - kv 31: quantize.imatrix.chunks_count i32 = 194<br/> llama_model_loader: - type f32: 41 tensors<br/> llama_model_loader: - type q5_K: 1 tensors<br/> llama_model_loader: - type iq3_xxs: 160 tensors<br/> llama_model_loader: - type iq3_s: 40 tensors<br/> llama_model_loader: - type iq2_s: 80 tensors<br/> llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect<br/> llm_load_vocab: special tokens cache size = 1008<br/> llm_load_vocab: token to piece cache size = 1.8528 MB<br/> llm_load_print_meta: format = GGUF V3 (latest)<br/> llm_load_print_meta: arch = command-r<br/> llm_load_print_meta: vocab type = BPE<br/> llm_load_print_meta: n_vocab = 256000<br/> llm_load_print_meta: n_merges = 253333<br/> llm_load_print_meta: vocab_only = 1<br/> llm_load_print_meta: model type = ?B<br/> llm_load_print_meta: model ftype = all F32<br/> llm_load_print_meta: model params = 34.98 B<br/> llm_load_print_meta: model size = 12.87 GiB (3.16 BPW) <br/> llm_load_print_meta: general.name = aya-23-35B<br/> llm_load_print_meta: BOS token = 5 '<BOS_TOKEN>'<br/> llm_load_print_meta: EOS token = 255001 '<|END_OF_TURN_TOKEN|>'<br/> llm_load_print_meta: PAD token = 0 '<PAD>'<br/> llm_load_print_meta: LF token = 136 'Ä'<br/> llm_load_print_meta: EOG token = 255001 '<|END_OF_TURN_TOKEN|>'<br/> llm_load_print_meta: max token length = 1024<br/> llama_model_load: vocab only - skipping tensors<br/> [GIN] 2024/11/11 - 20:55:42 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:55:42 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:55:43 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:56:56 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:56:56 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:56:57 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:56:57 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:56:58 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:56:58 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:56:59 | 200 | 0s | 127.0.0.1 | HEAD "/"<br/> [GIN] 2024/11/11 - 20:56:59 | 200 | 0s | 127.0.0.1 | GET "/api/ps"<br/> [GIN] 2024/11/11 - 20:57:42 | 200 | 75.2673ms | 10.0.0.220 | GET "/api/tags"<br/> [GIN] 2024/11/11 - 20:57:43 | 200 | 57.2µs | 10.0.0.220 | GET "/api/version"<br/> </details> <details> <summary> Laptop: </summary> Hardware: <br/> OS: Windows 10 22H2<br/> CPU: i7-8750H <br/> RAM: 32GB DDR4 2133MHZ <br/> MAIN Drive: WD BLACK 1TB SN850X NVME <br/> 2ND Drive: Micron 256GB NVME <br/> 3RD Drive: Seagate BarraCuda 2TB <br/> Software: fresh install of the latest ollama <br/> Docker running open-webui <br/> </details>
GiteaMirror commented 2026-04-12 15:51:18 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 11, 2024):

--cache-type-k and --cache-type-v are not options from source head. Can you duplicate this issue with an official release?

<!-- gh-comment-id:2469197222 --> @rick-github commented on GitHub (Nov 11, 2024): `--cache-type-k` and `--cache-type-v` are not options from source head. Can you duplicate this issue with an official release?
GiteaMirror commented 2026-04-12 15:51:18 -05:00
Author
Owner
Copy Link

@AncientMystic commented on GitHub (Nov 11, 2024):

--cache-type-k and --cache-type-v are not options from source head. Can you duplicate this issue with an official release?

that is why my laptop is also listed, that has the latest ollama from source without any changes whatsoever. i test it on both to ensure those changes are not the problem and it does indeed occur on both.
( I'll need to run it again on my laptop and get it to produce the error again for logs from that too but same thing happens on both so that aspect is irrelevant to the issue i believe)

<!-- gh-comment-id:2469221818 --> @AncientMystic commented on GitHub (Nov 11, 2024): > `--cache-type-k` and `--cache-type-v` are not options from source head. Can you duplicate this issue with an official release? that is why my laptop is also listed, that has the latest ollama from source without any changes whatsoever. i test it on both to ensure those changes are not the problem and it does indeed occur on both. ( I'll need to run it again on my laptop and get it to produce the error again for logs from that too but same thing happens on both so that aspect is irrelevant to the issue i believe)
GiteaMirror commented 2026-04-12 15:51:19 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 11, 2024):

There are no logs from the laptop.

<!-- gh-comment-id:2469230577 --> @rick-github commented on GitHub (Nov 11, 2024): There are no logs from the laptop.
GiteaMirror commented 2026-04-12 15:51:19 -05:00
Author
Owner
Copy Link

@AncientMystic commented on GitHub (Nov 11, 2024):

yes, i have not included the logs yet from the laptop, just included the details and fact it has been tested on it too and indeed does occur (mainly because of the server being not the standard from source), as i said ill have to run it again and get it to produce the error again to collect logs from the laptop too.

<!-- gh-comment-id:2469239004 --> @AncientMystic commented on GitHub (Nov 11, 2024): yes, i have not included the logs yet from the laptop, just included the details and fact it has been tested on it too and indeed does occur (mainly because of the server being not the standard from source), as i said ill have to run it again and get it to produce the error again to collect logs from the laptop too.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4855
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?