github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-21 13:11:39 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #7024] The request for ollama with qwen2.5 never returned, and after a reboot, I found that it had timed out for 30 minutes. #4454

New Issue
Closed
opened 2026-04-12 15:22:59 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 15:22:59 -05:00
Owner
Copy Link

Originally created by @flyinsea on GitHub (Sep 29, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7024

What is the issue?

A request was made to ollama and never returned, after restarting ollama I found that the request was processed for 30 minutes, doesn't ollama have a timeout mechanism for processing requests?

The LLM used is Qwen2.5 14B Instruct.

The request occurred at 00:10:02, and you can see this only after restart ollama.

9月 29 00:10:02 ubuntu-homaer ollama[27605]: [GIN] 2024/09/29 - 00:10:02 | 500 | 32m31s | 192.168.2.210 | POST "/api/generate"

Here is the full log:

9月 28 23:36:39 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:36:39 | 200 | 13.83000742s | 192.168.2.210 | POST "/api/generate"
9月 28 23:36:51 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:36:51 | 200 | 12.323051093s | 192.168.2.210 | POST "/api/generate"
9月 28 23:37:03 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:03 | 200 | 11.960134825s | 192.168.2.210 | POST "/api/generate"
9月 28 23:37:20 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:20 | 200 | 17.062595475s | 192.168.2.210 | POST "/api/generate"
9月 28 23:37:31 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:31 | 200 | 10.729482469s | 192.168.2.210 | POST "/api/generate"
9月 29 00:10:02 ubuntu-homaer systemd[1]: Stopping Ollama Service...
9月 29 00:10:02 ubuntu-homaer ollama[27605]: [GIN] 2024/09/29 - 00:10:02 | 500 | 32m31s | 192.168.2.210 | POST "/api/generate"
9月 29 00:10:03 ubuntu-homaer systemd[1]: Stopped Ollama Service.
9月 29 00:10:03 ubuntu-homaer systemd[1]: Started Ollama Service.
9月 29 00:10:03 ubuntu-homaer ollama[83275]: 2024/09/29 00:10:03 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:4 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.428+08:00 level=INFO source=images.go:753 msg="total blobs: 16"
9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.429+08:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0"
9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.430+08:00 level=INFO source=routes.go:1200 msg="Listening on [::]:11434 (version 0.3.12)"
9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.430+08:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama1346886307/runners
9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.184+08:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx cpu_avx2 cuda_v11 cuda_v12 rocm_v60102 cpu]"
9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.184+08:00 level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.582+08:00 level=INFO source=types.go:107 msg="inference compute" id=GPU-f0bd24b0-8186-8ecd-b7df-d90cf917bb40 library=cuda variant=v12 compute=6.1 driver=12.4 name="NVIDIA GeForce GTX 1080 Ti" total="10.9 GiB" available="10.7 GiB"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.538+08:00 level=INFO source=server.go:103 msg="system memory" total="110.1 GiB" free="16.8 GiB" free_swap="0 B"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.540+08:00 level=INFO source=memory.go:326 msg="offload to cuda" layers.requested=-1 layers.model=49 layers.offload=48 layers.split="" memory.available="[10.7 GiB]" memory.gpu_overhead="0 B" memory.required.full="11.0 GiB" memory.required.partial="10.4 GiB" memory.required.kv="1.5 GiB" memory.required.allocations="[10.4 GiB]" memory.weights.total="8.9 GiB" memory.weights.repeating="8.3 GiB" memory.weights.nonrepeating="609.1 MiB" memory.graph.full="676.0 MiB" memory.graph.partial="916.1 MiB"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.552+08:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama1346886307/runners/cuda_v12/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 48 --parallel 4 --port 40394"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=sched.go:449 msg="loaded runners" count=1
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] build info | build=10 commit="188ed47" tid="140343337033728" timestamp=1727539845
9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] system info | n_threads=44 n_threads_batch=44 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140343337033728" timestamp=1727539845 total_threads=88
9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="87" port="40394" tid="140343337033728" timestamp=1727539845
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: loaded meta data with 34 key-value pairs and 579 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54 (version GGUF V3 (latest))
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 0: general.architecture str = qwen2
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 1: general.type str = model
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 2: general.name str = Qwen2.5 14B Instruct
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 3: general.finetune str = Instruct
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 4: general.basename str = Qwen2.5
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 5: general.size_label str = 14B
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 6: general.license str = apache-2.0
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 7: general.license.link str = https://huggingface.co/Qwen/Qwen2.5-1...
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 8: general.base_model.count u32 = 1
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 9: general.base_model.0.name str = Qwen2.5 14B
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 10: general.base_model.0.organization str = Qwen
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 11: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen2.5-14B
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 12: general.tags arr[str,2] = ["chat", "text-generation"]
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 13: general.languages arr[str,1] = ["en"]
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 14: qwen2.block_count u32 = 48
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 15: qwen2.context_length u32 = 32768
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 16: qwen2.embedding_length u32 = 5120
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 17: qwen2.feed_forward_length u32 = 13824
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 18: qwen2.attention.head_count u32 = 40
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 19: qwen2.attention.head_count_kv u32 = 8
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 20: qwen2.rope.freq_base f32 = 1000000.000000
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 21: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 22: general.file_type u32 = 15
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 24: tokenizer.ggml.pre str = qwen2
9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.805+08:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model"
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,152064] = ["!", """, "#", "$", "%", "&", "'", ...
9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 28: tokenizer.ggml.eos_token_id u32 = 151645
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 29: tokenizer.ggml.padding_token_id u32 = 151643
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 151643
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 31: tokenizer.ggml.add_bos_token bool = false
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 32: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>...
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 33: general.quantization_version u32 = 2
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type f32: 241 tensors
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type q4_K: 289 tensors
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type q6_K: 49 tensors
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_vocab: special tokens cache size = 22
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_vocab: token to piece cache size = 0.9310 MB
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: format = GGUF V3 (latest)
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: arch = qwen2
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: vocab type = BPE
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_vocab = 152064
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_merges = 151387
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: vocab_only = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ctx_train = 32768
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd = 5120
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_layer = 48
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_head = 40
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_head_kv = 8
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_rot = 128
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_swa = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_head_k = 128
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_head_v = 128
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_gqa = 5
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_k_gqa = 1024
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_v_gqa = 1024
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_norm_eps = 0.0e+00
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_norm_rms_eps = 1.0e-06
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_clamp_kqv = 0.0e+00
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_logit_scale = 0.0e+00
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ff = 13824
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_expert = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_expert_used = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: causal attn = 1
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: pooling type = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope type = 2
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope scaling = linear
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: freq_base_train = 1000000.0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: freq_scale_train = 1
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ctx_orig_yarn = 32768
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope_finetuned = unknown
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_conv = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_inner = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_state = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_dt_rank = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_dt_b_c_rms = 0
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model type = ?B
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model ftype = Q4_K - Medium
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model params = 14.77 B
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model size = 8.37 GiB (4.87 BPW)
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: general.name = Qwen2.5 14B Instruct
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: BOS token = 151643 '<|endoftext|>'
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: EOS token = 151645 '<|im_end|>'
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: PAD token = 151643 '<|endoftext|>'
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: LF token = 148848 'ÄĬ'
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: EOT token = 151645 '<|im_end|>'
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: max token length = 256
9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: found 1 CUDA devices:
9月 29 00:10:46 ubuntu-homaer ollama[83275]: Device 0: NVIDIA GeForce GTX 1080 Ti, compute capability 6.1, VMM: yes
9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_tensors: ggml ctx size = 0.51 MiB
9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: offloading 48 repeating layers to GPU
9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: offloaded 48/49 layers to GPU
9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: CPU buffer size = 8566.04 MiB
9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: CUDA0 buffer size = 7539.28 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_ctx = 8192
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_batch = 512
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_ubatch = 512
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: flash_attn = 0
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: freq_base = 1000000.0
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: freq_scale = 1
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_kv_cache_init: CUDA0 KV buffer size = 1536.00 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: KV self size = 1536.00 MiB, K (f16): 768.00 MiB, V (f16): 768.00 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA_Host output buffer size = 2.40 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA0 compute buffer size = 916.08 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA_Host compute buffer size = 26.01 MiB
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: graph nodes = 1686
9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: graph splits = 4
9月 29 00:10:54 ubuntu-homaer ollama[83275]: INFO [main] model loaded | tid="140343337033728" timestamp=1727539854
9月 29 00:10:54 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:54.374+08:00 level=INFO source=server.go:626 msg="llama runner started in 8.82 seconds"
9月 29 00:11:22 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:22 | 200 | 36.944087978s | 192.168.2.210 | POST "/api/generate"
9月 29 00:11:37 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:37 | 200 | 15.780919416s | 192.168.2.210 | POST "/api/generate"
9月 29 00:11:53 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:53 | 200 | 15.722272501s | 192.168.2.210 | POST "/api/generate"
9月 29 00:12:11 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:11 | 200 | 17.488382477s | 192.168.2.210 | POST "/api/generate"
9月 29 00:12:21 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:21 | 200 | 10.294996116s | 192.168.2.210 | POST "/api/generate"
9月 29 00:12:27 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:27 | 200 | 6.487865341s | 192.168.2.210 | POST "/api/generate"
9月 29 00:12:42 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:42 | 200 | 14.513810414s | 192.168.2.210 | POST "/api/generate"

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.3.12

Originally created by @flyinsea on GitHub (Sep 29, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7024 ### What is the issue? A request was made to ollama and never returned, after restarting ollama I found that the request was processed for 30 minutes, doesn't ollama have a timeout mechanism for processing requests? The LLM used is Qwen2.5 14B Instruct. The request occurred at 00:10:02, and you can see this only after restart ollama. 9月 29 00:10:02 ubuntu-homaer ollama[27605]: [GIN] 2024/09/29 - 00:10:02 | 500 | 32m31s | 192.168.2.210 | POST "/api/generate" Here is the full log: 9月 28 23:36:39 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:36:39 | 200 | 13.83000742s | 192.168.2.210 | POST "/api/generate" 9月 28 23:36:51 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:36:51 | 200 | 12.323051093s | 192.168.2.210 | POST "/api/generate" 9月 28 23:37:03 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:03 | 200 | 11.960134825s | 192.168.2.210 | POST "/api/generate" 9月 28 23:37:20 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:20 | 200 | 17.062595475s | 192.168.2.210 | POST "/api/generate" 9月 28 23:37:31 ubuntu-homaer ollama[27605]: [GIN] 2024/09/28 - 23:37:31 | 200 | 10.729482469s | 192.168.2.210 | POST "/api/generate" 9月 29 00:10:02 ubuntu-homaer systemd[1]: Stopping Ollama Service... 9月 29 00:10:02 ubuntu-homaer ollama[27605]: [GIN] 2024/09/29 - 00:10:02 | 500 | 32m31s | 192.168.2.210 | POST "/api/generate" 9月 29 00:10:03 ubuntu-homaer systemd[1]: Stopped Ollama Service. 9月 29 00:10:03 ubuntu-homaer systemd[1]: Started Ollama Service. 9月 29 00:10:03 ubuntu-homaer ollama[83275]: 2024/09/29 00:10:03 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/usr/share/ollama/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:4 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]" 9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.428+08:00 level=INFO source=images.go:753 msg="total blobs: 16" 9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.429+08:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0" 9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.430+08:00 level=INFO source=routes.go:1200 msg="Listening on [::]:11434 (version 0.3.12)" 9月 29 00:10:03 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:03.430+08:00 level=INFO source=common.go:135 msg="extracting embedded files" dir=/tmp/ollama1346886307/runners 9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.184+08:00 level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu_avx cpu_avx2 cuda_v11 cuda_v12 rocm_v60102 cpu]" 9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.184+08:00 level=INFO source=gpu.go:199 msg="looking for compatible GPUs" 9月 29 00:10:21 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:21.582+08:00 level=INFO source=types.go:107 msg="inference compute" id=GPU-f0bd24b0-8186-8ecd-b7df-d90cf917bb40 library=cuda variant=v12 compute=6.1 driver=12.4 name="NVIDIA GeForce GTX 1080 Ti" total="10.9 GiB" available="10.7 GiB" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.538+08:00 level=INFO source=server.go:103 msg="system memory" total="110.1 GiB" free="16.8 GiB" free_swap="0 B" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.540+08:00 level=INFO source=memory.go:326 msg="offload to cuda" layers.requested=-1 layers.model=49 layers.offload=48 layers.split="" memory.available="[10.7 GiB]" memory.gpu_overhead="0 B" memory.required.full="11.0 GiB" memory.required.partial="10.4 GiB" memory.required.kv="1.5 GiB" memory.required.allocations="[10.4 GiB]" memory.weights.total="8.9 GiB" memory.weights.repeating="8.3 GiB" memory.weights.nonrepeating="609.1 MiB" memory.graph.full="676.0 MiB" memory.graph.partial="916.1 MiB" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.552+08:00 level=INFO source=server.go:388 msg="starting llama server" cmd="/tmp/ollama1346886307/runners/cuda_v12/ollama_llama_server --model /usr/share/ollama/.ollama/models/blobs/sha256-2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 48 --parallel 4 --port 40394" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=sched.go:449 msg="loaded runners" count=1 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=server.go:587 msg="waiting for llama runner to start responding" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.553+08:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server error" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] build info | build=10 commit="188ed47" tid="140343337033728" timestamp=1727539845 9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] system info | n_threads=44 n_threads_batch=44 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140343337033728" timestamp=1727539845 total_threads=88 9月 29 00:10:45 ubuntu-homaer ollama[83275]: INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="87" port="40394" tid="140343337033728" timestamp=1727539845 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: loaded meta data with 34 key-value pairs and 579 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54 (version GGUF V3 (latest)) 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 0: general.architecture str = qwen2 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 1: general.type str = model 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 2: general.name str = Qwen2.5 14B Instruct 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 3: general.finetune str = Instruct 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 4: general.basename str = Qwen2.5 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 5: general.size_label str = 14B 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 6: general.license str = apache-2.0 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 7: general.license.link str = https://huggingface.co/Qwen/Qwen2.5-1... 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 8: general.base_model.count u32 = 1 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 9: general.base_model.0.name str = Qwen2.5 14B 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 10: general.base_model.0.organization str = Qwen 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 11: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen2.5-14B 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 12: general.tags arr[str,2] = ["chat", "text-generation"] 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 13: general.languages arr[str,1] = ["en"] 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 14: qwen2.block_count u32 = 48 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 15: qwen2.context_length u32 = 32768 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 16: qwen2.embedding_length u32 = 5120 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 17: qwen2.feed_forward_length u32 = 13824 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 18: qwen2.attention.head_count u32 = 40 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 19: qwen2.attention.head_count_kv u32 = 8 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 20: qwen2.rope.freq_base f32 = 1000000.000000 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 21: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 22: general.file_type u32 = 15 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 23: tokenizer.ggml.model str = gpt2 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 24: tokenizer.ggml.pre str = qwen2 9月 29 00:10:45 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:45.805+08:00 level=INFO source=server.go:621 msg="waiting for server to become available" status="llm server loading model" 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 25: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... 9月 29 00:10:45 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 26: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 27: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 28: tokenizer.ggml.eos_token_id u32 = 151645 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 29: tokenizer.ggml.padding_token_id u32 = 151643 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 30: tokenizer.ggml.bos_token_id u32 = 151643 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 31: tokenizer.ggml.add_bos_token bool = false 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 32: tokenizer.chat_template str = {%- if tools %}\n {{- '<|im_start|>... 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - kv 33: general.quantization_version u32 = 2 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type f32: 241 tensors 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type q4_K: 289 tensors 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llama_model_loader: - type q6_K: 49 tensors 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_vocab: special tokens cache size = 22 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_vocab: token to piece cache size = 0.9310 MB 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: format = GGUF V3 (latest) 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: arch = qwen2 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: vocab type = BPE 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_vocab = 152064 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_merges = 151387 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: vocab_only = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ctx_train = 32768 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd = 5120 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_layer = 48 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_head = 40 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_head_kv = 8 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_rot = 128 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_swa = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_head_k = 128 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_head_v = 128 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_gqa = 5 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_k_gqa = 1024 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_embd_v_gqa = 1024 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_norm_eps = 0.0e+00 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_norm_rms_eps = 1.0e-06 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_clamp_kqv = 0.0e+00 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_max_alibi_bias = 0.0e+00 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: f_logit_scale = 0.0e+00 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ff = 13824 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_expert = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_expert_used = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: causal attn = 1 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: pooling type = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope type = 2 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope scaling = linear 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: freq_base_train = 1000000.0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: freq_scale_train = 1 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: n_ctx_orig_yarn = 32768 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: rope_finetuned = unknown 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_conv = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_inner = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_d_state = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_dt_rank = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: ssm_dt_b_c_rms = 0 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model type = ?B 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model ftype = Q4_K - Medium 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model params = 14.77 B 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: model size = 8.37 GiB (4.87 BPW) 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: general.name = Qwen2.5 14B Instruct 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: BOS token = 151643 '<|endoftext|>' 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: EOS token = 151645 '<|im_end|>' 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: PAD token = 151643 '<|endoftext|>' 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: LF token = 148848 'ÄĬ' 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: EOT token = 151645 '<|im_end|>' 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_print_meta: max token length = 256 9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no 9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no 9月 29 00:10:46 ubuntu-homaer ollama[83275]: ggml_cuda_init: found 1 CUDA devices: 9月 29 00:10:46 ubuntu-homaer ollama[83275]: Device 0: NVIDIA GeForce GTX 1080 Ti, compute capability 6.1, VMM: yes 9月 29 00:10:46 ubuntu-homaer ollama[83275]: llm_load_tensors: ggml ctx size = 0.51 MiB 9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: offloading 48 repeating layers to GPU 9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: offloaded 48/49 layers to GPU 9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: CPU buffer size = 8566.04 MiB 9月 29 00:10:51 ubuntu-homaer ollama[83275]: llm_load_tensors: CUDA0 buffer size = 7539.28 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_ctx = 8192 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_batch = 512 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: n_ubatch = 512 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: flash_attn = 0 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: freq_base = 1000000.0 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: freq_scale = 1 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_kv_cache_init: CUDA0 KV buffer size = 1536.00 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: KV self size = 1536.00 MiB, K (f16): 768.00 MiB, V (f16): 768.00 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA_Host output buffer size = 2.40 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA0 compute buffer size = 916.08 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: CUDA_Host compute buffer size = 26.01 MiB 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: graph nodes = 1686 9月 29 00:10:54 ubuntu-homaer ollama[83275]: llama_new_context_with_model: graph splits = 4 9月 29 00:10:54 ubuntu-homaer ollama[83275]: INFO [main] model loaded | tid="140343337033728" timestamp=1727539854 9月 29 00:10:54 ubuntu-homaer ollama[83275]: time=2024-09-29T00:10:54.374+08:00 level=INFO source=server.go:626 msg="llama runner started in 8.82 seconds" 9月 29 00:11:22 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:22 | 200 | 36.944087978s | 192.168.2.210 | POST "/api/generate" 9月 29 00:11:37 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:37 | 200 | 15.780919416s | 192.168.2.210 | POST "/api/generate" 9月 29 00:11:53 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:11:53 | 200 | 15.722272501s | 192.168.2.210 | POST "/api/generate" 9月 29 00:12:11 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:11 | 200 | 17.488382477s | 192.168.2.210 | POST "/api/generate" 9月 29 00:12:21 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:21 | 200 | 10.294996116s | 192.168.2.210 | POST "/api/generate" 9月 29 00:12:27 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:27 | 200 | 6.487865341s | 192.168.2.210 | POST "/api/generate" 9月 29 00:12:42 ubuntu-homaer ollama[83275]: [GIN] 2024/09/29 - 00:12:42 | 200 | 14.513810414s | 192.168.2.210 | POST "/api/generate" ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.3.12
GiteaMirror added the needs more infobug labels 2026-04-12 15:22:59 -05:00
GiteaMirror commented 2026-04-12 15:23:01 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 29, 2024):

ollama doesn't know how long it will take to process a request, it leaves it up to the client to manage the request and any associated timeouts. 32 minutes is an usually long time though. Do you know what the query was?

<!-- gh-comment-id:2381244352 --> @rick-github commented on GitHub (Sep 29, 2024): ollama doesn't know how long it will take to process a request, it leaves it up to the client to manage the request and any associated timeouts. 32 minutes is an usually long time though. Do you know what the query was?
GiteaMirror commented 2026-04-12 15:23:01 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Nov 6, 2024):

That seems too long for a GPU to respond when 48/49 layers are loaded in VRAM. It feels like a hang scenario. Please give 0.4.0 a try and let us know if that clears up the problem. If not, please set OLLAMA_DEBUG=1 on the server and share updated server logs so we can investigate.

<!-- gh-comment-id:2458449446 --> @dhiltgen commented on GitHub (Nov 6, 2024): That seems too long for a GPU to respond when 48/49 layers are loaded in VRAM. It feels like a hang scenario. Please give 0.4.0 a try and let us know if that clears up the problem. If not, please set OLLAMA_DEBUG=1 on the server and share updated server logs so we can investigate.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4454
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?