github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6092] Error: timed out waiting for llama runner to start - progress 1.00 - #3809

New Issue
Closed
opened 2026-04-12 14:38:28 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 14:38:28 -05:00
Owner
Copy Link

Originally created by @JasonJasonXU on GitHub (Jul 31, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6092

What is the issue?

Details:

[root@i-zdrHahqvL ~]# OLLAMA_DEBUG=1 ollama serve 2>&1 | tee server.log
2024/07/31 15:08:52 routes.go:1099: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-07-31T15:08:52.553+08:00 level=INFO source=images.go:784 msg="total blobs: 0"
time=2024-07-31T15:08:52.553+08:00 level=INFO source=images.go:791 msg="total unused blobs removed: 0"
time=2024-07-31T15:08:52.553+08:00 level=INFO source=routes.go:1146 msg="Listening on 127.0.0.1:11434 (version 0.3.0)"
time=2024-07-31T15:08:52.553+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1636953247/runners
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/deps.txt.gz
time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/ollama_llama_server.gz
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server
time=2024-07-31T15:08:57.310+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v60102 cpu cpu_avx]"
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=sched.go:102 msg="starting llm scheduler"
time=2024-07-31T15:08:57.310+08:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:91 msg="searching for GPU discovery libraries for NVIDIA"
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcuda.so
time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/usr/local/cuda/lib64/libcuda.so** /root/libcuda.so** /usr/local/cuda*/targets//lib/libcuda.so /usr/lib/-linux-gnu/nvidia/current/libcuda.so /usr/lib/-linux-gnu/libcuda.so /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers//libcuda.so /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2024-07-31T15:08:57.311+08:00 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths="[/usr/lib/libcuda.so.470.256.02 /usr/lib64/libcuda.so.470.256.02]"
library /usr/lib/libcuda.so.470.256.02 load err: /usr/lib/libcuda.so.470.256.02: wrong ELF class: ELFCLASS32
time=2024-07-31T15:08:57.311+08:00 level=DEBUG source=gpu.go:562 msg="skipping 32bit library" library=/usr/lib/libcuda.so.470.256.02
CUDA driver version: 11.4
time=2024-07-31T15:08:57.317+08:00 level=DEBUG source=gpu.go:124 msg="detected GPUs" count=1 library=/usr/lib64/libcuda.so.470.256.02
[GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] CUDA totalMem 8020 mb
[GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] CUDA freeMem 7634 mb
[GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] Compute Capability 8.6
time=2024-07-31T15:08:57.450+08:00 level=DEBUG source=amd_linux.go:356 msg="amdgpu driver not detected /sys/module/amdgpu"
releasing cuda driver library
time=2024-07-31T15:08:57.450+08:00 level=INFO source=types.go:105 msg="inference compute" id=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a library=cuda compute=8.6 driver=11.4 name="NVIDIA A40-8Q" total="7.8 GiB" available="7.5 GiB"
[GIN] 2024/07/31 - 15:10:15 | 200 | 80.04µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/07/31 - 15:10:15 | 200 | 240.079µs | 127.0.0.1 | GET "/api/tags"
[GIN] 2024/07/31 - 15:10:22 | 200 | 32.088µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/07/31 - 15:10:22 | 200 | 97.172µs | 127.0.0.1 | GET "/api/ps"
[GIN] 2024/07/31 - 15:10:33 | 200 | 30.201µs | 127.0.0.1 | HEAD "/"
[GIN] 2024/07/31 - 15:10:33 | 404 | 291.581µs | 127.0.0.1 | POST "/api/show"
time=2024-07-31T15:10:36.318+08:00 level=INFO source=download.go:136 msg="downloading 43f7a214e532 in 45 100 MB part(s)"
time=2024-07-31T15:10:46.974+08:00 level=INFO source=images.go:1053 msg="request failed: Get "43f7a214e5/data": net/http: TLS handshake timeout"
time=2024-07-31T15:10:46.974+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 33 attempt 0 failed: Get "43f7a214e5/data": net/http: TLS handshake timeout, retrying in 1s"
time=2024-07-31T15:13:56.654+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 9 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2024-07-31T15:14:30.692+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 25 attempt 0 failed: unexpected EOF, retrying in 1s"
time=2024-07-31T15:20:57.544+08:00 level=INFO source=download.go:136 msg="downloading 62fbfd9ed093 in 1 182 B part(s)"
time=2024-07-31T15:20:59.558+08:00 level=INFO source=download.go:136 msg="downloading c156170b718e in 1 11 KB part(s)"
time=2024-07-31T15:21:01.575+08:00 level=INFO source=download.go:136 msg="downloading f02dd72bb242 in 1 59 B part(s)"
time=2024-07-31T15:21:03.577+08:00 level=INFO source=download.go:136 msg="downloading 648f809ced2b in 1 485 B part(s)"
[GIN] 2024/07/31 - 15:21:10 | 200 | 10m36s | 127.0.0.1 | POST "/api/pull"
[GIN] 2024/07/31 - 15:21:10 | 200 | 19.308683ms | 127.0.0.1 | POST "/api/show"
time=2024-07-31T15:21:10.115+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="59.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="59.4 GiB" now.free_swap="0 B"
CUDA driver version: 11.4
time=2024-07-31T15:21:10.240+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="7.5 GiB" now.total="7.8 GiB" now.free="7.5 GiB" now.used="385.8 MiB"
releasing cuda driver library
time=2024-07-31T15:21:10.240+08:00 level=DEBUG source=sched.go:177 msg="updating default concurrency" OLLAMA_MAX_LOADED_MODELS=3 gpu_count=1
time=2024-07-31T15:21:10.258+08:00 level=DEBUG source=sched.go:214 msg="loading first model" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:21:10.258+08:00 level=DEBUG source=memory.go:101 msg=evaluating library=cuda gpu_count=1 available="[7.5 GiB]"
time=2024-07-31T15:21:10.259+08:00 level=INFO source=sched.go:701 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a parallel=4 available=8005419008 required="5.3 GiB"
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=server.go:100 msg="system memory" total="63.2 GiB" free="59.4 GiB" free_swap="0 B"
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=memory.go:101 msg=evaluating library=cuda gpu_count=1 available="[7.5 GiB]"
time=2024-07-31T15:21:10.259+08:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[7.5 GiB]" memory.required.full="5.3 GiB" memory.required.partial="5.3 GiB" memory.required.kv="448.0 MiB" memory.required.allocations="[5.3 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.4 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="478.0 MiB" memory.graph.partial="730.4 MiB"
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server
time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server
time=2024-07-31T15:21:10.260+08:00 level=INFO source=server.go:383 msg="starting llama server" cmd="/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 29 --verbose --parallel 4 --port 36523"
time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=server.go:400 msg=subprocess environment="[LD_LIBRARY_PATH=/tmp/ollama1636953247/runners/cuda_v11:/tmp/ollama1636953247/runners:/usr/local/cuda/lib64: PATH=/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin CUDA_VISIBLE_DEVICES=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a]"
time=2024-07-31T15:21:10.260+08:00 level=INFO source=sched.go:437 msg="loaded runners" count=1
time=2024-07-31T15:21:10.260+08:00 level=INFO source=server.go:583 msg="waiting for llama runner to start responding"
time=2024-07-31T15:21:10.261+08:00 level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server error"
INFO [main] build info | build=1 commit="d94c6e0" tid="139961436442624" timestamp=1722410470
INFO [main] system info | n_threads=16 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139961436442624" timestamp=1722410470 total_threads=16
INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="36523" tid="139961436442624" timestamp=1722410470
llama_model_loader: loaded meta data with 21 key-value pairs and 339 tensors from /root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen2
llama_model_loader: - kv 1: general.name str = Qwen2-7B-Instruct
llama_model_loader: - kv 2: qwen2.block_count u32 = 28
llama_model_loader: - kv 3: qwen2.context_length u32 = 32768
llama_model_loader: - kv 4: qwen2.embedding_length u32 = 3584
llama_model_loader: - kv 5: qwen2.feed_forward_length u32 = 18944
llama_model_loader: - kv 6: qwen2.attention.head_count u32 = 28
llama_model_loader: - kv 7: qwen2.attention.head_count_kv u32 = 4
llama_model_loader: - kv 8: qwen2.rope.freq_base f32 = 1000000.000000
llama_model_loader: - kv 9: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 10: general.file_type u32 = 2
llama_model_loader: - kv 11: tokenizer.ggml.model str = gpt2
llama_model_loader: - kv 12: tokenizer.ggml.pre str = qwen2
llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,152064] = ["!", """, "#", "$", "%", "&", "'", ...
llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
time=2024-07-31T15:21:10.512+08:00 level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server loading model"
llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32 = 151645
llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32 = 151643
llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 151643
llama_model_loader: - kv 19: tokenizer.chat_template str = {% for message in messages %}{% if lo...
llama_model_loader: - kv 20: general.quantization_version u32 = 2
llama_model_loader: - type f32: 141 tensors
llama_model_loader: - type q4_0: 197 tensors
llama_model_loader: - type q6_K: 1 tensors
llm_load_vocab: special tokens cache size = 421
llm_load_vocab: token to piece cache size = 0.9352 MB
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = qwen2
llm_load_print_meta: vocab type = BPE
llm_load_print_meta: n_vocab = 152064
llm_load_print_meta: n_merges = 151387
llm_load_print_meta: vocab_only = 0
llm_load_print_meta: n_ctx_train = 32768
llm_load_print_meta: n_embd = 3584
llm_load_print_meta: n_layer = 28
llm_load_print_meta: n_head = 28
llm_load_print_meta: n_head_kv = 4
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_swa = 0
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 7
llm_load_print_meta: n_embd_k_gqa = 512
llm_load_print_meta: n_embd_v_gqa = 512
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-06
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale = 0.0e+00
llm_load_print_meta: n_ff = 18944
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: causal attn = 1
llm_load_print_meta: pooling type = 0
llm_load_print_meta: rope type = 2
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn = 32768
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: ssm_d_conv = 0
llm_load_print_meta: ssm_d_inner = 0
llm_load_print_meta: ssm_d_state = 0
llm_load_print_meta: ssm_dt_rank = 0
llm_load_print_meta: model type = ?B
llm_load_print_meta: model ftype = Q4_0
llm_load_print_meta: model params = 7.62 B
llm_load_print_meta: model size = 4.12 GiB (4.65 BPW)
llm_load_print_meta: general.name = Qwen2-7B-Instruct
llm_load_print_meta: BOS token = 151643 '<|endoftext|>'
llm_load_print_meta: EOS token = 151645 '<|im_end|>'
llm_load_print_meta: PAD token = 151643 '<|endoftext|>'
llm_load_print_meta: LF token = 148848 'ÄĬ'
llm_load_print_meta: EOT token = 151645 '<|im_end|>'
llm_load_print_meta: max token length = 256
ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA A40-8Q, compute capability 8.6, VMM: no
llm_load_tensors: ggml ctx size = 0.30 MiB
llm_load_tensors: offloading 28 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 29/29 layers to GPU
llm_load_tensors: CPU buffer size = 292.36 MiB
llm_load_tensors: CUDA0 buffer size = 3928.07 MiB
time=2024-07-31T15:22:00.443+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.07"
time=2024-07-31T15:22:00.695+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.17"
time=2024-07-31T15:22:01.948+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.18"
time=2024-07-31T15:22:02.199+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.20"
time=2024-07-31T15:22:03.454+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.22"
time=2024-07-31T15:22:03.704+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.23"
time=2024-07-31T15:22:04.959+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.25"
time=2024-07-31T15:22:05.210+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.26"
time=2024-07-31T15:22:06.464+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.28"
time=2024-07-31T15:22:06.715+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.29"
time=2024-07-31T15:22:07.718+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.30"
time=2024-07-31T15:22:07.968+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.32"
time=2024-07-31T15:22:09.223+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.33"
time=2024-07-31T15:22:09.474+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.35"
time=2024-07-31T15:22:10.728+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.36"
time=2024-07-31T15:22:10.979+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.38"
time=2024-07-31T15:22:12.234+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.40"
time=2024-07-31T15:22:12.484+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.41"
time=2024-07-31T15:22:13.738+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.43"
time=2024-07-31T15:22:13.989+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.44"
time=2024-07-31T15:22:15.243+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.46"
time=2024-07-31T15:22:15.494+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.47"
time=2024-07-31T15:22:16.498+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.48"
time=2024-07-31T15:22:16.749+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.50"
time=2024-07-31T15:22:18.004+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.51"
time=2024-07-31T15:22:18.255+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.53"
time=2024-07-31T15:22:19.509+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.54"
time=2024-07-31T15:22:19.761+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.56"
time=2024-07-31T15:22:21.014+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.58"
time=2024-07-31T15:22:21.265+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.59"
time=2024-07-31T15:22:22.520+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.61"
time=2024-07-31T15:22:23.022+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.62"
time=2024-07-31T15:22:24.026+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.64"
time=2024-07-31T15:22:24.527+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.65"
time=2024-07-31T15:22:25.280+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.66"
time=2024-07-31T15:22:25.531+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.67"
time=2024-07-31T15:22:25.782+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.68"
time=2024-07-31T15:22:26.785+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.69"
time=2024-07-31T15:22:27.036+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.70"
time=2024-07-31T15:22:27.287+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.71"
time=2024-07-31T15:22:28.290+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.72"
time=2024-07-31T15:22:28.541+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.73"
time=2024-07-31T15:22:29.043+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.74"
time=2024-07-31T15:22:29.796+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.75"
time=2024-07-31T15:22:30.047+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.76"
time=2024-07-31T15:22:30.549+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.77"
time=2024-07-31T15:22:31.302+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.78"
time=2024-07-31T15:22:31.552+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.79"
time=2024-07-31T15:22:32.055+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.80"
time=2024-07-31T15:22:32.807+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.81"
time=2024-07-31T15:22:33.058+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.82"
time=2024-07-31T15:22:33.560+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.83"
time=2024-07-31T15:22:34.313+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.85"
time=2024-07-31T15:22:35.066+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.86"
time=2024-07-31T15:22:35.820+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.88"
time=2024-07-31T15:22:36.573+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.89"
time=2024-07-31T15:22:37.326+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.91"
time=2024-07-31T15:22:38.580+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.93"
time=2024-07-31T15:22:38.831+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.94"
time=2024-07-31T15:22:40.086+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.96"
time=2024-07-31T15:22:40.337+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.97"
time=2024-07-31T15:22:41.591+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.99"
llama_new_context_with_model: n_ctx = 8192
llama_new_context_with_model: n_batch = 512
llama_new_context_with_model: n_ubatch = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base = 1000000.0
llama_new_context_with_model: freq_scale = 1
time=2024-07-31T15:22:41.842+08:00 level=DEBUG source=server.go:628 msg="model load progress 1.00"
llama_kv_cache_init: CUDA0 KV buffer size = 448.00 MiB
llama_new_context_with_model: KV self size = 448.00 MiB, K (f16): 224.00 MiB, V (f16): 224.00 MiB
llama_new_context_with_model: CUDA_Host output buffer size = 2.38 MiB
llama_new_context_with_model: CUDA0 compute buffer size = 492.00 MiB
llama_new_context_with_model: CUDA_Host compute buffer size = 23.01 MiB
llama_new_context_with_model: graph nodes = 986
llama_new_context_with_model: graph splits = 2
time=2024-07-31T15:22:42.093+08:00 level=DEBUG source=server.go:631 msg="model load completed, waiting for server to become available" status="llm server loading model"
time=2024-07-31T15:27:42.142+08:00 level=ERROR source=sched.go:443 msg="error loading llama server" error="timed out waiting for llama runner to start - progress 1.00 - "
time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:446 msg="triggering expiration for failed load" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:347 msg="runner expired event received" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:363 msg="got lock to unload" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
[GIN] 2024/07/31 - 15:27:42 | 500 | 6m32s | 127.0.0.1 | POST "/api/chat"
time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="59.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="58.4 GiB" now.free_swap="0 B"
CUDA driver version: 11.4
time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="7.5 GiB" now.total="7.8 GiB" now.free="2.0 GiB" now.used="5.8 GiB"
releasing cuda driver library
time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=server.go:1039 msg="stopping llama server"
time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=server.go:1045 msg="waiting for llama server to exit"
time=2024-07-31T15:27:42.353+08:00 level=DEBUG source=server.go:1049 msg="llama server stopped"
time=2024-07-31T15:27:42.353+08:00 level=DEBUG source=sched.go:368 msg="runner released" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:27:42.518+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="58.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="59.4 GiB" now.free_swap="0 B"
CUDA driver version: 11.4
time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="2.0 GiB" now.total="7.8 GiB" now.free="7.5 GiB" now.used="385.8 MiB"
releasing cuda driver library
time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:647 msg="gpu VRAM free memory converged after 0.50 seconds" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:372 msg="sending an unloaded event" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5
time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:295 msg="ignoring unload event with no pending requests"

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

ollama version is 0.3.0

Originally created by @JasonJasonXU on GitHub (Jul 31, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6092 ### What is the issue? Details: [root@i-zdrHahqvL ~]# OLLAMA_DEBUG=1 ollama serve 2>&1 | tee server.log 2024/07/31 15:08:52 routes.go:1099: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-07-31T15:08:52.553+08:00 level=INFO source=images.go:784 msg="total blobs: 0" time=2024-07-31T15:08:52.553+08:00 level=INFO source=images.go:791 msg="total unused blobs removed: 0" time=2024-07-31T15:08:52.553+08:00 level=INFO source=routes.go:1146 msg="Listening on 127.0.0.1:11434 (version 0.3.0)" time=2024-07-31T15:08:52.553+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1636953247/runners time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/deps.txt.gz time=2024-07-31T15:08:52.553+08:00 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/ollama_llama_server.gz time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server time=2024-07-31T15:08:57.310+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v60102 cpu cpu_avx]" time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=sched.go:102 msg="starting llm scheduler" time=2024-07-31T15:08:57.310+08:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs" time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:91 msg="searching for GPU discovery libraries for NVIDIA" time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcuda.so* time=2024-07-31T15:08:57.310+08:00 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/usr/local/cuda/lib64/libcuda.so** /root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" time=2024-07-31T15:08:57.311+08:00 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths="[/usr/lib/libcuda.so.470.256.02 /usr/lib64/libcuda.so.470.256.02]" library /usr/lib/libcuda.so.470.256.02 load err: /usr/lib/libcuda.so.470.256.02: wrong ELF class: ELFCLASS32 time=2024-07-31T15:08:57.311+08:00 level=DEBUG source=gpu.go:562 msg="skipping 32bit library" library=/usr/lib/libcuda.so.470.256.02 CUDA driver version: 11.4 time=2024-07-31T15:08:57.317+08:00 level=DEBUG source=gpu.go:124 msg="detected GPUs" count=1 library=/usr/lib64/libcuda.so.470.256.02 [GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] CUDA totalMem 8020 mb [GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] CUDA freeMem 7634 mb [GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a] Compute Capability 8.6 time=2024-07-31T15:08:57.450+08:00 level=DEBUG source=amd_linux.go:356 msg="amdgpu driver not detected /sys/module/amdgpu" releasing cuda driver library time=2024-07-31T15:08:57.450+08:00 level=INFO source=types.go:105 msg="inference compute" id=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a library=cuda compute=8.6 driver=11.4 name="NVIDIA A40-8Q" total="7.8 GiB" available="7.5 GiB" [GIN] 2024/07/31 - 15:10:15 | 200 | 80.04µs | 127.0.0.1 | HEAD "/" [GIN] 2024/07/31 - 15:10:15 | 200 | 240.079µs | 127.0.0.1 | GET "/api/tags" [GIN] 2024/07/31 - 15:10:22 | 200 | 32.088µs | 127.0.0.1 | HEAD "/" [GIN] 2024/07/31 - 15:10:22 | 200 | 97.172µs | 127.0.0.1 | GET "/api/ps" [GIN] 2024/07/31 - 15:10:33 | 200 | 30.201µs | 127.0.0.1 | HEAD "/" [GIN] 2024/07/31 - 15:10:33 | 404 | 291.581µs | 127.0.0.1 | POST "/api/show" time=2024-07-31T15:10:36.318+08:00 level=INFO source=download.go:136 msg="downloading 43f7a214e532 in 45 100 MB part(s)" time=2024-07-31T15:10:46.974+08:00 level=INFO source=images.go:1053 msg="request failed: Get \"https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/43/43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20240731%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20240731T071235Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=519cf733df9fd6b10bf26ec24a94cddfe36246417e19142e6dd552b25c636520\": net/http: TLS handshake timeout" time=2024-07-31T15:10:46.974+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 33 attempt 0 failed: Get \"https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/43/43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5/data?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=66040c77ac1b787c3af820529859349a%2F20240731%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20240731T071235Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=519cf733df9fd6b10bf26ec24a94cddfe36246417e19142e6dd552b25c636520\": net/http: TLS handshake timeout, retrying in 1s" time=2024-07-31T15:13:56.654+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 9 attempt 0 failed: unexpected EOF, retrying in 1s" time=2024-07-31T15:14:30.692+08:00 level=INFO source=download.go:178 msg="43f7a214e532 part 25 attempt 0 failed: unexpected EOF, retrying in 1s" time=2024-07-31T15:20:57.544+08:00 level=INFO source=download.go:136 msg="downloading 62fbfd9ed093 in 1 182 B part(s)" time=2024-07-31T15:20:59.558+08:00 level=INFO source=download.go:136 msg="downloading c156170b718e in 1 11 KB part(s)" time=2024-07-31T15:21:01.575+08:00 level=INFO source=download.go:136 msg="downloading f02dd72bb242 in 1 59 B part(s)" time=2024-07-31T15:21:03.577+08:00 level=INFO source=download.go:136 msg="downloading 648f809ced2b in 1 485 B part(s)" [GIN] 2024/07/31 - 15:21:10 | 200 | 10m36s | 127.0.0.1 | POST "/api/pull" [GIN] 2024/07/31 - 15:21:10 | 200 | 19.308683ms | 127.0.0.1 | POST "/api/show" time=2024-07-31T15:21:10.115+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="59.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="59.4 GiB" now.free_swap="0 B" CUDA driver version: 11.4 time=2024-07-31T15:21:10.240+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="7.5 GiB" now.total="7.8 GiB" now.free="7.5 GiB" now.used="385.8 MiB" releasing cuda driver library time=2024-07-31T15:21:10.240+08:00 level=DEBUG source=sched.go:177 msg="updating default concurrency" OLLAMA_MAX_LOADED_MODELS=3 gpu_count=1 time=2024-07-31T15:21:10.258+08:00 level=DEBUG source=sched.go:214 msg="loading first model" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:21:10.258+08:00 level=DEBUG source=memory.go:101 msg=evaluating library=cuda gpu_count=1 available="[7.5 GiB]" time=2024-07-31T15:21:10.259+08:00 level=INFO source=sched.go:701 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a parallel=4 available=8005419008 required="5.3 GiB" time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=server.go:100 msg="system memory" total="63.2 GiB" free="59.4 GiB" free_swap="0 B" time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=memory.go:101 msg=evaluating library=cuda gpu_count=1 available="[7.5 GiB]" time=2024-07-31T15:21:10.259+08:00 level=INFO source=memory.go:309 msg="offload to cuda" layers.requested=-1 layers.model=29 layers.offload=29 layers.split="" memory.available="[7.5 GiB]" memory.required.full="5.3 GiB" memory.required.partial="5.3 GiB" memory.required.kv="448.0 MiB" memory.required.allocations="[5.3 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.4 GiB" memory.weights.nonrepeating="426.4 MiB" memory.graph.full="478.0 MiB" memory.graph.partial="730.4 MiB" time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server time=2024-07-31T15:21:10.259+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cpu_avx2/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama1636953247/runners/rocm_v60102/ollama_llama_server time=2024-07-31T15:21:10.260+08:00 level=INFO source=server.go:383 msg="starting llama server" cmd="/tmp/ollama1636953247/runners/cuda_v11/ollama_llama_server --model /root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 29 --verbose --parallel 4 --port 36523" time=2024-07-31T15:21:10.260+08:00 level=DEBUG source=server.go:400 msg=subprocess environment="[LD_LIBRARY_PATH=/tmp/ollama1636953247/runners/cuda_v11:/tmp/ollama1636953247/runners:/usr/local/cuda/lib64: PATH=/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin CUDA_VISIBLE_DEVICES=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a]" time=2024-07-31T15:21:10.260+08:00 level=INFO source=sched.go:437 msg="loaded runners" count=1 time=2024-07-31T15:21:10.260+08:00 level=INFO source=server.go:583 msg="waiting for llama runner to start responding" time=2024-07-31T15:21:10.261+08:00 level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server error" INFO [main] build info | build=1 commit="d94c6e0" tid="139961436442624" timestamp=1722410470 INFO [main] system info | n_threads=16 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="139961436442624" timestamp=1722410470 total_threads=16 INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="15" port="36523" tid="139961436442624" timestamp=1722410470 llama_model_loader: loaded meta data with 21 key-value pairs and 339 tensors from /root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen2 llama_model_loader: - kv 1: general.name str = Qwen2-7B-Instruct llama_model_loader: - kv 2: qwen2.block_count u32 = 28 llama_model_loader: - kv 3: qwen2.context_length u32 = 32768 llama_model_loader: - kv 4: qwen2.embedding_length u32 = 3584 llama_model_loader: - kv 5: qwen2.feed_forward_length u32 = 18944 llama_model_loader: - kv 6: qwen2.attention.head_count u32 = 28 llama_model_loader: - kv 7: qwen2.attention.head_count_kv u32 = 4 llama_model_loader: - kv 8: qwen2.rope.freq_base f32 = 1000000.000000 llama_model_loader: - kv 9: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 10: general.file_type u32 = 2 llama_model_loader: - kv 11: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 12: tokenizer.ggml.pre str = qwen2 llama_model_loader: - kv 13: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... time=2024-07-31T15:21:10.512+08:00 level=INFO source=server.go:617 msg="waiting for server to become available" status="llm server loading model" llama_model_loader: - kv 15: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32 = 151645 llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32 = 151643 llama_model_loader: - kv 18: tokenizer.ggml.bos_token_id u32 = 151643 llama_model_loader: - kv 19: tokenizer.chat_template str = {% for message in messages %}{% if lo... llama_model_loader: - kv 20: general.quantization_version u32 = 2 llama_model_loader: - type f32: 141 tensors llama_model_loader: - type q4_0: 197 tensors llama_model_loader: - type q6_K: 1 tensors llm_load_vocab: special tokens cache size = 421 llm_load_vocab: token to piece cache size = 0.9352 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = qwen2 llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 152064 llm_load_print_meta: n_merges = 151387 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 32768 llm_load_print_meta: n_embd = 3584 llm_load_print_meta: n_layer = 28 llm_load_print_meta: n_head = 28 llm_load_print_meta: n_head_kv = 4 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 7 llm_load_print_meta: n_embd_k_gqa = 512 llm_load_print_meta: n_embd_v_gqa = 512 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-06 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 18944 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 2 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 1000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 32768 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: model type = ?B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 7.62 B llm_load_print_meta: model size = 4.12 GiB (4.65 BPW) llm_load_print_meta: general.name = Qwen2-7B-Instruct llm_load_print_meta: BOS token = 151643 '<|endoftext|>' llm_load_print_meta: EOS token = 151645 '<|im_end|>' llm_load_print_meta: PAD token = 151643 '<|endoftext|>' llm_load_print_meta: LF token = 148848 'ÄĬ' llm_load_print_meta: EOT token = 151645 '<|im_end|>' llm_load_print_meta: max token length = 256 ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 CUDA devices: Device 0: NVIDIA A40-8Q, compute capability 8.6, VMM: no llm_load_tensors: ggml ctx size = 0.30 MiB llm_load_tensors: offloading 28 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 29/29 layers to GPU llm_load_tensors: CPU buffer size = 292.36 MiB llm_load_tensors: CUDA0 buffer size = 3928.07 MiB time=2024-07-31T15:22:00.443+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.07" time=2024-07-31T15:22:00.695+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.17" time=2024-07-31T15:22:01.948+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.18" time=2024-07-31T15:22:02.199+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.20" time=2024-07-31T15:22:03.454+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.22" time=2024-07-31T15:22:03.704+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.23" time=2024-07-31T15:22:04.959+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.25" time=2024-07-31T15:22:05.210+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.26" time=2024-07-31T15:22:06.464+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.28" time=2024-07-31T15:22:06.715+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.29" time=2024-07-31T15:22:07.718+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.30" time=2024-07-31T15:22:07.968+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.32" time=2024-07-31T15:22:09.223+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.33" time=2024-07-31T15:22:09.474+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.35" time=2024-07-31T15:22:10.728+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.36" time=2024-07-31T15:22:10.979+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.38" time=2024-07-31T15:22:12.234+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.40" time=2024-07-31T15:22:12.484+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.41" time=2024-07-31T15:22:13.738+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.43" time=2024-07-31T15:22:13.989+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.44" time=2024-07-31T15:22:15.243+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.46" time=2024-07-31T15:22:15.494+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.47" time=2024-07-31T15:22:16.498+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.48" time=2024-07-31T15:22:16.749+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.50" time=2024-07-31T15:22:18.004+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.51" time=2024-07-31T15:22:18.255+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.53" time=2024-07-31T15:22:19.509+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.54" time=2024-07-31T15:22:19.761+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.56" time=2024-07-31T15:22:21.014+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.58" time=2024-07-31T15:22:21.265+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.59" time=2024-07-31T15:22:22.520+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.61" time=2024-07-31T15:22:23.022+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.62" time=2024-07-31T15:22:24.026+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.64" time=2024-07-31T15:22:24.527+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.65" time=2024-07-31T15:22:25.280+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.66" time=2024-07-31T15:22:25.531+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.67" time=2024-07-31T15:22:25.782+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.68" time=2024-07-31T15:22:26.785+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.69" time=2024-07-31T15:22:27.036+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.70" time=2024-07-31T15:22:27.287+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.71" time=2024-07-31T15:22:28.290+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.72" time=2024-07-31T15:22:28.541+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.73" time=2024-07-31T15:22:29.043+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.74" time=2024-07-31T15:22:29.796+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.75" time=2024-07-31T15:22:30.047+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.76" time=2024-07-31T15:22:30.549+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.77" time=2024-07-31T15:22:31.302+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.78" time=2024-07-31T15:22:31.552+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.79" time=2024-07-31T15:22:32.055+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.80" time=2024-07-31T15:22:32.807+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.81" time=2024-07-31T15:22:33.058+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.82" time=2024-07-31T15:22:33.560+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.83" time=2024-07-31T15:22:34.313+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.85" time=2024-07-31T15:22:35.066+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.86" time=2024-07-31T15:22:35.820+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.88" time=2024-07-31T15:22:36.573+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.89" time=2024-07-31T15:22:37.326+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.91" time=2024-07-31T15:22:38.580+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.93" time=2024-07-31T15:22:38.831+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.94" time=2024-07-31T15:22:40.086+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.96" time=2024-07-31T15:22:40.337+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.97" time=2024-07-31T15:22:41.591+08:00 level=DEBUG source=server.go:628 msg="model load progress 0.99" llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 1000000.0 llama_new_context_with_model: freq_scale = 1 time=2024-07-31T15:22:41.842+08:00 level=DEBUG source=server.go:628 msg="model load progress 1.00" llama_kv_cache_init: CUDA0 KV buffer size = 448.00 MiB llama_new_context_with_model: KV self size = 448.00 MiB, K (f16): 224.00 MiB, V (f16): 224.00 MiB llama_new_context_with_model: CUDA_Host output buffer size = 2.38 MiB llama_new_context_with_model: CUDA0 compute buffer size = 492.00 MiB llama_new_context_with_model: CUDA_Host compute buffer size = 23.01 MiB llama_new_context_with_model: graph nodes = 986 llama_new_context_with_model: graph splits = 2 time=2024-07-31T15:22:42.093+08:00 level=DEBUG source=server.go:631 msg="model load completed, waiting for server to become available" status="llm server loading model" time=2024-07-31T15:27:42.142+08:00 level=ERROR source=sched.go:443 msg="error loading llama server" error="timed out waiting for llama runner to start - progress 1.00 - " time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:446 msg="triggering expiration for failed load" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:347 msg="runner expired event received" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=sched.go:363 msg="got lock to unload" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 [GIN] 2024/07/31 - 15:27:42 | 500 | 6m32s | 127.0.0.1 | POST "/api/chat" time=2024-07-31T15:27:42.142+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="59.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="58.4 GiB" now.free_swap="0 B" CUDA driver version: 11.4 time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="7.5 GiB" now.total="7.8 GiB" now.free="2.0 GiB" now.used="5.8 GiB" releasing cuda driver library time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=server.go:1039 msg="stopping llama server" time=2024-07-31T15:27:42.267+08:00 level=DEBUG source=server.go:1045 msg="waiting for llama server to exit" time=2024-07-31T15:27:42.353+08:00 level=DEBUG source=server.go:1049 msg="llama server stopped" time=2024-07-31T15:27:42.353+08:00 level=DEBUG source=sched.go:368 msg="runner released" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:27:42.518+08:00 level=DEBUG source=gpu.go:358 msg="updating system memory data" before.total="63.2 GiB" before.free="58.4 GiB" before.free_swap="0 B" now.total="63.2 GiB" now.free="59.4 GiB" now.free_swap="0 B" CUDA driver version: 11.4 time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=gpu.go:406 msg="updating cuda memory data" gpu=GPU-66762d9a-49a1-11ef-a56b-0f40675dcf0a name="NVIDIA A40-8Q" overhead="0 B" before.total="7.8 GiB" before.free="2.0 GiB" now.total="7.8 GiB" now.free="7.5 GiB" now.used="385.8 MiB" releasing cuda driver library time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:647 msg="gpu VRAM free memory converged after 0.50 seconds" model=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:372 msg="sending an unloaded event" modelPath=/root/.ollama/models/blobs/sha256-43f7a214e5329f672bb05404cfba1913cbb70fdaa1a17497224e1925046b0ed5 time=2024-07-31T15:27:42.644+08:00 level=DEBUG source=sched.go:295 msg="ignoring unload event with no pending requests" ### OS Linux ### GPU Nvidia ### CPU _No response_ ### Ollama version ollama version is 0.3.0
GiteaMirror added the bug label 2026-04-12 14:38:28 -05:00
GiteaMirror commented 2026-04-12 14:38:29 -05:00
Author
Owner
Copy Link

@JasonJasonXU commented on GitHub (Aug 1, 2024):

solved after reinstall the vGPU license

<!-- gh-comment-id:2262241775 --> @JasonJasonXU commented on GitHub (Aug 1, 2024): solved after reinstall the vGPU license
GiteaMirror commented 2026-04-12 14:38:30 -05:00
Author
Owner
Copy Link

@JorgeAlberto91MS commented on GitHub (Sep 1, 2024):

solved after reinstall the vGPU license

Please could you tell me how can I do that on Jetson Orin NX 16GB?

Thank's

<!-- gh-comment-id:2323077769 --> @JorgeAlberto91MS commented on GitHub (Sep 1, 2024): > solved after reinstall the vGPU license Please could you tell me how can I do that on Jetson Orin NX 16GB? Thank's
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3809
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?