github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9974] 运行模型时，gpu利用率占满，但是功率很低，gpu没有正常使用起来 #6533

New Issue
Closed
opened 2026-04-12 18:08:53 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 18:08:53 -05:00
Owner
Copy Link

Originally created by @save-FGG on GitHub (Mar 25, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9974

What is the issue?

在运行模型时，使用ollama run deepseek-r1:32b --verbose运行模式时

Image

显示(base) root@xunwei:~# ollama ps
NAME ID SIZE PROCESSOR UNTIL
deepseek-r1:32b 38056bbcbb2d 21 GB 100% GPU 4 minutes from now
已经全部加载到GPU，但是交互问答时发现：

(base) root@xunwei:/var/log# nvidia-smi
Tue Mar 25 08:16:28 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.10 Driver Version: 570.86.10 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 Tesla V100S-PCIE-32GB On | 00000000:86:00.0 Off | Off |
| N/A 82C P0 61W / 250W | 21580MiB / 32768MiB | 100% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 309279 C /usr/local/bin/ollama 21576MiB |
+-----------------------------------------------------------------------------------------+
显示gpu利用率100%，但是功率很低，跟平常待机功率差不多，然后模型运行也很慢。。

演示：

介绍下你自己

用户再次要求“介绍下你自己”，可能他们希望得到更全面的信息或确认之前的回答。

我会详细说明我的功能，比如帮助编程、数据分析、文档撰写等，并强调我在这里的目的是协助他们完成任务。此外，鼓励他们提出具体问题，以便我能
更好地提供支持。

您好！我是DeepSeek-R1，一个由中国的深度求索（DeepSeek）公司开发的人工智能助手。我的主要功能是通过理解和分析用户的查询，提供相关信息、
解答问题或提供建议。

在您的扫雷项目或其他任何领域的问题上，我会尽我所能为您提供帮助。如果您有任何具体的需求或遇到困难的地方，请随时告诉我！

total duration: 1m10.642081389s
load duration: 8.873016922s
prompt eval count: 1624 token(s)
prompt eval duration: 17.044928439s
prompt eval rate: 95.28 tokens/s
eval count: 141 token(s)
eval duration: 44.672026277s
eval rate: 3.16 tokens/s

环境：
cuda：12.8
显卡：v100s -32GE显存 ---/驱动：570.86.10
系统：ubutnu24.04
ollama：0.6.2

Relevant log output

运行日志：
Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model:  CUDA_Host  output buffer size =     2.40 MiB
Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model:      CUDA0 compute buffer size =   696.00 MiB
Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model:  CUDA_Host compute buffer size =    26.01 MiB
Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: graph nodes  = 2246
Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: graph splits = 2
Mar 25 07:12:56 xunwei ollama[307837]: time=2025-03-25T07:12:56.320Z level=INFO source=server.go:619 msg="llama runner started in 7.02 seconds"
Mar 25 07:12:56 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:12:56 | 200 |  8.646598221s |       127.0.0.1 | POST     "/api/generate"
Mar 25 07:13:20 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:20 | 200 |      49.297µs |       127.0.0.1 | HEAD     "/"
Mar 25 07:13:20 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:20 | 200 |      40.767µs |       127.0.0.1 | GET      "/api/ps"
Mar 25 07:13:43 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:43 | 200 |  33.95496122s |       127.0.0.1 | POST     "/api/chat"
Mar 25 07:24:52 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:24:52 | 200 |      53.366µs |       127.0.0.1 | HEAD     "/"
Mar 25 07:24:52 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:24:52 | 200 |   28.351229ms |       127.0.0.1 | POST     "/api/show"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.143Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.vision.block_count default=0
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.key_length default=128
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.value_length default=128
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 gpu=GPU-70c41d0d-d690-921a-a5fe-daec442f4624 parallel=4 available=33747566592 required="21.5 GiB"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.305Z level=INFO source=server.go:105 msg="system memory" total="503.4 GiB" free="492.4 GiB" free_swap="7.3 GiB"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.305Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.vision.block_count default=0
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.306Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.key_length default=128
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.306Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.value_length default=128
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.307Z level=INFO source=server.go:138 msg=offload library=cuda layers.requested=-1 layers.model=65 layers.offload=65 layers.split="" memory.available="[31.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="21.5 GiB" memory.required.partial="21.5 GiB" memory.required.kv="2.0 GiB" memory.required.allocations="[21.5 GiB]" memory.weights.total="17.5 GiB" memory.weights.repeating="17.5 GiB" memory.weights.nonrepeating="609.1 MiB" memory.graph.full="676.0 MiB" memory.graph.partial="916.1 MiB"
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: loaded meta data with 26 key-value pairs and 771 tensors from /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 (version GGUF V3 (latest))
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   0:                       general.architecture str              = qwen2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   1:                               general.type str              = model
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   2:                               general.name str              = DeepSeek R1 Distill Qwen 32B
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   3:                           general.basename str              = DeepSeek-R1-Distill-Qwen
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   4:                         general.size_label str              = 32B
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   5:                          qwen2.block_count u32              = 64
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   6:                       qwen2.context_length u32              = 131072
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   7:                     qwen2.embedding_length u32              = 5120
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   8:                  qwen2.feed_forward_length u32              = 27648
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   9:                 qwen2.attention.head_count u32              = 40
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  10:              qwen2.attention.head_count_kv u32              = 8
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  11:                       qwen2.rope.freq_base f32              = 1000000.000000
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  12:     qwen2.attention.layer_norm_rms_epsilon f32              = 0.000010
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  13:                          general.file_type u32              = 15
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = gpt2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  15:                         tokenizer.ggml.pre str              = deepseek-r1-qwen
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  16:                      tokenizer.ggml.tokens arr[str,152064]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,152064]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  18:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  19:                tokenizer.ggml.bos_token_id u32              = 151646
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 151643
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 151643
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  22:               tokenizer.ggml.add_bos_token bool             = true
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  23:               tokenizer.ggml.add_eos_token bool             = false
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {% if not add_generation_prompt is de...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  25:               general.quantization_version u32              = 2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type  f32:  321 tensors
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q4_K:  385 tensors
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q6_K:   65 tensors
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file format = GGUF V3 (latest)
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file type   = Q4_K - Medium
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file size   = 18.48 GiB (4.85 BPW)
Mar 25 07:24:53 xunwei ollama[307837]: load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
Mar 25 07:24:53 xunwei ollama[307837]: load: special tokens cache size = 22
Mar 25 07:24:53 xunwei ollama[307837]: load: token to piece cache size = 0.9310 MB
Mar 25 07:24:53 xunwei ollama[307837]: print_info: arch             = qwen2
Mar 25 07:24:53 xunwei ollama[307837]: print_info: vocab_only       = 1
Mar 25 07:24:53 xunwei ollama[307837]: print_info: model type       = ?B
Mar 25 07:24:53 xunwei ollama[307837]: print_info: model params     = 32.76 B
Mar 25 07:24:53 xunwei ollama[307837]: print_info: general.name     = DeepSeek R1 Distill Qwen 32B
Mar 25 07:24:53 xunwei ollama[307837]: print_info: vocab type       = BPE
Mar 25 07:24:53 xunwei ollama[307837]: print_info: n_vocab          = 152064
Mar 25 07:24:53 xunwei ollama[307837]: print_info: n_merges         = 151387
Mar 25 07:24:53 xunwei ollama[307837]: print_info: BOS token        = 151646 '<｜begin▁of▁sentence｜>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOS token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOT token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: PAD token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: LF token         = 198 'Ċ'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM MID token    = 151660 '<|fim_middle|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM PAD token    = 151662 '<|fim_pad|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM REP token    = 151663 '<|repo_name|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM SEP token    = 151664 '<|file_sep|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token        = 151662 '<|fim_pad|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token        = 151663 '<|repo_name|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token        = 151664 '<|file_sep|>'
Mar 25 07:24:53 xunwei ollama[307837]: print_info: max token length = 256
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_load: vocab only - skipping tensors
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 --ctx-size 8192 --batch-size 512 --n-gpu-layers 65 --threads 20 --parallel 4 --port 42483"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=sched.go:450 msg="loaded runners" count=1
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=server.go:580 msg="waiting for llama runner to start responding"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.625Z level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error"
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.642Z level=INFO source=runner.go:846 msg="starting go runner"
Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: found 1 CUDA devices:
Mar 25 07:24:53 xunwei ollama[307837]:   Device 0: Tesla V100S-PCIE-32GB, compute capability 7.0, VMM: yes
Mar 25 07:24:53 xunwei ollama[307837]: load_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v12/libggml-cuda.so
Mar 25 07:24:53 xunwei ollama[307837]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-skylakex.so
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.735Z level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.AVX512=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.736Z level=INFO source=runner.go:906 msg="Server listening on 127.0.0.1:42483"
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_load_from_file_impl: using device CUDA0 (Tesla V100S-PCIE-32GB) - 32184 MiB free
Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.876Z level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model"
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: loaded meta data with 26 key-value pairs and 771 tensors from /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 (version GGUF V3 (latest))
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   0:                       general.architecture str              = qwen2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   1:                               general.type str              = model
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   2:                               general.name str              = DeepSeek R1 Distill Qwen 32B
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   3:                           general.basename str              = DeepSeek-R1-Distill-Qwen
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   4:                         general.size_label str              = 32B
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   5:                          qwen2.block_count u32              = 64
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   6:                       qwen2.context_length u32              = 131072
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   7:                     qwen2.embedding_length u32              = 5120
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   8:                  qwen2.feed_forward_length u32              = 27648
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv   9:                 qwen2.attention.head_count u32              = 40
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  10:              qwen2.attention.head_count_kv u32              = 8
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  11:                       qwen2.rope.freq_base f32              = 1000000.000000
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  12:     qwen2.attention.layer_norm_rms_epsilon f32              = 0.000010
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  13:                          general.file_type u32              = 15
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  14:                       tokenizer.ggml.model str              = gpt2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  15:                         tokenizer.ggml.pre str              = deepseek-r1-qwen
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  16:                      tokenizer.ggml.tokens arr[str,152064]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  17:                  tokenizer.ggml.token_type arr[i32,152064]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  18:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  19:                tokenizer.ggml.bos_token_id u32              = 151646
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  20:                tokenizer.ggml.eos_token_id u32              = 151643
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  21:            tokenizer.ggml.padding_token_id u32              = 151643
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  22:               tokenizer.ggml.add_bos_token bool             = true
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  23:               tokenizer.ggml.add_eos_token bool             = false
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  24:                    tokenizer.chat_template str              = {% if not add_generation_prompt is de...
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv  25:               general.quantization_version u32              = 2
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type  f32:  321 tensors
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q4_K:  385 tensors
Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q6_K:   65 tensors
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file format = GGUF V3 (latest)
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file type   = Q4_K - Medium
Mar 25 07:24:53 xunwei ollama[307837]: print_info: file size   = 18.48 GiB (4.85 BPW)
Mar 25 07:24:54 xunwei ollama[307837]: load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
Mar 25 07:24:54 xunwei ollama[307837]: load: special tokens cache size = 22
Mar 25 07:24:54 xunwei ollama[307837]: load: token to piece cache size = 0.9310 MB
Mar 25 07:24:54 xunwei ollama[307837]: print_info: arch             = qwen2
Mar 25 07:24:54 xunwei ollama[307837]: print_info: vocab_only       = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ctx_train      = 131072
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd           = 5120
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_layer          = 64
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_head           = 40
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_head_kv        = 8
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_rot            = 128
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_swa            = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_head_k    = 128
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_head_v    = 128
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_gqa            = 5
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_k_gqa     = 1024
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_v_gqa     = 1024
Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_norm_eps       = 0.0e+00
Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_norm_rms_eps   = 1.0e-05
Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_clamp_kqv      = 0.0e+00
Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_max_alibi_bias = 0.0e+00
Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_logit_scale    = 0.0e+00
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ff             = 27648
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_expert         = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_expert_used    = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: causal attn      = 1
Mar 25 07:24:54 xunwei ollama[307837]: print_info: pooling type     = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope type        = 2
Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope scaling     = linear
Mar 25 07:24:54 xunwei ollama[307837]: print_info: freq_base_train  = 1000000.0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: freq_scale_train = 1
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ctx_orig_yarn  = 131072
Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope_finetuned   = unknown
Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_conv       = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_inner      = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_state      = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_dt_rank      = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_dt_b_c_rms   = 0
Mar 25 07:24:54 xunwei ollama[307837]: print_info: model type       = 32B
Mar 25 07:24:54 xunwei ollama[307837]: print_info: model params     = 32.76 B
Mar 25 07:24:54 xunwei ollama[307837]: print_info: general.name     = DeepSeek R1 Distill Qwen 32B
Mar 25 07:24:54 xunwei ollama[307837]: print_info: vocab type       = BPE
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_vocab          = 152064
Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_merges         = 151387
Mar 25 07:24:54 xunwei ollama[307837]: print_info: BOS token        = 151646 '<｜begin▁of▁sentence｜>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOS token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOT token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: PAD token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: LF token         = 198 'Ċ'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM MID token    = 151660 '<|fim_middle|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM PAD token    = 151662 '<|fim_pad|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM REP token    = 151663 '<|repo_name|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM SEP token    = 151664 '<|file_sep|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token        = 151643 '<｜end▁of▁sentence｜>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token        = 151662 '<|fim_pad|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token        = 151663 '<|repo_name|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token        = 151664 '<|file_sep|>'
Mar 25 07:24:54 xunwei ollama[307837]: print_info: max token length = 256
Mar 25 07:24:54 xunwei ollama[307837]: load_tensors: loading model tensors, this can take a while... (mmap = true)
Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloading 64 repeating layers to GPU
Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloading output layer to GPU
Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloaded 65/65 layers to GPU
Mar 25 07:24:55 xunwei ollama[307837]: load_tensors:        CUDA0 model buffer size = 18508.35 MiB
Mar 25 07:24:55 xunwei ollama[307837]: load_tensors:   CPU_Mapped model buffer size =   417.66 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_seq_max     = 4
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx         = 8192
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx_per_seq = 2048
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_batch       = 2048
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ubatch      = 512
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: flash_attn    = 0
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: freq_base     = 1000000.0
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: freq_scale    = 1
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
Mar 25 07:25:00 xunwei ollama[307837]: llama_kv_cache_init: kv_size = 8192, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 64, can_shift = 1
Mar 25 07:25:00 xunwei ollama[307837]: llama_kv_cache_init:      CUDA0 KV buffer size =  2048.00 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: KV self size  = 2048.00 MiB, K (f16): 1024.00 MiB, V (f16): 1024.00 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model:  CUDA_Host  output buffer size =     2.40 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model:      CUDA0 compute buffer size =   696.00 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model:  CUDA_Host compute buffer size =    26.01 MiB
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: graph nodes  = 2246
Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: graph splits = 2
Mar 25 07:25:00 xunwei ollama[307837]: time=2025-03-25T07:25:00.651Z level=INFO source=server.go:619 msg="llama runner started in 7.03 seconds"
Mar 25 07:25:00 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:25:00 | 200 |   7.82406288s |       127.0.0.1 | POST     "/api/generate"

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.6.2

Originally created by @save-FGG on GitHub (Mar 25, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9974 ### What is the issue? 在运行模型时，使用ollama run deepseek-r1:32b --verbose运行模式时 ![Image](https://github.com/user-attachments/assets/8f5becc8-7f17-4351-ba8c-2cd47bba917c) 显示(base) root@xunwei:~# ollama ps NAME ID SIZE PROCESSOR UNTIL deepseek-r1:32b 38056bbcbb2d 21 GB 100% GPU 4 minutes from now 已经全部加载到GPU，但是交互问答时发现： (base) root@xunwei:/var/log# nvidia-smi Tue Mar 25 08:16:28 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 570.86.10 Driver Version: 570.86.10 CUDA Version: 12.8 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 Tesla V100S-PCIE-32GB On | 00000000:86:00.0 Off | Off | | N/A 82C P0 61W / 250W | 21580MiB / 32768MiB | 100% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 309279 C /usr/local/bin/ollama 21576MiB | +-----------------------------------------------------------------------------------------+ 显示gpu利用率100%，但是功率很低，跟平常待机功率差不多，然后模型运行也很慢。。 演示： >>> 介绍下你自己 <think> 用户再次要求“介绍下你自己”，可能他们希望得到更全面的信息或确认之前的回答。 我会详细说明我的功能，比如帮助编程、数据分析、文档撰写等，并强调我在这里的目的是协助他们完成任务。此外，鼓励他们提出具体问题，以便我能 更好地提供支持。 </think> 您好！我是DeepSeek-R1，一个由中国的深度求索（DeepSeek）公司开发的人工智能助手。我的主要功能是通过理解和分析用户的查询，提供相关信息、 解答问题或提供建议。 在您的扫雷项目或其他任何领域的问题上，我会尽我所能为您提供帮助。如果您有任何具体的需求或遇到困难的地方，请随时告诉我！ total duration: 1m10.642081389s load duration: 8.873016922s prompt eval count: 1624 token(s) prompt eval duration: 17.044928439s prompt eval rate: 95.28 tokens/s eval count: 141 token(s) eval duration: 44.672026277s eval rate: 3.16 tokens/s >>> 环境： cuda：12.8 显卡：v100s -32GE显存 ---/驱动：570.86.10 系统：ubutnu24.04 ollama：0.6.2 ### Relevant log output ```shell 运行日志： Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: CUDA_Host output buffer size = 2.40 MiB Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: CUDA0 compute buffer size = 696.00 MiB Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: CUDA_Host compute buffer size = 26.01 MiB Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: graph nodes = 2246 Mar 25 07:12:56 xunwei ollama[307837]: llama_init_from_model: graph splits = 2 Mar 25 07:12:56 xunwei ollama[307837]: time=2025-03-25T07:12:56.320Z level=INFO source=server.go:619 msg="llama runner started in 7.02 seconds" Mar 25 07:12:56 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:12:56 | 200 | 8.646598221s | 127.0.0.1 | POST "/api/generate" Mar 25 07:13:20 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:20 | 200 | 49.297µs | 127.0.0.1 | HEAD "/" Mar 25 07:13:20 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:20 | 200 | 40.767µs | 127.0.0.1 | GET "/api/ps" Mar 25 07:13:43 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:13:43 | 200 | 33.95496122s | 127.0.0.1 | POST "/api/chat" Mar 25 07:24:52 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:24:52 | 200 | 53.366µs | 127.0.0.1 | HEAD "/" Mar 25 07:24:52 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:24:52 | 200 | 28.351229ms | 127.0.0.1 | POST "/api/show" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.143Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.vision.block_count default=0 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.key_length default=128 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.value_length default=128 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.144Z level=INFO source=sched.go:715 msg="new model will fit in available VRAM in single GPU, loading" model=/mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 gpu=GPU-70c41d0d-d690-921a-a5fe-daec442f4624 parallel=4 available=33747566592 required="21.5 GiB" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.305Z level=INFO source=server.go:105 msg="system memory" total="503.4 GiB" free="492.4 GiB" free_swap="7.3 GiB" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.305Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.vision.block_count default=0 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.306Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.key_length default=128 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.306Z level=WARN source=ggml.go:149 msg="key not found" key=qwen2.attention.value_length default=128 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.307Z level=INFO source=server.go:138 msg=offload library=cuda layers.requested=-1 layers.model=65 layers.offload=65 layers.split="" memory.available="[31.4 GiB]" memory.gpu_overhead="0 B" memory.required.full="21.5 GiB" memory.required.partial="21.5 GiB" memory.required.kv="2.0 GiB" memory.required.allocations="[21.5 GiB]" memory.weights.total="17.5 GiB" memory.weights.repeating="17.5 GiB" memory.weights.nonrepeating="609.1 MiB" memory.graph.full="676.0 MiB" memory.graph.partial="916.1 MiB" Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: loaded meta data with 26 key-value pairs and 771 tensors from /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 (version GGUF V3 (latest)) Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 0: general.architecture str = qwen2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 1: general.type str = model Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 2: general.name str = DeepSeek R1 Distill Qwen 32B Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 3: general.basename str = DeepSeek-R1-Distill-Qwen Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 4: general.size_label str = 32B Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 5: qwen2.block_count u32 = 64 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 6: qwen2.context_length u32 = 131072 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 7: qwen2.embedding_length u32 = 5120 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 8: qwen2.feed_forward_length u32 = 27648 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 9: qwen2.attention.head_count u32 = 40 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 10: qwen2.attention.head_count_kv u32 = 8 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 11: qwen2.rope.freq_base f32 = 1000000.000000 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 12: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000010 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 13: general.file_type u32 = 15 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 15: tokenizer.ggml.pre str = deepseek-r1-qwen Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151646 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151643 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 151643 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 22: tokenizer.ggml.add_bos_token bool = true Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 23: tokenizer.ggml.add_eos_token bool = false Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 24: tokenizer.chat_template str = {% if not add_generation_prompt is de... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 25: general.quantization_version u32 = 2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type f32: 321 tensors Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q4_K: 385 tensors Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q6_K: 65 tensors Mar 25 07:24:53 xunwei ollama[307837]: print_info: file format = GGUF V3 (latest) Mar 25 07:24:53 xunwei ollama[307837]: print_info: file type = Q4_K - Medium Mar 25 07:24:53 xunwei ollama[307837]: print_info: file size = 18.48 GiB (4.85 BPW) Mar 25 07:24:53 xunwei ollama[307837]: load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect Mar 25 07:24:53 xunwei ollama[307837]: load: special tokens cache size = 22 Mar 25 07:24:53 xunwei ollama[307837]: load: token to piece cache size = 0.9310 MB Mar 25 07:24:53 xunwei ollama[307837]: print_info: arch = qwen2 Mar 25 07:24:53 xunwei ollama[307837]: print_info: vocab_only = 1 Mar 25 07:24:53 xunwei ollama[307837]: print_info: model type = ?B Mar 25 07:24:53 xunwei ollama[307837]: print_info: model params = 32.76 B Mar 25 07:24:53 xunwei ollama[307837]: print_info: general.name = DeepSeek R1 Distill Qwen 32B Mar 25 07:24:53 xunwei ollama[307837]: print_info: vocab type = BPE Mar 25 07:24:53 xunwei ollama[307837]: print_info: n_vocab = 152064 Mar 25 07:24:53 xunwei ollama[307837]: print_info: n_merges = 151387 Mar 25 07:24:53 xunwei ollama[307837]: print_info: BOS token = 151646 '<｜begin▁of▁sentence｜>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOS token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOT token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: PAD token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: LF token = 198 'Ċ' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM PRE token = 151659 '<|fim_prefix|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM SUF token = 151661 '<|fim_suffix|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM MID token = 151660 '<|fim_middle|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM PAD token = 151662 '<|fim_pad|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM REP token = 151663 '<|repo_name|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: FIM SEP token = 151664 '<|file_sep|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token = 151662 '<|fim_pad|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token = 151663 '<|repo_name|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: EOG token = 151664 '<|file_sep|>' Mar 25 07:24:53 xunwei ollama[307837]: print_info: max token length = 256 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_load: vocab only - skipping tensors Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 --ctx-size 8192 --batch-size 512 --n-gpu-layers 65 --threads 20 --parallel 4 --port 42483" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=sched.go:450 msg="loaded runners" count=1 Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.624Z level=INFO source=server.go:580 msg="waiting for llama runner to start responding" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.625Z level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error" Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.642Z level=INFO source=runner.go:846 msg="starting go runner" Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Mar 25 07:24:53 xunwei ollama[307837]: ggml_cuda_init: found 1 CUDA devices: Mar 25 07:24:53 xunwei ollama[307837]: Device 0: Tesla V100S-PCIE-32GB, compute capability 7.0, VMM: yes Mar 25 07:24:53 xunwei ollama[307837]: load_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v12/libggml-cuda.so Mar 25 07:24:53 xunwei ollama[307837]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-skylakex.so Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.735Z level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.AVX512=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.736Z level=INFO source=runner.go:906 msg="Server listening on 127.0.0.1:42483" Mar 25 07:24:53 xunwei ollama[307837]: llama_model_load_from_file_impl: using device CUDA0 (Tesla V100S-PCIE-32GB) - 32184 MiB free Mar 25 07:24:53 xunwei ollama[307837]: time=2025-03-25T07:24:53.876Z level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model" Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: loaded meta data with 26 key-value pairs and 771 tensors from /mnt/deepseek/ollama-md/blobs/sha256-6150cb382311b69f09cc0f9a1b69fc029cbd742b66bb8ec531aa5ecf5c613e93 (version GGUF V3 (latest)) Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 0: general.architecture str = qwen2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 1: general.type str = model Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 2: general.name str = DeepSeek R1 Distill Qwen 32B Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 3: general.basename str = DeepSeek-R1-Distill-Qwen Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 4: general.size_label str = 32B Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 5: qwen2.block_count u32 = 64 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 6: qwen2.context_length u32 = 131072 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 7: qwen2.embedding_length u32 = 5120 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 8: qwen2.feed_forward_length u32 = 27648 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 9: qwen2.attention.head_count u32 = 40 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 10: qwen2.attention.head_count_kv u32 = 8 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 11: qwen2.rope.freq_base f32 = 1000000.000000 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 12: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000010 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 13: general.file_type u32 = 15 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 14: tokenizer.ggml.model str = gpt2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 15: tokenizer.ggml.pre str = deepseek-r1-qwen Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 16: tokenizer.ggml.tokens arr[str,152064] = ["!", "\"", "#", "$", "%", "&", "'", ... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 17: tokenizer.ggml.token_type arr[i32,152064] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 18: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 19: tokenizer.ggml.bos_token_id u32 = 151646 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 20: tokenizer.ggml.eos_token_id u32 = 151643 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 21: tokenizer.ggml.padding_token_id u32 = 151643 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 22: tokenizer.ggml.add_bos_token bool = true Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 23: tokenizer.ggml.add_eos_token bool = false Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 24: tokenizer.chat_template str = {% if not add_generation_prompt is de... Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - kv 25: general.quantization_version u32 = 2 Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type f32: 321 tensors Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q4_K: 385 tensors Mar 25 07:24:53 xunwei ollama[307837]: llama_model_loader: - type q6_K: 65 tensors Mar 25 07:24:53 xunwei ollama[307837]: print_info: file format = GGUF V3 (latest) Mar 25 07:24:53 xunwei ollama[307837]: print_info: file type = Q4_K - Medium Mar 25 07:24:53 xunwei ollama[307837]: print_info: file size = 18.48 GiB (4.85 BPW) Mar 25 07:24:54 xunwei ollama[307837]: load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect Mar 25 07:24:54 xunwei ollama[307837]: load: special tokens cache size = 22 Mar 25 07:24:54 xunwei ollama[307837]: load: token to piece cache size = 0.9310 MB Mar 25 07:24:54 xunwei ollama[307837]: print_info: arch = qwen2 Mar 25 07:24:54 xunwei ollama[307837]: print_info: vocab_only = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ctx_train = 131072 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd = 5120 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_layer = 64 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_head = 40 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_head_kv = 8 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_rot = 128 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_swa = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_head_k = 128 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_head_v = 128 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_gqa = 5 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_k_gqa = 1024 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_embd_v_gqa = 1024 Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_norm_eps = 0.0e+00 Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_norm_rms_eps = 1.0e-05 Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_clamp_kqv = 0.0e+00 Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_max_alibi_bias = 0.0e+00 Mar 25 07:24:54 xunwei ollama[307837]: print_info: f_logit_scale = 0.0e+00 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ff = 27648 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_expert = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_expert_used = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: causal attn = 1 Mar 25 07:24:54 xunwei ollama[307837]: print_info: pooling type = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope type = 2 Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope scaling = linear Mar 25 07:24:54 xunwei ollama[307837]: print_info: freq_base_train = 1000000.0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: freq_scale_train = 1 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_ctx_orig_yarn = 131072 Mar 25 07:24:54 xunwei ollama[307837]: print_info: rope_finetuned = unknown Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_conv = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_inner = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_d_state = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_dt_rank = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: ssm_dt_b_c_rms = 0 Mar 25 07:24:54 xunwei ollama[307837]: print_info: model type = 32B Mar 25 07:24:54 xunwei ollama[307837]: print_info: model params = 32.76 B Mar 25 07:24:54 xunwei ollama[307837]: print_info: general.name = DeepSeek R1 Distill Qwen 32B Mar 25 07:24:54 xunwei ollama[307837]: print_info: vocab type = BPE Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_vocab = 152064 Mar 25 07:24:54 xunwei ollama[307837]: print_info: n_merges = 151387 Mar 25 07:24:54 xunwei ollama[307837]: print_info: BOS token = 151646 '<｜begin▁of▁sentence｜>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOS token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOT token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: PAD token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: LF token = 198 'Ċ' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM PRE token = 151659 '<|fim_prefix|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM SUF token = 151661 '<|fim_suffix|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM MID token = 151660 '<|fim_middle|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM PAD token = 151662 '<|fim_pad|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM REP token = 151663 '<|repo_name|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: FIM SEP token = 151664 '<|file_sep|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token = 151643 '<｜end▁of▁sentence｜>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token = 151662 '<|fim_pad|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token = 151663 '<|repo_name|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: EOG token = 151664 '<|file_sep|>' Mar 25 07:24:54 xunwei ollama[307837]: print_info: max token length = 256 Mar 25 07:24:54 xunwei ollama[307837]: load_tensors: loading model tensors, this can take a while... (mmap = true) Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloading 64 repeating layers to GPU Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloading output layer to GPU Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: offloaded 65/65 layers to GPU Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: CUDA0 model buffer size = 18508.35 MiB Mar 25 07:24:55 xunwei ollama[307837]: load_tensors: CPU_Mapped model buffer size = 417.66 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_seq_max = 4 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx = 8192 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx_per_seq = 2048 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_batch = 2048 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ubatch = 512 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: flash_attn = 0 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: freq_base = 1000000.0 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: freq_scale = 1 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized Mar 25 07:25:00 xunwei ollama[307837]: llama_kv_cache_init: kv_size = 8192, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 64, can_shift = 1 Mar 25 07:25:00 xunwei ollama[307837]: llama_kv_cache_init: CUDA0 KV buffer size = 2048.00 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: KV self size = 2048.00 MiB, K (f16): 1024.00 MiB, V (f16): 1024.00 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: CUDA_Host output buffer size = 2.40 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: CUDA0 compute buffer size = 696.00 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: CUDA_Host compute buffer size = 26.01 MiB Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: graph nodes = 2246 Mar 25 07:25:00 xunwei ollama[307837]: llama_init_from_model: graph splits = 2 Mar 25 07:25:00 xunwei ollama[307837]: time=2025-03-25T07:25:00.651Z level=INFO source=server.go:619 msg="llama runner started in 7.03 seconds" Mar 25 07:25:00 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:25:00 | 200 | 7.82406288s | 127.0.0.1 | POST "/api/generate" ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.6.2
GiteaMirror added the bug label 2026-04-12 18:08:53 -05:00
GiteaMirror commented 2026-04-12 18:08:54 -05:00
Author
Owner
Copy Link

@save-FGG commented on GitHub (Mar 25, 2025):

在运行时，cpu占用也高，但是ollama ps显示都加载到GPU了，不明白为什么会跑到cpu上去

Image

<!-- gh-comment-id:2750540588 --> @save-FGG commented on GitHub (Mar 25, 2025): 在运行时，cpu占用也高，但是ollama ps显示都加载到GPU了，不明白为什么会跑到cpu上去 ![Image](https://github.com/user-attachments/assets/87ad7f37-a68a-4c17-92b0-71d3e1a844b3)
GiteaMirror commented 2026-04-12 18:08:55 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 25, 2025):

Mar 25 07:25:00 xunwei ollama[307837]: time=2025-03-25T07:25:00.651Z level=INFO source=server.go:619 msg="llama runner started in 7.03 seconds"
Mar 25 07:25:00 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:25:00 | 200 |   7.82406288s |       127.0.0.1 | POST     "/api/generate"

The log show loading and answering a request in less than 8 seconds. This looks normal.

在运行时，cpu占用也高，但是ollama ps显示都加载到GPU了，不明白为什么会跑到cpu上去

The CPU is sending instructions and receiving responses from the GPU. It's not doing inference, but it is busy controlling the GPU.

<!-- gh-comment-id:2751153420 --> @rick-github commented on GitHub (Mar 25, 2025): ``` Mar 25 07:25:00 xunwei ollama[307837]: time=2025-03-25T07:25:00.651Z level=INFO source=server.go:619 msg="llama runner started in 7.03 seconds" Mar 25 07:25:00 xunwei ollama[307837]: [GIN] 2025/03/25 - 07:25:00 | 200 | 7.82406288s | 127.0.0.1 | POST "/api/generate" ``` The log show loading and answering a request in less than 8 seconds. This looks normal. > 在运行时，cpu占用也高，但是ollama ps显示都加载到GPU了，不明白为什么会跑到cpu上去 The CPU is sending instructions and receiving responses from the GPU. It's not doing inference, but it is busy controlling the GPU.
GiteaMirror commented 2026-04-12 18:08:55 -05:00
Author
Owner
Copy Link

@Xmenlin commented on GitHub (Mar 27, 2025):

Same. when chat with R1 32B q8, it is so slow with ollama 0.6.2，but it is ok with 0.5.x

<!-- gh-comment-id:2756665199 --> @Xmenlin commented on GitHub (Mar 27, 2025): Same. when chat with R1 32B q8, it is so slow with ollama 0.6.2，but it is ok with 0.5.x
GiteaMirror commented 2026-04-12 18:08:56 -05:00
Author
Owner
Copy Link

@aufwies commented on GitHub (Apr 2, 2025):

Same as me. Maybe, I should fallback to 0.5.x.

I have solved this issue. I reinstalled the ollama wtih 0.6.3.

<!-- gh-comment-id:2771042255 --> @aufwies commented on GitHub (Apr 2, 2025): ~~Same as me. Maybe, I should fallback to 0.5.x.~~ I have solved this issue. I reinstalled the ollama wtih 0.6.3.
GiteaMirror commented 2026-04-12 18:08:57 -05:00
Author
Owner
Copy Link

@zhiyudangchu commented on GitHub (Jun 8, 2025):

https://github.com/ollama/ollama/blob/main/docs/gpu.md#linux-suspend-resume 官方解答：在 Linux 上，经过挂起/恢复循环后，Ollama 有时会无法识别你的 NVIDIA GPU，并回退到 CPU 上运行。你可以通过使用以下命令重新加载 NVIDIA UVM 驱动程序来解决此驱动程序错误：sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm

<!-- gh-comment-id:2953987795 --> @zhiyudangchu commented on GitHub (Jun 8, 2025): https://github.com/ollama/ollama/blob/main/docs/gpu.md#linux-suspend-resume 官方解答：在 Linux 上，经过挂起/恢复循环后，Ollama 有时会无法识别你的 NVIDIA GPU，并回退到 CPU 上运行。你可以通过使用以下命令重新加载 NVIDIA UVM 驱动程序来解决此驱动程序错误：sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#6533
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?