github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10427] Mix using CPU and GPU even the GTT size is enough to load the Model on GTT of iGPU of AMD Ryzen AI 395 #6853

New Issue
Closed
opened 2026-04-12 18:39:43 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 18:39:43 -05:00
Owner
Copy Link

Originally created by @alexhegit on GitHub (Apr 27, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10427

What is the issue?

I run llama3.1:8b on the iGPU of AMD Ryzen AI 395.

OS+Kernel Ubuntu24.04 with kernel 6.11.0-24-generic.

Ollama: v0.6.6

The radeontop show the GTT is 62194MiB and VRAM is 3940M. ( My system has 128GB DDR total, GTT share about half of it)

Test commdand: ollama run llama3.1:8b --verbose "why is sky blue"

Use ollama ps and radeontop to monitor the hardware usage which show the llama3.1:8b run on CPU+GPU. It should be expect run iGPU 100%.

Image

Relevant log output

Every 2.0s: ollama ps                                                                                                                               GZ302EA: Sun Apr 27 12:49:56 2025

NAME           ID              SIZE      PROCESSOR          UNTIL
llama3.1:8b    46e0c10c039e    6.3 GB    44%/56% CPU/GPU    4 minutes from now

When try qwen2.5:1.5b and llama3.2:3b , it run 100% GPU and the model loaded into GTT. So it looks there is a bug about how to count the memory size of iGPU to do the auto layers offload to GPU.

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

v0.6.6

Originally created by @alexhegit on GitHub (Apr 27, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10427 ### What is the issue? I run llama3.1:8b on the iGPU of AMD Ryzen AI 395. OS+Kernel Ubuntu24.04 with kernel 6.11.0-24-generic. Ollama: v0.6.6 The `radeontop` show the GTT is 62194MiB and VRAM is 3940M. ( My system has 128GB DDR total, GTT share about half of it) Test commdand: ollama run llama3.1:8b --verbose "why is sky blue" Use `ollama ps` and `radeontop` to monitor the hardware usage which show the llama3.1:8b run on CPU+GPU. It should be expect run iGPU 100%. ![Image](https://github.com/user-attachments/assets/759e8404-6b48-4776-bab2-2f650ba9e010) ### Relevant log output ```shell Every 2.0s: ollama ps GZ302EA: Sun Apr 27 12:49:56 2025 NAME ID SIZE PROCESSOR UNTIL llama3.1:8b 46e0c10c039e 6.3 GB 44%/56% CPU/GPU 4 minutes from now ``` When try `qwen2.5:1.5b` and `llama3.2:3b` , it run 100% GPU and the model loaded into GTT. So it looks there is a bug about how to count the memory size of iGPU to do the auto layers offload to GPU. ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version v0.6.6
GiteaMirror added the bug label 2026-04-12 18:39:43 -05:00
GiteaMirror commented 2026-04-12 18:39:44 -05:00
Author
Owner
Copy Link

@alexhegit commented on GitHub (Apr 27, 2025):

Check the log by journalctl -u ollama after ollama run llama3.18b

Key log show only 16/33 layers to GPU

Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: loading model tensors, this can take a while... (mmap = true)
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloading 16 repeating layers to GPU
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloaded 16/33 layers to GPU
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors:        ROCm0 model buffer size =  2011.72 MiB
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors:   CPU_Mapped model buffer size =  4685.30 MiB

Full log,

Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   0:                       general.architecture str              = llama
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   1:                               general.type str              = model
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   3:                           general.finetune str              = Instruct
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   5:                         general.size_label str              = 8B
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   6:                            general.license str              = llama3.1
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   7:                               general.tags arr[str,6]       = ["facebook", "meta", "pytorch", "llam...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   8:                          general.languages arr[str,8]       = ["en", "de", "fr", "it", "pt", "hi", ...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv   9:                          llama.block_count u32              = 32
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  10:                       llama.context_length u32              = 131072
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  17:                          general.file_type u32              = 15
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv  28:               general.quantization_version u32              = 2
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type  f32:   66 tensors
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type q4_K:  193 tensors
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type q6_K:   33 tensors
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file format = GGUF V3 (latest)
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file type   = Q4_K - Medium
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file size   = 4.58 GiB (4.89 BPW)
Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.520+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading mod>
Apr 27 12:49:01 GZ302EA ollama[2470]: load: special tokens cache size = 256
Apr 27 12:49:01 GZ302EA ollama[2470]: load: token to piece cache size = 0.7999 MB
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: arch             = llama
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: vocab_only       = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ctx_train      = 131072
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd           = 4096
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_layer          = 32
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_head           = 32
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_head_kv        = 8
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_rot            = 128
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_swa            = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_swa_pattern    = 1
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_head_k    = 128
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_head_v    = 128
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_gqa            = 4
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_k_gqa     = 1024
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_v_gqa     = 1024
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_norm_eps       = 0.0e+00
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_norm_rms_eps   = 1.0e-05
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_clamp_kqv      = 0.0e+00
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_max_alibi_bias = 0.0e+00
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_logit_scale    = 0.0e+00
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_attn_scale     = 0.0e+00
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ff             = 14336
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_expert         = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_expert_used    = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: causal attn      = 1
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: pooling type     = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope type        = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope scaling     = linear
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: freq_base_train  = 500000.0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: freq_scale_train = 1
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ctx_orig_yarn  = 131072
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope_finetuned   = unknown
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_conv       = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_inner      = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_state      = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_dt_rank      = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_dt_b_c_rms   = 0
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: model type       = 8B
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: model params     = 8.03 B
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: general.name     = Meta Llama 3.1 8B Instruct
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: vocab type       = BPE
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_vocab          = 128256
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_merges         = 280147
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: BOS token        = 128000 '<|begin_of_text|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOS token        = 128009 '<|eot_id|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOT token        = 128009 '<|eot_id|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOM token        = 128008 '<|eom_id|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: LF token         = 198 'Ċ'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOG token        = 128008 '<|eom_id|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOG token        = 128009 '<|eot_id|>'
Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: max token length = 256
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: loading model tensors, this can take a while... (mmap = true)
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloading 16 repeating layers to GPU
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloaded 16/33 layers to GPU
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors:        ROCm0 model buffer size =  2011.72 MiB
Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors:   CPU_Mapped model buffer size =  4685.30 MiB
Apr 27 12:49:01 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:01 | 200 |      25.722µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:01 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:01 | 200 |      23.267µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: constructing llama_context
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_seq_max     = 1
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx         = 2048
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx_per_seq = 2048
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_batch       = 512
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ubatch      = 512
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: causal_attn   = 1
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: flash_attn    = 0
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: freq_base     = 500000.0
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: freq_scale    = 1
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context:        CPU  output buffer size =     0.50 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: init: kv_size = 2048, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1
Apr 27 12:49:02 GZ302EA ollama[2470]: init:      ROCm0 KV buffer size =   128.00 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: init:        CPU KV buffer size =   128.00 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context:      ROCm0 compute buffer size =   669.48 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context:  ROCm_Host compute buffer size =    12.01 MiB
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: graph nodes  = 1094
Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: graph splits = 181 (with bs=512), 3 (with bs=1)
Apr 27 12:49:02 GZ302EA ollama[2470]: time=2025-04-27T12:49:02.272+08:00 level=INFO source=server.go:619 msg="llama runner started in 1.51 seconds"
Apr 27 12:49:02 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:02 | 200 |  1.771128402s |       127.0.0.1 | POST     "/api/generate"
Apr 27 12:49:03 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:03 | 200 |      36.123µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:03 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:03 | 200 |      40.271µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:05 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:05 | 200 |      36.323µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:05 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:05 | 200 |      42.747µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 |      26.504µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.633+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.645+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 |    25.40694ms |       127.0.0.1 | POST     "/api/show"
Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.660+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 |      65.823µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 |      55.823µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:09 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:09 | 200 |      41.805µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:09 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:09 | 200 |      29.169µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:11 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:11 | 200 |       29.46µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:11 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:11 | 200 |      36.383µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:13 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:13 | 200 |      56.434µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:13 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:13 | 200 |     111.867µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:15 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:15 | 200 |      39.721µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:15 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:15 | 200 |      35.111µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:18 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:18 | 200 |       84.04µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:18 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:18 | 200 |      43.127µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:20 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:20 | 200 |      30.091µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:20 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:20 | 200 |      23.618µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:22 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:22 | 200 |      29.289µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:22 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:22 | 200 |      27.987µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 |      40.673µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 |       44.77µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 | 17.233304364s |       127.0.0.1 | POST     "/api/generate"
Apr 27 12:49:26 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:26 | 200 |      31.003µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:26 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:26 | 200 |      32.576µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:28 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:28 | 200 |       49.21µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:28 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:28 | 200 |      42.857µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:30 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:30 | 200 |      30.872µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:30 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:30 | 200 |      35.341µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:32 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:32 | 200 |      27.395µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:32 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:32 | 200 |      27.115µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:34 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:34 | 200 |      38.658µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:34 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:34 | 200 |      36.894µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:36 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:36 | 200 |      41.313µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:36 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:36 | 200 |      23.657µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:38 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:38 | 200 |      27.346µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:38 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:38 | 200 |      38.257µs |       127.0.0.1 | GET      "/api/ps"
Apr 27 12:49:40 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:40 | 200 |      26.453µs |       127.0.0.1 | HEAD     "/"
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=INFO source=server.go:105 msg="system memory" total="121.5 GiB" free="118.1 GiB" free_swap="8.0 GiB"
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=33 layers.offload>
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6f>
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   0:                       general.architecture str              = llama
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   1:                               general.type str              = model
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   3:                           general.finetune str              = Instruct
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   5:                         general.size_label str              = 8B
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   6:                            general.license str              = llama3.1
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   7:                               general.tags arr[str,6]       = ["facebook", "meta", "pytorch", "llam...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   8:                          general.languages arr[str,8]       = ["en", "de", "fr", "it", "pt", "hi", ...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv   9:                          llama.block_count u32              = 32
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  10:                       llama.context_length u32              = 131072
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  17:                          general.file_type u32              = 15
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv  28:               general.quantization_version u32              = 2
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type  f32:   66 tensors
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type q4_K:  193 tensors
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type q6_K:   33 tensors
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file format = GGUF V3 (latest)
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file type   = Q4_K - Medium
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file size   = 4.58 GiB (4.89 BPW)
Apr 27 12:49:00 GZ302EA ollama[2470]: load: special tokens cache size = 256
Apr 27 12:49:00 GZ302EA ollama[2470]: load: token to piece cache size = 0.7999 MB
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: arch             = llama
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: vocab_only       = 1
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: model type       = ?B
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: model params     = 8.03 B
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: general.name     = Meta Llama 3.1 8B Instruct
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: vocab type       = BPE
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: n_vocab          = 128256
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: n_merges         = 280147
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: BOS token        = 128000 '<|begin_of_text|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOS token        = 128009 '<|eot_id|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOT token        = 128009 '<|eot_id|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOM token        = 128008 '<|eom_id|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: LF token         = 198 'Ċ'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOG token        = 128008 '<|eom_id|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOG token        = 128009 '<|eot_id|>'
Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: max token length = 256
Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_load: vocab only - skipping tensors
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/>
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=sched.go:451 msg="loaded runners" count=1
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding"
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.767+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error"
Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.775+08:00 level=INFO source=runner.go:853 msg="starting go runner"
Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: found 1 ROCm devices:
Apr 27 12:49:01 GZ302EA ollama[2470]:   Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32
Apr 27 12:49:01 GZ302EA ollama[2470]: load_backend: loaded ROCm backend from /usr/local/lib/ollama/rocm/libggml-hip.so
Apr 27 12:49:01 GZ302EA ollama[2470]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so
Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.451+08:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 C>
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) - 62052 MiB free
Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.451+08:00 level=INFO source=runner.go:913 msg="Server listening on 127.0.0.1:42765"
Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6f>
<!-- gh-comment-id:2833135726 --> @alexhegit commented on GitHub (Apr 27, 2025): Check the log by `journalctl -u ollama` after `ollama run llama3.18b` Key log show only 16/33 layers to GPU ``` Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: loading model tensors, this can take a while... (mmap = true) Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloading 16 repeating layers to GPU Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloaded 16/33 layers to GPU Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: ROCm0 model buffer size = 2011.72 MiB Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: CPU_Mapped model buffer size = 4685.30 MiB ``` Full log, ```shell Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 0: general.architecture str = llama Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 1: general.type str = model Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 2: general.name str = Meta Llama 3.1 8B Instruct Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 3: general.finetune str = Instruct Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 4: general.basename str = Meta-Llama-3.1 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 5: general.size_label str = 8B Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 6: general.license str = llama3.1 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 7: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 8: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 9: llama.block_count u32 = 32 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 10: llama.context_length u32 = 131072 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 11: llama.embedding_length u32 = 4096 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 12: llama.feed_forward_length u32 = 14336 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 13: llama.attention.head_count u32 = 32 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 14: llama.attention.head_count_kv u32 = 8 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 15: llama.rope.freq_base f32 = 500000.000000 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 16: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 17: general.file_type u32 = 15 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 18: llama.vocab_size u32 = 128256 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 19: llama.rope.dimension_count u32 = 128 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 20: tokenizer.ggml.model str = gpt2 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 21: tokenizer.ggml.pre str = llama-bpe Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 22: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 128000 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 128009 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 27: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ... Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - kv 28: general.quantization_version u32 = 2 Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type f32: 66 tensors Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type q4_K: 193 tensors Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: - type q6_K: 33 tensors Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file format = GGUF V3 (latest) Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file type = Q4_K - Medium Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: file size = 4.58 GiB (4.89 BPW) Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.520+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading mod> Apr 27 12:49:01 GZ302EA ollama[2470]: load: special tokens cache size = 256 Apr 27 12:49:01 GZ302EA ollama[2470]: load: token to piece cache size = 0.7999 MB Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: arch = llama Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: vocab_only = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ctx_train = 131072 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd = 4096 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_layer = 32 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_head = 32 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_head_kv = 8 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_rot = 128 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_swa = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_swa_pattern = 1 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_head_k = 128 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_head_v = 128 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_gqa = 4 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_k_gqa = 1024 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_embd_v_gqa = 1024 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_norm_eps = 0.0e+00 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_norm_rms_eps = 1.0e-05 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_clamp_kqv = 0.0e+00 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_max_alibi_bias = 0.0e+00 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_logit_scale = 0.0e+00 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: f_attn_scale = 0.0e+00 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ff = 14336 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_expert = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_expert_used = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: causal attn = 1 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: pooling type = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope type = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope scaling = linear Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: freq_base_train = 500000.0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: freq_scale_train = 1 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_ctx_orig_yarn = 131072 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: rope_finetuned = unknown Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_conv = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_inner = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_d_state = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_dt_rank = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: ssm_dt_b_c_rms = 0 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: model type = 8B Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: model params = 8.03 B Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: general.name = Meta Llama 3.1 8B Instruct Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: vocab type = BPE Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_vocab = 128256 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: n_merges = 280147 Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: BOS token = 128000 '<|begin_of_text|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOS token = 128009 '<|eot_id|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOT token = 128009 '<|eot_id|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOM token = 128008 '<|eom_id|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: LF token = 198 'Ċ' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOG token = 128008 '<|eom_id|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: EOG token = 128009 '<|eot_id|>' Apr 27 12:49:01 GZ302EA ollama[2470]: print_info: max token length = 256 Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: loading model tensors, this can take a while... (mmap = true) Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloading 16 repeating layers to GPU Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: offloaded 16/33 layers to GPU Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: ROCm0 model buffer size = 2011.72 MiB Apr 27 12:49:01 GZ302EA ollama[2470]: load_tensors: CPU_Mapped model buffer size = 4685.30 MiB Apr 27 12:49:01 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:01 | 200 | 25.722µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:01 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:01 | 200 | 23.267µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: constructing llama_context Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_seq_max = 1 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx = 2048 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx_per_seq = 2048 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_batch = 512 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ubatch = 512 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: causal_attn = 1 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: flash_attn = 0 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: freq_base = 500000.0 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: freq_scale = 1 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: CPU output buffer size = 0.50 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: init: kv_size = 2048, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1 Apr 27 12:49:02 GZ302EA ollama[2470]: init: ROCm0 KV buffer size = 128.00 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: init: CPU KV buffer size = 128.00 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: ROCm0 compute buffer size = 669.48 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: ROCm_Host compute buffer size = 12.01 MiB Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: graph nodes = 1094 Apr 27 12:49:02 GZ302EA ollama[2470]: llama_context: graph splits = 181 (with bs=512), 3 (with bs=1) Apr 27 12:49:02 GZ302EA ollama[2470]: time=2025-04-27T12:49:02.272+08:00 level=INFO source=server.go:619 msg="llama runner started in 1.51 seconds" Apr 27 12:49:02 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:02 | 200 | 1.771128402s | 127.0.0.1 | POST "/api/generate" Apr 27 12:49:03 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:03 | 200 | 36.123µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:03 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:03 | 200 | 40.271µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:05 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:05 | 200 | 36.323µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:05 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:05 | 200 | 42.747µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 | 26.504µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.633+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.645+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 | 25.40694ms | 127.0.0.1 | POST "/api/show" Apr 27 12:49:07 GZ302EA ollama[2470]: time=2025-04-27T12:49:07.660+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 | 65.823µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:07 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:07 | 200 | 55.823µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:09 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:09 | 200 | 41.805µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:09 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:09 | 200 | 29.169µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:11 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:11 | 200 | 29.46µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:11 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:11 | 200 | 36.383µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:13 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:13 | 200 | 56.434µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:13 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:13 | 200 | 111.867µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:15 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:15 | 200 | 39.721µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:15 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:15 | 200 | 35.111µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:18 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:18 | 200 | 84.04µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:18 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:18 | 200 | 43.127µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:20 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:20 | 200 | 30.091µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:20 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:20 | 200 | 23.618µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:22 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:22 | 200 | 29.289µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:22 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:22 | 200 | 27.987µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 | 40.673µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 | 44.77µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:24 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:24 | 200 | 17.233304364s | 127.0.0.1 | POST "/api/generate" Apr 27 12:49:26 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:26 | 200 | 31.003µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:26 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:26 | 200 | 32.576µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:28 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:28 | 200 | 49.21µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:28 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:28 | 200 | 42.857µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:30 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:30 | 200 | 30.872µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:30 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:30 | 200 | 35.341µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:32 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:32 | 200 | 27.395µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:32 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:32 | 200 | 27.115µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:34 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:34 | 200 | 38.658µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:34 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:34 | 200 | 36.894µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:36 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:36 | 200 | 41.313µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:36 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:36 | 200 | 23.657µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:38 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:38 | 200 | 27.346µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:38 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:38 | 200 | 38.257µs | 127.0.0.1 | GET "/api/ps" Apr 27 12:49:40 GZ302EA ollama[2470]: [GIN] 2025/04/27 - 12:49:40 | 200 | 26.453µs | 127.0.0.1 | HEAD "/" Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=INFO source=server.go:105 msg="system memory" total="121.5 GiB" free="118.1 GiB" free_swap="8.0 GiB" Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.587+08:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=33 layers.offload> Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6f> Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 0: general.architecture str = llama Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 1: general.type str = model Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 2: general.name str = Meta Llama 3.1 8B Instruct Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 3: general.finetune str = Instruct Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 4: general.basename str = Meta-Llama-3.1 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 5: general.size_label str = 8B Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 6: general.license str = llama3.1 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 7: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 8: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 9: llama.block_count u32 = 32 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 10: llama.context_length u32 = 131072 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 11: llama.embedding_length u32 = 4096 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 12: llama.feed_forward_length u32 = 14336 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 13: llama.attention.head_count u32 = 32 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 14: llama.attention.head_count_kv u32 = 8 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 15: llama.rope.freq_base f32 = 500000.000000 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 16: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 17: general.file_type u32 = 15 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 18: llama.vocab_size u32 = 128256 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 19: llama.rope.dimension_count u32 = 128 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 20: tokenizer.ggml.model str = gpt2 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 21: tokenizer.ggml.pre str = llama-bpe Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 22: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 128000 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 128009 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 27: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ... Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - kv 28: general.quantization_version u32 = 2 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type f32: 66 tensors Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type q4_K: 193 tensors Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_loader: - type q6_K: 33 tensors Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file format = GGUF V3 (latest) Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file type = Q4_K - Medium Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: file size = 4.58 GiB (4.89 BPW) Apr 27 12:49:00 GZ302EA ollama[2470]: load: special tokens cache size = 256 Apr 27 12:49:00 GZ302EA ollama[2470]: load: token to piece cache size = 0.7999 MB Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: arch = llama Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: vocab_only = 1 Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: model type = ?B Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: model params = 8.03 B Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: general.name = Meta Llama 3.1 8B Instruct Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: vocab type = BPE Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: n_vocab = 128256 Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: n_merges = 280147 Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: BOS token = 128000 '<|begin_of_text|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOS token = 128009 '<|eot_id|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOT token = 128009 '<|eot_id|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOM token = 128008 '<|eom_id|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: LF token = 198 'Ċ' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOG token = 128008 '<|eom_id|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: EOG token = 128009 '<|eot_id|>' Apr 27 12:49:00 GZ302EA ollama[2470]: print_info: max token length = 256 Apr 27 12:49:00 GZ302EA ollama[2470]: llama_model_load: vocab only - skipping tensors Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/> Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=sched.go:451 msg="loaded runners" count=1 Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.766+08:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding" Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.767+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error" Apr 27 12:49:00 GZ302EA ollama[2470]: time=2025-04-27T12:49:00.775+08:00 level=INFO source=runner.go:853 msg="starting go runner" Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Apr 27 12:49:01 GZ302EA ollama[2470]: ggml_cuda_init: found 1 ROCm devices: Apr 27 12:49:01 GZ302EA ollama[2470]: Device 0: AMD Radeon Graphics, gfx1151 (0x1151), VMM: no, Wave Size: 32 Apr 27 12:49:01 GZ302EA ollama[2470]: load_backend: loaded ROCm backend from /usr/local/lib/ollama/rocm/libggml-hip.so Apr 27 12:49:01 GZ302EA ollama[2470]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.451+08:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 C> Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) - 62052 MiB free Apr 27 12:49:01 GZ302EA ollama[2470]: time=2025-04-27T12:49:01.451+08:00 level=INFO source=runner.go:913 msg="Server listening on 127.0.0.1:42765" Apr 27 12:49:01 GZ302EA ollama[2470]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6f> ```
GiteaMirror commented 2026-04-12 18:39:44 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 27, 2025):

The log is truncated on the right, use this instead: journalctl -u ollama --no-pager.

<!-- gh-comment-id:2833433208 --> @rick-github commented on GitHub (Apr 27, 2025): The log is truncated on the right, use this instead: `journalctl -u ollama --no-pager`.
GiteaMirror commented 2026-04-12 18:39:45 -05:00
Author
Owner
Copy Link

@alexhegit commented on GitHub (Apr 28, 2025):

journalctl -u ollama --no-pager.

Update log @rick-github

Apr 28 17:19:22 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:19:22 | 200 |    1.282913ms |       127.0.0.1 | HEAD     "/"
Apr 28 17:19:22 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:19:22 | 200 |     299.496µs |       127.0.0.1 | GET      "/api/ps"
Apr 28 17:20:23 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:23 | 200 |      35.256µs |       127.0.0.1 | HEAD     "/"
Apr 28 17:20:23 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:23 | 404 |     537.194µs |       127.0.0.1 | POST     "/api/show"
Apr 28 17:20:24 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:24 | 200 |  1.187817355s |       127.0.0.1 | POST     "/api/pull"
Apr 28 17:20:32 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:32 | 200 |      27.221µs |       127.0.0.1 | HEAD     "/"
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.815+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.827+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 28 17:20:32 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:32 | 200 |   48.295939ms |       127.0.0.1 | POST     "/api/show"
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.842+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.854+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.865+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.866+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=INFO source=server.go:105 msg="system memory" total="121.5 GiB" free="118.7 GiB" free_swap="8.0 GiB"
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128
Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=33 layers.offload=17 layers.split="" memory.available="[3.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.9 GiB" memory.required.partial="3.4 GiB" memory.required.kv="256.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="4.3 GiB" memory.weights.repeating="3.9 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="258.5 MiB" memory.graph.partial="677.5 MiB"
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 (version GGUF V3 (latest))
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   0:                       general.architecture str              = llama
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   1:                               general.type str              = model
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   3:                           general.finetune str              = Instruct
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   5:                         general.size_label str              = 8B
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   6:                            general.license str              = llama3.1
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   7:                               general.tags arr[str,6]       = ["facebook", "meta", "pytorch", "llam...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   8:                          general.languages arr[str,8]       = ["en", "de", "fr", "it", "pt", "hi", ...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv   9:                          llama.block_count u32              = 32
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  10:                       llama.context_length u32              = 131072
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  17:                          general.file_type u32              = 15
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv  28:               general.quantization_version u32              = 2
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type  f32:   66 tensors
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type q4_K:  193 tensors
Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type q6_K:   33 tensors
Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file format = GGUF V3 (latest)
Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file type   = Q4_K - Medium
Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file size   = 4.58 GiB (4.89 BPW)
Apr 28 17:20:33 GZ302EA ollama[2328]: load: special tokens cache size = 256
Apr 28 17:20:33 GZ302EA ollama[2328]: load: token to piece cache size = 0.7999 MB
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: arch             = llama
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: vocab_only       = 1
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: model type       = ?B
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: model params     = 8.03 B
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: general.name     = Meta Llama 3.1 8B Instruct
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: vocab type       = BPE
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: n_vocab          = 128256
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: n_merges         = 280147
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: BOS token        = 128000 '<|begin_of_text|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOS token        = 128009 '<|eot_id|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOT token        = 128009 '<|eot_id|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOM token        = 128008 '<|eom_id|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: LF token         = 198 'Ċ'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOG token        = 128008 '<|eom_id|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOG token        = 128009 '<|eot_id|>'
Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: max token length = 256
Apr 28 17:20:33 GZ302EA ollama[2328]: llama_model_load: vocab only - skipping tensors
Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.060+08:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 --ctx-size 2048 --batch-size 512 --n-gpu-layers 17 --threads 16 --parallel 1 --port 35989"
Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=sched.go:451 msg="loaded runners" count=1
Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding"
Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error"
Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.069+08:00 level=INFO source=runner.go:853 msg="starting go runner"
Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: found 1 ROCm devices:
Apr 28 17:20:34 GZ302EA ollama[2328]:   Device 0: AMD Radeon Graphics, gfx1100 (0x1100), VMM: no, Wave Size: 32
Apr 28 17:20:34 GZ302EA ollama[2328]: load_backend: loaded ROCm backend from /usr/local/lib/ollama/rocm/libggml-hip.so
Apr 28 17:20:34 GZ302EA ollama[2328]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so
Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.041+08:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.042+08:00 level=INFO source=runner.go:913 msg="Server listening on 127.0.0.1:35989"
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) - 62036 MiB free
Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.065+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model"
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 (version GGUF V3 (latest))
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   0:                       general.architecture str              = llama
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   1:                               general.type str              = model
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   3:                           general.finetune str              = Instruct
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   5:                         general.size_label str              = 8B
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   6:                            general.license str              = llama3.1
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   7:                               general.tags arr[str,6]       = ["facebook", "meta", "pytorch", "llam...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   8:                          general.languages arr[str,8]       = ["en", "de", "fr", "it", "pt", "hi", ...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv   9:                          llama.block_count u32              = 32
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  10:                       llama.context_length u32              = 131072
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  17:                          general.file_type u32              = 15
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv  28:               general.quantization_version u32              = 2
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type  f32:   66 tensors
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type q4_K:  193 tensors
Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type q6_K:   33 tensors
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file format = GGUF V3 (latest)
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file type   = Q4_K - Medium
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file size   = 4.58 GiB (4.89 BPW)
Apr 28 17:20:34 GZ302EA ollama[2328]: load: special tokens cache size = 256
Apr 28 17:20:34 GZ302EA ollama[2328]: load: token to piece cache size = 0.7999 MB
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: arch             = llama
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: vocab_only       = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ctx_train      = 131072
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd           = 4096
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_layer          = 32
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_head           = 32
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_head_kv        = 8
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_rot            = 128
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_swa            = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_swa_pattern    = 1
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_head_k    = 128
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_head_v    = 128
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_gqa            = 4
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_k_gqa     = 1024
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_v_gqa     = 1024
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_norm_eps       = 0.0e+00
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_norm_rms_eps   = 1.0e-05
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_clamp_kqv      = 0.0e+00
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_max_alibi_bias = 0.0e+00
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_logit_scale    = 0.0e+00
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_attn_scale     = 0.0e+00
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ff             = 14336
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_expert         = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_expert_used    = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: causal attn      = 1
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: pooling type     = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope type        = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope scaling     = linear
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: freq_base_train  = 500000.0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: freq_scale_train = 1
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ctx_orig_yarn  = 131072
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope_finetuned   = unknown
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_conv       = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_inner      = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_state      = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_dt_rank      = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_dt_b_c_rms   = 0
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: model type       = 8B
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: model params     = 8.03 B
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: general.name     = Meta Llama 3.1 8B Instruct
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: vocab type       = BPE
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_vocab          = 128256
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_merges         = 280147
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: BOS token        = 128000 '<|begin_of_text|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOS token        = 128009 '<|eot_id|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOT token        = 128009 '<|eot_id|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOM token        = 128008 '<|eom_id|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: LF token         = 198 'Ċ'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOG token        = 128008 '<|eom_id|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOG token        = 128009 '<|eot_id|>'
Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: max token length = 256
Apr 28 17:20:34 GZ302EA ollama[2328]: load_tensors: loading model tensors, this can take a while... (mmap = true)
Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: offloading 17 repeating layers to GPU
Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: offloaded 17/33 layers to GPU
Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors:        ROCm0 model buffer size =  2128.75 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors:   CPU_Mapped model buffer size =  4685.30 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: constructing llama_context
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_seq_max     = 1
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx         = 2048
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx_per_seq = 2048
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_batch       = 512
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ubatch      = 512
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: causal_attn   = 1
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: flash_attn    = 0
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: freq_base     = 500000.0
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: freq_scale    = 1
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context:        CPU  output buffer size =     0.50 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: init: kv_size = 2048, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1
Apr 28 17:20:35 GZ302EA ollama[2328]: init:      ROCm0 KV buffer size =   136.00 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: init:        CPU KV buffer size =   120.00 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context:      ROCm0 compute buffer size =   669.48 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context:  ROCm_Host compute buffer size =    12.01 MiB
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: graph nodes  = 1094
Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: graph splits = 170 (with bs=512), 3 (with bs=1)
Apr 28 17:20:35 GZ302EA ollama[2328]: time=2025-04-28T17:20:35.821+08:00 level=INFO source=server.go:619 msg="llama runner started in 2.76 seconds"
Apr 28 17:20:56 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:56 | 200 | 23.506355456s |       127.0.0.1 | POST     "/api/generate"
<!-- gh-comment-id:2834576384 --> @alexhegit commented on GitHub (Apr 28, 2025): > `journalctl -u ollama --no-pager`. Update log @rick-github ```shell Apr 28 17:19:22 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:19:22 | 200 | 1.282913ms | 127.0.0.1 | HEAD "/" Apr 28 17:19:22 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:19:22 | 200 | 299.496µs | 127.0.0.1 | GET "/api/ps" Apr 28 17:20:23 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:23 | 200 | 35.256µs | 127.0.0.1 | HEAD "/" Apr 28 17:20:23 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:23 | 404 | 537.194µs | 127.0.0.1 | POST "/api/show" Apr 28 17:20:24 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:24 | 200 | 1.187817355s | 127.0.0.1 | POST "/api/pull" Apr 28 17:20:32 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:32 | 200 | 27.221µs | 127.0.0.1 | HEAD "/" Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.815+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.827+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 28 17:20:32 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:32 | 200 | 48.295939ms | 127.0.0.1 | POST "/api/show" Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.842+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.854+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.865+08:00 level=WARN source=ggml.go:152 msg="key not found" key=general.alignment default=32 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.866+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.867+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.868+08:00 level=INFO source=server.go:105 msg="system memory" total="121.5 GiB" free="118.7 GiB" free_swap="8.0 GiB" Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.vision.block_count default=0 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.key_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=WARN source=ggml.go:152 msg="key not found" key=llama.attention.value_length default=128 Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=33 layers.offload=17 layers.split="" memory.available="[3.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.9 GiB" memory.required.partial="3.4 GiB" memory.required.kv="256.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="4.3 GiB" memory.weights.repeating="3.9 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="258.5 MiB" memory.graph.partial="677.5 MiB" Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 (version GGUF V3 (latest)) Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 0: general.architecture str = llama Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 1: general.type str = model Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 2: general.name str = Meta Llama 3.1 8B Instruct Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 3: general.finetune str = Instruct Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 4: general.basename str = Meta-Llama-3.1 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 5: general.size_label str = 8B Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 6: general.license str = llama3.1 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 7: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 8: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 9: llama.block_count u32 = 32 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 10: llama.context_length u32 = 131072 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 11: llama.embedding_length u32 = 4096 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 12: llama.feed_forward_length u32 = 14336 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 13: llama.attention.head_count u32 = 32 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 14: llama.attention.head_count_kv u32 = 8 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 15: llama.rope.freq_base f32 = 500000.000000 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 16: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 17: general.file_type u32 = 15 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 18: llama.vocab_size u32 = 128256 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 19: llama.rope.dimension_count u32 = 128 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 20: tokenizer.ggml.model str = gpt2 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 21: tokenizer.ggml.pre str = llama-bpe Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 22: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 128000 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 128009 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 27: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ... Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - kv 28: general.quantization_version u32 = 2 Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type f32: 66 tensors Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type q4_K: 193 tensors Apr 28 17:20:32 GZ302EA ollama[2328]: llama_model_loader: - type q6_K: 33 tensors Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file format = GGUF V3 (latest) Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file type = Q4_K - Medium Apr 28 17:20:32 GZ302EA ollama[2328]: print_info: file size = 4.58 GiB (4.89 BPW) Apr 28 17:20:33 GZ302EA ollama[2328]: load: special tokens cache size = 256 Apr 28 17:20:33 GZ302EA ollama[2328]: load: token to piece cache size = 0.7999 MB Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: arch = llama Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: vocab_only = 1 Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: model type = ?B Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: model params = 8.03 B Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: general.name = Meta Llama 3.1 8B Instruct Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: vocab type = BPE Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: n_vocab = 128256 Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: n_merges = 280147 Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: BOS token = 128000 '<|begin_of_text|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOS token = 128009 '<|eot_id|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOT token = 128009 '<|eot_id|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOM token = 128008 '<|eom_id|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: LF token = 198 'Ċ' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOG token = 128008 '<|eom_id|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: EOG token = 128009 '<|eot_id|>' Apr 28 17:20:33 GZ302EA ollama[2328]: print_info: max token length = 256 Apr 28 17:20:33 GZ302EA ollama[2328]: llama_model_load: vocab only - skipping tensors Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.060+08:00 level=INFO source=server.go:405 msg="starting llama server" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 --ctx-size 2048 --batch-size 512 --n-gpu-layers 17 --threads 16 --parallel 1 --port 35989" Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=sched.go:451 msg="loaded runners" count=1 Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=server.go:580 msg="waiting for llama runner to start responding" Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.061+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server error" Apr 28 17:20:33 GZ302EA ollama[2328]: time=2025-04-28T17:20:33.069+08:00 level=INFO source=runner.go:853 msg="starting go runner" Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no Apr 28 17:20:34 GZ302EA ollama[2328]: ggml_cuda_init: found 1 ROCm devices: Apr 28 17:20:34 GZ302EA ollama[2328]: Device 0: AMD Radeon Graphics, gfx1100 (0x1100), VMM: no, Wave Size: 32 Apr 28 17:20:34 GZ302EA ollama[2328]: load_backend: loaded ROCm backend from /usr/local/lib/ollama/rocm/libggml-hip.so Apr 28 17:20:34 GZ302EA ollama[2328]: load_backend: loaded CPU backend from /usr/local/lib/ollama/libggml-cpu-icelake.so Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.041+08:00 level=INFO source=ggml.go:109 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.AVX512=1 CPU.0.AVX512_VBMI=1 CPU.0.AVX512_VNNI=1 CPU.0.LLAMAFILE=1 CPU.1.LLAMAFILE=1 ROCm.0.NO_VMM=1 ROCm.0.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.042+08:00 level=INFO source=runner.go:913 msg="Server listening on 127.0.0.1:35989" Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_load_from_file_impl: using device ROCm0 (AMD Radeon Graphics) - 62036 MiB free Apr 28 17:20:34 GZ302EA ollama[2328]: time=2025-04-28T17:20:34.065+08:00 level=INFO source=server.go:614 msg="waiting for server to become available" status="llm server loading model" Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29 (version GGUF V3 (latest)) Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 0: general.architecture str = llama Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 1: general.type str = model Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 2: general.name str = Meta Llama 3.1 8B Instruct Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 3: general.finetune str = Instruct Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 4: general.basename str = Meta-Llama-3.1 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 5: general.size_label str = 8B Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 6: general.license str = llama3.1 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 7: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 8: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 9: llama.block_count u32 = 32 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 10: llama.context_length u32 = 131072 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 11: llama.embedding_length u32 = 4096 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 12: llama.feed_forward_length u32 = 14336 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 13: llama.attention.head_count u32 = 32 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 14: llama.attention.head_count_kv u32 = 8 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 15: llama.rope.freq_base f32 = 500000.000000 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 16: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 17: general.file_type u32 = 15 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 18: llama.vocab_size u32 = 128256 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 19: llama.rope.dimension_count u32 = 128 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 20: tokenizer.ggml.model str = gpt2 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 21: tokenizer.ggml.pre str = llama-bpe Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 22: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 128000 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 128009 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 27: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ... Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - kv 28: general.quantization_version u32 = 2 Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type f32: 66 tensors Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type q4_K: 193 tensors Apr 28 17:20:34 GZ302EA ollama[2328]: llama_model_loader: - type q6_K: 33 tensors Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file format = GGUF V3 (latest) Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file type = Q4_K - Medium Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: file size = 4.58 GiB (4.89 BPW) Apr 28 17:20:34 GZ302EA ollama[2328]: load: special tokens cache size = 256 Apr 28 17:20:34 GZ302EA ollama[2328]: load: token to piece cache size = 0.7999 MB Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: arch = llama Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: vocab_only = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ctx_train = 131072 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd = 4096 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_layer = 32 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_head = 32 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_head_kv = 8 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_rot = 128 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_swa = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_swa_pattern = 1 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_head_k = 128 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_head_v = 128 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_gqa = 4 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_k_gqa = 1024 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_embd_v_gqa = 1024 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_norm_eps = 0.0e+00 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_norm_rms_eps = 1.0e-05 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_clamp_kqv = 0.0e+00 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_max_alibi_bias = 0.0e+00 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_logit_scale = 0.0e+00 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: f_attn_scale = 0.0e+00 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ff = 14336 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_expert = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_expert_used = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: causal attn = 1 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: pooling type = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope type = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope scaling = linear Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: freq_base_train = 500000.0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: freq_scale_train = 1 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_ctx_orig_yarn = 131072 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: rope_finetuned = unknown Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_conv = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_inner = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_d_state = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_dt_rank = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: ssm_dt_b_c_rms = 0 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: model type = 8B Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: model params = 8.03 B Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: general.name = Meta Llama 3.1 8B Instruct Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: vocab type = BPE Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_vocab = 128256 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: n_merges = 280147 Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: BOS token = 128000 '<|begin_of_text|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOS token = 128009 '<|eot_id|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOT token = 128009 '<|eot_id|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOM token = 128008 '<|eom_id|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: LF token = 198 'Ċ' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOG token = 128008 '<|eom_id|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: EOG token = 128009 '<|eot_id|>' Apr 28 17:20:34 GZ302EA ollama[2328]: print_info: max token length = 256 Apr 28 17:20:34 GZ302EA ollama[2328]: load_tensors: loading model tensors, this can take a while... (mmap = true) Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: offloading 17 repeating layers to GPU Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: offloaded 17/33 layers to GPU Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: ROCm0 model buffer size = 2128.75 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: load_tensors: CPU_Mapped model buffer size = 4685.30 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: constructing llama_context Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_seq_max = 1 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx = 2048 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx_per_seq = 2048 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_batch = 512 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ubatch = 512 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: causal_attn = 1 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: flash_attn = 0 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: freq_base = 500000.0 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: freq_scale = 1 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: n_ctx_per_seq (2048) < n_ctx_train (131072) -- the full capacity of the model will not be utilized Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: CPU output buffer size = 0.50 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: init: kv_size = 2048, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 32, can_shift = 1 Apr 28 17:20:35 GZ302EA ollama[2328]: init: ROCm0 KV buffer size = 136.00 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: init: CPU KV buffer size = 120.00 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: ROCm0 compute buffer size = 669.48 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: ROCm_Host compute buffer size = 12.01 MiB Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: graph nodes = 1094 Apr 28 17:20:35 GZ302EA ollama[2328]: llama_context: graph splits = 170 (with bs=512), 3 (with bs=1) Apr 28 17:20:35 GZ302EA ollama[2328]: time=2025-04-28T17:20:35.821+08:00 level=INFO source=server.go:619 msg="llama runner started in 2.76 seconds" Apr 28 17:20:56 GZ302EA ollama[2328]: [GIN] 2025/04/28 - 17:20:56 | 200 | 23.506355456s | 127.0.0.1 | POST "/api/generate" ```
GiteaMirror commented 2026-04-12 18:39:45 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 28, 2025):

Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=INFO source=server.go:138
 msg=offload  library=rocm layers.requested=-1 layers.model=33 layers.offload=17 layers.split=""
 memory.available="[3.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.9 GiB"
 memory.required.partial="3.4 GiB" memory.required.kv="256.0 MiB" memory.required.allocations="[3.4 GiB]"
 memory.weights.total="4.3 GiB" memory.weights.repeating="3.9 GiB" memory.weights.nonrepeating="411.0 MiB"
 memory.graph.full="258.5 MiB" memory.graph.partial="677.5 MiB"

ollama sees the VRAM, but it doesn't support GTT so that's not included in the memory calculations. See here for a PR that adds GTT support.

<!-- gh-comment-id:2834622304 --> @rick-github commented on GitHub (Apr 28, 2025): ``` Apr 28 17:20:32 GZ302EA ollama[2328]: time=2025-04-28T17:20:32.869+08:00 level=INFO source=server.go:138 msg=offload library=rocm layers.requested=-1 layers.model=33 layers.offload=17 layers.split="" memory.available="[3.5 GiB]" memory.gpu_overhead="0 B" memory.required.full="5.9 GiB" memory.required.partial="3.4 GiB" memory.required.kv="256.0 MiB" memory.required.allocations="[3.4 GiB]" memory.weights.total="4.3 GiB" memory.weights.repeating="3.9 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="258.5 MiB" memory.graph.partial="677.5 MiB" ``` ollama sees the VRAM, but it doesn't support GTT so that's not included in the memory calculations. See [here](https://github.com/ollama/ollama/issues/5471) for a PR that adds GTT support.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#6853
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?