Ollama running Granite 4 Hybrid reports wrong memory estimation #8428

New Issue

GiteaMirror · 2025-11-12T14:41:45-06:00

GiteaMirror commented

2025-11-12 14:41:45 -06:00

Originally created by @rrtt217 on GitHub (Oct 19, 2025).

What is the issue?

granite4:micro-h, granite4:tiny-h, granite4:small-h are hybrid models that combines Mamba2 with conventional transformers architecture.
When I run one of the models (as for me, specifically granite4:micro-h) with a large context size (e.g 65536), ollama says:

model requires more system memory (5.5 GiB) than is available (3.4 GiB)

In journalctl -xeu ollama log:

msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B"

After freeing some memory and then running the same model with the same context 65536, the actual memory usage is about

Relevant log output

10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: loaded meta data with 42 key-value pairs and 506 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 (version GGUF V3 (latest))
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   0:                       general.architecture str              = granitehybrid
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   1:                               general.type str              = model
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   2:                               general.name str              = Granite 4.0 H Micro
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   3:                         general.size_label str              = 3.2B
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   4:                            general.license str              = apache-2.0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   5:                               general.tags arr[str,2]       = ["language", "granite-4.0"]
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   6:                  granitehybrid.block_count u32              = 40
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   7:               granitehybrid.context_length u32              = 1048576
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   8:             granitehybrid.embedding_length u32              = 2048
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   9:          granitehybrid.feed_forward_length u32              = 8192
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  10:         granitehybrid.attention.head_count u32              = 32
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  11:      granitehybrid.attention.head_count_kv arr[i32,40]      = [0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, ...
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  12:               granitehybrid.rope.freq_base f32              = 10000.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  13: granitehybrid.attention.layer_norm_rms_epsilon f32              = 0.000010
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  14:                 granitehybrid.expert_count u32              = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  15:            granitehybrid.expert_used_count u32              = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  16:                   granitehybrid.vocab_size u32              = 100352
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  17:         granitehybrid.rope.dimension_count u32              = 64
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  18:              granitehybrid.attention.scale f32              = 0.015625
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  19:              granitehybrid.embedding_scale f32              = 12.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  20:               granitehybrid.residual_scale f32              = 0.220000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  21:                  granitehybrid.logit_scale f32              = 8.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  22: granitehybrid.expert_shared_feed_forward_length u32              = 8192
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  23:              granitehybrid.ssm.conv_kernel u32              = 4
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  24:               granitehybrid.ssm.state_size u32              = 128
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  25:              granitehybrid.ssm.group_count u32              = 1
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  26:               granitehybrid.ssm.inner_size u32              = 4096
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  27:           granitehybrid.ssm.time_step_rank u32              = 64
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  28:       granitehybrid.rope.scaling.finetuned bool             = false
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  29:                       tokenizer.ggml.model str              = gpt2
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  30:                         tokenizer.ggml.pre str              = dbrx
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  31:                      tokenizer.ggml.tokens arr[str,100352]  = ["!", "\"", "#", "$", "%", "&", "'", ...
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  32:                  tokenizer.ggml.token_type arr[i32,100352]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  33:                      tokenizer.ggml.merges arr[str,100000]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  34:                tokenizer.ggml.bos_token_id u32              = 100257
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  35:                tokenizer.ggml.eos_token_id u32              = 100257
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  36:            tokenizer.ggml.unknown_token_id u32              = 100269
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  37:            tokenizer.ggml.padding_token_id u32              = 100256
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  38:               tokenizer.ggml.add_bos_token bool             = false
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  39:                    tokenizer.chat_template str              = {%- set tools_system_message_prefix =...
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  40:               general.quantization_version u32              = 2
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  41:                          general.file_type u32              = 15
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type  f32:  297 tensors
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type q4_K:  186 tensors
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type q6_K:   23 tensors
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file format = GGUF V3 (latest)
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file type   = Q4_K - Medium
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file size   = 1.81 GiB (4.86 BPW)
10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: printing all EOG tokens:
10月 19 15:17:54 VM9890-ubuntu ollama[89]: load:   - 100257 ('<|end_of_text|>')
10月 19 15:17:54 VM9890-ubuntu ollama[89]: load:   - 100261 ('<|fim_pad|>')
10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: special tokens cache size = 96
10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: token to piece cache size = 0.6152 MB
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: arch             = granitehybrid
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: vocab_only       = 1
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_conv       = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_inner      = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_state      = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_dt_rank      = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_n_group      = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_dt_b_c_rms   = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: model type       = ?B
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: model params     = 3.19 B
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: general.name     = Granite 4.0 H Micro
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_embedding_scale = 0.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_residual_scale  = 0.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_attention_scale = 0.000000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_ff_shexp        = 0
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: vocab type       = BPE
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_vocab          = 100352
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_merges         = 100000
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: BOS token        = 100257 '<|end_of_text|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOS token        = 100257 '<|end_of_text|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOT token        = 100257 '<|end_of_text|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: UNK token        = 100269 '<|unk|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: PAD token        = 100256 '<|pad|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: LF token         = 198 'Ċ'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM PRE token    = 100258 '<|fim_prefix|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM SUF token    = 100260 '<|fim_suffix|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM MID token    = 100259 '<|fim_middle|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM PAD token    = 100261 '<|fim_pad|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOG token        = 100257 '<|end_of_text|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOG token        = 100261 '<|fim_pad|>'
10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: max token length = 256
10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_load: vocab only - skipping tensors
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.339+08:00 level=INFO source=server.go:216 msg="enabling flash attention"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.339+08:00 level=INFO source=server.go:400 msg="starting runner" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 --port 35709"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.343+08:00 level=INFO source=server.go:505 msg="system memory" total="6.0 GiB" free="3.4 GiB" free_swap="0 B"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.345+08:00 level=WARN source=server.go:540 msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.345+08:00 level=INFO source=sched.go:450 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 error="model requires more system memory (5.5 GiB) than is available (3.4 GiB)"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.359+08:00 level=INFO source=runner.go:893 msg="starting go runner"
10月 19 15:17:54 VM9890-ubuntu ollama[89]: [GIN] 2025/10/19 - 15:17:54 | 500 |  488.865497ms |       127.0.0.1 | POST     "/api/generate"
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: loaded meta data with 42 key-value pairs and 506 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 (version GGUF V3 (latest))
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   0:                       general.architecture str              = granitehybrid
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   1:                               general.type str              = model
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   2:                               general.name str              = Granite 4.0 H Micro
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   3:                         general.size_label str              = 3.2B
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   4:                            general.license str              = apache-2.0
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   5:                               general.tags arr[str,2]       = ["language", "granite-4.0"]
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   6:                  granitehybrid.block_count u32              = 40
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   7:               granitehybrid.context_length u32              = 1048576
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   8:             granitehybrid.embedding_length u32              = 2048
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv   9:          granitehybrid.feed_forward_length u32              = 8192
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  10:         granitehybrid.attention.head_count u32              = 32
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  11:      granitehybrid.attention.head_count_kv arr[i32,40]      = [0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, ...
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  12:               granitehybrid.rope.freq_base f32              = 10000.000000
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  13: granitehybrid.attention.layer_norm_rms_epsilon f32              = 0.000010
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  14:                 granitehybrid.expert_count u32              = 0
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  15:            granitehybrid.expert_used_count u32              = 0
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  16:                   granitehybrid.vocab_size u32              = 100352
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  17:         granitehybrid.rope.dimension_count u32              = 64
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  18:              granitehybrid.attention.scale f32              = 0.015625
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  19:              granitehybrid.embedding_scale f32              = 12.000000
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  20:               granitehybrid.residual_scale f32              = 0.220000
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  21:                  granitehybrid.logit_scale f32              = 8.000000
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  22: granitehybrid.expert_shared_feed_forward_length u32              = 8192
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  23:              granitehybrid.ssm.conv_kernel u32              = 4
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  24:               granitehybrid.ssm.state_size u32              = 128
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  25:              granitehybrid.ssm.group_count u32              = 1
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  26:               granitehybrid.ssm.inner_size u32              = 4096
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  27:           granitehybrid.ssm.time_step_rank u32              = 64
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  28:       granitehybrid.rope.scaling.finetuned bool             = false
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  29:                       tokenizer.ggml.model str              = gpt2
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  30:                         tokenizer.ggml.pre str              = dbrx
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  31:                      tokenizer.ggml.tokens arr[str,100352]  = ["!", "\"", "#", "$", "%", "&", "'", ...
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  32:                  tokenizer.ggml.token_type arr[i32,100352]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  33:                      tokenizer.ggml.merges arr[str,100000]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  34:                tokenizer.ggml.bos_token_id u32              = 100257
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  35:                tokenizer.ggml.eos_token_id u32              = 100257
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  36:            tokenizer.ggml.unknown_token_id u32              = 100269
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  37:            tokenizer.ggml.padding_token_id u32              = 100256
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  38:               tokenizer.ggml.add_bos_token bool             = false
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  39:                    tokenizer.chat_template str              = {%- set tools_system_message_prefix =...
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  40:               general.quantization_version u32              = 2
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv  41:                          general.file_type u32              = 15
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type  f32:  297 tensors
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type q4_K:  186 tensors
10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type q6_K:   23 tensors
10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file format = GGUF V3 (latest)
10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file type   = Q4_K - Medium
10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file size   = 1.81 GiB (4.86 BPW)
10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: printing all EOG tokens:
10月 19 15:17:56 VM9890-ubuntu ollama[89]: load:   - 100257 ('<|end_of_text|>')
10月 19 15:17:56 VM9890-ubuntu ollama[89]: load:   - 100261 ('<|fim_pad|>')
10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: special tokens cache size = 96
10月 19 15:17:57 VM9890-ubuntu ollama[89]: load: token to piece cache size = 0.6152 MB
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: arch             = granitehybrid
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: vocab_only       = 1
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_conv       = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_inner      = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_state      = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_dt_rank      = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_n_group      = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_dt_b_c_rms   = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: model type       = ?B
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: model params     = 3.19 B
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: general.name     = Granite 4.0 H Micro
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_embedding_scale = 0.000000
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_residual_scale  = 0.000000
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_attention_scale = 0.000000
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_ff_shexp        = 0
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: vocab type       = BPE
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_vocab          = 100352
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_merges         = 100000
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: BOS token        = 100257 '<|end_of_text|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOS token        = 100257 '<|end_of_text|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOT token        = 100257 '<|end_of_text|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: UNK token        = 100269 '<|unk|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: PAD token        = 100256 '<|pad|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: LF token         = 198 'Ċ'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM PRE token    = 100258 '<|fim_prefix|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM SUF token    = 100260 '<|fim_suffix|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM MID token    = 100259 '<|fim_middle|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM PAD token    = 100261 '<|fim_pad|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOG token        = 100257 '<|end_of_text|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOG token        = 100261 '<|fim_pad|>'
10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: max token length = 256
10月 19 15:17:57 VM9890-ubuntu ollama[89]: llama_model_load: vocab only - skipping tensors
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.086+08:00 level=INFO source=server.go:216 msg="enabling flash attention"
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.086+08:00 level=INFO source=server.go:400 msg="starting runner" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 --port 41475"
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.092+08:00 level=INFO source=server.go:505 msg="system memory" total="6.0 GiB" free="3.4 GiB" free_swap="0 B"
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.097+08:00 level=WARN source=server.go:540 msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B"
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.097+08:00 level=INFO source=sched.go:450 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 error="model requires more system memory (5.5 GiB) than is available (3.4 GiB)"
10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.126+08:00 level=INFO source=runner.go:893 msg="starting go runner"

OS

Linux

GPU

No response

CPU

Intel

Ollama version

0.12.6

Originally created by @rrtt217 on GitHub (Oct 19, 2025). ### What is the issue? `granite4:micro-h`, `granite4:tiny-h`, `granite4:small-h` are hybrid models that combines Mamba2 with conventional transformers architecture. When I run one of the models (as for me, specifically `granite4:micro-h`) with a large context size (e.g 65536), ollama says: model requires more system memory (5.5 GiB) than is available (3.4 GiB) In `journalctl -xeu ollama` log: msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B" After freeing some memory and then running the same model with the same context `65536`, the actual memory usage is about ### Relevant log output ```shell 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: loaded meta data with 42 key-value pairs and 506 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 (version GGUF V3 (latest)) 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 0: general.architecture str = granitehybrid 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 1: general.type str = model 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 2: general.name str = Granite 4.0 H Micro 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 3: general.size_label str = 3.2B 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 4: general.license str = apache-2.0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 5: general.tags arr[str,2] = ["language", "granite-4.0"] 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 6: granitehybrid.block_count u32 = 40 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 7: granitehybrid.context_length u32 = 1048576 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 8: granitehybrid.embedding_length u32 = 2048 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 9: granitehybrid.feed_forward_length u32 = 8192 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 10: granitehybrid.attention.head_count u32 = 32 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 11: granitehybrid.attention.head_count_kv arr[i32,40] = [0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, ... 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 12: granitehybrid.rope.freq_base f32 = 10000.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 13: granitehybrid.attention.layer_norm_rms_epsilon f32 = 0.000010 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 14: granitehybrid.expert_count u32 = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 15: granitehybrid.expert_used_count u32 = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 16: granitehybrid.vocab_size u32 = 100352 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 17: granitehybrid.rope.dimension_count u32 = 64 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 18: granitehybrid.attention.scale f32 = 0.015625 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 19: granitehybrid.embedding_scale f32 = 12.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 20: granitehybrid.residual_scale f32 = 0.220000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 21: granitehybrid.logit_scale f32 = 8.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 22: granitehybrid.expert_shared_feed_forward_length u32 = 8192 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 23: granitehybrid.ssm.conv_kernel u32 = 4 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 24: granitehybrid.ssm.state_size u32 = 128 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 25: granitehybrid.ssm.group_count u32 = 1 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 26: granitehybrid.ssm.inner_size u32 = 4096 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 27: granitehybrid.ssm.time_step_rank u32 = 64 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 28: granitehybrid.rope.scaling.finetuned bool = false 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 30: tokenizer.ggml.pre str = dbrx 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,100352] = ["!", "\"", "#", "$", "%", "&", "'", ... 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,100352] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,100000] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 100257 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 100257 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 36: tokenizer.ggml.unknown_token_id u32 = 100269 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 100256 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 38: tokenizer.ggml.add_bos_token bool = false 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 39: tokenizer.chat_template str = {%- set tools_system_message_prefix =... 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 40: general.quantization_version u32 = 2 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 41: general.file_type u32 = 15 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type f32: 297 tensors 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type q4_K: 186 tensors 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_loader: - type q6_K: 23 tensors 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file format = GGUF V3 (latest) 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file type = Q4_K - Medium 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: file size = 1.81 GiB (4.86 BPW) 10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: printing all EOG tokens: 10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: - 100257 ('<|end_of_text|>') 10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: - 100261 ('<|fim_pad|>') 10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: special tokens cache size = 96 10月 19 15:17:54 VM9890-ubuntu ollama[89]: load: token to piece cache size = 0.6152 MB 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: arch = granitehybrid 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: vocab_only = 1 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_conv = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_inner = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_d_state = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_dt_rank = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_n_group = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: ssm_dt_b_c_rms = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: model type = ?B 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: model params = 3.19 B 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: general.name = Granite 4.0 H Micro 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_embedding_scale = 0.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_residual_scale = 0.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: f_attention_scale = 0.000000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_ff_shexp = 0 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: vocab type = BPE 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_vocab = 100352 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: n_merges = 100000 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: BOS token = 100257 '<|end_of_text|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOS token = 100257 '<|end_of_text|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOT token = 100257 '<|end_of_text|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: UNK token = 100269 '<|unk|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: PAD token = 100256 '<|pad|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: LF token = 198 'Ċ' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM PRE token = 100258 '<|fim_prefix|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM SUF token = 100260 '<|fim_suffix|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM MID token = 100259 '<|fim_middle|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: FIM PAD token = 100261 '<|fim_pad|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOG token = 100257 '<|end_of_text|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: EOG token = 100261 '<|fim_pad|>' 10月 19 15:17:54 VM9890-ubuntu ollama[89]: print_info: max token length = 256 10月 19 15:17:54 VM9890-ubuntu ollama[89]: llama_model_load: vocab only - skipping tensors 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.339+08:00 level=INFO source=server.go:216 msg="enabling flash attention" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.339+08:00 level=INFO source=server.go:400 msg="starting runner" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 --port 35709" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.343+08:00 level=INFO source=server.go:505 msg="system memory" total="6.0 GiB" free="3.4 GiB" free_swap="0 B" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.345+08:00 level=WARN source=server.go:540 msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.345+08:00 level=INFO source=sched.go:450 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 error="model requires more system memory (5.5 GiB) than is available (3.4 GiB)" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:54.359+08:00 level=INFO source=runner.go:893 msg="starting go runner" 10月 19 15:17:54 VM9890-ubuntu ollama[89]: [GIN] 2025/10/19 - 15:17:54 | 500 | 488.865497ms | 127.0.0.1 | POST "/api/generate" 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: loaded meta data with 42 key-value pairs and 506 tensors from /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 (version GGUF V3 (latest)) 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 0: general.architecture str = granitehybrid 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 1: general.type str = model 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 2: general.name str = Granite 4.0 H Micro 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 3: general.size_label str = 3.2B 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 4: general.license str = apache-2.0 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 5: general.tags arr[str,2] = ["language", "granite-4.0"] 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 6: granitehybrid.block_count u32 = 40 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 7: granitehybrid.context_length u32 = 1048576 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 8: granitehybrid.embedding_length u32 = 2048 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 9: granitehybrid.feed_forward_length u32 = 8192 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 10: granitehybrid.attention.head_count u32 = 32 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 11: granitehybrid.attention.head_count_kv arr[i32,40] = [0, 0, 0, 0, 0, 8, 0, 0, 0, 0, 0, 0, ... 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 12: granitehybrid.rope.freq_base f32 = 10000.000000 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 13: granitehybrid.attention.layer_norm_rms_epsilon f32 = 0.000010 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 14: granitehybrid.expert_count u32 = 0 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 15: granitehybrid.expert_used_count u32 = 0 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 16: granitehybrid.vocab_size u32 = 100352 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 17: granitehybrid.rope.dimension_count u32 = 64 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 18: granitehybrid.attention.scale f32 = 0.015625 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 19: granitehybrid.embedding_scale f32 = 12.000000 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 20: granitehybrid.residual_scale f32 = 0.220000 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 21: granitehybrid.logit_scale f32 = 8.000000 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 22: granitehybrid.expert_shared_feed_forward_length u32 = 8192 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 23: granitehybrid.ssm.conv_kernel u32 = 4 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 24: granitehybrid.ssm.state_size u32 = 128 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 25: granitehybrid.ssm.group_count u32 = 1 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 26: granitehybrid.ssm.inner_size u32 = 4096 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 27: granitehybrid.ssm.time_step_rank u32 = 64 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 28: granitehybrid.rope.scaling.finetuned bool = false 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 30: tokenizer.ggml.pre str = dbrx 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,100352] = ["!", "\"", "#", "$", "%", "&", "'", ... 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,100352] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,100000] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 34: tokenizer.ggml.bos_token_id u32 = 100257 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 35: tokenizer.ggml.eos_token_id u32 = 100257 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 36: tokenizer.ggml.unknown_token_id u32 = 100269 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 37: tokenizer.ggml.padding_token_id u32 = 100256 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 38: tokenizer.ggml.add_bos_token bool = false 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 39: tokenizer.chat_template str = {%- set tools_system_message_prefix =... 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 40: general.quantization_version u32 = 2 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - kv 41: general.file_type u32 = 15 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type f32: 297 tensors 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type q4_K: 186 tensors 10月 19 15:17:56 VM9890-ubuntu ollama[89]: llama_model_loader: - type q6_K: 23 tensors 10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file format = GGUF V3 (latest) 10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file type = Q4_K - Medium 10月 19 15:17:56 VM9890-ubuntu ollama[89]: print_info: file size = 1.81 GiB (4.86 BPW) 10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: printing all EOG tokens: 10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: - 100257 ('<|end_of_text|>') 10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: - 100261 ('<|fim_pad|>') 10月 19 15:17:56 VM9890-ubuntu ollama[89]: load: special tokens cache size = 96 10月 19 15:17:57 VM9890-ubuntu ollama[89]: load: token to piece cache size = 0.6152 MB 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: arch = granitehybrid 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: vocab_only = 1 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_conv = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_inner = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_d_state = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_dt_rank = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_n_group = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: ssm_dt_b_c_rms = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: model type = ?B 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: model params = 3.19 B 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: general.name = Granite 4.0 H Micro 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_embedding_scale = 0.000000 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_residual_scale = 0.000000 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: f_attention_scale = 0.000000 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_ff_shexp = 0 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: vocab type = BPE 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_vocab = 100352 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: n_merges = 100000 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: BOS token = 100257 '<|end_of_text|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOS token = 100257 '<|end_of_text|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOT token = 100257 '<|end_of_text|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: UNK token = 100269 '<|unk|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: PAD token = 100256 '<|pad|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: LF token = 198 'Ċ' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM PRE token = 100258 '<|fim_prefix|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM SUF token = 100260 '<|fim_suffix|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM MID token = 100259 '<|fim_middle|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: FIM PAD token = 100261 '<|fim_pad|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOG token = 100257 '<|end_of_text|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: EOG token = 100261 '<|fim_pad|>' 10月 19 15:17:57 VM9890-ubuntu ollama[89]: print_info: max token length = 256 10月 19 15:17:57 VM9890-ubuntu ollama[89]: llama_model_load: vocab only - skipping tensors 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.086+08:00 level=INFO source=server.go:216 msg="enabling flash attention" 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.086+08:00 level=INFO source=server.go:400 msg="starting runner" cmd="/usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 --port 41475" 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.092+08:00 level=INFO source=server.go:505 msg="system memory" total="6.0 GiB" free="3.4 GiB" free_swap="0 B" 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.097+08:00 level=WARN source=server.go:540 msg="model request too large for system" requested="5.5 GiB" available="3.4 GiB" total="6.0 GiB" free="3.4 GiB" swap="0 B" 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.097+08:00 level=INFO source=sched.go:450 msg="Load failed" model=/usr/share/ollama/.ollama/models/blobs/sha256-2e81b93552f5aef967d68b0a1e45703177f685d528f3f10b7e4da2e3166aa700 error="model requires more system memory (5.5 GiB) than is available (3.4 GiB)" 10月 19 15:17:57 VM9890-ubuntu ollama[89]: time=2025-10-19T15:17:57.126+08:00 level=INFO source=runner.go:893 msg="starting go runner" ``` ### OS Linux ### GPU _No response_ ### CPU Intel ### Ollama version 0.12.6

GiteaMirror added the

bug

label 2025-11-12 14:41:45 -06:00

GiteaMirror referenced this issue

2025-11-12 16:14:19 -06:00

[PR #8428] [MERGED] Fix absolute path names + gguf detection #11443

Sign in to join this conversation.

Branches Tags

main

parth/olmo3-thinking-renderer

drifkin/responses

jmorganca/write-int32

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

nicole/olmo-model

jmorganca/cmd-eval

grace/deepseek-parser

nicole/nomic-embed-text-fix

jessegross/multi_chunk_reserve

grace/deepseek-renderer-parser

hoyyeva/proxy-ollama-serve

hoyyeva/model-capabilities-after-download

mxyng/ropeopts

mxyng/lint-2

nicole/nomic-embed-text-v2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

hoyyeva/app-ollama-apis

mxyng/expand-path

mxyng/environ-2

hoyyeva/thinking-rendering

nicole/truncation

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama-ollama#8428