github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #15834] qwen3.6-35b-a3b/gemma-4-E2B-it-GGUF not loading properly on ollama #72152

New Issue
Open
opened 2026-05-05 03:33:15 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-05 03:33:15 -05:00
Owner
Copy Link

Originally created by @orKL3mlz on GitHub (Apr 27, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15834

What is the issue?

I can use the model qwen3.6:35b without any issues. However, when trying to run this model hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M, ollama returns unknown model architecture: 'qwen35moe'

Full loading log below

ollama  | llama_model_loader: loaded meta data with 54 key-value pairs and 733 tensors from /root/.ollama/models/blobs/sha256-ac0e2c1189e055faa36eff361580e79c5bd6f8e76bffb4ce547f167d53e31a61 (version GGUF V3 (latest))
ollama  | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
ollama  | llama_model_loader: - kv   0:                       general.architecture str              = qwen35moe
ollama  | llama_model_loader: - kv   1:                               general.type str              = model
ollama  | llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
ollama  | llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
ollama  | llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
ollama  | llama_model_loader: - kv   5:                               general.name str              = Qwen3.6-35B-A3B
ollama  | llama_model_loader: - kv   6:                           general.basename str              = Qwen3.6-35B-A3B
ollama  | llama_model_loader: - kv   7:                       general.quantized_by str              = Unsloth
ollama  | llama_model_loader: - kv   8:                         general.size_label str              = 35B-A3B
ollama  | llama_model_loader: - kv   9:                            general.license str              = apache-2.0
ollama  | llama_model_loader: - kv  10:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3.6-3...
ollama  | llama_model_loader: - kv  11:                           general.repo_url str              = https://huggingface.co/unsloth
ollama  | llama_model_loader: - kv  12:                   general.base_model.count u32              = 1
ollama  | llama_model_loader: - kv  13:                  general.base_model.0.name str              = Qwen3.6 35B A3B
ollama  | llama_model_loader: - kv  14:          general.base_model.0.organization str              = Qwen
ollama  | llama_model_loader: - kv  15:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3.6-3...
ollama  | llama_model_loader: - kv  16:                               general.tags arr[str,3]       = ["qwen3_5_moe", "qwen", "image-text-t...
ollama  | llama_model_loader: - kv  17:                      qwen35moe.block_count u32              = 40
ollama  | llama_model_loader: - kv  18:                   qwen35moe.context_length u32              = 262144
ollama  | llama_model_loader: - kv  19:                 qwen35moe.embedding_length u32              = 2048
ollama  | llama_model_loader: - kv  20:             qwen35moe.attention.head_count u32              = 16
ollama  | llama_model_loader: - kv  21:          qwen35moe.attention.head_count_kv u32              = 2
ollama  | llama_model_loader: - kv  22:          qwen35moe.rope.dimension_sections arr[i32,4]       = [11, 11, 10, 0]
ollama  | llama_model_loader: - kv  23:                   qwen35moe.rope.freq_base f32              = 10000000.000000
ollama  | llama_model_loader: - kv  24: qwen35moe.attention.layer_norm_rms_epsilon f32              = 0.000001
ollama  | llama_model_loader: - kv  25:                     qwen35moe.expert_count u32              = 256
ollama  | llama_model_loader: - kv  26:                qwen35moe.expert_used_count u32              = 8
ollama  | llama_model_loader: - kv  27:             qwen35moe.attention.key_length u32              = 256
ollama  | llama_model_loader: - kv  28:           qwen35moe.attention.value_length u32              = 256
ollama  | llama_model_loader: - kv  29:       qwen35moe.expert_feed_forward_length u32              = 512
ollama  | llama_model_loader: - kv  30: qwen35moe.expert_shared_feed_forward_length u32              = 512
ollama  | llama_model_loader: - kv  31:                  qwen35moe.ssm.conv_kernel u32              = 4
ollama  | llama_model_loader: - kv  32:                   qwen35moe.ssm.state_size u32              = 128
ollama  | llama_model_loader: - kv  33:                  qwen35moe.ssm.group_count u32              = 16
ollama  | llama_model_loader: - kv  34:               qwen35moe.ssm.time_step_rank u32              = 32
ollama  | llama_model_loader: - kv  35:                   qwen35moe.ssm.inner_size u32              = 4096
ollama  | llama_model_loader: - kv  36:          qwen35moe.full_attention_interval u32              = 4
ollama  | llama_model_loader: - kv  37:             qwen35moe.rope.dimension_count u32              = 64
ollama  | llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
ollama  | llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen35
ollama  | llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
ollama  | llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
ollama  | llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,247587]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
ollama  | llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 248046
ollama  | llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 248055
ollama  | llama_model_loader: - kv  45:                tokenizer.ggml.bos_token_id u32              = 248044
ollama  | llama_model_loader: - kv  46:               tokenizer.ggml.add_bos_token bool             = false
ollama  | llama_model_loader: - kv  47:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
ollama  | llama_model_loader: - kv  48:               general.quantization_version u32              = 2
ollama  | llama_model_loader: - kv  49:                          general.file_type u32              = 15
ollama  | llama_model_loader: - kv  50:                      quantize.imatrix.file str              = Qwen3.6-35B-A3B-GGUF/imatrix_unsloth....
ollama  | llama_model_loader: - kv  51:                   quantize.imatrix.dataset str              = unsloth_calibration_Qwen3.6-35B-A3B.txt
ollama  | llama_model_loader: - kv  52:             quantize.imatrix.entries_count u32              = 510
ollama  | llama_model_loader: - kv  53:              quantize.imatrix.chunks_count u32              = 76
ollama  | llama_model_loader: - type  f32:  361 tensors
ollama  | llama_model_loader: - type q8_0:  251 tensors
ollama  | llama_model_loader: - type q4_K:   80 tensors
ollama  | llama_model_loader: - type q5_K:   37 tensors
ollama  | llama_model_loader: - type q6_K:    4 tensors
ollama  | print_info: file format = GGUF V3 (latest)
ollama  | print_info: file type   = Q4_K - Medium
ollama  | print_info: file size   = 20.60 GiB (5.11 BPW) 
ollama  | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe'
ollama  | llama_model_load_from_file_impl: failed to load model
ollama  | time=2026-04-27T11:20:49.345Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-ac0e2c1189e055faa36eff361580e79c5bd6f8e76bffb4ce547f167d53e31a61"

Except that ollama seems to know this architecture ? The qwen model available at ollama.com is from the same family.
Maybe the error shown doesn't corresponds to error happening ?

root@9b5930073a52:/# ollama show qwen3.6:35b
  Model
    architecture        qwen35moe    
    parameters          36.0B        
    context length      262144       
    embedding length    2048         
    quantization        Q4_K_M       

  Capabilities
    completion    
    vision        
    tools         
    thinking      

  Parameters
    min_p               0       
    presence_penalty    1.5     
    repeat_penalty      1       
    temperature         1       
    top_k               20      
    top_p               0.95    

  License
    Apache License               
    Version 2.0, January 2004    
    ...                          

root@9b5930073a52:/# ollama show hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M
  Model
    architecture        qwen35moe    
    parameters          34.7B        
    context length      262144       
    embedding length    2048         
    quantization        unknown      

  Capabilities
    completion    
    vision        

  Projector
    architecture        clip       
    parameters          446.57M    
    embedding length    1152       
    dimensions          2048       

  Parameters
    stop    "<|im_start|>"    
    stop    "<|im_end|>"      
    stop    "<think>"         
    stop    ".Prompt          }}<|im_end|>"

Thank you for your help

Relevant log output

OS

Docker

GPU

Nvidia

CPU

AMD

Ollama version

0.21.2

Originally created by @orKL3mlz on GitHub (Apr 27, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15834 ### What is the issue? I can use the model `qwen3.6:35b` without any issues. However, when trying to run this model `hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M`, ollama returns `unknown model architecture: 'qwen35moe'` Full loading log below ``` ollama | llama_model_loader: loaded meta data with 54 key-value pairs and 733 tensors from /root/.ollama/models/blobs/sha256-ac0e2c1189e055faa36eff361580e79c5bd6f8e76bffb4ce547f167d53e31a61 (version GGUF V3 (latest)) ollama | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. ollama | llama_model_loader: - kv 0: general.architecture str = qwen35moe ollama | llama_model_loader: - kv 1: general.type str = model ollama | llama_model_loader: - kv 2: general.sampling.top_k i32 = 20 ollama | llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 ollama | llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 ollama | llama_model_loader: - kv 5: general.name str = Qwen3.6-35B-A3B ollama | llama_model_loader: - kv 6: general.basename str = Qwen3.6-35B-A3B ollama | llama_model_loader: - kv 7: general.quantized_by str = Unsloth ollama | llama_model_loader: - kv 8: general.size_label str = 35B-A3B ollama | llama_model_loader: - kv 9: general.license str = apache-2.0 ollama | llama_model_loader: - kv 10: general.license.link str = https://huggingface.co/Qwen/Qwen3.6-3... ollama | llama_model_loader: - kv 11: general.repo_url str = https://huggingface.co/unsloth ollama | llama_model_loader: - kv 12: general.base_model.count u32 = 1 ollama | llama_model_loader: - kv 13: general.base_model.0.name str = Qwen3.6 35B A3B ollama | llama_model_loader: - kv 14: general.base_model.0.organization str = Qwen ollama | llama_model_loader: - kv 15: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3.6-3... ollama | llama_model_loader: - kv 16: general.tags arr[str,3] = ["qwen3_5_moe", "qwen", "image-text-t... ollama | llama_model_loader: - kv 17: qwen35moe.block_count u32 = 40 ollama | llama_model_loader: - kv 18: qwen35moe.context_length u32 = 262144 ollama | llama_model_loader: - kv 19: qwen35moe.embedding_length u32 = 2048 ollama | llama_model_loader: - kv 20: qwen35moe.attention.head_count u32 = 16 ollama | llama_model_loader: - kv 21: qwen35moe.attention.head_count_kv u32 = 2 ollama | llama_model_loader: - kv 22: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0] ollama | llama_model_loader: - kv 23: qwen35moe.rope.freq_base f32 = 10000000.000000 ollama | llama_model_loader: - kv 24: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001 ollama | llama_model_loader: - kv 25: qwen35moe.expert_count u32 = 256 ollama | llama_model_loader: - kv 26: qwen35moe.expert_used_count u32 = 8 ollama | llama_model_loader: - kv 27: qwen35moe.attention.key_length u32 = 256 ollama | llama_model_loader: - kv 28: qwen35moe.attention.value_length u32 = 256 ollama | llama_model_loader: - kv 29: qwen35moe.expert_feed_forward_length u32 = 512 ollama | llama_model_loader: - kv 30: qwen35moe.expert_shared_feed_forward_length u32 = 512 ollama | llama_model_loader: - kv 31: qwen35moe.ssm.conv_kernel u32 = 4 ollama | llama_model_loader: - kv 32: qwen35moe.ssm.state_size u32 = 128 ollama | llama_model_loader: - kv 33: qwen35moe.ssm.group_count u32 = 16 ollama | llama_model_loader: - kv 34: qwen35moe.ssm.time_step_rank u32 = 32 ollama | llama_model_loader: - kv 35: qwen35moe.ssm.inner_size u32 = 4096 ollama | llama_model_loader: - kv 36: qwen35moe.full_attention_interval u32 = 4 ollama | llama_model_loader: - kv 37: qwen35moe.rope.dimension_count u32 = 64 ollama | llama_model_loader: - kv 38: tokenizer.ggml.model str = gpt2 ollama | llama_model_loader: - kv 39: tokenizer.ggml.pre str = qwen35 ollama | llama_model_loader: - kv 40: tokenizer.ggml.tokens arr[str,248320] = ["!", "\"", "#", "$", "%", "&", "'", ... ollama | llama_model_loader: - kv 41: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... ollama | llama_model_loader: - kv 42: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... ollama | llama_model_loader: - kv 43: tokenizer.ggml.eos_token_id u32 = 248046 ollama | llama_model_loader: - kv 44: tokenizer.ggml.padding_token_id u32 = 248055 ollama | llama_model_loader: - kv 45: tokenizer.ggml.bos_token_id u32 = 248044 ollama | llama_model_loader: - kv 46: tokenizer.ggml.add_bos_token bool = false ollama | llama_model_loader: - kv 47: tokenizer.chat_template str = {%- set image_count = namespace(value... ollama | llama_model_loader: - kv 48: general.quantization_version u32 = 2 ollama | llama_model_loader: - kv 49: general.file_type u32 = 15 ollama | llama_model_loader: - kv 50: quantize.imatrix.file str = Qwen3.6-35B-A3B-GGUF/imatrix_unsloth.... ollama | llama_model_loader: - kv 51: quantize.imatrix.dataset str = unsloth_calibration_Qwen3.6-35B-A3B.txt ollama | llama_model_loader: - kv 52: quantize.imatrix.entries_count u32 = 510 ollama | llama_model_loader: - kv 53: quantize.imatrix.chunks_count u32 = 76 ollama | llama_model_loader: - type f32: 361 tensors ollama | llama_model_loader: - type q8_0: 251 tensors ollama | llama_model_loader: - type q4_K: 80 tensors ollama | llama_model_loader: - type q5_K: 37 tensors ollama | llama_model_loader: - type q6_K: 4 tensors ollama | print_info: file format = GGUF V3 (latest) ollama | print_info: file type = Q4_K - Medium ollama | print_info: file size = 20.60 GiB (5.11 BPW) ollama | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe' ollama | llama_model_load_from_file_impl: failed to load model ollama | time=2026-04-27T11:20:49.345Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-ac0e2c1189e055faa36eff361580e79c5bd6f8e76bffb4ce547f167d53e31a61" ``` Except that ollama seems to know this architecture ? The qwen model available at ollama.com is from the same family. Maybe the error shown doesn't corresponds to error happening ? ``` root@9b5930073a52:/# ollama show qwen3.6:35b Model architecture qwen35moe parameters 36.0B context length 262144 embedding length 2048 quantization Q4_K_M Capabilities completion vision tools thinking Parameters min_p 0 presence_penalty 1.5 repeat_penalty 1 temperature 1 top_k 20 top_p 0.95 License Apache License Version 2.0, January 2004 ... root@9b5930073a52:/# ollama show hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M Model architecture qwen35moe parameters 34.7B context length 262144 embedding length 2048 quantization unknown Capabilities completion vision Projector architecture clip parameters 446.57M embedding length 1152 dimensions 2048 Parameters stop "<|im_start|>" stop "<|im_end|>" stop "<think>" stop ".Prompt }}<|im_end|>" ``` Thank you for your help ### Relevant log output ```shell ``` ### OS Docker ### GPU Nvidia ### CPU AMD ### Ollama version 0.21.2
GiteaMirror added the bug label 2026-05-05 03:33:16 -05:00
GiteaMirror commented 2026-05-05 03:33:19 -05:00
Author
Owner
Copy Link

@orKL3mlz commented on GitHub (Apr 27, 2026):

I also tried this model without success hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M

However, this model is loading and answering properly hf.co/hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF:Q4_K_M

Loading log of hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M below

ollama  | llama_model_loader: loaded meta data with 41 key-value pairs and 733 tensors from /root/.ollama/models/blobs/sha256-4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d (version GGUF V3 (latest))
ollama  | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
ollama  | llama_model_loader: - kv   0:                       general.architecture str              = qwen35moe
ollama  | llama_model_loader: - kv   1:                               general.type str              = model
ollama  | llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
ollama  | llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
ollama  | llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
ollama  | llama_model_loader: - kv   5:                               general.name str              = Qwen_Qwen3.6 35B A3B
ollama  | llama_model_loader: - kv   6:                           general.basename str              = Qwen_Qwen3.6
ollama  | llama_model_loader: - kv   7:                         general.size_label str              = 35B-A3B
ollama  | llama_model_loader: - kv   8:                      qwen35moe.block_count u32              = 40
ollama  | llama_model_loader: - kv   9:                   qwen35moe.context_length u32              = 262144
ollama  | llama_model_loader: - kv  10:                 qwen35moe.embedding_length u32              = 2048
ollama  | llama_model_loader: - kv  11:             qwen35moe.attention.head_count u32              = 16
ollama  | llama_model_loader: - kv  12:          qwen35moe.attention.head_count_kv u32              = 2
ollama  | llama_model_loader: - kv  13:          qwen35moe.rope.dimension_sections arr[i32,4]       = [11, 11, 10, 0]
ollama  | llama_model_loader: - kv  14:                   qwen35moe.rope.freq_base f32              = 10000000.000000
ollama  | llama_model_loader: - kv  15: qwen35moe.attention.layer_norm_rms_epsilon f32              = 0.000001
ollama  | llama_model_loader: - kv  16:                     qwen35moe.expert_count u32              = 256
ollama  | llama_model_loader: - kv  17:                qwen35moe.expert_used_count u32              = 8
ollama  | llama_model_loader: - kv  18:             qwen35moe.attention.key_length u32              = 256
ollama  | llama_model_loader: - kv  19:           qwen35moe.attention.value_length u32              = 256
ollama  | llama_model_loader: - kv  20:       qwen35moe.expert_feed_forward_length u32              = 512
ollama  | llama_model_loader: - kv  21: qwen35moe.expert_shared_feed_forward_length u32              = 512
ollama  | llama_model_loader: - kv  22:                  qwen35moe.ssm.conv_kernel u32              = 4
ollama  | llama_model_loader: - kv  23:                   qwen35moe.ssm.state_size u32              = 128
ollama  | llama_model_loader: - kv  24:                  qwen35moe.ssm.group_count u32              = 16
ollama  | llama_model_loader: - kv  25:               qwen35moe.ssm.time_step_rank u32              = 32
ollama  | llama_model_loader: - kv  26:                   qwen35moe.ssm.inner_size u32              = 4096
ollama  | llama_model_loader: - kv  27:          qwen35moe.full_attention_interval u32              = 4
ollama  | llama_model_loader: - kv  28:             qwen35moe.rope.dimension_count u32              = 64
ollama  | llama_model_loader: - kv  29:                       tokenizer.ggml.model str              = gpt2
ollama  | llama_model_loader: - kv  30:                         tokenizer.ggml.pre str              = qwen35
ollama  | llama_model_loader: - kv  31:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
ollama  | llama_model_loader: - kv  32:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
ollama  | llama_model_loader: - kv  33:                      tokenizer.ggml.merges arr[str,247587]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
ollama  | llama_model_loader: - kv  34:                tokenizer.ggml.eos_token_id u32              = 248046
ollama  | llama_model_loader: - kv  35:            tokenizer.ggml.padding_token_id u32              = 248044
ollama  | llama_model_loader: - kv  36:                tokenizer.ggml.bos_token_id u32              = 248044
ollama  | llama_model_loader: - kv  37:               tokenizer.ggml.add_bos_token bool             = false
ollama  | llama_model_loader: - kv  38:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
ollama  | llama_model_loader: - kv  39:               general.quantization_version u32              = 2
ollama  | llama_model_loader: - kv  40:                          general.file_type u32              = 15
ollama  | llama_model_loader: - type  f32:  301 tensors
ollama  | llama_model_loader: - type q4_K:  371 tensors
ollama  | llama_model_loader: - type q6_K:   61 tensors
ollama  | print_info: file format = GGUF V3 (latest)
ollama  | print_info: file type   = Q4_K - Medium
ollama  | print_info: file size   = 19.70 GiB (4.88 BPW) 
ollama  | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe'
ollama  | llama_model_load_from_file_impl: failed to load model
ollama  | time=2026-04-27T12:11:22.441Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d"
<!-- gh-comment-id:4326859871 --> @orKL3mlz commented on GitHub (Apr 27, 2026): I also tried this model without success `hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M` However, this model is loading and answering properly `hf.co/hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF:Q4_K_M` Loading log of `hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M` below ``` ollama | llama_model_loader: loaded meta data with 41 key-value pairs and 733 tensors from /root/.ollama/models/blobs/sha256-4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d (version GGUF V3 (latest)) ollama | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. ollama | llama_model_loader: - kv 0: general.architecture str = qwen35moe ollama | llama_model_loader: - kv 1: general.type str = model ollama | llama_model_loader: - kv 2: general.sampling.top_k i32 = 20 ollama | llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 ollama | llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 ollama | llama_model_loader: - kv 5: general.name str = Qwen_Qwen3.6 35B A3B ollama | llama_model_loader: - kv 6: general.basename str = Qwen_Qwen3.6 ollama | llama_model_loader: - kv 7: general.size_label str = 35B-A3B ollama | llama_model_loader: - kv 8: qwen35moe.block_count u32 = 40 ollama | llama_model_loader: - kv 9: qwen35moe.context_length u32 = 262144 ollama | llama_model_loader: - kv 10: qwen35moe.embedding_length u32 = 2048 ollama | llama_model_loader: - kv 11: qwen35moe.attention.head_count u32 = 16 ollama | llama_model_loader: - kv 12: qwen35moe.attention.head_count_kv u32 = 2 ollama | llama_model_loader: - kv 13: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0] ollama | llama_model_loader: - kv 14: qwen35moe.rope.freq_base f32 = 10000000.000000 ollama | llama_model_loader: - kv 15: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001 ollama | llama_model_loader: - kv 16: qwen35moe.expert_count u32 = 256 ollama | llama_model_loader: - kv 17: qwen35moe.expert_used_count u32 = 8 ollama | llama_model_loader: - kv 18: qwen35moe.attention.key_length u32 = 256 ollama | llama_model_loader: - kv 19: qwen35moe.attention.value_length u32 = 256 ollama | llama_model_loader: - kv 20: qwen35moe.expert_feed_forward_length u32 = 512 ollama | llama_model_loader: - kv 21: qwen35moe.expert_shared_feed_forward_length u32 = 512 ollama | llama_model_loader: - kv 22: qwen35moe.ssm.conv_kernel u32 = 4 ollama | llama_model_loader: - kv 23: qwen35moe.ssm.state_size u32 = 128 ollama | llama_model_loader: - kv 24: qwen35moe.ssm.group_count u32 = 16 ollama | llama_model_loader: - kv 25: qwen35moe.ssm.time_step_rank u32 = 32 ollama | llama_model_loader: - kv 26: qwen35moe.ssm.inner_size u32 = 4096 ollama | llama_model_loader: - kv 27: qwen35moe.full_attention_interval u32 = 4 ollama | llama_model_loader: - kv 28: qwen35moe.rope.dimension_count u32 = 64 ollama | llama_model_loader: - kv 29: tokenizer.ggml.model str = gpt2 ollama | llama_model_loader: - kv 30: tokenizer.ggml.pre str = qwen35 ollama | llama_model_loader: - kv 31: tokenizer.ggml.tokens arr[str,248320] = ["!", "\"", "#", "$", "%", "&", "'", ... ollama | llama_model_loader: - kv 32: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... ollama | llama_model_loader: - kv 33: tokenizer.ggml.merges arr[str,247587] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... ollama | llama_model_loader: - kv 34: tokenizer.ggml.eos_token_id u32 = 248046 ollama | llama_model_loader: - kv 35: tokenizer.ggml.padding_token_id u32 = 248044 ollama | llama_model_loader: - kv 36: tokenizer.ggml.bos_token_id u32 = 248044 ollama | llama_model_loader: - kv 37: tokenizer.ggml.add_bos_token bool = false ollama | llama_model_loader: - kv 38: tokenizer.chat_template str = {%- set image_count = namespace(value... ollama | llama_model_loader: - kv 39: general.quantization_version u32 = 2 ollama | llama_model_loader: - kv 40: general.file_type u32 = 15 ollama | llama_model_loader: - type f32: 301 tensors ollama | llama_model_loader: - type q4_K: 371 tensors ollama | llama_model_loader: - type q6_K: 61 tensors ollama | print_info: file format = GGUF V3 (latest) ollama | print_info: file type = Q4_K - Medium ollama | print_info: file size = 19.70 GiB (4.88 BPW) ollama | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe' ollama | llama_model_load_from_file_impl: failed to load model ollama | time=2026-04-27T12:11:22.441Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/lmstudio-community/Qwen3.6-35B-A3B-GGUF:Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-4ac6a06bce551257267f49ad2226f8671a22519ccc1a4dde9d5b433d1f2a410d" ```
GiteaMirror commented 2026-05-05 03:33:20 -05:00
Author
Owner
Copy Link

@gotnochill815-web commented on GitHub (Apr 27, 2026):

I opened a PR that appears to address this parser mismatch:

Root cause seems to be that qwen35 / qwen35moe are referenced in higher-level Ollama code, but llm_arch_from_string() in the bundled llama.cpp parser did not recognize these architecture names, causing:

unknown model architecture: 'qwen35moe'

The PR adds compatibility aliases for:

  • qwen35
  • qwen35moe

I also validated this against a previously failing model (hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M), which successfully returned metadata after patching.

https://github.com/ollama/ollama/pull/15836

<!-- gh-comment-id:4327991375 --> @gotnochill815-web commented on GitHub (Apr 27, 2026): I opened a PR that appears to address this parser mismatch: Root cause seems to be that `qwen35` / `qwen35moe` are referenced in higher-level Ollama code, but `llm_arch_from_string()` in the bundled llama.cpp parser did not recognize these architecture names, causing: `unknown model architecture: 'qwen35moe'` The PR adds compatibility aliases for: * `qwen35` * `qwen35moe` I also validated this against a previously failing model (`hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M`), which successfully returned metadata after patching. https://github.com/ollama/ollama/pull/15836
GiteaMirror commented 2026-05-05 03:33:21 -05:00
Author
Owner
Copy Link

@orKL3mlz commented on GitHub (Apr 30, 2026):

Hi, following up on the work you've done, is the issue related to this model unsloth/gemma-4-E2B-it-GGUF could be linked to the same PR that you already made ?

Full loading logs

ollama  | llama_model_loader: loaded meta data with 56 key-value pairs and 601 tensors from /root/.ollama/models/blobs/sha256-ac0069ebccd39925d836f24a88c0f0c858d20578c29b21ab7cedce66ee576845 (version GGUF V3 (latest))
ollama  | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
ollama  | llama_model_loader: - kv   0:                       general.architecture str              = gemma4
ollama  | llama_model_loader: - kv   1:                               general.type str              = model
ollama  | llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 64
ollama  | llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
ollama  | llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
ollama  | llama_model_loader: - kv   5:                               general.name str              = Gemma-4-E2B-It
ollama  | llama_model_loader: - kv   6:                           general.basename str              = Gemma-4-E2B-It
ollama  | llama_model_loader: - kv   7:                       general.quantized_by str              = Unsloth
ollama  | llama_model_loader: - kv   8:                         general.size_label str              = 4.6B
ollama  | llama_model_loader: - kv   9:                            general.license str              = apache-2.0
ollama  | llama_model_loader: - kv  10:                       general.license.link str              = https://ai.google.dev/gemma/docs/gemm...
ollama  | llama_model_loader: - kv  11:                           general.repo_url str              = https://huggingface.co/unsloth
ollama  | llama_model_loader: - kv  12:                   general.base_model.count u32              = 1
ollama  | llama_model_loader: - kv  13:                  general.base_model.0.name str              = Gemma 4 E2B It
ollama  | llama_model_loader: - kv  14:          general.base_model.0.organization str              = Google
ollama  | llama_model_loader: - kv  15:              general.base_model.0.repo_url str              = https://huggingface.co/google/gemma-4...
ollama  | llama_model_loader: - kv  16:                               general.tags arr[str,2]       = ["unsloth", "any-to-any"]
ollama  | llama_model_loader: - kv  17:                         gemma4.block_count u32              = 35
ollama  | llama_model_loader: - kv  18:                      gemma4.context_length u32              = 131072
ollama  | llama_model_loader: - kv  19:                    gemma4.embedding_length u32              = 1536
ollama  | llama_model_loader: - kv  20:                 gemma4.feed_forward_length arr[i32,35]      = [6144, 6144, 6144, 6144, 6144, 6144, ...
ollama  | llama_model_loader: - kv  21:                gemma4.attention.head_count u32              = 8
ollama  | llama_model_loader: - kv  22:             gemma4.attention.head_count_kv u32              = 1
ollama  | llama_model_loader: - kv  23:                      gemma4.rope.freq_base f32              = 1000000.000000
ollama  | llama_model_loader: - kv  24:                  gemma4.rope.freq_base_swa f32              = 10000.000000
ollama  | llama_model_loader: - kv  25:    gemma4.attention.layer_norm_rms_epsilon f32              = 0.000001
ollama  | llama_model_loader: - kv  26:                gemma4.attention.key_length u32              = 512
ollama  | llama_model_loader: - kv  27:              gemma4.attention.value_length u32              = 512
ollama  | llama_model_loader: - kv  28:             gemma4.final_logit_softcapping f32              = 30.000000
ollama  | llama_model_loader: - kv  29:            gemma4.attention.sliding_window u32              = 512
ollama  | llama_model_loader: - kv  30:          gemma4.attention.shared_kv_layers u32              = 20
ollama  | llama_model_loader: - kv  31:    gemma4.embedding_length_per_layer_input u32              = 256
ollama  | llama_model_loader: - kv  32:    gemma4.attention.sliding_window_pattern arr[bool,35]     = [true, true, true, true, false, true,...
ollama  | llama_model_loader: - kv  33:            gemma4.attention.key_length_swa u32              = 256
ollama  | llama_model_loader: - kv  34:          gemma4.attention.value_length_swa u32              = 256
ollama  | llama_model_loader: - kv  35:                gemma4.rope.dimension_count u32              = 512
ollama  | llama_model_loader: - kv  36:            gemma4.rope.dimension_count_swa u32              = 256
ollama  | llama_model_loader: - kv  37:                       tokenizer.ggml.model str              = gemma4
ollama  | llama_model_loader: - kv  38:                      tokenizer.ggml.tokens arr[str,262144]  = ["<pad>", "<eos>", "<bos>", "<unk>", ...
ollama  | llama_model_loader: - kv  39:                      tokenizer.ggml.scores arr[f32,262144]  = [-1000.000000, -1000.000000, -1000.00...
ollama  | llama_model_loader: - kv  40:                  tokenizer.ggml.token_type arr[i32,262144]  = [3, 1, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ...
ollama  | llama_model_loader: - kv  41:                      tokenizer.ggml.merges arr[str,514906]  = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ...
ollama  | llama_model_loader: - kv  42:                tokenizer.ggml.bos_token_id u32              = 2
ollama  | llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 106
ollama  | llama_model_loader: - kv  44:            tokenizer.ggml.unknown_token_id u32              = 3
ollama  | llama_model_loader: - kv  45:            tokenizer.ggml.padding_token_id u32              = 0
ollama  | llama_model_loader: - kv  46:               tokenizer.ggml.mask_token_id u32              = 4
ollama  | llama_model_loader: - kv  47:                    tokenizer.chat_template str              = {%- macro format_parameters(propertie...
ollama  | llama_model_loader: - kv  48:            tokenizer.ggml.add_space_prefix bool             = false
ollama  | llama_model_loader: - kv  49:               tokenizer.ggml.add_bos_token bool             = true
ollama  | llama_model_loader: - kv  50:               general.quantization_version u32              = 2
ollama  | llama_model_loader: - kv  51:                          general.file_type u32              = 15
ollama  | llama_model_loader: - kv  52:                      quantize.imatrix.file str              = gemma-4-E2B-it-GGUF/imatrix_unsloth.gguf
ollama  | llama_model_loader: - kv  53:                   quantize.imatrix.dataset str              = unsloth_calibration_gemma-4-E2B-it.txt
ollama  | llama_model_loader: - kv  54:             quantize.imatrix.entries_count u32              = 275
ollama  | llama_model_loader: - kv  55:              quantize.imatrix.chunks_count u32              = 141
ollama  | llama_model_loader: - type  f32:  353 tensors
ollama  | llama_model_loader: - type q4_K:  212 tensors
ollama  | llama_model_loader: - type q5_K:    1 tensors
ollama  | llama_model_loader: - type q6_K:   34 tensors
ollama  | llama_model_loader: - type bf16:    1 tensors
ollama  | print_info: file format = GGUF V3 (latest)
ollama  | print_info: file type   = Q4_K - Medium
ollama  | print_info: file size   = 2.88 GiB (5.32 BPW) 
ollama  | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4'
ollama  | llama_model_load_from_file_impl: failed to load model
ollama  | time=2026-04-30T13:13:37.257Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/unsloth/gemma-4-E2B-it-GGUF:Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-ac0069ebccd39925d836f24a88c0f0c858d20578c29b21ab7cedce66ee576845"
<!-- gh-comment-id:4352772870 --> @orKL3mlz commented on GitHub (Apr 30, 2026): Hi, following up on the work you've done, is the issue related to this model `unsloth/gemma-4-E2B-it-GGUF` could be linked to the same PR that you already made ? ## Full loading logs ``` ollama | llama_model_loader: loaded meta data with 56 key-value pairs and 601 tensors from /root/.ollama/models/blobs/sha256-ac0069ebccd39925d836f24a88c0f0c858d20578c29b21ab7cedce66ee576845 (version GGUF V3 (latest)) ollama | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. ollama | llama_model_loader: - kv 0: general.architecture str = gemma4 ollama | llama_model_loader: - kv 1: general.type str = model ollama | llama_model_loader: - kv 2: general.sampling.top_k i32 = 64 ollama | llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 ollama | llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 ollama | llama_model_loader: - kv 5: general.name str = Gemma-4-E2B-It ollama | llama_model_loader: - kv 6: general.basename str = Gemma-4-E2B-It ollama | llama_model_loader: - kv 7: general.quantized_by str = Unsloth ollama | llama_model_loader: - kv 8: general.size_label str = 4.6B ollama | llama_model_loader: - kv 9: general.license str = apache-2.0 ollama | llama_model_loader: - kv 10: general.license.link str = https://ai.google.dev/gemma/docs/gemm... ollama | llama_model_loader: - kv 11: general.repo_url str = https://huggingface.co/unsloth ollama | llama_model_loader: - kv 12: general.base_model.count u32 = 1 ollama | llama_model_loader: - kv 13: general.base_model.0.name str = Gemma 4 E2B It ollama | llama_model_loader: - kv 14: general.base_model.0.organization str = Google ollama | llama_model_loader: - kv 15: general.base_model.0.repo_url str = https://huggingface.co/google/gemma-4... ollama | llama_model_loader: - kv 16: general.tags arr[str,2] = ["unsloth", "any-to-any"] ollama | llama_model_loader: - kv 17: gemma4.block_count u32 = 35 ollama | llama_model_loader: - kv 18: gemma4.context_length u32 = 131072 ollama | llama_model_loader: - kv 19: gemma4.embedding_length u32 = 1536 ollama | llama_model_loader: - kv 20: gemma4.feed_forward_length arr[i32,35] = [6144, 6144, 6144, 6144, 6144, 6144, ... ollama | llama_model_loader: - kv 21: gemma4.attention.head_count u32 = 8 ollama | llama_model_loader: - kv 22: gemma4.attention.head_count_kv u32 = 1 ollama | llama_model_loader: - kv 23: gemma4.rope.freq_base f32 = 1000000.000000 ollama | llama_model_loader: - kv 24: gemma4.rope.freq_base_swa f32 = 10000.000000 ollama | llama_model_loader: - kv 25: gemma4.attention.layer_norm_rms_epsilon f32 = 0.000001 ollama | llama_model_loader: - kv 26: gemma4.attention.key_length u32 = 512 ollama | llama_model_loader: - kv 27: gemma4.attention.value_length u32 = 512 ollama | llama_model_loader: - kv 28: gemma4.final_logit_softcapping f32 = 30.000000 ollama | llama_model_loader: - kv 29: gemma4.attention.sliding_window u32 = 512 ollama | llama_model_loader: - kv 30: gemma4.attention.shared_kv_layers u32 = 20 ollama | llama_model_loader: - kv 31: gemma4.embedding_length_per_layer_input u32 = 256 ollama | llama_model_loader: - kv 32: gemma4.attention.sliding_window_pattern arr[bool,35] = [true, true, true, true, false, true,... ollama | llama_model_loader: - kv 33: gemma4.attention.key_length_swa u32 = 256 ollama | llama_model_loader: - kv 34: gemma4.attention.value_length_swa u32 = 256 ollama | llama_model_loader: - kv 35: gemma4.rope.dimension_count u32 = 512 ollama | llama_model_loader: - kv 36: gemma4.rope.dimension_count_swa u32 = 256 ollama | llama_model_loader: - kv 37: tokenizer.ggml.model str = gemma4 ollama | llama_model_loader: - kv 38: tokenizer.ggml.tokens arr[str,262144] = ["<pad>", "<eos>", "<bos>", "<unk>", ... ollama | llama_model_loader: - kv 39: tokenizer.ggml.scores arr[f32,262144] = [-1000.000000, -1000.000000, -1000.00... ollama | llama_model_loader: - kv 40: tokenizer.ggml.token_type arr[i32,262144] = [3, 1, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ... ollama | llama_model_loader: - kv 41: tokenizer.ggml.merges arr[str,514906] = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ... ollama | llama_model_loader: - kv 42: tokenizer.ggml.bos_token_id u32 = 2 ollama | llama_model_loader: - kv 43: tokenizer.ggml.eos_token_id u32 = 106 ollama | llama_model_loader: - kv 44: tokenizer.ggml.unknown_token_id u32 = 3 ollama | llama_model_loader: - kv 45: tokenizer.ggml.padding_token_id u32 = 0 ollama | llama_model_loader: - kv 46: tokenizer.ggml.mask_token_id u32 = 4 ollama | llama_model_loader: - kv 47: tokenizer.chat_template str = {%- macro format_parameters(propertie... ollama | llama_model_loader: - kv 48: tokenizer.ggml.add_space_prefix bool = false ollama | llama_model_loader: - kv 49: tokenizer.ggml.add_bos_token bool = true ollama | llama_model_loader: - kv 50: general.quantization_version u32 = 2 ollama | llama_model_loader: - kv 51: general.file_type u32 = 15 ollama | llama_model_loader: - kv 52: quantize.imatrix.file str = gemma-4-E2B-it-GGUF/imatrix_unsloth.gguf ollama | llama_model_loader: - kv 53: quantize.imatrix.dataset str = unsloth_calibration_gemma-4-E2B-it.txt ollama | llama_model_loader: - kv 54: quantize.imatrix.entries_count u32 = 275 ollama | llama_model_loader: - kv 55: quantize.imatrix.chunks_count u32 = 141 ollama | llama_model_loader: - type f32: 353 tensors ollama | llama_model_loader: - type q4_K: 212 tensors ollama | llama_model_loader: - type q5_K: 1 tensors ollama | llama_model_loader: - type q6_K: 34 tensors ollama | llama_model_loader: - type bf16: 1 tensors ollama | print_info: file format = GGUF V3 (latest) ollama | print_info: file type = Q4_K - Medium ollama | print_info: file size = 2.88 GiB (5.32 BPW) ollama | llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' ollama | llama_model_load_from_file_impl: failed to load model ollama | time=2026-04-30T13:13:37.257Z level=INFO source=sched.go:462 msg="failed to create server" model=hf.co/unsloth/gemma-4-E2B-it-GGUF:Q4_K_M error="unable to load model: /root/.ollama/models/blobs/sha256-ac0069ebccd39925d836f24a88c0f0c858d20578c29b21ab7cedce66ee576845" ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#72152
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?