[GH-ISSUE #14481] Cannot run Qwen3.5-35B-A3B on v0.17.2 #9392

Closed
opened 2026-04-12 22:18:30 -05:00 by GiteaMirror · 0 comments
Owner

Originally created by @azdlonky on GitHub (Feb 26, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/14481

What is the issue?

I tried on both v0.17.0 and v0.17.2, both fail to run the Qwen model with the error unknown model architecture: 'qwen35moe'.

Supposedly (based on this comment) the model works in v0.17.1-rc2, so I'd assumed it'd also work in v0.17.2.

Relevant log output

time=2026-02-26T23:48:01.512Z level=INFO source=routes.go:1663 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/<me>/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES:GPU-<GPU-UUID> http_proxy: https_proxy: no_proxy:]"
time=2026-02-26T23:48:01.512Z level=INFO source=routes.go:1665 msg="Ollama cloud disabled: false"
time=2026-02-26T23:48:01.512Z level=INFO source=images.go:473 msg="total blobs: 12"
time=2026-02-26T23:48:01.512Z level=INFO source=images.go:480 msg="total unused blobs removed: 0"
time=2026-02-26T23:48:01.513Z level=INFO source=routes.go:1718 msg="Listening on 127.0.0.1:11434 (version 0.17.2)"
time=2026-02-26T23:48:01.513Z level=INFO source=runner.go:67 msg="discovering available GPUs..."
time=2026-02-26T23:48:01.513Z level=WARN source=runner.go:485 msg="user overrode visible devices" ROCR_VISIBLE_DEVICES=GPU-<GPU-UUID>
time=2026-02-26T23:48:01.513Z level=WARN source=runner.go:489 msg="if GPUs are not correctly discovered, unset and try again"
time=2026-02-26T23:48:01.514Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46391"
time=2026-02-26T23:48:01.529Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41449"
time=2026-02-26T23:48:02.718Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled.  To enable, set OLLAMA_VULKAN=1"
time=2026-02-26T23:48:02.719Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46133"
time=2026-02-26T23:48:02.734Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 39363"
time=2026-02-26T23:48:04.713Z level=INFO source=types.go:42 msg="inference compute" id=GPU-<GPU-UUID> filter_id="" library=ROCm compute=gfx1100 name=ROCm0 description="AMD Radeon Graphics" libdirs=ollama,rocm driver=60342.13 pci_id=0000:03:00.0 type=discrete total="24.0 GiB" available="20.5 GiB"
time=2026-02-26T23:48:04.713Z level=INFO source=routes.go:1768 msg="vram-based default context" total_vram="24.0 GiB" default_num_ctx=32768
[GIN] 2026/02/26 - 23:48:08 | 200 |      34.826\C2\B5s |       127.0.0.1 | HEAD     "/"
[GIN] 2026/02/26 - 23:48:08 | 200 |  129.926185ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/02/26 - 23:48:08 | 200 |  129.540863ms |       127.0.0.1 | POST     "/api/show"
time=2026-02-26T23:48:09.005Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46847"
llama_model_loader: loaded meta data with 52 key-value pairs and 733 tensors from /home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen35moe
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 20
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   5:                               general.name str              = Qwen3.5-35B-A3B
llama_model_loader: - kv   6:                           general.basename str              = Qwen3.5-35B-A3B
llama_model_loader: - kv   7:                       general.quantized_by str              = Unsloth
llama_model_loader: - kv   8:                         general.size_label str              = 35B-A3B
llama_model_loader: - kv   9:                            general.license str              = apache-2.0
llama_model_loader: - kv  10:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3.5-3...
llama_model_loader: - kv  11:                           general.repo_url str              = https://huggingface.co/unsloth
llama_model_loader: - kv  12:                   general.base_model.count u32              = 1
llama_model_loader: - kv  13:                  general.base_model.0.name str              = Qwen3.5 35B A3B
llama_model_loader: - kv  14:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  15:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3.5-3...
llama_model_loader: - kv  16:                               general.tags arr[str,2]       = ["unsloth", "image-text-to-text"]
llama_model_loader: - kv  17:                      qwen35moe.block_count u32              = 40
llama_model_loader: - kv  18:                   qwen35moe.context_length u32              = 262144
llama_model_loader: - kv  19:                 qwen35moe.embedding_length u32              = 2048
llama_model_loader: - kv  20:             qwen35moe.attention.head_count u32              = 16
llama_model_loader: - kv  21:          qwen35moe.attention.head_count_kv u32              = 2
llama_model_loader: - kv  22:          qwen35moe.rope.dimension_sections arr[i32,4]       = [11, 11, 10, 0]
llama_model_loader: - kv  23:                   qwen35moe.rope.freq_base f32              = 10000000.000000
llama_model_loader: - kv  24: qwen35moe.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  25:                     qwen35moe.expert_count u32              = 256
llama_model_loader: - kv  26:                qwen35moe.expert_used_count u32              = 8
llama_model_loader: - kv  27:             qwen35moe.attention.key_length u32              = 256
llama_model_loader: - kv  28:           qwen35moe.attention.value_length u32              = 256
llama_model_loader: - kv  29:       qwen35moe.expert_feed_forward_length u32              = 512
llama_model_loader: - kv  30: qwen35moe.expert_shared_feed_forward_length u32              = 512
llama_model_loader: - kv  31:                  qwen35moe.ssm.conv_kernel u32              = 4
llama_model_loader: - kv  32:                   qwen35moe.ssm.state_size u32              = 128
llama_model_loader: - kv  33:                  qwen35moe.ssm.group_count u32              = 16
llama_model_loader: - kv  34:               qwen35moe.ssm.time_step_rank u32              = 32
llama_model_loader: - kv  35:                   qwen35moe.ssm.inner_size u32              = 4096
llama_model_loader: - kv  36:          qwen35moe.full_attention_interval u32              = 4
llama_model_loader: - kv  37:             qwen35moe.rope.dimension_count u32              = 64
llama_model_loader: - kv  38:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  39:                         tokenizer.ggml.pre str              = qwen35
llama_model_loader: - kv  40:                      tokenizer.ggml.tokens arr[str,248320]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  41:                  tokenizer.ggml.token_type arr[i32,248320]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  42:                      tokenizer.ggml.merges arr[str,247587]  = ["\C4\A0 \C4\A0", "\C4\A0\C4\A0 \C4\A0\C4\A0", "i n", "\C4\A0 t",...
llama_model_loader: - kv  43:                tokenizer.ggml.eos_token_id u32              = 248046
llama_model_loader: - kv  44:            tokenizer.ggml.padding_token_id u32              = 248055
llama_model_loader: - kv  45:                    tokenizer.chat_template str              = {%- set image_count = namespace(value...
llama_model_loader: - kv  46:               general.quantization_version u32              = 2
llama_model_loader: - kv  47:                          general.file_type u32              = 15
llama_model_loader: - kv  48:                      quantize.imatrix.file str              = Qwen3.5-35B-A3B-GGUF/imatrix_unsloth....
llama_model_loader: - kv  49:                   quantize.imatrix.dataset str              = unsloth_calibration_Qwen3.5-35B-A3B.txt
llama_model_loader: - kv  50:             quantize.imatrix.entries_count u32              = 510
llama_model_loader: - kv  51:              quantize.imatrix.chunks_count u32              = 80
llama_model_loader: - type  f32:  301 tensors
llama_model_loader: - type q8_0:   74 tensors
llama_model_loader: - type q4_K:    1 tensors
llama_model_loader: - type q5_K:   31 tensors
llama_model_loader: - type q6_K:   51 tensors
llama_model_loader: - type mxfp4:  275 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 18.32 GiB (4.54 BPW)
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe'
llama_model_load_from_file_impl: failed to load model
time=2026-02-26T23:48:10.343Z level=INFO source=sched.go:473 msg="NewLlamaServer failed" model=/home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158 error="unable to load model: /home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158"
[GIN] 2026/02/26 - 23:48:10 | 500 |  1.498637448s |       127.0.0.1 | POST     "/api/generate"

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.17.2

Originally created by @azdlonky on GitHub (Feb 26, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/14481 ### What is the issue? I tried on both v0.17.0 and v0.17.2, both fail to run the Qwen model with the error `unknown model architecture: 'qwen35moe'`. Supposedly (based on [this comment](https://github.com/ollama/ollama/issues/14428#issuecomment-3963072517)) the model works in v0.17.1-rc2, so I'd assumed it'd also work in v0.17.2. ### Relevant log output ```shell time=2026-02-26T23:48:01.512Z level=INFO source=routes.go:1663 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:INFO OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/<me>/.ollama/models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NEW_ENGINE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:false ROCR_VISIBLE_DEVICES:GPU-<GPU-UUID> http_proxy: https_proxy: no_proxy:]" time=2026-02-26T23:48:01.512Z level=INFO source=routes.go:1665 msg="Ollama cloud disabled: false" time=2026-02-26T23:48:01.512Z level=INFO source=images.go:473 msg="total blobs: 12" time=2026-02-26T23:48:01.512Z level=INFO source=images.go:480 msg="total unused blobs removed: 0" time=2026-02-26T23:48:01.513Z level=INFO source=routes.go:1718 msg="Listening on 127.0.0.1:11434 (version 0.17.2)" time=2026-02-26T23:48:01.513Z level=INFO source=runner.go:67 msg="discovering available GPUs..." time=2026-02-26T23:48:01.513Z level=WARN source=runner.go:485 msg="user overrode visible devices" ROCR_VISIBLE_DEVICES=GPU-<GPU-UUID> time=2026-02-26T23:48:01.513Z level=WARN source=runner.go:489 msg="if GPUs are not correctly discovered, unset and try again" time=2026-02-26T23:48:01.514Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46391" time=2026-02-26T23:48:01.529Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41449" time=2026-02-26T23:48:02.718Z level=INFO source=runner.go:106 msg="experimental Vulkan support disabled. To enable, set OLLAMA_VULKAN=1" time=2026-02-26T23:48:02.719Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46133" time=2026-02-26T23:48:02.734Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 39363" time=2026-02-26T23:48:04.713Z level=INFO source=types.go:42 msg="inference compute" id=GPU-<GPU-UUID> filter_id="" library=ROCm compute=gfx1100 name=ROCm0 description="AMD Radeon Graphics" libdirs=ollama,rocm driver=60342.13 pci_id=0000:03:00.0 type=discrete total="24.0 GiB" available="20.5 GiB" time=2026-02-26T23:48:04.713Z level=INFO source=routes.go:1768 msg="vram-based default context" total_vram="24.0 GiB" default_num_ctx=32768 [GIN] 2026/02/26 - 23:48:08 | 200 | 34.826\C2\B5s | 127.0.0.1 | HEAD "/" [GIN] 2026/02/26 - 23:48:08 | 200 | 129.926185ms | 127.0.0.1 | POST "/api/show" [GIN] 2026/02/26 - 23:48:08 | 200 | 129.540863ms | 127.0.0.1 | POST "/api/show" time=2026-02-26T23:48:09.005Z level=INFO source=server.go:431 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 46847" llama_model_loader: loaded meta data with 52 key-value pairs and 733 tensors from /home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158 (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = qwen35moe llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.sampling.top_k i32 = 20 llama_model_loader: - kv 3: general.sampling.top_p f32 = 0.950000 llama_model_loader: - kv 4: general.sampling.temp f32 = 1.000000 llama_model_loader: - kv 5: general.name str = Qwen3.5-35B-A3B llama_model_loader: - kv 6: general.basename str = Qwen3.5-35B-A3B llama_model_loader: - kv 7: general.quantized_by str = Unsloth llama_model_loader: - kv 8: general.size_label str = 35B-A3B llama_model_loader: - kv 9: general.license str = apache-2.0 llama_model_loader: - kv 10: general.license.link str = https://huggingface.co/Qwen/Qwen3.5-3... llama_model_loader: - kv 11: general.repo_url str = https://huggingface.co/unsloth llama_model_loader: - kv 12: general.base_model.count u32 = 1 llama_model_loader: - kv 13: general.base_model.0.name str = Qwen3.5 35B A3B llama_model_loader: - kv 14: general.base_model.0.organization str = Qwen llama_model_loader: - kv 15: general.base_model.0.repo_url str = https://huggingface.co/Qwen/Qwen3.5-3... llama_model_loader: - kv 16: general.tags arr[str,2] = ["unsloth", "image-text-to-text"] llama_model_loader: - kv 17: qwen35moe.block_count u32 = 40 llama_model_loader: - kv 18: qwen35moe.context_length u32 = 262144 llama_model_loader: - kv 19: qwen35moe.embedding_length u32 = 2048 llama_model_loader: - kv 20: qwen35moe.attention.head_count u32 = 16 llama_model_loader: - kv 21: qwen35moe.attention.head_count_kv u32 = 2 llama_model_loader: - kv 22: qwen35moe.rope.dimension_sections arr[i32,4] = [11, 11, 10, 0] llama_model_loader: - kv 23: qwen35moe.rope.freq_base f32 = 10000000.000000 llama_model_loader: - kv 24: qwen35moe.attention.layer_norm_rms_epsilon f32 = 0.000001 llama_model_loader: - kv 25: qwen35moe.expert_count u32 = 256 llama_model_loader: - kv 26: qwen35moe.expert_used_count u32 = 8 llama_model_loader: - kv 27: qwen35moe.attention.key_length u32 = 256 llama_model_loader: - kv 28: qwen35moe.attention.value_length u32 = 256 llama_model_loader: - kv 29: qwen35moe.expert_feed_forward_length u32 = 512 llama_model_loader: - kv 30: qwen35moe.expert_shared_feed_forward_length u32 = 512 llama_model_loader: - kv 31: qwen35moe.ssm.conv_kernel u32 = 4 llama_model_loader: - kv 32: qwen35moe.ssm.state_size u32 = 128 llama_model_loader: - kv 33: qwen35moe.ssm.group_count u32 = 16 llama_model_loader: - kv 34: qwen35moe.ssm.time_step_rank u32 = 32 llama_model_loader: - kv 35: qwen35moe.ssm.inner_size u32 = 4096 llama_model_loader: - kv 36: qwen35moe.full_attention_interval u32 = 4 llama_model_loader: - kv 37: qwen35moe.rope.dimension_count u32 = 64 llama_model_loader: - kv 38: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 39: tokenizer.ggml.pre str = qwen35 llama_model_loader: - kv 40: tokenizer.ggml.tokens arr[str,248320] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 41: tokenizer.ggml.token_type arr[i32,248320] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 42: tokenizer.ggml.merges arr[str,247587] = ["\C4\A0 \C4\A0", "\C4\A0\C4\A0 \C4\A0\C4\A0", "i n", "\C4\A0 t",... llama_model_loader: - kv 43: tokenizer.ggml.eos_token_id u32 = 248046 llama_model_loader: - kv 44: tokenizer.ggml.padding_token_id u32 = 248055 llama_model_loader: - kv 45: tokenizer.chat_template str = {%- set image_count = namespace(value... llama_model_loader: - kv 46: general.quantization_version u32 = 2 llama_model_loader: - kv 47: general.file_type u32 = 15 llama_model_loader: - kv 48: quantize.imatrix.file str = Qwen3.5-35B-A3B-GGUF/imatrix_unsloth.... llama_model_loader: - kv 49: quantize.imatrix.dataset str = unsloth_calibration_Qwen3.5-35B-A3B.txt llama_model_loader: - kv 50: quantize.imatrix.entries_count u32 = 510 llama_model_loader: - kv 51: quantize.imatrix.chunks_count u32 = 80 llama_model_loader: - type f32: 301 tensors llama_model_loader: - type q8_0: 74 tensors llama_model_loader: - type q4_K: 1 tensors llama_model_loader: - type q5_K: 31 tensors llama_model_loader: - type q6_K: 51 tensors llama_model_loader: - type mxfp4: 275 tensors print_info: file format = GGUF V3 (latest) print_info: file type = Q4_K - Medium print_info: file size = 18.32 GiB (4.54 BPW) llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35moe' llama_model_load_from_file_impl: failed to load model time=2026-02-26T23:48:10.343Z level=INFO source=sched.go:473 msg="NewLlamaServer failed" model=/home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158 error="unable to load model: /home/<me>/.ollama/models/blobs/sha256-70f7b15b38aa5d778e3ecd7158fc35ecc7b9c36d11696d4ec59a116b75a78158" [GIN] 2026/02/26 - 23:48:10 | 500 | 1.498637448s | 127.0.0.1 | POST "/api/generate" ``` ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.17.2
GiteaMirror added the bug label 2026-04-12 22:18:30 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#9392