github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6649] Intel GPU - model > 4b nonsense? #50696

New Issue
Closed
opened 2026-04-28 16:48:16 -05:00 by GiteaMirror · 12 comments
GiteaMirror commented 2026-04-28 16:48:16 -05:00
Owner
Copy Link

Originally created by @cyear on GitHub (Sep 5, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6649

What is the issue?

qwen4b works fine, all other models larger than 4b are gibberish

time=2024-09-05T11:35:49.569+08:00 level=INFO source=download.go:175 msg="downloading 8eeb52dfb3bb in 16 291 MB part(s)"
time=2024-09-05T11:37:19.112+08:00 level=INFO source=download.go:370 msg="8eeb52dfb3bb part 0 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
time=2024-09-05T11:37:21.112+08:00 level=INFO source=download.go:370 msg="8eeb52dfb3bb part 4 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
[GIN] 2024/09/05 - 11:41:40 | 200 |         5m55s |       10.0.0.18 | POST     "/api/pull"
[GIN] 2024/09/05 - 11:51:04 | 200 |       1.182ms |       10.0.0.18 | GET      "/api/tags"
[GIN] 2024/09/05 - 11:51:05 | 200 |            0s |       10.0.0.18 | GET      "/api/version"
[GIN] 2024/09/05 - 11:51:24 | 200 |       510.7µs |       10.0.0.18 | GET      "/api/version"
[GIN] 2024/09/05 - 11:51:33 | 200 |            0s |       10.0.0.18 | GET      "/api/version"
time=2024-09-05T11:51:51.177+08:00 level=INFO source=download.go:175 msg="downloading 8eeb52dfb3bb in 16 291 MB part(s)"
time=2024-09-05T11:51:58.238+08:00 level=INFO source=download.go:175 msg="downloading 73b313b5552d in 1 1.4 KB part(s)"
time=2024-09-05T11:52:01.269+08:00 level=INFO source=download.go:175 msg="downloading 0ba8f0e314b4 in 1 12 KB part(s)"
time=2024-09-05T11:52:04.339+08:00 level=INFO source=download.go:175 msg="downloading 56bb8bd477a5 in 1 96 B part(s)"
time=2024-09-05T11:52:07.492+08:00 level=INFO source=download.go:175 msg="downloading 1a4c3c319823 in 1 485 B part(s)"
[GIN] 2024/09/05 - 11:52:14 | 200 |   28.5001976s |       10.0.0.18 | POST     "/api/pull"
[GIN] 2024/09/05 - 11:52:14 | 200 |      1.0817ms |       10.0.0.18 | GET      "/api/tags"
[GIN] 2024/09/05 - 11:52:18 | 200 |            0s |       10.0.0.18 | GET      "/api/version"
time=2024-09-05T11:52:23.514+08:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=33 layers.offload=0 layers.split="" memory.available="[20.3 GiB]" memory.required.full="4.6 GiB" memory.required.partial="0 B" memory.required.kv="256.0 MiB" memory.required.allocations="[4.6 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.5 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB"
time=2024-09-05T11:52:23.520+08:00 level=INFO source=server.go:395 msg="starting llama server" cmd="C:\\Users\\12742\\Desktop\\llama-cpp\\dist\\windows-amd64\\lib\\ollama\\runners\\cpu_avx2\\ollama_llama_server.exe --model C:\\Users\\12742\\.ollama\\models\\blobs\\sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --no-mmap --parallel 1 --port 55176"
time=2024-09-05T11:52:23.546+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1
time=2024-09-05T11:52:23.546+08:00 level=INFO source=server.go:595 msg="waiting for llama runner to start responding"
time=2024-09-05T11:52:23.547+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server error"
INFO [wmain] build info | build=1 commit="c455d1d" tid="6776" timestamp=1725508343
INFO [wmain] system info | n_threads=14 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="6776" timestamp=1725508343 total_threads=20
INFO [wmain] HTTP server listening | hostname="127.0.0.1" n_threads_http="19" port="55176" tid="6776" timestamp=1725508343
llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from C:\Users\12742\.ollama\models\blobs\sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Meta Llama 3.1 8B Instruct
llama_model_loader: - kv   3:                           general.finetune str              = Instruct
llama_model_loader: - kv   4:                           general.basename str              = Meta-Llama-3.1
llama_model_loader: - kv   5:                         general.size_label str              = 8B
llama_model_loader: - kv   6:                            general.license str              = llama3.1
llama_model_loader: - kv   7:                               general.tags arr[str,6]       = ["facebook", "meta", "pytorch", "llam...
llama_model_loader: - kv   8:                          general.languages arr[str,8]       = ["en", "de", "fr", "it", "pt", "hi", ...
llama_model_loader: - kv   9:                          llama.block_count u32              = 32
llama_model_loader: - kv  10:                       llama.context_length u32              = 131072
llama_model_loader: - kv  11:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv  12:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv  13:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv  14:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv  15:                       llama.rope.freq_base f32              = 500000.000000
llama_model_loader: - kv  16:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  17:                          general.file_type u32              = 2
llama_model_loader: - kv  18:                           llama.vocab_size u32              = 128256
llama_model_loader: - kv  19:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  20:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  21:                         tokenizer.ggml.pre str              = llama-bpe
llama_model_loader: - kv  22:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  24:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
llama_model_loader: - kv  25:                tokenizer.ggml.bos_token_id u32              = 128000
llama_model_loader: - kv  26:                tokenizer.ggml.eos_token_id u32              = 128009
llama_model_loader: - kv  27:                    tokenizer.chat_template str              = {{- bos_token }}\n{%- if custom_tools ...
llama_model_loader: - kv  28:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   66 tensors
llama_model_loader: - type q4_0:  225 tensors
llama_model_loader: - type q6_K:    1 tensors
time=2024-09-05T11:52:23.809+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server loading model"
llm_load_vocab: special tokens cache size = 256
llm_load_vocab: token to piece cache size = 0.7999 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 128256
llm_load_print_meta: n_merges         = 280147
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 131072
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 500000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn  = 131072
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: ssm_dt_b_c_rms   = 0
llm_load_print_meta: model type       = 8B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 8.03 B
llm_load_print_meta: model size       = 4.33 GiB (4.64 BPW)
llm_load_print_meta: general.name     = Meta Llama 3.1 8B Instruct
llm_load_print_meta: BOS token        = 128000 '<|begin_of_text|>'
llm_load_print_meta: EOS token        = 128009 '<|eot_id|>'
llm_load_print_meta: LF token         = 128 'Ä'
llm_load_print_meta: EOT token        = 128009 '<|eot_id|>'
llm_load_print_meta: max token length = 256
ggml_sycl_init: GGML_SYCL_FORCE_MMQ:   no
ggml_sycl_init: SYCL_USE_XMX: yes
ggml_sycl_init: found 1 SYCL devices:
llm_load_tensors: ggml ctx size =    0.27 MiB
llm_load_tensors: offloading 32 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 33/33 layers to GPU
llm_load_tensors:      SYCL0 buffer size =  4156.00 MiB
llm_load_tensors:  SYCL_Host buffer size =   281.81 MiB
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 500000.0
llama_new_context_with_model: freq_scale = 1
[SYCL] call ggml_check_sycl
ggml_check_sycl: GGML_SYCL_DEBUG: 0
ggml_check_sycl: GGML_SYCL_F16: no
found 1 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|               Intel Arc A730M Graphics|    1.5|    384|    1024|   32| 12514M|            1.3.30398|
llama_kv_cache_init:      SYCL0 KV buffer size =   256.00 MiB
llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_new_context_with_model:  SYCL_Host  output buffer size =     0.50 MiB
llama_new_context_with_model:      SYCL0 compute buffer size =   258.50 MiB
llama_new_context_with_model:  SYCL_Host compute buffer size =    12.01 MiB
llama_new_context_with_model: graph nodes  = 1062
llama_new_context_with_model: graph splits = 2
INFO [wmain] model loaded | tid="6776" timestamp=1725508352
time=2024-09-05T11:52:32.341+08:00 level=INFO source=server.go:634 msg="llama runner started in 8.80 seconds"

image

OS

Linux, Windows

GPU

Intel

CPU

Intel

Ollama version

0.3.6-ipexllm-20240905

Originally created by @cyear on GitHub (Sep 5, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6649 ### What is the issue? qwen4b works fine, all other models larger than 4b are gibberish ``` time=2024-09-05T11:35:49.569+08:00 level=INFO source=download.go:175 msg="downloading 8eeb52dfb3bb in 16 291 MB part(s)" time=2024-09-05T11:37:19.112+08:00 level=INFO source=download.go:370 msg="8eeb52dfb3bb part 0 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection." time=2024-09-05T11:37:21.112+08:00 level=INFO source=download.go:370 msg="8eeb52dfb3bb part 4 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection." [GIN] 2024/09/05 - 11:41:40 | 200 | 5m55s | 10.0.0.18 | POST "/api/pull" [GIN] 2024/09/05 - 11:51:04 | 200 | 1.182ms | 10.0.0.18 | GET "/api/tags" [GIN] 2024/09/05 - 11:51:05 | 200 | 0s | 10.0.0.18 | GET "/api/version" [GIN] 2024/09/05 - 11:51:24 | 200 | 510.7µs | 10.0.0.18 | GET "/api/version" [GIN] 2024/09/05 - 11:51:33 | 200 | 0s | 10.0.0.18 | GET "/api/version" time=2024-09-05T11:51:51.177+08:00 level=INFO source=download.go:175 msg="downloading 8eeb52dfb3bb in 16 291 MB part(s)" time=2024-09-05T11:51:58.238+08:00 level=INFO source=download.go:175 msg="downloading 73b313b5552d in 1 1.4 KB part(s)" time=2024-09-05T11:52:01.269+08:00 level=INFO source=download.go:175 msg="downloading 0ba8f0e314b4 in 1 12 KB part(s)" time=2024-09-05T11:52:04.339+08:00 level=INFO source=download.go:175 msg="downloading 56bb8bd477a5 in 1 96 B part(s)" time=2024-09-05T11:52:07.492+08:00 level=INFO source=download.go:175 msg="downloading 1a4c3c319823 in 1 485 B part(s)" [GIN] 2024/09/05 - 11:52:14 | 200 | 28.5001976s | 10.0.0.18 | POST "/api/pull" [GIN] 2024/09/05 - 11:52:14 | 200 | 1.0817ms | 10.0.0.18 | GET "/api/tags" [GIN] 2024/09/05 - 11:52:18 | 200 | 0s | 10.0.0.18 | GET "/api/version" time=2024-09-05T11:52:23.514+08:00 level=INFO source=memory.go:309 msg="offload to cpu" layers.requested=-1 layers.model=33 layers.offload=0 layers.split="" memory.available="[20.3 GiB]" memory.required.full="4.6 GiB" memory.required.partial="0 B" memory.required.kv="256.0 MiB" memory.required.allocations="[4.6 GiB]" memory.weights.total="3.9 GiB" memory.weights.repeating="3.5 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB" time=2024-09-05T11:52:23.520+08:00 level=INFO source=server.go:395 msg="starting llama server" cmd="C:\\Users\\12742\\Desktop\\llama-cpp\\dist\\windows-amd64\\lib\\ollama\\runners\\cpu_avx2\\ollama_llama_server.exe --model C:\\Users\\12742\\.ollama\\models\\blobs\\sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 999 --no-mmap --parallel 1 --port 55176" time=2024-09-05T11:52:23.546+08:00 level=INFO source=sched.go:450 msg="loaded runners" count=1 time=2024-09-05T11:52:23.546+08:00 level=INFO source=server.go:595 msg="waiting for llama runner to start responding" time=2024-09-05T11:52:23.547+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server error" INFO [wmain] build info | build=1 commit="c455d1d" tid="6776" timestamp=1725508343 INFO [wmain] system info | n_threads=14 n_threads_batch=-1 system_info="AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="6776" timestamp=1725508343 total_threads=20 INFO [wmain] HTTP server listening | hostname="127.0.0.1" n_threads_http="19" port="55176" tid="6776" timestamp=1725508343 llama_model_loader: loaded meta data with 29 key-value pairs and 292 tensors from C:\Users\12742\.ollama\models\blobs\sha256-8eeb52dfb3bb9aefdf9d1ef24b3bdbcfbe82238798c4b918278320b6fcef18fe (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = llama llama_model_loader: - kv 1: general.type str = model llama_model_loader: - kv 2: general.name str = Meta Llama 3.1 8B Instruct llama_model_loader: - kv 3: general.finetune str = Instruct llama_model_loader: - kv 4: general.basename str = Meta-Llama-3.1 llama_model_loader: - kv 5: general.size_label str = 8B llama_model_loader: - kv 6: general.license str = llama3.1 llama_model_loader: - kv 7: general.tags arr[str,6] = ["facebook", "meta", "pytorch", "llam... llama_model_loader: - kv 8: general.languages arr[str,8] = ["en", "de", "fr", "it", "pt", "hi", ... llama_model_loader: - kv 9: llama.block_count u32 = 32 llama_model_loader: - kv 10: llama.context_length u32 = 131072 llama_model_loader: - kv 11: llama.embedding_length u32 = 4096 llama_model_loader: - kv 12: llama.feed_forward_length u32 = 14336 llama_model_loader: - kv 13: llama.attention.head_count u32 = 32 llama_model_loader: - kv 14: llama.attention.head_count_kv u32 = 8 llama_model_loader: - kv 15: llama.rope.freq_base f32 = 500000.000000 llama_model_loader: - kv 16: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 llama_model_loader: - kv 17: general.file_type u32 = 2 llama_model_loader: - kv 18: llama.vocab_size u32 = 128256 llama_model_loader: - kv 19: llama.rope.dimension_count u32 = 128 llama_model_loader: - kv 20: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 21: tokenizer.ggml.pre str = llama-bpe llama_model_loader: - kv 22: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 23: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 24: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... llama_model_loader: - kv 25: tokenizer.ggml.bos_token_id u32 = 128000 llama_model_loader: - kv 26: tokenizer.ggml.eos_token_id u32 = 128009 llama_model_loader: - kv 27: tokenizer.chat_template str = {{- bos_token }}\n{%- if custom_tools ... llama_model_loader: - kv 28: general.quantization_version u32 = 2 llama_model_loader: - type f32: 66 tensors llama_model_loader: - type q4_0: 225 tensors llama_model_loader: - type q6_K: 1 tensors time=2024-09-05T11:52:23.809+08:00 level=INFO source=server.go:629 msg="waiting for server to become available" status="llm server loading model" llm_load_vocab: special tokens cache size = 256 llm_load_vocab: token to piece cache size = 0.7999 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = llama llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 128256 llm_load_print_meta: n_merges = 280147 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 131072 llm_load_print_meta: n_embd = 4096 llm_load_print_meta: n_layer = 32 llm_load_print_meta: n_head = 32 llm_load_print_meta: n_head_kv = 8 llm_load_print_meta: n_rot = 128 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 4 llm_load_print_meta: n_embd_k_gqa = 1024 llm_load_print_meta: n_embd_v_gqa = 1024 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.0e-05 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 14336 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 500000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 131072 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: ssm_dt_b_c_rms = 0 llm_load_print_meta: model type = 8B llm_load_print_meta: model ftype = Q4_0 llm_load_print_meta: model params = 8.03 B llm_load_print_meta: model size = 4.33 GiB (4.64 BPW) llm_load_print_meta: general.name = Meta Llama 3.1 8B Instruct llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>' llm_load_print_meta: EOS token = 128009 '<|eot_id|>' llm_load_print_meta: LF token = 128 'Ä' llm_load_print_meta: EOT token = 128009 '<|eot_id|>' llm_load_print_meta: max token length = 256 ggml_sycl_init: GGML_SYCL_FORCE_MMQ: no ggml_sycl_init: SYCL_USE_XMX: yes ggml_sycl_init: found 1 SYCL devices: llm_load_tensors: ggml ctx size = 0.27 MiB llm_load_tensors: offloading 32 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 33/33 layers to GPU llm_load_tensors: SYCL0 buffer size = 4156.00 MiB llm_load_tensors: SYCL_Host buffer size = 281.81 MiB llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 500000.0 llama_new_context_with_model: freq_scale = 1 [SYCL] call ggml_check_sycl ggml_check_sycl: GGML_SYCL_DEBUG: 0 ggml_check_sycl: GGML_SYCL_F16: no found 1 SYCL devices: | | | | |Max | |Max |Global | | | | | | |compute|Max work|sub |mem | | |ID| Device Type| Name|Version|units |group |group|size | Driver version| |--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------| | 0| [level_zero:gpu:0]| Intel Arc A730M Graphics| 1.5| 384| 1024| 32| 12514M| 1.3.30398| llama_kv_cache_init: SYCL0 KV buffer size = 256.00 MiB llama_new_context_with_model: KV self size = 256.00 MiB, K (f16): 128.00 MiB, V (f16): 128.00 MiB llama_new_context_with_model: SYCL_Host output buffer size = 0.50 MiB llama_new_context_with_model: SYCL0 compute buffer size = 258.50 MiB llama_new_context_with_model: SYCL_Host compute buffer size = 12.01 MiB llama_new_context_with_model: graph nodes = 1062 llama_new_context_with_model: graph splits = 2 INFO [wmain] model loaded | tid="6776" timestamp=1725508352 time=2024-09-05T11:52:32.341+08:00 level=INFO source=server.go:634 msg="llama runner started in 8.80 seconds" ``` ![image](https://github.com/user-attachments/assets/616e39ab-9f78-48f3-8f86-fbc65a7b87d6) ### OS Linux, Windows ### GPU Intel ### CPU Intel ### Ollama version 0.3.6-ipexllm-20240905
GiteaMirror added the intelbug labels 2026-04-28 16:48:16 -05:00
GiteaMirror commented 2026-04-28 16:48:18 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Sep 5, 2024):

Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps.

<!-- gh-comment-id:2332151687 --> @dhiltgen commented on GitHub (Sep 5, 2024): Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps.
GiteaMirror commented 2026-04-28 16:48:18 -05:00
Author
Owner
Copy Link

@cyear commented on GitHub (Sep 5, 2024):

Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps.

I have tried, and only when it is 0, the output is normal, but the GPU will not be used

<!-- gh-comment-id:2332157051 --> @cyear commented on GitHub (Sep 5, 2024): > Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps. I have tried, and only when it is 0, the output is normal, but the GPU will not be used
GiteaMirror commented 2026-04-28 16:48:19 -05:00
Author
Owner
Copy Link

@cyear commented on GitHub (Sep 5, 2024):

Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps.

However, I only encountered one correct response, which was a 11b model. Perhaps it was a coincidence, but I feel that the slightly larger model is not working properly on my GPU

<!-- gh-comment-id:2332168488 --> @cyear commented on GitHub (Sep 5, 2024): > Intel GPUs aren't officially supported yet, but often this behavior is related to loading too many layers. You can try to set num_gpu to a lower value and see if that helps. However, I only encountered one correct response, which was a 11b model. Perhaps it was a coincidence, but I feel that the slightly larger model is not working properly on my GPU
GiteaMirror commented 2026-04-28 16:48:19 -05:00
Author
Owner
Copy Link

@ayttop commented on GitHub (Sep 9, 2024):

https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md

<!-- gh-comment-id:2338762275 --> @ayttop commented on GitHub (Sep 9, 2024): https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md
GiteaMirror commented 2026-04-28 16:48:20 -05:00
Author
Owner
Copy Link

@cyear commented on GitHub (Sep 9, 2024):

https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md

Just looking at this operation

<!-- gh-comment-id:2338769745 --> @cyear commented on GitHub (Sep 9, 2024): > https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md Just looking at this operation
GiteaMirror commented 2026-04-28 16:48:20 -05:00
Author
Owner
Copy Link

@pepijndevos commented on GitHub (Nov 10, 2024):

I also have problems with larger models not working. I can go up to about 14b on my a770 but above that it simply doesn't work. I'm still debugging why.

<!-- gh-comment-id:2466566295 --> @pepijndevos commented on GitHub (Nov 10, 2024): I also have problems with larger models not working. I can go up to about 14b on my a770 but above that it simply doesn't work. I'm still debugging why.
GiteaMirror commented 2026-04-28 16:48:21 -05:00
Author
Owner
Copy Link

@SiewwenL commented on GitHub (Feb 19, 2025):

Hi @cyear ,you can refer to this github issue https://github.com/intel/ipex-llm/issues/12761 and see if this can helps you.

<!-- gh-comment-id:2667538891 --> @SiewwenL commented on GitHub (Feb 19, 2025): Hi @cyear ,you can refer to this github issue https://github.com/intel/ipex-llm/issues/12761 and see if this can helps you.
GiteaMirror commented 2026-04-28 16:48:21 -05:00
Author
Owner
Copy Link

@cyear commented on GitHub (Feb 19, 2025):

Hi @cyear ,you can refer to this github issue https://github.com/intel/ipex-llm/issues/12761 and see if this can helps you.

Perhaps it will be effective. This problem occurs more frequently when using the webui. Thank you very much. I will try it when I have time

<!-- gh-comment-id:2667547451 --> @cyear commented on GitHub (Feb 19, 2025): > Hi @cyear ,you can refer to this github issue https://github.com/intel/ipex-llm/issues/12761 and see if this can helps you. Perhaps it will be effective. This problem occurs more frequently when using the webui. Thank you very much. I will try it when I have time
GiteaMirror commented 2026-04-28 16:48:21 -05:00
Author
Owner
Copy Link

@NeoZhangJianyu commented on GitHub (Feb 20, 2025):

@cyear
Because you use ipex-llm, instead of ollama directly, suggest asking support to ipex-llm.
ipex-llm would update the ollama & llama.cpp with special code.
It's hard to trouble shooting by ollama & llama.cpp teams.

If you could reproduce the issue on ollama official release, it's OK to get support from ollama & llama.cpp teams.
Please provide the detailed log on official ollama.

<!-- gh-comment-id:2670184239 --> @NeoZhangJianyu commented on GitHub (Feb 20, 2025): @cyear Because you use ipex-llm, instead of ollama directly, suggest asking support to ipex-llm. ipex-llm would update the ollama & llama.cpp with special code. It's hard to trouble shooting by ollama & llama.cpp teams. If you could reproduce the issue on ollama official release, it's OK to get support from ollama & llama.cpp teams. Please provide the detailed log on official ollama.
GiteaMirror commented 2026-04-28 16:48:22 -05:00
Author
Owner
Copy Link

@desmondsow commented on GitHub (Feb 20, 2025):

The fastest way to verify this is to use the latest container https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md#start-docker-container.

This should fix the gibberish output issue that you are facing. Also, I believe openwebui add additional prompt when feeding the input to ollama. Use ollama run <model> to verify.

Ensure you have the latest Ubuntu 24.10, sudo apt upgrade and reboot to upgrade to the latest kernel version. Provide the sycl-ls output from the container.

<!-- gh-comment-id:2670397526 --> @desmondsow commented on GitHub (Feb 20, 2025): The fastest way to verify this is to use the latest container https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md#start-docker-container. This should fix the gibberish output issue that you are facing. Also, I believe openwebui add additional prompt when feeding the input to ollama. Use `ollama run <model>` to verify. Ensure you have the latest Ubuntu 24.10, `sudo apt upgrade` and reboot to upgrade to the latest kernel version. Provide the `sycl-ls` output from the container.
GiteaMirror commented 2026-04-28 16:48:22 -05:00
Author
Owner
Copy Link

@cyear commented on GitHub (Feb 20, 2025):

The fastest way to verify this is to use the latest container https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md#start-docker-container.

This should fix the gibberish output issue that you are facing. Also, I believe openwebui add additional prompt when feeding the input to ollama. Use ollama run <model> to verify.

Ensure you have the latest Ubuntu 24.10, sudo apt upgrade and reboot to upgrade to the latest kernel version. Provide the sycl-ls output from the container.

Unfortunately, I am using Gentoo and currently using ipex. There may be issues compiling the backend, possibly due to a problem with my device. Currently, I am using Terminal on Windows and it is running well

I think we can close this issue now. For mobile i-cards, this graphics card may be too old

Thank you very much.

<!-- gh-comment-id:2670405513 --> @cyear commented on GitHub (Feb 20, 2025): > The fastest way to verify this is to use the latest container https://github.com/intel/ipex-llm/blob/main/docs/mddocs/DockerGuides/docker_cpp_xpu_quickstart.md#start-docker-container. > > This should fix the gibberish output issue that you are facing. Also, I believe openwebui add additional prompt when feeding the input to ollama. Use `ollama run <model>` to verify. > > Ensure you have the latest Ubuntu 24.10, `sudo apt upgrade` and reboot to upgrade to the latest kernel version. Provide the `sycl-ls` output from the container. Unfortunately, I am using Gentoo and currently using ipex. There may be issues compiling the backend, possibly due to a problem with my device. Currently, I am using Terminal on Windows and it is running well I think we can close this issue now. For mobile i-cards, this graphics card may be too old Thank you very much.
GiteaMirror commented 2026-04-28 16:48:23 -05:00
Author
Owner
Copy Link

@desmondsow commented on GitHub (Feb 20, 2025):

Thanks for responding.

<!-- gh-comment-id:2670533107 --> @desmondsow commented on GitHub (Feb 20, 2025): Thanks for responding.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug intel
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#50696
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?