github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 21:25:46 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5563] glm-4-9b-chat responding not correctly #81158

New Issue
Closed
opened 2026-05-09 10:34:47 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-05-09 10:34:47 -05:00
Owner
Copy Link

Originally created by @loveyume520 on GitHub (Jul 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5563

What is the issue?

What is the issue?
glm-4-9b-chat sometimes output not correctly:

(base) PS G:\glm\repository> ollama list
NAME                            ID              SIZE    MODIFIED
glm-4-9b-chat:latest            3e2d51d40440    10.0 GB 20 minutes ago
(base) PS G:\glm\repository> ollama run glm-4-9b-chat
>>> hi
Hi, how can I assist you?

>>> who are u
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG

>>> what
GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG

Other models responding correctly.

Here is my modelfile:

FROM ./glm-4-9b-chat-Q8_0.gguf
TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"
PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER stop <|eot_id|>
SYSTEM Format the reply in MarkDown format.

Ollama server output:

(base) PS G:\glm\repository> ollama serve
2024/07/09 14:49:48 routes.go:1033: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS:C:\\Users\\account\\.ollama\\models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR:C:\\Users\\account\\AppData\\Local\\Programs\\Ollama\\ollama_runners OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-07-09T14:49:48.921 level=INFO source=images.go:751 msg="total blobs: 14"
time=2024-07-09T14:49:48.922 level=INFO source=images.go:758 msg="total unused blobs removed: 0"
time=2024-07-09T14:49:48.923 level=INFO source=routes.go:1080 msg="Listening on 127.0.0.1:11434 (version 0.2.0)"
time=2024-07-09T14:49:48.924 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11.3 rocm_v5.7]"
time=2024-07-09T14:49:48.924 level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
time=2024-07-09T14:49:49.269 level=INFO source=types.go:103 msg="inference compute" id=0 library=rocm compute=gfx1030 driver=0.0 name="AMD Radeon RX 6800 XT" total="16.0 GiB" available="15.9 GiB"
[GIN] 2024/07/09 - 14:50:18 | 200 |            0s |       127.0.0.1 | GET      "/"
[GIN] 2024/07/09 - 14:51:34 | 200 |            0s |       127.0.0.1 | HEAD     "/"
[GIN] 2024/07/09 - 14:51:34 | 200 |     12.8023ms |       127.0.0.1 | POST     "/api/show"
time=2024-07-09T14:51:35.283 level=INFO source=sched.go:188 msg="one or more GPUs detected that are unable to accurately report free memory - disabling default concurrency"
time=2024-07-09T14:51:35.300 level=INFO source=sched.go:741 msg="new model will fit in available VRAM in single GPU, loading" model=C:\Users\account\.ollama\models\blobs\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a gpu=0 parallel=4 available=17028874240 required="10.5 GiB"
time=2024-07-09T14:51:35.301 level=INFO source=memory.go:309 msg="offload to rocm" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[15.9 GiB]" memory.required.full="10.5 GiB" memory.required.partial="10.5 GiB" memory.required.kv="320.0 MiB" memory.required.allocations="[10.5 GiB]" memory.weights.total="8.4 GiB" memory.weights.repeating="7.8 GiB" memory.weights.nonrepeating="629.0 MiB" memory.graph.full="853.3 MiB" memory.graph.partial="853.3 MiB"
time=2024-07-09T14:51:35.307 level=INFO source=server.go:375 msg="starting llama server" cmd="C:\\Users\\account\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\rocm_v5.7\\ollama_llama_server.exe --model C:\\Users\\account\\.ollama\\models\\blobs\\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --parallel 4 --port 5559"
time=2024-07-09T14:51:35.310 level=INFO source=sched.go:477 msg="loaded runners" count=1
time=2024-07-09T14:51:35.311 level=INFO source=server.go:563 msg="waiting for llama runner to start responding"
time=2024-07-09T14:51:35.311 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error"
INFO [wmain] build info | build=3337 commit="a8db2a9c" tid="8704" timestamp=1720507895
INFO [wmain] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="8704" timestamp=1720507895 total_threads=12
INFO [wmain] HTTP server listening | hostname="127.0.0.1" n_threads_http="11" port="5559" tid="8704" timestamp=1720507895
llama_model_loader: loaded meta data with 24 key-value pairs and 283 tensors from C:\Users\account\.ollama\models\blobs\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = chatglm
llama_model_loader: - kv   1:                               general.name str              = glm-4-9b-chat
llama_model_loader: - kv   2:                     chatglm.context_length u32              = 131072
llama_model_loader: - kv   3:                   chatglm.embedding_length u32              = 4096
llama_model_loader: - kv   4:                chatglm.feed_forward_length u32              = 13696
llama_model_loader: - kv   5:                        chatglm.block_count u32              = 40
llama_model_loader: - kv   6:               chatglm.attention.head_count u32              = 32
llama_model_loader: - kv   7:            chatglm.attention.head_count_kv u32              = 2
llama_model_loader: - kv   8:   chatglm.attention.layer_norm_rms_epsilon f32              = 0.000000
llama_model_loader: - kv   9:                          general.file_type u32              = 7
llama_model_loader: - kv  10:               chatglm.rope.dimension_count u32              = 64
llama_model_loader: - kv  11:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  12:                     chatglm.rope.freq_base f32              = 5000000.000000
llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  14:                         tokenizer.ggml.pre str              = chatglm-bpe
llama_model_loader: - kv  15:                      tokenizer.ggml.tokens arr[str,151552]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  16:                  tokenizer.ggml.token_type arr[i32,151552]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  17:                      tokenizer.ggml.merges arr[str,151073]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  18:            tokenizer.ggml.padding_token_id u32              = 151329
llama_model_loader: - kv  19:                tokenizer.ggml.eos_token_id u32              = 151329
llama_model_loader: - kv  20:                tokenizer.ggml.eot_token_id u32              = 151336
llama_model_loader: - kv  21:            tokenizer.ggml.unknown_token_id u32              = 151329
llama_model_loader: - kv  22:                    tokenizer.chat_template str              = [gMASK]<sop>{% for item in messages %...
llama_model_loader: - kv  23:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:  121 tensors
llama_model_loader: - type q8_0:  162 tensors
llm_load_vocab: special tokens cache size = 223
llm_load_vocab: token to piece cache size = 0.9732 MB
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = chatglm
llm_load_print_meta: vocab type       = BPE
llm_load_print_meta: n_vocab          = 151552
llm_load_print_meta: n_merges         = 151073
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 131072
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_layer          = 40
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 2
llm_load_print_meta: n_rot            = 64
llm_load_print_meta: n_swa            = 0
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 16
llm_load_print_meta: n_embd_k_gqa     = 256
llm_load_print_meta: n_embd_v_gqa     = 256
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.6e-07
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale    = 0.0e+00
llm_load_print_meta: n_ff             = 13696
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: causal attn      = 1
llm_load_print_meta: pooling type     = 0
llm_load_print_meta: rope type        = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 5000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn  = 131072
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: ssm_d_conv       = 0
llm_load_print_meta: ssm_d_inner      = 0
llm_load_print_meta: ssm_d_state      = 0
llm_load_print_meta: ssm_dt_rank      = 0
llm_load_print_meta: model type       = 9B
llm_load_print_meta: model ftype      = Q8_0
llm_load_print_meta: model params     = 9.40 B
llm_load_print_meta: model size       = 9.30 GiB (8.50 BPW)
llm_load_print_meta: general.name     = glm-4-9b-chat
llm_load_print_meta: EOS token        = 151329 '<|endoftext|>'
llm_load_print_meta: UNK token        = 151329 '<|endoftext|>'
llm_load_print_meta: PAD token        = 151329 '<|endoftext|>'
llm_load_print_meta: LF token         = 128 'Ä'
llm_load_print_meta: EOT token        = 151336 '<|user|>'
llm_load_print_meta: max token length = 1024
time=2024-07-09T14:51:35.576 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model"
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon RX 6800 XT, compute capability 10.3, VMM: no
llm_load_tensors: ggml ctx size =    0.28 MiB
llm_load_tensors: offloading 40 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 41/41 layers to GPU
llm_load_tensors:      ROCm0 buffer size =  8897.23 MiB
llm_load_tensors:        CPU buffer size =   629.00 MiB
llama_new_context_with_model: n_ctx      = 8192
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base  = 5000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      ROCm0 KV buffer size =   320.00 MiB
llama_new_context_with_model: KV self size  =  320.00 MiB, K (f16):  160.00 MiB, V (f16):  160.00 MiB
llama_new_context_with_model:  ROCm_Host  output buffer size =     2.38 MiB
llama_new_context_with_model:      ROCm0 compute buffer size =   561.00 MiB
llama_new_context_with_model:  ROCm_Host compute buffer size =    24.01 MiB
llama_new_context_with_model: graph nodes  = 1606
llama_new_context_with_model: graph splits = 2
INFO [wmain] model loaded | tid="8704" timestamp=1720507900
time=2024-07-09T14:51:40.994 level=INFO source=server.go:609 msg="llama runner started in 5.68 seconds"
[GIN] 2024/07/09 - 14:51:40 | 200 |    6.0141945s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/07/09 - 14:52:10 | 200 |    3.4439825s |       127.0.0.1 | POST     "/v1/chat/completions"
[GIN] 2024/07/09 - 14:52:10 | 200 |    3.4856972s |       127.0.0.1 | POST     "/v1/chat/completions"
[GIN] 2024/07/09 - 14:52:12 | 200 |    5.4047883s |       127.0.0.1 | POST     "/v1/chat/completions"

OS

Windows

GPU

AMD

CPU

AMD

Ollama version

0.2.0

Originally created by @loveyume520 on GitHub (Jul 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5563 ### What is the issue? What is the issue? glm-4-9b-chat sometimes output not correctly: ``` (base) PS G:\glm\repository> ollama list NAME ID SIZE MODIFIED glm-4-9b-chat:latest 3e2d51d40440 10.0 GB 20 minutes ago (base) PS G:\glm\repository> ollama run glm-4-9b-chat >>> hi Hi, how can I assist you? >>> who are u GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG >>> what GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG ``` Other models responding correctly. Here is my modelfile: ``` FROM ./glm-4-9b-chat-Q8_0.gguf TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>" PARAMETER stop <|start_header_id|> PARAMETER stop <|end_header_id|> PARAMETER stop <|eot_id|> SYSTEM Format the reply in MarkDown format. ``` Ollama server output: ``` (base) PS G:\glm\repository> ollama serve 2024/07/09 14:49:48 routes.go:1033: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS:C:\\Users\\account\\.ollama\\models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR:C:\\Users\\account\\AppData\\Local\\Programs\\Ollama\\ollama_runners OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-07-09T14:49:48.921 level=INFO source=images.go:751 msg="total blobs: 14" time=2024-07-09T14:49:48.922 level=INFO source=images.go:758 msg="total unused blobs removed: 0" time=2024-07-09T14:49:48.923 level=INFO source=routes.go:1080 msg="Listening on 127.0.0.1:11434 (version 0.2.0)" time=2024-07-09T14:49:48.924 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11.3 rocm_v5.7]" time=2024-07-09T14:49:48.924 level=INFO source=gpu.go:205 msg="looking for compatible GPUs" time=2024-07-09T14:49:49.269 level=INFO source=types.go:103 msg="inference compute" id=0 library=rocm compute=gfx1030 driver=0.0 name="AMD Radeon RX 6800 XT" total="16.0 GiB" available="15.9 GiB" [GIN] 2024/07/09 - 14:50:18 | 200 | 0s | 127.0.0.1 | GET "/" [GIN] 2024/07/09 - 14:51:34 | 200 | 0s | 127.0.0.1 | HEAD "/" [GIN] 2024/07/09 - 14:51:34 | 200 | 12.8023ms | 127.0.0.1 | POST "/api/show" time=2024-07-09T14:51:35.283 level=INFO source=sched.go:188 msg="one or more GPUs detected that are unable to accurately report free memory - disabling default concurrency" time=2024-07-09T14:51:35.300 level=INFO source=sched.go:741 msg="new model will fit in available VRAM in single GPU, loading" model=C:\Users\account\.ollama\models\blobs\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a gpu=0 parallel=4 available=17028874240 required="10.5 GiB" time=2024-07-09T14:51:35.301 level=INFO source=memory.go:309 msg="offload to rocm" layers.requested=-1 layers.model=41 layers.offload=41 layers.split="" memory.available="[15.9 GiB]" memory.required.full="10.5 GiB" memory.required.partial="10.5 GiB" memory.required.kv="320.0 MiB" memory.required.allocations="[10.5 GiB]" memory.weights.total="8.4 GiB" memory.weights.repeating="7.8 GiB" memory.weights.nonrepeating="629.0 MiB" memory.graph.full="853.3 MiB" memory.graph.partial="853.3 MiB" time=2024-07-09T14:51:35.307 level=INFO source=server.go:375 msg="starting llama server" cmd="C:\\Users\\account\\AppData\\Local\\Programs\\Ollama\\ollama_runners\\rocm_v5.7\\ollama_llama_server.exe --model C:\\Users\\account\\.ollama\\models\\blobs\\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a --ctx-size 8192 --batch-size 512 --embedding --log-disable --n-gpu-layers 41 --parallel 4 --port 5559" time=2024-07-09T14:51:35.310 level=INFO source=sched.go:477 msg="loaded runners" count=1 time=2024-07-09T14:51:35.311 level=INFO source=server.go:563 msg="waiting for llama runner to start responding" time=2024-07-09T14:51:35.311 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server error" INFO [wmain] build info | build=3337 commit="a8db2a9c" tid="8704" timestamp=1720507895 INFO [wmain] system info | n_threads=6 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 0 | " tid="8704" timestamp=1720507895 total_threads=12 INFO [wmain] HTTP server listening | hostname="127.0.0.1" n_threads_http="11" port="5559" tid="8704" timestamp=1720507895 llama_model_loader: loaded meta data with 24 key-value pairs and 283 tensors from C:\Users\account\.ollama\models\blobs\sha256-e5e708922aaecfffaa635d6bf0a05433290462fe7c45826a7401010ddd617a6a (version GGUF V3 (latest)) llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. llama_model_loader: - kv 0: general.architecture str = chatglm llama_model_loader: - kv 1: general.name str = glm-4-9b-chat llama_model_loader: - kv 2: chatglm.context_length u32 = 131072 llama_model_loader: - kv 3: chatglm.embedding_length u32 = 4096 llama_model_loader: - kv 4: chatglm.feed_forward_length u32 = 13696 llama_model_loader: - kv 5: chatglm.block_count u32 = 40 llama_model_loader: - kv 6: chatglm.attention.head_count u32 = 32 llama_model_loader: - kv 7: chatglm.attention.head_count_kv u32 = 2 llama_model_loader: - kv 8: chatglm.attention.layer_norm_rms_epsilon f32 = 0.000000 llama_model_loader: - kv 9: general.file_type u32 = 7 llama_model_loader: - kv 10: chatglm.rope.dimension_count u32 = 64 llama_model_loader: - kv 11: tokenizer.ggml.add_bos_token bool = false llama_model_loader: - kv 12: chatglm.rope.freq_base f32 = 5000000.000000 llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2 llama_model_loader: - kv 14: tokenizer.ggml.pre str = chatglm-bpe llama_model_loader: - kv 15: tokenizer.ggml.tokens arr[str,151552] = ["!", "\"", "#", "$", "%", "&", "'", ... llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,151552] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... llama_model_loader: - kv 17: tokenizer.ggml.merges arr[str,151073] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",... llama_model_loader: - kv 18: tokenizer.ggml.padding_token_id u32 = 151329 llama_model_loader: - kv 19: tokenizer.ggml.eos_token_id u32 = 151329 llama_model_loader: - kv 20: tokenizer.ggml.eot_token_id u32 = 151336 llama_model_loader: - kv 21: tokenizer.ggml.unknown_token_id u32 = 151329 llama_model_loader: - kv 22: tokenizer.chat_template str = [gMASK]<sop>{% for item in messages %... llama_model_loader: - kv 23: general.quantization_version u32 = 2 llama_model_loader: - type f32: 121 tensors llama_model_loader: - type q8_0: 162 tensors llm_load_vocab: special tokens cache size = 223 llm_load_vocab: token to piece cache size = 0.9732 MB llm_load_print_meta: format = GGUF V3 (latest) llm_load_print_meta: arch = chatglm llm_load_print_meta: vocab type = BPE llm_load_print_meta: n_vocab = 151552 llm_load_print_meta: n_merges = 151073 llm_load_print_meta: vocab_only = 0 llm_load_print_meta: n_ctx_train = 131072 llm_load_print_meta: n_embd = 4096 llm_load_print_meta: n_layer = 40 llm_load_print_meta: n_head = 32 llm_load_print_meta: n_head_kv = 2 llm_load_print_meta: n_rot = 64 llm_load_print_meta: n_swa = 0 llm_load_print_meta: n_embd_head_k = 128 llm_load_print_meta: n_embd_head_v = 128 llm_load_print_meta: n_gqa = 16 llm_load_print_meta: n_embd_k_gqa = 256 llm_load_print_meta: n_embd_v_gqa = 256 llm_load_print_meta: f_norm_eps = 0.0e+00 llm_load_print_meta: f_norm_rms_eps = 1.6e-07 llm_load_print_meta: f_clamp_kqv = 0.0e+00 llm_load_print_meta: f_max_alibi_bias = 0.0e+00 llm_load_print_meta: f_logit_scale = 0.0e+00 llm_load_print_meta: n_ff = 13696 llm_load_print_meta: n_expert = 0 llm_load_print_meta: n_expert_used = 0 llm_load_print_meta: causal attn = 1 llm_load_print_meta: pooling type = 0 llm_load_print_meta: rope type = 0 llm_load_print_meta: rope scaling = linear llm_load_print_meta: freq_base_train = 5000000.0 llm_load_print_meta: freq_scale_train = 1 llm_load_print_meta: n_ctx_orig_yarn = 131072 llm_load_print_meta: rope_finetuned = unknown llm_load_print_meta: ssm_d_conv = 0 llm_load_print_meta: ssm_d_inner = 0 llm_load_print_meta: ssm_d_state = 0 llm_load_print_meta: ssm_dt_rank = 0 llm_load_print_meta: model type = 9B llm_load_print_meta: model ftype = Q8_0 llm_load_print_meta: model params = 9.40 B llm_load_print_meta: model size = 9.30 GiB (8.50 BPW) llm_load_print_meta: general.name = glm-4-9b-chat llm_load_print_meta: EOS token = 151329 '<|endoftext|>' llm_load_print_meta: UNK token = 151329 '<|endoftext|>' llm_load_print_meta: PAD token = 151329 '<|endoftext|>' llm_load_print_meta: LF token = 128 'Ä' llm_load_print_meta: EOT token = 151336 '<|user|>' llm_load_print_meta: max token length = 1024 time=2024-07-09T14:51:35.576 level=INFO source=server.go:604 msg="waiting for server to become available" status="llm server loading model" ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 1 ROCm devices: Device 0: AMD Radeon RX 6800 XT, compute capability 10.3, VMM: no llm_load_tensors: ggml ctx size = 0.28 MiB llm_load_tensors: offloading 40 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 41/41 layers to GPU llm_load_tensors: ROCm0 buffer size = 8897.23 MiB llm_load_tensors: CPU buffer size = 629.00 MiB llama_new_context_with_model: n_ctx = 8192 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 0 llama_new_context_with_model: freq_base = 5000000.0 llama_new_context_with_model: freq_scale = 1 llama_kv_cache_init: ROCm0 KV buffer size = 320.00 MiB llama_new_context_with_model: KV self size = 320.00 MiB, K (f16): 160.00 MiB, V (f16): 160.00 MiB llama_new_context_with_model: ROCm_Host output buffer size = 2.38 MiB llama_new_context_with_model: ROCm0 compute buffer size = 561.00 MiB llama_new_context_with_model: ROCm_Host compute buffer size = 24.01 MiB llama_new_context_with_model: graph nodes = 1606 llama_new_context_with_model: graph splits = 2 INFO [wmain] model loaded | tid="8704" timestamp=1720507900 time=2024-07-09T14:51:40.994 level=INFO source=server.go:609 msg="llama runner started in 5.68 seconds" [GIN] 2024/07/09 - 14:51:40 | 200 | 6.0141945s | 127.0.0.1 | POST "/api/chat" [GIN] 2024/07/09 - 14:52:10 | 200 | 3.4439825s | 127.0.0.1 | POST "/v1/chat/completions" [GIN] 2024/07/09 - 14:52:10 | 200 | 3.4856972s | 127.0.0.1 | POST "/v1/chat/completions" [GIN] 2024/07/09 - 14:52:12 | 200 | 5.4047883s | 127.0.0.1 | POST "/v1/chat/completions" ``` ### OS Windows ### GPU AMD ### CPU AMD ### Ollama version 0.2.0
GiteaMirror added the bug label 2026-05-09 10:34:47 -05:00
GiteaMirror commented 2026-05-09 10:34:49 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Jul 9, 2024):

@loveyume520 I think you might need to change the template, try this one: https://ollama.com/library/glm4/blobs/e7e7aebd710c

<!-- gh-comment-id:2216815840 --> @jmorganca commented on GitHub (Jul 9, 2024): @loveyume520 I think you might need to change the template, try this one: https://ollama.com/library/glm4/blobs/e7e7aebd710c
GiteaMirror commented 2026-05-09 10:34:49 -05:00
Author
Owner
Copy Link

@loveyume520 commented on GitHub (Jul 9, 2024):

@jmorganca Thank you for your help, but I have previously tried glm4:9b on Ollama and encountered the same issue. It doesn't seem to be related to the template.

<!-- gh-comment-id:2216855827 --> @loveyume520 commented on GitHub (Jul 9, 2024): @jmorganca Thank you for your help, but I have previously tried glm4:9b on Ollama and encountered the same issue. It doesn't seem to be related to the template.
GiteaMirror commented 2026-05-09 10:34:50 -05:00
Author
Owner
Copy Link

@lunlu commented on GitHub (Jul 10, 2024):

I have the same problem

<!-- gh-comment-id:2219463551 --> @lunlu commented on GitHub (Jul 10, 2024): I have the same problem
GiteaMirror commented 2026-05-09 10:34:50 -05:00
Author
Owner
Copy Link

@tobias-varden commented on GitHub (Jul 10, 2024):

Same problem, suddenly it starts spouting G's.
I pulled the model today, so I have the latest one.

<!-- gh-comment-id:2220147719 --> @tobias-varden commented on GitHub (Jul 10, 2024): Same problem, suddenly it starts spouting G's. I pulled the model today, so I have the latest one.
GiteaMirror commented 2026-05-09 10:34:51 -05:00
Author
Owner
Copy Link

@darwinvelez58 commented on GitHub (Jul 11, 2024):

Same with different models

<!-- gh-comment-id:2223000823 --> @darwinvelez58 commented on GitHub (Jul 11, 2024): Same with different models
GiteaMirror commented 2026-05-09 10:34:51 -05:00
Author
Owner
Copy Link

@tobias-varden commented on GitHub (Jul 11, 2024):

This commit might fix it, even though it is for Qwen2: https://github.com/ggerganov/llama.cpp/pull/8412

<!-- gh-comment-id:2223318808 --> @tobias-varden commented on GitHub (Jul 11, 2024): This commit might fix it, even though it is for Qwen2: https://github.com/ggerganov/llama.cpp/pull/8412
GiteaMirror commented 2026-05-09 10:34:52 -05:00
Author
Owner
Copy Link

@tin2tin commented on GitHub (Aug 20, 2024):

@tobias-varden Did the commit fix it? (I have encountered the same problem elsewhere)

<!-- gh-comment-id:2298057844 --> @tin2tin commented on GitHub (Aug 20, 2024): @tobias-varden Did the commit fix it? (I have encountered the same problem elsewhere)
GiteaMirror commented 2026-05-09 10:34:53 -05:00
Author
Owner
Copy Link

@tobias-varden commented on GitHub (Aug 20, 2024):

@tobias-varden Did the commit fix it? (I have encountered the same problem elsewhere)

I deleted the model yesterday so I cannot test it anymore.

<!-- gh-comment-id:2298585437 --> @tobias-varden commented on GitHub (Aug 20, 2024): > @tobias-varden Did the commit fix it? (I have encountered the same problem elsewhere) I deleted the model yesterday so I cannot test it anymore.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#81158
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?