github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 04:34:16 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #13863] GLM 4.7 flash uses too much VRAM #9075

New Issue
Closed
opened 2026-04-12 21:55:05 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-12 21:55:05 -05:00
Owner
Copy Link

Originally created by @balki on GitHub (Jan 23, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/13863

What is the issue?

18GB model blows up to 145GB VRAM usage for 128K context length.

❯ ollama list | grep "glm-4.7-flash:latest"
glm-4.7-flash:latest              ff14144f31df    18 GB     29 hours ago


❯ ollama ps
NAME                    ID              SIZE      PROCESSOR          CONTEXT    UNTIL
glm-4.7-flash:latest    ff14144f31df    145 GB    35%/65% CPU/GPU    128000     4 minutes from now

From below discussion, this is likely a bug.
https://huggingface.co/zai-org/GLM-4.7-Flash/discussions/3

Relevant log output

Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1250.00 MiB on device 0: cudaMalloc failed: out of memory
Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.928-05:00 level=INFO source=server.go:877 msg="model layout did not fit, applying backoff" backoff=0.10
Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.929-05:00 level=INFO source=server.go:877 msg="model layout did not fit, applying backoff" backoff=0.20
Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.929-05:00 level=INFO source=runner.go:1278 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:128000 KvCacheType: NumThreads:16 GPULayers:35[ID:0 Layers:35(12..46)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"
Jan 23 13:53:34 cachy▒▒▒▒▒ ollama[76530]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1250.00 MiB on device 0: cudaMalloc failed: out of memory
Jan 23 13:53:35 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:35.232-05:00 level=INFO source=runner.go:1278 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:128000 KvCacheType: NumThreads:16 GPULayers:32[ID:0 Layers:32(15..46)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}"

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.14.3

Originally created by @balki on GitHub (Jan 23, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/13863 ### What is the issue? 18GB model blows up to 145GB VRAM usage for 128K context length. ``` ❯ ollama list | grep "glm-4.7-flash:latest" glm-4.7-flash:latest ff14144f31df 18 GB 29 hours ago ❯ ollama ps NAME ID SIZE PROCESSOR CONTEXT UNTIL glm-4.7-flash:latest ff14144f31df 145 GB 35%/65% CPU/GPU 128000 4 minutes from now ``` From below discussion, this is likely a bug. https://huggingface.co/zai-org/GLM-4.7-Flash/discussions/3 ### Relevant log output ```shell Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1250.00 MiB on device 0: cudaMalloc failed: out of memory Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.928-05:00 level=INFO source=server.go:877 msg="model layout did not fit, applying backoff" backoff=0.10 Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.929-05:00 level=INFO source=server.go:877 msg="model layout did not fit, applying backoff" backoff=0.20 Jan 23 13:53:29 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:29.929-05:00 level=INFO source=runner.go:1278 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:128000 KvCacheType: NumThreads:16 GPULayers:35[ID:0 Layers:35(12..46)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" Jan 23 13:53:34 cachy▒▒▒▒▒ ollama[76530]: ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1250.00 MiB on device 0: cudaMalloc failed: out of memory Jan 23 13:53:35 cachy▒▒▒▒▒ ollama[76530]: time=2026-01-23T13:53:35.232-05:00 level=INFO source=runner.go:1278 msg=load request="{Operation:alloc LoraPath:[] Parallel:1 BatchSize:512 FlashAttention:Enabled KvSize:128000 KvCacheType: NumThreads:16 GPULayers:32[ID:0 Layers:32(15..46)] MultiUserCache:false ProjectorPath: MainGPU:0 UseMmap:false}" ``` ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.14.3
GiteaMirror added the bug label 2026-04-12 21:55:05 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#9075
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?