[GH-ISSUE #7783] Ram and Vram usage so hight #101884

New Issue

GiteaMirror · 2026-05-17T23:16:29-05:00

GiteaMirror commented

2026-05-17 23:16:29 -05:00

Originally created by @shpinog on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7783

Hello, I'm still a beginner when it comes to everything related to LLVM. My hardware is RX6800 + R7-3700X and 32GB RAM.

I use open-webui + ollama and encounter some strange behavior with models like 4Q and others around 12-13GB in size. Some models in the chat behave quite oddly, they may respond to one request or a few normally, after which they go crazy and might output "@#&&#*" or something similar.

At the same time, I notice that the 22B models take up quite a lot of VRAM, yet there is still around 1-2GB of free space. However, smaller models also take up quite a bit. The context size is set between 2048-4096 (varies).

I thought this was normal since other applications using ollama behave in a similar way, but I tried running these models with coboldcpp, using Vulkan and 40 GPU layers, and I get VRAM consumption of about 10 000MB at startup and a small RAM usage of about 5000MB (virt) and using context 8196!!! The model runs quite fast, and visually there’s no significant difference, everything happens interactively in the chat. At the same time, there are no problems with work and any “@#&&#*”.

My question is, how can I achieve this in openwebui? I understand that openwebui is essentially just a frontend over ollama in this case, but is there a way to use a more efficient and resource-optimized backend? Thank you.

Originally created by @shpinog on GitHub (Dec 11, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/7783 Hello, I'm still a beginner when it comes to everything related to LLVM. My hardware is RX6800 + R7-3700X and 32GB RAM. I use open-webui + ollama and encounter some strange behavior with models like 4Q and others around 12-13GB in size. Some models in the chat behave quite oddly, they may respond to one request or a few normally, after which they go crazy and might output "@#&&#*" or something similar. At the same time, I notice that the 22B models take up quite a lot of VRAM, yet there is still around 1-2GB of free space. However, smaller models also take up quite a bit. The context size is set between 2048-4096 (varies). I thought this was normal since other applications using ollama behave in a similar way, but I tried running these models with coboldcpp, using Vulkan and 40 GPU layers, and I get VRAM consumption of about 10 000MB at startup and a small RAM usage of about 5000MB (virt) and using context 8196!!! The model runs quite fast, and visually there’s no significant difference, everything happens interactively in the chat. At the same time, there are no problems with work and any “@#&&#*”. My question is, how can I achieve this in openwebui? I understand that openwebui is essentially just a frontend over ollama in this case, but is there a way to use a more efficient and resource-optimized backend? Thank you.

GiteaMirror closed this issue

2026-05-17 23:16:30 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#101884