I can't modify the "num of gpu layers" setting in Open WebUI's Ollama. #729

New Issue

2025-11-11T14:29:59-06:00

GiteaMirror commented

2025-11-11 14:29:59 -06:00

Originally created by @Gally-Youko on GitHub (Apr 25, 2024).

Bug Report

Description

Bug Summary:
I can't modify the "num of gpu layers" setting in Open WebUI's Ollama.

Steps to Reproduce:
[Outline the steps to reproduce the bug. Be as detailed as possible.]

Expected Behavior:
How can I modify the "num of gpu layers" setting in Open WebUI's Ollama?
I checked Ollama's logs and found that the number of GPU layers loaded was 33. Then, in the Windows 10 CMD command line, I ran "ollama run llama3" followed by "/set parameter num_gpu 30", and it returned "Set parameter 'num_gpu' to '30'" (as shown on this page: "https://github.com/ollama/ollama/issues/1855"). Afterwards, the ttft and inference speed in my conversation with llama3 in the command line were significantly faster than in Open WebUI's llama3. However, when I restarted the conversation with the same model in Open WebUI, I checked Ollama's logs and found that the number of GPU layers was still 33, unchanged, indicating that changing the GPU layers in the Windows command line does not modify the same configuration in Open WebUI.

Actual Behavior:
the number of GPU layers was still 33,the ttft and inference speed in my conversation with llama3 in Open WebUI's llama3 still long and slow.

Environment

Open WebUI Version: [v0.1.120]
Ollama (if applicable): [0.1.32]
Operating System: [Windows 10]
Browser (if applicable): [Chrome]

Reproduction Details

How can I modify the "num of gpu layers" setting in Open WebUI's Ollama?
I want to change the number of GPU layers to shorten the time-to-first-token (ttft) and speed up inference when running the llama3 model on my computer. I checked Ollama's logs and found that the number of GPU layers loaded was 33. Then, in the Windows 10 CMD command line, I ran "ollama run llama3" followed by "/set parameter num_gpu 30", and it returned "Set parameter 'num_gpu' to '30'" (as shown on this page: "https://github.com/ollama/ollama/issues/1855"). Afterwards, the ttft and inference speed in my conversation with llama3 in the command line were significantly faster than in Open WebUI's llama3. However, when I restarted the conversation with the same model in Open WebUI, I checked Ollama's logs and found that the number of GPU layers was still 33, unchanged, indicating that changing the GPU layers in the Windows command line does not modify the same configuration in Open WebUI. Therefore, I would like to know how to modify the GPU layers in Open WebUI's Ollama to make my use of llama3 faster and more comfortable? (I strongly suggest adding a corresponding modification UI in Open WebUI in the future to facilitate changing GPU layers.)

[Y] I have read and followed all the instructions provided in the README.md.
[Y] I am on the latest version of both Open WebUI and Ollama.
[N] I have included the browser console logs.
[N] I have included the Docker container logs.

Installation Method

First, install Ollama through the official channels, then install Docker, and finally download and install via the command line provided by the official Open WebUI.

Additional Information

There may be a way to configure the GPU layer loading for models in Open WebUI's Ollama, but please forgive my ignorance. I have tried my best to search extensively but still have not found a way. I don't know how to program or understand programming languages, so I'm not sure where to start or how to solve this.

Originally created by @Gally-Youko on GitHub (Apr 25, 2024). # Bug Report ## Description **Bug Summary:** I can't modify the "num of gpu layers" setting in Open WebUI's Ollama. **Steps to Reproduce:** [Outline the steps to reproduce the bug. Be as detailed as possible.] **Expected Behavior:** How can I modify the "num of gpu layers" setting in Open WebUI's Ollama? I checked Ollama's logs and found that the number of GPU layers loaded was 33. Then, in the Windows 10 CMD command line, I ran "ollama run llama3" followed by "/set parameter num_gpu 30", and it returned "Set parameter 'num_gpu' to '30'" (as shown on this page: "https://github.com/ollama/ollama/issues/1855"). Afterwards, the ttft and inference speed in my conversation with llama3 in the command line were significantly faster than in Open WebUI's llama3. However, when I restarted the conversation with the same model in Open WebUI, I checked Ollama's logs and found that the number of GPU layers was still 33, unchanged, indicating that changing the GPU layers in the Windows command line does not modify the same configuration in Open WebUI. **Actual Behavior:** the number of GPU layers was still 33,the ttft and inference speed in my conversation with llama3 in Open WebUI's llama3 still long and slow. ## Environment - **Open WebUI Version:** [v0.1.120] - **Ollama (if applicable):** [0.1.32] - **Operating System:** [Windows 10] - **Browser (if applicable):** [Chrome] ## Reproduction Details How can I modify the "num of gpu layers" setting in Open WebUI's Ollama? I want to change the number of GPU layers to shorten the time-to-first-token (ttft) and speed up inference when running the llama3 model on my computer. I checked Ollama's logs and found that the number of GPU layers loaded was 33. Then, in the Windows 10 CMD command line, I ran "ollama run llama3" followed by "/set parameter num_gpu 30", and it returned "Set parameter 'num_gpu' to '30'" (as shown on this page: "https://github.com/ollama/ollama/issues/1855"). Afterwards, the ttft and inference speed in my conversation with llama3 in the command line were significantly faster than in Open WebUI's llama3. However, when I restarted the conversation with the same model in Open WebUI, I checked Ollama's logs and found that the number of GPU layers was still 33, unchanged, indicating that changing the GPU layers in the Windows command line does not modify the same configuration in Open WebUI. Therefore, I would like to know how to modify the GPU layers in Open WebUI's Ollama to make my use of llama3 faster and more comfortable? (I strongly suggest adding a corresponding modification UI in Open WebUI in the future to facilitate changing GPU layers.) - [Y] I have read and followed all the instructions provided in the README.md. - [Y] I am on the latest version of both Open WebUI and Ollama. - [N] I have included the browser console logs. - [N] I have included the Docker container logs. ## Installation Method First, install Ollama through the official channels, then install Docker, and finally download and install via the command line provided by the official Open WebUI. ## Additional Information There may be a way to configure the GPU layer loading for models in Open WebUI's Ollama, but please forgive my ignorance. I have tried my best to search extensively but still have not found a way. I don't know how to program or understand programming languages, so I'm not sure where to start or how to solve this.

GiteaMirror closed this issue