github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10336] Using Ollama + OpenWebUI on AWS Tesla T4 #53301

New Issue
Closed
opened 2026-04-29 02:34:11 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-29 02:34:11 -05:00
Owner
Copy Link

Originally created by @Ara3096 on GitHub (Apr 18, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10336

What is the issue?

Ollama is really slow answering. I'm using mistral-small3.1:24b-instruct-2503-q8_0
I just installed everything using Docker on Ubuntu (AWS). Before installing the docker compose plugin and everything necessary. I ran into issues with the Nvidia toolbox. So I had to reinstall cuda. to get the issue fixed with my Nvidia server versions. "containers cudnn-linux-x86_64-8.9.7.29_cuda11-archive.tar.xz snap
cuda-keyring_1.0-1_all.deb cudnn-linux-x86_64-8.9.7.29_cuda11-archive.tar.xz.1" is shown when I list all objects within my home directory right now.
I'm running everything on a AWS EC2 Instance. It's g4dn.2xlarge which should use a Tesla T4. It's shown when I use the command sudo Nvidia-smi. But it is not utilized it still shows 3MiB / 15360MiB and there is no GPU processes running.
What went wrong? I couldn't figure it out on my own.

Image

Relevant log output

OS

Linux, Docker

GPU

Nvidia

CPU

No response

Ollama version

mistral-small3.1:24b-instruct-2503-q8_0

Originally created by @Ara3096 on GitHub (Apr 18, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10336 ### What is the issue? Ollama is really slow answering. I'm using mistral-small3.1:24b-instruct-2503-q8_0 I just installed everything using Docker on Ubuntu (AWS). Before installing the docker compose plugin and everything necessary. I ran into issues with the Nvidia toolbox. So I had to reinstall cuda. to get the issue fixed with my Nvidia server versions. "containers cudnn-linux-x86_64-8.9.7.29_cuda11-archive.tar.xz snap cuda-keyring_1.0-1_all.deb cudnn-linux-x86_64-8.9.7.29_cuda11-archive.tar.xz.1" is shown when I list all objects within my home directory right now. I'm running everything on a AWS EC2 Instance. It's g4dn.2xlarge which should use a Tesla T4. It's shown when I use the command sudo Nvidia-smi. But it is not utilized it still shows 3MiB / 15360MiB and there is no GPU processes running. What went wrong? I couldn't figure it out on my own. ![Image](https://github.com/user-attachments/assets/fc00bfc9-e876-4b42-b357-983329f8955d) ### Relevant log output ```shell ``` ### OS Linux, Docker ### GPU Nvidia ### CPU _No response_ ### Ollama version mistral-small3.1:24b-instruct-2503-q8_0
GiteaMirror added the bugneeds more info labels 2026-04-29 02:34:11 -05:00
GiteaMirror commented 2026-04-29 02:34:14 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2025):

Server logs will aid in debugging.

<!-- gh-comment-id:2815388515 --> @rick-github commented on GitHub (Apr 18, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) will aid in debugging.
GiteaMirror commented 2026-04-29 02:34:15 -05:00
Author
Owner
Copy Link

@divyansh1110 commented on GitHub (Apr 21, 2025):

Hi @rick-github ,

Did you try this

Installing Open WebUI with Bundled Ollama Support

This installation method uses a single container image that bundles Open WebUI with Ollama, allowing for a streamlined setup via a single command. Choose the appropriate command based on your hardware setup:

With GPU Support: Utilize GPU resources by running the following command:

docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama

<!-- gh-comment-id:2819475740 --> @divyansh1110 commented on GitHub (Apr 21, 2025): Hi @rick-github , Did you try this # Installing Open WebUI with Bundled Ollama Support This installation method uses a single container image that bundles Open WebUI with Ollama, allowing for a streamlined setup via a single command. Choose the appropriate command based on your hardware setup: With GPU Support: Utilize GPU resources by running the following command: `docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama`
GiteaMirror commented 2026-04-29 02:34:15 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 21, 2025):

Did you try this

Presumably the question is meant for @Ara3096.

<!-- gh-comment-id:2819481337 --> @rick-github commented on GitHub (Apr 21, 2025): > Did you try this Presumably the question is meant for @Ara3096.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#53301
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?