Hi
My system has a NVDIA 1070 8 GB GPU card running linux mint
I used the latest Docker images as installed on the 15th May
I have installed the Docker image with Ollama installed and it runs very slow.
I then installed the Docker image with Ollama and Open WebUI seperately and it runs the fastest.
I then ran the same prompt directly on the Ollama terminal and it had similar speed.
The prompt was :
Write me a python code for the game centipede
Model LLama 3:lastest
The code generated by Ollama directly and with a seperate Ollama/Open WebUI docker image was the same and used the pygame library. I did not test the code but is used clasees, and more advanced code
The code generated by the combined Olllama/Webui docker image create a very small code using no libraries and I doubt it would have run.
The error is that the combined Ollama/OpenWebUI docker miage creates carbage and is a lot slower.
Error with ModelFilles
Also I coulld not import the modelfiles from the website. It just does nothing. Also when you import the model it looks for ".json" files but the downloaded modelfile have a ".txt" extension. I would presume the error is significant. All model files are on the recommended home page of the website and should be compatible.
Originally created by @PietFourie on GitHub (May 16, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/2306
Hi
My system has a NVDIA 1070 8 GB GPU card running linux mint
I used the latest Docker images as installed on the 15th May
1) I have installed the Docker image with Ollama installed and it runs very slow.
2) I then installed the Docker image with Ollama and Open WebUI seperately and it runs the fastest.
3) I then ran the same prompt directly on the Ollama terminal and it had similar speed.
The prompt was :
Write me a python code for the game centipede
Model LLama 3:lastest
1) The code generated by Ollama directly and with a seperate Ollama/Open WebUI docker image was the same and used the pygame library. I did not test the code but is used clasees, and more advanced code
2) The code generated by the combined Olllama/Webui docker image create a very small code using no libraries and I doubt it would have run.
The error is that the combined Ollama/OpenWebUI docker miage creates carbage and is a lot slower.
Error with ModelFilles
Also I coulld not import the modelfiles from the website. It just does nothing. Also when you import the model it looks for ".json" files but the downloaded modelfile have a ".txt" extension. I would presume the error is significant. All model files are on the recommended home page of the website and should be compatible.
<!-- gh-comment-id:2114482299 -->
@PietFourie commented on GitHub (May 16, 2024):
Here is the code generated
[Ollama direct Inference Lama3.txt](https://github.com/open-webui/open-webui/files/15331931/Ollama.direct.Inference.Lama3.txt)
[Open WEBUI and Ollama Seperated Docker images.txt](https://github.com/open-webui/open-webui/files/15331932/Open.WEBUI.and.Ollama.Seperated.Docker.images.txt)
[OpenWebUI with Ollama InStalled Inference Lama3.txt](https://github.com/open-webui/open-webui/files/15331933/OpenWebUI.with.Ollama.InStalled.Inference.Lama3.txt)
GPU memory used in the combined version was lower than in the separate Ollama docker images. It also used CPU cores.
<!-- gh-comment-id:2114485835 -->
@PietFourie commented on GitHub (May 16, 2024):
GPU memory used in the combined version was lower than in the separate Ollama docker images. It also used CPU cores.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @PietFourie on GitHub (May 16, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/2306
Hi
My system has a NVDIA 1070 8 GB GPU card running linux mint
I used the latest Docker images as installed on the 15th May
The prompt was :
Write me a python code for the game centipede
Model LLama 3:lastest
The error is that the combined Ollama/OpenWebUI docker miage creates carbage and is a lot slower.
Error with ModelFilles
Also I coulld not import the modelfiles from the website. It just does nothing. Also when you import the model it looks for ".json" files but the downloaded modelfile have a ".txt" extension. I would presume the error is significant. All model files are on the recommended home page of the website and should be compatible.
@PietFourie commented on GitHub (May 16, 2024):
Here is the code generated
Ollama direct Inference Lama3.txt
Open WEBUI and Ollama Seperated Docker images.txt
OpenWebUI with Ollama InStalled Inference Lama3.txt
@PietFourie commented on GitHub (May 16, 2024):
GPU memory used in the combined version was lower than in the separate Ollama docker images. It also used CPU cores.