I have installed a model and run with MLX server, but I don't know how to use in this web UI. I run the server at localhost:9000 and I added it to Manage OpenAI API Connections, but it cannot find the model.
I don't want to use Ollama since it needs more memory than MLX.
Originally created by @HarryK4673 on GitHub (Feb 6, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9508
Hi everyone!
I have installed a model and run with MLX server, but I don't know how to use in this web UI. I run the server at `localhost:9000` and I added it to `Manage OpenAI API Connections`, but it cannot find the model.
I don't want to use Ollama since it needs more memory than MLX.
Not working. It seems localhost:9000/v1/models returns an empty list, but I can use it through request in Python.
<!-- gh-comment-id:2640350220 -->
@HarryK4673 commented on GitHub (Feb 6, 2025):
> try localhost:9000/v1
Not working. It seems `localhost:9000/v1/models` returns an empty list, but I can use it through `request` in Python.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @HarryK4673 on GitHub (Feb 6, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9508
Hi everyone!
I have installed a model and run with MLX server, but I don't know how to use in this web UI. I run the server at
localhost:9000and I added it toManage OpenAI API Connections, but it cannot find the model.I don't want to use Ollama since it needs more memory than MLX.
@Anaphylaxis commented on GitHub (Feb 6, 2025):
try localhost:9000/v1
@HarryK4673 commented on GitHub (Feb 6, 2025):
Not working. It seems
localhost:9000/v1/modelsreturns an empty list, but I can use it throughrequestin Python.