Mixtral support #80

New Issue

GiteaMirror · 2025-11-11T14:04:09-06:00

GiteaMirror commented

2025-11-11 14:04:09 -06:00

Originally created by @RandomLegend on GitHub (Dec 12, 2023).

Heyho,

afaik the mixtral branch of llama.cpp just updated to be able to use the new 8x7b model.

Will this be available here aswell? If yes, do you have an estimate on when?

Originally created by @RandomLegend on GitHub (Dec 12, 2023). Heyho, afaik the mixtral branch of llama.cpp just [updated](https://github.com/ggerganov/llama.cpp/pull/4406) to be able to use the new [8x7b model](https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF). Will this be available here aswell? If yes, do you have an estimate on when?

GiteaMirror closed this issue

2025-11-11 14:04:09 -06:00

GiteaMirror commented

2025-11-11 14:04:11 -06:00

@r7l commented on GitHub (Dec 12, 2023):

@RandomLegend Not a maintainer here but this seems to be the wrong repository for the issue. This is just the UI and not the Ollama backend.

Even so, how should anyone give you an estimate? This isn't even part of the original llama.cpp code. It's just a work in progress PR and not even merged. So you shouldn't expect this in any of the projects (seems like you've opened multiple issues) prior to the PR being merged in upstream llama.cpp.

If you're in such a hurry, you can compile llama.cpp yourself and use the working branch of that PR.

@r7l commented on GitHub (Dec 12, 2023): @RandomLegend Not a maintainer here but this seems to be the wrong repository for the issue. This is just the UI and not the Ollama backend. Even so, how should anyone give you an estimate? This isn't even part of the original llama.cpp code. It's just a work in progress PR and not even merged. So you shouldn't expect this in any of the projects (seems like you've opened multiple issues) prior to the PR being merged in upstream llama.cpp. If you're in such a hurry, you can compile llama.cpp yourself and use the working branch of that PR.

GiteaMirror commented

2025-11-11 14:04:12 -06:00

@RandomLegend commented on GitHub (Dec 12, 2023):

Yup, correct about the wrong repo. My bad.

I already built the mixtral branch and it works for me, but it would be awesome to have this in the webui, maybe they could add it as an auto-detect if we use the mixtral model or as option.

@RandomLegend commented on GitHub (Dec 12, 2023): Yup, correct about the wrong repo. My bad. I already built the mixtral branch and it works for me, but it would be awesome to have this in the webui, maybe they could add it as an auto-detect if we use the mixtral model or as option.