github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #649] Request: ensemble Llamas 🦙 (llama2:13b-ensemble) #289

New Issue
Closed
opened 2026-04-12 09:49:49 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 09:49:49 -05:00
Owner
Copy Link

Originally created by @jamesbraza on GitHub (Sep 29, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/649

From Hugging Face's Open LLM leaderboard: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

A 13b model ranked somewhat highly is yeontaek/llama-2-13B-ensemble-v5.

image

I believe TheBloke exposes it here via GGUF: https://huggingface.co/TheBloke/Llama-2-13B-Ensemble-v5-GGUF

It would be cool to add it to the llama2 offerings as 13b-ensemble, 13b-ensemble-q4_0.

Originally created by @jamesbraza on GitHub (Sep 29, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/649 From Hugging Face's Open LLM leaderboard: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard A 13b model ranked somewhat highly is [`yeontaek/llama-2-13B-ensemble-v5`](https://huggingface.co/datasets/open-llm-leaderboard/details_yeontaek__llama-2-13B-ensemble-v5). ![image](https://github.com/jmorganca/ollama/assets/8990777/a602fffb-0148-4202-9608-c186a9037d4a) I believe TheBloke exposes it here via GGUF: https://huggingface.co/TheBloke/Llama-2-13B-Ensemble-v5-GGUF It would be cool to add it to the [llama2](https://ollama.ai/library/llama2) offerings as `13b-ensemble`, `13b-ensemble-q4_0`.
GiteaMirror added the model label 2026-04-12 09:49:49 -05:00
GiteaMirror commented 2026-04-12 09:49:51 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Sep 30, 2023):

hey! Would you like to try to import it, and perhaps even upload it into Ollama library (if you want). We support importing both GGML and GGUF files. [Would seriously love input to make this experience better]

  1. Create a file named Modelfile, and add a FROM instruction with the local file path to the model you want to import.
FROM ./llama-2-13b-ensemble-v5.Q4_0.gguf
  1. Create the model in Ollama
ollama create name -f path_to_modelfile
  1. Run the model
ollama run name

Now you'll be able to try changing the prompt template in the modelfile. For example:

FROM ./llama-2-13b-ensemble-v5.Q4_0.gguf 

TEMPLATE """ 

### Instruction:
{{.Prompt}}

### Response:

"""

Now for uploading the model:
Sign up: https://ollama.ai/signup
(Bear with me on this, we're working hard to improve this experience before more broadly sharing it)

and once you upload your public keys, you'll be able to push the model to your own namespace

ie.)
ollama create jamesbraza/llama2-ensemble

and then ollama push jamesbraza/llama2-ensemble

<!-- gh-comment-id:1741674730 --> @mchiang0610 commented on GitHub (Sep 30, 2023): hey! Would you like to try to import it, and perhaps even upload it into Ollama library (if you want). We support importing both GGML and GGUF files. [Would seriously love input to make this experience better] 1. Create a file named Modelfile, and add a FROM instruction with the local file path to the model you want to import. ``` FROM ./llama-2-13b-ensemble-v5.Q4_0.gguf ``` 2. Create the model in Ollama ``` ollama create name -f path_to_modelfile ``` 3. Run the model ``` ollama run name ``` Now you'll be able to try changing the prompt template in the modelfile. For example: ``` FROM ./llama-2-13b-ensemble-v5.Q4_0.gguf TEMPLATE """ ### Instruction: {{.Prompt}} ### Response: """ ``` Now for uploading the model: Sign up: https://ollama.ai/signup (Bear with me on this, we're working hard to improve this experience before more broadly sharing it) and once you upload your public keys, you'll be able to push the model to your own namespace ie.) `ollama create jamesbraza/llama2-ensemble` and then `ollama push jamesbraza/llama2-ensemble`
GiteaMirror commented 2026-04-12 09:49:51 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Sep 30, 2023):

@jamesbraza hope all is well! Just wanted to check if you were running into any trouble with this

<!-- gh-comment-id:1741869973 --> @mchiang0610 commented on GitHub (Sep 30, 2023): @jamesbraza hope all is well! Just wanted to check if you were running into any trouble with this
GiteaMirror commented 2026-04-12 09:49:51 -05:00
Author
Owner
Copy Link

@jamesbraza commented on GitHub (Sep 30, 2023):

Yeah I appreciate the follow up and also the excellent instructions!

I haven't gotten around to this yet but it's one of the first things I will do Monday. I was going to upstream any notable learnings from the process to https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md in a PR that closes this issue

<!-- gh-comment-id:1741870988 --> @jamesbraza commented on GitHub (Sep 30, 2023): Yeah I appreciate the follow up and also the excellent instructions! I haven't gotten around to this yet but it's one of the first things I will do Monday. I was going to upstream any notable learnings from the process to https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md in a PR that closes this issue
GiteaMirror commented 2026-04-12 09:49:52 -05:00
Author
Owner
Copy Link

@jamesbraza commented on GitHub (Oct 2, 2023):

Okay I did hit a hiccup. I previously downloaded llama-2-13b-ensemble-v5.Q4_K_M.gguf using huggingface_hub.hf_hub_download.

When I use FROM llama2 in the Modelfile and then ollama/ollama create llama-ensemble -f Modelfile, Ollama starts downloading llama2. How can I get it not to download llama2? I want to use my GGUF

<!-- gh-comment-id:1743749744 --> @jamesbraza commented on GitHub (Oct 2, 2023): Okay I did hit a hiccup. I previously downloaded [llama-2-13b-ensemble-v5.Q4_K_M.gguf](https://huggingface.co/TheBloke/Llama-2-13B-Ensemble-v5-GGUF/blob/main/llama-2-13b-ensemble-v5.Q4_K_M.gguf) using [huggingface_hub.hf_hub_download](https://huggingface.co/docs/huggingface_hub/guides/download). When I use `FROM llama2` in the `Modelfile` and then `ollama/ollama create llama-ensemble -f Modelfile`, Ollama starts downloading `llama2`. How can I get it not to download `llama2`? I want to use my GGUF
GiteaMirror commented 2026-04-12 09:49:53 -05:00
Author
Owner
Copy Link

@jamesbraza commented on GitHub (Oct 6, 2023):

Alright, to answer my question, you have to use local FROM: https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#build-from-a-bin-file

I will pick this back up next week. If .gguf works, we should update that docs title to be #build-from-a-local-file (not specific to .bin)

<!-- gh-comment-id:1751390275 --> @jamesbraza commented on GitHub (Oct 6, 2023): Alright, to answer my question, you have to use local `FROM`: https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md#build-from-a-bin-file I will pick this back up next week. If `.gguf` works, we should update that docs title to be `#build-from-a-local-file` (not specific to `.bin`)
GiteaMirror commented 2026-04-12 09:49:53 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Dec 4, 2023):

I think it looks like this issue has been resolved with Michaels help. I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.

<!-- gh-comment-id:1839388273 --> @technovangelist commented on GitHub (Dec 4, 2023): I think it looks like this issue has been resolved with Michaels help. I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#289
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?