github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4319] NikolayKozloff/Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-GGUF #2694

New Issue
Open
opened 2026-04-12 13:01:03 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 13:01:03 -05:00
Owner
Copy Link

Originally created by @adrianpuiu on GitHub (May 10, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4319

please add Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-GGUF

converted to GGUF without changing tensor data type. Moreover, the new correct pre-tokenizer llama-bpe is used (ref) and the EOS token is correctly set

Originally created by @adrianpuiu on GitHub (May 10, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4319 please add [Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-GGUF](https://huggingface.co/NikolayKozloff/Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-GGUF) converted to GGUF without changing tensor data type. Moreover, the new correct pre-tokenizer llama-bpe is used ([ref](https://github.com/ggerganov/llama.cpp/pull/6745#issuecomment-2094991999)) and the EOS token is correctly set
GiteaMirror added the model label 2026-04-12 13:01:03 -05:00
GiteaMirror commented 2026-04-12 13:01:04 -05:00
Author
Owner
Copy Link

@oldgithubman commented on GitHub (May 10, 2024):

You might want to wait. I think I'm still dragging more changes out of the huggingface/meta guys. So frustrating

<!-- gh-comment-id:2105287068 --> @oldgithubman commented on GitHub (May 10, 2024): You might want to wait. I think I'm still dragging more changes out of the huggingface/meta guys. So frustrating
GiteaMirror commented 2026-04-12 13:01:04 -05:00
Author
Owner
Copy Link

@adrianpuiu commented on GitHub (May 11, 2024):

You might want to wait. I think I'm still dragging more changes out of the huggingface/meta guys. So frustrating

jesus christ , how long do they need to get their thing together ?:))

<!-- gh-comment-id:2105656652 --> @adrianpuiu commented on GitHub (May 11, 2024): > You might want to wait. I think I'm still dragging more changes out of the huggingface/meta guys. So frustrating jesus christ , how long do they need to get their thing together ?:))
GiteaMirror commented 2026-04-12 13:01:05 -05:00
Author
Owner
Copy Link

@oldgithubman commented on GitHub (May 11, 2024):

I wish I knew. What's clear to me is they haven't given this proper attention yet and I'd caution everyone to slow down. Please spread the word. I deleted my reddit account years ago

<!-- gh-comment-id:2105985717 --> @oldgithubman commented on GitHub (May 11, 2024): I wish I knew. What's clear to me is they haven't given this proper attention yet and I'd caution everyone to slow down. Please spread the word. I deleted my reddit account years ago
GiteaMirror commented 2026-04-12 13:01:05 -05:00
Author
Owner
Copy Link

@adrianpuiu commented on GitHub (May 12, 2024):

they already know about it on reddit but I guess not many pay attention to
it and they use what's already available

On Sat, 11 May 2024 at 21:37, Old Man @.***> wrote:

I wish I knew. What's clear to me is they haven't given this proper
attention yet and I'd caution everyone to slow down. Please spread the
word. I deleted my reddit account years ago

—
Reply to this email directly, view it on GitHub
https://github.com/ollama/ollama/issues/4319#issuecomment-2105985717,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABNVRWT7TMILRN6W5HCA7TZBZQQBAVCNFSM6AAAAABHRDS22OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVHE4DKNZRG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

<!-- gh-comment-id:2106228796 --> @adrianpuiu commented on GitHub (May 12, 2024): they already know about it on reddit but I guess not many pay attention to it and they use what's already available On Sat, 11 May 2024 at 21:37, Old Man ***@***.***> wrote: > I wish I knew. What's clear to me is they haven't given this proper > attention yet and I'd caution everyone to slow down. Please spread the > word. I deleted my reddit account years ago > > — > Reply to this email directly, view it on GitHub > <https://github.com/ollama/ollama/issues/4319#issuecomment-2105985717>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AABNVRWT7TMILRN6W5HCA7TZBZQQBAVCNFSM6AAAAABHRDS22OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVHE4DKNZRG4> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
GiteaMirror commented 2026-04-12 13:01:06 -05:00
Author
Owner
Copy Link

@adrianpuiu commented on GitHub (May 12, 2024):

problem solved

huggingface-cli download \
  NikolayKozloff/Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-Q6_k-Q4_K_M-GGUF \
  Meta-Llama-3-8B-Instruct-correct-pre-tokenizer-and-EOS-token-Q8_0.gguf]GGUF \
  --local-dir downloads \
  --local-dir-use-symlinks False

create he modelfile file and paste into it :
FROM ./downloads/Meta-Llama-3-8B-Instruct-correct-pre-tokenizer-and-EOS-token-Q8_0.gguf

then in terminal : ollama create Meta-Llama-3-8B-Instruct-Q8_0.gguf -f Modelfile

ollama list
ollama run

i did not checked the output when running it , things might get messy, you might wanna use the tokenizer template

<html> tokenizer.chat_template | {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<\|start_header_id\|>' + message['role'] + '<\|end_header_id\|> '+ message['content'] \| trim + '<\|eot_id\|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<\|start_header_id\|>assistant<\|end_header_id\|> ' }}{% endif %} -- | -- </html>
<!-- gh-comment-id:2106308744 --> @adrianpuiu commented on GitHub (May 12, 2024): problem solved ``` huggingface-cli download \ NikolayKozloff/Meta-Llama-3-8B-Instruct-bf16-correct-pre-tokenizer-and-EOS-token-Q8_0-Q6_k-Q4_K_M-GGUF \ Meta-Llama-3-8B-Instruct-correct-pre-tokenizer-and-EOS-token-Q8_0.gguf]GGUF \ --local-dir downloads \ --local-dir-use-symlinks False ``` create he modelfile file and paste into it : `FROM ./downloads/Meta-Llama-3-8B-Instruct-correct-pre-tokenizer-and-EOS-token-Q8_0.gguf` then in terminal : `ollama create Meta-Llama-3-8B-Instruct-Q8_0.gguf -f Modelfile` ollama list ollama run i did not checked the output when running it , things might get messy, you might wanna use the tokenizer template <html> <body> <!--StartFragment--> tokenizer.chat_template | {% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<\|start_header_id\|>' + message['role'] + '<\|end_header_id\|> '+ message['content'] \| trim + '<\|eot_id\|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<\|start_header_id\|>assistant<\|end_header_id\|> ' }}{% endif %} -- | -- <!--EndFragment--> </body> </html>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2694
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?