[GH-ISSUE #11019] [Any custom model] Infinity text generation #7266

Closed
opened 2026-04-12 19:18:54 -05:00 by GiteaMirror · 12 comments
Owner

Originally created by @ZikViM on GitHub (Jun 8, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11019

What is the issue?

When i convert any models from huggingface (gguf or safetensor nvm.) it succefully converting, but when i try to text,ask or just typing randomly it generating infinitely a poem idk why, and it does not depend on model type, instruct, storyteller, or just base model, it goes generate dialog by self, and it only stops when I press ctrl + d, models from ollama repo like deepseek, ollama 3.2 work great, below deepseek v1

Image

and this is a Llama-3.2-3B-Instruct-uncensored from hugging face

Image

this is parameters in modelfiles

Image

Relevant log output


OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @ZikViM on GitHub (Jun 8, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11019 ### What is the issue? When i convert any models from huggingface (gguf or safetensor nvm.) it succefully converting, but when i try to text,ask or just typing randomly it generating infinitely a poem idk why, and it does not depend on model type, instruct, storyteller, or just base model, it goes generate dialog by self, and it only stops when I press ctrl + d, models from ollama repo like deepseek, ollama 3.2 work great, below deepseek v1 ![Image](https://github.com/user-attachments/assets/df2357f1-f9f9-4ed0-b6bc-1ed27473addc) and this is a Llama-3.2-3B-Instruct-uncensored from hugging face ![Image](https://github.com/user-attachments/assets/45420587-000c-4f23-9a0d-4abb5df1cfef) this is parameters in modelfiles ![Image](https://github.com/user-attachments/assets/6cb55d58-41c9-44e4-bd3f-d8749873ba61) ### Relevant log output ```shell ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-12 19:18:54 -05:00
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

You have to supply a template, see documentation. ollama will try to auto-detect a suitable template when you import a model, but if you are using modified models, it may not find a suitable match and will fall back to a basic prompt that might not work correctly.

Also, text is much better than screenshots.

<!-- gh-comment-id:2954154845 --> @rick-github commented on GitHub (Jun 8, 2025): You have to supply a template, see [documentation](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#template). ollama will try to auto-detect a suitable template when you import a model, but if you are using modified models, it may not find a suitable match and will fall back to a basic prompt that might not work correctly. Also, text is much better than screenshots.
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

Im jently add a TEMPLATE to Modelfile and its looking like that
FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

not gonna work, it still doesn't work, still going generate some infinitly bullshit, or ,the mostly, i just to dumb to understand how template works

<!-- gh-comment-id:2954165161 --> @ZikViM commented on GitHub (Jun 8, 2025): Im jently add a TEMPLATE to Modelfile and its looking like that FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf TEMPLATE """{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>user {{ .Prompt }}<|im_end|> {{ end }}<|im_start|>assistant """ not gonna work, it still doesn't work, still going generate some infinitly bullshit, or ,the mostly, i just to dumb to understand how template works
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

This doesn't look like a llama3.1 template. llama3.1 uses <|start_header_id|> to delimit the roles, not <|im_start|>.

Note that most GGUF models on HF have a "Use this model" button on the right hand side of the model card that gives instructions on pulling the model along with parameters and template. However, the template that accompanies this model looks like it's actually for a discrimination model, so I don't think it will work any better.

<!-- gh-comment-id:2954172941 --> @rick-github commented on GitHub (Jun 8, 2025): This doesn't look like a llama3.1 [template](https://ollama.com/library/llama3.1:latest/blobs/948af2743fc7). llama3.1 uses `<|start_header_id|>` to delimit the roles, not `<|im_start|>`. Note that most GGUF models on HF have a "Use this model" button on the right hand side of the model card that gives instructions on pulling the model along with parameters and template. However, the template that accompanies this model looks like it's actually for a discrimination model, so I don't think it will work any better.
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

Try this, it looks more like the chat template embedded in the GGUf for this model:

TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"
<!-- gh-comment-id:2954175042 --> @rick-github commented on GitHub (Jun 8, 2025): Try this, it looks more like the chat template embedded in the GGUf for this model: ``` TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>" ```
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

Looking like that

FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf
TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"

For example what it generate:

Who are you
?
You're a visitor to our website!
You can choose to allow or refuse cookies on this site. Most browsers accept cookies by default, but you can
change your browser settings to reject cookies if you wish.
What are cookies? Why do we use them?
Cookies are small text files that websites store in your computer's hard disk or temporary memory when you visit a
website. They contain information about your visit and allow us to remember your preferences, such as language or
region. Cookies can also be used for analytical purposes - like finding out what pages of our site were most
popular.
Why do we use cookies?
We use cookies for several reasons:
To personalize the content of our web pages according to your preferences
To count how many times a particular page has been visited and by whom
To ensure that when you visit our website again, it looks like you left it last time (because we remember your
language or region)
To help us improve our website by seeing where visitors come from and what they do on the site.
What kinds of cookies do we use?
We only use two types of cookies:
Session cookies - these are deleted once you leave our site
Persistent cookies - these stay in your computer's hard disk for a longer time, depending on the settings
When do we use cookies?
We use cookies when you first visit our website. We also use them every time you navigate through the site and
when you log off.
How to change or remove cookies?
You can easily change or delete cookies using your browser settings. If you do not want us to remember your
preferences, just decline our cookie usage next time you come to this site.
We hope that helps! If you have any questions about how we use cookies, please don't hesitate to contact us.
Cookies (text files) are stored on the user's computer or mobile device and allow a website to recognize a user
over time and across multiple visits to different pages. When a cookie is placed in your web browser, it may be
used

as you can see it still doen't work , i only asked Who are you, sad_face

<!-- gh-comment-id:2954181078 --> @ZikViM commented on GitHub (Jun 8, 2025): Looking like that FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>" For example what it generate: >>> Who are you ? You're a visitor to our website! You can choose to allow or refuse cookies on this site. Most browsers accept cookies by default, but you can change your browser settings to reject cookies if you wish. What are cookies? Why do we use them? Cookies are small text files that websites store in your computer's hard disk or temporary memory when you visit a website. They contain information about your visit and allow us to remember your preferences, such as language or region. Cookies can also be used for analytical purposes - like finding out what pages of our site were most popular. Why do we use cookies? We use cookies for several reasons: To personalize the content of our web pages according to your preferences To count how many times a particular page has been visited and by whom To ensure that when you visit our website again, it looks like you left it last time (because we remember your language or region) To help us improve our website by seeing where visitors come from and what they do on the site. What kinds of cookies do we use? We only use two types of cookies: Session cookies - these are deleted once you leave our site Persistent cookies - these stay in your computer's hard disk for a longer time, depending on the settings When do we use cookies? We use cookies when you first visit our website. We also use them every time you navigate through the site and when you log off. How to change or remove cookies? You can easily change or delete cookies using your browser settings. If you do not want us to remember your preferences, just decline our cookie usage next time you come to this site. We hope that helps! If you have any questions about how we use cookies, please don't hesitate to contact us. Cookies (text files) are stored on the user's computer or mobile device and allow a website to recognize a user over time and across multiple visits to different pages. When a cookie is placed in your web browser, it may be used as you can see it still doen't work , i only asked Who are you, sad_face
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

This doesn't look like a llama3.1 template. llama3.1 uses <|start_header_id|> to delimit the roles, not <|im_start|>.

Note that most GGUF models on HF have a "Use this model" button on the right hand side of the model card that gives instructions on pulling the model along with parameters and template. However, the template that accompanies this model looks like it's actually for a discrimination model, so I don't think it will work any better.

Yes i found that button but on half of the models don't have them

<!-- gh-comment-id:2954183872 --> @ZikViM commented on GitHub (Jun 8, 2025): > This doesn't look like a llama3.1 [template](https://ollama.com/library/llama3.1:latest/blobs/948af2743fc7). llama3.1 uses `<|start_header_id|>` to delimit the roles, not `<|im_start|>`. > > Note that most GGUF models on HF have a "Use this model" button on the right hand side of the model card that gives instructions on pulling the model along with parameters and template. However, the template that accompanies this model looks like it's actually for a discrimination model, so I don't think it will work any better. Yes i found that button but on half of the models don't have them
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf
TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"
$ ollama create 11019
gathering model components 
copying file sha256:e0759cd1a8592cae8ca19e8644e6f18e12a5596d340d006d78f4797eb61dc408 100% 
parsing GGUF 
using existing layer sha256:e0759cd1a8592cae8ca19e8644e6f18e12a5596d340d006d78f4797eb61dc408 
using existing layer sha256:8ab4849b038cf0abc5b1c9b8ee1443dca6b93a045c2272180d985126eb40bf6f 
writing manifest 
success 
$ ollama run 11019
>>> Who are you?
I'm an AI assistant. I can help with a variety of tasks such as answering questions, completing math problems, and generating text. What do you need assistance with today?

>>> 

<!-- gh-comment-id:2954184106 --> @rick-github commented on GitHub (Jun 8, 2025): ```modelfile FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|> {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> {{ .Response }}<|eot_id|>" ``` ```console $ ollama create 11019 gathering model components copying file sha256:e0759cd1a8592cae8ca19e8644e6f18e12a5596d340d006d78f4797eb61dc408 100% parsing GGUF using existing layer sha256:e0759cd1a8592cae8ca19e8644e6f18e12a5596d340d006d78f4797eb61dc408 using existing layer sha256:8ab4849b038cf0abc5b1c9b8ee1443dca6b93a045c2272180d985126eb40bf6f writing manifest success $ ollama run 11019 >>> Who are you? I'm an AI assistant. I can help with a variety of tasks such as answering questions, completing math problems, and generating text. What do you need assistance with today? >>> ```
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

Image

idk what am i doing wrong

<!-- gh-comment-id:2954187289 --> @ZikViM commented on GitHub (Jun 8, 2025): ![Image](https://github.com/user-attachments/assets/d7e2f43c-a2fd-44cc-893f-a854cc94bce3) idk what am i doing wrong
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

Only one layer was created. You can see from my example that two layers were created, one for the model weights (sha256:e0759...) and one for the template (sha256:8ab4849...). In your screenshot (text is better, btw) only one layer is created, the one for the model weights. So either you are using the wrong modelfile to create the model, or the modelfile either doesn't have a template in it or is invalid in some way.

<!-- gh-comment-id:2954189315 --> @rick-github commented on GitHub (Jun 8, 2025): Only one layer was created. You can see from my example that two layers were created, one for the model weights (sha256:e0759...) and one for the template (sha256:8ab4849...). In your screenshot (text is better, btw) only one layer is created, the one for the model weights. So either you are using the wrong modelfile to create the model, or the modelfile either doesn't have a template in it or is invalid in some way.
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

1.Modelfile.txt in root directory of Ollama
2.in txt write:

FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf

TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"

3.in same directory putting DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf
4.then run cmd from folder
5.type ollama create Name, it create only one layer
at what point am I doing something wrong

<!-- gh-comment-id:2954194609 --> @ZikViM commented on GitHub (Jun 8, 2025): 1.Modelfile.txt in root directory of Ollama 2.in txt write: > FROM ./DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf > TEMPLATE "{{ if .System }}<|start_header_id|>system<|end_header_id|> > {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|> > {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|> > {{ .Response }}<|eot_id|>" 3.in same directory putting DarkIdol-Llama-3.1-8B-Instruct-1.2-Uncensored.Q4_K_M.gguf 4.then run cmd from folder 5.type ollama create Name, it create only one layer at what point am I doing something wrong
Author
Owner

@rick-github commented on GitHub (Jun 8, 2025):

The default file is called Modelfile, not Modelfile.txt. Try this instead:

ollama create Name -f Modelfile.txt
<!-- gh-comment-id:2954198884 --> @rick-github commented on GitHub (Jun 8, 2025): The default file is called `Modelfile`, not `Modelfile.txt`. Try this instead: ``` ollama create Name -f Modelfile.txt ```
Author
Owner

@ZikViM commented on GitHub (Jun 8, 2025):

The default file is called Modelfile, not Modelfile.txt. Try this instead:

ollama create Name -f Modelfile.txt

well, some small detail can ruin all day, it really not a txt file, i just delete .txt extension and it start work, now i have two layers, thx a good man for help

<!-- gh-comment-id:2954200383 --> @ZikViM commented on GitHub (Jun 8, 2025): > The default file is called `Modelfile`, not `Modelfile.txt`. Try this instead: > > ``` > ollama create Name -f Modelfile.txt > ``` well, some small detail can ruin all day, it really not a txt file, i just delete .txt extension and it start work, now i have two layers, thx a good man for help
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7266