github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6226] Error: unexpected EOF: #65928

New Issue
Open
opened 2026-05-03 23:12:05 -05:00 by GiteaMirror · 11 comments
GiteaMirror commented 2026-05-03 23:12:05 -05:00
Owner
Copy Link

Originally created by @KangInKoo on GitHub (Aug 7, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6226

Originally assigned to: @pdevine on GitHub.

What is the issue?

hi? I'm studying fine tuning.
I learned using the "unsloth/gemma-2-2b-it" model.
I created the dataset myself and it contains less than 100 cases.
I want to use only the fine-tuned model without combining it with the existing model.
I was able to use the fine-tuned model using the code below.

`
pipe_finetuned = pipeline(
"text-generation",
model = finetuned_model,
tokenizer = tokenizer,
max_new_tokens = 512
)

outputs = pipe_finetuned(
prompt,
do_sample = True,
temperature = 0.35,
top_k = 5,
top_p = 0.95,
add_special_tokens = True
)

print(outputs[0]["generated_text"])
`

Finally, I plan to deploy the fine-tuned model to ollama.
So I created a gguf file using llama.cpp.
After that I created a Modelfile.

`
FROM gemma-2-2B-it-F16.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>{{ end }}
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop <|start_header_id|>
PARAMETER stop <|end_header_id|>
PARAMETER <|eot_id|>
`

And I did ollama create, but an EOF error occurred.

ollama create gemma2 -f Modelfile

How can I fix this error?
help me ....
file_list

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

ollama version is 0.1.47

Originally created by @KangInKoo on GitHub (Aug 7, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6226 Originally assigned to: @pdevine on GitHub. ### What is the issue? hi? I'm studying fine tuning. I learned using the "unsloth/gemma-2-2b-it" model. I created the dataset myself and it contains less than 100 cases. I want to use only the fine-tuned model without combining it with the existing model. I was able to use the fine-tuned model using the code below. ` pipe_finetuned = pipeline( "text-generation", model = finetuned_model, tokenizer = tokenizer, max_new_tokens = 512 ) outputs = pipe_finetuned( prompt, do_sample = True, temperature = 0.35, top_k = 5, top_p = 0.95, add_special_tokens = True ) print(outputs[0]["generated_text"]) ` Finally, I plan to deploy the fine-tuned model to ollama. So I created a gguf file using llama.cpp. After that I created a Modelfile. ` FROM gemma-2-2B-it-F16.gguf TEMPLATE """{{ if .System }}<|im_start|>system {{ .System }}<|im_end|>{{ end }} <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant """ PARAMETER stop <|start_header_id|> PARAMETER stop <|end_header_id|> PARAMETER <|eot_id|> ` And I did ollama create, but an EOF error occurred. ` ollama create gemma2 -f Modelfile ` How can I fix this error? help me .... ![file_list](https://github.com/user-attachments/assets/5f3c8811-e994-40de-9c2e-23d77c189bc6) ### OS Linux ### GPU Nvidia ### CPU _No response_ ### Ollama version ollama version is 0.1.47
GiteaMirror added the bug label 2026-05-03 23:12:06 -05:00
GiteaMirror commented 2026-05-03 23:12:48 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 7, 2024):

Last line of your modelfile should be PARAMETER stop <|eot_id|>.

<!-- gh-comment-id:2273040374 --> @rick-github commented on GitHub (Aug 7, 2024): Last line of your modelfile should be `PARAMETER stop <|eot_id|>`.
GiteaMirror commented 2026-05-03 23:12:51 -05:00
Author
Owner
Copy Link

@KangInKoo commented on GitHub (Aug 16, 2024):

hi @rick-github
I tried what you recommended. But still an error occurs.

transferring model data Error: invalid file magic

What is this error??
Thanks.

<!-- gh-comment-id:2292537006 --> @KangInKoo commented on GitHub (Aug 16, 2024): hi @rick-github I tried what you recommended. But still an error occurs. ` transferring model data Error: invalid file magic ` What is this error?? Thanks.
GiteaMirror commented 2026-05-03 23:12:57 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 16, 2024):

The file is not recognized as a valid GGUF file by ollama. What is the result of:

hd gemma-2-2B-it-F16.gguf | head
<!-- gh-comment-id:2292545204 --> @rick-github commented on GitHub (Aug 16, 2024): The file is not recognized as a valid GGUF file by ollama. What is the result of: ``` hd gemma-2-2B-it-F16.gguf | head ```
GiteaMirror commented 2026-05-03 23:13:00 -05:00
Author
Owner
Copy Link

@KangInKoo commented on GitHub (Aug 16, 2024):

hi, @rick-github
This is a screenshot of the result.

image

<!-- gh-comment-id:2292547522 --> @KangInKoo commented on GitHub (Aug 16, 2024): hi, @rick-github This is a screenshot of the result. ![image](https://github.com/user-attachments/assets/9e239177-4c8b-4a23-b386-231ff1c96ee3)
GiteaMirror commented 2026-05-03 23:13:02 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 16, 2024):

What was the full command you used to create gemma-2-2B-it-F16.gguf? What is the current contents of the Modelfile?

<!-- gh-comment-id:2292571333 --> @rick-github commented on GitHub (Aug 16, 2024): What was the full command you used to create gemma-2-2B-it-F16.gguf? What is the current contents of the Modelfile?
GiteaMirror commented 2026-05-03 23:13:03 -05:00
Author
Owner
Copy Link

@KangInKoo commented on GitHub (Aug 16, 2024):

hi, @rick-github

create command gemma-2-2B-it-F16.gguf :
image

and current Modelfile :
image

Thanks

<!-- gh-comment-id:2292580911 --> @KangInKoo commented on GitHub (Aug 16, 2024): hi, @rick-github create command gemma-2-2B-it-F16.gguf : ![image](https://github.com/user-attachments/assets/eb7f4fac-3ed3-4b44-9f63-388fa249b08a) and current Modelfile : ![image](https://github.com/user-attachments/assets/856b2391-f52c-4dfb-820a-241e71f8ec2f) Thanks
GiteaMirror commented 2026-05-03 23:13:03 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 18, 2024):

How big is gemma2-2-2B-it-F16.gguf? This line:

INFO:gguf.gguf_writer:gemma22b-tts/gemma-2-2B-it-F16.gguf: n_tensors = 0, total_size = negligible - metadata only

leads me to believe that the file contains no model weights.

What do you get if you run

python ../llama.cpp/scripts/gguf_dump.py gemma-2-2B-it-F16.gguf
<!-- gh-comment-id:2295307686 --> @rick-github commented on GitHub (Aug 18, 2024): How big is gemma2-2-2B-it-F16.gguf? This line: ``` INFO:gguf.gguf_writer:gemma22b-tts/gemma-2-2B-it-F16.gguf: n_tensors = 0, total_size = negligible - metadata only ``` leads me to believe that the file contains no model weights. What do you get if you run ``` python ../llama.cpp/scripts/gguf_dump.py gemma-2-2B-it-F16.gguf ```
GiteaMirror commented 2026-05-03 23:13:04 -05:00
Author
Owner
Copy Link

@igorschlum commented on GitHub (Sep 2, 2024):

@KangInKoo did you try with a newer version of Ollama? We are at version 0.3.9 now.

<!-- gh-comment-id:2324798523 --> @igorschlum commented on GitHub (Sep 2, 2024): @KangInKoo did you try with a newer version of Ollama? We are at version 0.3.9 now.
GiteaMirror commented 2026-05-03 23:13:05 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Sep 2, 2024):

@pdevine PTAL when you have a chance

<!-- gh-comment-id:2325374611 --> @jmorganca commented on GitHub (Sep 2, 2024): @pdevine PTAL when you have a chance
GiteaMirror commented 2026-05-03 23:13:05 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Sep 6, 2024):

Just an update on this:

@KangInKoo the problem you're running into is there is some issue w/ the gguf file that you made. I've tried separately to fine tune the 2b-it model but ran into some problems. I have had success with fine tuning the 9b model w/ MLX, but have yet to be able to figure out what's wrong w/ fine tuning the 2b model. I'll keep poking at this though.

Separately, while I was testing I found that the conversion for gemma2 2b directly in Ollama was broken (there were some duplicate tensor names for some of the tensors). This is now fixed with #6645 and should be in the next version. This lets you convert your safetensors model inside of the Modelfile instead of using the convert_hf_to_gguf.py script. More details here

<!-- gh-comment-id:2332980489 --> @pdevine commented on GitHub (Sep 6, 2024): Just an update on this: @KangInKoo the problem you're running into is there is some issue w/ the gguf file that you made. I've tried separately to fine tune the 2b-it model but ran into some problems. I have had success with fine tuning the 9b model w/ MLX, but have yet to be able to figure out what's wrong w/ fine tuning the 2b model. I'll keep poking at this though. Separately, while I was testing I found that the conversion for gemma2 2b directly in Ollama was broken (there were some duplicate tensor names for some of the tensors). This is now fixed with #6645 and should be in the next version. This lets you convert your safetensors model inside of the `Modelfile` instead of using the `convert_hf_to_gguf.py` script. More details [here](https://github.com/ollama/ollama/blob/main/docs/import.md)
GiteaMirror commented 2026-05-03 23:13:06 -05:00
Author
Owner
Copy Link

@igorschlum commented on GitHub (Apr 20, 2025):

@pdevine and news on this issue?

<!-- gh-comment-id:2817231039 --> @igorschlum commented on GitHub (Apr 20, 2025): @pdevine and news on this issue?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#65928
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?