github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9882] *BUG* "http://127.0.0.1:36365/completion": EOF with image attachments #32228

New Issue
Closed
opened 2026-04-22 13:17:27 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-22 13:17:27 -05:00
Owner
Copy Link

Originally created by @TheWhiteWord on GitHub (Mar 19, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9882

What is the issue?

I am getting an error whenever i try to use image with any quant imported in Ollama from huggingface.
THe official Q_4 seems to work, but others I tried don't:

Here is the report form comfy UI (ollama node), but he issue persists no matter the app I use for inference:

HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 500 Internal Server Error"

Also I found this on reddit, of somebody that said to have fixed it, but haven't been able to reproduce yet:

Okay, so now I have finally solved this.

I had to download the raw, safetensor files of Gemma 3, and then quantize them with the ollama --quantize command, with the Modelfile. Now, Gemma 3 Q5_K_M works fine with vision too!

Relevant log output

025-03-18T20:53:30.442972 - Prompt executed in 3.83 seconds
2025-03-18T20:54:08.259647 - HTTP Request: GET http://127.0.0.1:11434/api/tags "HTTP/1.1 200 OK"
2025-03-18T20:54:22.098317 - got prompt
2025-03-18T20:54:22.364119 - [Ollama Vision]
request query params:

- query: PLease, describe in details the image prvided, giving particular attention to provide details about technical aspect like:
- drawing genre and sub-genere,
- drawing style,
- coloring,
- overall stylistic mood,
with the aim to create a context that will aloow for continuity in generating images from the same visual novel.
- url: http://127.0.0.1:11434
- model: hf.co/bartowski/google_gemma-3-4b-it-GGUF:Q8_0

2025-03-18T20:54:22.364167 -
2025-03-18T20:54:22.474160 - HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 500 Internal Server Error"
2025-03-18T20:54:22.475569 - !!! Exception during processing !!! POST predict: Post "http://127.0.0.1:36365/completion": EOF

OS

No response

GPU

No response

CPU

No response

Ollama version

ollama version is 0.6.2

Originally created by @TheWhiteWord on GitHub (Mar 19, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9882 ### What is the issue? I am getting an error whenever i try to use image with any quant imported in Ollama from huggingface. THe official Q_4 seems to work, but others I tried don't: Here is the report form comfy UI (ollama node), but he issue persists no matter the app I use for inference: HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 500 Internal Server Error" Also I found this on reddit, of somebody that said to have fixed it, but haven't been able to reproduce yet: ``` Okay, so now I have finally solved this. I had to download the raw, safetensor files of Gemma 3, and then quantize them with the ollama --quantize command, with the Modelfile. Now, Gemma 3 Q5_K_M works fine with vision too! ``` ### Relevant log output ```shell 025-03-18T20:53:30.442972 - Prompt executed in 3.83 seconds 2025-03-18T20:54:08.259647 - HTTP Request: GET http://127.0.0.1:11434/api/tags "HTTP/1.1 200 OK" 2025-03-18T20:54:22.098317 - got prompt 2025-03-18T20:54:22.364119 - [Ollama Vision] request query params: - query: PLease, describe in details the image prvided, giving particular attention to provide details about technical aspect like: - drawing genre and sub-genere, - drawing style, - coloring, - overall stylistic mood, with the aim to create a context that will aloow for continuity in generating images from the same visual novel. - url: http://127.0.0.1:11434 - model: hf.co/bartowski/google_gemma-3-4b-it-GGUF:Q8_0 2025-03-18T20:54:22.364167 - 2025-03-18T20:54:22.474160 - HTTP Request: POST http://127.0.0.1:11434/api/generate "HTTP/1.1 500 Internal Server Error" 2025-03-18T20:54:22.475569 - !!! Exception during processing !!! POST predict: Post "http://127.0.0.1:36365/completion": EOF ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version ollama version is 0.6.2
GiteaMirror added the bug label 2026-04-22 13:17:27 -05:00
GiteaMirror commented 2026-04-22 13:17:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 19, 2025):

Server logs may help in debugging.

<!-- gh-comment-id:2735846689 --> @rick-github commented on GitHub (Mar 19, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may help in debugging.
GiteaMirror commented 2026-04-22 13:17:29 -05:00
Author
Owner
Copy Link

@TheWhiteWord commented on GitHub (Mar 19, 2025):

Yes.. so i can confirm that creating a new model from the safetensors and quantize it with the correct ModelFile does indeed work.
I have posted on reddit the solution for anyone that needs it.

Create Gemma 3 quants

Unfortunately now I don't have the old gguf anymore but I was using the ones form bartowski.

<!-- gh-comment-id:2735876077 --> @TheWhiteWord commented on GitHub (Mar 19, 2025): Yes.. so i can confirm that creating a new model from the safetensors and quantize it with the correct ModelFile does indeed work. I have posted on reddit the solution for anyone that needs it. [Create Gemma 3 quants](https://www.reddit.com/r/ollama/comments/1jesz6w/temp_fix_http12700136365completion_eof_with_image/) Unfortunately now I don't have the old gguf anymore but I was using the ones form bartowski.
GiteaMirror commented 2026-04-22 13:17:29 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 19, 2025):

Hi @TheWhiteWord thanks for the issue! The vision components of some GGUF files aren't compatible with Ollama – sometimes it comes down to one tensor name or metadata element that may be different. We're working on better detecting these differences at import/ollama create time. Thanks so much for the reddit post!

<!-- gh-comment-id:2737640667 --> @jmorganca commented on GitHub (Mar 19, 2025): Hi @TheWhiteWord thanks for the issue! The vision components of some GGUF files aren't compatible with Ollama – sometimes it comes down to one tensor name or metadata element that may be different. We're working on better detecting these differences at import/`ollama create` time. Thanks so much for the reddit post!
GiteaMirror commented 2026-04-22 13:17:30 -05:00
Author
Owner
Copy Link

@TheWhiteWord commented on GitHub (Mar 20, 2025):

My pleasure, and thank you all for all you do for the open-source community.
😉

<!-- gh-comment-id:2740728424 --> @TheWhiteWord commented on GitHub (Mar 20, 2025): My pleasure, and thank you all for all you do for the open-source community. 😉
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#32228
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?