[GH-ISSUE #10970] Support for MedGemma #7229

Open
opened 2026-04-12 19:14:25 -05:00 by GiteaMirror · 25 comments
Owner

Originally created by @fighter3005 on GitHub (Jun 4, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10970

Hi, I would love to run MedGemma (4B with vision is broken for me) on my local WebUI! Since Gemma3 is supported, this might actually be easy?

Originally created by @fighter3005 on GitHub (Jun 4, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10970 Hi, I would love to run MedGemma (4B with vision is broken for me) on my local WebUI! Since Gemma3 is supported, this might actually be easy?
GiteaMirror added the model label 2026-04-12 19:14:25 -05:00
Author
Owner

@matrix07012 commented on GitHub (Jun 8, 2025):

For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with ollama create -q q8_0 model

<!-- gh-comment-id:2954198213 --> @matrix07012 commented on GitHub (Jun 8, 2025): For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with `ollama create -q q8_0 model`
Author
Owner

@fighter3005 commented on GitHub (Jun 10, 2025):

The model does run, but it seems that image encoder is not part of the model weights!? I am not sure what is going on there. I get:
level=INFO source=server.go:809 msg="llm predict error: Failed to create new sequence: failed to process inputs: this model is missing data required for image input".

<!-- gh-comment-id:2958138082 --> @fighter3005 commented on GitHub (Jun 10, 2025): The model does run, but it seems that image encoder is not part of the model weights!? I am not sure what is going on there. I get: `level=INFO source=server.go:809 msg="llm predict error: Failed to create new sequence: failed to process inputs: this model is missing data required for image input"`.
Author
Owner

@pupphelper commented on GitHub (Jun 11, 2025):

For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with ollama create -q q8_0 model

which fp16 model works? can you link it? I tried some F16 GGUF same problem, I'm having the same issue, vison not working on openwebui

<!-- gh-comment-id:2964511772 --> @pupphelper commented on GitHub (Jun 11, 2025): > For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with `ollama create -q q8_0 model` which fp16 model works? can you link it? I tried some F16 GGUF same problem, I'm having the same issue, vison not working on openwebui
Author
Owner

@matrix07012 commented on GitHub (Jun 12, 2025):

For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with ollama create -q q8_0 model

which fp16 model works? can you link it? I tried some F16 GGUF same problem, I'm having the same issue, vison not working on openwebui

Not GGUF, the original safetensor version straight from Google

<!-- gh-comment-id:2965952397 --> @matrix07012 commented on GitHub (Jun 12, 2025): > > For some reason the quantizations from HF don't work with vision. You have to download full fp16 version of the model and either use that or quantize it yourself with `ollama create -q q8_0 model` > > which fp16 model works? can you link it? I tried some F16 GGUF same problem, I'm having the same issue, vison not working on openwebui Not GGUF, the original safetensor version straight from Google
Author
Owner

@amsaravi commented on GitHub (Jun 13, 2025):

@fighter3005 I created model for you. you can find it here:
https://ollama.com/amsaravi/medgemma-4b-it-q8

take a look at this notebook to see how to do that:
https://colab.research.google.com/drive/12MW3bwp_-rXVFuC_6dVBp9q8VPCvVFT7?usp=sharing

<!-- gh-comment-id:2968660105 --> @amsaravi commented on GitHub (Jun 13, 2025): @fighter3005 I created model for you. you can find it here: https://ollama.com/amsaravi/medgemma-4b-it-q8 take a look at this notebook to see how to do that: https://colab.research.google.com/drive/12MW3bwp_-rXVFuC_6dVBp9q8VPCvVFT7?usp=sharing
Author
Owner

@fighter3005 commented on GitHub (Jun 16, 2025):

@fighter3005 I created model for you. you can find it here: https://ollama.com/amsaravi/medgemma-4b-it-q8

take a look at this notebook to see how to do that: https://colab.research.google.com/drive/12MW3bwp_-rXVFuC_6dVBp9q8VPCvVFT7?usp=sharing

Thank you for the nice tutorial & the Model! However, if I just pull your model and run it in open webui, I get gibberish. Do I need to set something in the Modelfile? Or do I need to create one?

<unused22><unused22><unused5><unused1><unused12><unused26>[multimodal]<unused16><unused18><unused10><unused22><unused31>

<!-- gh-comment-id:2975882741 --> @fighter3005 commented on GitHub (Jun 16, 2025): > [@fighter3005](https://github.com/fighter3005) I created model for you. you can find it here: https://ollama.com/amsaravi/medgemma-4b-it-q8 > > take a look at this notebook to see how to do that: https://colab.research.google.com/drive/12MW3bwp_-rXVFuC_6dVBp9q8VPCvVFT7?usp=sharing Thank you for the nice tutorial & the Model! However, if I just pull your model and run it in open webui, I get gibberish. Do I need to set something in the Modelfile? Or do I need to create one? `<unused22><unused22><unused5><unused1><unused12><unused26>[multimodal]<unused16><unused18><unused10><unused22><unused31>`
Author
Owner

@amsaravi commented on GitHub (Jun 29, 2025):

i am working on it. perhaps the templates must be updated. stay tuned!

<!-- gh-comment-id:3016951638 --> @amsaravi commented on GitHub (Jun 29, 2025): i am working on it. perhaps the templates must be updated. stay tuned!
Author
Owner

@amsaravi commented on GitHub (Jun 30, 2025):

I recreated the model using ollama 0.6.2. text prompt works. I've no time to test if image support works as well. plz give me feedback.
https://ollama.com/amsaravi/medgemma-4b-it
ollama run amsaravi/medgemma-4b-it:q8

<!-- gh-comment-id:3020347554 --> @amsaravi commented on GitHub (Jun 30, 2025): I recreated the model using ollama 0.6.2. text prompt works. I've no time to test if image support works as well. plz give me feedback. https://ollama.com/amsaravi/medgemma-4b-it `ollama run amsaravi/medgemma-4b-it:q8`
Author
Owner

@amsaravi commented on GitHub (Jul 1, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

<!-- gh-comment-id:3024125811 --> @amsaravi commented on GitHub (Jul 1, 2025): i have tested on ollama 0.9.0. image and text, both of them works
Author
Owner

@fighter3005 commented on GitHub (Jul 7, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

Thank you very much! Works!

<!-- gh-comment-id:3045367294 --> @fighter3005 commented on GitHub (Jul 7, 2025): > i have tested on ollama 0.9.0. image and text, both of them works Thank you very much! Works!
Author
Owner

@XwK-P commented on GitHub (Jul 11, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it
If you don't mind, could you please create the ollama model for 27b as well?

Edit:
I've created it myself.
https://ollama.com/puyangwang/medgemma-27b-it

<!-- gh-comment-id:3063329259 --> @XwK-P commented on GitHub (Jul 11, 2025): > i have tested on ollama 0.9.0. image and text, both of them works @amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it
Author
Owner

@illusdolphin commented on GitHub (Jul 14, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well?

Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it

May I ask you go add also Q4 version?

<!-- gh-comment-id:3069796025 --> @illusdolphin commented on GitHub (Jul 14, 2025): > > i have tested on ollama 0.9.0. image and text, both of them works > > [@amsaravi](https://github.com/amsaravi) Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? > > Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it May I ask you go add also Q4 version?
Author
Owner

@XwK-P commented on GitHub (Jul 14, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well?
Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it

May I ask you go add also Q4 version?

Q4_0 and Q4_K_M uploaded.
https://ollama.com/puyangwang/medgemma-27b-it/tags

<!-- gh-comment-id:3070079515 --> @XwK-P commented on GitHub (Jul 14, 2025): > > > i have tested on ollama 0.9.0. image and text, both of them works > > > > > > [@amsaravi](https://github.com/amsaravi) Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? > > Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it > > May I ask you go add also Q4 version? Q4_0 and Q4_K_M uploaded. https://ollama.com/puyangwang/medgemma-27b-it/tags
Author
Owner

@illusdolphin commented on GitHub (Jul 14, 2025):

Thank you very much!

<!-- gh-comment-id:3070560259 --> @illusdolphin commented on GitHub (Jul 14, 2025): Thank you very much!
Author
Owner

@yenjia commented on GitHub (Jul 29, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well?

Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it

Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you!

<!-- gh-comment-id:3133816814 --> @yenjia commented on GitHub (Jul 29, 2025): > > i have tested on ollama 0.9.0. image and text, both of them works > > [@amsaravi](https://github.com/amsaravi) Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? > > Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you!
Author
Owner

@XwK-P commented on GitHub (Jul 29, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well?
Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it

Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you!

Need to downgrade Ollama to 0.6.2 for quantization. Quantized model will still work with latest Ollama.

<!-- gh-comment-id:3133965394 --> @XwK-P commented on GitHub (Jul 29, 2025): > > > i have tested on ollama 0.9.0. image and text, both of them works > > > > > > [@amsaravi](https://github.com/amsaravi) Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? > > Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it > > Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you! Need to downgrade Ollama to 0.6.2 for quantization. Quantized model will still work with latest Ollama.
Author
Owner

@yenjia commented on GitHub (Jul 29, 2025):

i have tested on ollama 0.9.0. image and text, both of them works

@amsaravi Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well?
Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it

Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you!

Need to downgrade Ollama to 0.6.2 for quantization. Quantized model will still work with latest Ollama.

Thank you very much!

<!-- gh-comment-id:3134090196 --> @yenjia commented on GitHub (Jul 29, 2025): > > > > i have tested on ollama 0.9.0. image and text, both of them works > > > > > > > > > [@amsaravi](https://github.com/amsaravi) Thanks for your work. I noticed that the multi modal 27B version of medgemma was released. https://huggingface.co/google/medgemma-27b-it If you don't mind, could you please create the ollama model for 27b as well? > > > Edit: I've created it myself. https://ollama.com/puyangwang/medgemma-27b-it > > > > > > Hi. I'm curious how to create the quantized ollama model. I tried different ways, and I got weird results after quantizing the model. Thank you! > > Need to downgrade Ollama to 0.6.2 for quantization. Quantized model will still work with latest Ollama. Thank you very much!
Author
Owner

@OdinVex commented on GitHub (Sep 22, 2025):

I was running medgemma-27b-text-it:latest (134e...) from Ollama successfully and it's been a while since, but now it generates gibberish. Has something changed, any way to diagnose?

<!-- gh-comment-id:3321029444 --> @OdinVex commented on GitHub (Sep 22, 2025): I was running `medgemma-27b-text-it:latest` (`134e...`) from Ollama successfully and it's been a while since, but now it generates gibberish. Has something changed, any way to diagnose?
Author
Owner

@njpogue commented on GitHub (Sep 23, 2025):

I am also trying to run medgemma-4b-it on ollama server from this HF repo (https://huggingface.co/SandLogicTechnologies/MedGemma-4B-IT-GGUF/blob/main/medgemma-4b-it-Q5_K_M.gguf), but does not appear to work. When I pass cURL command with "images" parameter in base64 format, I also get Failed to create new sequence: failed to process inputs: this model is missing data required for image input"

<!-- gh-comment-id:3324671193 --> @njpogue commented on GitHub (Sep 23, 2025): I am also trying to run medgemma-4b-it on ollama server from this HF repo (https://huggingface.co/SandLogicTechnologies/MedGemma-4B-IT-GGUF/blob/main/medgemma-4b-it-Q5_K_M.gguf), but does not appear to work. When I pass cURL command with "images" parameter in base64 format, I also get `Failed to create new sequence: failed to process inputs: this model is missing data required for image input"`
Author
Owner

@code-forge-temple commented on GitHub (Jan 13, 2026):

Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it?

<!-- gh-comment-id:3747036325 --> @code-forge-temple commented on GitHub (Jan 13, 2026): Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it?
Author
Owner

@illusdolphin commented on GitHub (Jan 14, 2026):

Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it?

What is not working? It's working for a long time already, use https://ollama.com/puyangwang/medgemma-27b-it for vision

<!-- gh-comment-id:3748552582 --> @illusdolphin commented on GitHub (Jan 14, 2026): > Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it? What is not working? It's working for a long time already, use https://ollama.com/puyangwang/medgemma-27b-it for vision
Author
Owner

@OdinVex commented on GitHub (Jan 14, 2026):

Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it?

What is not working? It's working for a long time already, use https://ollama.com/puyangwang/medgemma-27b-it for vision

This is MedGemma-27b-it, not some random upload: https://huggingface.co/google/medgemma-27b-it/tree/main That random upload you referenced is the one sometimes spitting out gibberish.

<!-- gh-comment-id:3749371772 --> @OdinVex commented on GitHub (Jan 14, 2026): > > Hi! Are there any new updates on the status of this issue? Has there been any progress toward MedGemma multimodal support or a timeline for resolving it? > > What is not working? It's working for a long time already, use https://ollama.com/puyangwang/medgemma-27b-it for vision *This* is MedGemma-27b-it, not some random upload: https://huggingface.co/google/medgemma-27b-it/tree/main That random upload you referenced is the one sometimes spitting out gibberish.
Author
Owner

@code-forge-temple commented on GitHub (Jan 14, 2026):

@illusdolphin Apologies for not providing more details earlier - thanks for the pointer.

I was referring specifically to Google's official MedGemma published on Hugging Face, for example:
https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release)

My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions.

<!-- gh-comment-id:3749540040 --> @code-forge-temple commented on GitHub (Jan 14, 2026): @illusdolphin Apologies for not providing more details earlier - thanks for the pointer. I was referring specifically to **Google's official MedGemma published on Hugging Face**, for example: https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release) My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions.
Author
Owner

@illusdolphin commented on GitHub (Jan 14, 2026):

@illusdolphin Apologies for not providing more details earlier - thanks for the pointer.

I was referring specifically to Google's official MedGemma published on Hugging Face, for example: https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release)

My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions.

Google published models using safetensors format, ollama works with GGUF (also safe, but other format). safetensors => GGUF is just change of format, not weight, not quality. You can't use https://huggingface.co/google/medgemma-1.5-4b-it link since it has no files in format that ollama expects. Options are: 1) beg for official GGUF-based release (don't think it makes sense) 2) do it by yourself 3) use GGUF published by other users

<!-- gh-comment-id:3749966424 --> @illusdolphin commented on GitHub (Jan 14, 2026): > [@illusdolphin](https://github.com/illusdolphin) Apologies for not providing more details earlier - thanks for the pointer. > > I was referring specifically to **Google's official MedGemma published on Hugging Face**, for example: https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release) > > My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions. Google published models using safetensors format, ollama works with GGUF (also safe, but other format). safetensors => GGUF is just change of format, not weight, not quality. You can't use https://huggingface.co/google/medgemma-1.5-4b-it link since it has no files in format that ollama expects. Options are: 1) beg for official GGUF-based release (don't think it makes sense) 2) do it by yourself 3) use GGUF published by other users
Author
Owner

@OdinVex commented on GitHub (Jan 14, 2026):

@illusdolphin Apologies for not providing more details earlier - thanks for the pointer.
I was referring specifically to Google's official MedGemma published on Hugging Face, for example: https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release)
My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions.

Google published models using safetensors format, ollama works with GGUF (also safe, but other format). safetensors => GGUF is just change of format, not weight, not quality. You can't use https://huggingface.co/google/medgemma-1.5-4b-it link since it has no files in format that ollama expects. Options are: 1) beg for official GGUF-based release (don't think it makes sense) 2) do it by yourself 3) use GGUF published by other users

Ollama partly works with GGUF. It still doesn't support split GGUF models. As for puyangwang/medgemma-27b-it, it started generating gibberish, so no thanks.

<!-- gh-comment-id:3749982070 --> @OdinVex commented on GitHub (Jan 14, 2026): > > [@illusdolphin](https://github.com/illusdolphin) Apologies for not providing more details earlier - thanks for the pointer. > > I was referring specifically to **Google's official MedGemma published on Hugging Face**, for example: https://huggingface.co/google/medgemma-1.5-4b-it or the full list (https://huggingface.co/collections/google/medgemma-release) > > My understanding is that this model has a different multimodal architecture compared to the community-converted Ollama versions. It would be extraordinary if Ollama could directly support Hugging Face MedGemma models (including both text and vision) without requiring custom conversions. > > Google published models using safetensors format, ollama works with GGUF (also safe, but other format). safetensors => GGUF is just change of format, not weight, not quality. You can't use https://huggingface.co/google/medgemma-1.5-4b-it link since it has no files in format that ollama expects. Options are: 1) beg for official GGUF-based release (don't think it makes sense) 2) do it by yourself 3) use GGUF published by other users Ollama *partly* works with GGUF. It *still* doesn't support split GGUF models. As for `puyangwang/medgemma-27b-it`, it started generating gibberish, so no thanks.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7229