github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-21 13:11:39 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #12089] panic: runtime error: integer divide by zero #54545

New Issue
Closed
opened 2026-04-29 06:17:55 -05:00 by GiteaMirror · 15 comments
GiteaMirror commented 2026-04-29 06:17:55 -05:00
Owner
Copy Link

Originally created by @gordan-bobic on GitHub (Aug 26, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12089

What is the issue?

Command:

ollama create llama3.3-vision:90b --file Modelfile --quantize q4_K_M

Model:

https://huggingface.co/gghfez/Llama-3.3-90B-Vision-merged

Result: Crash before conversion is complete.
A directory with incompletely constructed f16 model is left over in ~/.ollama/models/ollama-safetensors1621793353/ (number at the end varies).

Relevant log output

Aug 26 21:30:06 overmind ollama[95284]: panic: runtime error: integer divide by zero
Aug 26 21:30:06 overmind ollama[95284]: goroutine 1448 [running]:
Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/convert.(*mllamaModel).Tensors.(*mllamaModel).repack.func3({0xc796e00000?, 0x1?}, {0xc797d72000, 0x190000, 0x19c000}, {0xc003b0fa80, 0x2, 0x30?})
Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/convert/convert_mllama.go:142 +0x4de
Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/convert.safetensor.WriteTo({{0x1776960, 0xc0003e1030}, {0xc0037aca20, 0x20}, {0xc003b0fa70, 0x4}, 0x1951f2956, 0x320000, 0xc000c93290}, {0x1777be0, ...})
Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/convert/reader_safetensors.go:170 +0x649
Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/fs/ggml.WriteGGUF.func2()
Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/fs/ggml/gguf.go:566 +0x2b
Aug 26 21:30:06 overmind ollama[95284]: golang.org/x/sync/errgroup.(*Group).Go.func1()
Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/vendor/golang.org/x/sync/errgroup/errgroup.go:78 +0x50
Aug 26 21:30:06 overmind ollama[95284]: created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 86
Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x93

OS

Linux

GPU

Nvidia

CPU

Intel, AMD

Ollama version

0.11.4

Originally created by @gordan-bobic on GitHub (Aug 26, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12089 ### What is the issue? Command: ``` ollama create llama3.3-vision:90b --file Modelfile --quantize q4_K_M ``` Model: ``` https://huggingface.co/gghfez/Llama-3.3-90B-Vision-merged ``` Result: Crash before conversion is complete. A directory with incompletely constructed f16 model is left over in `~/.ollama/models/ollama-safetensors1621793353/` (number at the end varies). ### Relevant log output ```shell Aug 26 21:30:06 overmind ollama[95284]: panic: runtime error: integer divide by zero Aug 26 21:30:06 overmind ollama[95284]: goroutine 1448 [running]: Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/convert.(*mllamaModel).Tensors.(*mllamaModel).repack.func3({0xc796e00000?, 0x1?}, {0xc797d72000, 0x190000, 0x19c000}, {0xc003b0fa80, 0x2, 0x30?}) Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/convert/convert_mllama.go:142 +0x4de Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/convert.safetensor.WriteTo({{0x1776960, 0xc0003e1030}, {0xc0037aca20, 0x20}, {0xc003b0fa70, 0x4}, 0x1951f2956, 0x320000, 0xc000c93290}, {0x1777be0, ...}) Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/convert/reader_safetensors.go:170 +0x649 Aug 26 21:30:06 overmind ollama[95284]: github.com/ollama/ollama/fs/ggml.WriteGGUF.func2() Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/fs/ggml/gguf.go:566 +0x2b Aug 26 21:30:06 overmind ollama[95284]: golang.org/x/sync/errgroup.(*Group).Go.func1() Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/vendor/golang.org/x/sync/errgroup/errgroup.go:78 +0x50 Aug 26 21:30:06 overmind ollama[95284]: created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 86 Aug 26 21:30:06 overmind ollama[95284]: #011/home/mock/rpmbuild/BUILD/ollama-0.11.4/vendor/golang.org/x/sync/errgroup/errgroup.go:75 +0x93 ``` ### OS Linux ### GPU Nvidia ### CPU Intel, AMD ### Ollama version 0.11.4
GiteaMirror added the bug label 2026-04-29 06:17:55 -05:00
GiteaMirror commented 2026-04-29 06:18:03 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 26, 2025):

Can you add some log from before the panic?

<!-- gh-comment-id:3225470162 --> @rick-github commented on GitHub (Aug 26, 2025): Can you add some log from before the panic?
GiteaMirror commented 2026-04-29 06:18:07 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 26, 2025):

Yes, attached.

llama3.3-vision-90b.log

<!-- gh-comment-id:3225508389 --> @gordan-bobic commented on GitHub (Aug 26, 2025): Yes, attached. [llama3.3-vision-90b.log](https://github.com/user-attachments/files/21994528/llama3.3-vision-90b.log)
GiteaMirror commented 2026-04-29 06:18:09 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Aug 26, 2025):

it looks like the model you're trying to import doesn't have the necessary config values convert, specifically vision_config..attention_heads. You'll need to touch up the config before importing. the model is missing other values such as vision_config.num_hidden_layers, vision_config.num_global_layers, etc. A full list of config values required for conversion can be found in convert/convert_mllama.go

<!-- gh-comment-id:3225615696 --> @mxyng commented on GitHub (Aug 26, 2025): it looks like the model you're trying to import doesn't have the necessary config values convert, specifically `vision_config..attention_heads`. You'll need to touch up the config before importing. the model is missing other values such as `vision_config.num_hidden_layers`, `vision_config.num_global_layers`, etc. A full list of config values required for conversion can be found in [`convert/convert_mllama.go`](https://github.com/ollama/ollama/blob/main/convert/convert_mllama.go#L18-L34)
GiteaMirror commented 2026-04-29 06:18:10 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 26, 2025):

I see, I'll grab those from llama3.2-vision:90b and try again, thanks for the pointer, most appreciated. I'll report back with results.

<!-- gh-comment-id:3225669951 --> @gordan-bobic commented on GitHub (Aug 26, 2025): I see, I'll grab those from llama3.2-vision:90b and try again, thanks for the pointer, most appreciated. I'll report back with results.
GiteaMirror commented 2026-04-29 06:18:10 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 27, 2025):

OK, that successfully converts but running the model produced crashes ollama:

Aug 27 05:23:57 overmind ollama[182588]: time=2025-08-27T05:23:57.321+03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.SSE3=1 CPU.1.SSSE3=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 CUDA.2.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.2.USE_GRAPHS=1 CUDA.2.PEER_MAX_BATCH_SIZE=128 CUDA.3.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.3.USE_GRAPHS=1 CUDA.3.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc)
Aug 27 05:23:57 overmind ollama[182588]: ggml.c:3411: GGML_ASSERT(ggml_nelements(a) == ne0*ne1*ne2) failed
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11c9d88]
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ca223]
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ca3ad]
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ce2ac]
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x119cae7]
Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x4a5c24]
Aug 27 05:23:57 overmind ollama[182588]: SIGABRT: abort
Aug 27 05:23:57 overmind ollama[182588]: PC=0x7f113e016edc m=19 sigcode=18446744073709551610
Aug 27 05:23:57 overmind ollama[182588]: signal arrived during cgo execution
Aug 27 05:23:57 overmind ollama[182588]: goroutine 76 gp=0xc0000fd500 m=19 mp=0xc000581008 [syscall]:
Aug 27 05:23:57 overmind ollama[182588]: runtime.cgocall(0x119cac0, 0xc000044ed0)
Aug 27 05:23:57 overmind ollama[182588]: #011/usr/lib/golang/src/runtime/cgocall.go:167 +0x4b fp=0xc000044ea8 sp=0xc000044e70 pc=0x49aaab

Full log attached.

llama3.3-vision-90b.2.log

Also attached is the patch I applied to config.json to add the missing necessary vision elements.

config.json.patch.txt

Note: I updated ollama to 0.11.7.

<!-- gh-comment-id:3226536720 --> @gordan-bobic commented on GitHub (Aug 27, 2025): OK, that successfully converts but running the model produced crashes ollama: ``` Aug 27 05:23:57 overmind ollama[182588]: time=2025-08-27T05:23:57.321+03:00 level=INFO source=ggml.go:104 msg=system CPU.0.SSE3=1 CPU.0.SSSE3=1 CPU.0.AVX=1 CPU.0.AVX2=1 CPU.0.F16C=1 CPU.0.FMA=1 CPU.0.BMI2=1 CPU.0.LLAMAFILE=1 CPU.1.SSE3=1 CPU.1.SSSE3=1 CPU.1.LLAMAFILE=1 CUDA.0.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.0.USE_GRAPHS=1 CUDA.0.PEER_MAX_BATCH_SIZE=128 CUDA.1.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.1.USE_GRAPHS=1 CUDA.1.PEER_MAX_BATCH_SIZE=128 CUDA.2.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.2.USE_GRAPHS=1 CUDA.2.PEER_MAX_BATCH_SIZE=128 CUDA.3.ARCHS=500,600,610,700,750,800,860,870,890,900,1200 CUDA.3.USE_GRAPHS=1 CUDA.3.PEER_MAX_BATCH_SIZE=128 compiler=cgo(gcc) Aug 27 05:23:57 overmind ollama[182588]: ggml.c:3411: GGML_ASSERT(ggml_nelements(a) == ne0*ne1*ne2) failed Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11c9d88] Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ca223] Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ca3ad] Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x11ce2ac] Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x119cae7] Aug 27 05:23:57 overmind ollama[182588]: /usr/bin/ollama[0x4a5c24] Aug 27 05:23:57 overmind ollama[182588]: SIGABRT: abort Aug 27 05:23:57 overmind ollama[182588]: PC=0x7f113e016edc m=19 sigcode=18446744073709551610 Aug 27 05:23:57 overmind ollama[182588]: signal arrived during cgo execution Aug 27 05:23:57 overmind ollama[182588]: goroutine 76 gp=0xc0000fd500 m=19 mp=0xc000581008 [syscall]: Aug 27 05:23:57 overmind ollama[182588]: runtime.cgocall(0x119cac0, 0xc000044ed0) Aug 27 05:23:57 overmind ollama[182588]: #011/usr/lib/golang/src/runtime/cgocall.go:167 +0x4b fp=0xc000044ea8 sp=0xc000044e70 pc=0x49aaab ``` Full log attached. [llama3.3-vision-90b.2.log](https://github.com/user-attachments/files/21999937/llama3.3-vision-90b.2.log) Also attached is the patch I applied to config.json to add the missing necessary vision elements. [config.json.patch.txt](https://github.com/user-attachments/files/21999977/config.json.patch.txt) Note: I updated ollama to 0.11.7.
GiteaMirror commented 2026-04-29 06:18:11 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 27, 2025):

--- config.json.1	2025-08-27 19:41:03.898387726 +0200
+++ config.json	2025-08-27 19:41:26.777732610 +0200
@@ -38,6 +38,7 @@
     "model_type": "mllama_text_model",
     "num_attention_heads": 64,
     "num_hidden_layers": 100,
+    "num_key_value_heads": 8,
     "rope_scaling": {
       "factor": 8.0,
       "high_freq_factor": 4.0,

This allows the model to load and use the text weights, but the vision component fails:

$ ollama run llama-3.3:90B-vision-merged-q4_K_M
>>> hello
Hello! How can I assist you today?

>>> describe this image: ./picture.png
Added image './picture.png'
Error: Post "http://:11437/api/chat": EOF

time=2025-08-28T01:22:49.046+02:00 level=TRACE source=bytepairencoding.go:205 msg=encoded string="describe this image:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" ids="[12612 420 2217 25 128009 128006 78191 128007 271]"
time=2025-08-28T01:22:49.046+02:00 level=DEBUG source=cache.go:140 msg="loading cache slot" id=0 cache=19 prompt=34 used=19 remaining=15
ml/backend/ggml/ggml/src/ggml-cpu/ops.cpp:6930: fatal error

The error occurs in ggml_compute_forward_im2col() due to a mismatched tensor type, GGML_TYPE_BF16 instead of GGML_TYPE_F32 or GGML_TYPE_F16.

<!-- gh-comment-id:3230111484 --> @rick-github commented on GitHub (Aug 27, 2025): ```diff --- config.json.1 2025-08-27 19:41:03.898387726 +0200 +++ config.json 2025-08-27 19:41:26.777732610 +0200 @@ -38,6 +38,7 @@ "model_type": "mllama_text_model", "num_attention_heads": 64, "num_hidden_layers": 100, + "num_key_value_heads": 8, "rope_scaling": { "factor": 8.0, "high_freq_factor": 4.0, ``` This allows the model to load and use the text weights, but the vision component fails: ```console $ ollama run llama-3.3:90B-vision-merged-q4_K_M >>> hello Hello! How can I assist you today? >>> describe this image: ./picture.png Added image './picture.png' Error: Post "http://:11437/api/chat": EOF ``` ``` time=2025-08-28T01:22:49.046+02:00 level=TRACE source=bytepairencoding.go:205 msg=encoded string="describe this image:<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" ids="[12612 420 2217 25 128009 128006 78191 128007 271]" time=2025-08-28T01:22:49.046+02:00 level=DEBUG source=cache.go:140 msg="loading cache slot" id=0 cache=19 prompt=34 used=19 remaining=15 ml/backend/ggml/ggml/src/ggml-cpu/ops.cpp:6930: fatal error ``` The error occurs in [ggml_compute_forward_im2col()](https://github.com/ollama/ollama/blob/9d97e6a9f1968a9ad85757aeccb2062f7249a3e9/ml/backend/ggml/ggml/src/ggml-cpu/ops.cpp#L6916) due to a mismatched tensor type, GGML_TYPE_BF16 instead of GGML_TYPE_F32 or GGML_TYPE_F16.
GiteaMirror commented 2026-04-29 06:18:11 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

+ "num_key_value_heads": 8,
This allows the model to load and use the text weights

Great, I'm going to try that shortly.

The error occurs in ggml_compute_forward_im2col() due to a mismatched tensor type, GGML_TYPE_BF16 instead of GGML_TYPE_F32 or GGML_TYPE_F16.

Interesting, doesn't the ollama create process convert all bf16 values to f16?

Edit:
Could this be related to freq_base not being in config.json?
https://github.com/ollama/ollama/blob/main/convert/convert_mllama.go#L33C1-L34C1

Edit 2:
It looks like the problem is like due to this missing in the config:

"use_bfloat16": false

I added those so will see if that improves things once my rebuild completes.

<!-- gh-comment-id:3230278179 --> @gordan-bobic commented on GitHub (Aug 28, 2025): > `+ "num_key_value_heads": 8,` > This allows the model to load and use the text weights Great, I'm going to try that shortly. > The error occurs in ggml_compute_forward_im2col() due to a mismatched tensor type, GGML_TYPE_BF16 instead of GGML_TYPE_F32 or GGML_TYPE_F16. Interesting, doesn't the `ollama create` process convert all bf16 values to f16? Edit: Could this be related to freq_base not being in config.json? https://github.com/ollama/ollama/blob/main/convert/convert_mllama.go#L33C1-L34C1 Edit 2: It looks like the problem is like due to this missing in the config: ``` "use_bfloat16": false ``` I added those so will see if that improves things once my rebuild completes.
GiteaMirror commented 2026-04-29 06:18:11 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

After I added "use_bfloat16": false in config.json, it looks like there is still something errant in the generated quantized model:

Aug 28 17:47:46 overmind ollama[71242]: /home/mock/rpmbuild/BUILD/ollama-0.11.4/ml/backend/ggml/ggml/src/ggml-cuda/im2col.cu:73: GGML_ASSERT(dst->type == GGML_TYPE_F16 || dst->type == GGML_TYPE_F32) failed

What is more concerning is that the model is producing really bizarre responses, as if there is over-quantization going on, or quantization of parameters that should remain unquantized.

<!-- gh-comment-id:3233894193 --> @gordan-bobic commented on GitHub (Aug 28, 2025): After I added `"use_bfloat16": false` in config.json, it looks like there is still something errant in the generated quantized model: ``` Aug 28 17:47:46 overmind ollama[71242]: /home/mock/rpmbuild/BUILD/ollama-0.11.4/ml/backend/ggml/ggml/src/ggml-cuda/im2col.cu:73: GGML_ASSERT(dst->type == GGML_TYPE_F16 || dst->type == GGML_TYPE_F32) failed ``` What is more concerning is that the model is producing really bizarre responses, as if there is over-quantization going on, or quantization of parameters that should remain unquantized.
GiteaMirror commented 2026-04-29 06:18:12 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

On one hand, it could be that this was fixed here 2 weeks ago:
https://github.com/ggml-org/llama.cpp/issues/15536

On the other hand, how did ollama create work on llama3.2-vision:90b last year considering the vision tower is the same?

<!-- gh-comment-id:3234117170 --> @gordan-bobic commented on GitHub (Aug 28, 2025): On one hand, it could be that this was fixed here 2 weeks ago: https://github.com/ggml-org/llama.cpp/issues/15536 On the other hand, how did `ollama create` work on llama3.2-vision:90b last year considering the vision tower is the same?
GiteaMirror commented 2026-04-29 06:18:13 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

Even without use_bfloat16:false, the quantized text model seems to be producing garbage output. Not semantically incoherent, but for example, it responded to "hello" with the implementation of a binary search algorithm. Truly bizarre.
So there seems to be something more fundamental going broken in how the quantization is working on this model, even though the text component is supposed to be standard llama3.3:70b

<!-- gh-comment-id:3234267688 --> @gordan-bobic commented on GitHub (Aug 28, 2025): Even without `use_bfloat16:false`, the quantized text model seems to be producing garbage output. Not semantically incoherent, but for example, it responded to "hello" with the implementation of a binary search algorithm. Truly bizarre. So there seems to be something more fundamental going broken in how the quantization is working on this model, even though the text component is supposed to be standard llama3.3:70b
GiteaMirror commented 2026-04-29 06:18:13 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 28, 2025):

Do you have a template in your Modelfile?

<!-- gh-comment-id:3234321241 --> @rick-github commented on GitHub (Aug 28, 2025): Do you have a template in your Modelfile?
GiteaMirror commented 2026-04-29 06:18:14 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

I do not. Should I?

My model file is extremely minimal:

FROM /scratch/models/llama-3.3-90b-vision
PARAMETER temperature 0.6
PARAMETER top_p 0.9

Should I extend it to something more like this?

FROM /scratch/models/llama-3.3-90b-vision
PARAMETER temperature 0.6
PARAMETER top_p 0.9
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
TEMPLATE """
{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
{{- if .System }}

{{ .System }}
{{- end }}
{{- if .Tools }}

Cutting Knowledge Date: December 2023

When you receive a tool call response, use the output to format an answer to the orginal user question.

You are a helpful assistant with tool calling capabilities.
{{- end }}<|eot_id|>
{{- end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 }}
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
{{- if and $.Tools $last }}

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.

{{ range $.Tools }}
{{- . }}
{{ end }}
Question: {{ .Content }}<|eot_id|>
{{- else }}

{{ .Content }}<|eot_id|>
{{- end }}{{ if $last }}<|start_header_id|>assistant<|end_header_id|>

{{ end }}
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
{{- if .ToolCalls }}
{{ range .ToolCalls }}
{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
{{- else }}

{{ .Content }}
{{- end }}{{ if not $last }}<|eot_id|>{{ end }}
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>

{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>

{{ end }}
{{- end }}
{{- end }}
"""

( That is a guess based on parameters/template on llama3.2 and llama3.3 models in ollama repository:
https://ollama.com/library/llama3.2-vision:90b
https://ollama.com/library/llama3.3:70b )

<!-- gh-comment-id:3234421263 --> @gordan-bobic commented on GitHub (Aug 28, 2025): I do not. Should I? My model file is extremely minimal: ``` FROM /scratch/models/llama-3.3-90b-vision PARAMETER temperature 0.6 PARAMETER top_p 0.9 ``` Should I extend it to something more like this? ``` FROM /scratch/models/llama-3.3-90b-vision PARAMETER temperature 0.6 PARAMETER top_p 0.9 PARAMETER stop "<|start_header_id|>" PARAMETER stop "<|end_header_id|>" PARAMETER stop "<|eot_id|>" TEMPLATE """ {{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|> {{- if .System }} {{ .System }} {{- end }} {{- if .Tools }} Cutting Knowledge Date: December 2023 When you receive a tool call response, use the output to format an answer to the orginal user question. You are a helpful assistant with tool calling capabilities. {{- end }}<|eot_id|> {{- end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 }} {{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|> {{- if and $.Tools $last }} Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables. {{ range $.Tools }} {{- . }} {{ end }} Question: {{ .Content }}<|eot_id|> {{- else }} {{ .Content }}<|eot_id|> {{- end }}{{ if $last }}<|start_header_id|>assistant<|end_header_id|> {{ end }} {{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|> {{- if .ToolCalls }} {{ range .ToolCalls }} {"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }} {{- else }} {{ .Content }} {{- end }}{{ if not $last }}<|eot_id|>{{ end }} {{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|> {{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|> {{ end }} {{- end }} {{- end }} """ ``` ( That is a guess based on parameters/template on llama3.2 and llama3.3 models in ollama repository: https://ollama.com/library/llama3.2-vision:90b https://ollama.com/library/llama3.3:70b )
GiteaMirror commented 2026-04-29 06:18:14 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 28, 2025):

echo FROM /path/to/gguf > Modelfile 
ollama show --modelfile llama3.3 | grep -v "^FROM" >> Modelfile
ollama create llama-3.3-90b-vision
<!-- gh-comment-id:3234434982 --> @rick-github commented on GitHub (Aug 28, 2025): ```console echo FROM /path/to/gguf > Modelfile ollama show --modelfile llama3.3 | grep -v "^FROM" >> Modelfile ollama create llama-3.3-90b-vision ```
GiteaMirror commented 2026-04-29 06:18:15 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

Yup, that's more or less similar, just without the license blocks.
I may be wrong, but I don't think temperature/top_p are likely to break things that badly, and top_p=0.9 seems to be the default anyway.
I'll try again with the template added.

<!-- gh-comment-id:3234448102 --> @gordan-bobic commented on GitHub (Aug 28, 2025): Yup, that's more or less similar, just without the license blocks. I may be wrong, but I don't *think* temperature/top_p are likely to break things that badly, and top_p=0.9 seems to be the default anyway. I'll try again with the template added.
GiteaMirror commented 2026-04-29 06:18:17 -05:00
Author
Owner
Copy Link

@gordan-bobic commented on GitHub (Aug 28, 2025):

Confirmed - with the template, text mode works, but image processing errors out with:

im2col.cu:84: GGML_ASSERT(dst->type == GGML_TYPE_F16 || dst->type == GGML_TYPE_F32) failed

The question, then, is why tensor(s) of wrong type end up in the converted+quantized model.

<!-- gh-comment-id:3235273585 --> @gordan-bobic commented on GitHub (Aug 28, 2025): Confirmed - with the template, text mode works, but image processing errors out with: ``` im2col.cu:84: GGML_ASSERT(dst->type == GGML_TYPE_F16 || dst->type == GGML_TYPE_F32) failed ``` The question, then, is why tensor(s) of wrong type end up in the converted+quantized model.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#54545
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?