github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #648] Model Parameters Not Getting Set #288

New Issue
Closed
opened 2026-04-12 09:49:46 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 09:49:46 -05:00
Owner
Copy Link

Originally created by @fmackenzie on GitHub (Sep 29, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/648

Originally assigned to: @BruceMacD on GitHub.

From what I can tell, the parameters set in the model file are not getting set properly. Taking the mario Modelfile as an example and adding an EMBED and a few PARAMETERS, it looks like in the server output that the PARAMETERS are having issues getting set to the appropriate type, and thus are not actually getting set as configured.

Here's the sample Modelfile

FROM llama2

EMBED /data/ollama/data/sample-content/*.txt

PARAMETER temperature 0.8
# PARAMETER num_thread 2
PARAMETER num_ctx 4096
PARAMETER num_gpu 1


SYSTEM """
You are Mario from super mario bros, acting as an assistant.
"""

and when the creation of the new model is run, against the started server, the following outcomes appear to indicate that there are issues setting the data values as configured:

2023/09/29 12:23:06 types.go:234: could not convert model parameter num_ctx to int, skipped
2023/09/29 12:23:06 types.go:234: could not convert model parameter num_gpu to int, skipped
2023/09/29 12:23:06 types.go:247: could not convert model parameter temperature to float32, skipped
Originally created by @fmackenzie on GitHub (Sep 29, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/648 Originally assigned to: @BruceMacD on GitHub. From what I can tell, the parameters set in the model file are not getting set properly. Taking the mario Modelfile as an example and adding an EMBED and a few PARAMETERS, it looks like in the server output that the PARAMETERS are having issues getting set to the appropriate type, and thus are not actually getting set as configured. Here's the sample Modelfile ``` FROM llama2 EMBED /data/ollama/data/sample-content/*.txt PARAMETER temperature 0.8 # PARAMETER num_thread 2 PARAMETER num_ctx 4096 PARAMETER num_gpu 1 SYSTEM """ You are Mario from super mario bros, acting as an assistant. """ ``` and when the creation of the new model is run, against the started server, the following outcomes appear to indicate that there are issues setting the data values as configured: ``` 2023/09/29 12:23:06 types.go:234: could not convert model parameter num_ctx to int, skipped 2023/09/29 12:23:06 types.go:234: could not convert model parameter num_gpu to int, skipped 2023/09/29 12:23:06 types.go:247: could not convert model parameter temperature to float32, skipped ```
GiteaMirror added the bug label 2026-04-12 09:49:46 -05:00
GiteaMirror commented 2026-04-12 09:49:47 -05:00
Author
Owner
Copy Link

@mchiang0610 commented on GitHub (Sep 30, 2023):

May I ask what version of ollama you're running? ollama -v and if you are running it with langchain? I believe the langchain integration passes null values if they are not set, and will be ignored.

<!-- gh-comment-id:1741675764 --> @mchiang0610 commented on GitHub (Sep 30, 2023): May I ask what version of ollama you're running? `ollama -v` and if you are running it with langchain? I believe the langchain integration passes null values if they are not set, and will be ignored.
GiteaMirror commented 2026-04-12 09:49:47 -05:00
Author
Owner
Copy Link

@fmackenzie commented on GitHub (Oct 1, 2023):

Sure. I'm using the latest build (main branch compiled as of Sept 30) on Ubuntu 23.04. As a side note, I believe it is the message that is incorrect.

The current code in types.go (lines 227-237ish):

				case reflect.Int:
					switch t := val.(type) {
					case int64:
						field.SetInt(t)
					case float64:
						// when JSON unmarshals numbers, it uses float64, not int
						field.SetInt(int64(t))
					default:
						log.Printf("could not convert model parameter %v to int, skipped", key)
					}

However, it looks like it shouldn't be indicating that it is being skipped in this case as it appears that the parameter is actually already an INT. I've updated my code to identify if we have the correct type (so we don't need to convert it), and it looks like this:

				case reflect.Int:
					switch t := val.(type) {
					case int:
						log.Printf("Using int parameter %v with value %v as provided", key, t)
					case int64:
						field.SetInt(t)
					case float64:
						// when JSON unmarshals numbers, it uses float64, not int
						field.SetInt(int64(t))
					default:
						log.Printf("could not convert model parameter %v to int, skipped", key)
						log.Printf("unknown type %s for %s", field.Kind(), key)
					}

This output now shows the following:

2023/10/01 07:43:35 images.go:317: [temperature] - 0.8
2023/10/01 07:43:35 images.go:317: [num_thread] - 4
2023/10/01 07:43:35 images.go:317: [num_ctx] - 8192
2023/10/01 07:43:35 images.go:317: [num_gpu] - 1
2023/10/01 07:43:35 images.go:317: [system] - 
2023/10/01 07:43:35 types.go:230: Using int parameter num_gpu with value 1 as provided
2023/10/01 07:43:35 types.go:251: Using float32 parameter temperature with value 0.8 as provided
2023/10/01 07:43:35 types.go:230: Using int parameter num_thread with value 4 as provided
2023/10/01 07:43:35 types.go:230: Using int parameter num_ctx with value 8192 as provided
2023/10/01 07:43:35 llama.go:313: starting llama runner

What I don't know, is that we are using the correct values further down the line, but I will investigate further.

<!-- gh-comment-id:1742056298 --> @fmackenzie commented on GitHub (Oct 1, 2023): Sure. I'm using the latest build (main branch compiled as of Sept 30) on Ubuntu 23.04. As a side note, I believe it is the message that is incorrect. The current code in types.go (lines 227-237ish): ``` case reflect.Int: switch t := val.(type) { case int64: field.SetInt(t) case float64: // when JSON unmarshals numbers, it uses float64, not int field.SetInt(int64(t)) default: log.Printf("could not convert model parameter %v to int, skipped", key) } ``` However, it looks like it shouldn't be indicating that it is being skipped in this case as it appears that the parameter is actually already an INT. I've updated my code to identify if we have the correct type (so we don't need to convert it), and it looks like this: ``` case reflect.Int: switch t := val.(type) { case int: log.Printf("Using int parameter %v with value %v as provided", key, t) case int64: field.SetInt(t) case float64: // when JSON unmarshals numbers, it uses float64, not int field.SetInt(int64(t)) default: log.Printf("could not convert model parameter %v to int, skipped", key) log.Printf("unknown type %s for %s", field.Kind(), key) } ``` This output now shows the following: ``` 2023/10/01 07:43:35 images.go:317: [temperature] - 0.8 2023/10/01 07:43:35 images.go:317: [num_thread] - 4 2023/10/01 07:43:35 images.go:317: [num_ctx] - 8192 2023/10/01 07:43:35 images.go:317: [num_gpu] - 1 2023/10/01 07:43:35 images.go:317: [system] - 2023/10/01 07:43:35 types.go:230: Using int parameter num_gpu with value 1 as provided 2023/10/01 07:43:35 types.go:251: Using float32 parameter temperature with value 0.8 as provided 2023/10/01 07:43:35 types.go:230: Using int parameter num_thread with value 4 as provided 2023/10/01 07:43:35 types.go:230: Using int parameter num_ctx with value 8192 as provided 2023/10/01 07:43:35 llama.go:313: starting llama runner ``` What I don't know, is that we are using the correct values further down the line, but I will investigate further.
GiteaMirror commented 2026-04-12 09:49:48 -05:00
Author
Owner
Copy Link

@JoseConseco commented on GitHub (Oct 2, 2023):

I have similar issue. eg. num_gpu seems completely ignored. I did not test too much of other params.

<!-- gh-comment-id:1742892395 --> @JoseConseco commented on GitHub (Oct 2, 2023): I have similar issue. eg. num_gpu seems completely ignored. I did not test too much of other params.
GiteaMirror commented 2026-04-12 09:49:48 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Oct 2, 2023):

@JoseConseco is this on MacOS or Linux? MacOS only supports 0/1 gpu (corresponding with cpu/metal).

<!-- gh-comment-id:1743526672 --> @BruceMacD commented on GitHub (Oct 2, 2023): @JoseConseco is this on MacOS or Linux? MacOS only supports 0/1 gpu (corresponding with cpu/metal).
GiteaMirror commented 2026-04-12 09:49:48 -05:00
Author
Owner
Copy Link

@JoseConseco commented on GitHub (Oct 2, 2023):

Linux. Ok, it seems to work after all. My bad. I just had to kill the ollama server, to make sure it new version with gpu layers is loaded.

<!-- gh-comment-id:1743593601 --> @JoseConseco commented on GitHub (Oct 2, 2023): Linux. Ok, it seems to work after all. My bad. I just had to kill the ollama server, to make sure it new version with gpu layers is loaded.
GiteaMirror commented 2026-04-12 09:49:49 -05:00
Author
Owner
Copy Link

@jtoy commented on GitHub (Oct 2, 2023):

this can be closed

<!-- gh-comment-id:1743643962 --> @jtoy commented on GitHub (Oct 2, 2023): this can be closed
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#288
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?