github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3102] Response_format not supported #48417

New Issue
Closed
opened 2026-04-28 08:12:44 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-28 08:12:44 -05:00
Owner
Copy Link

Originally created by @halcwb on GitHub (Mar 13, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3102

When sending this request to the open AI endpoint I don't get the requested JSON. The payload is the actual content send to the api. Exactly the same content works with for example the fireworks api.

ℹ INFO:
EndPoint: http://localhost:11434/api/chat
Payload:
{"format":"json","messages":[{"content":"What is the minimal age for a neonate 28 weeks to 32 weeks corrected gestational age. Reply just in one JSON.","role":"user"}],"model":"llama2","options":{"num_keep":null,"seed":101,"num_predict":null,"top_k":null,"top_p":null,"tfs_z":null,"typical_p":null,"repeat_last_n":64,"temperature":0.0,"repeat_penalty":null,"presence_penalty":null,"frequency_penalty":null,"mirostat":0,"mirostat_tau":null,"mirostat_eta":null,"penalize_newline":null,"stop":[],"numa":null,"num_ctx":2048,"num_batch":null,"num_gqa":null,"num_gpu":null,"main_gpu":null,"low_vram":null,"f16_kv":null,"vocab_only":null,"use_mmap":null,"use_mlock":null,"rope_frequency_base":null,"rope_frequency_scale":null,"num_thread":null},"response_format":{"schema":{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "<>f__AnonymousType3408661278OfIntegerAndString",
"type": "object",
"additionalProperties": false,
"properties": {
"number": {
"type": "integer",
"format": "int32"
},
"unit": {
"type": [
"null",
"string"
]
}
}
},"type":"json_object"},"stream":false}

val it: Result<Response<Ollama.OllamaResponse>,string> =
Ok
{ Original =
"{"model":"llama2","created_at":"2024-03-13T12:22:59.29158Z","message":{"role":"assistant","content":"{\n"minimal age": 4000,\n"units": "days"\n}\n\n\n\n"},"done":true,"total_duration":809391791,"load_duration":2388625,"prompt_eval_count":50,"prompt_eval_duration":341193000,"eval_count":28,"eval_duration":464799000}"
Response =
{ error = null
model = "llama2"
created_at = "2024-03-13T12:22:59.29158Z"
response = null
message = { content = "{
"minimal age": 4000,
"units": "days"
}

"
role = "assistant" }
done = true
context = null
total_duration = 809391791L
load_duration = 2388625L
prompt_eval_duration = 341193000L
eval_count = 28
eval_duration = 464799000L } }

The workaround mentioned in #2844 I don't get to work

Originally created by @halcwb on GitHub (Mar 13, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3102 When sending this request to the open AI endpoint I don't get the requested JSON. The payload is the actual content send to the api. Exactly the same content works with for example the fireworks api. ℹ INFO: EndPoint: http://localhost:11434/api/chat Payload: {"format":"json","messages":[{"content":"What is the minimal age for a neonate 28 weeks to 32 weeks corrected gestational age. Reply just in one JSON.","role":"user"}],"model":"llama2","options":{"num_keep":null,"seed":101,"num_predict":null,"top_k":null,"top_p":null,"tfs_z":null,"typical_p":null,"repeat_last_n":64,"temperature":0.0,"repeat_penalty":null,"presence_penalty":null,"frequency_penalty":null,"mirostat":0,"mirostat_tau":null,"mirostat_eta":null,"penalize_newline":null,"stop":[],"numa":null,"num_ctx":2048,"num_batch":null,"num_gqa":null,"num_gpu":null,"main_gpu":null,"low_vram":null,"f16_kv":null,"vocab_only":null,"use_mmap":null,"use_mlock":null,"rope_frequency_base":null,"rope_frequency_scale":null,"num_thread":null},"response_format":{"schema":{ "$schema": "http://json-schema.org/draft-04/schema#", "title": "<>f__AnonymousType3408661278OfIntegerAndString", "type": "object", "additionalProperties": false, "properties": { "number": { "type": "integer", "format": "int32" }, "unit": { "type": [ "null", "string" ] } } },"type":"json_object"},"stream":false} val it: Result<Response<Ollama.OllamaResponse>,string> = Ok { Original = "{"model":"llama2","created_at":"2024-03-13T12:22:59.29158Z","message":{"role":"assistant","content":"{\n\"minimal age\": 4000,\n\"units\": \"days\"\n}\n\n\n\n"},"done":true,"total_duration":809391791,"load_duration":2388625,"prompt_eval_count":50,"prompt_eval_duration":341193000,"eval_count":28,"eval_duration":464799000}" Response = { error = null model = "llama2" created_at = "2024-03-13T12:22:59.29158Z" response = null message = { content = "{ "minimal age": 4000, "units": "days" } " role = "assistant" } done = true context = null total_duration = 809391791L load_duration = 2388625L prompt_eval_duration = 341193000L eval_count = 28 eval_duration = 464799000L } } The workaround mentioned in #2844 I don't get to work
GiteaMirror added the question label 2026-04-28 08:12:44 -05:00
GiteaMirror commented 2026-04-28 08:12:48 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Mar 13, 2024):

Hi @halcwb it looks like you may be sending a response schematic in the a response_format parameter. This isn't supported by Ollama currently, only format: json. Please let me know if that helps.

<!-- gh-comment-id:1994558805 --> @BruceMacD commented on GitHub (Mar 13, 2024): Hi @halcwb it looks like you may be sending a response schematic in the a `response_format` parameter. This isn't supported by Ollama currently, only `format: json`. Please let me know if that helps.
GiteaMirror commented 2026-04-28 08:12:52 -05:00
Author
Owner
Copy Link

@halcwb commented on GitHub (Mar 13, 2024):

@BruceMacD : Thanks, saw that in the ollama code. Also, trying to add a schema as a string to the prompt seems not to work. The work around is just to add manually the json structure like:


"""
Use schema: { number: int; unit: string }
What is the minimal age for a neonate 28 weeks to 32 weeks corrected gestational age.
Reply just in one JSON."""
|> Message.user
|> Ollama.json<{| number: int; unit: string |}>
    Ollama.Models.llama2
    []
|> Async.RunSynchronously

First I had the code translate the anonymous object definition (generic parameter in the function call) to a regular json schema and then I tried to add that to the prompt as the schema to use. But that didn't work.

Hope ollama is going to support the schema field as well for the response_format. But I already am very enthusiastic about ollama!

<!-- gh-comment-id:1994603881 --> @halcwb commented on GitHub (Mar 13, 2024): @BruceMacD : Thanks, saw that in the ollama code. Also, trying to add a schema as a string to the prompt seems not to work. The work around is just to add manually the json structure like: ```fsharp """ Use schema: { number: int; unit: string } What is the minimal age for a neonate 28 weeks to 32 weeks corrected gestational age. Reply just in one JSON.""" |> Message.user |> Ollama.json<{| number: int; unit: string |}> Ollama.Models.llama2 [] |> Async.RunSynchronously ``` First I had the code translate the anonymous object definition (generic parameter in the function call) to a regular json schema and then I tried to add that to the prompt as the schema to use. But that didn't work. Hope ollama is going to support the schema field as well for the response_format. But I already am very enthusiastic about ollama!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48417
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?