github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4074] Grammar Guided response from model. #28289

New Issue
Closed
opened 2026-04-22 06:17:09 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-22 06:17:09 -05:00
Owner
Copy Link

Originally created by @NeevJewalkar on GitHub (May 1, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4074

Llama.cpp already has an implementation of grammars. Adding this to the ollama API, would bring more flexibility. This would also help in making reliable agentic llms.

Originally created by @NeevJewalkar on GitHub (May 1, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4074 Llama.cpp already has an implementation of grammars. Adding this to the ollama API, would bring more flexibility. This would also help in making reliable agentic llms.
GiteaMirror added the feature request label 2026-04-22 06:17:09 -05:00
GiteaMirror commented 2026-04-22 06:17:11 -05:00
Author
Owner
Copy Link

@grantCelley commented on GitHub (May 5, 2024):

There is a few PRs over the past year.

PR #565
Last updated: December 19th, 2024
The only thing I can tell is it opens up grammar from llama.cpp to the API. This was simple implementation but stopped it seems it was updated in pr #1606. This has merge conflicts.

PR #830
Last updated: April 29th, 2024
It allowed a json schema as the grammars. The last comment just explained why this is a good idea. Before that the last activity on the Pull request. This has merge conflicts.

PR #1606
Last updated: April 30th, 2024
This is an updated version of pr #565. This got a lot of comments. This has merge conflicts.

PR #3618
Last updated: April 23rd, 2024
This was an modification of pr #1606. It works for a JSON schema. It seems it worked as of April 23rd.

PR #2404
Last update: April 30th, 2024
This used format argument to allow grammar. Instead JSON you put in the Grammar. This is has conflicts now.

PR #2754
Last update: April 9th, 2024
This would allow a cli output onto ollama. This has conflicts now.

PR #3303
Last update: April 11th, 2024
This would add function calling. It says that it is a work in progress but does not have any recent activity nor any comments on how well this is going.

Why am I adding this to here?
This seems to be wanted by a bunch of people. None of the pull requests were merged and with 1.33 I think it caused a bunch of these pull requests not to work. I am trying to use LLMs for decision making. I need them to output in a standard format that I set.

Why aren't these merged yet?
I really don't know. This is something I am trying to figure out. I am wondering if this is not how the maintainers want the API to be. Some guidance would be nice.

<!-- gh-comment-id:2094860328 --> @grantCelley commented on GitHub (May 5, 2024): There is a few PRs over the past year. PR #565 Last updated: December 19th, 2024 The only thing I can tell is it opens up `grammar` from llama.cpp to the API. This was simple implementation but stopped it seems it was updated in pr #1606. This has merge conflicts. PR #830 Last updated: April 29th, 2024 It allowed a json schema as the grammars. The last comment just explained why this is a good idea. Before that the last activity on the Pull request. This has merge conflicts. PR #1606 Last updated: April 30th, 2024 This is an updated version of pr #565. This got a lot of comments. This has merge conflicts. PR #3618 Last updated: April 23rd, 2024 This was an modification of pr #1606. It works for a JSON schema. It seems it worked as of April 23rd. PR #2404 Last update: April 30th, 2024 This used format argument to allow grammar. Instead JSON you put in the Grammar. This is has conflicts now. PR #2754 Last update: April 9th, 2024 This would allow a cli output onto ollama. This has conflicts now. PR #3303 Last update: April 11th, 2024 This would add function calling. It says that it is a work in progress but does not have any recent activity nor any comments on how well this is going. Why am I adding this to here? This seems to be wanted by a bunch of people. None of the pull requests were merged and with 1.33 I think it caused a bunch of these pull requests not to work. I am trying to use LLMs for decision making. I need them to output in a standard format that I set. Why aren't these merged yet? I really don't know. This is something I am trying to figure out. I am wondering if this is not how the maintainers want the API to be. Some guidance would be nice.
GiteaMirror commented 2026-04-22 06:17:14 -05:00
Author
Owner
Copy Link

@richardanaya commented on GitHub (May 7, 2024):

i am able to easily break Ollama's JSON support like this even with embedding JSON schema in prompts.

image

<!-- gh-comment-id:2097418956 --> @richardanaya commented on GitHub (May 7, 2024): i am able to easily break Ollama's JSON support like this even with embedding JSON schema in prompts. ![image](https://github.com/ollama/ollama/assets/294042/615c5c5e-65cd-458b-8c22-4f0f3087a979)
GiteaMirror commented 2026-04-22 06:17:15 -05:00
Author
Owner
Copy Link

@NeevJewalkar commented on GitHub (May 16, 2024):

it wld be helpful if ollama had support for custom grammar, since the json grammar file provided by llama.cpp isnt really that helpful

<!-- gh-comment-id:2114058019 --> @NeevJewalkar commented on GitHub (May 16, 2024): it wld be helpful if ollama had support for custom grammar, since the json grammar file provided by llama.cpp isnt really that helpful
GiteaMirror commented 2026-04-22 06:17:17 -05:00
Author
Owner
Copy Link

@richardanaya commented on GitHub (May 19, 2024):

I created an extensive PR for supporting grammar and go-side server validation here

https://github.com/ollama/ollama/pull/4525

<!-- gh-comment-id:2119351176 --> @richardanaya commented on GitHub (May 19, 2024): I created an extensive PR for supporting grammar and go-side server validation here https://github.com/ollama/ollama/pull/4525
GiteaMirror commented 2026-04-22 06:17:18 -05:00
Author
Owner
Copy Link

@rhohndorf commented on GitHub (Jul 5, 2024):

It is extremely frustrating that the ollama core team is so silent about this topic.
I am working on small llm local agents. I would love to use ollama but I need grammar support.

<!-- gh-comment-id:2210303161 --> @rhohndorf commented on GitHub (Jul 5, 2024): It is extremely frustrating that the ollama core team is so silent about this topic. I am working on small llm local agents. I would love to use ollama but I need grammar support.
GiteaMirror commented 2026-04-22 06:17:19 -05:00
Author
Owner
Copy Link

@Kinglord commented on GitHub (Aug 7, 2024):

Hey all, I know there's an automated ping here but just to better align everyone please check out and comment on my new call to the Ollama team for clarity here. As always please be civil and stay on topic! 😄 - https://github.com/ollama/ollama/issues/6237

<!-- gh-comment-id:2274314837 --> @Kinglord commented on GitHub (Aug 7, 2024): Hey all, I know there's an automated ping here but just to better align everyone please check out and comment on my new call to the Ollama team for clarity here. As always please be civil and stay on topic! 😄 - https://github.com/ollama/ollama/issues/6237
GiteaMirror commented 2026-04-22 06:17:20 -05:00
Author
Owner
Copy Link

@ParthSareen commented on GitHub (Dec 5, 2024):

Hey everyone! Going to close this out as we're supporting structured outputs through https://github.com/ollama/ollama/pull/7900

Left a comment with some background as well: https://github.com/ollama/ollama/issues/6237#issuecomment-2518836758

<!-- gh-comment-id:2518847278 --> @ParthSareen commented on GitHub (Dec 5, 2024): Hey everyone! Going to close this out as we're supporting structured outputs through https://github.com/ollama/ollama/pull/7900 Left a comment with some background as well: https://github.com/ollama/ollama/issues/6237#issuecomment-2518836758
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#28289
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?