github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #2754] [CLOSED] Introduce GBNF grammar parameter: run flags, REPL and API #42215

New Issue
Closed
opened 2026-04-24 22:00:08 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-24 22:00:08 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/2754
Author: @zopieux
Created: 2/26/2024
Status: ❌ Closed

Base: main ← Head: grammar

📝 Commits (1)

  • af30144 Introduce GBNF grammar parameter: run flags, REPL and API

📊 Changes

6 files changed (+144 additions, -41 deletions)

View changed files

📝 api/types.go (+5 -0)
📝 cmd/cmd.go (+40 -3)
📝 cmd/interactive.go (+38 -11)
📝 llm/server.go (+3 -2)
📝 openai/openai.go (+12 -21)
📝 server/routes.go (+46 -4)

📄 Description

This patch adds full support for the llama.cpp "grammar" option, which defaults to "" i.e. no grammar constraint.

For backwards compatability, the existing format parameter remains available in flags, the REPL and the API. Only one of grammar or format at a time is legal, since format: json is just a shortcut for the hard-coded JSON grammar.

Tested locally:

$ ./ollama run mistral-openorca --grammar-file json-two-string-array.bnf \
  'Write the result of "3+5" followed by the english spelling of the number, as two JSON strings'
["8", "Eight"]

$ ./ollama run mistral-openorca --grammar ' root ::= "[" [0-9]+ ", \"" [a-z]+ "\"]" ' \
  'Write the result of "3+5" followed by the english spelling of the number, as a json number and a json string'
[8, "eight"]

This more or less conflicts with #2404 which instead decided to go with an Options, which AFAICT is more geared towards creating derived models with an embedded grammar, whereas this patch is more about runtime override. It would be nice to eventually converge on something that does both, but today the concept of "format", which is very limiting (list of well-known formats and only JSON today), is not an Options but a top-level parameter and I wasn't sure how to make this play nice with #2404 without too many changes.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/2754 **Author:** [@zopieux](https://github.com/zopieux) **Created:** 2/26/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `grammar` --- ### 📝 Commits (1) - [`af30144`](https://github.com/ollama/ollama/commit/af30144cf3e5b317972e7d14244391ec1ce38c3e) Introduce GBNF grammar parameter: run flags, REPL and API ### 📊 Changes **6 files changed** (+144 additions, -41 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+5 -0) 📝 `cmd/cmd.go` (+40 -3) 📝 `cmd/interactive.go` (+38 -11) 📝 `llm/server.go` (+3 -2) 📝 `openai/openai.go` (+12 -21) 📝 `server/routes.go` (+46 -4) </details> ### 📄 Description This patch adds full support for the llama.cpp "grammar" option, which defaults to "" i.e. no grammar constraint. For backwards compatability, the existing `format` parameter remains available in flags, the REPL and the API. Only one of `grammar` or `format` at a time is legal, since `format: json` is just a shortcut for the hard-coded JSON grammar. Tested locally: ```console $ ./ollama run mistral-openorca --grammar-file json-two-string-array.bnf \ 'Write the result of "3+5" followed by the english spelling of the number, as two JSON strings' ["8", "Eight"] $ ./ollama run mistral-openorca --grammar ' root ::= "[" [0-9]+ ", \"" [a-z]+ "\"]" ' \ 'Write the result of "3+5" followed by the english spelling of the number, as a json number and a json string' [8, "eight"] ``` This more or less conflicts with #2404 which instead decided to go with an `Options`, which AFAICT is more geared towards creating derived models with an embedded grammar, whereas this patch is more about runtime override. It would be nice to eventually converge on something that does both, but today the concept of "format", which is very limiting (list of well-known formats and only JSON today), is not an `Options` but a top-level parameter and I wasn't sure how to make this play nice with #2404 without too many changes. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-24 22:00:08 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#42215
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?