github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3019] Automatic sub-language constraint sections #1856

New Issue
Open
opened 2026-04-12 11:55:07 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 11:55:07 -05:00
Owner
Copy Link

Originally created by @mirek on GitHub (Mar 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3019

It would be useful if ollama supported automatic, text based, plugin driven grammar sections.

Every time triple quote section is used in text it enters that language mode constraints, for example:

  1. "```json" enters json bnf
  2. "```json:Foo" enters json bnf + json schema for Foo object
  3. "```python" enters python bnf
  4. "```quote:documentRef" enters trie based constraints for quotes
  5. "```whatever:whatever?" enters whatever
  6. final "\n```" pops the mode

Available grammars could be:

  1. server/model defined – avoiding network overhead
  2. client defined – flexible under client side control (probably much better)

The api for plugin/handler could be:

  1. character based – given text from beginning of opening "```foo" input it returns allowed characters, or
  2. native model token based – given same input returns list of allowed tokens, or
  3. generic suffix text – given same input returns list of allowed, arbitrarily sized completions (so plugin can adopt, ie. for wide fanout it can return list of short strings, ie. single characters, for less fanout it can produce longer output with possible single output optimization where llm doesn't need to be consulted <<if there is single completion allowed, no need to run it though the model at all>>), or
  4. optimized trie output – given same input returns 3. in compressed, trie format.

Plugin can internally use whatever strategy it wants to conform to autocompletion interface (BNF, trie based index for document quotes or something else).

Heading can be uri based, ie. "```trie://mydb/docs?id=123" so plugins can register themselves as handlers for specific modes.

Originally created by @mirek on GitHub (Mar 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3019 It would be useful if ollama supported automatic, text based, plugin driven grammar sections. Every time triple quote section is used in text it enters that language mode constraints, for example: 1. "```json" enters json bnf 2. "```json:Foo" enters json bnf + json schema for Foo object 3. "```python" enters python bnf 4. "```quote:documentRef" enters trie based constraints for quotes 5. "```whatever:whatever?" enters whatever 6. final "\n```" pops the mode Available grammars could be: 1. server/model defined – avoiding network overhead 2. client defined – flexible under client side control (probably much better) The api for plugin/handler could be: 1. character based – given text from beginning of opening "```foo" input it returns allowed characters, or 2. native model token based – given same input returns list of allowed tokens, or 3. generic suffix text – given same input returns list of allowed, arbitrarily sized completions (so plugin can adopt, ie. for wide fanout it can return list of short strings, ie. single characters, for less fanout it can produce longer output with possible single output optimization where llm doesn't need to be consulted <<if there is single completion allowed, no need to run it though the model at all>>), or 4. optimized trie output – given same input returns 3. in compressed, trie format. Plugin can internally use whatever strategy it wants to conform to autocompletion interface (BNF, trie based index for document quotes or something else). Heading can be uri based, ie. "```trie://mydb/docs?id=123" so plugins can register themselves as handlers for specific modes.
GiteaMirror added the feature request label 2026-04-12 11:55:07 -05:00
GiteaMirror commented 2026-04-12 11:55:08 -05:00
Author
Owner
Copy Link

@mirek commented on GitHub (Mar 12, 2024):

@jmorganca this proposal is not related to supporting more format options beyond current json.

It's about automatically recognizing triple quote sections within text itself (and ideally handing over control to the client).

It can be though of as simple to use solution that sits between high level "format" and low level aici.

<!-- gh-comment-id:1990912965 --> @mirek commented on GitHub (Mar 12, 2024): @jmorganca this proposal is not related to supporting more `format` options beyond current `json`. It's about automatically recognizing triple quote sections within text itself (and ideally handing over control to the client). It can be though of as simple to use solution that sits between high level "format" and low level [aici](https://github.com/microsoft/aici).
GiteaMirror commented 2026-04-12 11:55:09 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Mar 12, 2024):

Ah sorry for misunderstanding!

<!-- gh-comment-id:1990935644 --> @jmorganca commented on GitHub (Mar 12, 2024): Ah sorry for misunderstanding!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1856
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?