github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #1031] [CLOSED] Added logit_bias support #10442

New Issue
Closed
opened 2026-04-12 23:00:42 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-12 23:00:42 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/1031
Author: @Vokturz
Created: 11/7/2023
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (2)

  • 472bb86 added logit_bias support
  • 21f2692 Merge branch 'main' into main

📊 Changes

3 files changed (+51 additions, -30 deletions)

View changed files

📝 api/types.go (+48 -30)
📝 docs/modelfile.md (+2 -0)
📝 llm/llama.go (+1 -0)

📄 Description

This PR brings the logit_bias functionality, already present in llama.cpp, which allows users to adjust the likelihood of token occurrences in generated text.

For example, for the prompt "Once upon a " we have:

  1. Without logit_bias

    curl -X POST http://localhost:11434/api/generate -d '{
   "model": "llama2:7b",
   "prompt": "Once upon a ",
   "stream": false,
   "options": {"seed": 0, "num_predict" : 10}
   }'
{"model":"llama2:7b", ... , "response":"time, in a far-off land, there", ...}

  2. With logit_bias removing "time" and "Time" words:

    curl -X POST http://localhost:11434/api/generate -d '{
   "model": "llama2:7b",
   "prompt": "Once upon a ",
   "stream": false,
   "options": {"seed": 0, "num_predict" : 10,
               "logit_bias": [[2230,false],[931,false],[2481,false],[5974,false]]}
   }'
{"model":"llama2:7b", ... , "response":"Once upon a...! What a great prompt!", ...}

  3. With logit_bias removing "time" and "Time" words, and increasing "night" likelihood :

    curl -X POST http://localhost:11434/api/generate -d '{
   "model": "llama2:7b",
   "prompt": "Once upon a ",
   "stream": false,
   "options": {"seed": 0, "num_predict" : 10,
               "logit_bias": [[2230,false],[931,false],[2481,false],[5974,false],[11147, 15]]}
   }'
{"model":"llama2:7b", ... , "response":"night, in a land far away, there lived"", ...}

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/1031 **Author:** [@Vokturz](https://github.com/Vokturz) **Created:** 11/7/2023 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (2) - [`472bb86`](https://github.com/ollama/ollama/commit/472bb86ac58b0359fdaf124f84f415c94aaee402) added logit_bias support - [`21f2692`](https://github.com/ollama/ollama/commit/21f2692bd16b7cc1ca72d30c56efe1ecaf093afb) Merge branch 'main' into main ### 📊 Changes **3 files changed** (+51 additions, -30 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+48 -30) 📝 `docs/modelfile.md` (+2 -0) 📝 `llm/llama.go` (+1 -0) </details> ### 📄 Description This PR brings the `logit_bias` functionality, already present in llama.cpp, which allows users to adjust the likelihood of token occurrences in generated text. For example, for the prompt `"Once upon a "` we have: 1. Without `logit_bias` ```bash curl -X POST http://localhost:11434/api/generate -d '{ "model": "llama2:7b", "prompt": "Once upon a ", "stream": false, "options": {"seed": 0, "num_predict" : 10} }' {"model":"llama2:7b", ... , "response":"time, in a far-off land, there", ...} ``` 2. With `logit_bias` removing "time" and "Time" words: ```bash curl -X POST http://localhost:11434/api/generate -d '{ "model": "llama2:7b", "prompt": "Once upon a ", "stream": false, "options": {"seed": 0, "num_predict" : 10, "logit_bias": [[2230,false],[931,false],[2481,false],[5974,false]]} }' {"model":"llama2:7b", ... , "response":"Once upon a...! What a great prompt!", ...} ``` 2. With `logit_bias` removing "time" and "Time" words, and increasing "night" likelihood : ```bash curl -X POST http://localhost:11434/api/generate -d '{ "model": "llama2:7b", "prompt": "Once upon a ", "stream": false, "options": {"seed": 0, "num_predict" : 10, "logit_bias": [[2230,false],[931,false],[2481,false],[5974,false],[11147, 15]]} }' {"model":"llama2:7b", ... , "response":"night, in a land far away, there lived"", ...} ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-12 23:00:42 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#10442
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?