github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #12899] [MERGED] Allow returning logprobs #19266

New Issue
Closed
opened 2026-04-16 07:02:21 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-16 07:02:21 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/12899
Author: @baptistejamin
Created: 11/1/2025
Status: ✅ Merged
Merged: 11/11/2025
Merged by: @jmorganca

Base: main ← Head: main

📝 Commits (8)

  • c83285c Allow returning logprobs
  • b420f73 Add unit tests and commonize runner functions
  • 0708f0a Remove missing debug request flag
  • 51e4202 Fix lint issue
  • 823f94b Use llm.Logprob instead of common.Logprob, and remove ToLLMLogprobs
  • 56a729d Add routes_generate_tests
  • 495ae27 Harden tests and and fix completion with stop tokens ans well as better buffering for chunk
  • 8f62edd lint pass

📊 Changes

13 files changed (+1367 additions, -47 deletions)

View changed files

📝 api/types.go (+42 -0)
📝 integration/api_test.go (+171 -0)
📝 llama/llama.go (+13 -0)
📝 llm/server.go (+23 -1)
📝 openai/openai.go (+44 -9)
📝 openai/openai_test.go (+216 -0)
➕ runner/common/logprob.go (+79 -0)
➕ runner/common/logprob_test.go (+498 -0)
📝 runner/llamarunner/runner.go (+52 -5)
📝 runner/ollamarunner/runner.go (+69 -20)
➕ server/logprob.go (+29 -0)
📝 server/routes.go (+51 -12)
📝 server/routes_generate_test.go (+80 -0)

📄 Description

Hey there.

This pull request provides a simple way to return logprobs for both LLama and Ollama runners.

This PR does not contain any unit tests for now, since I don't know if this can be merged or not.

curl http://localhost:11434/api/generate -d '{
   "model": "llama3.2",
    "prompt": "The capital of France is",
    "stream": false,
    "logprobs": true,
    "top_logprobs": 1
}'

{
    "model": "llama3.2",
    "created_at": "2025-11-01T12:46:02.322107Z",
    "response": "Paris.",
    "done": true,
    "done_reason": "stop",
    "context": [
        128006,
        9125,
        128007,
        271,
        38766,
        1303,
        33025,
        2696,
        25,
        6790,
        220,
        2366,
        18,
        271,
        128009,
        128006,
        882,
        128007,
        271,
        791,
        6864,
        315,
        9822,
        374,
        128009,
        128006,
        78191,
        128007,
        271,
        60704,
        13
    ],
    "total_duration": 661622667,
    "load_duration": 535133167,
    "prompt_eval_count": 30,
    "prompt_eval_duration": 51514750,
    "eval_count": 3,
    "eval_duration": 16629751,
    "logprobs": [
        {
            "token": "Paris",
            "logprob": -0.0820460096001625,
            "top_logprobs": [
                {
                    "token": "Paris",
                    "logprob": -0.0820460096001625
                }
            ]
        }
    ]
}

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/12899 **Author:** [@baptistejamin](https://github.com/baptistejamin) **Created:** 11/1/2025 **Status:** ✅ Merged **Merged:** 11/11/2025 **Merged by:** [@jmorganca](https://github.com/jmorganca) **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (8) - [`c83285c`](https://github.com/ollama/ollama/commit/c83285cbecec1953e7b7c87289161e35ac11fa5b) Allow returning logprobs - [`b420f73`](https://github.com/ollama/ollama/commit/b420f737eb77e9100b5babb0eb2bded55d8ac076) Add unit tests and commonize runner functions - [`0708f0a`](https://github.com/ollama/ollama/commit/0708f0a91384c660569bf41e80ed2092c6eece1c) Remove missing debug request flag - [`51e4202`](https://github.com/ollama/ollama/commit/51e4202d2beede0feaf8ee2e321c878381fcc9b6) Fix lint issue - [`823f94b`](https://github.com/ollama/ollama/commit/823f94bdfd4f318a4430cbc752413ab0117de18b) Use llm.Logprob instead of common.Logprob, and remove ToLLMLogprobs - [`56a729d`](https://github.com/ollama/ollama/commit/56a729d4cded9253ec5d3d94b536d6f30540376c) Add routes_generate_tests - [`495ae27`](https://github.com/ollama/ollama/commit/495ae27076ce8ea6bfa452ed9726f908e87aa484) Harden tests and and fix completion with stop tokens ans well as better buffering for chunk - [`8f62edd`](https://github.com/ollama/ollama/commit/8f62edd12d6b0254cbcbed81f5bc1a3f5e2d3dc8) lint pass ### 📊 Changes **13 files changed** (+1367 additions, -47 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+42 -0) 📝 `integration/api_test.go` (+171 -0) 📝 `llama/llama.go` (+13 -0) 📝 `llm/server.go` (+23 -1) 📝 `openai/openai.go` (+44 -9) 📝 `openai/openai_test.go` (+216 -0) ➕ `runner/common/logprob.go` (+79 -0) ➕ `runner/common/logprob_test.go` (+498 -0) 📝 `runner/llamarunner/runner.go` (+52 -5) 📝 `runner/ollamarunner/runner.go` (+69 -20) ➕ `server/logprob.go` (+29 -0) 📝 `server/routes.go` (+51 -12) 📝 `server/routes_generate_test.go` (+80 -0) </details> ### 📄 Description Hey there. This pull request provides a simple way to return logprobs for both LLama and Ollama runners. This PR does not contain any unit tests for now, since I don't know if this can be merged or not. ``` curl http://localhost:11434/api/generate -d '{ "model": "llama3.2", "prompt": "The capital of France is", "stream": false, "logprobs": true, "top_logprobs": 1 }' ``` ``` { "model": "llama3.2", "created_at": "2025-11-01T12:46:02.322107Z", "response": "Paris.", "done": true, "done_reason": "stop", "context": [ 128006, 9125, 128007, 271, 38766, 1303, 33025, 2696, 25, 6790, 220, 2366, 18, 271, 128009, 128006, 882, 128007, 271, 791, 6864, 315, 9822, 374, 128009, 128006, 78191, 128007, 271, 60704, 13 ], "total_duration": 661622667, "load_duration": 535133167, "prompt_eval_count": 30, "prompt_eval_duration": 51514750, "eval_count": 3, "eval_duration": 16629751, "logprobs": [ { "token": "Paris", "logprob": -0.0820460096001625, "top_logprobs": [ { "token": "Paris", "logprob": -0.0820460096001625 } ] } ] } ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-16 07:02:21 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#19266
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?