github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 17:49:24 -05:00
Code Issues 14.5k Packages Projects Releases 122 Wiki Activity

[PR #11315] report progress during load #24044

New Issue
Open
opened 2026-04-19 17:21:00 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-19 17:21:00 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/11315
Author: @dhiltgen
Created: 7/6/2025
Status: 🔄 Open

Base: main ← Head: load_progress

📝 Commits (1)

  • 82c0f46 report progress during load

📊 Changes

7 files changed (+156 additions, -47 deletions)

View changed files

📝 api/types.go (+7 -0)
📝 cmd/cmd.go (+7 -0)
📝 llm/server.go (+8 -2)
📝 server/routes.go (+118 -31)
📝 server/routes_generate_test.go (+6 -6)
📝 server/sched.go (+4 -2)
📝 server/sched_test.go (+6 -6)

📄 Description

This extends the Chat and Generate APIs to stream progress updates during model load. A side-effect of the change is loading errors can be reported in the stream if they occur after we start streaming progress updates.

Example curl output:

% curl http://localhost:11434/api/chat -d '{
  "model": "gemma3n",
  "messages":[{"role":"user","content":"hello"}]
}'
{"model":"gemma3n","created_at":"2025-07-06T20:31:51.83739Z","message":{"role":"","content":""},"done":false,"load_progress":11}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.088228Z","message":{"role":"","content":""},"done":false,"load_progress":48}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.33935Z","message":{"role":"","content":""},"done":false,"load_progress":70}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.589736Z","message":{"role":"","content":""},"done":false,"load_progress":92}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.841086Z","message":{"role":"","content":""},"done":false,"load_progress":100}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.945664Z","message":{"role":"assistant","content":"Hello"},"done":false}
{"model":"gemma3n","created_at":"2025-07-06T20:31:52.964794Z","message":{"role":"assistant","content":" there"},"done":false}
...
{"model":"gemma3n","created_at":"2025-07-06T20:31:53.705994Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":2524677375,"load_duration":1659822750,"prompt_eval_count":10,"prompt_eval_duration":103464958,"eval_count":43,"eval_duration":760736709}

% curl http://localhost:11434/api/generate -d '{
  "model": "gemma3n",
  "prompt": "hello"
}'
{"model":"gemma3n","created_at":"2025-07-06T20:52:18.804913Z","response":"","done":false,"load_progress":8}
{"model":"gemma3n","created_at":"2025-07-06T20:52:19.056388Z","response":"","done":false,"load_progress":37}
{"model":"gemma3n","created_at":"2025-07-06T20:52:19.307753Z","response":"","done":false,"load_progress":47}
{"model":"gemma3n","created_at":"2025-07-06T20:52:19.558891Z","response":"","done":false,"load_progress":64}
{"model":"gemma3n","created_at":"2025-07-06T20:52:19.810266Z","response":"","done":false,"load_progress":82}
{"model":"gemma3n","created_at":"2025-07-06T20:52:20.061133Z","response":"","done":false,"load_progress":100}
{"model":"gemma3n","created_at":"2025-07-06T20:52:20.176715Z","response":"Hello","done":false}
{"model":"gemma3n","created_at":"2025-07-06T20:52:20.194763Z","response":" there","done":false}
...
{"model":"gemma3n","created_at":"2025-07-06T20:52:23.026797Z","response":"","done":true,"done_reason":"stop","context":[105,2364,107,23391,106,107,105,4368,107,9259,993,236888,103453,236743,107,107,3910,740,564,1601,611,3124,236881,138,107,107,236777,740,236787,107,107,236829,5213,7925,822,4137,53121,236743,29020,786,614,109542,522,611,236789,500,23210,1003,236888,107,236829,5213,43204,9578,3004,53121,236743,33403,236764,510,4755,1356,236764,3393,236764,32524,236764,13906,9097,236764,4553,236764,11739,236764,4044,236761,107,236829,5213,40414,21389,44070,3624,53121,236743,564,740,17866,1534,1551,1607,21389,44070,3624,236761,107,236829,5213,160773,969,1816,53121,236743,23097,786,496,1440,4676,532,564,236789,859,2583,611,496,63510,12323,236761,107,236829,5213,67400,4393,741,6549,53121,236743,19704,1601,4891,872,607,6549,573,496,2203,236881,564,740,1601,236888,107,236829,5213,11896,11184,236888,1018,236743,18692,625,236789,236751,6290,531,735,1647,50105,866,531,2910,531,236761,107,107,11896,1531,786,1281,1144,611,236789,236753,1133,786,531,776,236888,73687,107,107,107,107],"total_duration":4879476959,"load_duration":1913893875,"prompt_eval_count":10,"prompt_eval_duration":114435209,"eval_count":157,"eval_duration":2850490458}

The interactive CLI keeps the spinner running throughout the load phase.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/11315 **Author:** [@dhiltgen](https://github.com/dhiltgen) **Created:** 7/6/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `load_progress` --- ### 📝 Commits (1) - [`82c0f46`](https://github.com/ollama/ollama/commit/82c0f466fc41dbb0cb741306c5e4236a92ba9cef) report progress during load ### 📊 Changes **7 files changed** (+156 additions, -47 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+7 -0) 📝 `cmd/cmd.go` (+7 -0) 📝 `llm/server.go` (+8 -2) 📝 `server/routes.go` (+118 -31) 📝 `server/routes_generate_test.go` (+6 -6) 📝 `server/sched.go` (+4 -2) 📝 `server/sched_test.go` (+6 -6) </details> ### 📄 Description This extends the Chat and Generate APIs to stream progress updates during model load. A side-effect of the change is loading errors can be reported in the stream if they occur after we start streaming progress updates. Example curl output: ``` % curl http://localhost:11434/api/chat -d '{ "model": "gemma3n", "messages":[{"role":"user","content":"hello"}] }' {"model":"gemma3n","created_at":"2025-07-06T20:31:51.83739Z","message":{"role":"","content":""},"done":false,"load_progress":11} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.088228Z","message":{"role":"","content":""},"done":false,"load_progress":48} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.33935Z","message":{"role":"","content":""},"done":false,"load_progress":70} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.589736Z","message":{"role":"","content":""},"done":false,"load_progress":92} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.841086Z","message":{"role":"","content":""},"done":false,"load_progress":100} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.945664Z","message":{"role":"assistant","content":"Hello"},"done":false} {"model":"gemma3n","created_at":"2025-07-06T20:31:52.964794Z","message":{"role":"assistant","content":" there"},"done":false} ... {"model":"gemma3n","created_at":"2025-07-06T20:31:53.705994Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":2524677375,"load_duration":1659822750,"prompt_eval_count":10,"prompt_eval_duration":103464958,"eval_count":43,"eval_duration":760736709} ``` ``` % curl http://localhost:11434/api/generate -d '{ "model": "gemma3n", "prompt": "hello" }' {"model":"gemma3n","created_at":"2025-07-06T20:52:18.804913Z","response":"","done":false,"load_progress":8} {"model":"gemma3n","created_at":"2025-07-06T20:52:19.056388Z","response":"","done":false,"load_progress":37} {"model":"gemma3n","created_at":"2025-07-06T20:52:19.307753Z","response":"","done":false,"load_progress":47} {"model":"gemma3n","created_at":"2025-07-06T20:52:19.558891Z","response":"","done":false,"load_progress":64} {"model":"gemma3n","created_at":"2025-07-06T20:52:19.810266Z","response":"","done":false,"load_progress":82} {"model":"gemma3n","created_at":"2025-07-06T20:52:20.061133Z","response":"","done":false,"load_progress":100} {"model":"gemma3n","created_at":"2025-07-06T20:52:20.176715Z","response":"Hello","done":false} {"model":"gemma3n","created_at":"2025-07-06T20:52:20.194763Z","response":" there","done":false} ... {"model":"gemma3n","created_at":"2025-07-06T20:52:23.026797Z","response":"","done":true,"done_reason":"stop","context":[105,2364,107,23391,106,107,105,4368,107,9259,993,236888,103453,236743,107,107,3910,740,564,1601,611,3124,236881,138,107,107,236777,740,236787,107,107,236829,5213,7925,822,4137,53121,236743,29020,786,614,109542,522,611,236789,500,23210,1003,236888,107,236829,5213,43204,9578,3004,53121,236743,33403,236764,510,4755,1356,236764,3393,236764,32524,236764,13906,9097,236764,4553,236764,11739,236764,4044,236761,107,236829,5213,40414,21389,44070,3624,53121,236743,564,740,17866,1534,1551,1607,21389,44070,3624,236761,107,236829,5213,160773,969,1816,53121,236743,23097,786,496,1440,4676,532,564,236789,859,2583,611,496,63510,12323,236761,107,236829,5213,67400,4393,741,6549,53121,236743,19704,1601,4891,872,607,6549,573,496,2203,236881,564,740,1601,236888,107,236829,5213,11896,11184,236888,1018,236743,18692,625,236789,236751,6290,531,735,1647,50105,866,531,2910,531,236761,107,107,11896,1531,786,1281,1144,611,236789,236753,1133,786,531,776,236888,73687,107,107,107,107],"total_duration":4879476959,"load_duration":1913893875,"prompt_eval_count":10,"prompt_eval_duration":114435209,"eval_count":157,"eval_duration":2850490458} ``` The interactive CLI keeps the spinner running throughout the load phase. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-19 17:21:00 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#24044
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?