github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-10 15:02:13 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[PR #14274] ps: report actual layer counts instead of percentage #40468

New Issue
Open
opened 2026-04-23 01:21:43 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-23 01:21:43 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/14274
Author: @ljluestc
Created: 2/16/2026
Status: 🔄 Open

Base: main ← Head: fix/ps-report-layer-counts

📝 Commits (1)

  • 5a86ebc ps: report actual layer counts instead of percentage

📊 Changes

9 files changed (+58 additions, -35 deletions)

View changed files

📝 api/types.go (+10 -8)
📝 cmd/cmd.go (+6 -11)
📝 docs/api.md (+3 -1)
📝 docs/faq.mdx (+6 -6)
📝 docs/openapi.yaml (+6 -0)
📝 llm/server.go (+10 -0)
📝 server/routes.go (+9 -7)
📝 server/sched.go (+6 -2)
📝 server/sched_test.go (+2 -0)

📄 Description

Summary

Changes ollama ps to display the number of GPU-offloaded layers out of
total layers (e.g. 33/35 GPU) instead of a percentage (e.g. 100% GPU).

This provides more actionable information when tuning the num_gpu parameter,
and makes the total layer count of a model visible at a glance.

Before

NAME        ID            SIZE    PROCESSOR         UNTIL
llama3:70b  bcfb190ca3a7  42 GB   100% GPU          4 minutes from now
llama3:8b   ab2c3d4e5f67  5 GB    48%/52% CPU/GPU   3 minutes from now

After

NAME        ID            SIZE    PROCESSOR        UNTIL
llama3:70b  bcfb190ca3a7  42 GB   81/81 GPU        4 minutes from now
llama3:8b   ab2c3d4e5f67  5 GB    16/33 CPU/GPU    3 minutes from now

Changes

  • llm/server.go: Add GPULayerCount() and TotalLayerCount() to LlamaServer interface, implemented on llmServer
  • server/sched.go: Store layer counts in runnerRef at load time
  • api/types.go: Add gpu_layers and total_layers fields to ProcessModelResponse
  • server/routes.go: Populate new fields in PsHandler
  • cmd/cmd.go: Update ListRunningHandler to display N/M GPU|CPU|CPU/GPU format
  • Docs: Updated api.md, openapi.yaml, faq.mdx
  • Tests: Updated mockLlm in sched_test.go

API Changes

The /api/ps response now includes two new fields per model:

{
  "models": [
    {
      "name": "llama3:70b",
      "size": 42000000000,
      "size_vram": 42000000000,
      "gpu_layers": 81,
      "total_layers": 81,
      "context_length": 2048
    }
  ]
}

How to Build

go build ./...

How to Test

Unit tests:

go test ./server/ -run TestSched -count=1 -timeout 30s
go test ./api/ -count=1 -timeout 30s

Manual testing:

  1. Start the server: go run . serve
  2. Load a model: go run . run llama3.2:1b --keepalive 5m
  3. Check output: go run . ps
  4. Check API: curl -s http://localhost:11434/api/ps | python3 -m json.tool

Partial offload (set num_gpu lower than total layers):

PROCESSOR
5/17 CPU/GPU

CPU-only (CUDA_VISIBLE_DEVICES=""):

PROCESSOR
17/17 CPU

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/14274 **Author:** [@ljluestc](https://github.com/ljluestc) **Created:** 2/16/2026 **Status:** 🔄 Open **Base:** `main` ← **Head:** `fix/ps-report-layer-counts` --- ### 📝 Commits (1) - [`5a86ebc`](https://github.com/ollama/ollama/commit/5a86ebc64dda249b434f9af1c356d4532a19029f) ps: report actual layer counts instead of percentage ### 📊 Changes **9 files changed** (+58 additions, -35 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+10 -8) 📝 `cmd/cmd.go` (+6 -11) 📝 `docs/api.md` (+3 -1) 📝 `docs/faq.mdx` (+6 -6) 📝 `docs/openapi.yaml` (+6 -0) 📝 `llm/server.go` (+10 -0) 📝 `server/routes.go` (+9 -7) 📝 `server/sched.go` (+6 -2) 📝 `server/sched_test.go` (+2 -0) </details> ### 📄 Description ### Summary Changes `ollama ps` to display the number of GPU-offloaded layers out of total layers (e.g. `33/35 GPU`) instead of a percentage (e.g. `100% GPU`). This provides more actionable information when tuning the `num_gpu` parameter, and makes the total layer count of a model visible at a glance. ### Before ``` NAME ID SIZE PROCESSOR UNTIL llama3:70b bcfb190ca3a7 42 GB 100% GPU 4 minutes from now llama3:8b ab2c3d4e5f67 5 GB 48%/52% CPU/GPU 3 minutes from now ``` ### After ``` NAME ID SIZE PROCESSOR UNTIL llama3:70b bcfb190ca3a7 42 GB 81/81 GPU 4 minutes from now llama3:8b ab2c3d4e5f67 5 GB 16/33 CPU/GPU 3 minutes from now ``` ### Changes - **`llm/server.go`**: Add `GPULayerCount()` and `TotalLayerCount()` to `LlamaServer` interface, implemented on `llmServer` - **`server/sched.go`**: Store layer counts in `runnerRef` at load time - **`api/types.go`**: Add `gpu_layers` and `total_layers` fields to `ProcessModelResponse` - **`server/routes.go`**: Populate new fields in `PsHandler` - **`cmd/cmd.go`**: Update `ListRunningHandler` to display `N/M GPU|CPU|CPU/GPU` format - **Docs**: Updated `api.md`, `openapi.yaml`, `faq.mdx` - **Tests**: Updated `mockLlm` in `sched_test.go` ### API Changes The `/api/ps` response now includes two new fields per model: ```json { "models": [ { "name": "llama3:70b", "size": 42000000000, "size_vram": 42000000000, "gpu_layers": 81, "total_layers": 81, "context_length": 2048 } ] } ``` ### How to Build ```bash go build ./... ``` ### How to Test **Unit tests:** ```bash go test ./server/ -run TestSched -count=1 -timeout 30s go test ./api/ -count=1 -timeout 30s ``` **Manual testing:** 1. Start the server: `go run . serve` 2. Load a model: `go run . run llama3.2:1b --keepalive 5m` 3. Check output: `go run . ps` 4. Check API: `curl -s http://localhost:11434/api/ps | python3 -m json.tool` **Partial offload** (set `num_gpu` lower than total layers): ``` PROCESSOR 5/17 CPU/GPU ``` **CPU-only** (`CUDA_VISIBLE_DEVICES=""`): ``` PROCESSOR 17/17 CPU ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-23 01:21:43 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#40468