github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #6537] Add metrics endpoint and basic request metrics otel based #74438

New Issue
Open
opened 2026-05-05 06:31:10 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-05 06:31:10 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/6537
Author: @amila-ku
Created: 8/27/2024
Status: 🔄 Open

Base: main ← Head: add-otel-metrics

📝 Commits (1)

  • c9c0e24 metrics: Add metrics endpoint and basic request metrics

📊 Changes

7 files changed (+324 additions, -18 deletions)

View changed files

📝 examples/go-chat/main.go (+4 -4)
📝 go.mod (+18 -4)
📝 go.sum (+41 -10)
📝 server/routes.go (+54 -0)
📝 server/routes_test.go (+38 -0)
➕ telemetry/metrics.go (+90 -0)
➕ telemetry/metrics_test.go (+79 -0)

📄 Description

Resolves https://github.com/ollama/ollama/issues/3144

This pull request is to add /metrics endpoint and http metrics as a starting point.
It uses otel metrics libary and exposes metrics in prometheus format.

This PR does not try to cover all metrics to keep it simple. If this looks good. I could add few more that will be useful.

How to test:
once Ollama server is running pull a model and list(or any other Ollama actions)

curl http://127.0.0.1:11434/metrics

example of custom metrics(not all are shown since i tried only few commands):

Ollama request metrics:

% curl http://localhost:11434/metrics | grep -i ollama
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5724    0  5724    0     0  1092k      0 --:--:-- --:--:-- --:--:-- 1117k
model_actions_total{action="list",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 1
otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1
requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 2
target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1

All metrics :

% curl http://localhost:11434/metrics
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 2.2875e-05
go_gc_duration_seconds{quantile="0.25"} 2.2875e-05
go_gc_duration_seconds{quantile="0.5"} 3.0375e-05
go_gc_duration_seconds{quantile="0.75"} 3.0375e-05
go_gc_duration_seconds{quantile="1"} 3.0375e-05
go_gc_duration_seconds_sum 5.325e-05
go_gc_duration_seconds_count 2
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 10
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.22.0"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 3.247952e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 5.271112e+06
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 11524
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 21294
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 3.159336e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 3.247952e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 1.998848e+06
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 5.7344e+06
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 18185
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 1.88416e+06
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 7.733248e+06
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.724793883369717e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 39479
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 9600
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 15600
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 137120
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 146880
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 5.69504e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 1.404724e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 655360
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 655360
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 1.3126672e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 11
# HELP list_requests_total The total number of model list requests that have been attempted.
# TYPE list_requests_total counter
list_requests_total{action="",otel_scope_name="ollama",otel_scope_version="",status="",status_code="0"} 4
# HELP otel_scope_info Instrumentation Scope metadata
# TYPE otel_scope_info gauge
otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 1
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
# HELP requests_total The total number of requests on all endpoints.
# TYPE requests_total counter
requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 3
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/6537 **Author:** [@amila-ku](https://github.com/amila-ku) **Created:** 8/27/2024 **Status:** 🔄 Open **Base:** `main` ← **Head:** `add-otel-metrics` --- ### 📝 Commits (1) - [`c9c0e24`](https://github.com/ollama/ollama/commit/c9c0e24a87e5821e1ca7234a4bfa79244a13fe84) metrics: Add metrics endpoint and basic request metrics ### 📊 Changes **7 files changed** (+324 additions, -18 deletions) <details> <summary>View changed files</summary> 📝 `examples/go-chat/main.go` (+4 -4) 📝 `go.mod` (+18 -4) 📝 `go.sum` (+41 -10) 📝 `server/routes.go` (+54 -0) 📝 `server/routes_test.go` (+38 -0) ➕ `telemetry/metrics.go` (+90 -0) ➕ `telemetry/metrics_test.go` (+79 -0) </details> ### 📄 Description Resolves https://github.com/ollama/ollama/issues/3144 This pull request is to add /metrics endpoint and http metrics as a starting point. It uses otel metrics libary and exposes metrics in prometheus format. This PR does not try to cover all metrics to keep it simple. If this looks good. I could add few more that will be useful. How to test: once Ollama server is running pull a model and list(or any other Ollama actions) ``` curl http://127.0.0.1:11434/metrics ``` example of custom metrics(not all are shown since i tried only few commands): Ollama request metrics: ``` % curl http://localhost:11434/metrics | grep -i ollama % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5724 0 5724 0 0 1092k 0 --:--:-- --:--:-- --:--:-- 1117k model_actions_total{action="list",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 1 otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1 requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 2 target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1 ``` All metrics : ``` % curl http://localhost:11434/metrics # HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles. # TYPE go_gc_duration_seconds summary go_gc_duration_seconds{quantile="0"} 2.2875e-05 go_gc_duration_seconds{quantile="0.25"} 2.2875e-05 go_gc_duration_seconds{quantile="0.5"} 3.0375e-05 go_gc_duration_seconds{quantile="0.75"} 3.0375e-05 go_gc_duration_seconds{quantile="1"} 3.0375e-05 go_gc_duration_seconds_sum 5.325e-05 go_gc_duration_seconds_count 2 # HELP go_goroutines Number of goroutines that currently exist. # TYPE go_goroutines gauge go_goroutines 10 # HELP go_info Information about the Go environment. # TYPE go_info gauge go_info{version="go1.22.0"} 1 # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. # TYPE go_memstats_alloc_bytes gauge go_memstats_alloc_bytes 3.247952e+06 # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. # TYPE go_memstats_alloc_bytes_total counter go_memstats_alloc_bytes_total 5.271112e+06 # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. # TYPE go_memstats_buck_hash_sys_bytes gauge go_memstats_buck_hash_sys_bytes 11524 # HELP go_memstats_frees_total Total number of frees. # TYPE go_memstats_frees_total counter go_memstats_frees_total 21294 # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. # TYPE go_memstats_gc_sys_bytes gauge go_memstats_gc_sys_bytes 3.159336e+06 # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. # TYPE go_memstats_heap_alloc_bytes gauge go_memstats_heap_alloc_bytes 3.247952e+06 # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. # TYPE go_memstats_heap_idle_bytes gauge go_memstats_heap_idle_bytes 1.998848e+06 # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. # TYPE go_memstats_heap_inuse_bytes gauge go_memstats_heap_inuse_bytes 5.7344e+06 # HELP go_memstats_heap_objects Number of allocated objects. # TYPE go_memstats_heap_objects gauge go_memstats_heap_objects 18185 # HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. # TYPE go_memstats_heap_released_bytes gauge go_memstats_heap_released_bytes 1.88416e+06 # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. # TYPE go_memstats_heap_sys_bytes gauge go_memstats_heap_sys_bytes 7.733248e+06 # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. # TYPE go_memstats_last_gc_time_seconds gauge go_memstats_last_gc_time_seconds 1.724793883369717e+09 # HELP go_memstats_lookups_total Total number of pointer lookups. # TYPE go_memstats_lookups_total counter go_memstats_lookups_total 0 # HELP go_memstats_mallocs_total Total number of mallocs. # TYPE go_memstats_mallocs_total counter go_memstats_mallocs_total 39479 # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. # TYPE go_memstats_mcache_inuse_bytes gauge go_memstats_mcache_inuse_bytes 9600 # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. # TYPE go_memstats_mcache_sys_bytes gauge go_memstats_mcache_sys_bytes 15600 # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. # TYPE go_memstats_mspan_inuse_bytes gauge go_memstats_mspan_inuse_bytes 137120 # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. # TYPE go_memstats_mspan_sys_bytes gauge go_memstats_mspan_sys_bytes 146880 # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. # TYPE go_memstats_next_gc_bytes gauge go_memstats_next_gc_bytes 5.69504e+06 # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. # TYPE go_memstats_other_sys_bytes gauge go_memstats_other_sys_bytes 1.404724e+06 # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. # TYPE go_memstats_stack_inuse_bytes gauge go_memstats_stack_inuse_bytes 655360 # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. # TYPE go_memstats_stack_sys_bytes gauge go_memstats_stack_sys_bytes 655360 # HELP go_memstats_sys_bytes Number of bytes obtained from system. # TYPE go_memstats_sys_bytes gauge go_memstats_sys_bytes 1.3126672e+07 # HELP go_threads Number of OS threads created. # TYPE go_threads gauge go_threads 11 # HELP list_requests_total The total number of model list requests that have been attempted. # TYPE list_requests_total counter list_requests_total{action="",otel_scope_name="ollama",otel_scope_version="",status="",status_code="0"} 4 # HELP otel_scope_info Instrumentation Scope metadata # TYPE otel_scope_info gauge otel_scope_info{otel_scope_name="ollama",otel_scope_version=""} 1 # HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served. # TYPE promhttp_metric_handler_requests_in_flight gauge promhttp_metric_handler_requests_in_flight 1 # HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code. # TYPE promhttp_metric_handler_requests_total counter promhttp_metric_handler_requests_total{code="200"} 1 promhttp_metric_handler_requests_total{code="500"} 0 promhttp_metric_handler_requests_total{code="503"} 0 # HELP requests_total The total number of requests on all endpoints. # TYPE requests_total counter requests_total{action="all",otel_scope_name="ollama",otel_scope_version="",status="OK",status_code="200"} 3 # HELP target_info Target metadata # TYPE target_info gauge target_info{service_name="unknown_service:ollama",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.27.0"} 1 ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-05 06:31:10 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#74438
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?