[GH-ISSUE #21675] enh: analytics #58217

Open
opened 2026-05-05 22:35:33 -05:00 by GiteaMirror · 24 comments
Owner

Originally created by @tjbck on GitHub (Feb 20, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/21675

Originally created by @tjbck on GitHub (Feb 20, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/21675
Author
Owner

@KevinRossiTC commented on GitHub (Feb 27, 2026):

I would love this also. Perhaps with a dropdown for Both / UI / API.

<!-- gh-comment-id:3975194520 --> @KevinRossiTC commented on GitHub (Feb 27, 2026): I would love this also. Perhaps with a dropdown for Both / UI / API.
Author
Owner

@godlobster6 commented on GitHub (Mar 8, 2026):

Great direction. To make this actionable and contributor-friendly, I suggest locking a minimal v1 scope:

  • Metrics: request count (UI/API split), tokens in/out/total, error rate, p95 latency
  • Filters: 24h / 7d / 30d + per-model breakdown
  • Event schema: timestamp, source(ui|api), model, tokens_in, tokens_out, latency_ms, status
  • UX: Both/UI/API toggle + sortable per-model table

Suggested rollout:

  1. event schema + ingestion
  2. aggregation endpoint
  3. dashboard cards/charts
  4. docs/migration notes

If maintainers agree, I can draft a concrete PR plan (API contract + test cases + docs checklist) so contributors can pick this up with less ambiguity.

<!-- gh-comment-id:4020044508 --> @godlobster6 commented on GitHub (Mar 8, 2026): Great direction. To make this actionable and contributor-friendly, I suggest locking a minimal v1 scope: - Metrics: request count (UI/API split), tokens in/out/total, error rate, p95 latency - Filters: 24h / 7d / 30d + per-model breakdown - Event schema: timestamp, source(ui|api), model, tokens_in, tokens_out, latency_ms, status - UX: Both/UI/API toggle + sortable per-model table Suggested rollout: 1) event schema + ingestion 2) aggregation endpoint 3) dashboard cards/charts 4) docs/migration notes If maintainers agree, I can draft a concrete PR plan (API contract + test cases + docs checklist) so contributors can pick this up with less ambiguity.
Author
Owner

@zaakiy commented on GitHub (Mar 9, 2026):

...minimal v1 scope...

Not a maintainer but I'm happy with this.

Would like to stress the urgency because we don't know which users are causing the most usage ($$$).

<!-- gh-comment-id:4021036295 --> @zaakiy commented on GitHub (Mar 9, 2026): > ...minimal v1 scope... Not a maintainer but I'm happy with this. Would like to stress the urgency because we don't know which users are causing the most usage ($$$).
Author
Owner

@zaakiy commented on GitHub (Mar 11, 2026):

Correction: Would like to stress the urgency because we don't know which users API consumers are causing the most usage ($$$).

<!-- gh-comment-id:4036182020 --> @zaakiy commented on GitHub (Mar 11, 2026): Correction: Would like to stress the urgency because we don't know which ~~users~~ API consumers are causing the most usage ($$$).
Author
Owner

@Classic298 commented on GitHub (Mar 11, 2026):

If you have the urgency, you can add a filter to track their usage for now.

<!-- gh-comment-id:4037371938 --> @Classic298 commented on GitHub (Mar 11, 2026): If you have the urgency, you can add a filter to track their usage for now.
Author
Owner

@zaakiy commented on GitHub (Mar 11, 2026):

If you have the urgency, you can add a filter to track their usage for now.

I don't see a way to do this @Classic298. Could you please elaborate?

Image

I've already put the API user into a group

Image

And I can confirm that the API user is definitely within this group even though no users are showing up in the Analytics view for this group

<!-- gh-comment-id:4039339056 --> @zaakiy commented on GitHub (Mar 11, 2026): > If you have the urgency, you can add a filter to track their usage for now. I don't see a way to do this @Classic298. Could you please elaborate? <img width="1364" height="714" alt="Image" src="https://github.com/user-attachments/assets/3a49c3db-6ac1-45ed-bd4a-e37efbb85a3a" /> I've already put the API user into a group <img width="1348" height="597" alt="Image" src="https://github.com/user-attachments/assets/ad898dee-ed52-4df2-8873-6e650dcd61d7" /> And I can confirm that the API user is definitely within this group even though no users are showing up in the Analytics view for this group
Author
Owner

@Classic298 commented on GitHub (Mar 11, 2026):

@zaakiy a filter

https://docs.openwebui.com/features/extensibility/plugin/functions/filter

<!-- gh-comment-id:4039393700 --> @Classic298 commented on GitHub (Mar 11, 2026): @zaakiy a filter https://docs.openwebui.com/features/extensibility/plugin/functions/filter
Author
Owner

@zaakiy commented on GitHub (Mar 15, 2026):

That's too complex for me. Is it really that difficult to add API usage in the analytics dashboard?

In my opinion, this would have been the obvious thing to do in the first place.

<!-- gh-comment-id:4062564391 --> @zaakiy commented on GitHub (Mar 15, 2026): That's too complex for me. Is it really that difficult to add API usage in the analytics dashboard? In my opinion, this would have been the obvious thing to do in the first place.
Author
Owner

@Classic298 commented on GitHub (Mar 15, 2026):

@zaakiy

If it's too complex for you, ask AI to do it. Give it the docs and tell it you want to track all requests in the inlet() filter.
(API-only requests don't reach the outlet(), so can only track the request itself)

And yes, it is not trivial.

<!-- gh-comment-id:4062738659 --> @Classic298 commented on GitHub (Mar 15, 2026): @zaakiy If it's too complex for you, ask AI to do it. Give it the docs and tell it you want to track all requests in the inlet() filter. (API-only requests don't reach the outlet(), so can only track the request itself) And yes, it is not trivial.
Author
Owner

@SFL79 commented on GitHub (Mar 23, 2026):

Been following this issue for a while as it's an extremely important enterprise feature.
@tjbck Is this something the maintainers are planning to do in your roadmap?

Or, perhaps following @godlobster6 's suggestion, is there any agreed upon, pending plan for implementation, or are you looking for the community to go ahead and implement it how they see fit?

<!-- gh-comment-id:4110217342 --> @SFL79 commented on GitHub (Mar 23, 2026): Been following this issue for a while as it's an extremely important enterprise feature. @tjbck Is this something the maintainers are planning to do in your roadmap? Or, perhaps following @godlobster6 's suggestion, is there any agreed upon, pending plan for implementation, or are you looking for the community to go ahead and implement it how they see fit?
Author
Owner

@zaakiy commented on GitHub (Mar 24, 2026):

@Classic298 I honestly would not know where to begin how to ask AI this

<!-- gh-comment-id:4116743310 --> @zaakiy commented on GitHub (Mar 24, 2026): @Classic298 I honestly would not know where to begin how to ask AI this
Author
Owner

@Classic298 commented on GitHub (Mar 24, 2026):

https://docs.openwebui.com/features/extensibility/plugin/functions/filter

<!-- gh-comment-id:4116748591 --> @Classic298 commented on GitHub (Mar 24, 2026): https://docs.openwebui.com/features/extensibility/plugin/functions/filter
Author
Owner

@zaakiy commented on GitHub (Mar 24, 2026):

@Classic298 I don't see how you giving me the same link that you did before is going to be of any benefit.

I'm not coming here as a contributor. Just because I'm on github following this issue doesn't mean I have infinite time or infinite AI credits to work on something

<!-- gh-comment-id:4116759871 --> @zaakiy commented on GitHub (Mar 24, 2026): @Classic298 I don't see how you giving me the same link that you did before is going to be of any benefit. I'm not coming here as a contributor. Just because I'm on github following this issue doesn't mean I have infinite time or infinite AI credits to work on something
Author
Owner

@smorello87 commented on GitHub (Apr 2, 2026):

Per #23323 — one specific use case for analytics enhancements: per-user and per-group usage limits.

The chat_message table already tracks token usage. Adding an enforcement layer (configurable token/message caps per user or group, with block-on-limit) would let institutions manage costs without external proxies.

<!-- gh-comment-id:4178216278 --> @smorello87 commented on GitHub (Apr 2, 2026): Per #23323 — one specific use case for analytics enhancements: **per-user and per-group usage limits**. The `chat_message` table already tracks token usage. Adding an enforcement layer (configurable token/message caps per user or group, with block-on-limit) would let institutions manage costs without external proxies.
Author
Owner

@smorello87 commented on GitHub (Apr 3, 2026):

Following up on the per-user/group usage limits idea — we'd be interested in contributing a minimal PR for this if it aligns with your plans.

Thinking something scoped to:

  • Per-group token cap (daily/monthly) using existing chat_message usage data
  • Pre-request middleware check that blocks with a clear message when limit is reached
  • Admin settings for configuring limits per group

Would you be open to a PR along these lines, and are there any design constraints we should follow? Happy to adapt to whatever direction you're taking #21675.

<!-- gh-comment-id:4183969758 --> @smorello87 commented on GitHub (Apr 3, 2026): Following up on the per-user/group usage limits idea — we'd be interested in contributing a minimal PR for this if it aligns with your plans. Thinking something scoped to: - Per-group token cap (daily/monthly) using existing `chat_message` usage data - Pre-request middleware check that blocks with a clear message when limit is reached - Admin settings for configuring limits per group Would you be open to a PR along these lines, and are there any design constraints we should follow? Happy to adapt to whatever direction you're taking #21675.
Author
Owner

@Classic298 commented on GitHub (Apr 3, 2026):

@smorello87 how would that work when token usage data is based on currently existing chats? that'd be trivially easy to bypass by just deleting chats - or just using the temporary chat mode.

<!-- gh-comment-id:4183975798 --> @Classic298 commented on GitHub (Apr 3, 2026): @smorello87 how would that work when token usage data is based on currently existing chats? that'd be trivially easy to bypass by just deleting chats - or just using the temporary chat mode.
Author
Owner

@smorello87 commented on GitHub (Apr 3, 2026):

@Classic298 Good catch — you're right that chat_message is unsuitable for enforcement since it's deleted with chats and never written in temporary mode.

A proper implementation would decouple usage accounting from chat persistence entirely. Here's a sketch:

Separate, append-only usage ledger

New usage_ledger table — no foreign key to chat, no cascade delete, no user-facing delete API:

usage_ledger
├── id (PK)
├── user_id (indexed)
├── model_id
├── input_tokens
├── output_tokens
├── created_at (indexed)

Write path: The streaming response middleware (generate_chat_completion pipeline) already extracts token counts from every model response. After the existing ChatMessages.upsert_message() dual-write, append a row to usage_ledger unconditionally — regardless of chat mode, before any user-controlled persistence logic runs.

Enforcement path: Pre-request check in the same middleware, before the model call:

SELECT COALESCE(SUM(input_tokens + output_tokens), 0)
FROM usage_ledger
WHERE user_id = :uid AND created_at >= :period_start

Compare against the user's group limit. If exceeded, return a 429 with a clear message — the request never reaches the model.

Limit configuration: Extend the existing group model with optional token_limit and limit_period (daily/monthly) fields. Admin sets these in the existing group settings UI. Users inherit the limit from their highest-priority group (consistent with current group permission model).

This keeps the scope small (one table, two middleware touchpoints, one group schema extension) while being immune to chat deletion and temp mode since the ledger is written and checked independently of the chat lifecycle.

<!-- gh-comment-id:4183994222 --> @smorello87 commented on GitHub (Apr 3, 2026): @Classic298 Good catch — you're right that `chat_message` is unsuitable for enforcement since it's deleted with chats and never written in temporary mode. A proper implementation would decouple usage accounting from chat persistence entirely. Here's a sketch: ### Separate, append-only usage ledger New `usage_ledger` table — no foreign key to `chat`, no cascade delete, no user-facing delete API: ``` usage_ledger ├── id (PK) ├── user_id (indexed) ├── model_id ├── input_tokens ├── output_tokens ├── created_at (indexed) ``` **Write path**: The streaming response middleware (`generate_chat_completion` pipeline) already extracts token counts from every model response. After the existing `ChatMessages.upsert_message()` dual-write, append a row to `usage_ledger` unconditionally — regardless of chat mode, before any user-controlled persistence logic runs. **Enforcement path**: Pre-request check in the same middleware, before the model call: ```sql SELECT COALESCE(SUM(input_tokens + output_tokens), 0) FROM usage_ledger WHERE user_id = :uid AND created_at >= :period_start ``` Compare against the user's group limit. If exceeded, return a 429 with a clear message — the request never reaches the model. **Limit configuration**: Extend the existing group model with optional `token_limit` and `limit_period` (daily/monthly) fields. Admin sets these in the existing group settings UI. Users inherit the limit from their highest-priority group (consistent with current group permission model). This keeps the scope small (one table, two middleware touchpoints, one group schema extension) while being immune to chat deletion and temp mode since the ledger is written and checked independently of the chat lifecycle.
Author
Owner

@Classic298 commented on GitHub (Apr 3, 2026):

yeah and that is out of scope for the analytics feature. that would be rate limiting.
please open idea in discussions

btw: same could probably be achieved with a filter and a sqlite database

btw: even if you implement what you proposed you'll run into other issues like correctly tracking all requests since not all requests in open webui go through the same path

<!-- gh-comment-id:4183999857 --> @Classic298 commented on GitHub (Apr 3, 2026): yeah and that is out of scope for the analytics feature. that would be rate limiting. please open idea in discussions btw: same could probably be achieved with a filter and a sqlite database btw: even if you implement what you proposed you'll run into other issues like correctly tracking all requests since not all requests in open webui go through the same path
Author
Owner

@smorello87 commented on GitHub (Apr 3, 2026):

@Classic298 Since @tjbck directed us here from #23323, we'll wait for their input on whether this is the right place or better suited as a separate discussion.

<!-- gh-comment-id:4184015111 --> @smorello87 commented on GitHub (Apr 3, 2026): @Classic298 Since @tjbck directed us here from #23323, we'll wait for their input on whether this is the right place or better suited as a separate discussion.
Author
Owner

@trevorhayes6561-maker commented on GitHub (Apr 3, 2026):

I agree that the usage ledger approach is the right way to handle
enforcement independently of the chat lifecycle. Given the overlap with
both analytics and rate limiting, let's move this specific implementation
proposal to a new Discussion. This will allow us to finalize the schema and
middleware logic without cluttering the main analytics issue.

On Fri, Apr 3, 2026, 11:48 AM Stefano Morello @.***>
wrote:

smorello87 left a comment (open-webui/open-webui#21675)
https://github.com/open-webui/open-webui/issues/21675#issuecomment-4184015111

@Classic298 https://github.com/Classic298 Since @tjbck
https://github.com/tjbck directed us here from #23323
https://github.com/open-webui/open-webui/issues/23323, we'll wait for
their input on whether this is the right place or better suited as a
separate discussion.


Reply to this email directly, view it on GitHub
https://github.com/open-webui/open-webui/issues/21675?email_source=notifications&email_token=B7HYZ6IGGDH5I2HK3GSHMKT4T7MLRA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJYGQYDCNJRGEY2M4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-4184015111,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/B7HYZ6NKXLMRB75SU54U6W34T7MLRAVCNFSM6AAAAACV27AJPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCOBUGAYTKMJRGE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

<!-- gh-comment-id:4184382930 --> @trevorhayes6561-maker commented on GitHub (Apr 3, 2026): I agree that the usage ledger approach is the right way to handle enforcement independently of the chat lifecycle. Given the overlap with both analytics and rate limiting, let's move this specific implementation proposal to a new Discussion. This will allow us to finalize the schema and middleware logic without cluttering the main analytics issue. On Fri, Apr 3, 2026, 11:48 AM Stefano Morello ***@***.***> wrote: > *smorello87* left a comment (open-webui/open-webui#21675) > <https://github.com/open-webui/open-webui/issues/21675#issuecomment-4184015111> > > @Classic298 <https://github.com/Classic298> Since @tjbck > <https://github.com/tjbck> directed us here from #23323 > <https://github.com/open-webui/open-webui/issues/23323>, we'll wait for > their input on whether this is the right place or better suited as a > separate discussion. > > — > Reply to this email directly, view it on GitHub > <https://github.com/open-webui/open-webui/issues/21675?email_source=notifications&email_token=B7HYZ6IGGDH5I2HK3GSHMKT4T7MLRA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJYGQYDCNJRGEY2M4TFMFZW63VKON2WE43DOJUWEZLEUVSXMZLOOSWGM33PORSXEX3DNRUWG2Y#issuecomment-4184015111>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/B7HYZ6NKXLMRB75SU54U6W34T7MLRAVCNFSM6AAAAACV27AJPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCOBUGAYTKMJRGE> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> >
Author
Owner

@amirparsadd commented on GitHub (Apr 5, 2026):

Following up on the per-user/group usage limits idea — we'd be interested in contributing a minimal PR for this if it aligns with your plans.

Thinking something scoped to:

* Per-group token cap (daily/monthly) using existing `chat_message` usage data

* Pre-request middleware check that blocks with a clear message when limit is reached

* Admin settings for configuring limits per group

Would you be open to a PR along these lines, and are there any design constraints we should follow? Happy to adapt to whatever direction you're taking #21675.

are input and output tokens alone enough? one model might be 100USD and another one might be 5
maybe a usage cost based limit might be better

<!-- gh-comment-id:4188868759 --> @amirparsadd commented on GitHub (Apr 5, 2026): > Following up on the per-user/group usage limits idea — we'd be interested in contributing a minimal PR for this if it aligns with your plans. > > Thinking something scoped to: > > * Per-group token cap (daily/monthly) using existing `chat_message` usage data > > * Pre-request middleware check that blocks with a clear message when limit is reached > > * Admin settings for configuring limits per group > > > Would you be open to a PR along these lines, and are there any design constraints we should follow? Happy to adapt to whatever direction you're taking [#21675](https://github.com/open-webui/open-webui/issues/21675). are input and output tokens alone enough? one model might be 100USD and another one might be 5 maybe a usage cost based limit might be better
Author
Owner

@smorello87 commented on GitHub (Apr 5, 2026):

@amirparsadd That's a valid point. In our case we primarily serve open-weight models where the cost difference between them is relatively small, so token-based limits are sufficient. Cost-based limits would add a pricing table that needs ongoing maintenance and updates — we actually have a script that pulls pricing from OpenRouter and Bedrock APIs, so it's doable, but it's extra complexity. Could be a follow-up enhancement rather than part of the initial scope.

<!-- gh-comment-id:4188983646 --> @smorello87 commented on GitHub (Apr 5, 2026): @amirparsadd That's a valid point. In our case we primarily serve open-weight models where the cost difference between them is relatively small, so token-based limits are sufficient. Cost-based limits would add a pricing table that needs ongoing maintenance and updates — we actually have a script that pulls pricing from OpenRouter and Bedrock APIs, so it's doable, but it's extra complexity. Could be a follow-up enhancement rather than part of the initial scope.
Author
Owner

@jijunwu commented on GitHub (Apr 21, 2026):

+1 I'd like to see per-user usage dashboard where each user can see their own token consumption and API history. A leaderboard would be great too!

<!-- gh-comment-id:4287368923 --> @jijunwu commented on GitHub (Apr 21, 2026): +1 I'd like to see per-user usage dashboard where each user can see their own token consumption and API history. A leaderboard would be great too!
Author
Owner

@SamirMoustafa commented on GitHub (May 3, 2026):

@tjbck @Classic298, Quick ask: does the API_analytics_21675.md plan align with your intent for resolving this issue (UI vs API usage tracing)? Any scope or sequencing you’d want changed before I open a PR?

<!-- gh-comment-id:4366790129 --> @SamirMoustafa commented on GitHub (May 3, 2026): @tjbck @Classic298, Quick ask: does the [API_analytics_21675.md](https://github.com/user-attachments/files/27322274/API_analytics_21675.md) plan align with your intent for resolving this issue (UI vs API usage tracing)? Any scope or sequencing you’d want changed before I open a PR?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#58217