feat: Optional user metadata forwarding to LiteLLM for cost tracking
Description
This PR introduces an opt-in feature to forward user metadata to LiteLLM for enhanced cost tracking. A new environment variable enables the inclusion of additional user information in the payload sent to LiteLLM Proxy.
Changes
Added new environment variable ENABLE_FORWARD_USER_INFO_LITELLM_TAGS in env.py
Implemented logic to include user metadata in payloads when enabled
Modified payload structure in routers/openai.py, routers/audio.py, routers/image.py, and retrieval/utils.py
Metadata Details
When enabled, the following tags are included:
Source: openwebui
Model name
User email
User name
Implementation
The feature is opt-in, requiring explicit activation via the environment variable
Payload structure now includes a metadata object with a list of 'tags' when enabled
Testing
Conducted comprehensive testing with various user scenarios to ensure accurate metadata generation and transmission
Privacy Considerations
Feature is disabled by default to maintain user privacy
Only activates when ENABLE_FORWARD_USER_INFO_LITELLM_TAGS is set
Screenshots
Impact
This feature enhances cost tracking capabilities through LiteLLM while maintaining user privacy as an opt-in feature.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/9398
**Author:** [@elabbarw](https://github.com/elabbarw)
**Created:** 2/5/2025
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `forward_tags_litellm`
---
### 📝 Commits (8)
- [`dc3b2f1`](https://github.com/open-webui/open-webui/commit/dc3b2f1f1e60dadfe6ca22e208b0f00d36f6a0f1) Merge pull request #9190 from open-webui/dev
- [`ab94468`](https://github.com/open-webui/open-webui/commit/ab94468ffa765c9ed169cff3a7caa18e85f434db) Merge pull request #9387 from open-webui/dev
- [`e30cfa9`](https://github.com/open-webui/open-webui/commit/e30cfa9c58e87b2ca6f24dcc315c3d763d0ac29e) introduce ENABLE_FORWARD_USER_INFO_LITELLM_TAGS
- [`5eeefd8`](https://github.com/open-webui/open-webui/commit/5eeefd8cb3c0c1c611ae4e7ad791cc060c6a5022) Send tags for embeddings
- [`b0c36fc`](https://github.com/open-webui/open-webui/commit/b0c36fc472e8cd9998cd0ab5d481a02bcdc5c409) Send tags for audio
- [`806e1ce`](https://github.com/open-webui/open-webui/commit/806e1ce5094294c973265bcccfc6cd60fa46df2d) send tags for images
- [`ef45c1f`](https://github.com/open-webui/open-webui/commit/ef45c1f06661c7ee247d594b1f03fafb0ec2b6ed) send tags for chat and speech
- [`0941ed6`](https://github.com/open-webui/open-webui/commit/0941ed69aa11c6b3a02bfdc7a6fb1bd7cd68684d) remove unnecessary param
### 📊 Changes
**5 files changed** (+106 additions, -29 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/env.py` (+11 -0)
📝 `backend/open_webui/retrieval/utils.py` (+22 -20)
📝 `backend/open_webui/routers/audio.py` (+15 -1)
📝 `backend/open_webui/routers/images.py` (+8 -1)
📝 `backend/open_webui/routers/openai.py` (+50 -7)
</details>
### 📄 Description
# feat: Optional user metadata forwarding to LiteLLM for cost tracking
## Description
This PR introduces an opt-in feature to forward user metadata to LiteLLM for enhanced cost tracking. A new environment variable enables the inclusion of additional user information in the payload sent to LiteLLM Proxy.
## Changes
- Added new environment variable `ENABLE_FORWARD_USER_INFO_LITELLM_TAGS` in env.py
- Implemented logic to include user metadata in payloads when enabled
- Modified payload structure in `routers/openai.py`, `routers/audio.py`, `routers/image.py`, and `retrieval/utils.py`
## Metadata Details
When enabled, the following tags are included:
- Source: openwebui
- Model name
- User email
- User name
## Implementation
- The feature is opt-in, requiring explicit activation via the environment variable
- Payload structure now includes a `metadata` object with a list of 'tags' when enabled
## Testing
- Conducted comprehensive testing with various user scenarios to ensure accurate metadata generation and transmission
## Privacy Considerations
- Feature is disabled by default to maintain user privacy
- Only activates when `ENABLE_FORWARD_USER_INFO_LITELLM_TAGS` is set
## Screenshots
## Impact
This feature enhances cost tracking capabilities through LiteLLM while maintaining user privacy as an opt-in feature.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/9398
Author: @elabbarw
Created: 2/5/2025
Status: ❌ Closed
Base:
dev← Head:forward_tags_litellm📝 Commits (8)
dc3b2f1Merge pull request #9190 from open-webui/devab94468Merge pull request #9387 from open-webui/deve30cfa9introduce ENABLE_FORWARD_USER_INFO_LITELLM_TAGS5eeefd8Send tags for embeddingsb0c36fcSend tags for audio806e1cesend tags for imagesef45c1fsend tags for chat and speech0941ed6remove unnecessary param📊 Changes
5 files changed (+106 additions, -29 deletions)
View changed files
📝
backend/open_webui/env.py(+11 -0)📝
backend/open_webui/retrieval/utils.py(+22 -20)📝
backend/open_webui/routers/audio.py(+15 -1)📝
backend/open_webui/routers/images.py(+8 -1)📝
backend/open_webui/routers/openai.py(+50 -7)📄 Description
feat: Optional user metadata forwarding to LiteLLM for cost tracking
Description
This PR introduces an opt-in feature to forward user metadata to LiteLLM for enhanced cost tracking. A new environment variable enables the inclusion of additional user information in the payload sent to LiteLLM Proxy.
Changes
ENABLE_FORWARD_USER_INFO_LITELLM_TAGSin env.pyrouters/openai.py,routers/audio.py,routers/image.py, andretrieval/utils.pyMetadata Details
When enabled, the following tags are included:
Implementation
metadataobject with a list of 'tags' when enabledTesting
Privacy Considerations
ENABLE_FORWARD_USER_INFO_LITELLM_TAGSis setScreenshots
Impact
This feature enhances cost tracking capabilities through LiteLLM while maintaining user privacy as an opt-in feature.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.