[GH-ISSUE #23012] feat: Support random cache_salt parameter in extra_body for cache isolation in each unique user #35395

New Issue

2026-04-25T09:36:29-05:00

GiteaMirror commented

2026-04-25 09:36:29 -05:00

Originally created by @ehfd on GitHub (Mar 25, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/23012

Check Existing Issues

I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Verify Feature Scope

I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.

Problem Description

In vLLM or SGLang, there is an option to support cache_salt, which allows secure Prefix Caching isolation for each user (https://github.com/vllm-project/vllm/issues/16016, https://github.com/vllm-project/vllm/pull/17045, https://github.com/sgl-project/sglang/issues/9163, https://github.com/sgl-project/sglang/pull/10317).

Since in many cases, the LLM API Token for the backend (such as vLLM or SGLang) is shared in web interfaces, this operation should be performed from the frontend-side.

Desired Solution you'd like

Add the below randomly generated key value for each user (the base64-encoded value should be from a salted hash unique to the user):

Example (base64-encoded from abcdefghijklmnopqrstuvwxyzabcdefghijklmnopq): extra_body={"cache_salt": "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXphYmNkZWZnaGlqa2xtbm9wcQ=="}

response = client.chat.completions.create(
    model=model,
    messages=messages,
    extra_body={
        "cache_salt": "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXphYmNkZWZnaGlqa2xtbm9wcQ==",
    },
)

Alternatives Considered

This needs to be implemented where individual users are managed.

Additional Context

No response

Originally created by @ehfd on GitHub (Mar 25, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/23012 ### Check Existing Issues - [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request. ### Verify Feature Scope - [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions. ### Problem Description In vLLM or SGLang, there is an option to support `cache_salt`, which allows secure Prefix Caching isolation for each user (https://github.com/vllm-project/vllm/issues/16016, https://github.com/vllm-project/vllm/pull/17045, https://github.com/sgl-project/sglang/issues/9163, https://github.com/sgl-project/sglang/pull/10317). Since in many cases, the LLM API Token for the backend (such as vLLM or SGLang) is shared in web interfaces, this operation should be performed from the frontend-side. ### Desired Solution you'd like Add the below randomly generated key value for each user (the base64-encoded value should be from a salted hash unique to the user): **Example** (base64-encoded from `abcdefghijklmnopqrstuvwxyzabcdefghijklmnopq`): `extra_body={"cache_salt": "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXphYmNkZWZnaGlqa2xtbm9wcQ=="}` ```python response = client.chat.completions.create( model=model, messages=messages, extra_body={ "cache_salt": "YWJjZGVmZ2hpamtsbW5vcHFyc3R1dnd4eXphYmNkZWZnaGlqa2xtbm9wcQ==", }, ) ``` ### Alternatives Considered This needs to be implemented where individual users are managed. ### Additional Context _No response_

GiteaMirror closed this issue

2026-04-25 09:36:29 -05:00

GiteaMirror commented

2026-04-25 09:36:30 -05:00

@tjbck commented on GitHub (Mar 25, 2026):

You can use filter function for this.

@tjbck commented on GitHub (Mar 25, 2026): You can use filter function for this.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#35395

[GH-ISSUE #23012] feat: Support random cache_salt parameter in extra_body for cache isolation in each unique user #35395