feat: set limit/cache for repetitive /models calls #5744

New Issue

2025-11-11T16:32:19-06:00

GiteaMirror commented

2025-11-11 16:32:19 -06:00

Originally created by @steveepreston on GitHub (Jul 12, 2025).

Check Existing Issues

I have searched the existing issues and discussions.

Problem Description

Problem explained here: https://github.com/open-webui/open-webui/issues/15662

Which leads to models list breakdown due 429 or another unsuccessful fetch among spam calls
and this leads to chat page breakdown and need to refresh page.

Desired Solution you'd like

please set a limit for this calls as cache, or let the call turned off, for conditions that models list is constant and user-defined

Alternatives Considered

No response

Additional Context

No response

Originally created by @steveepreston on GitHub (Jul 12, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Problem Description Problem explained here: https://github.com/open-webui/open-webui/issues/15662 Which leads to models list breakdown due 429 or another unsuccessful fetch among spam calls and this leads to chat page breakdown and need to refresh page. ### Desired Solution you'd like please set a limit for this calls as cache, or let the call turned off, for conditions that models list is constant and user-defined ### Alternatives Considered _No response_ ### Additional Context _No response_

GiteaMirror closed this issue

2025-11-11 16:32:19 -06:00

GiteaMirror commented

2025-11-11 16:32:20 -06:00

@rgaricano commented on GitHub (Jul 12, 2025):

I commented in https://github.com/open-webui/open-webui/issues/15662#issuecomment-3064937168

@rgaricano commented on GitHub (Jul 12, 2025): I commented in https://github.com/open-webui/open-webui/issues/15662#issuecomment-3064937168

GiteaMirror commented

2025-11-11 16:32:20 -06:00

@tjbck commented on GitHub (Jul 12, 2025):

Model Cache feature already exists in dev, and calling /models multiple times should NOT break the system and this has been already battle tested with deployments over 50k+ users. I'd suggest you check for your reverse proxy configuration.

@tjbck commented on GitHub (Jul 12, 2025): Model Cache feature already exists in dev, and calling `/models` multiple times should NOT break the system and this has been already battle tested with deployments over 50k+ users. I'd suggest you check for your reverse proxy configuration.

GiteaMirror commented

2025-11-11 16:32:20 -06:00

@steveepreston commented on GitHub (Jul 12, 2025):

@tjbck Thanks for note 🙏

Model Cache feature already exists in dev

will dev merged with main for next version?

calling /models multiple times should NOT break the system

agree, but it break in other way: third-party endpoint server rate limit because of calling an api multiple time at once, this leads that one of /models responses is not ok, then open-webui clears the models menu, and page need refresh to load models again after 429 state

@steveepreston commented on GitHub (Jul 12, 2025): @tjbck Thanks for note 🙏 > Model Cache feature already exists in dev will `dev` merged with main for next version? > calling /models multiple times should NOT break the system agree, but it break in other way: third-party endpoint server rate limit because of calling an api multiple time at once, this leads that one of `/models` responses is not ok, then open-webui clears the models menu, and page need refresh to load models again after 429 state

feat: set limit/cache for repetitive /models calls #5744

Check Existing Issues

Problem Description

Desired Solution you'd like

Alternatives Considered

Additional Context

feat: set limit/cache for repetitive `/models` calls #5744