Description: Briefly describe the changes in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Description
This introduces a load balancer scheme in ollama app of the backend, resolving issue #1081. The implementation allows setting different strategies in the .env file, with support for RoundRobin and WeightedRoundRobin currently available.
Changelog Entry
Added
Implemented load balancer scheme for ollama backend
Added RoundRobin and WeightedRoundRobin strategies
Configured strategies in .env.example file for easy customization
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/1276
**Author:** [@asedmammad](https://github.com/asedmammad)
**Created:** 3/24/2024
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `feat/ollama-load-balance`
---
### 📝 Commits (6)
- [`bcf772a`](https://github.com/open-webui/open-webui/commit/bcf772a5b6f53dacdd15c7353f9b1b141b81e8b2) WIP: Add load balancing to backend ollama app
- [`b05f37a`](https://github.com/open-webui/open-webui/commit/b05f37a77f958e07898eb706ee7bead972dfb70a) WIP: Add UI to select load balancer policy
- [`04f70f6`](https://github.com/open-webui/open-webui/commit/04f70f6b8ab96ea05311594c731d2b7a771fd8ce) feat(lb): Get ollama lb values from the backend
- [`055f97c`](https://github.com/open-webui/open-webui/commit/055f97c119b05b95470316c94a3bb7eadf897ab7) Merge branch 'dev' into feat/ollama-load-balance
- [`f816fc0`](https://github.com/open-webui/open-webui/commit/f816fc0c39ff36f4caa8dddbcdd78513b43827f8) fix: Formatting
- [`4c92b69`](https://github.com/open-webui/open-webui/commit/4c92b69257fbdc33ca9225e43aa729987d95fcb9) fix: Rename get load balancer function
### 📊 Changes
**7 files changed** (+270 additions, -19 deletions)
<details>
<summary>View changed files</summary>
📝 `.env.example` (+3 -1)
➕ `backend/apps/ollama/load_balancer.py` (+73 -0)
📝 `backend/apps/ollama/main.py` (+63 -11)
📝 `backend/config.py` (+8 -0)
📝 `backend/dev.sh` (+0 -0)
📝 `src/lib/apis/ollama/index.ts` (+68 -0)
📝 `src/lib/components/chat/Settings/Connections.svelte` (+55 -7)
</details>
### 📄 Description
## Pull Request Checklist
- [X] **Description:** Briefly describe the changes in this pull request.
- [X] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** Have you updated relevant documentation?
- [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
---
## Description
This introduces a load balancer scheme in ollama app of the backend, resolving issue #1081. The implementation allows setting different strategies in the `.env` file, with support for RoundRobin and WeightedRoundRobin currently available.
---
### Changelog Entry
### Added
- Implemented load balancer scheme for ollama backend
- Added RoundRobin and WeightedRoundRobin strategies
- Configured strategies in `.env.example` file for easy customization
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/1276
Author: @asedmammad
Created: 3/24/2024
Status: ❌ Closed
Base:
dev← Head:feat/ollama-load-balance📝 Commits (6)
bcf772aWIP: Add load balancing to backend ollama appb05f37aWIP: Add UI to select load balancer policy04f70f6feat(lb): Get ollama lb values from the backend055f97cMerge branch 'dev' into feat/ollama-load-balancef816fc0fix: Formatting4c92b69fix: Rename get load balancer function📊 Changes
7 files changed (+270 additions, -19 deletions)
View changed files
📝
.env.example(+3 -1)➕
backend/apps/ollama/load_balancer.py(+73 -0)📝
backend/apps/ollama/main.py(+63 -11)📝
backend/config.py(+8 -0)📝
backend/dev.sh(+0 -0)📝
src/lib/apis/ollama/index.ts(+68 -0)📝
src/lib/components/chat/Settings/Connections.svelte(+55 -7)📄 Description
Pull Request Checklist
Description
This introduces a load balancer scheme in ollama app of the backend, resolving issue #1081. The implementation allows setting different strategies in the
.envfile, with support for RoundRobin and WeightedRoundRobin currently available.Changelog Entry
Added
Implemented load balancer scheme for ollama backend
Added RoundRobin and WeightedRoundRobin strategies
Configured strategies in
.env.examplefile for easy customization🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.