Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:
Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Note: I couldn't see any tests for num_ctx etc... so I assume this isn't required for adding these two params to the frontend
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix: To cleary categorize this pull request, prefix the pull request title
Changelog Entry
Description
Added two parameters that are configurable in Ollama (Modelfiles/API/Clients).
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/3161
**Author:** [@sammcj](https://github.com/sammcj)
**Created:** 6/14/2024
**Status:** ✅ Merged
**Merged:** 6/14/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `num_batch`
---
### 📝 Commits (1)
- [`3f5e362`](https://github.com/open-webui/open-webui/commit/3f5e36271f6129f77e0d9c3df2ed2c7643a19bd4) feat: add num_keep, num_batch
### 📊 Changes
**41 files changed** (+182 additions, -0 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/apps/ollama/main.py` (+8 -0)
📝 `src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte` (+94 -0)
📝 `src/lib/components/chat/Settings/General.svelte` (+4 -0)
📝 `src/lib/i18n/locales/ar-BH/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/bg-BG/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/bn-BD/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/ca-ES/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/ceb-PH/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/de-DE/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/dg-DG/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/en-GB/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/en-US/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/es-ES/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/fa-IR/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/fi-FI/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/fr-CA/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/fr-FR/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/he-IL/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/hi-IN/translation.json` (+2 -0)
📝 `src/lib/i18n/locales/hr-HR/translation.json` (+2 -0)
_...and 21 more files_
</details>
### 📄 Description
# Pull Request Checklist
### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) and describe your changes before submitting a pull request.
**Before submitting, make sure you've checked the following:**
- [x] **Target branch:** Please verify that the pull request targets the `dev` branch.
- [x] **Description:** Provide a concise description of the changes made in this pull request.
- [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [x] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources?
- [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [ ] **Testing:** Have you written and run sufficient tests for validating the changes?
- _Note: I couldn't see any tests for num_ctx etc... so I assume this isn't required for adding these two params to the frontend_
- [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [x] **Prefix:** To cleary categorize this pull request, prefix the pull request title
# Changelog Entry
### Description
Added two parameters that are configurable in Ollama (Modelfiles/API/Clients).
Re: https://github.com/open-webui/open-webui/issues/3148
### Added
- Add parameter for `num_batch`
- Add parameter for `num_keep`
### Fixed
- https://github.com/open-webui/open-webui/issues/3148
---
### Additional Information
- Reference
- batch size: https://github.com/ggerganov/llama.cpp/blob/master/examples/batched-bench/README.md
- number of tokens to keep on context refresh: https://github.com/ggerganov/llama.cpp/blob/master/examples/main/main.cpp#L330
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/3161
Author: @sammcj
Created: 6/14/2024
Status: ✅ Merged
Merged: 6/14/2024
Merged by: @tjbck
Base:
dev← Head:num_batch📝 Commits (1)
3f5e362feat: add num_keep, num_batch📊 Changes
41 files changed (+182 additions, -0 deletions)
View changed files
📝
backend/apps/ollama/main.py(+8 -0)📝
src/lib/components/chat/Settings/Advanced/AdvancedParams.svelte(+94 -0)📝
src/lib/components/chat/Settings/General.svelte(+4 -0)📝
src/lib/i18n/locales/ar-BH/translation.json(+2 -0)📝
src/lib/i18n/locales/bg-BG/translation.json(+2 -0)📝
src/lib/i18n/locales/bn-BD/translation.json(+2 -0)📝
src/lib/i18n/locales/ca-ES/translation.json(+2 -0)📝
src/lib/i18n/locales/ceb-PH/translation.json(+2 -0)📝
src/lib/i18n/locales/de-DE/translation.json(+2 -0)📝
src/lib/i18n/locales/dg-DG/translation.json(+2 -0)📝
src/lib/i18n/locales/en-GB/translation.json(+2 -0)📝
src/lib/i18n/locales/en-US/translation.json(+2 -0)📝
src/lib/i18n/locales/es-ES/translation.json(+2 -0)📝
src/lib/i18n/locales/fa-IR/translation.json(+2 -0)📝
src/lib/i18n/locales/fi-FI/translation.json(+2 -0)📝
src/lib/i18n/locales/fr-CA/translation.json(+2 -0)📝
src/lib/i18n/locales/fr-FR/translation.json(+2 -0)📝
src/lib/i18n/locales/he-IL/translation.json(+2 -0)📝
src/lib/i18n/locales/hi-IN/translation.json(+2 -0)📝
src/lib/i18n/locales/hr-HR/translation.json(+2 -0)...and 21 more files
📄 Description
Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:
devbranch.Changelog Entry
Description
Added two parameters that are configurable in Ollama (Modelfiles/API/Clients).
Re: https://github.com/open-webui/open-webui/issues/3148
Added
num_batchnum_keepFixed
Additional Information
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.