Performance issue #4035 has additional details.
This PR optimizes the query made to the database by selecting only the required columns.
The images which are stored in the database chat history make the slow down apparent when querying that endpoint, which happens on every reload.
This change will allow for significantly improved performance with large database sizes.
This PR doesn't solve the root issue, as the issue is caused by excessive database size.
Added
a function, similar to the existing database query function used in the /chats endpoint, which only queries the fields it needs
Changed
changed /chats and /chats/list endpoints to use the new function.
the function also had a bug which has now been resolved: skip: int was passed as a positional argument to the function parameter include_archived: bool
# argument #2 should not be "skip"returnChats.get_chat_list_by_user_id(user.id,skip,limit)
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/4046
**Author:** [@thearyadev](https://github.com/thearyadev)
**Created:** 7/22/2024
**Status:** ✅ Merged
**Merged:** 7/24/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `chat-list-optimization`
---
### 📝 Commits (3)
- [`2b78e61`](https://github.com/open-webui/open-webui/commit/2b78e613a490f86425899bc145adb6b17dc607e5) add func to get chat list with more specific sql query
- [`a0667df`](https://github.com/open-webui/open-webui/commit/a0667dfd1b095900a1400f953a99b461cc82baa5) change `/chats/` and `/chats/list` to utilize new function
- [`f531a51`](https://github.com/open-webui/open-webui/commit/f531a51e91e83aadb4c862215fbb40b158077c0f) chore: formatting
### 📊 Changes
**2 files changed** (+35 additions, -1 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/apps/webui/models/chats.py` (+34 -0)
📝 `backend/apps/webui/routers/chats.py` (+1 -1)
</details>
### 📄 Description
# Changelog Entry
## Description
Performance issue #4035 has additional details.
This PR optimizes the query made to the database by selecting only the required columns.
The images which are stored in the database chat history make the slow down apparent when querying that endpoint, which happens on every reload.
This change will allow for significantly improved performance with large database sizes.
This PR doesn't solve the root issue, as the issue is caused by excessive database size.
## Added
- a function, similar to the existing database query function used in the `/chats` endpoint, which only queries the fields it needs
## Changed
- changed `/chats` and `/chats/list` endpoints to use the new function.
- the function also had a bug which has now been resolved:
`skip: int` was passed as a positional argument to the function parameter `include_archived: bool`
```python
# argument #2 should not be "skip"
return Chats.get_chat_list_by_user_id(user.id, skip, limit)
```
changed to
```python
return Chats.get_chat_title_id_list_by_user_id(user.id, skip=skip, limit=limit)
```
using keyword arguments to safely pass the params.
## Screenshots or Videos
**test done with 500mb database, artificially bloated with images**
before optimization
https://github.com/user-attachments/assets/16419820-adbe-4b8b-85ec-391dbfdfb129
after optimization
https://github.com/user-attachments/assets/4f0bdd5b-7ab1-438e-bff4-64fa5d530e0c
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/4046
Author: @thearyadev
Created: 7/22/2024
Status: ✅ Merged
Merged: 7/24/2024
Merged by: @tjbck
Base:
dev← Head:chat-list-optimization📝 Commits (3)
2b78e61add func to get chat list with more specific sql querya0667dfchange/chats/and/chats/listto utilize new functionf531a51chore: formatting📊 Changes
2 files changed (+35 additions, -1 deletions)
View changed files
📝
backend/apps/webui/models/chats.py(+34 -0)📝
backend/apps/webui/routers/chats.py(+1 -1)📄 Description
Changelog Entry
Description
Performance issue #4035 has additional details.
This PR optimizes the query made to the database by selecting only the required columns.
The images which are stored in the database chat history make the slow down apparent when querying that endpoint, which happens on every reload.
This change will allow for significantly improved performance with large database sizes.
This PR doesn't solve the root issue, as the issue is caused by excessive database size.
Added
/chatsendpoint, which only queries the fields it needsChanged
/chatsand/chats/listendpoints to use the new function.skip: intwas passed as a positional argument to the function parameterinclude_archived: boolchanged to
using keyword arguments to safely pass the params.
Screenshots or Videos
test done with 500mb database, artificially bloated with images
before optimization
https://github.com/user-attachments/assets/16419820-adbe-4b8b-85ec-391dbfdfb129
after optimization
https://github.com/user-attachments/assets/4f0bdd5b-7ab1-438e-bff4-64fa5d530e0c
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.