Files
open-webui/backend
Classic298andGitHub 517f824eb4 perf: kb_exec reads the full text of the whole knowledge base on every call (#33332)
Every kb_exec command (ls, cat, grep and the rest) and the knowledge base sync check listed the knowledge base by loading each file's full extracted text, even though only names, folders, sizes and hashes were used. On large knowledge bases every call took seconds, held hundreds of MB of memory and blocked the server for other users while it ran. The listing now reads only those fields, and file text is still loaded per file when a command needs it, as before. Output is byte-identical.

5,005 files (5,000 of 50 KB in 50 folders plus 5 of 10 MB, 292 MB of text), median of 3 calls, Postgres 17:

| kb_exec command | before | after | peak memory before | after |
|---|---|---|---|---|
| `ls` | 2.34 s | 0.20 s | 874 MB | 243 MB |
| `ls folder_000/` | 2.15 s | 0.20 s | 873 MB | 243 MB |
| `cat folder_000/doc_000_001.md` | 1.07 s | 0.17 s | 855 MB | 243 MB |
| `grep -n needle folder_000/` | 1.24 s | 0.31 s | 871 MB | 243 MB |
| `grep -c needle large_0.md` | 1.19 s | 0.22 s | 874 MB | 306 MB |

On SQLite the same calls drop from 0.50-1.03 s to 0.16-0.29 s.

Fixes #33322
2026-10-29 05:26:02 +09:00
..
…
…