when you delete a "knowledge" from the UI one would expect that the files be removed
same for when an upload fails or is duplicate
that doesn't happen in open-webui
Actual Behavior
in 24h with half a dozen knowledge stores i have:
### Total disk usage: 7.7 GiB Apparent size: 7.6 GiB Items: 9881
in vector db folder
Steps to Reproduce
click delete knowledge or upload a duplicate or fail an upload
check vector_db folder
Logs & Screenshots
2025-03-28 11:50:45.583 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document templates.md 596c0f1e-437b-4b1b-a5a0-19b3314b533d - {}
2025-03-28 11:50:45.590 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:796 - Document with hash b4f3d9c3c928cfbc654173784645ba7d2f428ed35584fe4c65b618a2c5fbad7a already exists - {}
2025-03-28 11:50:45.590 | ERROR | open_webui.routers.retrieval:process_file:1078 - Duplicate content detected. Please provide unique content to proceed. - {}
Traceback (most recent call last):
File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0xffff9f104860>
└ <WorkerThread(AnyIO worker thread, started 281466134983072)>
File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
│ └ <function WorkerThread.run at 0xfffe9770af20>
└ <WorkerThread(AnyIO worker thread, started 281466134983072)>
File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
│ │ │ └ ()
│ │ └ functools.partial(<function add_file_to_knowledge_by_id at 0xffff5e245300>, user=UserModel(id='81fc19c9-ddb2-483b-ba63-8a13a7...
│ └ <method 'run' of '_contextvars.Context' objects>
└ <_contextvars.Context object at 0xfffe9770cac0>
File "/app/backend/open_webui/routers/knowledge.py", line 291, in add_file_to_knowledge_by_id
process_file(
└ <function process_file at 0xffff5e85dee0>
> File "/app/backend/open_webui/routers/retrieval.py", line 1068, in process_file
raise e
└ ValueError(<ERROR_MESSAGES.DUPLICATE_CONTENT: 'Duplicate content detected. Please provide unique content to proceed.'>)
File "/app/backend/open_webui/routers/retrieval.py", line 1040, in process_file
result = save_docs_to_vector_db(
└ <function save_docs_to_vector_db at 0xffff5e81fba0>
File "/app/backend/open_webui/routers/retrieval.py", line 797, in save_docs_to_vector_db
raise ValueError(ERROR_MESSAGES.DUPLICATE_CONTENT)
│ └ <ERROR_MESSAGES.DUPLICATE_CONTENT: 'Duplicate content detected. Please provide unique content to proceed.'>
└ <enum 'ERROR_MESSAGES'>
ValueError: Duplicate content detected. Please provide unique content to proceed.
Check Existing Issues
I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.
Installation Method
Docker
Open WebUI Version
v0.5
Ollama Version (if applicable)
No response
Operating System
linux
Browser (if applicable)
No response
Confirmation
I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.
Originally created by @gilbrotheraway on GitHub (Mar 28, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12155
### Expected Behavior
when you delete a "knowledge" from the UI one would expect that the files be removed
same for when an upload fails or is duplicate
that doesn't happen in open-webui
### Actual Behavior
in 24h with half a dozen knowledge stores i have:
**### Total disk usage: 7.7 GiB Apparent size: 7.6 GiB Items: 9881**
in vector db folder
### Steps to Reproduce
click delete knowledge or upload a duplicate or fail an upload
check vector_db folder
### Logs & Screenshots
```
2025-03-28 11:50:45.583 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:782 - save_docs_to_vector_db: document templates.md 596c0f1e-437b-4b1b-a5a0-19b3314b533d - {}
2025-03-28 11:50:45.590 | INFO | open_webui.routers.retrieval:save_docs_to_vector_db:796 - Document with hash b4f3d9c3c928cfbc654173784645ba7d2f428ed35584fe4c65b618a2c5fbad7a already exists - {}
2025-03-28 11:50:45.590 | ERROR | open_webui.routers.retrieval:process_file:1078 - Duplicate content detected. Please provide unique content to proceed. - {}
Traceback (most recent call last):
File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
self._bootstrap_inner()
│ └ <function Thread._bootstrap_inner at 0xffff9f104860>
└ <WorkerThread(AnyIO worker thread, started 281466134983072)>
File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
self.run()
│ └ <function WorkerThread.run at 0xfffe9770af20>
└ <WorkerThread(AnyIO worker thread, started 281466134983072)>
File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
│ │ │ └ ()
│ │ └ functools.partial(<function add_file_to_knowledge_by_id at 0xffff5e245300>, user=UserModel(id='81fc19c9-ddb2-483b-ba63-8a13a7...
│ └ <method 'run' of '_contextvars.Context' objects>
└ <_contextvars.Context object at 0xfffe9770cac0>
File "/app/backend/open_webui/routers/knowledge.py", line 291, in add_file_to_knowledge_by_id
process_file(
└ <function process_file at 0xffff5e85dee0>
> File "/app/backend/open_webui/routers/retrieval.py", line 1068, in process_file
raise e
└ ValueError(<ERROR_MESSAGES.DUPLICATE_CONTENT: 'Duplicate content detected. Please provide unique content to proceed.'>)
File "/app/backend/open_webui/routers/retrieval.py", line 1040, in process_file
result = save_docs_to_vector_db(
└ <function save_docs_to_vector_db at 0xffff5e81fba0>
File "/app/backend/open_webui/routers/retrieval.py", line 797, in save_docs_to_vector_db
raise ValueError(ERROR_MESSAGES.DUPLICATE_CONTENT)
│ └ <ERROR_MESSAGES.DUPLICATE_CONTENT: 'Duplicate content detected. Please provide unique content to proceed.'>
└ <enum 'ERROR_MESSAGES'>
ValueError: Duplicate content detected. Please provide unique content to proceed.
```
### Check Existing Issues
- [x] I have searched the existing issues and discussions.
- [x] I am using the latest version of Open WebUI.
### Installation Method
Docker
### Open WebUI Version
v0.5
### Ollama Version (if applicable)
_No response_
### Operating System
linux
### Browser (if applicable)
_No response_
### Confirmation
- [x] I have read and followed all instructions in `README.md`.
- [x] I am using the latest version of **both** Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
- [x] I have listed steps to reproduce the bug in detail.
GiteaMirror
added the bug label 2026-05-20 21:13:14 -05:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @gilbrotheraway on GitHub (Mar 28, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12155
Expected Behavior
when you delete a "knowledge" from the UI one would expect that the files be removed
same for when an upload fails or is duplicate
that doesn't happen in open-webui
Actual Behavior
in 24h with half a dozen knowledge stores i have:
### Total disk usage: 7.7 GiB Apparent size: 7.6 GiB Items: 9881
in vector db folder
Steps to Reproduce
click delete knowledge or upload a duplicate or fail an upload
check vector_db folder
Logs & Screenshots
Check Existing Issues
Installation Method
Docker
Open WebUI Version
v0.5
Ollama Version (if applicable)
No response
Operating System
linux
Browser (if applicable)
No response
Confirmation
README.md.@tjbck commented on GitHub (Mar 28, 2025):
Please search and check previous discussions and issues before creating one. Intended behaviour.