This fixes an issue with RAG scan that stops loading documents as soon as it reaches a file with unsupported mime type (or any other exceptions).
When the mime type is not supported, the scan operation stops with 'NoneType' object has no attribute 'find'.
Check if file_content_type is not None before searching for text/ and use TextLoader
Move try/except into the for loop so that we continue loading next files even if loading a document fails.
In case of the triage_process.graffle we fallback to TextLoader, but it seem to fail with loading, so we continue with the next document.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/866
**Author:** [@asedmammad](https://github.com/asedmammad)
**Created:** 2/23/2024
**Status:** ✅ Merged
**Merged:** 2/23/2024
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `main` ← **Head:** `fix-rag-scan-unsupported-mime`
---
### 📝 Commits (1)
- [`b473ad5`](https://github.com/open-webui/open-webui/commit/b473ad574fee48fd56a903e58614a4c7865ec7c4) fix: RAG scan unsupported mimetype
### 📊 Changes
**1 file changed** (+5 additions, -5 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/apps/rag/main.py` (+5 -5)
</details>
### 📄 Description
## Description
This fixes an issue with RAG scan that stops loading documents as soon as it reaches a file with unsupported mime type (or any other exceptions).
When the mime type is not supported, the scan operation stops with `'NoneType' object has no attribute 'find'`.
Sample unsupported mime type: [triage_process.graffle](https://github.com/django/django/blob/main/docs/internals/_images/triage_process.graffle)
## Explanation of changes
- Check if `file_content_type` is not None before searching for `text/` and use `TextLoader`
- Move try/except into the for loop so that we continue loading next files even if loading a document fails.
In case of the `triage_process.graffle` we fallback to TextLoader, but it seem to fail with loading, so we continue with the next document.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/866
Author: @asedmammad
Created: 2/23/2024
Status: ✅ Merged
Merged: 2/23/2024
Merged by: @tjbck
Base:
main← Head:fix-rag-scan-unsupported-mime📝 Commits (1)
b473ad5fix: RAG scan unsupported mimetype📊 Changes
1 file changed (+5 additions, -5 deletions)
View changed files
📝
backend/apps/rag/main.py(+5 -5)📄 Description
Description
This fixes an issue with RAG scan that stops loading documents as soon as it reaches a file with unsupported mime type (or any other exceptions).
When the mime type is not supported, the scan operation stops with
'NoneType' object has no attribute 'find'.Sample unsupported mime type: triage_process.graffle
Explanation of changes
file_content_typeis not None before searching fortext/and useTextLoaderIn case of the
triage_process.grafflewe fallback to TextLoader, but it seem to fail with loading, so we continue with the next document.🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.