I have searched the existing issues and discussions.
Problem Description
The new Content Extraction Engine with docling serve is amazing. But it's also very slow. For creating a permanent knowledge base (via workspace -> Knowledge) this is not a problem. As it's a one-off task.
But for uploading (larger) files to an ad-hoc chat, docling serve is just too slow.
Desired Solution you'd like
There are two ideas in my mind. Either:
One can set the default Content Extraction Engine in Admin -> Settings (Just as is already implement). When uploading a file in the permanent knowledge base, we get another screen where we can optionally select a different content extraction engine.
Alternatively, there are two different Settings in Admin -> Settings -> Documents. One for the Chat - RAG (where you upload files to the chat) and one for the permanent knowledge base.
Alternatives Considered
No response
Additional Context
No response
Originally created by @andnig on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12619
### Check Existing Issues
- [x] I have searched the existing issues and discussions.
### Problem Description
The new Content Extraction Engine with docling serve is amazing. But it's also very slow. For creating a permanent knowledge base (via workspace -> Knowledge) this is not a problem. As it's a one-off task.
But for uploading (larger) files to an ad-hoc chat, docling serve is just too slow.
### Desired Solution you'd like
There are two ideas in my mind. Either:
1. One can set the default Content Extraction Engine in Admin -> Settings (Just as is already implement). When uploading a file in the permanent knowledge base, we get another screen where we can optionally select a different content extraction engine.
2. Alternatively, there are two different Settings in Admin -> Settings -> Documents. One for the Chat - RAG (where you upload files to the chat) and one for the permanent knowledge base.
### Alternatives Considered
_No response_
### Additional Context
_No response_
This is an excellent suggestion. I'd like to expand on it by proposing separate settings for:
Chat uploaded files: optimized for individual document processing with Full Context Mode
Knowledge base collections: optimized for traditional RAG with chunking
Currently, the "Full Context Mode" in Admin → Settings → Documents settings applies globally, which creates challenges. Even with large context window models, using full context mode on extensive knowledge collections often results in context window limitation errors.
Having separate controls for these different use cases would help optimize both:
Accuracy: allowing full context processing for individual chat documents where appropriate
Performance/Cost: enabling proper chunking for large collections where needed
#11227 (Discussion about Full Context Mode RAG on Knowledge-Base vs Document-Upload)
#11065 (Proposal for conditional parameter for Full Context Document Mode)
<!-- gh-comment-id:2801559854 -->
@akhilpotdar1 commented on GitHub (Apr 14, 2025):
This is an excellent suggestion. I'd like to expand on it by proposing separate settings for:
- Chat uploaded files: optimized for individual document processing with Full Context Mode
- Knowledge base collections: optimized for traditional RAG with chunking
Currently, the "Full Context Mode" in Admin → Settings → Documents settings applies globally, which creates challenges. Even with large context window models, using full context mode on extensive knowledge collections often results in context window limitation errors.
Having separate controls for these different use cases would help optimize both:
- Accuracy: allowing full context processing for individual chat documents where appropriate
- Performance/Cost: enabling proper chunking for large collections where needed
Related issues/discussions:
- #12459 (Full context mode per-user, per-chat)
- #11227 (Discussion about Full Context Mode RAG on Knowledge-Base vs Document-Upload)
- #11065 (Proposal for conditional parameter for Full Context Document Mode)
I tried to play with the basecode and if we want to emulate full context for document uploaded in chat (with the "+" button) and, at the same time, chat with collections selected by "#" command using RAG we could do something like that.
let enableFullContent = true;
$: isPDF = item?.meta?.content_type === 'application/pdf' ||
(item?.name && item?.name.toLowerCase().endsWith('.pdf'));
onMount(() => {
// Explicitly set to true if context is 'full'
// Set to false if context has any other defined value
// Keep default (true) if context is undefined
if (item?.context === 'full') {
enableFullContent = true;
} else if (item?.context !== undefined) {
enableFullContent = false;
}
});
This implementation preserves full context for uploaded documents (as default behaviour) while properly handling RAG collections referenced with the "#" command.
For uploaded chat documents we could use RAG by switching off the fullContext toggle
Does this implementation make technical sense? Are there any potential issues or improvements I should consider?
<!-- gh-comment-id:2821150859 -->
@sir3mat commented on GitHub (Apr 22, 2025):
I tried to play with the basecode and if we want to emulate full context for document uploaded in chat (with the "+" button) and, at the same time, chat with collections selected by "#" command using RAG we could do something like that.
Set fullContext: true in the uploadFileHandler() function call at:
https://github.com/open-webui/open-webui/blob/852d9dcbe9b4406e1da230a14642f35ac7bb0d2b/src/lib/components/chat/MessageInput.svelte#L281
Modify the [FileItemModal.svelte]( https://github.com/open-webui/open-webui/blob/852d9dcbe9b4406e1da230a14642f35ac7bb0d2b/src/lib/components/common/FileItemModal.svelte#L18) with this logic
```
let enableFullContent = true;
$: isPDF = item?.meta?.content_type === 'application/pdf' ||
(item?.name && item?.name.toLowerCase().endsWith('.pdf'));
onMount(() => {
// Explicitly set to true if context is 'full'
// Set to false if context has any other defined value
// Keep default (true) if context is undefined
if (item?.context === 'full') {
enableFullContent = true;
} else if (item?.context !== undefined) {
enableFullContent = false;
}
});
```
This implementation preserves full context for uploaded documents (as default behaviour) while properly handling RAG collections referenced with the "#" command.
For uploaded chat documents we could use RAG by switching off the fullContext toggle
Does this implementation make technical sense? Are there any potential issues or improvements I should consider?
I would also recommend adjusting the top_k setting for web search. The chunks generated with Playwright from SearxNG web results often have poor formatting and low-quality text content. As a result, the similarity scores during RAG-based web search are very low (often below 0.15%), which sometimes leads to zero chunks being passed to the model for answer generation.
<!-- gh-comment-id:2826644426 -->
@sir3mat commented on GitHub (Apr 24, 2025):
I would also recommend adjusting the top_k setting for web search. The chunks generated with Playwright from SearxNG web results often have poor formatting and low-quality text content. As a result, the similarity scores during RAG-based web search are very low (often below 0.15%), which sometimes leads to zero chunks being passed to the model for answer generation.
@sir3mat This is a great solution for handling chat uploads differently from knowledge base content.
Building on this, if we could extend the FileItemModal.svelte logic to handle knowledge base collections, it would help address the per-user, per-chat control request in #12459. Perhaps we could:
Add a similar toggle in the knowledge base UI for individual documents/collections
Store these preferences in the same way your implementation handles chat upload preferences
However your current solution works well as a standalone improvement too.
<!-- gh-comment-id:2845003732 -->
@akhilpotdar1 commented on GitHub (May 1, 2025):
@sir3mat This is a great solution for handling chat uploads differently from knowledge base content.
Building on this, if we could extend the `FileItemModal.svelte` logic to handle knowledge base collections, it would help address the per-user, per-chat control request in #12459. Perhaps we could:
1. Add a similar toggle in the knowledge base UI for individual documents/collections
2. Store these preferences in the same way your implementation handles chat upload preferences
However your current solution works well as a standalone improvement too.
Is "Content Extraction Engine" only relevant when building a knowledge base to be used with RAG? I can barely find any documentation explaining this.
Say if I just simply upload a PDF document to a local AI model, this "Content Extraction Engine" won't be used, right? Then who is doing the content extraction job? I'm so confused.
I followed gpt's suggestion and use pymupdf and Uvicorn to run a service on a port to extract text from PDF ("external" is chosen in settings). I then built a knowledge base with a few PDF files and then create a test model from it. When I interact with this model, the service was never called. Who is doing the content extraction job here? I'm again so confused.
Also it seems whether the option "Bypass Embedding and Retrieval" is on or off does not affect the observations above.
Any suggestions or more insights will be greatly appreciated!
<!-- gh-comment-id:2915395773 -->
@astroboylrx commented on GitHub (May 28, 2025):
Is "Content Extraction Engine" only relevant when building a knowledge base to be used with RAG? I can barely find any documentation explaining this.
Say if I just simply upload a PDF document to a local AI model, this "Content Extraction Engine" won't be used, right? Then who is doing the content extraction job? I'm so confused.
I followed gpt's suggestion and use `pymupdf` and `Uvicorn` to run a service on a port to extract text from PDF ("external" is chosen in settings). I then built a knowledge base with a few PDF files and then create a test model from it. When I interact with this model, the service was never called. Who is doing the content extraction job here? I'm again so confused.
Also it seems whether the option "Bypass Embedding and Retrieval" is on or off does not affect the observations above.
Any suggestions or more insights will be greatly appreciated!
@astroboylrx content extraction engine is used for knoledge base, single files upload on chat and web search results
<!-- gh-comment-id:2915773707 -->
@sir3mat commented on GitHub (May 28, 2025):
@astroboylrx content extraction engine is used for knoledge base, single files upload on chat and web search results
I provided an external endpoint for pdf-to-text in the settings and enabled "Bypass Embedding and Retrieval". However webui seems to ignore anyway my settings when I throw a PDF into chat. The detailed logging suggested it just parse the PDF byitself anyway. Is this expected behaviors?
@astroboylrx how did you implement your endpoint? Does it follow the input output requirements?
Pls also attach the documents page settings
<!-- gh-comment-id:2917321954 -->
@sir3mat commented on GitHub (May 28, 2025):
@astroboylrx how did you implement your endpoint? Does it follow the input output requirements?
Pls also attach the documents page settings
@astroboylrx how did you implement your endpoint? Does it follow the input output requirements?
Pls also attach the documents page settings
@sir3mat Thank you again for the response!
Sorry, where can I find this "input output requirements"?
The simplest thing that I got from GPT's suggestions is a loader.py that get text out of a PDF (somehow it works on some PDF---I mean when I directly use it, not on webui---whereas the default one failed to extract text according to webui's log; an example can be found here, which is an research article):
# loader.pyfromfastapiimportFastAPI,HTTPException,QueryfrompydanticimportBaseModelimportfitz# PyMuPDFfromPILimportImageimportpytesseractimportioapp=FastAPI(title="PyMuPDF + OCR External Loader")classDocument(BaseModel):pages:list[str]@app.get("/load_document",response_model=Document)defload_document(filepath:str=Query(...,description="Absolute path to the PDF file"),ocr_threshold:int=Query(20,description="If extracted text length < this, run OCR"),dpi:int=Query(300,description="DPI for page rasterization before OCR")):"""
Example:
GET /load_document?filepath=/home/user/doc.pdf
&ocr_threshold=20&dpi=300
Returns:
{ "pages": ["text page 1", "text page 2", …] }
"""try:doc=fitz.open(filepath)exceptExceptionase:raiseHTTPException(status_code=400,detail=f"Cannot open PDF: {e}")pages_text:list[str]=[]forpageindoc:# 1) Extract embedded texttext=page.get_text("text").strip()# 2) If text is too short, rasterize + OCRiflen(text)<ocr_threshold:try:pix=page.get_pixmap(dpi=dpi)img=Image.open(io.BytesIO(pix.tobytes()))ocr_result=pytesseract.image_to_string(img,lang="eng+chi_sim")# Merge and de-duplicatecombined="\n".join(filter(None,{text,ocr_result.strip()}))text=combinedifcombinedelsetextexceptExceptionase:text=textorf"[OCR failed: {e}]"pages_text.append(text)returnDocument(pages=pages_text)
And then start the service with uvicorn loader:app --host 0.0.0.0 --port 8083.
In the documents page settings I have:
Maybe I totally misunderstood how it works?
<!-- gh-comment-id:2917598493 -->
@astroboylrx commented on GitHub (May 28, 2025):
> [@astroboylrx](https://github.com/astroboylrx) how did you implement your endpoint? Does it follow the input output requirements?
>
> Pls also attach the documents page settings
@sir3mat Thank you again for the response!
Sorry, where can I find this "input output requirements"?
The simplest thing that I got from GPT's suggestions is a `loader.py` that get text out of a PDF (somehow it works on some PDF---I mean when I directly use it, not on webui---whereas the default one failed to extract text according to webui's log; an example can be found [here](https://www.rixinli.com/misc/s41550-023-01945-7.pdf), which is an research article):
```python
# loader.py
from fastapi import FastAPI, HTTPException, Query
from pydantic import BaseModel
import fitz # PyMuPDF
from PIL import Image
import pytesseract
import io
app = FastAPI(title="PyMuPDF + OCR External Loader")
class Document(BaseModel):
pages: list[str]
@app.get("/load_document", response_model=Document)
def load_document(
filepath: str = Query(..., description="Absolute path to the PDF file"),
ocr_threshold: int = Query(20, description="If extracted text length < this, run OCR"),
dpi: int = Query(300, description="DPI for page rasterization before OCR")
):
"""
Example:
GET /load_document?filepath=/home/user/doc.pdf
&ocr_threshold=20&dpi=300
Returns:
{ "pages": ["text page 1", "text page 2", …] }
"""
try:
doc = fitz.open(filepath)
except Exception as e:
raise HTTPException(status_code=400, detail=f"Cannot open PDF: {e}")
pages_text: list[str] = []
for page in doc:
# 1) Extract embedded text
text = page.get_text("text").strip()
# 2) If text is too short, rasterize + OCR
if len(text) < ocr_threshold:
try:
pix = page.get_pixmap(dpi=dpi)
img = Image.open(io.BytesIO(pix.tobytes()))
ocr_result = pytesseract.image_to_string(img, lang="eng+chi_sim")
# Merge and de-duplicate
combined = "\n".join(filter(None, {text, ocr_result.strip()}))
text = combined if combined else text
except Exception as e:
text = text or f"[OCR failed: {e}]"
pages_text.append(text)
return Document(pages=pages_text)
```
And then start the service with `uvicorn loader:app --host 0.0.0.0 --port 8083`.
In the documents page settings I have:
<img width="950" alt="Image" src="https://github.com/user-attachments/assets/84345475-589d-4fa3-b7f2-e9fc86117fb3" />
Maybe I totally misunderstood how it works?
endpoint have to be: /process
input: data & headers (in headers we have "Authorization" & "Content-Type" of data)
output: json with page content (string text) & metadata (optional, Arbitrary metadata associated with the content)
<!-- gh-comment-id:2917727309 -->
@rgaricano commented on GitHub (May 28, 2025):
you can find Input/output requirements on https://github.com/open-webui/open-webui/blob/main/backend/open_webui/retrieval/loaders/external_document.py
endpoint have to be: /process
input: data & headers (in headers we have "Authorization" & "Content-Type" of data)
output: json with page content (string text) & metadata (optional, Arbitrary metadata associated with the content)
Sorry I'm not very familir with Web-related coding. I asked GPT to adjust my loader.py based on the external_document.py and now it looks like:
fromfastapiimportFastAPI,Request,Header,HTTPExceptionfrompydanticimportBaseModelimportfitz# PyMuPDFfromPILimportImageimportpytesseractimportioapp=FastAPI(title="PyMuPDF + OCR External Loader")print("🔌 loader.py imported – FastAPI app created")classLoaderResponse(BaseModel):page_content:strmetadata:dict={}# Accept both PUT (what Open WebUI uses) and POST@app.api_route("/process",methods=["PUT","POST"],response_model=LoaderResponse)asyncdefprocess(request:Request,authorization:str|None=Header(None),content_type:str=Header(...),):print(f"📥 /process called Authorization={authorization!r} Content-Type={content_type!r}")data=awaitrequest.body()ifnotdata:raiseHTTPException(status_code=400,detail="No document data provided")try:doc=fitz.open(stream=data,filetype="pdf")exceptExceptionase:raiseHTTPException(status_code=400,detail=f"Error opening PDF: {e}")pages=[]ocr_threshold=20# fallback to OCR if too little textdpi=300# raster DPI for OCRfori,pageinenumerate(doc,start=1):......# do stuffreturnLoaderResponse(page_content=full_text,metadata={})
And I change the address in the documents page settings to simply http://localhost:8083.
Still, when I drag a PDF file to a chat, the detailed log from WebUI shows it never try to use the exteranl service and just took care of everything as usual.
@astroboylrx try to set a fake api Key in documents setting page and click save in the bottom of the page
<!-- gh-comment-id:2918389834 -->
@sir3mat commented on GitHub (May 29, 2025):
@astroboylrx try to set a fake api Key in documents setting page and click save in the bottom of the page
Hello, could this be extended, so that document settings is settable per knowledge? E.g., select for each knowledge instance embedding model and corresponding options. I believe, this would be very usable, if user can tune the options..
<!-- gh-comment-id:2959952771 -->
@xhejtman commented on GitHub (Jun 10, 2025):
Hello, could this be extended, so that document settings is settable per knowledge? E.g., select for each knowledge instance embedding model and corresponding options. I believe, this would be very usable, if user can tune the options..
@arunbugkiller commented on GitHub (Jun 20, 2025):
@astroboylrx @sir3mat I am planning to use "markitdown" (https://github.com/microsoft/markitdown) as external document loader that would convert uploaded docs to markdown files before moving forward , based on your experience do you think it will be possible to achieve this?
<!-- gh-comment-id:2989832155 -->
@arunbugkiller commented on GitHub (Jun 20, 2025):
@astroboylrx @sir3mat I am planning to use "markitdown" (https://github.com/microsoft/markitdown) as external document loader that would convert uploaded docs to markdown files before moving forward , based on your experience do you think it will be possible to achieve this?
<!-- gh-comment-id:2997129827 -->
@Mte90 commented on GitHub (Jun 23, 2025):
Based on the snippet in this issue I did a tiny github project, already tested, for a document extractor to use as external.
It is based on https://github.com/markitdown/markitdown and a plugin that can use LLM if needed for an image, so 0 OCR used.
https://github.com/CodeAtCode/deadsimple
I would like to suggest that, on top of the different options to chat and knowledge-base, also an option for different engines based on different documents formats. Since docling doesn't work with .msg e-mails, for example, but it's excellent for tables. With that in mind I could use Tika for .msg and docling for everything else!
<!-- gh-comment-id:3197500308 -->
@danielwsky commented on GitHub (Aug 18, 2025):
I would like to suggest that, on top of the different options to chat and knowledge-base, also an option for different engines based on different documents formats. Since docling doesn't work with .msg e-mails, for example, but it's excellent for tables. With that in mind I could use Tika for .msg and docling for everything else!
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @andnig on GitHub (Apr 8, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12619
Check Existing Issues
Problem Description
The new Content Extraction Engine with docling serve is amazing. But it's also very slow. For creating a permanent knowledge base (via workspace -> Knowledge) this is not a problem. As it's a one-off task.
But for uploading (larger) files to an ad-hoc chat, docling serve is just too slow.
Desired Solution you'd like
There are two ideas in my mind. Either:
One can set the default Content Extraction Engine in Admin -> Settings (Just as is already implement). When uploading a file in the permanent knowledge base, we get another screen where we can optionally select a different content extraction engine.
Alternatively, there are two different Settings in Admin -> Settings -> Documents. One for the Chat - RAG (where you upload files to the chat) and one for the permanent knowledge base.
Alternatives Considered
No response
Additional Context
No response
@akhilpotdar1 commented on GitHub (Apr 14, 2025):
This is an excellent suggestion. I'd like to expand on it by proposing separate settings for:
Currently, the "Full Context Mode" in Admin → Settings → Documents settings applies globally, which creates challenges. Even with large context window models, using full context mode on extensive knowledge collections often results in context window limitation errors.
Having separate controls for these different use cases would help optimize both:
Related issues/discussions:
@sir3mat commented on GitHub (Apr 22, 2025):
I tried to play with the basecode and if we want to emulate full context for document uploaded in chat (with the "+" button) and, at the same time, chat with collections selected by "#" command using RAG we could do something like that.
Set fullContext: true in the uploadFileHandler() function call at:
https://github.com/open-webui/open-webui/blob/852d9dcbe9b4406e1da230a14642f35ac7bb0d2b/src/lib/components/chat/MessageInput.svelte#L281
Modify the FileItemModal.svelte with this logic
This implementation preserves full context for uploaded documents (as default behaviour) while properly handling RAG collections referenced with the "#" command.
For uploaded chat documents we could use RAG by switching off the fullContext toggle
Does this implementation make technical sense? Are there any potential issues or improvements I should consider?
@sir3mat commented on GitHub (Apr 24, 2025):
I would also recommend adjusting the top_k setting for web search. The chunks generated with Playwright from SearxNG web results often have poor formatting and low-quality text content. As a result, the similarity scores during RAG-based web search are very low (often below 0.15%), which sometimes leads to zero chunks being passed to the model for answer generation.
@akhilpotdar1 commented on GitHub (May 1, 2025):
@sir3mat This is a great solution for handling chat uploads differently from knowledge base content.
Building on this, if we could extend the
FileItemModal.sveltelogic to handle knowledge base collections, it would help address the per-user, per-chat control request in #12459. Perhaps we could:However your current solution works well as a standalone improvement too.
@astroboylrx commented on GitHub (May 28, 2025):
Is "Content Extraction Engine" only relevant when building a knowledge base to be used with RAG? I can barely find any documentation explaining this.
Say if I just simply upload a PDF document to a local AI model, this "Content Extraction Engine" won't be used, right? Then who is doing the content extraction job? I'm so confused.
I followed gpt's suggestion and use
pymupdfandUvicornto run a service on a port to extract text from PDF ("external" is chosen in settings). I then built a knowledge base with a few PDF files and then create a test model from it. When I interact with this model, the service was never called. Who is doing the content extraction job here? I'm again so confused.Also it seems whether the option "Bypass Embedding and Retrieval" is on or off does not affect the observations above.
Any suggestions or more insights will be greatly appreciated!
@sir3mat commented on GitHub (May 28, 2025):
@astroboylrx content extraction engine is used for knoledge base, single files upload on chat and web search results
@astroboylrx commented on GitHub (May 28, 2025):
@sir3mat Many thanks for the info!
I provided an external endpoint for pdf-to-text in the settings and enabled "Bypass Embedding and Retrieval". However webui seems to ignore anyway my settings when I throw a PDF into chat. The detailed logging suggested it just parse the PDF byitself anyway. Is this expected behaviors?
Is the log above suggest that the default "Content Extraction Engine" is being used?
@sir3mat commented on GitHub (May 28, 2025):
@astroboylrx how did you implement your endpoint? Does it follow the input output requirements?
Pls also attach the documents page settings
@astroboylrx commented on GitHub (May 28, 2025):
@sir3mat Thank you again for the response!
Sorry, where can I find this "input output requirements"?
The simplest thing that I got from GPT's suggestions is a
loader.pythat get text out of a PDF (somehow it works on some PDF---I mean when I directly use it, not on webui---whereas the default one failed to extract text according to webui's log; an example can be found here, which is an research article):And then start the service with
uvicorn loader:app --host 0.0.0.0 --port 8083.In the documents page settings I have:
Maybe I totally misunderstood how it works?
@rgaricano commented on GitHub (May 28, 2025):
you can find Input/output requirements on https://github.com/open-webui/open-webui/blob/main/backend/open_webui/retrieval/loaders/external_document.py
endpoint have to be: /process
input: data & headers (in headers we have "Authorization" & "Content-Type" of data)
output: json with page content (string text) & metadata (optional, Arbitrary metadata associated with the content)
@astroboylrx commented on GitHub (May 29, 2025):
@rgaricano Thank you so much for the info!
Sorry I'm not very familir with Web-related coding. I asked GPT to adjust my loader.py based on the
external_document.pyand now it looks like:And I change the address in the documents page settings to simply
http://localhost:8083.Still, when I drag a PDF file to a chat, the detailed log from WebUI shows it never try to use the exteranl service and just took care of everything as usual.
Again, any suggestions or more insights will be greatly appreciated!
@sir3mat commented on GitHub (May 29, 2025):
@astroboylrx try to set a fake api Key in documents setting page and click save in the bottom of the page
@astroboylrx commented on GitHub (Jun 1, 2025):
@sir3mat Thank you so much!! A fake api key did the job...
@xhejtman commented on GitHub (Jun 10, 2025):
Hello, could this be extended, so that document settings is settable per knowledge? E.g., select for each knowledge instance embedding model and corresponding options. I believe, this would be very usable, if user can tune the options..
@arunbugkiller commented on GitHub (Jun 20, 2025):
@astroboylrx @sir3mat I am planning to use "markitdown" (https://github.com/microsoft/markitdown) as external document loader that would convert uploaded docs to markdown files before moving forward , based on your experience do you think it will be possible to achieve this?
@sir3mat commented on GitHub (Jun 20, 2025):
Yes no problem @arunbugkiller
@Mte90 commented on GitHub (Jun 23, 2025):
Based on the snippet in this issue I did a tiny github project, already tested, for a document extractor to use as external.
It is based on https://github.com/markitdown/markitdown and a plugin that can use LLM if needed for an image, so 0 OCR used.
https://github.com/CodeAtCode/deadsimple
@danielwsky commented on GitHub (Aug 18, 2025):
I would like to suggest that, on top of the different options to chat and knowledge-base, also an option for different engines based on different documents formats. Since docling doesn't work with .msg e-mails, for example, but it's excellent for tables. With that in mind I could use Tika for .msg and docling for everything else!