[GH-ISSUE #10061] Improved chunking options for RAG #54414

New Issue

GiteaMirror · 2026-05-05T16:14:27-05:00

GiteaMirror commented

2026-05-05 16:14:27 -05:00

Originally created by @subashc2023 on GitHub (Feb 15, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/10061

Feature Request

Right now it seems the only options are chunk size, and overlap. Can we introduce additional chunking options that will automatically chunk codeblocks, and split by paragraph in Markdown docs? The current chunking algorithm destroys code, and the retrieval system does not work very will with this arbitrarily chunked code.

Originally created by @subashc2023 on GitHub (Feb 15, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/10061 # Feature Request Right now it seems the only options are chunk size, and overlap. Can we introduce additional chunking options that will automatically chunk codeblocks, and split by paragraph in Markdown docs? The current chunking algorithm destroys code, and the retrieval system does not work very will with this arbitrarily chunked code.

GiteaMirror closed this issue

2026-05-05 16:14:28 -05:00

GiteaMirror commented

2026-05-05 16:14:30 -05:00

@Schwenn2002 commented on GitHub (Feb 15, 2025):

A suggestion for additional optimizing the RAG function.

Currently, you can improve the quality of the query using Top k and ReRanking.

It would be optimal to filter out the most important documents using Top k=50 or 70 and ReRanking (0,5 - 0.8). Then another parameter would be optimal if, after the reranking, maximum the best (Top Best=10 or 20) hits were passed on to the LLM as context.

This way, the context length can be kept shorter and the response time improved. This also ensures that the context cannot become larger than configured in the LLM.

The current behavior of the LLM is that a context that is too large cannot be processed (i.e. it is interpreted as empty). It would therefore make sense to truncate the context from the RAG query after reranking to the context length of the LLM, so that only the best hits are passed on.

@Schwenn2002 commented on GitHub (Feb 15, 2025): A suggestion for additional optimizing the RAG function. Currently, you can improve the quality of the query using Top k and ReRanking. It would be optimal to filter out the most important documents using Top k=50 or 70 and ReRanking (0,5 - 0.8). Then another parameter would be optimal if, after the reranking, maximum the best (Top Best=10 or 20) hits were passed on to the LLM as context. This way, the context length can be kept shorter and the response time improved. This also ensures that the context cannot become larger than configured in the LLM. The current behavior of the LLM is that a context that is too large cannot be processed (i.e. it is interpreted as empty). It would therefore make sense to truncate the context from the RAG query after reranking to the context length of the LLM, so that only the best hits are passed on.

GiteaMirror commented

2026-05-05 16:14:31 -05:00

@Schwenn2002 commented on GitHub (Feb 15, 2025):

Furthermore, I have replaced the integrated chromadb with qdrant. Retrieving information seems to work much better with qdrant!

I have customized the model suggested above in the source code. This makes the retrieval much better. But I also chose 50% overlap in the RAG.

Nevertheless, the semantic formation of vectors would certainly be even better for the RAG.

@Schwenn2002 commented on GitHub (Feb 15, 2025): Furthermore, I have replaced the integrated chromadb with qdrant. Retrieving information seems to work much better with qdrant! I have customized the model suggested above in the source code. This makes the retrieval much better. But I also chose 50% overlap in the RAG. Nevertheless, the semantic formation of vectors would certainly be even better for the RAG.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#54414