Optimizing the RAG function #3455

New Issue

2025-11-11T15:32:04-06:00

GiteaMirror commented

2025-11-11 15:32:04 -06:00

Originally created by @Schwenn2002 on GitHub (Jan 27, 2025).

A suggestion for optimizing the RAG function.

Currently, you can improve the quality of the query using Top k and ReRanking.

It would be optimal to filter out the most important documents using Top k=50 and ReRanking (0.8). Then another parameter would be optimal if, after the reranking, maximum the best (Top Best=20) hits were passed on to the LLM as context.

This way, the context length can be kept shorter and the response time improved. This also ensures that the context cannot become larger than configured in the LLM.

The current behavior of the LLM is that a context that is too large cannot be processed (i.e. it is interpreted as empty). It would therefore make sense to truncate the context from the RAG query after reranking to the context length of the LLM, so that only the best hits are passed on

Originally created by @Schwenn2002 on GitHub (Jan 27, 2025). A suggestion for optimizing the RAG function. Currently, you can improve the quality of the query using Top k and ReRanking. It would be optimal to filter out the most important documents using Top k=50 and ReRanking (0.8). Then another parameter would be optimal if, after the reranking, maximum the best (Top Best=20) hits were passed on to the LLM as context. This way, the context length can be kept shorter and the response time improved. This also ensures that the context cannot become larger than configured in the LLM. The current behavior of the LLM is that a context that is too large cannot be processed (i.e. it is interpreted as empty). It would therefore make sense to truncate the context from the RAG query after reranking to the context length of the LLM, so that only the best hits are passed on

GiteaMirror closed this issue

2025-11-11 15:32:04 -06:00

GiteaMirror referenced this issue

2026-04-19 20:03:49 -05:00

[GH-ISSUE #3455] docker run binds on 0.0.0.0 instead of localhost #13276

GiteaMirror referenced this issue

2026-04-25 03:21:22 -05:00

[GH-ISSUE #3455] docker run binds on 0.0.0.0 instead of localhost #28804

GiteaMirror referenced this issue

2026-05-05 13:10:19 -05:00

[GH-ISSUE #3455] docker run binds on 0.0.0.0 instead of localhost #51942

GiteaMirror referenced this issue