feat: whole/full document mode #1245

New Issue

Closed

opened 2025-11-11 14:40:56 -06:00 by GiteaMirror · 20 comments

GiteaMirror commented 2025-11-11 14:40:56 -06:00

Owner

Copy Link

Originally created by @PkmX on GitHub (Jun 13, 2024).

Originally assigned to: @tjbck on GitHub.

Is your feature request related to a problem? Please describe.
Sometimes it may be ideal to pass the whole document to LLM for tasks like summarization. Currently, if you upload a document and ask the LLM to summarize it, the RAG would most likely just return no result. As the user can't turn off the retrieval part, the LLM basically becomes useless in this rather common use case. Passing the full document is also useful of tasks like translation or sentiment analysis.

Describe the solution you'd like
When using an uploaded document or fetched webpage, there should be a checkbox allowing the user to pass the entire ingested document as the context. This should be a relative easy change to just skip the retrieval straight to the query.

Ideally, there should be a warning if the content is larger than the LLM's context size, so truncation or degraded output does not occur.

Describe alternatives you've considered

• Just copy/paste the full document into the chatbox: It works but is not very convenient.
• Have a separate pipeline specifically made for summarization: This can use a myriad of techniques to provide even better results with smaller context sizes, but requires significantly more effort to implement.

Additional context
This feature is also present in some other chat UI's, e.g., “document pinning” in AnythingLLM.

I'm not sure how this will interact with pipelines, since pipelines can basically do anything. I think such a flag can also be passed to compatible RAG pipelines to let it know if the user wants to perform context retrieval or just do a full-context insertion.

Originally created by @PkmX on GitHub (Jun 13, 2024). Originally assigned to: @tjbck on GitHub. **Is your feature request related to a problem? Please describe.** Sometimes it may be ideal to pass the whole document to LLM for tasks like summarization. Currently, if you upload a document and ask the LLM to summarize it, the RAG would most likely just return no result. As the user can't turn off the retrieval part, the LLM basically becomes useless in this rather common use case. Passing the full document is also useful of tasks like translation or sentiment analysis. **Describe the solution you'd like** When using an uploaded document or fetched webpage, there should be a checkbox allowing the user to pass the entire ingested document as the context. This should be a relative easy change to just skip the retrieval straight to the query. Ideally, there should be a warning if the content is larger than the LLM's context size, so truncation or degraded output does not occur. **Describe alternatives you've considered** * Just copy/paste the full document into the chatbox: It works but is not very convenient. * Have a separate pipeline specifically made for summarization: This can use a myriad of techniques to provide even better results with smaller context sizes, but requires significantly more effort to implement. **Additional context** This feature is also present in some other chat UI's, e.g., “document pinning” in AnythingLLM. I'm not sure how this will interact with pipelines, since pipelines can basically do anything. I think such a flag can also be passed to compatible RAG pipelines to let it know if the user wants to perform context retrieval or just do a full-context insertion.

GiteaMirror closed this issue 2025-11-11 14:40:57 -06:00

GiteaMirror commented 2025-11-11 14:40:57 -06:00

Author

Owner

Copy Link

@bunnyfu commented on GitHub (Jun 14, 2024):

Full support, with 32-128k context models now being more common, sometimes it's just easier and more failsafe to pass the whole document into the context if you want a summary.

@bunnyfu commented on GitHub (Jun 14, 2024): Full support, with 32-128k context models now being more common, sometimes it's just easier and more failsafe to pass the whole document into the context if you want a summary.

GiteaMirror commented 2025-11-11 14:40:58 -06:00

Author

Owner

Copy Link

@Peter-De-Ath commented on GitHub (Jun 14, 2024):

Yes I like this.
I currently open the document then paste it into chat, then do some editing of the messages to get the prompting right for the task i want.

on Summarization I have a modelfile which was setup just for summarizing which i can just paste my entire document (text) and return just the summary

@Peter-De-Ath commented on GitHub (Jun 14, 2024): Yes I like this. I currently open the document then paste it into chat, then do some editing of the messages to get the prompting right for the task i want. on Summarization I have a modelfile which was setup just for summarizing which i can just paste my entire document (text) and return just the summary

GiteaMirror commented 2025-11-11 14:40:58 -06:00

Author

Owner

Copy Link

@rvkwi commented on GitHub (Jul 22, 2024):

That would be really useful for several edge cases, recently ran into this not with summaries but some analysis. It was not about the content but patterns in it (so the entire context was important to survive at once). Ended up copying and pasting a lot, and eventually throwing it into Claude because this got tedious quickly.

RAG is amazing, but there's cases where a simple 1:1 is all you need.

@rvkwi commented on GitHub (Jul 22, 2024): That would be really useful for several edge cases, recently ran into this not with summaries but some analysis. It was not about the content but patterns in it (so the entire context was important to survive at once). Ended up copying and pasting a lot, and eventually throwing it into Claude because this got tedious quickly. RAG is amazing, but there's cases where a simple 1:1 is all you need.

GiteaMirror commented 2025-11-11 14:40:58 -06:00

Author

Owner

Copy Link

@pbasov commented on GitHub (Jul 23, 2024):

With llama3.1 supporting 128k context this would be an important feature to have

@pbasov commented on GitHub (Jul 23, 2024): With llama3.1 supporting 128k context this would be an important feature to have

GiteaMirror commented 2025-11-11 14:40:58 -06:00

Author

Owner

Copy Link

@justinh-rahb commented on GitHub (Jul 23, 2024):

With llama3.1 supporting 128k context this would be an important feature to have

I think a lot of people are going to soon realize that Ollama doesn't actually run models at their full context size by default, and that if you wanted to run even 8B at 128K you'd need over 100GB of VRAM...

@justinh-rahb commented on GitHub (Jul 23, 2024): > With llama3.1 supporting 128k context this would be an important feature to have I think a lot of people are going to soon realize that Ollama doesn't actually run models at their full context size by default, and that if you wanted to run even 8B at 128K you'd need over 100GB of VRAM...

GiteaMirror commented 2025-11-11 14:40:59 -06:00

Author

Owner

Copy Link

@rvkwi commented on GitHub (Jul 24, 2024):

With llama3.1 supporting 128k context this would be an important feature to have

I think a lot of people are going to soon realize that Ollama doesn't actually run models at their full context size by default, and that if you wanted to run even 8B at 128K you'd need over 100GB of VRAM...

just because some can't fit it, doesn't mean others can't. num_ctx is not exactly hidden or advanced to understand, and you don't have to max it out just because it can. on top not everything uses standard quadratic attention to sequence length.

open webui even has it more clearly worded as context length in the model file.
i'm just saying, RAG is great, but there are also plenty of uses where you specifically don't want to cut a context insertion apart. a lot of these cases don't even need a massive context, but it's not unreasonable that people run high contexts with the webui. i think it would be a helpful feature to have as a toggle.

@rvkwi commented on GitHub (Jul 24, 2024): > > With llama3.1 supporting 128k context this would be an important feature to have > > I think a lot of people are going to soon realize that Ollama doesn't actually run models at their full context size by default, and that if you wanted to run even 8B at 128K you'd need over 100GB of VRAM... just because some can't fit it, doesn't mean others can't. num_ctx is not exactly hidden or advanced to understand, and you don't have to max it out just because it can. on top not everything uses standard quadratic attention to sequence length. open webui even has it more clearly worded as context length in the model file. i'm just saying, RAG is great, but there are also plenty of uses where you specifically don't want to cut a context insertion apart. a lot of these cases don't even need a massive context, but it's not unreasonable that people run high contexts with the webui. i think it would be a helpful feature to have as a toggle.

GiteaMirror commented 2025-11-11 14:40:59 -06:00

Author

Owner

Copy Link

@justinh-rahb commented on GitHub (Jul 24, 2024):

I wasn't arguing against it @rvkwi, you can see that I did thumbs-up the OP. You and I may know how num_ctx works full-well, but there are a LOT of WebUI users that don't understand that.

@justinh-rahb commented on GitHub (Jul 24, 2024): I wasn't arguing against it @rvkwi, you can see that I did thumbs-up the OP. You and I may know how `num_ctx` works full-well, but there are a LOT of WebUI users that don't understand that.

GiteaMirror commented 2025-11-11 14:40:59 -06:00

Author

Owner

Copy Link

@formigarafa commented on GitHub (Jul 30, 2024):

Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system.

ATM, the biggest issue is that it is hard to integrate any alternative with a RAG on the way.

I did not check but yet if it is available, but with that in hand and with an api to retrieve/read the documents we could implement alternative methods of context insertion with filters, pipes, tools or whatever we dream.

@formigarafa commented on GitHub (Jul 30, 2024): Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system. ATM, the biggest issue is that it is hard to integrate any alternative with a RAG on the way. I did not check but yet if it is available, but with that in hand and with an api to retrieve/read the documents we could implement alternative methods of context insertion with filters, pipes, tools or whatever we dream.

GiteaMirror commented 2025-11-11 14:41:00 -06:00

Author

Owner

Copy Link

@justinh-rahb commented on GitHub (Jul 30, 2024):

Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system.

Example for Tools:

class Tools:
    def __init__(self):
        # If set to true it will prevent default RAG pipeline
        self.file_handler = True

From: https://openwebui.com/t/hub/get_files

@justinh-rahb commented on GitHub (Jul 30, 2024): > Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system. Example for Tools: ```python class Tools: def __init__(self): # If set to true it will prevent default RAG pipeline self.file_handler = True ``` From: https://openwebui.com/t/hub/get_files

GiteaMirror commented 2025-11-11 14:41:00 -06:00

Author

Owner

Copy Link

@pomoke commented on GitHub (Aug 10, 2024):

If we cannot fit the whole file into the ctx, we will need to take some other approaches like memGPT.

@pomoke commented on GitHub (Aug 10, 2024): If we cannot fit the whole file into the ctx, we will need to take some other approaches like memGPT.

GiteaMirror commented 2025-11-11 14:41:01 -06:00

Author

Owner

Copy Link

@formigarafa commented on GitHub (Aug 11, 2024):

If we cannot fit the whole file into the ctx, we will need to take some other approaches like memGPT.

ATM, what happens the given prompt (and chat history) does not fit the context? I mean, this problem already exists, doesn't it?

@formigarafa commented on GitHub (Aug 11, 2024): > If we cannot fit the whole file into the ctx, we will need to take some other approaches like memGPT. ATM, what happens the given prompt (and chat history) does not fit the context? I mean, this problem already exists, doesn't it?

GiteaMirror commented 2025-11-11 14:41:01 -06:00

Author

Owner

Copy Link

@pomoke commented on GitHub (Aug 13, 2024):

Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system.

Example for Tools:

class Tools:
    def __init__(self):
        # If set to true it will prevent default RAG pipeline
        self.file_handler = True

From: https://openwebui.com/t/hub/get_files

However, the default RAG is still seems to be enabled with this and .citation = True.

@pomoke commented on GitHub (Aug 13, 2024): > > Is it possible to bypass the current RAG system? I mean, having the documents inserted in the chat but opt to not use/apply them through the built-in RAG system. > > Example for Tools: > > ```python > class Tools: > def __init__(self): > # If set to true it will prevent default RAG pipeline > self.file_handler = True > ``` > > From: https://openwebui.com/t/hub/get_files However, the default RAG is still seems to be enabled with this and `.citation = True`.

GiteaMirror commented 2025-11-11 14:41:01 -06:00

Author

Owner

Copy Link

@NeverOccurs commented on GitHub (Sep 10, 2024):

Try this:
https://github.com/NeverOccurs/full_doc_pipe
Use it as a filter function in Open-Webui. It allows uploading a file and push it fully into system prompt. Then you can direct ask the LLM to summarize it or whatever. It doesn't support changing file midway through the chat though.
Still optimizing it but for now I think it is enough for my use case. I'm trying to incorporating it with Claude's prompt caching to minimize cost.

@NeverOccurs commented on GitHub (Sep 10, 2024): Try this: https://github.com/NeverOccurs/full_doc_pipe Use it as a filter function in Open-Webui. It allows uploading a file and push it fully into system prompt. Then you can direct ask the LLM to summarize it or whatever. It doesn't support changing file midway through the chat though. Still optimizing it but for now I think it is enough for my use case. I'm trying to incorporating it with Claude's prompt caching to minimize cost.

GiteaMirror commented 2025-11-11 14:41:01 -06:00

Author

Owner

Copy Link

@tjbck commented on GitHub (Sep 29, 2024):

Full document mode toggle option has been added to latest dev, you can find the toggleable option by clicking on the file item component. Testing wanted here!

@tjbck commented on GitHub (Sep 29, 2024): <img width="1047" alt="image" src="https://github.com/user-attachments/assets/75e205e7-c240-4cc1-9c1c-7449ab0818c5"> Full document mode toggle option has been added to latest dev, you can find the toggleable option by clicking on the file item component. Testing wanted here!

GiteaMirror commented 2025-11-11 14:41:01 -06:00

Author

Owner

Copy Link

@daniel-dona commented on GitHub (Oct 13, 2024):

Looks like the context is not being passed to the LLM with the "full document" option.

The query received by Ollama:

ollama  | time=2024-10-13T14:43:54.332Z level=DEBUG source=routes.go:1422 msg="chat request" images=0 prompt="<|im_start|>system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>
<|im_start|>user
You are given a user query, some textual context and rules, all inside xml tags. You have to answer the query based on the context while respecting the rules.

<context>
BK72XX_SDK_User_Manual-3.0.3-1.pdf:
</context>

<rules>
- If you don't know, just say so.
- If you are not sure, ask for clarification.
- Answer in the same language as the user query.
- If the context appears unreadable or of poor quality, tell the user then answer as best as you can.
- If the answer is not in the context but you think you know the answer, explain that to the user then answer with your own knowledge.
- Answer directly and without using xml tags.
</rules>

<user_query>
Summarize this
</user_query>

Summarize this<|im_end|>
<|im_start|>assistant

Used document:

BK72XX_SDK_User_Manual-3.0.3-1.pdf

@daniel-dona commented on GitHub (Oct 13, 2024): Looks like the context is not being passed to the LLM with the "full document" option.    The query received by Ollama: ``` ollama | time=2024-10-13T14:43:54.332Z level=DEBUG source=routes.go:1422 msg="chat request" images=0 prompt="<|im_start|>system You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|> <|im_start|>user You are given a user query, some textual context and rules, all inside xml tags. You have to answer the query based on the context while respecting the rules. <context> BK72XX_SDK_User_Manual-3.0.3-1.pdf: </context> <rules> - If you don't know, just say so. - If you are not sure, ask for clarification. - Answer in the same language as the user query. - If the context appears unreadable or of poor quality, tell the user then answer as best as you can. - If the answer is not in the context but you think you know the answer, explain that to the user then answer with your own knowledge. - Answer directly and without using xml tags. </rules> <user_query> Summarize this </user_query> Summarize this<|im_end|> <|im_start|>assistant ``` Used document: [BK72XX_SDK_User_Manual-3.0.3-1.pdf](https://github.com/user-attachments/files/17356037/BK72XX_SDK_User_Manual-3.0.3-1.pdf)

GiteaMirror commented 2025-11-11 14:41:02 -06:00

Author

Owner

Copy Link

@daniel-dona commented on GitHub (Oct 13, 2024):

@daniel-dona It sounds like you are also facing #291 (please check the logs and confirm).

Nope, not a single 40X error found in the Developer Console, I don't think is a CORS problem

@daniel-dona commented on GitHub (Oct 13, 2024): > @daniel-dona It sounds like you are also facing #291 (please check the logs and confirm). Nope, not a single 40X error found in the Developer Console, I don't think is a CORS problem

GiteaMirror commented 2025-11-11 14:41:02 -06:00

Author

Owner

Copy Link

@a-mart commented on GitHub (Oct 21, 2024):

Seeing the same issue. Only handing 5 random pages of the document to the model out of roughly 50.

@a-mart commented on GitHub (Oct 21, 2024): Seeing the same issue. Only handing 5 random pages of the document to the model out of roughly 50.

GiteaMirror commented 2025-11-11 14:41:02 -06:00

Author

Owner

Copy Link

@a-mart commented on GitHub (Oct 28, 2024):

I've also confirmed that I'm not seeing the errors from #291 but still experiencing the same issue

@a-mart commented on GitHub (Oct 28, 2024): I've also confirmed that I'm not seeing the errors from #291 but still experiencing the same issue

GiteaMirror commented 2025-11-11 14:41:02 -06:00

Author

Owner

Copy Link

@maurizioaiello commented on GitHub (May 1, 2025):

Try this: https://github.com/NeverOccurs/full_doc_pipe Use it as a filter function in Open-Webui. It allows uploading a file and push it fully into system prompt. Then you can direct ask the LLM to summarize it or whatever. It doesn't support changing file midway through the chat though. Still optimizing it but for now I think it is enough for my use case. I'm trying to incorporating it with Claude's prompt caching to minimize cost.

How does it work? Can you explain better how to use it?

@maurizioaiello commented on GitHub (May 1, 2025): > Try this: https://github.com/NeverOccurs/full_doc_pipe Use it as a filter function in Open-Webui. It allows uploading a file and push it fully into system prompt. Then you can direct ask the LLM to summarize it or whatever. It doesn't support changing file midway through the chat though. Still optimizing it but for now I think it is enough for my use case. I'm trying to incorporating it with Claude's prompt caching to minimize cost. How does it work? Can you explain better how to use it?

GiteaMirror commented 2025-11-11 14:41:03 -06:00

Author

Owner

Copy Link

@jayendra19 commented on GitHub (Nov 3, 2025):

If i have pdf and i uploaded it when i say summarise it won’t be able to do it how can i enable it ?

@jayendra19 commented on GitHub (Nov 3, 2025): If i have pdf and i uploaded it when i say summarise it won’t be able to do it how can i enable it ?

GiteaMirror referenced this issue 2026-04-19 19:19:58 -05:00

[GH-ISSUE #1245] How can I upload 7B model ? #12411

GiteaMirror referenced this issue 2026-04-25 02:42:25 -05:00

[GH-ISSUE #1245] How can I upload 7B model ? #27939

GiteaMirror referenced this issue 2026-05-05 11:55:58 -05:00

[GH-ISSUE #1245] How can I upload 7B model ? #51076

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dev

v0.9.2

v0.9.1

v0.9.0

v0.8.12

v0.8.11

v0.8.10

v0.8.9

v0.8.8

v0.8.7

v0.8.6

v0.8.5

v0.8.4

v0.8.3

v0.8.2

v0.8.1

v0.8.0

v0.7.2

v0.7.1

v0.7.0

v0.6.43

v0.6.42

v0.6.41

v0.6.40

v0.6.39

v0.6.38

v0.6.37

v0.6.36

v0.6.35

v0.6.34

v0.6.33

v0.6.32

v0.6.31

v0.6.30

v0.6.29

v0.6.28

v0.6.27

v0.6.26

v0.6.25

v0.6.24

v0.6.23

v0.6.22

v0.6.21

v0.6.20

v0.6.19

v0.6.18

v0.6.17

v0.6.16

v0.6.15

v0.6.14

v0.6.13

v0.6.12

v0.6.11

v0.6.10

v0.6.9

v0.6.8

v0.6.7

v0.6.6

v0.6.5

v0.6.4

v0.6.3

v0.6.2

v0.6.1

v0.6.0

v0.5.20

v0.5.19

v0.5.18

v0.5.17

v0.5.16

v0.5.15

v0.5.14

v0.5.13

v0.5.12

v0.5.11

v0.5.10

v0.5.9

v0.5.8

v0.5.7

v0.5.6

v0.5.5

v0.5.4

v0.5.3

v0.5.2

v0.5.1

v0.5.0

v0.4.8

v0.4.7

v0.4.6

v0.4.5

v0.4.4

v0.4.3

v0.4.2

v0.4.1

v0.4.0

v0.3.35

v0.3.34

v0.3.33

v0.3.32

v0.3.31

v0.3.30

v0.3.29

v0.3.28

v0.3.27

v0.3.26

v0.3.25

v0.3.24

v0.3.23

v0.3.22

v0.3.21

v0.3.20

v0.3.19

v0.3.18

v0.3.17

v0.3.16

v0.3.15

v0.3.14

v0.3.13

v0.3.12

v0.3.11

v0.3.10

v0.3.9

v0.3.8

v0.3.7

v0.3.6

v0.3.5

v0.3.4

v0.3.3

v0.3.2

v0.3.1

v0.3.0

v0.2.5

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.125

v0.1.124

v0.1.123

v0.1.122

v0.1.121

v0.1.120

v0.1.119

v0.1.118

v0.1.117

v0.1.116

v0.1.115

v0.1.114

v0.1.113

v0.1.112

v0.1.111

v0.1.110

v0.1.109

v0.1.108

v0.1.107

v0.1.106

v0.1.105

v0.1.104

v0.1.103

v0.1.102

Labels

Clear labels

bug

confirmed

confirmed issue

core

documentation

enhancement

good first issue

help wanted

non-core

pull-request

Mirrored from GitHub Pull Request

python

question

testing wanted

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

GiteaMirror ninjasurge

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#1245