mirror of
https://github.com/open-webui/open-webui.git
synced 2026-05-21 00:52:24 -05:00
[GH-ISSUE #586] feat: rag api integration support (web search) #12130
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#12130
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @tjbck on GitHub (Jan 27, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/586
Extension of #464
@zinwelzl commented on GitHub (Jan 28, 2024):
Hi.
Can you give example how to use it.
I try with
count words from #http://example.com/
but doesn't work for me
@tjbck commented on GitHub (Jan 29, 2024):
@zinwelzl
You should start the prompt using the '#' command followed by the website url like such:
Make sure to click on the button to include the website as a document.
@ChingWeiChan commented on GitHub (Jan 30, 2024):
@tjbck
I think we can make the url block clickable (hypertext) or catch the website title then can check the url whether we expected or not.
@tjbck commented on GitHub (Feb 1, 2024):
@ChingWeiChan Great idea, I'll make it clickable!
Just merged #616, should work as expected!
@adan89lion commented on GitHub (Feb 21, 2024):
Hi,
I tried some news articles with RAG, but it seems like models I tried are unable to process the "correct" content.
For instance, given the example article from CNN, the model is unable to extract answers from the article.
And if I want the model to introduce the document, it responded with content that are irrelevant to the article (website headers or SEO data I presume).
I think an integration with Mozilla's Readability library or similar projects can vastly improve the efficiency of website RAG support for open-webui.
@tbendien commented on GitHub (Mar 1, 2024):
DuckDuckGo search API is free: https://pypi.org/project/duckduckgo-search/
@dillfrescott commented on GitHub (Mar 22, 2024):
Can't wait for this! Super cool stuff!
@d416 commented on GitHub (Mar 22, 2024):
Huggingface chat is one of the best implementations of web search I’ve seen for UX so it’d be great if open-webui used this flow: https://huggingface.co/chat/
HuggingChat also open sourced and is OpenAPI compatible so it can be used with Ollama. The web search can be configured to use different search engines including searxng (private open source search engine)
https://github.com/huggingface/chat-ui?tab=readme-ov-file#web-search-config
Edit: chat-ui has Ollama support built in (it’s in the readme)
@strikeoncmputrz commented on GitHub (Apr 4, 2024):
LLM_Web_search is a similar capability implemented in Text Generation WebUI that uses Langchain for RAG. It's worked very well for me and with a good system or character prompt I don't even need to ask it to search the web. https://github.com/mamei16/LLM_Web_search. I made a minor bug fix but otherwise am not affiliated.
@dillfrescott commented on GitHub (Apr 5, 2024):
I think if open webui were to use a search engine, I nominate searxng (running locally) 100%.
I'm using it with another similar project and it works flawlessly and its totally free too, unlike a lot of these API's.
@sammcj commented on GitHub (Apr 11, 2024):
Seconding the recommendation of searxng, it's really very good, self-hostable and works with many backend search engine providers.
@spergware commented on GitHub (Apr 25, 2024):
+1
Can't wait for a web search feature, game changer!
@MohamedAliRashad commented on GitHub (Apr 25, 2024):
Any updates ?
@knd775 commented on GitHub (Apr 25, 2024):
If you don't have anything to add, please don't comment like this. It just makes it harder for other people to follow.
@9cento
Reactions exist for a reason, use them instead
@MohamedAliRashad
Do you see any updates?
@ProjectMoon commented on GitHub (May 20, 2024):
Idea for how to implement this: make a checkbox/toggle in the message prompt to enable "use web search for enhanced response accuracy". If this toggle is enabled, first send a hidden message to the LLM asking it to analyze the user's message for terms to search for. Then make a search request for each of these terms using some supported search engine, and send the top results to the normal web RAG pipeline.
This would require the admin to configure some search API endpoint. Starting with SearxNG would probably be good because it has an easily configurable JSON API.
@spergware commented on GitHub (May 20, 2024):
That was a bump, frustrated mass-replier. And this is another bump.
@tjbck commented on GitHub (May 27, 2024):
Implemented in dev.