github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-06-02 23:08:26 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

bug: RAG_RERANKING not functional with manual Win10 install #957

New Issue
Closed
opened 2025-11-11 14:34:22 -06:00 by GiteaMirror · 3 comments
GiteaMirror commented 2025-11-11 14:34:22 -06:00
Owner
Copy Link

Originally created by @chrisoutwright on GitHub (May 17, 2024).

Bug Report

Description

Bug Summary:
After fetching and processing files, the reranking model does not function as expected. Weights from the DebertaV2ForSequenceClassification are not properly initialized, leading to ineffective reranking outcomes. Additionally, there seems to be a lack of interaction from the reranker with the documents, suggesting it cannot interpret or process document content effectively.

Steps to Reproduce:

  1. Run start_windows.bat with the following environment variables set:
ENABLE_RAG_HYBRID_SEARCH=True
RAG_EMBEDDING_MODEL=snowflake-arctic-embed:latest
RAG_EMBEDDING_ENGINE=ollama 
OLLAMA_BASE_URLS=http://host.docker.internal:11434;http://host.docker.internal:11435;http://192.168.1.225:11439;http://192.168.1.225:11437;http://192.168.1.225:11436
CHUNK_SIZE=500
CHUNK_OVERLAP=100
RAG_RERANKING_MODEL_AUTO_UPDATE=True
RAG_RERANKING_MODEL=mightbe/Better-PairRM
  1. Ensure that the RAG_RERANKING_MODEL variable points to a functional model.
  2. Observe the system logs for errors or warnings related to model initialization and embedding generation.
  3. Note the behavior of the reranking model during document processing.

Expected Behavior:
The reranking model should successfully initialize with pre-trained weights and effectively interpret and process documents to enhance search and retrieval outcomes.

Actual Behavior:

  • Initialization errors occur due to some weights not being loaded from the checkpoint.
  • The reranker does not interact effectively with document content, leading to no meaningful enhancements in search or document retrieval functionality as no results used.

Environment

  • Open WebUI Version: v0.1.124
  • Operating System: Windows 10, manual install, CUDA support
  • Browser: Chrome

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of Open WebUI.
  • I have included the browser console logs.
  • I have included the CMD logs.

Additional Notes:

  • The issues occur consistently across multiple attempts with varying document types and contents.
  • Potential compatibility issues with the mightbe/Better-PairRM model could be a contributing factor to the initialization problems.
  • Other Reranking models with same issue.

Logs and Screenshots

WARNING:models.DebertaV2:Some weights of DebertaV2ForSequenceClassification were not initialized from the model checkpoint at C:\Users\Chris\.cache\huggingface\hub\models--mightbe--Better-PairRM\snapshots\c26058437c327e56e878163f74b345b4fb75ce98 and are newly initialized
INFO:apps.rag.main:file.content_type: application/pdf
INFO:apps.rag.main:store_data_in_vector_db
INFO:apps.ollama.main:generate_ollama_embeddings model='snowflake-arctic-embed:latest' prompt='[details of prompt not fully captured]'
INFO:     192.168.1.224:61610 - "GET /_app/immutable/nodes/5.9117d953.js HTTP/1.1" 304 Not Modified
INFO:apps.ollama.main:generate_ollama_embeddings

Effectively it uses:

INFO:apps.ollama.main:generate_ollama_embeddings model='snowflake-arctic-embed:latest' prompt='who involved' options=None keep_alive=None
INFO:apps.ollama.main:url: http://localhost:11435

and then does
INFO:apps.ollama.main:generate_ollama_embeddings

where is reranking seen there?

Originally created by @chrisoutwright on GitHub (May 17, 2024). ## Bug Report ## Description **Bug Summary:** After fetching and processing files, the reranking model does not function as expected. Weights from the DebertaV2ForSequenceClassification are not properly initialized, leading to ineffective reranking outcomes. Additionally, there seems to be a lack of interaction from the reranker with the documents, suggesting it cannot interpret or process document content effectively. **Steps to Reproduce:** 1. Run `start_windows.bat` with the following environment variables set: ``` ENABLE_RAG_HYBRID_SEARCH=True RAG_EMBEDDING_MODEL=snowflake-arctic-embed:latest RAG_EMBEDDING_ENGINE=ollama OLLAMA_BASE_URLS=http://host.docker.internal:11434;http://host.docker.internal:11435;http://192.168.1.225:11439;http://192.168.1.225:11437;http://192.168.1.225:11436 CHUNK_SIZE=500 CHUNK_OVERLAP=100 RAG_RERANKING_MODEL_AUTO_UPDATE=True RAG_RERANKING_MODEL=mightbe/Better-PairRM ``` 2. Ensure that the `RAG_RERANKING_MODEL` variable points to a functional model. 3. Observe the system logs for errors or warnings related to model initialization and embedding generation. 4. Note the behavior of the reranking model during document processing. **Expected Behavior:** The reranking model should successfully initialize with pre-trained weights and effectively interpret and process documents to enhance search and retrieval outcomes. **Actual Behavior:** - Initialization errors occur due to some weights not being loaded from the checkpoint. - The reranker does not interact effectively with document content, leading to no meaningful enhancements in search or document retrieval functionality as no results used. ## Environment - **Open WebUI Version:** v0.1.124 - **Operating System:** Windows 10, manual install, CUDA support - **Browser:** Chrome ## Reproduction Details **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of Open WebUI. - [ ] I have included the browser console logs. - [ ] I have included the CMD logs. **Additional Notes:** - The issues occur consistently across multiple attempts with varying document types and contents. - Potential compatibility issues with the `mightbe/Better-PairRM` model could be a contributing factor to the initialization problems. - Other Reranking models with same issue. ## Logs and Screenshots ``` WARNING:models.DebertaV2:Some weights of DebertaV2ForSequenceClassification were not initialized from the model checkpoint at C:\Users\Chris\.cache\huggingface\hub\models--mightbe--Better-PairRM\snapshots\c26058437c327e56e878163f74b345b4fb75ce98 and are newly initialized INFO:apps.rag.main:file.content_type: application/pdf INFO:apps.rag.main:store_data_in_vector_db INFO:apps.ollama.main:generate_ollama_embeddings model='snowflake-arctic-embed:latest' prompt='[details of prompt not fully captured]' INFO: 192.168.1.224:61610 - "GET /_app/immutable/nodes/5.9117d953.js HTTP/1.1" 304 Not Modified INFO:apps.ollama.main:generate_ollama_embeddings ``` Effectively it uses: ``` INFO:apps.ollama.main:generate_ollama_embeddings model='snowflake-arctic-embed:latest' prompt='who involved' options=None keep_alive=None INFO:apps.ollama.main:url: http://localhost:11435 ``` and then does `INFO:apps.ollama.main:generate_ollama_embeddings ` where is reranking seen there?
GiteaMirror commented 2025-11-11 14:34:23 -06:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (May 17, 2024):

with reranking:

image
image

without reranking
image
image

image

@chrisoutwright commented on GitHub (May 17, 2024): with reranking: ![image](https://github.com/open-webui/open-webui/assets/27736055/48220994-8b81-4434-b7e3-b02798e9ef7d) ![image](https://github.com/open-webui/open-webui/assets/27736055/86dad83d-210d-440f-b83e-e237d9211a22) without reranking ![image](https://github.com/open-webui/open-webui/assets/27736055/1d98e74e-82cb-4789-bb92-506dc2b584c6) ![image](https://github.com/open-webui/open-webui/assets/27736055/a3a665a1-f5f3-435f-9209-8ccdcc806926) ![image](https://github.com/open-webui/open-webui/assets/27736055/a9c47f01-df3f-4811-8e34-bc5027e3dc3e)
GiteaMirror commented 2025-11-11 14:34:23 -06:00
Author
Owner
Copy Link

@tjbck commented on GitHub (May 18, 2024):

PR welcome!

@tjbck commented on GitHub (May 18, 2024): PR welcome!
GiteaMirror commented 2025-11-11 14:34:23 -06:00
Author
Owner
Copy Link

@chrisoutwright commented on GitHub (May 18, 2024):

With hybrid search one would expect RerankCompressor being involved and then being logged with the query_doc_with_hybrid_search:result

I will try again in the Win10 setup, as for the logs. We need to understand the challenge better first, or are there known ones?

@chrisoutwright commented on GitHub (May 18, 2024): With hybrid search one would expect [RerankCompressor ](https://github.com/open-webui/open-webui/blob/90503be2edef1a1f7ce2074286b6316d5cb8868a/backend/apps/rag/utils.py#L78) being involved and then being logged with the `query_doc_with_hybrid_search:result` I will try again in the Win10 setup, as for the logs. We need to understand the challenge better first, or are there known ones?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#957
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?