I've been working on improving the hybrid search functionality and Milvus integration, addressing a few issues that were causing inconsistent behavior. Here's a summary of the problems and the proposed fixes:
Issues Addressed
Hybrid Search Returning Empty Results: The RerankCompressor in utils.py occasionally returned no documents due to strict score filtering or errors in reranking logic, especially when used with the Milvus client.
Float32 Serialization Error: Scores stored as numpy.float32 in the metadata caused a TypeError: Object of type float32 is not JSON serializable during Socket.IO emission, breaking the chat response flow.
Inconsistent Data Handling in Milvus: The original milvus.py didn't robustly handle variations in Milvus result formats (e.g., malformed JSON in data or metadata), leading to potential failures.
Proposed Changes
utils.py:
Updated RerankCompressor to:
Use Optional[float] for r_score (allowing None to disable filtering).
Add fallback to original documents if score calculation fails, improving robustness.
Convert scores to float to fix serialization issues.
Adjusted query_doc_with_hybrid_search to:
Include a fallback to query_doc for empty collections.
Ensure scores in result_dict are serialized as float.
milvus.py:
Enhanced _result_to_get_result and _result_to_search_result to:
Sanitize data and metadata (handle malformed JSON and ensure dict type).
Add score to metadata in search results for compatibility with hybrid search.
Added has_collection checks in query and get to prevent errors on non-existent collections.
Benefits
Hybrid search now consistently returns reranked documents, avoiding empty results.
Improves Milvus compatibility with robust data handling, aligning with Chroma's expected format.
Proposed Code
Attached are the updated versions of utils.py and milvus.py. These changes maintain the original structure and intent while fixing the identified issues. You can test them by replacing the current files and running a query like "habeas corpus pode discutir provas relacionadas à inocência" with r_score=0.2 and top_k=10.
How to Test
Replace open_webui/retrieval/utils.py and open_webui/retrieval/vector/milvus.py with the attached files.
Set r_score=0.2 and top_k=10 in your environment or configuration.
Run a query and verify that the LLM receives reranked chunks without errors.
Thanks for considering this contribution! I'm happy to adjust or provide more details if needed.
Best regards,
Pablo
Originally created by @pablocerdeira on GitHub (Feb 23, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/10630
[milvus.py.txt](https://github.com/user-attachments/files/18932517/milvus.py.txt)
[utils.py.txt](https://github.com/user-attachments/files/18932518/utils.py.txt)
Hi Open-WebUI team,
I've been working on improving the hybrid search functionality and Milvus integration, addressing a few issues that were causing inconsistent behavior. Here's a summary of the problems and the proposed fixes:
#### Issues Addressed
1. **Hybrid Search Returning Empty Results**: The `RerankCompressor` in `utils.py` occasionally returned no documents due to strict score filtering or errors in reranking logic, especially when used with the Milvus client.
2. **Float32 Serialization Error**: Scores stored as `numpy.float32` in the metadata caused a `TypeError: Object of type float32 is not JSON serializable` during Socket.IO emission, breaking the chat response flow.
3. **Inconsistent Data Handling in Milvus**: The original `milvus.py` didn't robustly handle variations in Milvus result formats (e.g., malformed JSON in `data` or `metadata`), leading to potential failures.
#### Proposed Changes
- **`utils.py`**:
- Updated `RerankCompressor` to:
- Use `Optional[float]` for `r_score` (allowing `None` to disable filtering).
- Add fallback to original documents if score calculation fails, improving robustness.
- Convert scores to `float` to fix serialization issues.
- Adjusted `query_doc_with_hybrid_search` to:
- Include a fallback to `query_doc` for empty collections.
- Ensure scores in `result_dict` are serialized as `float`.
- **`milvus.py`**:
- Enhanced `_result_to_get_result` and `_result_to_search_result` to:
- Sanitize `data` and `metadata` (handle malformed JSON and ensure `dict` type).
- Add `score` to `metadata` in search results for compatibility with hybrid search.
- Added `has_collection` checks in `query` and `get` to prevent errors on non-existent collections.
#### Benefits
- Hybrid search now consistently returns reranked documents, avoiding empty results.
- Eliminates serialization errors, ensuring smooth chat responses.
- Improves Milvus compatibility with robust data handling, aligning with Chroma's expected format.
#### Proposed Code
Attached are the updated versions of `utils.py` and `milvus.py`. These changes maintain the original structure and intent while fixing the identified issues. You can test them by replacing the current files and running a query like "habeas corpus pode discutir provas relacionadas à inocência" with `r_score=0.2` and `top_k=10`.
#### How to Test
1. Replace `open_webui/retrieval/utils.py` and `open_webui/retrieval/vector/milvus.py` with the attached files.
2. Set `r_score=0.2` and `top_k=10` in your environment or configuration.
3. Run a query and verify that the LLM receives reranked chunks without errors.
Thanks for considering this contribution! I'm happy to adjust or provide more details if needed.
Best regards,
Pablo
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @pablocerdeira on GitHub (Feb 23, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/10630
milvus.py.txt
utils.py.txt
Hi Open-WebUI team,
I've been working on improving the hybrid search functionality and Milvus integration, addressing a few issues that were causing inconsistent behavior. Here's a summary of the problems and the proposed fixes:
Issues Addressed
RerankCompressorinutils.pyoccasionally returned no documents due to strict score filtering or errors in reranking logic, especially when used with the Milvus client.numpy.float32in the metadata caused aTypeError: Object of type float32 is not JSON serializableduring Socket.IO emission, breaking the chat response flow.milvus.pydidn't robustly handle variations in Milvus result formats (e.g., malformed JSON indataormetadata), leading to potential failures.Proposed Changes
utils.py:RerankCompressorto:Optional[float]forr_score(allowingNoneto disable filtering).floatto fix serialization issues.query_doc_with_hybrid_searchto:query_docfor empty collections.result_dictare serialized asfloat.milvus.py:_result_to_get_resultand_result_to_search_resultto:dataandmetadata(handle malformed JSON and ensuredicttype).scoretometadatain search results for compatibility with hybrid search.has_collectionchecks inqueryandgetto prevent errors on non-existent collections.Benefits
Proposed Code
Attached are the updated versions of
utils.pyandmilvus.py. These changes maintain the original structure and intent while fixing the identified issues. You can test them by replacing the current files and running a query like "habeas corpus pode discutir provas relacionadas à inocência" withr_score=0.2andtop_k=10.How to Test
open_webui/retrieval/utils.pyandopen_webui/retrieval/vector/milvus.pywith the attached files.r_score=0.2andtop_k=10in your environment or configuration.Thanks for considering this contribution! I'm happy to adjust or provide more details if needed.
Best regards,
Pablo