PgvectorClient.search() Fails After Connection Loss – Causes Stuck Transactions and Query Failures #3641

New Issue

2025-11-11T15:35:59-06:00

GiteaMirror commented

2025-11-11 15:35:59 -06:00

Originally created by @tintina95 on GitHub (Feb 6, 2025).

Bug Report

Important Notes

Before submitting a bug report: Please check the Issues or Discussions section to see if a similar issue or feature request has already been posted. It's likely we're already tracking it! If you’re unsure, start a discussion post first. This will help us efficiently focus on improving the project.
Collaborate respectfully: We value a constructive attitude, so please be mindful of your communication. If negativity is part of your approach, our capacity to engage may be limited. We’re here to help if you’re open to learning and communicating positively. Remember, Open WebUI is a volunteer-driven project managed by a single maintainer and supported by contributors who also have full-time jobs. We appreciate your time and ask that you respect ours.
Contributing: If you encounter an issue, we highly encourage you to submit a pull request or fork the project. We actively work to prevent contributor burnout to maintain the quality and continuity of Open WebUI.
Bug reproducibility: If a bug cannot be reproduced with a :main or :dev Docker setup, or a pip install with Python 3.11, it may require additional help from the community. In such cases, we will move it to the "issues" Discussions section due to our limited resources. We encourage the community to assist with these issues. Remember, it’s not that the issue doesn’t exist; we need your help!

Note: Please remove the notes above when submitting your post. Thank you for your understanding and support!

Installation Method

Docker

Environment

Open WebUI Version: v0.4.8
Ollama (if applicable): NA
Operating System: Windows 10
Browser (if applicable): Chrome 131.0.6778.70

Confirmation:

[x ] I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
[x ] I have included the browser console logs.
[ x] I have included the Docker container logs.
[ x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

If the database connection is lost, Open Web UI should:

Rollback any stuck transactions and attempt to reconnect.
Retry or reinitialize the PgvectorClient.search() connection.
Display a meaningful error message instead of failing silently.

Actual Behavior:

When a PostgreSQL connection loss occurs (e.g., AWS RDS SSL timeout, network disruption, etc.), Open Web UI:

Fails to rollback the transaction, causing subsequent queries to fail.
Returns an empty response ("") instead of an error, making it difficult to debug.
Becomes non-operational until a manual rollback or database restart is performed.

Description

Bug Summary:
We encountered an issue where Open Web UI fails to properly handle PostgreSQL connection failures in PgvectorClient.search(), resulting in stuck transactions, inability to reconnect, and silent failures in RAG responses. When an SSL connection is unexpectedly closed, Open Web UI does not rollback the transaction or retry the search, causing the system to become non-operational until a manual intervention is performed.

Reproduction Details

Steps to Reproduce:
If the database connection drops unexpectedly (e.g., AWS timeout, SSL disconnection), the following occurs:
1. The connection is lost with an error:
The error(exception) is happening in PgvectorClient.search()

log of losing connection:
"Error during search: (psycopg2.OperationalError) SSL connection has been closed unexpectedly"

2. The following AttributeError appears in logs:
AttributeError: 'NoneType' object has no attribute 'ids' ERROR [open_webui.apps.retrieval.utils] Error when querying the collection: 'NoneType' object has no attribute 'ids'

3. Further transactions fail because the previous transaction was not rolled back:
Error during search:
Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)

4. Open Web UI silently fails by returning empty responses ("") instead of notifying users of a backend failure.

Logs and Screenshots

Browser Console Logs:
NA

Docker Container Logs:
Error during search:
(psycopg2.OperationalError) SSL connection has been closed unexpectedly
AttributeError: 'NoneType' object has no attribute 'ids' ERROR [open_webui.apps.retrieval.utils] Error when querying the collection: 'NoneType' object has no attribute 'ids'
Error during search:
Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)

Screenshots/Screen Recordings (if applicable):
NA

Originally created by @tintina95 on GitHub (Feb 6, 2025). # Bug Report ## Important Notes - **Before submitting a bug report**: Please check the Issues or Discussions section to see if a similar issue or feature request has already been posted. It's likely we're already tracking it! If you’re unsure, start a discussion post first. This will help us efficiently focus on improving the project. - **Collaborate respectfully**: We value a constructive attitude, so please be mindful of your communication. If negativity is part of your approach, our capacity to engage may be limited. We’re here to help if you’re open to learning and communicating positively. Remember, Open WebUI is a volunteer-driven project managed by a single maintainer and supported by contributors who also have full-time jobs. We appreciate your time and ask that you respect ours. - **Contributing**: If you encounter an issue, we highly encourage you to submit a pull request or fork the project. We actively work to prevent contributor burnout to maintain the quality and continuity of Open WebUI. - **Bug reproducibility**: If a bug cannot be reproduced with a `:main` or `:dev` Docker setup, or a pip install with Python 3.11, it may require additional help from the community. In such cases, we will move it to the "issues" Discussions section due to our limited resources. We encourage the community to assist with these issues. Remember, it’s not that the issue doesn’t exist; we need your help! Note: Please remove the notes above when submitting your post. Thank you for your understanding and support! --- ## Installation Method Docker ## Environment - **Open WebUI Version:** v0.4.8 - **Ollama (if applicable):** NA - **Operating System:** Windows 10 - **Browser (if applicable):** Chrome 131.0.6778.70 **Confirmation:** - [x ] I have read and followed all the instructions provided in the README.md. - [ ] I am on the latest version of both Open WebUI and Ollama. - [x ] I have included the browser console logs. - [ x] I have included the Docker container logs. - [ x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: If the **database connection is lost**, Open Web UI should: - **Rollback any stuck transactions** and attempt to reconnect. - **Retry or reinitialize the PgvectorClient.search() connection.** - **Display a meaningful error message** instead of failing silently. ## Actual Behavior: When a **PostgreSQL connection loss occurs** (e.g., AWS RDS SSL timeout, network disruption, etc.), Open Web UI: 1. **Fails to rollback the transaction**, causing subsequent queries to fail. 2. **Returns an empty response (`""`) instead of an error**, making it difficult to debug. 3. **Becomes non-operational** until a manual rollback or database restart is performed. ## Description **Bug Summary:** We encountered an issue where Open Web UI fails to properly handle PostgreSQL connection failures in PgvectorClient.search(), resulting in stuck transactions, inability to reconnect, and silent failures in RAG responses. When an SSL connection is unexpectedly closed, Open Web UI does not rollback the transaction or retry the search, causing the system to become non-operational until a manual intervention is performed. ## Reproduction Details **Steps to Reproduce:** If the database connection drops unexpectedly (e.g., AWS timeout, SSL disconnection), the following occurs: **1. The connection is lost with an error:** The error(exception) is happening in `PgvectorClient.search()` **log of losing connection:** `"Error during search: (psycopg2.OperationalError) SSL connection has been closed unexpectedly"` **2. The following AttributeError appears in logs:** `AttributeError: 'NoneType' object has no attribute 'ids' ERROR [open_webui.apps.retrieval.utils] Error when querying the collection: 'NoneType' object has no attribute 'ids'` **3. Further transactions fail because the previous transaction was not rolled back:** **Error during search:** ` Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)` **4. Open Web UI silently fails by returning empty responses ("") instead of notifying users of a backend failure.** ## Logs and Screenshots **Browser Console Logs:** NA **Docker Container Logs:** Error during search: `(psycopg2.OperationalError) SSL connection has been closed unexpectedly` `AttributeError: 'NoneType' object has no attribute 'ids' ERROR [open_webui.apps.retrieval.utils] Error when querying the collection: 'NoneType' object has no attribute 'ids'` Error during search: `Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)` **Screenshots/Screen Recordings (if applicable):** NA

GiteaMirror closed this issue

2025-11-11 15:35:59 -06:00

GiteaMirror commented

2025-11-11 15:36:00 -06:00