With `DATABASE_ENABLE_IAM_TOKEN_AUTH=true` and `VECTOR_DB=pgvector`, startup failed at vector store initialisation with `fe_sendauth: no password supplied`, so the two features could not be used together.
`PgvectorClient` builds its own engine and never got the `do_connect` listener that refreshes the RDS IAM token, and the `ScopedSession` branch that would have reused the instrumented main engine is unreachable because `PGVECTOR_DB_URL` defaults to `DATABASE_URL` and is therefore never falsy.
The pgvector engine now goes through `enable_iam_token_auth()` like the main and Alembic engines. Since a token authenticates exactly one host/port/user, that function now attaches the listener only to engines pointing at the same target, so a `PGVECTOR_DB_URL` aimed at a separate database keeps the password from its own URL instead of having it overwritten; the skip is logged with both identities.
Fixes#27752
Raising GLOBAL_LOG_LEVEL to WARNING buys quieter output but not less work: 241 INFO call sites interpolate their payload into an f-string before the logging call gets to drop it. The heaviest is get_doc, which logs every chunk id and metadata dict in a collection, so on the full-context retrieval path that is the entire knowledge base, once per chat request.
That one line at WARNING, CPython 3.12:
| knowledge base | payload | before | after |
| -------------- | ------- | -------- | ------- |
| top-k of 3 | 1.2 kB | 3.8 us | 0.07 us |
| 500 chunks | 201 kB | 583.6 us | 0.08 us |
| 5000 chunks | 2.0 MB | 5.8 ms | 0.15 us |
The lazy form log.info('query_doc:result %s %s', result.ids, result.metadatas) hands the payload to record.getMessage(), which the InterceptHandler only reaches once a record has passed the level check. Output at INFO is byte-identical. Two sites that already built their message eagerly, one str concat and one % operator, move to the same lazy form.
* Adding hnsw index type for pgvector, allowing vector dimensions larger than 2000
* remove some variable assignments
* Make USE_HALFVEC variable configurable
* Simplify USE_HALFVEC handling
* Raise runtime error if the index requires rebuilt
---------
Co-authored-by: Moritz <moritz.mueller2@tu-dresden.de>
Replace direct CREATE EXTENSION commands with conditional checks to avoid
permission errors on Azure PostgreSQL Flexible Server where only
azure_pg_admin members can create extensions.
- Check pg_extension table before attempting to create vector extension
- Apply same fix to pgcrypto extension for consistency
- Allows following least privilege principle for database users
Fixes#12453
- Created `VectorDBBase` as an abstract base class to standardize vector database operations.
- Added required methods for common vector database operations: `has_collection`, `delete_collection`, `insert`, `upsert`, `search`, `query`, `get`, `delete`, `reset`.
- The base class can now be extended by any vector database implementation (e.g., Qdrant, Pinecone) to ensure a consistent API across different database systems.
Classic298andGitHub
Timothy Jaeryang Baek
Rain6435
Jan Kessler
psy42aandGitHub
Alexander Grimm
Athanasios OikonomouandAthanasios Oikonomou
Marko Henning
Yifang Deng
Jason Kidd