Add |middletruncate:n, |start:n, and |end:n pipe filters to the
{{MESSAGES}} template variable, enabling per-message character
truncation for task models (title, tags, follow-up, etc.).
Example: {{MESSAGES:END:2|middletruncate:500}}
This optimizes task model prompt size for conversations with very
long messages (e.g. pasted documents), reducing latency for local
models and API costs.
Closes#21499
The get_token_usage_by_user query lacked group_id filtering, while the
companion get_message_count_by_user query already supported it. When an
admin filtered analytics by user group, message counts were correctly
scoped to the group but token usage totals included data from all users.
Add the group_id parameter and subquery filter to get_token_usage_by_user,
matching the pattern used by get_message_count_by_user and other analytics
queries, and pass group_id through from the analytics endpoint.
The locale for Azure TTS SSML was being extracted with `split("-")[:1]`,
which only takes the first segment (e.g., "en" from "en-US"). The
xml:lang attribute in SSML requires a full locale like "en-US", not just
a language code. This caused Azure TTS to either fail or use incorrect
pronunciation rules.
Changed `[:1]` to `[:2]` to properly extract the locale (e.g., "en-US").
Co-authored-by: gambletan <ethanchang32@gmail.com>
- URL-encodes the OAuth error message when constructing the redirect URL in the OIDC callback handler
- Without encoding, error messages containing spaces, ampersands, or other special characters produce malformed URLs that the frontend cannot parse correctly
- The custom OAuth client callback handler already correctly uses urllib.parse.quote_plus() for the same purpose; this fix brings the OIDC handler in line with that pattern
Co-authored-by: gambletan <tan@gambletan.com>
json.loads(event_data.get("user", {})) crashes with TypeError when
the "user" key is absent because the default value {} is a dict, not
a JSON string. json.loads expects str/bytes, not dict.
Also handle the case where "user" is already a dict (not serialized
JSON) to make the webhook more robust.
Co-authored-by: gambletan <ethanchang32@gmail.com>
In both inlet and outlet filter processing, response.json() was called
BEFORE response.raise_for_status(). When a filter endpoint returns an
HTTP error, the user's chat payload gets silently overwritten with the
error response body. If the error is not caught, the corrupted payload
propagates through subsequent filters and into the chat completion.
Swapped the order so raise_for_status() runs first — payload is only
updated on success.
Co-authored-by: gambletan <ethanchang32@gmail.com>
`raise "string"` in Python raises TypeError instead of the intended
error, making error messages confusing and debugging difficult.
Co-authored-by: gambletan <ethanchang32@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace bare except with except Exception in main.py
* fix: replace bare except with Exception in oauth.py
In Python 3, bare 'except:' is discouraged as it catches all
SystemExit and KeyboardInterrupt exceptions. Changed to 'except Exception:'
to only catch actual exceptions.
Add Chat.id as a secondary sort key to all paginated chat queries
that use offset/limit pagination. When multiple chats share the same
updated_at timestamp, the database does not guarantee a stable order
across page boundaries, causing chats to appear on multiple pages.
This produces duplicate keys in the Svelte sidebar each-block
(each_key_duplicate error). Adding Chat.id as a tiebreaker ensures
fully deterministic ordering.
Extends the fix from #22383 (which addressed get_chat_ids_by_model_id)
to all remaining paginated chat queries.
* fix: normalize usage tokens + migration streaming/batching
- Migration: replace .fetchall() with yield_per streaming, replace per-message INSERT+SAVEPOINT with batched inserts (5k/batch) with fallback to row-by-row on error, add progress logging
- Write path: call normalize_usage() in upsert_message() before saving to ensure input_tokens/output_tokens always present
- Read path: analytics queries now COALESCE across input_tokens/prompt_tokens and output_tokens/completion_tokens so historical data with OpenAI-format keys is visible
* fix: restore defensive timestamp conversion in migration
Re-add try/except around int(float(timestamp)) that was accidentally dropped. Without this, a non-numeric timestamp string would cause a TypeError on the subsequent comparison, breaking the entire upgrade.
* revert: remove changes to chat_messages.py
The non-streaming response handler was saving assistant messages without
their usage/token data. While the streaming handler correctly extracted
and saved usage information, the non-streaming path discarded it entirely.
This caused assistant messages from non-streaming completions to have
NULL usage in the chat_message table, making them invisible to the
analytics token aggregation queries and contributing to the '0 tokens'
display in Admin Panel Analytics.
Extract and normalize the usage data from the API response and include
it in the database upsert, matching the pattern already used by the
streaming handler.
