[PR #24777] [CLOSED] fix: prefer source_url for document citation links #98872

New Issue

GiteaMirror · 2026-05-16T01:47:25-05:00

GiteaMirror commented

2026-05-16 01:47:25 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24777
Author: @DuncanSARapp
Created: 5/15/2026
Status: ❌ Closed

Base: dev ← Head: codex/source-url-citations

📝 Commits (1)

023b1e7 fix: prefer source url for document citations

📊 Changes

7 files changed (+337 additions, -24 deletions)

View changed files

➕ backend/open_webui/retrieval/source.py (+95 -0)
📝 backend/open_webui/retrieval/utils.py (+13 -1)
📝 backend/open_webui/routers/retrieval.py (+13 -1)
➕ backend/open_webui/test/retrieval/test_source.py (+148 -0)
📝 src/lib/components/chat/Messages/Citations.svelte (+28 -4)
📝 src/lib/components/chat/Messages/Citations/CitationModal.svelte (+39 -17)
📝 src/lib/components/chat/Messages/ContentRenderer.svelte (+1 -1)

📄 Description

Pull Request Checklist

Note to first-time contributors: A discussion has been opened before this PR: https://github.com/open-webui/open-webui/discussions/24773

Before submitting, make sure you've checked the following:

Linked Issue/Discussion: This PR references an existing Issue or Discussion — Closes #24775; Relates to https://github.com/open-webui/open-webui/discussions/24773.
Target branch: The pull request targets the dev branch.
Description: A concise description of the changes is provided below.
Changelog: A changelog entry following Keep a Changelog format is included at the bottom.
Documentation: Documentation impact is described below. This uses existing document metadata and introduces no new public API endpoint.
Dependencies: No new or updated dependencies are introduced.
Testing: Manual/container tests were performed and edge cases are listed below.
No Unchecked AI Code: This PR was AI-assisted, but the changes were manually reviewed and manually tested.
Self-Review: A self-review of the code has been performed.
Architecture: This keeps existing fallback behavior and treats source_url only as citation presentation metadata.
Git Hygiene: The PR is atomic, rebased on dev, and contains no unrelated commits.
Title Prefix: The PR title uses the fix prefix.

Description

This PR updates citation URL resolution so API-uploaded documents can preserve their authoritative source URL.

When citation metadata includes a source URL, Open WebUI now prefers that URL for citation links before falling back to the internal file endpoint. This keeps current behavior for normal file uploads while improving API ingestion workflows where documents originate from another system.

No new dependencies are added.

Reproduction

Before this change:

Upload a document through the API with metadata:
{"source_url": "https://gitlab.example.com/group/project/-/blob/main/docs/spec.md"}
Process the file and use it in chat/RAG.
Open the generated citation.

Expected: the citation opens the provided source_url.
Actual: the citation can fall back to Open WebUI's internal file endpoint.

Changed

Adds a small source URL resolver for citation metadata.
Preserves existing internal file citation fallback behavior when source_url is absent or invalid.
Uses source_url only as a presentation hint for citation links.

Testing

Tested locally with:

Podman build:
- podman build --env NODE_OPTIONS=--max-old-space-size=4096 --build-arg USE_SLIM=true --build-arg BUILD_HASH=023b1e735 -t localhost/open-webui-source-url:source-url-citations .
Runtime:
- OLLAMA_BASE_URL=http://host.containers.internal:11434
Healthcheck:
- GET /health
- result: {"status": true}
Unit tests inside the container:
- cd /app/backend && python -m pytest open_webui/test/retrieval/test_source.py
- result: 37 passed
Edge cases covered by tests:
- absent, empty, null, and non-string source URL metadata
- valid http and https source URLs
- local, IPv4, IPv6, and IDN hostnames
- relative URLs and missing-host URLs
- rejected raw whitespace, invalid ports, userinfo, parser-confusing backslashes, and malformed hosts
- rejected javascript:, data:, file:, blob:, and mailto: URLs
- encoded URL characters

Documentation

Documentation impact: this uses the existing document metadata convention source_url when present.
No new public API endpoint is introduced.
Existing citation fallback behavior remains unchanged when source_url is absent or invalid.

Security Considerations

source_url is only used as a citation presentation hint.
Only absolute http and https URLs with a valid host are accepted.
Invalid or unsupported schemes fall back to existing citation behavior.
source_url is never used for authorization, file lookup, deletion, backend fetches, or access decisions.
External links use the existing Svelte rendering path and rel="noopener noreferrer" when opened with target="_blank".
No new automatic external requests are introduced; document citations using source_url are excluded from the Google favicon lookup used for regular URL citations.

AI-assisted Code Note

This PR was AI-assisted, but the changes were manually reviewed and manually tested by the author. The implementation was verified in a running Open WebUI container, including the steps listed above.

Changelog Entry

Description

Prefer provided source URLs for document citation links before falling back to internal file links.

Added

Source URL resolution helper and tests.

Changed

Citation rendering now uses resolved source URLs when available.

Deprecated

None.

Removed

None.

Fixed

API-uploaded documents with source URLs can now show citations that link to the original source instead of only the internal file endpoint.

Security

No new automatic external requests are introduced by this change.
Document citations that use source_url are not added to the Google favicon lookup used for regular URL citations.

Breaking Changes

None.

Additional Information

This PR targets dev.
No new dependencies.
Related discussion: https://github.com/open-webui/open-webui/discussions/24773
Closes [PR #18390] [CLOSED] fix: Docling RAG - Fix typo in UI parameter name (#24775)

Screenshots or Videos

No screenshot included. This is citation metadata resolution behavior rather than a new UI surface.

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24777 **Author:** [@DuncanSARapp](https://github.com/DuncanSARapp) **Created:** 5/15/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `codex/source-url-citations` --- ### 📝 Commits (1) - [`023b1e7`](https://github.com/open-webui/open-webui/commit/023b1e735af30443f236ccb9ea7ce67e41891fb1) fix: prefer source url for document citations ### 📊 Changes **7 files changed** (+337 additions, -24 deletions) <details> <summary>View changed files</summary> ➕ `backend/open_webui/retrieval/source.py` (+95 -0) 📝 `backend/open_webui/retrieval/utils.py` (+13 -1) 📝 `backend/open_webui/routers/retrieval.py` (+13 -1) ➕ `backend/open_webui/test/retrieval/test_source.py` (+148 -0) 📝 `src/lib/components/chat/Messages/Citations.svelte` (+28 -4) 📝 `src/lib/components/chat/Messages/Citations/CitationModal.svelte` (+39 -17) 📝 `src/lib/components/chat/Messages/ContentRenderer.svelte` (+1 -1) </details> ### 📄 Description  # Pull Request Checklist ### Note to first-time contributors: A discussion has been opened before this PR: https://github.com/open-webui/open-webui/discussions/24773 **Before submitting, make sure you've checked the following:** - [x] **Linked Issue/Discussion:** This PR references an existing Issue or Discussion — Closes #24775; Relates to https://github.com/open-webui/open-webui/discussions/24773. - [x] **Target branch:** The pull request targets the `dev` branch. - [x] **Description:** A concise description of the changes is provided below. - [x] **Changelog:** A changelog entry following Keep a Changelog format is included at the bottom. - [x] **Documentation:** Documentation impact is described below. This uses existing document metadata and introduces no new public API endpoint. - [x] **Dependencies:** No new or updated dependencies are introduced. - [x] **Testing:** Manual/container tests were performed and edge cases are listed below. - [x] **No Unchecked AI Code:** This PR was AI-assisted, but the changes were manually reviewed and manually tested. - [x] **Self-Review:** A self-review of the code has been performed. - [x] **Architecture:** This keeps existing fallback behavior and treats `source_url` only as citation presentation metadata. - [x] **Git Hygiene:** The PR is atomic, rebased on `dev`, and contains no unrelated commits. - [x] **Title Prefix:** The PR title uses the `fix` prefix. ## Description This PR updates citation URL resolution so API-uploaded documents can preserve their authoritative source URL. When citation metadata includes a source URL, Open WebUI now prefers that URL for citation links before falling back to the internal file endpoint. This keeps current behavior for normal file uploads while improving API ingestion workflows where documents originate from another system. No new dependencies are added. ## Reproduction Before this change: 1. Upload a document through the API with metadata: `{"source_url": "https://gitlab.example.com/group/project/-/blob/main/docs/spec.md"}` 2. Process the file and use it in chat/RAG. 3. Open the generated citation. Expected: the citation opens the provided `source_url`. Actual: the citation can fall back to Open WebUI's internal file endpoint. ## Changed - Adds a small source URL resolver for citation metadata. - Preserves existing internal file citation fallback behavior when `source_url` is absent or invalid. - Uses `source_url` only as a presentation hint for citation links. ## Testing Tested locally with: - Podman build: - `podman build --env NODE_OPTIONS=--max-old-space-size=4096 --build-arg USE_SLIM=true --build-arg BUILD_HASH=023b1e735 -t localhost/open-webui-source-url:source-url-citations .` - Runtime: - `OLLAMA_BASE_URL=http://host.containers.internal:11434` - Healthcheck: - `GET /health` - result: `{"status": true}` - Unit tests inside the container: - `cd /app/backend && python -m pytest open_webui/test/retrieval/test_source.py` - result: `37 passed` - Edge cases covered by tests: - absent, empty, null, and non-string source URL metadata - valid `http` and `https` source URLs - local, IPv4, IPv6, and IDN hostnames - relative URLs and missing-host URLs - rejected raw whitespace, invalid ports, userinfo, parser-confusing backslashes, and malformed hosts - rejected `javascript:`, `data:`, `file:`, `blob:`, and `mailto:` URLs - encoded URL characters ## Documentation - Documentation impact: this uses the existing document metadata convention `source_url` when present. - No new public API endpoint is introduced. - Existing citation fallback behavior remains unchanged when `source_url` is absent or invalid. ## Security Considerations - `source_url` is only used as a citation presentation hint. - Only absolute `http` and `https` URLs with a valid host are accepted. - Invalid or unsupported schemes fall back to existing citation behavior. - `source_url` is never used for authorization, file lookup, deletion, backend fetches, or access decisions. - External links use the existing Svelte rendering path and `rel="noopener noreferrer"` when opened with `target="_blank"`. - No new automatic external requests are introduced; document citations using `source_url` are excluded from the Google favicon lookup used for regular URL citations. ## AI-assisted Code Note This PR was AI-assisted, but the changes were manually reviewed and manually tested by the author. The implementation was verified in a running Open WebUI container, including the steps listed above. # Changelog Entry ### Description - Prefer provided source URLs for document citation links before falling back to internal file links. ### Added - Source URL resolution helper and tests. ### Changed - Citation rendering now uses resolved source URLs when available. ### Deprecated - None. ### Removed - None. ### Fixed - API-uploaded documents with source URLs can now show citations that link to the original source instead of only the internal file endpoint. ### Security - No new automatic external requests are introduced by this change. - Document citations that use `source_url` are not added to the Google favicon lookup used for regular URL citations. ### Breaking Changes - None. --- ### Additional Information - This PR targets `dev`. - No new dependencies. - Related discussion: https://github.com/open-webui/open-webui/discussions/24773 - Closes #24775 ### Screenshots or Videos No screenshot included. This is citation metadata resolution behavior rather than a new UI surface. ### Contributor License Agreement  - [x] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-16 01:47:25 -05:00

GiteaMirror closed this issue

2026-05-16 01:47:33 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#98872