[PR #14659] [MERGED] **fix**: do_picture_description feature when using Docling as an external document parser #62396

New Issue

2026-05-06T06:32:44-05:00

GiteaMirror commented

2026-05-06 06:32:44 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/14659
Author: @vaclcer
Created: 6/4/2025
Status: ✅ Merged
Merged: 6/4/2025
Merged by: @tjbck

Base: dev ← Head: vaclavs-picture-description

📝 Commits (5)

8644e81 feat(loader): add picture description configuration for DoclingLoader
c4278f4 fix description vs classification mismatch
c71236b feat(loader): enhance picture description prompt for improved detail and clarity
0b3719c This commit fixes do_picture_description feature when using Docling as an external document parser.
9772c18 fix(loader): remove deprecated picture description configuration

📊 Changes

1 file changed (+7 additions, -7 deletions)

View changed files

📝 backend/open_webui/retrieval/loaders/main.py (+7 -7)

📄 Description

Changelog Entry

Description

do_picture_description feature of Docling was implemented, but not working properly

Added

so far, I have added only an hard-coded defaults for Docling's prefered VLM "HuggingFaceTB/SmolVLM-256M-Instruct" including a prompt to extract the information.

Changed

There were mismatch between do_picture_classification and do_picture_description. Fixed.

Deprecated

none

Removed

none

Fixed

There were mismatch between do_picture_classification and do_picture_description. Fixed.

Breaking Changes

none

Additional Information

In next iteration, I should make (at least) the model and the prompt configurable. There is also Docling's feature to use remote API for picture descriptions. That should be added too.

Screenshots or Videos

Input:

Output:

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/14659 **Author:** [@vaclcer](https://github.com/vaclcer) **Created:** 6/4/2025 **Status:** ✅ Merged **Merged:** 6/4/2025 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `vaclavs-picture-description` --- ### 📝 Commits (5) - [`8644e81`](https://github.com/open-webui/open-webui/commit/8644e81a1c352c127d3e1decef6fdc389c6f81dc) feat(loader): add picture description configuration for DoclingLoader - [`c4278f4`](https://github.com/open-webui/open-webui/commit/c4278f4784e31a0f0819478292864e5111faf452) fix description vs classification mismatch - [`c71236b`](https://github.com/open-webui/open-webui/commit/c71236ba07fbed0abeb132b5d1ba65cadf050079) feat(loader): enhance picture description prompt for improved detail and clarity - [`0b3719c`](https://github.com/open-webui/open-webui/commit/0b3719cc65b9f3a4742a25b5599882cc34099200) This commit fixes do_picture_description feature when using Docling as an external document parser. - [`9772c18`](https://github.com/open-webui/open-webui/commit/9772c18b20d826396b3e600452726f4aafc64b7c) fix(loader): remove deprecated picture description configuration ### 📊 Changes **1 file changed** (+7 additions, -7 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/retrieval/loaders/main.py` (+7 -7) </details> ### 📄 Description # Changelog Entry ### Description `do_picture_description` feature of Docling was implemented, but not working properly ### Added so far, I have added only an hard-coded defaults for Docling's prefered VLM "HuggingFaceTB/SmolVLM-256M-Instruct" including a prompt to extract the information. ### Changed There were mismatch between `do_picture_classification` and `do_picture_description`. Fixed. ### Deprecated none ### Removed none ### Fixed There were mismatch between `do_picture_classification` and `do_picture_description`. Fixed. ### Breaking Changes none --- ### Additional Information In next iteration, I should make (at least) the model and the prompt configurable. There is also Docling's feature to use remote API for picture descriptions. That should be added too. ### Screenshots or Videos Input: ![one](https://github.com/user-attachments/assets/e9bf5788-7158-4669-822f-cc1239428a41) Output: ![2](https://github.com/user-attachments/assets/dbd48bcd-4307-4dfa-8c44-e5d1ef35a86c) ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-06 06:32:44 -05:00

GiteaMirror closed this issue

2026-05-06 06:32:47 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#62396