github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #10971] [Qwen-2.5vl] Support Video Attachments for conversation #7230

New Issue
Closed
opened 2026-04-12 19:14:50 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 19:14:50 -05:00
Owner
Copy Link

Originally created by @Notbici on GitHub (Jun 4, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10971

Qwen-2.5vl supports video and on their huggingface they also have a demo mode for this if you want to try it out.

Here's an example model for reference and ollama already has this up on its repository https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct

I believe there is some torch level wiring required to feed a video into the model, is this something Ollama can support?

Basically making it possible for us with Ollama's API attach a 'video' input to a message, similar to how we do images.

Originally created by @Notbici on GitHub (Jun 4, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10971 Qwen-2.5vl supports video and on their [huggingface](https://huggingface.co/spaces/Qwen/Qwen2.5-VL-32B-Instruct) they also have a demo mode for this if you want to try it out. Here's an example model for reference and ollama already has this up on its repository https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct I believe there is some torch level wiring required to feed a video into the model, is this something Ollama can support? Basically making it possible for us with Ollama's API attach a 'video' input to a message, similar to how we do images.
GiteaMirror added the feature request label 2026-04-12 19:14:50 -05:00
GiteaMirror commented 2026-04-12 19:14:53 -05:00
Author
Owner
Copy Link

@GreazySpoon commented on GitHub (Jun 10, 2025):

I think, you simply need to specify the mime type and feed it the base64 of the video. thats the required "wiring" you talking about i think.

<!-- gh-comment-id:2958655800 --> @GreazySpoon commented on GitHub (Jun 10, 2025): I think, you simply need to specify the mime type and feed it the base64 of the video. thats the required "wiring" you talking about i think.
GiteaMirror commented 2026-04-12 19:14:56 -05:00
Author
Owner
Copy Link

@Notbici commented on GitHub (Jun 10, 2025):

I think, you simply need to specify the mime type and feed it the base64 of the video. thats the required "wiring" you talking about i think.

Not sure if I did it right, though when I feed a base64 image to ollama it refuses with:

failed to process inputs: image: unknown

<!-- gh-comment-id:2960810190 --> @Notbici commented on GitHub (Jun 10, 2025): > I think, you simply need to specify the mime type and feed it the base64 of the video. thats the required "wiring" you talking about i think. Not sure if I did it right, though when I feed a base64 image to ollama it refuses with: > failed to process inputs: image: unknown
GiteaMirror commented 2026-04-12 19:14:57 -05:00
Author
Owner
Copy Link

@bmabir17 commented on GitHub (Jun 19, 2025):

response = client.chat.completions.create(
            model=f"{model_name}",  # Use the latest vision-capable model
            messages=[
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": master_prompt},
                        {
                            "type": "image_url",
                            "image_url": {
                                "url": f"data:video/mp4;base64,{base64_video}"
                            },
                        },
                    ],
                }
            ],
            # Use JSON mode for reliable, structured output
            response_format={"type": "json_object"},
            temperature=0.7, # A little creativity is good for analysis
        )

@Yami-Bitshark I tried the above with video file converted to base64
Is this the correct mime type to feed video into ollama's openAI compatible api?

I tried the code above but it gives the following error

Error code: 400 - {'error': {'message': 'invalid image input', 'type': 'invalid_request_error', 'param': None, 'code': None}}

If the video can not be directly fed into the model as base64, do we have to convert the video into image frames and then add all those frames as base64 images?

<!-- gh-comment-id:2987591328 --> @bmabir17 commented on GitHub (Jun 19, 2025): ```python response = client.chat.completions.create( model=f"{model_name}", # Use the latest vision-capable model messages=[ { "role": "user", "content": [ {"type": "text", "text": master_prompt}, { "type": "image_url", "image_url": { "url": f"data:video/mp4;base64,{base64_video}" }, }, ], } ], # Use JSON mode for reliable, structured output response_format={"type": "json_object"}, temperature=0.7, # A little creativity is good for analysis ) ``` @Yami-Bitshark I tried the above with video file converted to base64 Is this the correct mime type to feed video into ollama's openAI compatible api? I tried the code above but it gives the following error ``` Error code: 400 - {'error': {'message': 'invalid image input', 'type': 'invalid_request_error', 'param': None, 'code': None}} ``` If the video can not be directly fed into the model as base64, do we have to convert the video into image frames and then add all those frames as base64 images?
GiteaMirror commented 2026-04-12 19:14:58 -05:00
Author
Owner
Copy Link

@Notbici commented on GitHub (Jun 26, 2025):

@rick-github would this be remotely possible without a substantial change to Ollama?
Basically in essence, passing a video as an attachment for the model to process instead of just images.

<!-- gh-comment-id:3009415504 --> @Notbici commented on GitHub (Jun 26, 2025): @rick-github would this be remotely possible without a substantial change to Ollama? Basically in essence, passing a video as an attachment for the model to process instead of just images.
GiteaMirror commented 2026-04-12 19:14:59 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 26, 2025):

Not currently. Supporting more modes of input and other modals is on the roadmap for ollama, but there's no schedule for releasing those features. The only way to process video at the moment would be to decompose to frames and use the image processor.

<!-- gh-comment-id:3009469523 --> @rick-github commented on GitHub (Jun 26, 2025): Not currently. Supporting more modes of input and other modals is on the roadmap for ollama, but there's no schedule for releasing those features. The only way to process video at the moment would be to decompose to frames and use the image processor.
GiteaMirror commented 2026-04-12 19:15:00 -05:00
Author
Owner
Copy Link

@dhandhalyabhavik commented on GitHub (Jul 10, 2025):

Bro, @rick-github

Does ollama resizes image? because I tried with Qwen-2.5vl to detect object, but its returning wrong coordinates.

<!-- gh-comment-id:3058337328 --> @dhandhalyabhavik commented on GitHub (Jul 10, 2025): Bro, @rick-github Does ollama resizes image? because I tried with Qwen-2.5vl to detect object, but its returning wrong coordinates.
GiteaMirror commented 2026-04-12 19:15:00 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jul 10, 2025):

Does ollama resizes image?

https://github.com/ollama/ollama/issues/11217#issuecomment-3013302452

<!-- gh-comment-id:3058345441 --> @rick-github commented on GitHub (Jul 10, 2025): > Does ollama resizes image? https://github.com/ollama/ollama/issues/11217#issuecomment-3013302452
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#7230
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?