github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 13:47:08 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #4296] Feature Request: Generate embedding for images using /api/embeddings endpoint #2682

New Issue
Open
opened 2026-04-12 13:00:38 -05:00 by GiteaMirror · 12 comments
GiteaMirror commented 2026-04-12 13:00:38 -05:00
Owner
Copy Link

Originally created by @eamonburns on GitHub (May 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4296

I think this feature would make a lot of sense.
I can add an "images" field in requests to /api/generate when using a multi-modal model, why can't I do the same for requests to /api/embeddings?

curl http://localhost:11434/api/embeddings -d '{
  "model": "llava",
  "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0A..."]
}'

Possible use case:
Store embeddings of images in a vector database for enhanced personal image search

Originally created by @eamonburns on GitHub (May 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4296 I think this feature would make a lot of sense. I can add an `"images"` field in requests to `/api/generate` when using a multi-modal model, why can't I do the same for requests to `/api/embeddings`? ```sh curl http://localhost:11434/api/embeddings -d '{ "model": "llava", "images": ["iVBORw0KGgoAAAANSUhEUgAAAG0A..."] }' ``` Possible use case: Store embeddings of images in a vector database for enhanced personal image search
GiteaMirror added the feature requestapi labels 2026-04-12 13:00:38 -05:00
GiteaMirror commented 2026-04-12 13:00:38 -05:00
Author
Owner
Copy Link

@PaulCapestany commented on GitHub (May 9, 2024):

I'd be very interested in this feature as well. As a potential workaround I was thinking of just using llava via /api/generate endpoint to generate a textual description of the image, and then making an embedding of that text

<!-- gh-comment-id:2103470811 --> @PaulCapestany commented on GitHub (May 9, 2024): I'd be very interested in this feature as well. As a potential workaround I was thinking of just using llava via `/api/generate` endpoint to generate a textual description of the image, and then making an embedding of that text
GiteaMirror commented 2026-04-12 13:00:39 -05:00
Author
Owner
Copy Link

@zty0510 commented on GitHub (May 11, 2024):

I'm also expecting this feature with which I can get text embedding and image embedding respectively from Llava, so that i can calculate the semantic similarity between them.

<!-- gh-comment-id:2105940386 --> @zty0510 commented on GitHub (May 11, 2024): I'm also expecting this feature with which I can get text embedding and image embedding respectively from Llava, so that i can calculate the semantic similarity between them.
GiteaMirror commented 2026-04-12 13:00:39 -05:00
Author
Owner
Copy Link

@kendelljoseph commented on GitHub (May 12, 2024):

I think this makes sense too.

<!-- gh-comment-id:2106118198 --> @kendelljoseph commented on GitHub (May 12, 2024): I think this makes sense too.
GiteaMirror commented 2026-04-12 13:00:39 -05:00
Author
Owner
Copy Link

@JoanFM commented on GitHub (May 24, 2024):

I think it would be really nice, not sure however if llama.cpp backend supports this yet

<!-- gh-comment-id:2129428865 --> @JoanFM commented on GitHub (May 24, 2024): I think it would be really nice, not sure however if llama.cpp backend supports this yet
GiteaMirror commented 2026-04-12 13:00:39 -05:00
Author
Owner
Copy Link

@pulinagrawal commented on GitHub (Jun 24, 2024):

upvote

<!-- gh-comment-id:2186996242 --> @pulinagrawal commented on GitHub (Jun 24, 2024): upvote
GiteaMirror commented 2026-04-12 13:00:40 -05:00
Author
Owner
Copy Link

@daboe01 commented on GitHub (Sep 12, 2024):

+1

<!-- gh-comment-id:2346121112 --> @daboe01 commented on GitHub (Sep 12, 2024): +1
GiteaMirror commented 2026-04-12 13:00:40 -05:00
Author
Owner
Copy Link

@Philcartmell commented on GitHub (Sep 20, 2024):

+1

<!-- gh-comment-id:2362949630 --> @Philcartmell commented on GitHub (Sep 20, 2024): +1
GiteaMirror commented 2026-04-12 13:00:41 -05:00
Author
Owner
Copy Link

@om2468 commented on GitHub (Nov 12, 2024):

+1

<!-- gh-comment-id:2471336742 --> @om2468 commented on GitHub (Nov 12, 2024): +1
GiteaMirror commented 2026-04-12 13:00:41 -05:00
Author
Owner
Copy Link

@luizcelsojr commented on GitHub (Nov 16, 2024):

Please!

<!-- gh-comment-id:2480540498 --> @luizcelsojr commented on GitHub (Nov 16, 2024): Please!
GiteaMirror commented 2026-04-12 13:00:41 -05:00
Author
Owner
Copy Link

@HeroBrine1st commented on GitHub (Nov 16, 2024):

Please stop sending "+1" and other "bump" messages without new info as you're triggering notifications and giving a false hope that this issue is updated.

Use 👍 reaction on first message as a replacement to bump messages.

<!-- gh-comment-id:2480563224 --> @HeroBrine1st commented on GitHub (Nov 16, 2024): Please stop sending "+1" and other "bump" messages without new info as you're triggering notifications and giving a false hope that this issue is updated. Use :+1: reaction on first message as a replacement to bump messages.
GiteaMirror commented 2026-04-12 13:00:42 -05:00
Author
Owner
Copy Link

@daboe01 commented on GitHub (Nov 30, 2024):

most likely this will not be useful without an e5-tuned multimodal embedding model.

<!-- gh-comment-id:2509270985 --> @daboe01 commented on GitHub (Nov 30, 2024): most likely this will not be useful without an e5-tuned multimodal embedding model.
GiteaMirror commented 2026-04-12 13:00:43 -05:00
Author
Owner
Copy Link

@Capsar commented on GitHub (Sep 2, 2025):

Another nice option would be to add an option in the chat endpoint to also return the embedding as part of the output.

<!-- gh-comment-id:3244557667 --> @Capsar commented on GitHub (Sep 2, 2025): Another nice option would be to add an option in the chat endpoint to also return the embedding as part of the output.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label api feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2682
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?