github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #10728] llm/api/runner: Add ability to generate multimodal embeddings via embeddings endpoints #13346

New Issue
Open
opened 2026-04-13 00:24:30 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-13 00:24:30 -05:00
Owner
Copy Link

Original Pull Request: https://github.com/ollama/ollama/pull/10728

State: open
Merged: No

Inre: https://github.com/ollama/ollama/issues/5304

Explain the problem you are trying to solve, not what you are trying to do.
I'm trying to generate embeddings of pdfs so that I can compare embeddings of webpage descriptions with embeddings for their pdfs and merge results.

Explain how the change will be tested.
I tested the code with added examples to the api example clients but I did not commit this code as that looked like mostly minimalistic in scope

Happy to change anything, add tests or address feedback but wanted to see if there was a reason this wasn't added already.

go run main.go foo.png
[-1.7418279647827148 1.9501290321350098 -1.991140604019165 3.5079853534698486 0.8471968173980713 -4.52871036529541 3.787987232208252 1.8196138143539429 -0.24654169380664825 -2.8415348529815674 -3.5892462730407715 -3.5571892261505127 0.5544070601463318 1.1104142665863037 -3.033186674118042 4.001904487609863

A simple test using llava can be done with...

	imgData, err := os.ReadFile("foo.png")
	if err != nil {
		log.Fatal(err)
	}

	url, err := url.Parse("http://localhost:11434")
	if err != nil {
		log.Fatal(err)
	}
	client := api.NewClient(url, http.DefaultClient)

	req := &api.EmbeddingRequest{
		Model: "llava",
		Image: api.ImageData(imgData),
	}

	ctx := context.Background()
	embedResponse, err := client.Embeddings(ctx, req)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(embedResponse.Embedding)
**Original Pull Request:** https://github.com/ollama/ollama/pull/10728 **State:** open **Merged:** No --- Inre: https://github.com/ollama/ollama/issues/5304 Explain the problem you are trying to solve, not what you are trying to do. I'm trying to generate embeddings of pdfs so that I can compare embeddings of webpage descriptions with embeddings for their pdfs and merge results. Explain how the change will be tested. I tested the code with added examples to the api example clients but I did not commit this code as that looked like mostly minimalistic in scope Happy to change anything, add tests or address feedback but wanted to see if there was a reason this wasn't added already. ``` go run main.go foo.png [-1.7418279647827148 1.9501290321350098 -1.991140604019165 3.5079853534698486 0.8471968173980713 -4.52871036529541 3.787987232208252 1.8196138143539429 -0.24654169380664825 -2.8415348529815674 -3.5892462730407715 -3.5571892261505127 0.5544070601463318 1.1104142665863037 -3.033186674118042 4.001904487609863 ``` A simple test using llava can be done with... ``` imgData, err := os.ReadFile("foo.png") if err != nil { log.Fatal(err) } url, err := url.Parse("http://localhost:11434") if err != nil { log.Fatal(err) } client := api.NewClient(url, http.DefaultClient) req := &api.EmbeddingRequest{ Model: "llava", Image: api.ImageData(imgData), } ctx := context.Background() embedResponse, err := client.Embeddings(ctx, req) if err != nil { log.Fatal(err) } fmt.Println(embedResponse.Embedding) ```
GiteaMirror added the pull-request label 2026-04-13 00:24:30 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#13346
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?