github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #3642] [CLOSED] Batch embeddings #11230

New Issue
Closed
opened 2026-04-12 23:24:56 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-12 23:24:56 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/3642
Author: @jmorganca
Created: 4/15/2024
Status: ❌ Closed

Base: main ← Head: jmorganca/batch-embeddings

📝 Commits (1)

📊 Changes

8 files changed (+241 additions, -70 deletions)

View changed files

📝 api/types.go (+6 -4)
📝 docs/api.md (+32 -1)
➕ integration/embedding_test.go (+64 -0)
📝 integration/utils_test.go (+83 -0)
📝 llm/ext_server/server.cpp (+12 -39)
📝 llm/server.go (+12 -10)
📝 server/routes.go (+30 -14)
📝 server/sched_test.go (+2 -2)

📄 Description

This change makes it possible to generate embeddings for more than one string at once, making it much faster to process many chunks simultaneously with the /api/embeddings endpoint:

 curl http://localhost:11434/api/embeddings -d '{
  "model": "all-minilm",
  "prompt_batch": [
    "Here is an article about llamas...",
    "Here is another article about llamas...",
    "Here is a third article about llamas...",
    ...
    "Here is yet another article..."
  ]
}'

The API will then respond with an embeddings field that contains a list of vector embeddings:

{
  "embedding_batch": [
    [0.010080209001898766 ... 0.308726966381073],
    ...
    [-0.15468695759773254 ... 0.09876912087202072]
  ]
}

TODO:

  • Finalize API
  • Finalize behavior when empty prompt is provided
  • Add more tests

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/3642 **Author:** [@jmorganca](https://github.com/jmorganca) **Created:** 4/15/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `jmorganca/batch-embeddings` --- ### 📝 Commits (1) - [`ad7e641`](https://github.com/ollama/ollama/commit/ad7e64181525cd22f312f5d053aaf4eb84685b0b) add batch embeddings ### 📊 Changes **8 files changed** (+241 additions, -70 deletions) <details> <summary>View changed files</summary> 📝 `api/types.go` (+6 -4) 📝 `docs/api.md` (+32 -1) ➕ `integration/embedding_test.go` (+64 -0) 📝 `integration/utils_test.go` (+83 -0) 📝 `llm/ext_server/server.cpp` (+12 -39) 📝 `llm/server.go` (+12 -10) 📝 `server/routes.go` (+30 -14) 📝 `server/sched_test.go` (+2 -2) </details> ### 📄 Description This change makes it possible to generate embeddings for more than one string at once, making it much faster to process many chunks simultaneously with the `/api/embeddings` endpoint: ``` curl http://localhost:11434/api/embeddings -d '{ "model": "all-minilm", "prompt_batch": [ "Here is an article about llamas...", "Here is another article about llamas...", "Here is a third article about llamas...", ... "Here is yet another article..." ] }' ``` The API will then respond with an `embeddings` field that contains a list of vector embeddings: ``` { "embedding_batch": [ [0.010080209001898766 ... 0.308726966381073], ... [-0.15468695759773254 ... 0.09876912087202072] ] } ``` TODO: - [ ] Finalize API - [ ] Finalize behavior when empty prompt is provided - [ ] Add more tests --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-12 23:24:56 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#11230
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?