github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #4698] ValueError: Error raised by inference API HTTP code: 500, {"error":"failed to generate embedding"} #2958

New Issue
Closed
opened 2026-04-12 13:20:09 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 13:20:09 -05:00
Owner
Copy Link

Originally created by @uzumakinaruto19 on GitHub (May 29, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4698

          ValueError: Error raised by inference API HTTP code: 500, {"error":"failed to generate embedding"}

still I'm getting with the latest ollama docker

Hi folks this should be fixed now - please let me know if that's not the case

@jmorganca

only with the llama/ollama:0.1.32 version it works,
doesn't work with 0.1.28,37 and latest(as far as I checked)
Randomly it works, but most of the time fails

@mchiang0610, do you have any idea how to fix this?

Originally posted by @uzumakinaruto19 in https://github.com/ollama/ollama/issues/1577#issuecomment-2126700878

Originally created by @uzumakinaruto19 on GitHub (May 29, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4698 ValueError: Error raised by inference API HTTP code: 500, {"error":"failed to generate embedding"} still I'm getting with the latest ollama docker > Hi folks this should be fixed now - please let me know if that's not the case @jmorganca only with the llama/ollama:0.1.32 version it works, doesn't work with 0.1.28,37 and latest(as far as I checked) Randomly it works, but most of the time fails @mchiang0610, do you have any idea how to fix this? _Originally posted by @uzumakinaruto19 in https://github.com/ollama/ollama/issues/1577#issuecomment-2126700878_
GiteaMirror commented 2026-04-12 13:20:10 -05:00
Author
Owner
Copy Link

@Mgarbia commented on GitHub (Jun 17, 2024):

I'm getting the same error

<!-- gh-comment-id:2171969326 --> @Mgarbia commented on GitHub (Jun 17, 2024): I'm getting the same error
GiteaMirror commented 2026-04-12 13:20:11 -05:00
Author
Owner
Copy Link

@box9527 commented on GitHub (Jul 8, 2024):

I'm suffering this problem too.
After narrow down docker version, I can confirm that only the version before 0.1.32 can work normally in embedding time. Otherwise the latest layer to embedding can always get HTTP 500 error.

Docker info:
Server Version: 19.09.12

Host OS:
rhel 7.9

Logs in Ollama:
time=2024-07-08T09:33:20.300Z level=INFO source=routes.go:404 msg="embedding generation failed: do embedding request: Post "http://127.0.0.1:37391/embedding": EOF"
[GIN] 2024/07/08 - 09:33:20 | 500| 9.541230688s| xxx.xxx.xxx.xxx | POST "/api/embeddings"

Logs in Apps:
File "/usr/local/lib/python3.11/site-packages/langchain_core/vectorstores.py", line 138, in add_documents
return self.add_texts(texts, metadatas, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_community/vectorstores/pgvector.py", line 555, in add_texts
embeddings = self.embedding_function.embed_documents(list(texts))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 211, in embed_documents
embeddings = self._embed(instruction_pairs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 199, in _embed
return [self.process_emb_response(prompt) for prompt in iter]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 199, in
return [self.process_emb_response(prompt) for prompt in iter]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 173, in _process_emb_response
raise ValueError(
ValueError: Error raised by inference API HTTP code: 500, {"error":"failed to generate embedding"}

<!-- gh-comment-id:2213455128 --> @box9527 commented on GitHub (Jul 8, 2024): I'm suffering this problem too. After narrow down docker version, I can confirm that only the version before 0.1.32 can work normally in embedding time. Otherwise the latest layer to embedding can always get HTTP 500 error. Docker info: Server Version: 19.09.12 Host OS: rhel 7.9 Logs in Ollama: time=2024-07-08T09:33:20.300Z level=INFO source=routes.go:404 msg="embedding generation failed: do embedding request: Post \"http://127.0.0.1:37391/embedding\": EOF" [GIN] 2024/07/08 - 09:33:20 | 500| 9.541230688s| xxx.xxx.xxx.xxx | POST "/api/embeddings" Logs in Apps: File "/usr/local/lib/python3.11/site-packages/langchain_core/vectorstores.py", line 138, in add_documents return self.add_texts(texts, metadatas, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_community/vectorstores/pgvector.py", line 555, in add_texts embeddings = self.embedding_function.embed_documents(list(texts)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 211, in embed_documents embeddings = self._embed(instruction_pairs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 199, in _embed return [self._process_emb_response(prompt) for prompt in iter_] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 199, in <listcomp> return [self._process_emb_response(prompt) for prompt in iter_] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/langchain_community/embeddings/ollama.py", line 173, in _process_emb_response raise ValueError( ValueError: Error raised by inference API HTTP code: 500, {"error":"failed to generate embedding"}
GiteaMirror commented 2026-04-12 13:20:11 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Sep 13, 2024):

This seems like a problem in langchain? It should use the /api/embed endpoint and not /api/embeddings.

You can try it with:

curl http://localhost:11434/api/embed -d '{
  "model": "all-minilm",
  "input": "Why is the sky blue?"
}'

Also, you should check out the ollama python client. I have updated the README there to show how to use the new endpoint.

<!-- gh-comment-id:2347444041 --> @pdevine commented on GitHub (Sep 13, 2024): This seems like a problem in langchain? It should use the `/api/embed` endpoint and not `/api/embeddings`. You can try it with: ``` curl http://localhost:11434/api/embed -d '{ "model": "all-minilm", "input": "Why is the sky blue?" }' ``` Also, you should check out the [ollama python client](https://github.com/ollama/ollama-python). I have updated the README there to show how to use the new endpoint.
GiteaMirror commented 2026-04-12 13:20:11 -05:00
Author
Owner
Copy Link

@gabripo commented on GitHub (Nov 10, 2024):

same problem here, running Docker container with ollama/ollama image.

2024-11-10 12:18:04 goroutine 54 gp=0x400020c540 m=nil [IO wait]: 2024-11-10 12:18:04 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2024-11-10 12:18:04 runtime/proc.go:402 +0xc8 fp=0x40000a7550 sp=0x40000a7530 pc=0xaaaab8191008 2024-11-10 12:18:04 runtime.netpollblock(0x0?, 0xffffffff?, 0xff?) 2024-11-10 12:18:04 runtime/netpoll.go:573 +0x158 fp=0x40000a7590 sp=0x40000a7550 pc=0xaaaab8189468 2024-11-10 12:18:04 internal/poll.runtime_pollWait(0xffff9c617eb0, 0x72) 2024-11-10 12:18:04 runtime/netpoll.go:345 +0xa0 fp=0x40000a75c0 sp=0x40000a7590 pc=0xaaaab81bed80 2024-11-10 12:18:04 internal/poll.(*pollDesc).wait(0x40001ae080?, 0x400010f571?, 0x0) 2024-11-10 12:18:04 internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x40000a75f0 sp=0x40000a75c0 pc=0xaaaab820e728 2024-11-10 12:18:04 internal/poll.(*pollDesc).waitRead(...) 2024-11-10 12:18:04 internal/poll/fd_poll_runtime.go:89 2024-11-10 12:18:04 internal/poll.(*FD).Read(0x40001ae080, {0x400010f571, 0x1, 0x1}) 2024-11-10 12:18:04 internal/poll/fd_unix.go:164 +0x200 fp=0x40000a7690 sp=0x40000a75f0 pc=0xaaaab820f240 2024-11-10 12:18:04 net.(*netFD).Read(0x40001ae080, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 net/fd_posix.go:55 +0x28 fp=0x40000a76e0 sp=0x40000a7690 pc=0xaaaab8275978 2024-11-10 12:18:04 net.(*conn).Read(0x40001b0008, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 net/net.go:185 +0x34 fp=0x40000a7730 sp=0x40000a76e0 pc=0xaaaab827f444 2024-11-10 12:18:04 net.(*TCPConn).Read(0x0?, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 <autogenerated>:1 +0x2c fp=0x40000a7760 sp=0x40000a7730 pc=0xaaaab828ac3c 2024-11-10 12:18:04 net/http.(*connReader).backgroundRead(0x400010f560) 2024-11-10 12:18:04 net/http/server.go:681 +0x40 fp=0x40000a77b0 sp=0x40000a7760 pc=0xaaaab8366f50 2024-11-10 12:18:04 net/http.(*connReader).startBackgroundRead.gowrap2() 2024-11-10 12:18:04 net/http/server.go:677 +0x28 fp=0x40000a77d0 sp=0x40000a77b0 pc=0xaaaab8366e38 2024-11-10 12:18:04 runtime.goexit({}) 2024-11-10 12:18:04 runtime/asm_arm64.s:1222 +0x4 fp=0x40000a77d0 sp=0x40000a77d0 pc=0xaaaab81c4604 2024-11-10 12:18:04 created by net/http.(*connReader).startBackgroundRead in goroutine 24 2024-11-10 12:18:04 net/http/server.go:677 +0xc8 2024-11-10 12:18:05 time=2024-11-10T11:18:05.015Z level=ERROR source=routes.go:453 msg="embedding generation failed" error="do embedding request: Post \"http://127.0.0.1:38233/embedding\": EOF"

@pdevine which is the latest version of Langchain working, in you opinion?

Reformulating the question: when (and if) has api/embed to api/embeddings changed?

EDIT: it works as expected with langchain 0.1.20 (old but gold)

<!-- gh-comment-id:2466698009 --> @gabripo commented on GitHub (Nov 10, 2024): same problem here, running Docker container with ollama/ollama image. `2024-11-10 12:18:04 goroutine 54 gp=0x400020c540 m=nil [IO wait]: 2024-11-10 12:18:04 runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?) 2024-11-10 12:18:04 runtime/proc.go:402 +0xc8 fp=0x40000a7550 sp=0x40000a7530 pc=0xaaaab8191008 2024-11-10 12:18:04 runtime.netpollblock(0x0?, 0xffffffff?, 0xff?) 2024-11-10 12:18:04 runtime/netpoll.go:573 +0x158 fp=0x40000a7590 sp=0x40000a7550 pc=0xaaaab8189468 2024-11-10 12:18:04 internal/poll.runtime_pollWait(0xffff9c617eb0, 0x72) 2024-11-10 12:18:04 runtime/netpoll.go:345 +0xa0 fp=0x40000a75c0 sp=0x40000a7590 pc=0xaaaab81bed80 2024-11-10 12:18:04 internal/poll.(*pollDesc).wait(0x40001ae080?, 0x400010f571?, 0x0) 2024-11-10 12:18:04 internal/poll/fd_poll_runtime.go:84 +0x28 fp=0x40000a75f0 sp=0x40000a75c0 pc=0xaaaab820e728 2024-11-10 12:18:04 internal/poll.(*pollDesc).waitRead(...) 2024-11-10 12:18:04 internal/poll/fd_poll_runtime.go:89 2024-11-10 12:18:04 internal/poll.(*FD).Read(0x40001ae080, {0x400010f571, 0x1, 0x1}) 2024-11-10 12:18:04 internal/poll/fd_unix.go:164 +0x200 fp=0x40000a7690 sp=0x40000a75f0 pc=0xaaaab820f240 2024-11-10 12:18:04 net.(*netFD).Read(0x40001ae080, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 net/fd_posix.go:55 +0x28 fp=0x40000a76e0 sp=0x40000a7690 pc=0xaaaab8275978 2024-11-10 12:18:04 net.(*conn).Read(0x40001b0008, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 net/net.go:185 +0x34 fp=0x40000a7730 sp=0x40000a76e0 pc=0xaaaab827f444 2024-11-10 12:18:04 net.(*TCPConn).Read(0x0?, {0x400010f571?, 0x0?, 0x0?}) 2024-11-10 12:18:04 <autogenerated>:1 +0x2c fp=0x40000a7760 sp=0x40000a7730 pc=0xaaaab828ac3c 2024-11-10 12:18:04 net/http.(*connReader).backgroundRead(0x400010f560) 2024-11-10 12:18:04 net/http/server.go:681 +0x40 fp=0x40000a77b0 sp=0x40000a7760 pc=0xaaaab8366f50 2024-11-10 12:18:04 net/http.(*connReader).startBackgroundRead.gowrap2() 2024-11-10 12:18:04 net/http/server.go:677 +0x28 fp=0x40000a77d0 sp=0x40000a77b0 pc=0xaaaab8366e38 2024-11-10 12:18:04 runtime.goexit({}) 2024-11-10 12:18:04 runtime/asm_arm64.s:1222 +0x4 fp=0x40000a77d0 sp=0x40000a77d0 pc=0xaaaab81c4604 2024-11-10 12:18:04 created by net/http.(*connReader).startBackgroundRead in goroutine 24 2024-11-10 12:18:04 net/http/server.go:677 +0xc8 2024-11-10 12:18:05 time=2024-11-10T11:18:05.015Z level=ERROR source=routes.go:453 msg="embedding generation failed" error="do embedding request: Post \"http://127.0.0.1:38233/embedding\": EOF"` @pdevine which is the latest version of Langchain working, in you opinion? Reformulating the question: when (and if) has api/embed to api/embeddings changed? EDIT: it works as expected with langchain 0.1.20 (old but gold)
GiteaMirror commented 2026-04-12 13:20:12 -05:00
Author
Owner
Copy Link

@FanZhang91 commented on GitHub (Mar 31, 2025):

ollama==0.3.13, langchain==0.1.20. I have the same problem. How to fix this problem?

<!-- gh-comment-id:2766064395 --> @FanZhang91 commented on GitHub (Mar 31, 2025): ollama==0.3.13, langchain==0.1.20. I have the same problem. How to fix this problem?
GiteaMirror commented 2026-04-12 13:20:12 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Mar 31, 2025):

@FanZhang91 the latest ollama is 0.6.3.

<!-- gh-comment-id:2766822191 --> @pdevine commented on GitHub (Mar 31, 2025): @FanZhang91 the latest ollama is `0.6.3`.
GiteaMirror commented 2026-04-12 13:20:13 -05:00
Author
Owner
Copy Link

@gabripo commented on GitHub (Apr 4, 2025):

@FanZhang91 a downgrade could be an option

check my previous comment

<!-- gh-comment-id:2777858483 --> @gabripo commented on GitHub (Apr 4, 2025): @FanZhang91 a downgrade could be an option check my previous comment
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2958
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?