[GH-ISSUE #6435] 0.3.6 /api/embed return 500 if more items are provided in input #4045

Closed
opened 2026-04-12 14:56:13 -05:00 by GiteaMirror · 5 comments
Owner

Originally created by @davidliudev on GitHub (Aug 20, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6435

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

This does not happen on 0.3.4. It only happen on the latest 0.3.6 (Haven't test on 0.3.5).

If I reduce the payload to a single item it is all good. It happens when I put more items.

I have to downgrade to the 0.3.4 until this is fixed.

Here is the log:
time=2024-08-20T13:38:29.761+08:00 level=ERROR source=routes.go:394 msg="embedding generation failed" error="health resp: Get "http://127.0.0.1:56157/health": read tcp 127.0.0.1:56190->127.0.0.1:56157: wsarecv: An existing connection was forcibly closed by the remote host."

Sample payload:

{
  "model" : "nomic-embed-text",
  "input" : [ "BREAKFAST", "MAPLE LEAVES", "SCENERY", "ROKKO MOUNTAIN", "SUNSET", "HOT SPRINGS", "PEAK", "NARITA AIRPORT", "SPEAKER", "MORNING", "MOUNTAIN", "RED-EYE FLIGHT", "STRING QUARTET", "AIR", "AUTUMN", "CHERRY BLOSSOM", "LAKE KAWAGUCHI", "SUBWAY", "ARAKURAYAMA SENGEN PARK", "MUSIC", "PARKS", "SCARY", "SYMPHONIES", "TEMPERATURE", "CHUREITO PAGODA", "HAKONE", "LAKE", "PINOCCHIO", "3 DEGREES", "THREE DEGREES CELSIUS", "NARRATOR", "SINGAPORE", "MUSIC BOX MUSEUM", "PIANO", "RING", "SAND ART", "FIVE-STORIED PAGODA", "HEAVENLY BELL", "KAWAGUCHIKO STATION", "MOUNT FUJI", "MOUNT FUJI AREA", "PIANOS", "SHINJUKU", "TENJOYAMA PARK", "BENCHES", "BUS", "HOTEL", "JAPAN", "TRAIL", "SWANS", "APRIL 3RD", "KANSAI", "LOCAL GIRL", "SHINJUJU", "NECKLACE", "PINOCCHIO'S STORY" ]
}

OS

Windows

GPU

Nvidia

CPU

AMD

Ollama version

0.3.6

Originally created by @davidliudev on GitHub (Aug 20, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6435 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? This does not happen on 0.3.4. It only happen on the latest 0.3.6 (Haven't test on 0.3.5). If I reduce the payload to a single item it is all good. It happens when I put more items. I have to downgrade to the 0.3.4 until this is fixed. Here is the log: time=2024-08-20T13:38:29.761+08:00 level=ERROR source=routes.go:394 msg="embedding generation failed" error="health resp: Get \"http://127.0.0.1:56157/health\": read tcp 127.0.0.1:56190->127.0.0.1:56157: wsarecv: An existing connection was forcibly closed by the remote host." Sample payload: ``` { "model" : "nomic-embed-text", "input" : [ "BREAKFAST", "MAPLE LEAVES", "SCENERY", "ROKKO MOUNTAIN", "SUNSET", "HOT SPRINGS", "PEAK", "NARITA AIRPORT", "SPEAKER", "MORNING", "MOUNTAIN", "RED-EYE FLIGHT", "STRING QUARTET", "AIR", "AUTUMN", "CHERRY BLOSSOM", "LAKE KAWAGUCHI", "SUBWAY", "ARAKURAYAMA SENGEN PARK", "MUSIC", "PARKS", "SCARY", "SYMPHONIES", "TEMPERATURE", "CHUREITO PAGODA", "HAKONE", "LAKE", "PINOCCHIO", "3 DEGREES", "THREE DEGREES CELSIUS", "NARRATOR", "SINGAPORE", "MUSIC BOX MUSEUM", "PIANO", "RING", "SAND ART", "FIVE-STORIED PAGODA", "HEAVENLY BELL", "KAWAGUCHIKO STATION", "MOUNT FUJI", "MOUNT FUJI AREA", "PIANOS", "SHINJUKU", "TENJOYAMA PARK", "BENCHES", "BUS", "HOTEL", "JAPAN", "TRAIL", "SWANS", "APRIL 3RD", "KANSAI", "LOCAL GIRL", "SHINJUJU", "NECKLACE", "PINOCCHIO'S STORY" ] } ``` ### OS Windows ### GPU Nvidia ### CPU AMD ### Ollama version 0.3.6
GiteaMirror added the bug label 2026-04-12 14:56:13 -05:00
Author
Owner

@mxyng commented on GitHub (Aug 21, 2024):

@davidliudev I'm unable to reproduce your issue. Can you provide some details on how you've reached this error? The log line you linked is abnormal since Ollama doesn't have a health endpoint nor does it (generally) serve on port 56157.

FWIW, here's the output of your example payload:

import ollama

inputs = ['BREAKFAST', 'MAPLE LEAVES', 'SCENERY', 'ROKKO MOUNTAIN', 'SUNSET', 'HOT SPRINGS', 'PEAK', 'NARITA AIRPORT', 'SPEAKER', 'MORNING', 'MOUNTAIN', 'RED-EYE FLIGHT', 'STRING QUARTET', 'AIR', 'AUTUMN', 'CHERRY BLOSSOM', 'LAKE KAWAGUCHI', 'SUBWAY', 'ARAKURAYAMA SENGEN PARK', 'MUSIC', 'PARKS', 'SCARY', 'SYMPHONIES', 'TEMPERATURE', 'CHUREITO PAGODA', 'HAKONE', 'LAKE', 'PINOCCHIO', '3 DEGREES', 'THREE DEGREES CELSIUS', 'NARRATOR', 'SINGAPORE', 'MUSIC BOX MUSEUM', 'PIANO', 'RING', 'SAND ART', 'FIVE-STORIED PAGODA', 'HEAVENLY BELL', 'KAWAGUCHIKO STATION', 'MOUNT FUJI', 'MOUNT FUJI AREA', 'PIANOS', 'SHINJUKU', 'TENJOYAMA PARK', 'BENCHES', 'BUS', 'HOTEL', 'JAPAN', 'TRAIL', 'SWANS', 'APRIL 3RD', 'KANSAI', 'LOCAL GIRL', 'SHINJUJU', 'NECKLACE', "PINOCCHIO'S STORY"]

embeddings = ollama.embed(model='nomic-embed-text', input=inputs)
for word, embedding in zip(inputs, embeddings['embeddings']):
  print(word, embedding[:5])
$ python embed.py
BREAKFAST [0.008739223, 0.045730893, -0.16131887, 0.0031437576, -0.038349766]
MAPLE LEAVES [0.06398161, -0.06967579, -0.13945945, -0.0333528, -0.042758334]
SCENERY [-0.012765189, 0.03290033, -0.21166044, -0.003989847, -0.02353204]
ROKKO MOUNTAIN [0.013675163, -0.0013703966, -0.13379748, 0.04967475, 0.059461594]
SUNSET [-0.020506589, -0.038180366, -0.18680866, 0.019179147, -0.025532402]
HOT SPRINGS [0.0252274, -0.011029283, -0.17439325, 0.019640828, 0.02458879]
PEAK [0.032953084, 0.04261463, -0.21061127, 0.030152997, 0.030941876]
NARITA AIRPORT [0.000578391, 0.023201877, -0.1250423, 0.0513831, 0.0055379267]
SPEAKER [-0.006622254, 0.041886065, -0.17437804, -0.049660973, 0.032186788]
MORNING [-0.027311591, -0.0033516185, -0.16685818, -0.008191577, -0.009172038]
MOUNTAIN [0.029238084, 0.03328353, -0.19173723, 0.023338366, 0.039173394]
RED-EYE FLIGHT [-0.046498954, 0.046370987, -0.16383094, 0.024812097, 0.05204731]
STRING QUARTET [-0.015339078, -0.02581633, -0.17397861, -0.04076196, 0.015117032]
AIR [0.0037941497, 0.022738842, -0.22011915, 0.07908051, -0.019416183]
AUTUMN [0.021555962, -0.06341981, -0.19074881, 0.012451607, 0.0025711225]
CHERRY BLOSSOM [0.058939587, 0.0033518476, -0.16395621, -0.013085613, -0.028672198]
LAKE KAWAGUCHI [-0.00040816038, 0.012328833, -0.12697187, -0.011389026, -0.0161123]
SUBWAY [-0.03356874, 0.07117735, -0.16424924, 0.054867998, -0.016875926]
ARAKURAYAMA SENGEN PARK [0.0007031584, 0.066544496, -0.13053793, 0.008732335, -0.022130517]
MUSIC [-0.005144564, -0.012165364, -0.21838534, 0.003808897, -0.04082421]
PARKS [0.04118935, 0.060493343, -0.15126446, 0.009695669, 0.0017490978]
SCARY [0.011967612, 0.057114914, -0.1568222, 0.025713747, 0.015946949]
SYMPHONIES [0.017533561, 0.07828965, -0.16089639, 0.00936415, -0.013971172]
TEMPERATURE [0.060464684, -0.02033767, -0.21924934, -0.0083500305, 0.0055289534]
CHUREITO PAGODA [-0.014063288, 0.06642195, -0.11888982, 0.006646133, -0.023541277]
HAKONE [-0.03513632, 0.038148396, -0.14236735, -0.0125797335, -0.003765748]
LAKE [-0.0057897232, 0.041829478, -0.1438062, -0.039227884, 0.011480182]
PINOCCHIO [0.059544116, 0.10514674, -0.13427463, -0.012096445, -0.0210777]
3 DEGREES [0.074253745, -0.03441007, -0.16229033, 0.0070045004, -0.0059312163]
THREE DEGREES CELSIUS [0.08147877, -0.0070242137, -0.17812873, -0.010499873, -0.0050736438]
NARRATOR [-0.004548263, -0.0021088244, -0.18229197, -0.07153085, 0.014255789]
SINGAPORE [0.016203085, -0.02830413, -0.12495192, -0.01169783, -0.012468399]
MUSIC BOX MUSEUM [-0.06864728, 0.058857616, -0.16358341, -0.030605467, -0.010344431]
PIANO [0.046430565, 0.029432388, -0.163196, -0.0034248384, -0.047273226]
RING [0.011789637, 0.06921108, -0.16021581, -0.0149869155, 0.033436995]
SAND ART [0.06159453, 0.05867689, -0.14117905, -0.062383473, 0.02107234]
FIVE-STORIED PAGODA [-0.01760132, 0.07202654, -0.17384647, -0.057261832, -0.013982573]
HEAVENLY BELL [0.009134987, 0.039536294, -0.148511, -0.06148186, 0.0075094444]
KAWAGUCHIKO STATION [-0.012179129, 0.001737424, -0.15419188, 0.0040292237, -0.0048527075]
MOUNT FUJI [0.049408976, 0.056463603, -0.1838918, 0.048378292, 0.06483141]
MOUNT FUJI AREA [0.030215079, 0.033897273, -0.17068794, 0.039174587, 0.03892817]
PIANOS [0.016480187, 0.01681652, -0.16930659, -0.022269255, -0.04790617]
SHINJUKU [-0.050184593, 0.032009974, -0.16963328, 0.04777218, -0.01213993]
TENJOYAMA PARK [0.015456956, 0.043137606, -0.13501713, -0.031795226, 0.021204393]
BENCHES [0.057052374, 0.06913541, -0.16950718, -0.055509545, 0.02376094]
BUS [-0.08689601, 0.03115404, -0.1756639, 0.01651755, 0.021232]
HOTEL [-0.07004754, 0.06234973, -0.187822, 0.031151313, -0.027771758]
JAPAN [0.03340825, 0.050580587, -0.16801456, 0.02650342, -0.046223685]
TRAIL [-0.025288368, 0.02833276, -0.19732459, 0.029448956, 0.012326132]
SWANS [-0.05994444, 0.022852037, -0.16746293, -0.05110516, -0.022832723]
APRIL 3RD [0.03477102, -0.038825084, -0.13017912, 0.052911825, -0.010114151]
KANSAI [-0.018736625, 0.042829517, -0.12185811, -0.01568517, -0.0804055]
LOCAL GIRL [0.040332254, -0.00332225, -0.15092635, -0.017235262, -0.012956413]
SHINJUJU [-0.05903753, 0.04099602, -0.15042555, 0.044082187, -0.038608752]
NECKLACE [0.013112308, 0.05554893, -0.14164281, -0.08650458, -0.025264328]
PINOCCHIO'S STORY [0.04576401, 0.10324291, -0.16271587, -0.030300887, -0.015238542]
<!-- gh-comment-id:2302967937 --> @mxyng commented on GitHub (Aug 21, 2024): @davidliudev I'm unable to reproduce your issue. Can you provide some details on how you've reached this error? The log line you linked is abnormal since Ollama doesn't have a health endpoint nor does it (generally) serve on port 56157. FWIW, here's the output of your example payload: ```python import ollama inputs = ['BREAKFAST', 'MAPLE LEAVES', 'SCENERY', 'ROKKO MOUNTAIN', 'SUNSET', 'HOT SPRINGS', 'PEAK', 'NARITA AIRPORT', 'SPEAKER', 'MORNING', 'MOUNTAIN', 'RED-EYE FLIGHT', 'STRING QUARTET', 'AIR', 'AUTUMN', 'CHERRY BLOSSOM', 'LAKE KAWAGUCHI', 'SUBWAY', 'ARAKURAYAMA SENGEN PARK', 'MUSIC', 'PARKS', 'SCARY', 'SYMPHONIES', 'TEMPERATURE', 'CHUREITO PAGODA', 'HAKONE', 'LAKE', 'PINOCCHIO', '3 DEGREES', 'THREE DEGREES CELSIUS', 'NARRATOR', 'SINGAPORE', 'MUSIC BOX MUSEUM', 'PIANO', 'RING', 'SAND ART', 'FIVE-STORIED PAGODA', 'HEAVENLY BELL', 'KAWAGUCHIKO STATION', 'MOUNT FUJI', 'MOUNT FUJI AREA', 'PIANOS', 'SHINJUKU', 'TENJOYAMA PARK', 'BENCHES', 'BUS', 'HOTEL', 'JAPAN', 'TRAIL', 'SWANS', 'APRIL 3RD', 'KANSAI', 'LOCAL GIRL', 'SHINJUJU', 'NECKLACE', "PINOCCHIO'S STORY"] embeddings = ollama.embed(model='nomic-embed-text', input=inputs) for word, embedding in zip(inputs, embeddings['embeddings']): print(word, embedding[:5]) ``` ``` $ python embed.py BREAKFAST [0.008739223, 0.045730893, -0.16131887, 0.0031437576, -0.038349766] MAPLE LEAVES [0.06398161, -0.06967579, -0.13945945, -0.0333528, -0.042758334] SCENERY [-0.012765189, 0.03290033, -0.21166044, -0.003989847, -0.02353204] ROKKO MOUNTAIN [0.013675163, -0.0013703966, -0.13379748, 0.04967475, 0.059461594] SUNSET [-0.020506589, -0.038180366, -0.18680866, 0.019179147, -0.025532402] HOT SPRINGS [0.0252274, -0.011029283, -0.17439325, 0.019640828, 0.02458879] PEAK [0.032953084, 0.04261463, -0.21061127, 0.030152997, 0.030941876] NARITA AIRPORT [0.000578391, 0.023201877, -0.1250423, 0.0513831, 0.0055379267] SPEAKER [-0.006622254, 0.041886065, -0.17437804, -0.049660973, 0.032186788] MORNING [-0.027311591, -0.0033516185, -0.16685818, -0.008191577, -0.009172038] MOUNTAIN [0.029238084, 0.03328353, -0.19173723, 0.023338366, 0.039173394] RED-EYE FLIGHT [-0.046498954, 0.046370987, -0.16383094, 0.024812097, 0.05204731] STRING QUARTET [-0.015339078, -0.02581633, -0.17397861, -0.04076196, 0.015117032] AIR [0.0037941497, 0.022738842, -0.22011915, 0.07908051, -0.019416183] AUTUMN [0.021555962, -0.06341981, -0.19074881, 0.012451607, 0.0025711225] CHERRY BLOSSOM [0.058939587, 0.0033518476, -0.16395621, -0.013085613, -0.028672198] LAKE KAWAGUCHI [-0.00040816038, 0.012328833, -0.12697187, -0.011389026, -0.0161123] SUBWAY [-0.03356874, 0.07117735, -0.16424924, 0.054867998, -0.016875926] ARAKURAYAMA SENGEN PARK [0.0007031584, 0.066544496, -0.13053793, 0.008732335, -0.022130517] MUSIC [-0.005144564, -0.012165364, -0.21838534, 0.003808897, -0.04082421] PARKS [0.04118935, 0.060493343, -0.15126446, 0.009695669, 0.0017490978] SCARY [0.011967612, 0.057114914, -0.1568222, 0.025713747, 0.015946949] SYMPHONIES [0.017533561, 0.07828965, -0.16089639, 0.00936415, -0.013971172] TEMPERATURE [0.060464684, -0.02033767, -0.21924934, -0.0083500305, 0.0055289534] CHUREITO PAGODA [-0.014063288, 0.06642195, -0.11888982, 0.006646133, -0.023541277] HAKONE [-0.03513632, 0.038148396, -0.14236735, -0.0125797335, -0.003765748] LAKE [-0.0057897232, 0.041829478, -0.1438062, -0.039227884, 0.011480182] PINOCCHIO [0.059544116, 0.10514674, -0.13427463, -0.012096445, -0.0210777] 3 DEGREES [0.074253745, -0.03441007, -0.16229033, 0.0070045004, -0.0059312163] THREE DEGREES CELSIUS [0.08147877, -0.0070242137, -0.17812873, -0.010499873, -0.0050736438] NARRATOR [-0.004548263, -0.0021088244, -0.18229197, -0.07153085, 0.014255789] SINGAPORE [0.016203085, -0.02830413, -0.12495192, -0.01169783, -0.012468399] MUSIC BOX MUSEUM [-0.06864728, 0.058857616, -0.16358341, -0.030605467, -0.010344431] PIANO [0.046430565, 0.029432388, -0.163196, -0.0034248384, -0.047273226] RING [0.011789637, 0.06921108, -0.16021581, -0.0149869155, 0.033436995] SAND ART [0.06159453, 0.05867689, -0.14117905, -0.062383473, 0.02107234] FIVE-STORIED PAGODA [-0.01760132, 0.07202654, -0.17384647, -0.057261832, -0.013982573] HEAVENLY BELL [0.009134987, 0.039536294, -0.148511, -0.06148186, 0.0075094444] KAWAGUCHIKO STATION [-0.012179129, 0.001737424, -0.15419188, 0.0040292237, -0.0048527075] MOUNT FUJI [0.049408976, 0.056463603, -0.1838918, 0.048378292, 0.06483141] MOUNT FUJI AREA [0.030215079, 0.033897273, -0.17068794, 0.039174587, 0.03892817] PIANOS [0.016480187, 0.01681652, -0.16930659, -0.022269255, -0.04790617] SHINJUKU [-0.050184593, 0.032009974, -0.16963328, 0.04777218, -0.01213993] TENJOYAMA PARK [0.015456956, 0.043137606, -0.13501713, -0.031795226, 0.021204393] BENCHES [0.057052374, 0.06913541, -0.16950718, -0.055509545, 0.02376094] BUS [-0.08689601, 0.03115404, -0.1756639, 0.01651755, 0.021232] HOTEL [-0.07004754, 0.06234973, -0.187822, 0.031151313, -0.027771758] JAPAN [0.03340825, 0.050580587, -0.16801456, 0.02650342, -0.046223685] TRAIL [-0.025288368, 0.02833276, -0.19732459, 0.029448956, 0.012326132] SWANS [-0.05994444, 0.022852037, -0.16746293, -0.05110516, -0.022832723] APRIL 3RD [0.03477102, -0.038825084, -0.13017912, 0.052911825, -0.010114151] KANSAI [-0.018736625, 0.042829517, -0.12185811, -0.01568517, -0.0804055] LOCAL GIRL [0.040332254, -0.00332225, -0.15092635, -0.017235262, -0.012956413] SHINJUJU [-0.05903753, 0.04099602, -0.15042555, 0.044082187, -0.038608752] NECKLACE [0.013112308, 0.05554893, -0.14164281, -0.08650458, -0.025264328] PINOCCHIO'S STORY [0.04576401, 0.10324291, -0.16271587, -0.030300887, -0.015238542] ```
Author
Owner

@davidliudev commented on GitHub (Aug 22, 2024):

@mxyng
It does listen on llama cpp server on a random port between 65535-49152 according to this line:

6bd8a4b0a1/llm/server.go (L301)

I suspect something breaks in communication between ollama and llamacpp server during recent updates.

My environment is on Windows 11, 0.3.6. My way of reproducing this issue is just installing and run, and get this error.. no other special setup

0.3.4 is guaranteed to not have such issues

<!-- gh-comment-id:2303564335 --> @davidliudev commented on GitHub (Aug 22, 2024): @mxyng It does listen on llama cpp server on a random port between **65535-49152** according to this line: https://github.com/ollama/ollama/blob/6bd8a4b0a1ac15d5718f52bbe1cd56f827beb694/llm/server.go#L301 I suspect something breaks in communication between ollama and llamacpp server during recent updates. My environment is on Windows 11, 0.3.6. My way of reproducing this issue is just installing and run, and get this error.. no other special setup 0.3.4 is guaranteed to not have such issues
Author
Owner

@davidliudev commented on GitHub (Aug 22, 2024):

What I suspect is that the llama server crashed, resulting a forcefully close of the http call to it, and somehow the log missed the crash information.

<!-- gh-comment-id:2303576678 --> @davidliudev commented on GitHub (Aug 22, 2024): What I suspect is that the llama server crashed, resulting a forcefully close of the http call to it, and somehow the log missed the crash information.
Author
Owner

@davidliudev commented on GitHub (Aug 22, 2024):

I roughly looked at the changes happened from 0.3.4 to 0.3.5 and this commit is potentially suspicious. I may be wrong because I am not familiar with the codebase but roughly after looking at the changed code this is the closest relevant changes that is around my above issue.

15c2d8fe14
Is there any possibility that this embedding parallelism may crash the llama server somehow?

cc @jmorganca

<!-- gh-comment-id:2303584663 --> @davidliudev commented on GitHub (Aug 22, 2024): I roughly looked at the changes happened from 0.3.4 to 0.3.5 and [this](https://github.com/ollama/ollama/commit/15c2d8fe149ba2b58aadbab615a6955f8821c7a9) commit is potentially suspicious. I may be wrong because I am not familiar with the codebase but roughly after looking at the changed code this is the closest relevant changes that is around my above issue. https://github.com/ollama/ollama/commit/15c2d8fe149ba2b58aadbab615a6955f8821c7a9 Is there any possibility that this embedding parallelism may crash the llama server somehow? cc @jmorganca
Author
Owner

@mxyng commented on GitHub (Aug 22, 2024):

You are correct. I forgot about the subprocess.

There seems to be some memory corruption. @dhiltgen is currently looking into it.

<!-- gh-comment-id:2305541855 --> @mxyng commented on GitHub (Aug 22, 2024): You are correct. I forgot about the subprocess. There seems to be some memory corruption. @dhiltgen is currently looking into it.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4045