github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6934] OpenAI client expects embeddings to be base64 encoded string, not json array of floats #66437

New Issue
Closed
opened 2026-05-04 05:12:35 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-05-04 05:12:35 -05:00
Owner
Copy Link

Originally created by @jasoncouture on GitHub (Sep 24, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6934

Originally assigned to: @npardal on GitHub.

What is the issue?

v1/embeddings support was added recently, but the C# OpenAPI client expects the API to return a base64 string, containing the embeddings as 4 byte floating point values.

The deserializer eventually calls this function

    private static ReadOnlyMemory<float> ConvertToVectorOfFloats(BinaryData binaryData)
    {
        ReadOnlySpan<byte> base64 = binaryData.ToMemory().Span;

        // Remove quotes around base64 string.
        if (base64.Length < 2 || base64[0] != (byte)'"' || base64[base64.Length - 1] != (byte)'"')
        {
            ThrowInvalidData();
        }
        base64 = base64.Slice(1, base64.Length - 2);

        // Decode base64 string to bytes.
        byte[] bytes = ArrayPool<byte>.Shared.Rent(Base64.GetMaxDecodedFromUtf8Length(base64.Length));
        OperationStatus status = Base64.DecodeFromUtf8(base64, bytes.AsSpan(), out int bytesConsumed, out int bytesWritten);
        if (status != OperationStatus.Done || bytesWritten % sizeof(float) != 0)
        {
            ThrowInvalidData();
        }

While I find this... goofy to say the least, this API is not currently compatible with OpenAI's embedding endpoint because of this difference.

Relates to: #5285

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.3.11

Originally created by @jasoncouture on GitHub (Sep 24, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6934 Originally assigned to: @npardal on GitHub. ### What is the issue? v1/embeddings support was added recently, but the C# OpenAPI client expects the API to return a base64 string, containing the embeddings as 4 byte floating point values. The deserializer eventually calls [this function](https://github.com/openai/openai-dotnet/blob/main/src/Custom/Embeddings/Embedding.cs#L107) ```csharp private static ReadOnlyMemory<float> ConvertToVectorOfFloats(BinaryData binaryData) { ReadOnlySpan<byte> base64 = binaryData.ToMemory().Span; // Remove quotes around base64 string. if (base64.Length < 2 || base64[0] != (byte)'"' || base64[base64.Length - 1] != (byte)'"') { ThrowInvalidData(); } base64 = base64.Slice(1, base64.Length - 2); // Decode base64 string to bytes. byte[] bytes = ArrayPool<byte>.Shared.Rent(Base64.GetMaxDecodedFromUtf8Length(base64.Length)); OperationStatus status = Base64.DecodeFromUtf8(base64, bytes.AsSpan(), out int bytesConsumed, out int bytesWritten); if (status != OperationStatus.Done || bytesWritten % sizeof(float) != 0) { ThrowInvalidData(); } ``` While I find this... goofy to say the least, this API is not currently compatible with OpenAI's embedding endpoint because of this difference. Relates to: #5285 ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.3.11
GiteaMirror added the bugapi labels 2026-05-04 05:12:37 -05:00
GiteaMirror commented 2026-05-04 05:12:45 -05:00
Author
Owner
Copy Link

@tomaszbk commented on GitHub (Mar 14, 2025):

using langchain's OpenAIEmbeddings with ollama embeddings doesnt work either:
openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}}

<!-- gh-comment-id:2722959782 --> @tomaszbk commented on GitHub (Mar 14, 2025): using langchain's OpenAIEmbeddings with ollama embeddings doesnt work either: openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}}
GiteaMirror commented 2026-05-04 05:12:46 -05:00
Author
Owner
Copy Link

@David-Morales-Norato commented on GitHub (Mar 18, 2025):

Please add the OpenAI integration for embeddings

I'm trying to use it with Langchain, when y use the base_url="http://localhost:11434/v1"

embedding_function = OpenAIEmbeddings(
    model="nomic-embed-text",
    base_url="http://localhost:11434/api",
    api_key="ollama",
)

print(embedding_function.embed_query("test"))

The error is something like this

...
    for embedding in obj.data:
TypeError: 'NoneType' object is not iterable

Using
"http://localhost:11434/v1"

...
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}}
<!-- gh-comment-id:2731298962 --> @David-Morales-Norato commented on GitHub (Mar 18, 2025): Please add the OpenAI integration for embeddings I'm trying to use it with Langchain, when y use the `base_url="http://localhost:11434/v1"` ``` embedding_function = OpenAIEmbeddings( model="nomic-embed-text", base_url="http://localhost:11434/api", api_key="ollama", ) print(embedding_function.embed_query("test")) ``` The error is something like this ``` ... for embedding in obj.data: TypeError: 'NoneType' object is not iterable ``` Using "http://localhost:11434/v1" ``` ... raise self._make_status_error_from_response(err.response) from None openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}} ```
GiteaMirror commented 2026-05-04 05:12:54 -05:00
Author
Owner
Copy Link

@ActuallyTrent commented on GitHub (Mar 31, 2025):

Please add the OpenAI integration for embeddings

I'm trying to use it with Langchain, when y use the base_url="http://localhost:11434/v1"

embedding_function = OpenAIEmbeddings(
    model="nomic-embed-text",
    base_url="http://localhost:11434/api",
    api_key="ollama",
)

print(embedding_function.embed_query("test"))

Your example does not have the /v1 and instead is using /api

<!-- gh-comment-id:2767571508 --> @ActuallyTrent commented on GitHub (Mar 31, 2025): > Please add the OpenAI integration for embeddings > > I'm trying to use it with Langchain, when y use the `base_url="http://localhost:11434/v1"` > > ``` > embedding_function = OpenAIEmbeddings( > model="nomic-embed-text", > base_url="http://localhost:11434/api", > api_key="ollama", > ) > > print(embedding_function.embed_query("test")) > ``` > Your example does not have the `/v1` and instead is using `/api`
GiteaMirror commented 2026-05-04 05:12:57 -05:00
Author
Owner
Copy Link

@David-Morales-Norato commented on GitHub (Apr 1, 2025):

Please add the OpenAI integration for embeddings
I'm trying to use it with Langchain, when y use the base_url="http://localhost:11434/v1"

embedding_function = OpenAIEmbeddings(
    model="nomic-embed-text",
    base_url="http://localhost:11434/api",
    api_key="ollama",
)

print(embedding_function.embed_query("test"))

Your example does not have the /v1 and instead is using /api

In my first reply, I added,

Using
"http://localhost:11434/v1"

...
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}}

That referenced the error when I used the /v1. But now I have solved, thanks to the langchain.embeddings.init_embeddings. This is my solution

    kwargs = {}
    if provider == "ollama":
        kwargs["base_url"] = os.getenv("OLLAMA_BASE_URL")
    elif provider == "openai":
        kwargs["api_key"] = openai_api_key
    else:
        raise("Not Implemented")
    embedding_result = init_embeddings(
        model=model,
        provider=provider,
        **kwargs,
    )
<!-- gh-comment-id:2767720238 --> @David-Morales-Norato commented on GitHub (Apr 1, 2025): > > Please add the OpenAI integration for embeddings > > I'm trying to use it with Langchain, when y use the `base_url="http://localhost:11434/v1"` > > ``` > > embedding_function = OpenAIEmbeddings( > > model="nomic-embed-text", > > base_url="http://localhost:11434/api", > > api_key="ollama", > > ) > > > > print(embedding_function.embed_query("test")) > > ``` > > Your example does not have the `/v1` and instead is using `/api` In my first reply, I added, > > Using > "http://localhost:11434/v1" > > ... > raise self._make_status_error_from_response(err.response) from None > openai.BadRequestError: Error code: 400 - {'error': {'message': 'invalid input type', 'type': 'api_error', 'param': None, 'code': None}} That referenced the error when I used the `/v1`. But now I have solved, thanks to the `langchain.embeddings.init_embeddings`. This is my solution ``` kwargs = {} if provider == "ollama": kwargs["base_url"] = os.getenv("OLLAMA_BASE_URL") elif provider == "openai": kwargs["api_key"] = openai_api_key else: raise("Not Implemented") embedding_result = init_embeddings( model=model, provider=provider, **kwargs, ) ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label api bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#66437
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?