github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3258] Problem with returning response with openai compatibility #2005

New Issue
Closed
opened 2026-04-12 12:13:11 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 12:13:11 -05:00
Owner
Copy Link

Originally created by @ejgutierrez74 on GitHub (Mar 19, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3258

What is the issue?

The response output is somehow a list:

('The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative.', 0)

What did you expect to see?

The text of the response....

The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative.

As for now i can access that with:

print(response[0]) or
print(response[0].strip())

Steps to reproduce

Execute this code;

`def llama_openai(prompt, 
          add_inst=True, #By default True, if you use a base model should write it as False
          model="llama2", 
          temperature=0.0, #By default in openai is 1.0 o 0.7 depends of the model, openai from 0.0 to 2.0, llama2 from 0.0 to 1.0
          max_tokens=1024,
          verbose=False
         ):
    
    if add_inst:
        prompt = f"[INST]{prompt}[/INST]"

    if verbose:
        print(f"Prompt:\n{prompt}\n")
        print(f"model: {model}")

    error = 0
  
    try:
        response = client.chat.completions.create(
            messages=[
                        {
                        'role': 'user',
                        'content': prompt,
                        }
                    ],
            model=model,
            max_tokens=max_tokens,
            temperature=temperature
                )

    except openai.APIError as e:
      #Handle API error here, e.g. retry or log
      print(f"Llama2: OpenAI API returned an API Error:{e} ")
      error = 1
      pass
    except openai.APIConnectionError as e:
      #Handle connection error here
      print(f"Llama2:Failed to connect to OpenAI API:{e} ")
      error = 2
      pass
    except openai.RateLimitError as e:
      #Handle rate limit error (we recommend using exponential backoff)
      print(f"Llama2: OpenAI API request exceeded rate limit:{e} ")
      error = 3
      pass
    except ConnectionError as e:
        # Handle ConnectionError here
        print(f"Llama2: Connection Refused, perhaps server not running, network problem or address ip and port incorrect: {e}")
        error = 4
        pass
    except Exception as e:
        # Handle all other exceptions.
        print(f"Llama2: Error general no reconegut: {e}")
        error = 5
        pass

    #Si no salta ninguna excpeción, todo va bien..Vamos a devolver la respuesta del sistema y el error que deberia ser cero
    else:    
        #Si no hay ninguna excepcion se ejecuta else
        return (response.choices[0].message.content, error)
    
    #Si ha saltado alguna excepcion devolvemos algo y el codigo de error
    return ("Default Value", error)
`

Then execute a test:

prompt = """
What is the sentiment of:
Hi Amit, thanks for the thoughtful birthday card!
"""
response = llama_openai(prompt)
print(response)

Then somehow the returned value:

('The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative.', 0)

Are there any recent changes that introduced the issue?

I think so, that in ollama v1.0.28 didnt happen...but not sure.

OS

Linux

Architecture

amd64

Platform

No response

Ollama version

0.1.29

GPU

No response

GPU info

No response

CPU

No response

Other software

No response

Originally created by @ejgutierrez74 on GitHub (Mar 19, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3258 ### What is the issue? The response output is somehow a list: > ('The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative.', 0) ### What did you expect to see? The text of the response.... > The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative. As for now i can access that with: ``` print(response[0]) or print(response[0].strip()) ``` ### Steps to reproduce Execute this code; ```python `def llama_openai(prompt, add_inst=True, #By default True, if you use a base model should write it as False model="llama2", temperature=0.0, #By default in openai is 1.0 o 0.7 depends of the model, openai from 0.0 to 2.0, llama2 from 0.0 to 1.0 max_tokens=1024, verbose=False ): if add_inst: prompt = f"[INST]{prompt}[/INST]" if verbose: print(f"Prompt:\n{prompt}\n") print(f"model: {model}") error = 0 try: response = client.chat.completions.create( messages=[ { 'role': 'user', 'content': prompt, } ], model=model, max_tokens=max_tokens, temperature=temperature ) except openai.APIError as e: #Handle API error here, e.g. retry or log print(f"Llama2: OpenAI API returned an API Error:{e} ") error = 1 pass except openai.APIConnectionError as e: #Handle connection error here print(f"Llama2:Failed to connect to OpenAI API:{e} ") error = 2 pass except openai.RateLimitError as e: #Handle rate limit error (we recommend using exponential backoff) print(f"Llama2: OpenAI API request exceeded rate limit:{e} ") error = 3 pass except ConnectionError as e: # Handle ConnectionError here print(f"Llama2: Connection Refused, perhaps server not running, network problem or address ip and port incorrect: {e}") error = 4 pass except Exception as e: # Handle all other exceptions. print(f"Llama2: Error general no reconegut: {e}") error = 5 pass #Si no salta ninguna excpeción, todo va bien..Vamos a devolver la respuesta del sistema y el error que deberia ser cero else: #Si no hay ninguna excepcion se ejecuta else return (response.choices[0].message.content, error) #Si ha saltado alguna excepcion devolvemos algo y el codigo de error return ("Default Value", error) ` ``` Then execute a test: ```python prompt = """ What is the sentiment of: Hi Amit, thanks for the thoughtful birthday card! """ response = llama_openai(prompt) print(response) ``` Then somehow the returned value: > ('The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative.', 0) ### Are there any recent changes that introduced the issue? I think so, that in ollama v1.0.28 didnt happen...but not sure. ### OS Linux ### Architecture amd64 ### Platform _No response_ ### Ollama version 0.1.29 ### GPU _No response_ ### GPU info _No response_ ### CPU _No response_ ### Other software _No response_
GiteaMirror added the bug label 2026-04-12 12:13:11 -05:00
GiteaMirror commented 2026-04-12 12:13:13 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Mar 25, 2024):

Hi Amit, there were no changes in Ollama which should have started causing this behavior. Do you see this when you use the CLI also?

<!-- gh-comment-id:2018468810 --> @BruceMacD commented on GitHub (Mar 25, 2024): Hi Amit, there were no changes in Ollama which should have started causing this behavior. Do you see this when you use the CLI also?
GiteaMirror commented 2026-04-12 12:13:14 -05:00
Author
Owner
Copy Link

@ejgutierrez74 commented on GitHub (Mar 29, 2024):

Are you writing to me ? As your wrote hi Amit....dont know who is Amit...( perhaps some english expression i dont understand)
I only use jupyter notebook in my courses and test...

<!-- gh-comment-id:2027077499 --> @ejgutierrez74 commented on GitHub (Mar 29, 2024): Are you writing to me ? As your wrote hi Amit....dont know who is Amit...( perhaps some english expression i dont understand) I only use jupyter notebook in my courses and test...
GiteaMirror commented 2026-04-12 12:13:14 -05:00
Author
Owner
Copy Link

@royjhan commented on GitHub (Jul 31, 2024):

response = client.chat.completions.create(
    model="llama3",
    messages=[
        {
            "role": "user",
            "content": "What's the weather like in San Francisco?"
        }
    ],
    max_tokens=300,
)

print(response.choices[0].message.content)

this is working as expected, please reopen the issue if you are still experiencing this problem. On quick glance, it looks like your code is returning (response.choices[0].message.content, error), which is probably why you're getting the tuple.

<!-- gh-comment-id:2261125297 --> @royjhan commented on GitHub (Jul 31, 2024): ``` response = client.chat.completions.create( model="llama3", messages=[ { "role": "user", "content": "What's the weather like in San Francisco?" } ], max_tokens=300, ) print(response.choices[0].message.content) ``` this is working as expected, please reopen the issue if you are still experiencing this problem. On quick glance, it looks like your code is returning (response.choices[0].message.content, error), which is probably why you're getting the tuple.
GiteaMirror commented 2026-04-12 12:13:15 -05:00
Author
Owner
Copy Link

@ejgutierrez74 commented on GitHub (Aug 7, 2024):

Yes but my function is returning as you said: response.choices[0].message.content that should be the reponse text in the example: "The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative."..so i think is not working as expected...

<!-- gh-comment-id:2273077872 --> @ejgutierrez74 commented on GitHub (Aug 7, 2024): Yes but my function is returning as you said: response.choices[0].message.content that should be the reponse text in the example: "The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative."..so i think is not working as expected...
GiteaMirror commented 2026-04-12 12:13:16 -05:00
Author
Owner
Copy Link

@royjhan commented on GitHub (Aug 7, 2024):

That looks like a reasonable response to me? What is not working from your perspective?

<!-- gh-comment-id:2273903527 --> @royjhan commented on GitHub (Aug 7, 2024): That looks like a reasonable response to me? What is not working from your perspective?
GiteaMirror commented 2026-04-12 12:13:17 -05:00
Author
Owner
Copy Link

@ejgutierrez74 commented on GitHub (Aug 10, 2024):

As said: response.choices[0].message.content should in my opinion return the reponse text in the example, only the text of the response:
The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative, not the tuple

<!-- gh-comment-id:2281343001 --> @ejgutierrez74 commented on GitHub (Aug 10, 2024): As said: response.choices[0].message.content should in my opinion return the reponse text in the example, only the text of the response: _The sentiment of "Hi Amit, thanks for the thoughtful birthday card!" is positive. The use of the word "thoughtful" suggests that the speaker appreciates the effort and care put into the card, and the exclamation mark at the end conveys a sense of gratitude and warmth. Overall, the sentiment is friendly, sincere, and appreciative_, not the tuple
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2005
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?