[GH-ISSUE #6127] llama3.1 always uses tool #65864

New Issue

GiteaMirror · 2026-05-03T22:56:33-05:00

GiteaMirror commented

2026-05-03 22:56:33 -05:00

Originally created by @tomaszbk on GitHub (Aug 1, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6127

What is the issue?

no matter what I prompt, llama3.1 always replies with a tool call

OS

Windows

GPU

Nvidia

CPU

AMD

Ollama version

0.10.58

Originally created by @tomaszbk on GitHub (Aug 1, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6127 ### What is the issue? no matter what I prompt, llama3.1 always replies with a tool call ### OS Windows ### GPU Nvidia ### CPU AMD ### Ollama version 0.10.58

GiteaMirror added the bug label 2026-05-03 22:56:33 -05:00

GiteaMirror closed this issue

2026-05-03 22:56:34 -05:00

GiteaMirror commented

2026-05-03 22:56:38 -05:00

@rick-github commented on GitHub (Aug 1, 2024):

An example of a prompt that replies with a tool call and server logs would help in debugging.

$ curl -s localhost:11434/api/generate -d '{"model":"llama3.1","prompt":"multiply 3 times 10", "stream":false}' | jq 'del(.context)'
{
  "model": "llama3.1",
  "created_at": "2024-08-01T22:08:24.784291922Z",
  "response": "The result of multiplying 3 by 10 is:\n\n30",
  "done": true,
  "done_reason": "stop",
  "total_duration": 226709571,
  "load_duration": 15261997,
  "prompt_eval_count": 16,
  "prompt_eval_duration": 20152000,
  "eval_count": 13,
  "eval_duration": 147840000
}

@rick-github commented on GitHub (Aug 1, 2024): An example of a prompt that replies with a tool call and server logs would help in debugging. ``` $ curl -s localhost:11434/api/generate -d '{"model":"llama3.1","prompt":"multiply 3 times 10", "stream":false}' | jq 'del(.context)' { "model": "llama3.1", "created_at": "2024-08-01T22:08:24.784291922Z", "response": "The result of multiplying 3 by 10 is:\n\n30", "done": true, "done_reason": "stop", "total_duration": 226709571, "load_duration": 15261997, "prompt_eval_count": 16, "prompt_eval_duration": 20152000, "eval_count": 13, "eval_duration": 147840000 } ```

GiteaMirror commented

2026-05-03 22:56:40 -05:00

@tomaszbk commented on GitHub (Aug 1, 2024):

from langchain_core.tools import tool
from langchain_ollama import ChatOllama

@tool
def add(x: int, y: int) -> int:
    """Adds 2 numbers and returns the result"""
    return x + y
tools = [add]
model = ChatOllama(
    model="llama3.1",
    temperature=0,
    prompt="Your name is botty"
).bind_tools(tools)
response = model.invoke("what is your name?")
print(f"response: {response.content}")
print(response.tool_calls)

output:
response:
[{'name': 'add', 'args': {'x': '0', 'y': '0'}, 'id': '62cf011c-7a66-481c-894a-0fa5a3c9e6a8', 'type': 'tool_call'}]

@tomaszbk commented on GitHub (Aug 1, 2024): ```python from langchain_core.tools import tool from langchain_ollama import ChatOllama @tool def add(x: int, y: int) -> int: """Adds 2 numbers and returns the result""" return x + y tools = [add] model = ChatOllama( model="llama3.1", temperature=0, prompt="Your name is botty" ).bind_tools(tools) response = model.invoke("what is your name?") print(f"response: {response.content}") print(response.tool_calls) ``` output: response: [{'name': 'add', 'args': {'x': '0', 'y': '0'}, 'id': '62cf011c-7a66-481c-894a-0fa5a3c9e6a8', 'type': 'tool_call'}]

GiteaMirror commented

2026-05-03 22:56:40 -05:00

@rick-github commented on GitHub (Aug 1, 2024):

Tool calls have higher priority for this model. Don't bind tools if you don't want a tool call.

@rick-github commented on GitHub (Aug 1, 2024): Tool calls have higher priority for this model. Don't bind tools if you don't want a tool call.

GiteaMirror commented

2026-05-03 22:56:41 -05:00

@tomaszbk commented on GitHub (Aug 2, 2024):

I mean, I could set one llm that decides if the user prompt requires a tool to be called, and then pass the prompt to a model with the tools binded, or else pass the prompt to a model that doesn't have the tools binded. Maybe this is the way I'm supposed to do it? but it doesn't sound like optimal

@tomaszbk commented on GitHub (Aug 2, 2024): I mean, I could set one llm that decides if the user prompt requires a tool to be called, and then pass the prompt to a model with the tools binded, or else pass the prompt to a model that doesn't have the tools binded. Maybe this is the way I'm supposed to do it? but it doesn't sound like optimal

GiteaMirror commented

2026-05-03 22:56:42 -05:00

@rick-github commented on GitHub (Aug 2, 2024):

The alternative is to modify the template and try to make the model decide whether it needs to use a tool or not. You can do this by getting the Modelfile (ollama show --modelfile llama3.1 > Modelfile), editing the TEMPLATE field, and creating a new model (ollama create llama3.1-tool -f Modelfile). For example, I changed the template:

@@ -19,9 +19,7 @@
 {{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|>
 {{- if and $.Tools $last }}
 
-Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
-
-Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
+Analyse the given prompt and decided whether or not it can be answered by a tool.  If it can, use the following functions to respond with a JSON for a function call with its proper arguments that best answers the given prompt.  Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
 
 {{ $.Tools }}
 {{- end }}

and now your script responds:

response: The prompt "what is your name?" cannot be answered by a tool, as it requires personal information. However, I can respond with my name.

Since the given function list does not include a function that directly answers this question, I will use a simple string response instead of a JSON object.

My name is Botty.
[]

and if I change the prompt to what is 2 + 2?, it responds:

response: 
[{'name': 'add', 'args': {'x': 2, 'y': 2}, 'id': 'bb51899f-3826-401a-bd79-11548c7bd0aa', 'type': 'tool_call'}]

If you take this approach you will need to craft the modified prompt to remove the "thinking" that it does and just return the answer.

@rick-github commented on GitHub (Aug 2, 2024): The alternative is to modify the template and try to make the model decide whether it needs to use a tool or not. You can do this by getting the Modelfile (`ollama show --modelfile llama3.1 > Modelfile`), editing the TEMPLATE field, and creating a new model (`ollama create llama3.1-tool -f Modelfile`). For example, I changed the template: ```diff @@ -19,9 +19,7 @@ {{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|> {{- if and $.Tools $last }} -Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. - -Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables. +Analyse the given prompt and decided whether or not it can be answered by a tool. If it can, use the following functions to respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables. {{ $.Tools }} {{- end }} ``` and now your script responds: ``` response: The prompt "what is your name?" cannot be answered by a tool, as it requires personal information. However, I can respond with my name. Since the given function list does not include a function that directly answers this question, I will use a simple string response instead of a JSON object. My name is Botty. [] ``` and if I change the prompt to `what is 2 + 2?`, it responds: ``` response: [{'name': 'add', 'args': {'x': 2, 'y': 2}, 'id': 'bb51899f-3826-401a-bd79-11548c7bd0aa', 'type': 'tool_call'}] ``` If you take this approach you will need to craft the modified prompt to remove the "thinking" that it does and just return the answer.

GiteaMirror commented

2026-05-03 22:56:43 -05:00

@tomaszbk commented on GitHub (Aug 2, 2024):

Sound great, thanks for your help @rick-github ! :)

@tomaszbk commented on GitHub (Aug 2, 2024): Sound great, thanks for your help @rick-github ! :)

GiteaMirror commented

2026-05-03 22:56:43 -05:00

@rick-github commented on GitHub (Aug 2, 2024):

BTW, your invocation of ChatOllama (on my system, at least) is not quite correct, it doesn't take a prompt parameter:

from langchain_core.tools import tool
from langchain_ollama import ChatOllama

@tool
def add(x: int, y: int) -> int:
    """Adds 2 numbers and returns the result"""
    return x + y
tools = [add]
model = ChatOllama(
    model="llama3.1-tool",
    temperature=0,
).bind_tools(tools)
messages = [
  ( "system", "Your name is botty"),
  ( "user", "what is 2 + 2?"),
]
response = model.invoke(messages)
print(f"response: {response.content}")
print(response.tool_calls)

@rick-github commented on GitHub (Aug 2, 2024): BTW, your invocation of ChatOllama (on my system, at least) is not quite correct, it doesn't take a `prompt` parameter: ```python from langchain_core.tools import tool from langchain_ollama import ChatOllama @tool def add(x: int, y: int) -> int: """Adds 2 numbers and returns the result""" return x + y tools = [add] model = ChatOllama( model="llama3.1-tool", temperature=0, ).bind_tools(tools) messages = [ ( "system", "Your name is botty"), ( "user", "what is 2 + 2?"), ] response = model.invoke(messages) print(f"response: {response.content}") print(response.tool_calls) ```

GiteaMirror commented

2026-05-03 22:56:46 -05:00

@SamuelBG13 commented on GitHub (Aug 13, 2024):

Hello! Thank you for your insights, the fix worked for me as well.

@rick-github I was wondering if it would be sensible to use this modified modelfile as default? Because one of the big advantages of function calling is having a model decide when (and if) to use a tool, for a given query.

Of course, this will probably be handled more holistically once streaming tool calling is working (i.e. parsing tool calls from an arbitrary assistant message) and calling them live. My guess is that this feature will take a while?

@SamuelBG13 commented on GitHub (Aug 13, 2024): Hello! Thank you for your insights, the fix worked for me as well. @rick-github I was wondering if it would be sensible to use this modified modelfile as default? Because one of the big advantages of function calling is having a model decide when (and _if_) to use a tool, for a given query. Of course, this will probably be handled more holistically once streaming tool calling is working (i.e. parsing tool calls from an arbitrary assistant message) and calling them live. My guess is that this feature will take a while?

GiteaMirror commented

2026-05-03 22:56:47 -05:00

@rick-github commented on GitHub (Aug 13, 2024):

The problem with allowing the model to decide whether to use the tool is that it opens up another path for hallucinations. Personally I prefer to have well defined roles for my LLM clients: if it's a tool user, bind tools to it, otherwise it's text generation only. No room for the text generator to take the phrase "stock up on apples" and return a toolcall to buy 100 shares of AAPL.

@rick-github commented on GitHub (Aug 13, 2024): The problem with allowing the model to decide whether to use the tool is that it opens up another path for hallucinations. Personally I prefer to have well defined roles for my LLM clients: if it's a tool user, bind tools to it, otherwise it's text generation only. No room for the text generator to take the phrase "stock up on apples" and return a toolcall to buy 100 shares of AAPL.

GiteaMirror commented

2026-05-03 22:56:49 -05:00

@SamuelBG13 commented on GitHub (Aug 18, 2024):

Personally I prefer to have well defined roles for my LLM clients: if it's a tool user, bind tools to it, otherwise it's text generation only. No room for the text generator to take the phrase "stock up on apples" and return a toolcall to buy 100 shares of AAPL.

I understand, and I fully agree with having well defined roles (and this is actually backed by a recent paper).

I am considering more scenarios with chain of thought or in-context learning strategies. I.e. I want the model to consider the tools it has, process a few ideas or heuristics, and then finally select the tool. As an alternative, sure, one can chain two prompts, one for reasoning and one for structuring. But then one needs to find some intermediate representation of the tools for the "reasoner" LLM (you can't pass the tools as tools) plus add the overhead of two prompts being processed.

@SamuelBG13 commented on GitHub (Aug 18, 2024): > Personally I prefer to have well defined roles for my LLM clients: if it's a tool user, bind tools to it, otherwise it's text generation only. No room for the text generator to take the phrase "stock up on apples" and return a toolcall to buy 100 shares of AAPL. I understand, and I fully agree with having well defined roles (and this is actually backed [by a recent paper](https://arxiv.org/html/2406.00507v1)). I am considering more scenarios with chain of thought or in-context learning strategies. I.e. I want the model to consider the tools it has, process a few ideas or heuristics, and then finally select the tool. As an alternative, sure, one can chain two prompts, one for reasoning and one for structuring. But then one needs to find some intermediate representation of the tools for the "reasoner" LLM (you can't pass the tools as tools) plus add the overhead of two prompts being processed.

GiteaMirror commented

2026-05-03 22:56:51 -05:00

@zhiftyDK commented on GitHub (Sep 18, 2024):

Another solution to this problem could be the following code. Instead of relying on the ollama built in tools function you can customize the user prompt to get an appropirate tools response with arguments, should work with any non tool trained model. Here is the example:

tools.json file:

[
    {
        "name": "set_light_state",
        "description": "Set the current state and brightness of lights",
        "parameters": {
            "type": "object",
            "properties": {
                "brightness": {
                    "type": "integer",
                    "description": "The brightness to set the lights to (0-100)."
                },
                "state": {
                    "type": "string",
                    "description": "The state to set the lights to, either 'on' or 'off'. If state is on and brightness is not specified set brightness to 100.",
                    "enum": ["on", "off"]
                }
            },
            "required": ["state", "brightness"]
        }
    },
    {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and country code, e.g. San Francisco, CA"
                }
            },
            "required": ["location"]
        }
    }
]

tools.py file:

import ollama
import json

with open("./models/tools.json") as f:
    tools = json.loads(f.read())

available_tools = "\n".join(f"Function {tool['name']} to {tool['description']}:\n{tool}" for tool in tools)

toolPrompt = f"""
Analyse the given prompt and decided whether or not it can be answered by a any of the following functions that you have access to:
{available_tools}
 
If you choose to call a function ONLY respond in the JSON format:

{{"name": function name, "parameters": dictionary of argument name and its value}}

Do not use variables.
 
Reminder:
- If looking for real time information use relevant functions before falling back to brave_search
- Function calls MUST follow the specified format
- Required parameters MUST always be specified in the response
- Only call one function at a time
- Put the entire function call reply on one line
"""

print(toolPrompt)

#This is just a demo system message/prompt
system_message = """
You should always give reasonably short answers.
"""
 
response = ollama.chat(model="llama3.1:8b-instruct-q4_0", messages=[
    {"role": "system", "content": system_message},
    {"role": "user", "content": "What is your name?"},
    {"role": "user", "content": toolPrompt},
])

try:
   # Try to load the returned json format if a tool/function was decided to be used
    tool = json.loads(response["message"]["content"])
    print(tool)
except:
    #Print the response if the llm decided not to use a tool/function
    print(response["message"]["content"])

# Here you can do anything you want with the tool variable in the format {"name:" "function name", "parameters": "function paramters/arguments"}

@zhiftyDK commented on GitHub (Sep 18, 2024): Another solution to this problem could be the following code. Instead of relying on the ollama built in tools function you can customize the user prompt to get an appropirate tools response with arguments, should work with any non tool trained model. Here is the example: tools.json file: ```json [ { "name": "set_light_state", "description": "Set the current state and brightness of lights", "parameters": { "type": "object", "properties": { "brightness": { "type": "integer", "description": "The brightness to set the lights to (0-100)." }, "state": { "type": "string", "description": "The state to set the lights to, either 'on' or 'off'. If state is on and brightness is not specified set brightness to 100.", "enum": ["on", "off"] } }, "required": ["state", "brightness"] } }, { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and country code, e.g. San Francisco, CA" } }, "required": ["location"] } } ] ``` tools.py file: ```py import ollama import json with open("./models/tools.json") as f: tools = json.loads(f.read()) available_tools = "\n".join(f"Function {tool['name']} to {tool['description']}:\n{tool}" for tool in tools) toolPrompt = f""" Analyse the given prompt and decided whether or not it can be answered by a any of the following functions that you have access to: {available_tools} If you choose to call a function ONLY respond in the JSON format: {{"name": function name, "parameters": dictionary of argument name and its value}} Do not use variables. Reminder: - If looking for real time information use relevant functions before falling back to brave_search - Function calls MUST follow the specified format - Required parameters MUST always be specified in the response - Only call one function at a time - Put the entire function call reply on one line """ print(toolPrompt) #This is just a demo system message/prompt system_message = """ You should always give reasonably short answers. """ response = ollama.chat(model="llama3.1:8b-instruct-q4_0", messages=[ {"role": "system", "content": system_message}, {"role": "user", "content": "What is your name?"}, {"role": "user", "content": toolPrompt}, ]) try: # Try to load the returned json format if a tool/function was decided to be used tool = json.loads(response["message"]["content"]) print(tool) except: #Print the response if the llm decided not to use a tool/function print(response["message"]["content"]) # Here you can do anything you want with the tool variable in the format {"name:" "function name", "parameters": "function paramters/arguments"} ```

GiteaMirror commented

2026-05-03 22:56:53 -05:00

@pannous commented on GitHub (Sep 27, 2024):

It finally worked after adding

NEVER make up your own parameter values as tool function arguments, like 'city=London'! 
NEVER use tool functions if not asked, instead revert to normal chat!

to the modelfile

Back to prompt engineering like it's 2023

@pannous commented on GitHub (Sep 27, 2024): It finally worked after adding ``` NEVER make up your own parameter values as tool function arguments, like 'city=London'! NEVER use tool functions if not asked, instead revert to normal chat! ``` to the modelfile Back to prompt engineering like it's 2023

GiteaMirror commented

2026-05-03 22:56:55 -05:00

@acastry commented on GitHub (Sep 27, 2024):

NEVER make up your own parameter values as tool function arguments, like 'city=London'!

can you show a complete model file please ?

@acastry commented on GitHub (Sep 27, 2024): > NEVER make up your own parameter values as tool function arguments, like 'city=London'! can you show a complete model file please ?

GiteaMirror commented

2026-05-03 22:56:56 -05:00

@pannous commented on GitHub (Sep 28, 2024):

@acastry same as rick-github's just with these two lines added below

@pannous commented on GitHub (Sep 28, 2024): @acastry same as [rick-github](https://github.com/rick-github)'s just with these two lines added below

GiteaMirror commented

2026-05-03 22:56:57 -05:00

@Master-Pr0grammer commented on GitHub (Oct 26, 2024):

I think this issue needs be fixed, but in the mean time, you can also add a 'respond_to_chat' function if u dont want the added delay from the double prompting method of deciding wether or not it needs a function call.

However since its using a function call to respond, the response might be hindered or limited because the model isnt meant to respond that way, however its good for quick responses.

@Master-Pr0grammer commented on GitHub (Oct 26, 2024): I think this issue needs be fixed, but in the mean time, you can also add a 'respond_to_chat' function if u dont want the added delay from the double prompting method of deciding wether or not it needs a function call. However since its using a function call to respond, the response might be hindered or limited because the model isnt meant to respond that way, however its good for quick responses.

GiteaMirror commented

2026-05-03 22:56:58 -05:00

@shjala commented on GitHub (Dec 31, 2024):

I managed to reduce the unnecessary tool call by checking the user input and only making the tools available if needed, surely not perfect

func toolNeeded(text string) bool {
	text = strings.ToLower(text)
        // add more like "search", "remind(er)", etc.
	return strings.Contains(text, "weather")
}
[...]
	availableTools := []openai.ChatCompletionToolParam{}
	// Check if the user input requires a tool
	if toolNeeded(userInput) {
		availableTools = append(availableTools, weatherTool)
	}

	params := openai.ChatCompletionNewParams{
		Messages: openai.F([]openai.ChatCompletionMessageParamUnion{
			openai.UserMessage(userInput),
		}),
		ParallelToolCalls: openai.F(true),
		Model:             openai.F("llama3.2:3b-tool"),
		Tools:             openai.F(availableTools),
	}
[...]

@shjala commented on GitHub (Dec 31, 2024): I managed to reduce the unnecessary tool call by checking the user input and only making the tools available if needed, surely not perfect ```go func toolNeeded(text string) bool { text = strings.ToLower(text) // add more like "search", "remind(er)", etc. return strings.Contains(text, "weather") } [...] availableTools := []openai.ChatCompletionToolParam{} // Check if the user input requires a tool if toolNeeded(userInput) { availableTools = append(availableTools, weatherTool) } params := openai.ChatCompletionNewParams{ Messages: openai.F([]openai.ChatCompletionMessageParamUnion{ openai.UserMessage(userInput), }), ParallelToolCalls: openai.F(true), Model: openai.F("llama3.2:3b-tool"), Tools: openai.F(availableTools), } [...] ```

GiteaMirror commented

2026-05-03 22:57:00 -05:00

@lemassykoi commented on GitHub (Mar 22, 2025):

import requests
import json
import colorama
YELLOW = colorama.Fore.YELLOW
RESET = colorama.Style.RESET_ALL

model = "llama3.2:latest"
user_query = 'Hello there!'

url = "http://localhost:11434/api/chat"

messages = [
    {
        "role": "user",
        "content": user_query
    }
]

## WITHOUT TOOLS
data = {
    "model": model,
    "messages": messages,
    "stream": False,
    "keep_alive": "1m",
    "options": {'temperature': 0.0, 'seed': 1234567890}
}
headers = {"Content-Type": "application/json"}
payload = json.dumps(data).encode("utf-8")
response = requests.post(url, headers=headers, data=payload, stream=False)

if response.status_code == 200:
    print(response.json())
    print(YELLOW + response.json()['message']['content'] + RESET)

else:
    print(f"Error: {response.status_code}")
    print(response.text)

print('\n')

## WITH TOOLS
tools = [
    {
        'type': 'function',
        'function': {
            'name': 'get_current_weather',
            'description': 'Get the current weather for a city',
            'parameters': {
                'type': 'object',
                'properties': {
                'city': {
                    'type': 'string',
                    'description': 'The name of the city',
                },
            },
            'required': ['city'],
            },
        },
    },
]

data = {
    "model": model,
    "messages": messages,
    "stream": False,
    "keep_alive": "1m",
    "tools": tools,
    "options": {'temperature': 0.0, 'seed': 1234567890}
}
headers = {"Content-Type": "application/json"}
payload = json.dumps(data).encode("utf-8")
response = requests.post(url, headers=headers, data=payload, stream=False)

if response.status_code == 200:
    print(response.json())
    print(YELLOW + response.json()['message']['content'] + RESET)

else:
    print(f"Error: {response.status_code}")
    print(response.text)

case 1:

{'model': 'llama3.2:latest', 'created_at': '2025-03-22T21:06:28.040493426Z', 'message': {'role': 'assistant', 'content': "It's nice to meet you. Is there something I can help you with or would you like to chat?"}, 'done_reason': 'stop', 'done': True, 'total_duration': 1438499087, 'load_duration': 1270149241, 'prompt_eval_count': 28, 'prompt_eval_duration': 22325287, 'eval_count': 23, 'eval_duration': 144218483}

It's nice to meet you. Is there something I can help you with or would you like to chat?

case 2:

{'model': 'llama3.2:latest', 'created_at': '2025-03-22T21:06:28.190441052Z', 'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'function': {'name': 'get_current_weather', 'arguments': {'city': 'New York'}}}]}, 'done_reason': 'stop', 'done': True, 'total_duration': 147643531, 'load_duration': 13826815, 'prompt_eval_count': 162, 'prompt_eval_duration': 3391734, 'eval_count': 19, 'eval_duration': 129858423}

None

This is really problematic

@lemassykoi commented on GitHub (Mar 22, 2025): ```python import requests import json import colorama YELLOW = colorama.Fore.YELLOW RESET = colorama.Style.RESET_ALL model = "llama3.2:latest" user_query = 'Hello there!' url = "http://localhost:11434/api/chat" messages = [ { "role": "user", "content": user_query } ] ## WITHOUT TOOLS data = { "model": model, "messages": messages, "stream": False, "keep_alive": "1m", "options": {'temperature': 0.0, 'seed': 1234567890} } headers = {"Content-Type": "application/json"} payload = json.dumps(data).encode("utf-8") response = requests.post(url, headers=headers, data=payload, stream=False) if response.status_code == 200: print(response.json()) print(YELLOW + response.json()['message']['content'] + RESET) else: print(f"Error: {response.status_code}") print(response.text) print('\n') ## WITH TOOLS tools = [ { 'type': 'function', 'function': { 'name': 'get_current_weather', 'description': 'Get the current weather for a city', 'parameters': { 'type': 'object', 'properties': { 'city': { 'type': 'string', 'description': 'The name of the city', }, }, 'required': ['city'], }, }, }, ] data = { "model": model, "messages": messages, "stream": False, "keep_alive": "1m", "tools": tools, "options": {'temperature': 0.0, 'seed': 1234567890} } headers = {"Content-Type": "application/json"} payload = json.dumps(data).encode("utf-8") response = requests.post(url, headers=headers, data=payload, stream=False) if response.status_code == 200: print(response.json()) print(YELLOW + response.json()['message']['content'] + RESET) else: print(f"Error: {response.status_code}") print(response.text) ``` case 1: ```shell {'model': 'llama3.2:latest', 'created_at': '2025-03-22T21:06:28.040493426Z', 'message': {'role': 'assistant', 'content': "It's nice to meet you. Is there something I can help you with or would you like to chat?"}, 'done_reason': 'stop', 'done': True, 'total_duration': 1438499087, 'load_duration': 1270149241, 'prompt_eval_count': 28, 'prompt_eval_duration': 22325287, 'eval_count': 23, 'eval_duration': 144218483} ``` ```text It's nice to meet you. Is there something I can help you with or would you like to chat? ``` case 2: ```shell {'model': 'llama3.2:latest', 'created_at': '2025-03-22T21:06:28.190441052Z', 'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'function': {'name': 'get_current_weather', 'arguments': {'city': 'New York'}}}]}, 'done_reason': 'stop', 'done': True, 'total_duration': 147643531, 'load_duration': 13826815, 'prompt_eval_count': 162, 'prompt_eval_duration': 3391734, 'eval_count': 19, 'eval_duration': 129858423} ``` ```text None ``` This is really problematic

GiteaMirror commented

2026-05-03 22:57:00 -05:00

@rick-github commented on GitHub (Mar 22, 2025):

llama3.2 doesn't have great discrimination. Try something from the qwen2.5 family.

#!/usr/bin/env python3

import requests
import json
import colorama
import argparse
YELLOW = colorama.Fore.YELLOW
RESET = colorama.Style.RESET_ALL

parser = argparse.ArgumentParser()
parser.add_argument("model", nargs='?', default="llama3.2:latest")
args = parser.parse_args()

user_query = 'Hello there!'

url = "http://localhost:11434/api/chat"

def get_response(user_query, tools):
  data = {
      "model": args.model,
      "messages": [{
        "role": "user",
        "content": user_query
      }],
      "stream": False,
      "keep_alive": "1m",
      "tools": tools,
      "options": {'temperature': 0.0, 'seed': 1234567890}
  }
  headers = {"Content-Type": "application/json"}
  payload = json.dumps(data).encode("utf-8")
  response = requests.post(url, headers=headers, data=payload, stream=False)

  if response.status_code == 200:
      print(response.json())
      print(YELLOW + response.json()['message']['content'] + RESET)

  else:
      print(f"Error: {response.status_code}")
      print(response.text)

  print('\n')

## WITH TOOLS
tools = [
    {
        'type': 'function',
        'function': {
            'name': 'get_current_weather',
            'description': 'Get the current weather for a city',
            'parameters': {
                'type': 'object',
                'properties': {
                'city': {
                    'type': 'string',
                    'description': 'The name of the city',
                },
            },
            'required': ['city'],
            },
        },
    },
]

get_response(user_query, None)
get_response(user_query, tools)
get_response("What's the weather like in Paris?", tools)

$ ./6127.py qwen2.5:0.5b
{'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:40.953201319Z', 'message': {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 299040394, 'load_duration': 251506763, 'prompt_eval_count': 32, 'prompt_eval_duration': 7939767, 'eval_count': 10, 'eval_duration': 35222937}
Hello! How can I assist you today?


{'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:41.235725336Z', 'message': {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 280486460, 'load_duration': 230412881, 'prompt_eval_count': 163, 'prompt_eval_duration': 6175926, 'eval_count': 10, 'eval_duration': 39423668}
Hello! How can I assist you today?


{'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:41.541227609Z', 'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'function': {'name': 'get_current_weather', 'arguments': {'city': 'Paris'}}}]}, 'done_reason': 'stop', 'done': True, 'total_duration': 304057078, 'load_duration': 221519148, 'prompt_eval_count': 168, 'prompt_eval_duration': 5164776, 'eval_count': 21, 'eval_duration': 72863163}

@rick-github commented on GitHub (Mar 22, 2025): llama3.2 doesn't have great discrimination. Try something from the qwen2.5 family. ```python #!/usr/bin/env python3 import requests import json import colorama import argparse YELLOW = colorama.Fore.YELLOW RESET = colorama.Style.RESET_ALL parser = argparse.ArgumentParser() parser.add_argument("model", nargs='?', default="llama3.2:latest") args = parser.parse_args() user_query = 'Hello there!' url = "http://localhost:11434/api/chat" def get_response(user_query, tools): data = { "model": args.model, "messages": [{ "role": "user", "content": user_query }], "stream": False, "keep_alive": "1m", "tools": tools, "options": {'temperature': 0.0, 'seed': 1234567890} } headers = {"Content-Type": "application/json"} payload = json.dumps(data).encode("utf-8") response = requests.post(url, headers=headers, data=payload, stream=False) if response.status_code == 200: print(response.json()) print(YELLOW + response.json()['message']['content'] + RESET) else: print(f"Error: {response.status_code}") print(response.text) print('\n') ## WITH TOOLS tools = [ { 'type': 'function', 'function': { 'name': 'get_current_weather', 'description': 'Get the current weather for a city', 'parameters': { 'type': 'object', 'properties': { 'city': { 'type': 'string', 'description': 'The name of the city', }, }, 'required': ['city'], }, }, }, ] get_response(user_query, None) get_response(user_query, tools) get_response("What's the weather like in Paris?", tools) ``` ```console $ ./6127.py qwen2.5:0.5b {'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:40.953201319Z', 'message': {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 299040394, 'load_duration': 251506763, 'prompt_eval_count': 32, 'prompt_eval_duration': 7939767, 'eval_count': 10, 'eval_duration': 35222937} Hello! How can I assist you today? {'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:41.235725336Z', 'message': {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 280486460, 'load_duration': 230412881, 'prompt_eval_count': 163, 'prompt_eval_duration': 6175926, 'eval_count': 10, 'eval_duration': 39423668} Hello! How can I assist you today? {'model': 'qwen2.5:0.5b', 'created_at': '2025-03-22T21:46:41.541227609Z', 'message': {'role': 'assistant', 'content': '', 'tool_calls': [{'function': {'name': 'get_current_weather', 'arguments': {'city': 'Paris'}}}]}, 'done_reason': 'stop', 'done': True, 'total_duration': 304057078, 'load_duration': 221519148, 'prompt_eval_count': 168, 'prompt_eval_duration': 5164776, 'eval_count': 21, 'eval_duration': 72863163} ```

GiteaMirror commented

2026-05-03 22:57:02 -05:00

@lemassykoi commented on GitHub (Mar 22, 2025):

llama3.2 doesn't have great discrimination. Try something from the qwen2.5 family.

your are right, it's ok for qwen2.5:14b-instruct

@lemassykoi commented on GitHub (Mar 22, 2025): > llama3.2 doesn't have great discrimination. Try something from the qwen2.5 family. your are right, it's ok for `qwen2.5:14b-instruct`

GiteaMirror commented

2026-05-03 22:57:03 -05:00

@adar2378 commented on GitHub (Mar 28, 2025):

is there no proper solution? :/ I overriding system prompt feels kind of like a hack.

@adar2378 commented on GitHub (Mar 28, 2025): is there no proper solution? :/ I overriding system prompt feels kind of like a hack.

GiteaMirror commented

2026-05-03 22:57:05 -05:00

@zhiftyDK commented on GitHub (Mar 29, 2025):

is there no proper solution? :/ I overriding system prompt feels kind of like a hack.

The only proper solution is using a better model, which is why i switched to openai api

@zhiftyDK commented on GitHub (Mar 29, 2025): > is there no proper solution? :/ I overriding system prompt feels kind of like a hack. The only proper solution is using a better model, which is why i switched to openai api

GiteaMirror commented

2026-05-03 22:57:08 -05:00

@JoHavel commented on GitHub (Jul 6, 2025):

Or you can create a tool that does nothing. It seems to work (if the model doesn't need to do anything, it runs this tool and then answers; otherwise, it uses the right tool):

@tool
def do_nothing():
    """ Does nothing """

The downside is that there are two calls to the model if no tool is used.

@JoHavel commented on GitHub (Jul 6, 2025): Or you can create a tool that does nothing. It seems to work (if the model doesn't need to do anything, it runs this tool and then answers; otherwise, it uses the right tool): ``` @tool def do_nothing(): """ Does nothing """ ``` The downside is that there are two calls to the model if no tool is used.

Sign in to join this conversation.

Branches Tags

main

dhiltgen/ci

parth-launch-plan-gating

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#65864