[GH-ISSUE #8287] The <toolcall> in nemotron-mini. Again. #67359

New Issue

GiteaMirror · 2026-05-04T10:05:04-05:00

GiteaMirror commented

2026-05-04 10:05:04 -05:00

Originally created by @tripolskypetr on GitHub (Jan 2, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8287

Originally assigned to: @ParthSareen on GitHub.

What is the issue?

The problem

I am trying to implement the agent swarm for ollama from scratch. I made the triage agent: the intent navigator. It should call the navigate_to_refund_agent_tool or navigate_to_sales_agent_tool depends on user's choose. Both tools got empty arguments like

function navigate_to_refund_agent_tool(/* no arguments */) {

}

function navigate_to_sales_agent_tool(/* no arguments */) {

}

When I send the following curl request it return the nasty XML output.

curl --location 'http://localhost:11434/api/chat' \
--header 'Content-Type: application/json' \
--data '{
    "model": "nemotron-mini",
    "messages": [
        {
            "role": "system",
            "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions"
        },
        {
            "role": "user",
            "content": "Navigate me to sales agent"
        }
    ],
    "stream": false,
    "options": {
        "top_k": 20,
        "top_p": 0.4,
        "temperature": 0.5
    },
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "navigate_to_sales_agent_tool",
                "description": "Navigate to sales agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "navigate_to_refund_agent_tool",
                "description": "Navigate to refund agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        }
    ]
}'

Time-to-time the output is

<toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>

Also, sometimes the JSON in <toolcall> tag is invalid

To solve the problem

How do I hardcode the model version? I am seeing you are updating models without publishing the new tag

This is definitely going to break the AI prompts, I need to fetch exactly the defined model version even if it was uploaded 10 years ago

When are you planning to clear all fake labels with tools support?

The XML output is a real common for all models with tools in the list. This is a fake. Give me the truly information about the state of the framework: without tools it unusable and pointless waste of time

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.5.4

Originally created by @tripolskypetr on GitHub (Jan 2, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8287 Originally assigned to: @ParthSareen on GitHub. ### What is the issue? # The problem I am trying to implement the agent swarm for ollama from scratch. I made the triage agent: the intent navigator. It should call the `navigate_to_refund_agent_tool` or `navigate_to_sales_agent_tool` depends on user's choose. Both tools got empty arguments like ```tsx function navigate_to_refund_agent_tool(/* no arguments */) { } function navigate_to_sales_agent_tool(/* no arguments */) { } ``` When I send the following curl request it return the nasty XML output. ```bash curl --location 'http://localhost:11434/api/chat' \ --header 'Content-Type: application/json' \ --data '{ "model": "nemotron-mini", "messages": [ { "role": "system", "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions" }, { "role": "user", "content": "Navigate me to sales agent" } ], "stream": false, "options": { "top_k": 20, "top_p": 0.4, "temperature": 0.5 }, "tools": [ { "type": "function", "function": { "name": "navigate_to_sales_agent_tool", "description": "Navigate to sales agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } }, { "type": "function", "function": { "name": "navigate_to_refund_agent_tool", "description": "Navigate to refund agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } } ] }' ``` Time-to-time the output is ``` <toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall> ``` Also, sometimes the JSON in `<toolcall>` tag is invalid # To solve the problem 1. **How do I hardcode the model version? I am seeing you are updating models without publishing the new tag** ![image](https://github.com/user-attachments/assets/ac357882-5803-40d0-9c6f-f4b3b1fc6d8f) This is definitely going to break the AI prompts, I need to fetch exactly the defined model version even if it was uploaded 10 years ago 2. **When are you planning to clear all fake labels with tools support?** ![image](https://github.com/user-attachments/assets/155d1fad-aa3f-47bd-b01c-66c8742f5d2e) The XML output is a real common for all models with tools in the list. This is a fake. Give me the truly information about the state of the framework: without tools it unusable and pointless waste of time ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.4

GiteaMirror added the bug label 2026-05-04 10:05:04 -05:00

GiteaMirror closed this issue

2026-05-04 10:05:05 -05:00

GiteaMirror commented

2026-05-04 10:05:07 -05:00

@tripolskypetr commented on GitHub (Jan 2, 2025):

The normal approach for Ollama

@tripolskypetr commented on GitHub (Jan 2, 2025): # The normal approach for Ollama ![image](https://github.com/user-attachments/assets/7e047acb-df4c-4788-9715-b96873ddb1fc)

GiteaMirror commented

2026-05-04 10:05:08 -05:00

@rick-github commented on GitHub (Jan 2, 2025):

LLMs are probabilistic generators, expecting 100% accuracy with untuned prompts and tiny models is setting the ground for failure and frustration.

#!/usr/bin/env python3

import ollama
import sys
import argparse
import json

system=[
"""
You are to triage a users request, and call a tool to transfer to the right agent.
There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool.
Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well.
Do not spam function executions"
""",
"""
You are a tool using triage agent.  Your purpose is to use the available
tools to transfer a user to the appropriate agent.

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>
"""
]
parser = argparse.ArgumentParser()
parser.add_argument("--prompt", default="Navigate me to sales agent")
parser.add_argument("--system", type=int, default=0)
parser.add_argument("model", nargs="?", default="nemotron-mini")
args = parser.parse_args()

def navigate_to_refund_agent_tool() -> bool:
  """
  Navigates the user to the refund agent.

  Args:
    None

  Returns:
    `True` if the navigation succeeded, `False` otherwise.
  """
  print("Navigate to refund agent", file=sys.stderr)
  return True

def navigate_to_sales_agent_tool() -> bool:
  """
  Navigates the user to the sales agent.

  Args:
    None

  Returns:
    `True` if the navigation succeeded, `False` otherwise.
  """
  print("Navigate to sales agent", file=sys.stderr)
  return True
  
tools = [ navigate_to_refund_agent_tool, navigate_to_sales_agent_tool ]

response = ollama.chat(
      args.model,
      messages=[{"role":"system","content":system[args.system]},
                {"role":"user","content":args.prompt}],
      tools=tools,
    )
print(json.dumps(dict(response.message), default=str))

$ for i in {1..100} ; do ./8287.py --system 0 ; done | sort | uniq -c
      5 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null}
     14 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null}
     32 {"role": "assistant", "content": " <toolcall> {\"type\":\"function\",\"arguments\":{\"name\":\"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null}
     16 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {}} </toolcall>", "images": null, "tool_calls": null}
      2 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null}
      5 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null}
      1 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {\n  \"name\": \"navigate_to_sales_agent_tool\"\n}} </toolcall>", "images": null, "tool_calls": null}
      2 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {}} </toolcall>", "images": null, "tool_calls": null}
     23 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"functionName\": \"navigate_to_sales_agent_tool\"} </toolcall>", "images": null, "tool_calls": null}

$  for i in {1..100} ; do ./8287.py --system 1 ; done | sort | uniq -c
     94 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]}
      3 {"role": "assistant", "content": " <tool_call>{\"name\":\"navigate_to_sales_agent_tool\",\"arguments\":{}</tool_call>", "images": null, "tool_calls": null}
      1 {"role": "assistant", "content": " <tool_call>{\"name\": \"navigate_to_sales_agent_tool\", \"\"}</tool_call>", "images": null, "tool_calls": null}
      2 {"role": "assistant", "content": " <tool_call>{\"name\":\"navigate_to_sales_agent_tool\"}</tool_call>", "images": null, "tool_calls": null}

$ for i in {1..100} ; do ./8287.py qwen2.5:0.5b --system 0 ; done | sort | uniq -c
     75 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]}
     25 {"role": "assistant", "content": "<tool_call>\n{\"name\": \"navigate_to_sales_agent_tool\", \"arguments\": null}\n</tool_call>", "images": null, "tool_calls": null}

$ for i in {1..100} ; do ./8287.py qwen2.5:0.5b --system 1 ; done | sort | uniq -c
    100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]}

@rick-github commented on GitHub (Jan 2, 2025): LLMs are probabilistic generators, expecting 100% accuracy with untuned prompts and tiny models is setting the ground for failure and frustration. ```python #!/usr/bin/env python3 import ollama import sys import argparse import json system=[ """ You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions" """, """ You are a tool using triage agent. Your purpose is to use the available tools to transfer a user to the appropriate agent. For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> {"name": <function-name>, "arguments": <args-json-object>} </tool_call> """ ] parser = argparse.ArgumentParser() parser.add_argument("--prompt", default="Navigate me to sales agent") parser.add_argument("--system", type=int, default=0) parser.add_argument("model", nargs="?", default="nemotron-mini") args = parser.parse_args() def navigate_to_refund_agent_tool() -> bool: """ Navigates the user to the refund agent. Args: None Returns: `True` if the navigation succeeded, `False` otherwise. """ print("Navigate to refund agent", file=sys.stderr) return True def navigate_to_sales_agent_tool() -> bool: """ Navigates the user to the sales agent. Args: None Returns: `True` if the navigation succeeded, `False` otherwise. """ print("Navigate to sales agent", file=sys.stderr) return True tools = [ navigate_to_refund_agent_tool, navigate_to_sales_agent_tool ] response = ollama.chat( args.model, messages=[{"role":"system","content":system[args.system]}, {"role":"user","content":args.prompt}], tools=tools, ) print(json.dumps(dict(response.message), default=str)) ``` ```console $ for i in {1..100} ; do ./8287.py --system 0 ; done | sort | uniq -c 5 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null} 14 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null} 32 {"role": "assistant", "content": " <toolcall> {\"type\":\"function\",\"arguments\":{\"name\":\"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null} 16 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {}} </toolcall>", "images": null, "tool_calls": null} 2 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null} 5 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null} 1 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {\n \"name\": \"navigate_to_sales_agent_tool\"\n}} </toolcall>", "images": null, "tool_calls": null} 2 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"data\": {}} </toolcall>", "images": null, "tool_calls": null} 23 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"functionName\": \"navigate_to_sales_agent_tool\"} </toolcall>", "images": null, "tool_calls": null} ``` ```console $ for i in {1..100} ; do ./8287.py --system 1 ; done | sort | uniq -c 94 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]} 3 {"role": "assistant", "content": " <tool_call>{\"name\":\"navigate_to_sales_agent_tool\",\"arguments\":{}</tool_call>", "images": null, "tool_calls": null} 1 {"role": "assistant", "content": " <tool_call>{\"name\": \"navigate_to_sales_agent_tool\", \"\"}</tool_call>", "images": null, "tool_calls": null} 2 {"role": "assistant", "content": " <tool_call>{\"name\":\"navigate_to_sales_agent_tool\"}</tool_call>", "images": null, "tool_calls": null} ``` ```console $ for i in {1..100} ; do ./8287.py qwen2.5:0.5b --system 0 ; done | sort | uniq -c 75 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]} 25 {"role": "assistant", "content": "<tool_call>\n{\"name\": \"navigate_to_sales_agent_tool\", \"arguments\": null}\n</tool_call>", "images": null, "tool_calls": null} ``` ```console $ for i in {1..100} ; do ./8287.py qwen2.5:0.5b --system 1 ; done | sort | uniq -c 100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]} ```

GiteaMirror commented

2026-05-04 10:05:08 -05:00

@tripolskypetr commented on GitHub (Jan 3, 2025):

@rick-github Ok, I got it, without monkey-patching the system prompt, the nemotron-mini model is unusable. Are you planning to make the tools setup process automatic?

tools to transfer a user to the appropriate agent.

For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>```

But this does not solve the fake tools label problem. For example the qwen2:0.5b model does not support tools...

curl --location 'http://localhost:11434/api/chat' \
--header 'Content-Type: application/json' \
--data '{
    "model": "qwen2:0.5b",
    "messages": [
        {
            "role": "system",
            "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions"
        },
        {
            "role": "user",
            "content": "Navigate me to sales agent"
        }
    ],
    "stream": false,
    "options": {
        "top_k": 20,
        "top_p": 0.4,
        "temperature": 0.5
    },
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "navigate_to_sales_agent_tool",
                "description": "Navigate to sales agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "navigate_to_refund_agent_tool",
                "description": "Navigate to refund agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        }
    ]
}'

@tripolskypetr commented on GitHub (Jan 3, 2025): @rick-github Ok, I got it, without [monkey-patching](https://en.wikipedia.org/wiki/Monkey_patch) the system prompt, the `nemotron-mini` model is unusable. **Are you planning to make the tools setup process automatic?** > ```You are a tool using triage agent. Your purpose is to use the available > tools to transfer a user to the appropriate agent. > > For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: > <tool_call> > {"name": <function-name>, "arguments": <args-json-object>} > </tool_call>``` But this does not solve the fake tools label problem. For example the `qwen2:0.5b` model does not support tools... ![image](https://github.com/user-attachments/assets/c68b8451-efc0-49a7-a4df-db989cfb3c25) ```bash curl --location 'http://localhost:11434/api/chat' \ --header 'Content-Type: application/json' \ --data '{ "model": "qwen2:0.5b", "messages": [ { "role": "system", "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions" }, { "role": "user", "content": "Navigate me to sales agent" } ], "stream": false, "options": { "top_k": 20, "top_p": 0.4, "temperature": 0.5 }, "tools": [ { "type": "function", "function": { "name": "navigate_to_sales_agent_tool", "description": "Navigate to sales agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } }, { "type": "function", "function": { "name": "navigate_to_refund_agent_tool", "description": "Navigate to refund agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } } ] }' ```

GiteaMirror commented

2026-05-04 10:05:08 -05:00

@rick-github commented on GitHub (Jan 3, 2025):

@rick-github Ok, I got it, without monkey-patching the system prompt, the nemotron-mini model is unusable. Are you planning to make the tools setup process automatic?

Prompt "engineering". Models require prompting. Getting good results requires putting effort into writing good prompts.

But this does not solve the fake tools label problem. For example the qwen2:0.5b model does not support tools...

So choose a model that does. Or modify the prompt template to support tools, it's not hard.

$ ollama show --modelfile qwen2:0.5b > Modelfile
$ ollama show --template qwen2:7b-instruct-q4_K_M >> Modelfile
# clean up Modelfile
$ ollama create qwen2-tools:0.5b-instruct-q4_K_M

$ curl -s --location 'http://localhost:11434/api/chat' --header 'Content-Type: application/json' --data '{
    "model": "qwen2-tools:0.5b-instruct-q4_K_M",
    "messages": [
        {
            "role": "system",
            "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions"
        },
        {
            "role": "user",
            "content": "Navigate me to sales agent"
        }
    ],
    "stream": false,
    "options": {
        "top_k": 20,
        "top_p": 0.4,
        "temperature": 0.5
    },
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "navigate_to_sales_agent_tool",
                "description": "Navigate to sales agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "navigate_to_refund_agent_tool",
                "description": "Navigate to refund agent",
                "parameters": {
                    "type": "object",
                    "required": [
                    ],
                    "properties": {
                    }
                }
            }
        }
    ]
}' | jq
{
  "model": "qwen2-tools:0.5b-instruct-q4_K_M",
  "created_at": "2025-01-03T12:21:17.981463621Z",
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "navigate_to_sales_agent_tool",
          "arguments": {
            "type": "object"
          }
        }
      }
    ]
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 141645527,
  "load_duration": 10852660,
  "prompt_eval_count": 240,
  "prompt_eval_duration": 4000000,
  "eval_count": 28,
  "eval_duration": 121000000
}

But there's a reason why it's not given tool capability off the bat - it's poor at it. Which is why time is better spent in choosing an appropriate model and writing good prompts.

@rick-github commented on GitHub (Jan 3, 2025): > @rick-github Ok, I got it, without [monkey-patching](https://en.wikipedia.org/wiki/Monkey_patch) the system prompt, the `nemotron-mini` model is unusable. **Are you planning to make the tools setup process automatic?** Prompt "engineering". Models require prompting. Getting good results requires putting effort into writing good prompts. > But this does not solve the fake tools label problem. For example the `qwen2:0.5b` model does not support tools... So choose a model that does. Or modify the prompt template to support tools, it's not hard. ```console $ ollama show --modelfile qwen2:0.5b > Modelfile $ ollama show --template qwen2:7b-instruct-q4_K_M >> Modelfile # clean up Modelfile $ ollama create qwen2-tools:0.5b-instruct-q4_K_M ``` ```console $ curl -s --location 'http://localhost:11434/api/chat' --header 'Content-Type: application/json' --data '{ "model": "qwen2-tools:0.5b-instruct-q4_K_M", "messages": [ { "role": "system", "content": "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents you can transfer to: navigate_to_refund_agent_tool and navigate_to_sales_agent_tool. Untill calling any function, you must ask the user for their agent. Before navigation make sure you choose well. Do not spam function executions" }, { "role": "user", "content": "Navigate me to sales agent" } ], "stream": false, "options": { "top_k": 20, "top_p": 0.4, "temperature": 0.5 }, "tools": [ { "type": "function", "function": { "name": "navigate_to_sales_agent_tool", "description": "Navigate to sales agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } }, { "type": "function", "function": { "name": "navigate_to_refund_agent_tool", "description": "Navigate to refund agent", "parameters": { "type": "object", "required": [ ], "properties": { } } } } ] }' | jq { "model": "qwen2-tools:0.5b-instruct-q4_K_M", "created_at": "2025-01-03T12:21:17.981463621Z", "message": { "role": "assistant", "content": "", "tool_calls": [ { "function": { "name": "navigate_to_sales_agent_tool", "arguments": { "type": "object" } } } ] }, "done_reason": "stop", "done": true, "total_duration": 141645527, "load_duration": 10852660, "prompt_eval_count": 240, "prompt_eval_duration": 4000000, "eval_count": 28, "eval_duration": 121000000 } ``` But there's a reason why it's not given tool capability off the bat - it's poor at it. Which is why time is better spent in choosing an appropriate model and writing good prompts.

GiteaMirror commented

2026-05-04 10:05:09 -05:00

@tripolskypetr commented on GitHub (Jan 3, 2025):

@rick-github

I got a discussion in the discord channel. To solve that issue, you need to update all system prompt for each model to match the tool execution protocol. Or make a documentation of the xml schema received from the model to execute the function

It should be

1. Build into system prompt out of the box

or

2. Well-documented so I can write the prompt on my own

right now both of options are not resolved

@tripolskypetr commented on GitHub (Jan 3, 2025): @rick-github I got a discussion in the discord channel. To solve that issue, you need to update all system prompt for each model to match the tool execution protocol. Or make a documentation of the xml schema received from the model to execute the function It should be **1. Build into system prompt out of the box** or **2. Well-documented so I can write the prompt on my own** right now both of options are not resolved ![image](https://github.com/user-attachments/assets/91108bbe-69a9-42d3-8d26-019c205cf3a6)

GiteaMirror commented

2026-05-04 10:05:09 -05:00

@tripolskypetr commented on GitHub (Jan 3, 2025):

@rick-github

In the case of nemotron-mini, the ollama system prompt is invalid. How do I rewrite it?

For example, the mistral system prompt is correct so the model execute the tools ok

@tripolskypetr commented on GitHub (Jan 3, 2025): @rick-github In the case of `nemotron-mini`, **the ollama system prompt is invalid. How do I rewrite it?** ![image](https://github.com/user-attachments/assets/0e904ecc-e4ab-4b3e-a3f0-dce322e88088) For example, the `mistral` system prompt is correct so the model execute the tools ok

GiteaMirror commented

2026-05-04 10:05:10 -05:00

@rick-github commented on GitHub (Jan 3, 2025):

Build into system prompt out of the box

It usually is, that's what the template is for. But as you've noted, sometimes it's not correct, and the model is updated like the original nemotron-mini in #6870. Other times, users feel that's it's not implemented correctly, and offer their own take, eg for glm4 in #6505.

Well-documented so I can write the prompt on my own

There are many examples of prompts in the library and in example. If you want to know more about go templating, see here.

In the case of nemotron-mini, the ollama system prompt is invalid. How do I rewrite it?

You can dump the modelfile, adjust the template, and create a model:

$ ollama show --modelfile nemotron-mini > Modelfile
$ vi Modelfile # or nano
$ ollama create nemotron-mini-mytemplate

If you use /api/generate, you can pass a template via the template field in the API call. This can be used for quick verification without having to re-create a model every time you want to try out a new iteration. The other way to do quick testing of a template is to create a simple go program that does the template instantiation and evaluation.

If you want to hew closer to the template from the nemotron-mini model source, you can look at it here.

@rick-github commented on GitHub (Jan 3, 2025): > 1. Build into system prompt out of the box It usually is, that's what the template is for. But as you've noted, sometimes it's not correct, and the model is updated like the original nemotron-mini in #6870. Other times, users feel that's it's not implemented correctly, and offer their own take, eg for glm4 in #6505. > 2. Well-documented so I can write the prompt on my own There are many examples of prompts in the library and in [example](https://github.com/ollama/ollama/tree/main/template). If you want to know more about go templating, see [here](https://pkg.go.dev/text/template). > In the case of nemotron-mini, the ollama system prompt is invalid. How do I rewrite it? You can dump the modelfile, adjust the template, and create a model: ```shell $ ollama show --modelfile nemotron-mini > Modelfile $ vi Modelfile # or nano $ ollama create nemotron-mini-mytemplate ``` If you use `/api/generate`, you can pass a template via the [`template`](https://github.com/ollama/ollama/blob/main/docs/api.md#:~:text=template,-%3A%20the%20prompt%20template) field in the API call. This can be used for quick verification without having to re-create a model every time you want to try out a new iteration. The other way to do quick testing of a template is to create a simple go program that does the template instantiation and evaluation. If you want to hew closer to the template from the `nemotron-mini` model source, you can look at it [here](https://huggingface.co/nvidia/Nemotron-Mini-4B-Instruct/blob/6a417790c444fd65a3da6a5c8821de6afc9654a6/tokenizer_config.json#L8030).

GiteaMirror commented

2026-05-04 10:05:10 -05:00

@tripolskypetr commented on GitHub (Jan 3, 2025):

@rick-github

It usually is, that's what the template is for...

It usually is means there are a lot of models with are untested for tools execution. All of them got a fake tools label

Well-documented so I can write the prompt on my own

There is no a single sentence in documentation which exactly XML i need to format to make the model execute a tool. All I know right now is the magic <tool_call></tool_call> XML tag, but I don't have any details about this XML schema at all

@tripolskypetr commented on GitHub (Jan 3, 2025): @rick-github > 1. It usually is, that's what the template is for... ***It usually is*** means there are a lot of models with are untested for tools execution. All of them got a fake `tools` label > 2. Well-documented so I can write the prompt on my own There is no a single sentence in documentation which exactly XML i need to format to make the model execute a tool. All I know right now is the magic `<tool_call></tool_call>` XML tag, but I don't have any details about this XML schema at all

GiteaMirror commented

2026-05-04 10:05:11 -05:00

@rick-github commented on GitHub (Jan 3, 2025):

All of them got a fake tools label

In your red circles, you have tools and 0.5b. These are attributes that some of the models in the qwen2 family have. There are other attributes listed there, like 7b and 72b. Not surprisingly, not all models in the qwen2 family have 72 billion parameters.

There is no a single sentence in documentation which exactly XML i need to format to make the model execute a tool.

That is model dependent. I've already pointed you to where you can find the model specific chat template. You take that template, convert it into a go template, and create a TEMPLATE in the Modelfile. Some models will have <tool_call></tool_call> constructs, others won't, eg granite3. In fact, the python script above that I used to test models uses the wrong form of tool construction for nemotron-mini. As it happens, the model training was such that it was mostly able to interpret the instruction even though it wasn't correct.

@rick-github commented on GitHub (Jan 3, 2025): > All of them got a fake tools label In your red circles, you have `tools` and `0.5b`. These are attributes that some of the models in the `qwen2` family have. There are other attributes listed there, like `7b` and `72b`. Not surprisingly, not all models in the `qwen2` family have 72 billion parameters. > There is no a single sentence in documentation which exactly XML i need to format to make the model execute a tool. That is model dependent. I've already pointed you to where you can find the model specific chat template. You take that template, convert it into a go template, and create a TEMPLATE in the Modelfile. Some models will have `<tool_call></tool_call>` constructs, others won't, eg [granite3](https://huggingface.co/ibm-granite/granite-3.0-8b-instruct/blob/5af56291a1c6ca9056df597aab0cff53edabddb0/tokenizer_config.json#L188). In fact, the python script above that I used to test models uses the wrong form of tool construction for nemotron-mini. As it happens, the model training was such that it was mostly able to interpret the instruction even though it wasn't correct.

GiteaMirror commented

2026-05-04 10:05:11 -05:00

@rick-github commented on GitHub (Jan 4, 2025):

--- Modelfile.orig	2025-01-04 17:01:06.384036908 +1000
+++ Modelfile	2025-01-04 17:02:38.365310711 +1000
@@ -9,8 +9,13 @@
 
 {{ end }}
 {{- if .Tools }}
+The following tools are available:
 {{- range .Tools }}<tool> {{ . }} </tool>{{ end }}
 
+For each tool call, return a json object with function name and arguments within <toolcall></toolcall> XML tags:
+<toolcall>
+{"name": <function-name>, "arguments": <args-json-object>}
+</toolcall>
 
 {{ end }}
 {{- end }}

@rick-github commented on GitHub (Jan 4, 2025): ```diff --- Modelfile.orig 2025-01-04 17:01:06.384036908 +1000 +++ Modelfile 2025-01-04 17:02:38.365310711 +1000 @@ -9,8 +9,13 @@ {{ end }} {{- if .Tools }} +The following tools are available: {{- range .Tools }}<tool> {{ . }} </tool>{{ end }} +For each tool call, return a json object with function name and arguments within <toolcall></toolcall> XML tags: +<toolcall> +{"name": <function-name>, "arguments": <args-json-object>} +</toolcall> {{ end }} {{- end }} ```

GiteaMirror commented

2026-05-04 10:05:14 -05:00

@tripolskypetr commented on GitHub (Jan 4, 2025):

@rick-github

Ok, I got it. The function call XML schema is different model to model

That is model dependent. I've already pointed you to where you can find the model specific chat template. You take that template, convert it into a go template, and create a TEMPLATE in the Modelfile. Some models will have <tool_call></tool_call> constructs,

I need to exactly know which schema is need to be returned by the model to make Ollama detect the tool call. This schema is defined in the Ollama code but not documented

{
  "tool_calls": {
    "type": "function",
    "function": {
      "name": "example-function",
      "arguments": {
        "foo": "bar"
      }
    }
  }
}

Is the next model output will be correct to make Ollama return the tool_calls object as a result of api/chat request?

<toolcall>
{"name": "example-function", "arguments": {"foo": "bar"}}
</toolcall>

Are you sure about <toolcall> tag. If so, why am I seeing it in the message.content text output? Maybe I need to use <tool_call> tag?

@tripolskypetr commented on GitHub (Jan 4, 2025): @rick-github Ok, I got it. The function call XML schema is different model to model > That is model dependent. I've already pointed you to where you can find the model specific chat template. You take that template, convert it into a go template, and create a TEMPLATE in the Modelfile. Some models will have <tool_call></tool_call> constructs, I need to exactly know **which schema is need to be returned by the model to make Ollama detect the tool call**. This schema is defined in the Ollama code but not documented ```json { "tool_calls": { "type": "function", "function": { "name": "example-function", "arguments": { "foo": "bar" } } } } ``` Is the next model output will be correct to make Ollama return the `tool_calls` object as a result of `api/chat` request? ```xml <toolcall> {"name": "example-function", "arguments": {"foo": "bar"}} </toolcall> ``` Are you sure about `<toolcall>` tag. If so, why am I seeing it in the `message.content` text output? Maybe I need to use `<tool_call>` tag?

GiteaMirror commented

2026-05-04 10:05:16 -05:00

@rick-github commented on GitHub (Jan 5, 2025):

You have to relax your grip on the idea of an "exact schema". LLMs are not schema processing engines, they interpret language.

The chat template of a model (nemotron-mini, mistral, qwen2.5) provides guidance on how the model is to interpret input and generate output. The client deciphers the output and takes necessary actions. In ollama, the chat template is converted to a go template in order to facilitate this step. The .ToolCalls property (as you pointed out for mistral, above) is used as the guide for extracting the tool call - content from the model that matches a JSON element in .ToolCalls is interpreted as a tool call. The XML tags and other structures are syntactic sugar - using then will likely improve the models adherence to instructions if the model was trained with them, but they are secondary to the ability of the model to generate output in accordance with the .ToolCalls structure. This is why nemotron-mini was able to generate tool calls in the python script even though I used the incorrect XML tags - the important part was matching .ToolCalls.

Let's take a look at the chat template for nemotron-mini:

{{'<extra_id_0>System'}}{% for message in messages %}{% if message['role'] == 'system' %}{{'
' + message['content'].strip()}}{% if tools or contexts %}{{'
'}}{% endif %}{% endif %}{% endfor %}{% if tools %}{% for tool in tools %}{{ '
<tool> ' + tool|tojson + ' </tool>' }}{% endfor %}{% endif %}{% if contexts %}{% if tools %}{{'
'}}{% endif %}{% for context in contexts %}{{ '
<context> ' + context.strip() + ' </context>' }}{% endfor %}{% endif %}{{'

'}}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<extra_id_1>User
' + message['content'].strip() + '
' }}{% elif message['role'] == 'assistant' %}{{ '<extra_id_1>Assistant
' + message['content'].strip() + '
' }}{% elif message['role'] == 'tool' %}{{ '<extra_id_1>Tool
' + message['content'].strip() + '
' }}{% endif %}{% endfor %}{%- if add_generation_prompt %}{{'<extra_id_1>Assistant
'}}{%- endif %}

We see here that the model authors enabled the enumeration of the tools, but didn't add context for generating tool call output. This got converted to a go template:

{{- if (or .Tools .System) }}<extra_id_0>System
{{ if .System }}{{ .System }}


{{ end }}
{{- if .Tools }}
{{- range .Tools }}<tool> {{ . }} </tool>{{ end }}


{{ end }}
{{- end }}
{{- range $i, $m := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if eq .Role "user" }}<extra_id_1>User
{{ .Content }}
{{- if $last }}
<extra_id_1>Assistant
{{- end }}
{{ else if eq .Role "tool" }}<extra_id_1>Tool
{{ .Content }}
{{- if $last }}
<extra_id_1>Assistant
{{- end }}
{{ else if eq .Role "assistant" }}<extra_id_1>Assistant
{{- if .ToolCalls }}
{{ range .ToolCalls }}<toolcall> {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} </toolcall> {{ end }}
{{ else }}
{{ .Content }}
{{- if not $last }}
{{ end }}
{{- end }}
{{- end }}
{{- end }}

We see that ollama will try to match model output of the form {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} as a tool call. It's important to note that the match must be exact. If we go back to my first response with the modified prompt, we see the following failure:

{"role": "assistant", "content": " <tool_call>{\"name\": \"navigate_to_sales_agent_tool\", \"\"}</tool_call>", "images": null, "tool_calls": null}

The response is well formed but missing the arguments field, and so is not detected as a tool call.

The ability of nemotron-mini to respond with tool calls can be improved by enhancing the prompt. By telling the model the sort of output we are expecting (https://github.com/ollama/ollama/issues/8287#issuecomment-2570442336) we get much better compliance.

$ for i in {1..100} ; do ./8287.py --system 0 nemotron-mini ; done | sort | uniq -c
      9 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null}
     23 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null}
     37 {"role": "assistant", "content": " <toolcall> {\"type\":\"function\",\"arguments\":{\"name\":\"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null}
      6 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {}} </toolcall>", "images": null, "tool_calls": null}
     25 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"functionName\": \"navigate_to_sales_agent_tool\"} </toolcall>", "images": null, "tool_calls": null}
$ diff <(ollama show --template nemotron-mini) <(ollama show --template nemotron-mini-mytemplate)
6a7
> The following tools are available:
8a10,13
> For each tool call, return a json object with function name and arguments within <toolcall></toolcall> XML tags:
> <toolcall>
> {"name": <function-name>, "arguments": <args-json-object>}
> </toolcall>
$ for i in {1..100} ; do ./8287.py --system 0 nemotron-mini-mytemplate ; done | sort | uniq -c
    100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]}

So, in summary:

Add .ToolCalls JSON templates to the prompt using the structure you want.
Tell the model to output tool calls using that template.
Be prepared for the occasional failure.

@rick-github commented on GitHub (Jan 5, 2025): You have to relax your grip on the idea of an "exact schema". LLMs are not schema processing engines, they interpret language. The chat template of a model ([nemotron-mini](https://huggingface.co/nvidia/Nemotron-Mini-4B-Instruct/blob/6a417790c444fd65a3da6a5c8821de6afc9654a6/tokenizer_config.json#L8030), [mistral](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3/blob/e0bc86c23ce5aae1db576c8cca6f06f1f73af2db/tokenizer_config.json#L6176), [qwen2.5](https://huggingface.co/Qwen/Qwen2.5-72B-Instruct/blob/d3d951150c1e5848237cd6a7ad11df4836aee842/tokenizer_config.json#L198)) provides guidance on how the model is to interpret input and generate output. The client deciphers the output and takes necessary actions. In ollama, the chat template is converted to a go template in order to facilitate this step. The `.ToolCalls` property (as you pointed out for mistral, above) is used as the guide for extracting the tool call - content from the model that matches a JSON element in `.ToolCalls` is interpreted as a tool call. The XML tags and other structures are syntactic sugar - using then will likely improve the models adherence to instructions if the model was trained with them, but they are secondary to the ability of the model to generate output in accordance with the `.ToolCalls` structure. This is why `nemotron-mini` was able to generate tool calls in the python script even though I used the incorrect XML tags - the important part was matching `.ToolCalls`. Let's take a look at the chat template for `nemotron-mini`: ```jinja {{'<extra_id_0>System'}}{% for message in messages %}{% if message['role'] == 'system' %}{{' ' + message['content'].strip()}}{% if tools or contexts %}{{' '}}{% endif %}{% endif %}{% endfor %}{% if tools %}{% for tool in tools %}{{ ' <tool> ' + tool|tojson + ' </tool>' }}{% endfor %}{% endif %}{% if contexts %}{% if tools %}{{' '}}{% endif %}{% for context in contexts %}{{ ' <context> ' + context.strip() + ' </context>' }}{% endfor %}{% endif %}{{' '}}{% for message in messages %}{% if message['role'] == 'user' %}{{ '<extra_id_1>User ' + message['content'].strip() + ' ' }}{% elif message['role'] == 'assistant' %}{{ '<extra_id_1>Assistant ' + message['content'].strip() + ' ' }}{% elif message['role'] == 'tool' %}{{ '<extra_id_1>Tool ' + message['content'].strip() + ' ' }}{% endif %}{% endfor %}{%- if add_generation_prompt %}{{'<extra_id_1>Assistant '}}{%- endif %} ``` We see here that the model authors enabled the enumeration of the tools, but didn't add context for generating tool call output. This got converted to a go template: ```go {{- if (or .Tools .System) }}<extra_id_0>System {{ if .System }}{{ .System }} {{ end }} {{- if .Tools }} {{- range .Tools }}<tool> {{ . }} </tool>{{ end }} {{ end }} {{- end }} {{- range $i, $m := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 -}} {{- if eq .Role "user" }}<extra_id_1>User {{ .Content }} {{- if $last }} <extra_id_1>Assistant {{- end }} {{ else if eq .Role "tool" }}<extra_id_1>Tool {{ .Content }} {{- if $last }} <extra_id_1>Assistant {{- end }} {{ else if eq .Role "assistant" }}<extra_id_1>Assistant {{- if .ToolCalls }} {{ range .ToolCalls }}<toolcall> {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} </toolcall> {{ end }} {{ else }} {{ .Content }} {{- if not $last }} {{ end }} {{- end }} {{- end }} {{- end }} ``` We see that ollama will try to match model output of the form `{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}` as a tool call. It's important to note that the match must be exact. If we go back to my first response with the modified prompt, we see the following failure: ``` {"role": "assistant", "content": " <tool_call>{\"name\": \"navigate_to_sales_agent_tool\", \"\"}</tool_call>", "images": null, "tool_calls": null} ``` The response is well formed but missing the `arguments` field, and so is not detected as a tool call. The ability of `nemotron-mini` to respond with tool calls can be improved by enhancing the prompt. By telling the model the sort of output we are expecting (https://github.com/ollama/ollama/issues/8287#issuecomment-2570442336) we get much better compliance. ```console $ for i in {1..100} ; do ./8287.py --system 0 nemotron-mini ; done | sort | uniq -c 9 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": { \"name\": \"navigate_to_sales_agent_tool\" }} </toolcall>", "images": null, "tool_calls": null} 23 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {\"name\": \"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null} 37 {"role": "assistant", "content": " <toolcall> {\"type\":\"function\",\"arguments\":{\"name\":\"navigate_to_sales_agent_tool\"}} </toolcall>", "images": null, "tool_calls": null} 6 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"arguments\": {}} </toolcall>", "images": null, "tool_calls": null} 25 {"role": "assistant", "content": " <toolcall> {\"type\": \"function\", \"functionName\": \"navigate_to_sales_agent_tool\"} </toolcall>", "images": null, "tool_calls": null} $ diff <(ollama show --template nemotron-mini) <(ollama show --template nemotron-mini-mytemplate) 6a7 > The following tools are available: 8a10,13 > For each tool call, return a json object with function name and arguments within <toolcall></toolcall> XML tags: > <toolcall> > {"name": <function-name>, "arguments": <args-json-object>} > </toolcall> $ for i in {1..100} ; do ./8287.py --system 0 nemotron-mini-mytemplate ; done | sort | uniq -c 100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_to_sales_agent_tool', arguments={})"]} ``` So, in summary: 1. Add `.ToolCalls` JSON templates to the prompt using the structure you want. 2. Tell the model to output tool calls using that template. 3. Be prepared for the occasional failure.

GiteaMirror commented

2026-05-04 10:05:17 -05:00

@tripolskypetr commented on GitHub (Jan 5, 2025):

@rick-github @mxyng

The problem is parseToolCalls function only really works if the input contains tool calls in some JSON format. I can't find where that JSON format is specified.

// parseToolCalls attempts to parse a JSON string into a slice of ToolCalls.
// mxyng: this only really works if the input contains tool calls in some JSON format
func (m *Model) parseToolCalls(s string) ([]api.ToolCall, bool) {
	// create a subtree from the node that ranges over .ToolCalls
	tmpl := m.Template.Subtree(func(n parse.Node) bool {
		if t, ok := n.(*parse.RangeNode); ok {
			return slices.Contains(template.Identifiers(t.Pipe), "ToolCalls")
		}

		return false
	})

	if tmpl == nil {
		return nil, false
	}
	
	...

This is not about models at all. This issue is related to the ollama parseToolCalls function does not accept the nemotron-mini tool calling format

@tripolskypetr commented on GitHub (Jan 5, 2025): @rick-github @mxyng The problem is [parseToolCalls](https://github.com/ollama/ollama/blob/86a622cbdc69e9fd501764ff7565e977fc98f00a/server/model.go#L158) function `only really works if the input contains tool calls in some JSON format`. I can't find where that `JSON format` is specified. ```go // parseToolCalls attempts to parse a JSON string into a slice of ToolCalls. // mxyng: this only really works if the input contains tool calls in some JSON format func (m *Model) parseToolCalls(s string) ([]api.ToolCall, bool) { // create a subtree from the node that ranges over .ToolCalls tmpl := m.Template.Subtree(func(n parse.Node) bool { if t, ok := n.(*parse.RangeNode); ok { return slices.Contains(template.Identifiers(t.Pipe), "ToolCalls") } return false }) if tmpl == nil { return nil, false } ... ``` This is not about models at all. This issue is related to the ollama `parseToolCalls` function does not accept the `nemotron-mini` tool calling format

GiteaMirror commented

2026-05-04 10:05:18 -05:00

@rick-github commented on GitHub (Jan 5, 2025):

OK, maybe a simpler example. This is the entire Modelfile for a new model using the GGUF file from nemotron-mini:

FROM ./sha256-1dcbd925825b41744ddc2fc3047db6d3ad0aecf8d336f4fadc044eaaf79779d5
TEMPLATE """
{{- if .Tools }}<extra_id_0>System

You have the following tools:
{{- range .Tools }}<tool> {{ . }} </tool>{{ end }}

For each tool call, return a json object as below:
{"name": <tool-name>, "arguments": <json-tool-arguments>}

{{ range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }}

{{ end }}
<extra_id_1>User
{{ range .Messages }}{{ .Content }}{{ end }}
<extra_id_1>Assistant
"""

Looking at the tool relevant portions: give the model the tools it can use. This comes from the chat template:

You have the following tools:
{{- range .Tools }}<tool> {{ . }} </tool>{{ end }}

Tell the model how to make to a tool call:

For each tool call, return a json object as below:
{"name": <tool-name>, "arguments": <json-tool-arguments>}

Tell ollama how to interpret a tool call:

{{ range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }}

Create the model:

$ ollama create nemotron-mini-simple

The prompt:

$ curl localhost:11434/api/chat -d '
{
  "model": "nemotron-mini-simple",
  "stream": false,
  "messages": [
    {
      "role": "user",
      "content": "what is 10 + 20 + 30?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "add",
        "parameters": {
          "type": "object",
          "properties": {
            "l": {
              "type": "array"
            }
          }
        }
      }
    }
  ]
}
'

The response:

{
  "model": "nemotron-mini-simple",
  "created_at": "2025-01-05T15:14:51.201437283Z",
  "message": {
    "role": "assistant",
    "content": "",
    "tool_calls": [
      {
        "function": {
          "name": "add",
          "arguments": {
            "l": [
              10,
              20,
              30
            ]
          }
        }
      }
    ]
  },
  "done_reason": "stop",
  "done": true,
  "total_duration": 207524009,
  "load_duration": 16277673,
  "prompt_eval_count": 111,
  "prompt_eval_duration": 2000000,
  "eval_count": 20,
  "eval_duration": 188000000
}

Because we have been economical with the function definition and the system message, the compliance rate of this model is not great, but it's illustrative of the process of setting up a model for tool use.

This issue is related to the ollama parseToolCalls function does not accept the nemotron-mini tool calling format

If you mean that the current prompt template is missing the instruction on generating a tool call, correct. There is no "nemotron-mini tool calling format".

@rick-github commented on GitHub (Jan 5, 2025): OK, maybe a simpler example. This is the entire Modelfile for a new model using the GGUF file from `nemotron-mini`: ```modelfile FROM ./sha256-1dcbd925825b41744ddc2fc3047db6d3ad0aecf8d336f4fadc044eaaf79779d5 TEMPLATE """ {{- if .Tools }}<extra_id_0>System You have the following tools: {{- range .Tools }}<tool> {{ . }} </tool>{{ end }} For each tool call, return a json object as below: {"name": <tool-name>, "arguments": <json-tool-arguments>} {{ range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }} {{ end }} <extra_id_1>User {{ range .Messages }}{{ .Content }}{{ end }} <extra_id_1>Assistant """ ``` Looking at the tool relevant portions: give the model the tools it can use. This comes from the chat template: ``` You have the following tools: {{- range .Tools }}<tool> {{ . }} </tool>{{ end }} ``` Tell the model how to make to a tool call: ``` For each tool call, return a json object as below: {"name": <tool-name>, "arguments": <json-tool-arguments>} ``` Tell ollama how to interpret a tool call: ``` {{ range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{ end }} ``` Create the model: ```console $ ollama create nemotron-mini-simple ``` The prompt: ``` $ curl localhost:11434/api/chat -d ' { "model": "nemotron-mini-simple", "stream": false, "messages": [ { "role": "user", "content": "what is 10 + 20 + 30?" } ], "tools": [ { "type": "function", "function": { "name": "add", "parameters": { "type": "object", "properties": { "l": { "type": "array" } } } } } ] } ' ``` The response: ```json { "model": "nemotron-mini-simple", "created_at": "2025-01-05T15:14:51.201437283Z", "message": { "role": "assistant", "content": "", "tool_calls": [ { "function": { "name": "add", "arguments": { "l": [ 10, 20, 30 ] } } } ] }, "done_reason": "stop", "done": true, "total_duration": 207524009, "load_duration": 16277673, "prompt_eval_count": 111, "prompt_eval_duration": 2000000, "eval_count": 20, "eval_duration": 188000000 } ``` Because we have been economical with the function definition and the system message, the compliance rate of this model is not great, but it's illustrative of the process of setting up a model for tool use. > This issue is related to the ollama parseToolCalls function does not accept the nemotron-mini tool calling format If you mean that the current prompt template is missing the instruction on generating a tool call, correct. There is no "nemotron-mini tool calling format".

GiteaMirror commented

2026-05-04 10:05:19 -05:00

@tripolskypetr commented on GitHub (Jan 5, 2025):

@rick-github Why don't the invalid model prompts are fixed out of the box of ollama?

@tripolskypetr commented on GitHub (Jan 5, 2025): @rick-github Why don't the invalid model prompts are fixed out of the box of ollama?

GiteaMirror commented

2026-05-04 10:05:20 -05:00

@rick-github commented on GitHub (Jan 5, 2025):

It's the holiday season, it's a newly discovered issue, it's easy to workaround, there are higher priority issues, other unknown reasons.

@rick-github commented on GitHub (Jan 5, 2025): It's the holiday season, it's a newly discovered issue, it's easy to workaround, there are higher priority issues, other unknown reasons.

GiteaMirror commented

2026-05-04 10:05:20 -05:00

@tripolskypetr commented on GitHub (Jan 5, 2025):

@mxyng @rick-github

Please pay attention to the fact that the ollama repository is being spammed by models with broken system prompts which do not work as expected. This is a red flag not to use Ollama.

@tripolskypetr commented on GitHub (Jan 5, 2025): @mxyng @rick-github Please pay attention to the fact that the ollama repository is being spammed by models with broken system prompts which do not work as expected. This is a red flag not to use Ollama.

GiteaMirror commented

2026-05-04 10:05:21 -05:00

@rick-github commented on GitHub (Jan 6, 2025):

The default nemotron-mini template doesn't work well with your prompt. It works fine for others as evidenced by the original issue (#6870). We can get improved results for your use case without modifying the template by modifying the prompt:

#!/usr/bin/env python3

import ollama
import sys
import argparse
import json

parser = argparse.ArgumentParser()
parser.add_argument("--prompt", default="Navigate me to sales agent")
parser.add_argument("model", nargs="?", default="nemotron-mini")
args = parser.parse_args()

def navigate_user_to_agent(agent: str) -> bool:
  """
  Navigates the user to an agent.  Agent types are `sales` or `refund`.

  Args:
    Agent type.

  Returns:
    `True` if the transfer succeeded, `False` otherwise.
  """
  print(f"Navigate to {agent} agent", file=sys.stderr)
  return True

tools = [ navigate_user_to_agent ]

response = ollama.chat(
      args.model,
      messages=[{"role":"user","content":args.prompt}],
      options={"top_k":20,"top_p":0.4,"temperature":0.5},
      tools=tools,
    )
print(json.dumps(dict(response.message), default=str))

$ for i in {1..100} ; do ./8287-2.py ; done | sort | uniq -c
    100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}

@rick-github commented on GitHub (Jan 6, 2025): The default nemotron-mini template doesn't work well with your prompt. It works fine for others as evidenced by the original issue (#6870). We can get improved results for your use case without modifying the template by modifying the prompt: ```python #!/usr/bin/env python3 import ollama import sys import argparse import json parser = argparse.ArgumentParser() parser.add_argument("--prompt", default="Navigate me to sales agent") parser.add_argument("model", nargs="?", default="nemotron-mini") args = parser.parse_args() def navigate_user_to_agent(agent: str) -> bool: """ Navigates the user to an agent. Agent types are `sales` or `refund`. Args: Agent type. Returns: `True` if the transfer succeeded, `False` otherwise. """ print(f"Navigate to {agent} agent", file=sys.stderr) return True tools = [ navigate_user_to_agent ] response = ollama.chat( args.model, messages=[{"role":"user","content":args.prompt}], options={"top_k":20,"top_p":0.4,"temperature":0.5}, tools=tools, ) print(json.dumps(dict(response.message), default=str)) ``` ```console $ for i in {1..100} ; do ./8287-2.py ; done | sort | uniq -c 100 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} ```

GiteaMirror commented

2026-05-04 10:05:22 -05:00

@ParthSareen commented on GitHub (Jan 8, 2025):

Thanks for pointing the template issue out @tripolskypetr - will check that out. Highly recommend following the comments laid out by @rick-github (thank you!) as they cover what all you can do right now for this to be a non-issue.

I'm aware of the constraint by the JSON generation for tools and it being an issue - it's something that I plan to address in the near future. I'd also highly recommend playing with structured outputs if you need an output generated. Most of your issues can just be fixed with prompting for now as Rick has mentioned.

There's a lot of structured outputs + tool calls and constrained sampling work to be done which should be improving this entire experience.

As for the "fake tool" tags could you point some out? If these are user defined there's not much for us to do as that is the onus of the user. However, for the ones that we publish we aim to have the template correct with the right tags. If you see any of those models feel free to link them here. Do realize that there are permutations of model configurations which do not support all features in the tags and that's fine for now as that's more so just UI/UX than a model not returning tools.

@ParthSareen commented on GitHub (Jan 8, 2025): Thanks for pointing the template issue out @tripolskypetr - will check that out. Highly recommend following the comments laid out by @rick-github (thank you!) as they cover what all you can do right now for this to be a non-issue. I'm aware of the constraint by the JSON generation for tools and it being an issue - it's something that I plan to address in the near future. I'd also highly recommend playing with structured outputs if you _need_ an output generated. Most of your issues can just be fixed with prompting for now as Rick has mentioned. There's a lot of structured outputs + tool calls and constrained sampling work to be done which should be improving this entire experience. As for the "fake tool" tags could you point some out? If these are user defined there's not much for us to do as that is the onus of the user. However, for the ones that we publish we aim to have the template correct with the right tags. If you see any of those models feel free to link them here. Do realize that there are permutations of model configurations which do not support all features in the tags and that's fine for now as that's more so just UI/UX than a model not returning tools.

GiteaMirror commented

2026-05-04 10:05:23 -05:00

@tripolskypetr commented on GitHub (Jan 9, 2025):

@ParthSareen

The same. Again. thanks for a lot of chatgpt spam, @rick-github

The qwen2 model does not support tools

on the official website it still has the tools label

The ollama parseToolCalls use the <tool_call></tool_call> tag

The nemotron-mini is using the <toolcall></toolcall> tag without underscore so tools not being called

@tripolskypetr commented on GitHub (Jan 9, 2025): @ParthSareen The same. Again. thanks for a lot of chatgpt spam, @rick-github 1. The qwen2 model does not support tools ![image](https://github.com/user-attachments/assets/71db5f3f-4c3d-4427-93e0-40c67a34ae9c) on the official website it still has the tools label ![image](https://github.com/user-attachments/assets/92b36a5e-04e7-4295-bb32-95a0826623e6) 2. The ollama [parseToolCalls](https://github.com/ollama/ollama/blob/86a622cbdc69e9fd501764ff7565e977fc98f00a/server/model.go#L158) use the `<tool_call></tool_call>` tag ![image](https://github.com/user-attachments/assets/c012b096-8a80-467b-beb1-12f6ce2fd84b) The nemotron-mini is using the `<toolcall></toolcall>` tag without underscore so tools not being called

GiteaMirror commented

2026-05-04 10:05:25 -05:00

@rick-github commented on GitHub (Jan 9, 2025):

All I can do is it explain it, it's up to you to understand it.

@rick-github commented on GitHub (Jan 9, 2025): All I can do is it explain it, it's up to you to understand it.

GiteaMirror commented

2026-05-04 10:05:26 -05:00

@tripolskypetr commented on GitHub (Jan 9, 2025):

@rick-github I understand It, thanks. Now, I need to know when the model list will be purged from the advertising fake.

@tripolskypetr commented on GitHub (Jan 9, 2025): @rick-github I understand It, thanks. Now, I need to know when the model list will be purged from the advertising fake. ![image](https://github.com/user-attachments/assets/dbe22af7-96b2-4ea1-90ff-c8785e88209d)

GiteaMirror commented

2026-05-04 10:05:29 -05:00

@ParthSareen commented on GitHub (Jan 9, 2025):

@tripolskypetr As both Rick and I mentioned previously, the larger qwen models support tools. It's not the most straightforward thing on the UI I get that, SUPER hard to understand it seems so. Will fix that. Thanks!

@ParthSareen commented on GitHub (Jan 9, 2025): @tripolskypetr As both Rick and I mentioned previously, the larger qwen models support tools. It's not the most straightforward thing on the UI I get that, SUPER hard to understand it seems so. Will fix that. Thanks!

GiteaMirror commented

2026-05-04 10:05:31 -05:00

@tripolskypetr commented on GitHub (Jan 9, 2025):

@ParthSareen @rick-github @mxyng I got that issue on a first look. After I seen the invalid system prompt in nemotron-mini I got a serious suspicion about the whole model list is untested

Please! Could you better test the tools execution cause without them the agent swarm pattern cannot be implemented. This affects third party tools like LangChain.js too

The agent swarm is the only way to make prompt engineering well structed. Without agent swarms, the ai code is nasty. So, by the first look this is low priority. But it definitely not!

@tripolskypetr commented on GitHub (Jan 9, 2025): @ParthSareen @rick-github @mxyng I got that issue on a first look. After I seen the invalid system prompt in `nemotron-mini` I got a serious suspicion about the whole model list is untested Please! Could you better test the tools execution cause without them [the agent swarm pattern](https://github.com/openai/swarm) cannot be implemented. This affects third party tools like [LangChain.js](https://js.langchain.com/docs/introduction/) too The agent swarm is the only way to make prompt engineering well structed. Without agent swarms, the ai code is nasty. So, by the first look this is low priority. But it definitely not!

GiteaMirror commented

2026-05-04 10:05:33 -05:00

@rick-github commented on GitHub (Jan 10, 2025):

This is the list of model families in the main ollama library that support tools:

llama3.3
qwq
llama3.2
llama3.1
mistral
qwen2
qwen2.5
qwen2.5-coder
mistral-nemo
mixtral:8x228
command-r
command-r-plus
mistral-large
smollm2
hermes3
athene-v2
mistral-small
nemotron-mini
llama3-groq-tool-use
nemotron
granite3-dense
granite3-moe
aya-expanse
firefunction-v2
granite3.1-dense
granite3.1-moe

For the first pass, I'm using 8287-2.py from above. This script implements the "navigate to agent" function from your initial post but is structured ("engineered") to provide better results for nemotron-mini. In tests above, I did 100 tests on a model but that would take too long with so many models, so it's a one-shot test for the initial results.

$ for i in $(cat tool.families) ; do ollama pull $i ; done
$ for i in $(cat tool.families) ; do printf "%-30s %s\n" $i "$(./8287-2.py $i)" ; done

llama3.3                       {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
qwq                            {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})", "function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
llama3.2                       {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
llama3.1                       {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
mistral                        {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
qwen2                          {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
qwen2.5                        {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
qwen2.5-coder                  {"role": "assistant", "content": "```xml\n<navigate_user_to_agent>\n    {\"agent\": \"sales\"}\n</navigate_user_to_agent>\n```", "images": null, "tool_calls": null}
mistral-nemo                   {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
mixtral:8x22b                  {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
command-r                      {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
command-r-plus                 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
mistral-large                  {"role": "assistant", "content": "\nI'll navigate you to a sales agent. Please hold on while I connect you.", "images": null, "tool_calls": null}
smollm2                        {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
hermes3                        {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
athene-v2                      {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
mistral-small                  {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
nemotron-mini                  {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
llama3-groq-tool-use           {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
nemotron                       {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
granite3-dense                 {"role": "assistant", "content": "I'm sorry for the inconvenience, but I'm unable to perform that action directly. However, I can guide you on how to do it. To navigate to a sales agent, you would need to use the `navigate_user_to_agent` function with the parameter `agent` set to `\"sales\"`. Here's an example:\n\n```json\n{\n  \"type\": \"function\",\n  \"function\": {\n    \"name\": \"navigate_user_to_agent\",\n    \"description\": \"Navigates the user to an agent. Agent types are 'sales' or 'refund'.\",\n    \"parameters\": {\n      \"type\": \"object\",\n      \"required\": [\"agent\"],\n      \"properties\": {\n        \"agent\": {\n          \"type\": \"string\",\n          \"description\": \"\"\n        }\n      }\n    }\n  }\n}\n```\n\nJust replace `\"sales\"` with the desired agent type and call this function.", "images": null, "tool_calls": null}
granite3-moe                   {"role": "assistant", "content": "Sure! Here's how you can navigate a user to the \"sales\" agent:\n\n```json\n{\n  \"type\": \"function\",\n  \"function\": {\n    \"name\": \"navigate_user_to_agent\",\n    \"description\": \"Navigates the user to an agent.  Agent types are `sales` or `refund`.\",\n    \"parameters\": {\n      \"type\": {\n        \"type\": \"string\",\n        \"required\": true,\n        \"values\": [\"sales\", \"refund\"],\n        \"description\": \"The type of agent to navigate to.\"\n      },\n      \"agent\": {\n        \"type\": \"object\",\n        \"required\": true,\n        \"properties\": {\n          \"name\": {\n            \"type\": \"string\",\n            \"description\": \"\"\n          }\n        }\n      }\n    }\n  }\n}\n```\n\nIn this example, the function `navigate_user_to_agent` takes two parameters: `type` and `agent`. The `type` parameter is an object that specifies the type of agent to navigate to. In this case, it's set to \"sales\" or \"refund\", depending on whether you want to navigate to a sales agent or a refund agent.\n\nThe `agent` parameter is an object that contains information about the specific agent you want to navigate to. In this example, it has a single property called \"name\", which contains the name of the agent. You can replace this with the actual name of the agent you want to navigate to.\n\nSo, if you wanted to navigate a user to a sales agent, you would call the function like this:\n\n```json\n{\n  \"type\": {\n    \"type\": \"object\",\n    \"required\": true,\n    \"values\": [\"sales\"],\n    \"description\": \"\"\n  },\n  \"agent\": {\n    \"name\": \"Sales Agent Name\"\n  }\n}\n```", "images": null, "tool_calls": null}
aya-expanse                    {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
firefunction-v2                {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
granite3.1-dense               {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}
granite3.1-moe                 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}

qwq returns two tool calls. They're both the same, but it's arguably incorrect. However, the tool function capability works.

qwen2.5-coder is an obvious mistake. The HF page makes no mention of function calling and it looks like the template is just the one from qwen2.5 with the addition of FIM support.

mistral-large touts "native function calling" so it's expected to do better. The script is modified to allow a system message to be included:

--- 8287-2.py	2025-01-10 00:02:08.276770019 +0100
+++ 8287-3.py	2025-01-10 02:25:25.431698305 +0100
@@ -7,6 +7,7 @@
 
 parser = argparse.ArgumentParser()
 parser.add_argument("--prompt", default="Navigate me to sales agent")
+parser.add_argument("--system", default="Use the available tools to respond to the user question.")
 parser.add_argument("model", nargs="?", default="nemotron-mini")
 args = parser.parse_args()
 
@@ -27,7 +28,10 @@
 
 response = ollama.chat(
       args.model,
-      messages=[{"role":"user","content":args.prompt}],
+      messages=[
+        {"role":"system","content":args.system},
+        {"role":"user","content":args.prompt}
+      ],
       options={"top_k":20,"top_p":0.4,"temperature":0.5},
       tools=tools,
     )

$ ./8287-3.py mistral-large --system 'Use the available tools to respond to the user question.'
{"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}

granite3-dense and granite3-moe know what to do, but return a format that is not detected as a tool call. granite3-dense can return the right response with a tuned prompt:

$ ./8287-3.py granite3-dense --system "You are a helpful AI assistant with access to the following tools. When a tool is required to answer the user's query, respond with a JSON list of tools used in the form {"name": <tool-name>, "arguments": <tool-arguments>}."
{"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]}

It's harder for granite-moe to comply because it's such a small model at 1b parameters. If you use the 3b version, it works better:

$ ./8287-3.py granite3-moe:3b --system "You are a helpful AI assistant with access to the following tools. When a tool is required to answer the user's query, respond with a JSON list of tools used in the form {"name": <tool-name>, "arguments": <tool-arguments>}."
{"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'type': 'sales'})"]}

Both granite3-moe and granite3-dense have been superseded by the 3.1 models, so it's probably not worth spending any further effort on them.

So the summary is that with the exception of qwen2.5-coder, all families with the tools tag have models that support tool calls. Some of the tools require tuned prompts to get good results. The qwen2.5-coder template should be updated to remove tool capability. granite3-moe and granite3-dense should be archived.

The mistral-large template could be enhanced for tool use, but since the default prompt doesn't include tool use instruction, and the tool use example at HF explicitly calls out the requirement for a system prompt, that can be left up to the user.

@rick-github commented on GitHub (Jan 10, 2025): This is the list of model families in the main ollama library that support tools: ``` llama3.3 qwq llama3.2 llama3.1 mistral qwen2 qwen2.5 qwen2.5-coder mistral-nemo mixtral:8x228 command-r command-r-plus mistral-large smollm2 hermes3 athene-v2 mistral-small nemotron-mini llama3-groq-tool-use nemotron granite3-dense granite3-moe aya-expanse firefunction-v2 granite3.1-dense granite3.1-moe ``` For the first pass, I'm using 8287-2.py from above. This script implements the "navigate to agent" function from your initial post but is structured ("engineered") to provide better results for nemotron-mini. In tests above, I did 100 tests on a model but that would take too long with so many models, so it's a one-shot test for the initial results. ```console $ for i in $(cat tool.families) ; do ollama pull $i ; done $ for i in $(cat tool.families) ; do printf "%-30s %s\n" $i "$(./8287-2.py $i)" ; done ``` ```console llama3.3 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} qwq {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})", "function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} llama3.2 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} llama3.1 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} mistral {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} qwen2 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} qwen2.5 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} qwen2.5-coder {"role": "assistant", "content": "```xml\n<navigate_user_to_agent>\n {\"agent\": \"sales\"}\n</navigate_user_to_agent>\n```", "images": null, "tool_calls": null} mistral-nemo {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} mixtral:8x22b {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} command-r {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} command-r-plus {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} mistral-large {"role": "assistant", "content": "\nI'll navigate you to a sales agent. Please hold on while I connect you.", "images": null, "tool_calls": null} smollm2 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} hermes3 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} athene-v2 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} mistral-small {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} nemotron-mini {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} llama3-groq-tool-use {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} nemotron {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} granite3-dense {"role": "assistant", "content": "I'm sorry for the inconvenience, but I'm unable to perform that action directly. However, I can guide you on how to do it. To navigate to a sales agent, you would need to use the `navigate_user_to_agent` function with the parameter `agent` set to `\"sales\"`. Here's an example:\n\n```json\n{\n \"type\": \"function\",\n \"function\": {\n \"name\": \"navigate_user_to_agent\",\n \"description\": \"Navigates the user to an agent. Agent types are 'sales' or 'refund'.\",\n \"parameters\": {\n \"type\": \"object\",\n \"required\": [\"agent\"],\n \"properties\": {\n \"agent\": {\n \"type\": \"string\",\n \"description\": \"\"\n }\n }\n }\n }\n}\n```\n\nJust replace `\"sales\"` with the desired agent type and call this function.", "images": null, "tool_calls": null} granite3-moe {"role": "assistant", "content": "Sure! Here's how you can navigate a user to the \"sales\" agent:\n\n```json\n{\n \"type\": \"function\",\n \"function\": {\n \"name\": \"navigate_user_to_agent\",\n \"description\": \"Navigates the user to an agent. Agent types are `sales` or `refund`.\",\n \"parameters\": {\n \"type\": {\n \"type\": \"string\",\n \"required\": true,\n \"values\": [\"sales\", \"refund\"],\n \"description\": \"The type of agent to navigate to.\"\n },\n \"agent\": {\n \"type\": \"object\",\n \"required\": true,\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"\"\n }\n }\n }\n }\n }\n}\n```\n\nIn this example, the function `navigate_user_to_agent` takes two parameters: `type` and `agent`. The `type` parameter is an object that specifies the type of agent to navigate to. In this case, it's set to \"sales\" or \"refund\", depending on whether you want to navigate to a sales agent or a refund agent.\n\nThe `agent` parameter is an object that contains information about the specific agent you want to navigate to. In this example, it has a single property called \"name\", which contains the name of the agent. You can replace this with the actual name of the agent you want to navigate to.\n\nSo, if you wanted to navigate a user to a sales agent, you would call the function like this:\n\n```json\n{\n \"type\": {\n \"type\": \"object\",\n \"required\": true,\n \"values\": [\"sales\"],\n \"description\": \"\"\n },\n \"agent\": {\n \"name\": \"Sales Agent Name\"\n }\n}\n```", "images": null, "tool_calls": null} aya-expanse {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} firefunction-v2 {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} granite3.1-dense {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} granite3.1-moe {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} ``` `qwq` returns two tool calls. They're both the same, but it's arguably incorrect. However, the tool function capability works. `qwen2.5-coder` is an obvious mistake. The HF page makes no mention of function calling and it looks like the template is just the one from `qwen2.5` with the addition of FIM support. `mistral-large` touts "native function calling" so it's expected to do better. The script is modified to allow a system message to be included: ```diff --- 8287-2.py 2025-01-10 00:02:08.276770019 +0100 +++ 8287-3.py 2025-01-10 02:25:25.431698305 +0100 @@ -7,6 +7,7 @@ parser = argparse.ArgumentParser() parser.add_argument("--prompt", default="Navigate me to sales agent") +parser.add_argument("--system", default="Use the available tools to respond to the user question.") parser.add_argument("model", nargs="?", default="nemotron-mini") args = parser.parse_args() @@ -27,7 +28,10 @@ response = ollama.chat( args.model, - messages=[{"role":"user","content":args.prompt}], + messages=[ + {"role":"system","content":args.system}, + {"role":"user","content":args.prompt} + ], options={"top_k":20,"top_p":0.4,"temperature":0.5}, tools=tools, ) ``` ```console $ ./8287-3.py mistral-large --system 'Use the available tools to respond to the user question.' {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} ``` `granite3-dense` and `granite3-moe` know what to do, but return a format that is not detected as a tool call. `granite3-dense` can return the right response with a tuned prompt: ```console $ ./8287-3.py granite3-dense --system "You are a helpful AI assistant with access to the following tools. When a tool is required to answer the user's query, respond with a JSON list of tools used in the form {"name": <tool-name>, "arguments": <tool-arguments>}." {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'agent': 'sales'})"]} ``` It's harder for `granite-moe` to comply because it's such a small model at 1b parameters. If you use the 3b version, it works better: ```console $ ./8287-3.py granite3-moe:3b --system "You are a helpful AI assistant with access to the following tools. When a tool is required to answer the user's query, respond with a JSON list of tools used in the form {"name": <tool-name>, "arguments": <tool-arguments>}." {"role": "assistant", "content": "", "images": null, "tool_calls": ["function=Function(name='navigate_user_to_agent', arguments={'type': 'sales'})"]} ``` Both `granite3-moe` and `granite3-dense` have been superseded by the 3.1 models, so it's probably not worth spending any further effort on them. So the summary is that with the exception of `qwen2.5-coder`, all families with the `tools` tag have models that support tool calls. Some of the tools require tuned prompts to get good results. The `qwen2.5-coder` template should be updated to remove tool capability. `granite3-moe` and `granite3-dense` should be archived. The `mistral-large` template could be enhanced for tool use, but since the [default prompt](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/blob/3a5cb136f6106edf5c1210369068eb5a4f787cab/tokenizer_config.json#L6177) doesn't include tool use instruction, and the [tool use example](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411#improved-function-calling) at HF explicitly calls out the requirement for a system prompt, that can be left up to the user.

GiteaMirror commented

2026-05-04 10:05:36 -05:00

@tripolskypetr commented on GitHub (Jan 25, 2025):

In addition to the previous conversation I managed to create the algorithm which will rescue the model when it calling not existing tools, calling tools with invalid arguments, return the invalid text output which suppose to be the tool call

_resurrectModel = async (reason?: string) => {
    this.params.logger.debug(
      `ClientAgent agentName=${this.params.agentName} _resurrectModel`
    );
    {
      await this.params.history.push({
        role: "resque",
        agentName: this.params.agentName,
        content: reason || "Unknown error",
      });
      await this.params.history.push({
        role: "user",
        agentName: this.params.agentName,
        content: GLOBAL_CONFIG.CC_TOOL_CALL_EXCEPTION_PROMPT,
      });
    }
    const message = await this.getCompletion();
    const result = message.content;
    let validation: string | null = null;
    if (validation = await this.params.validate(result)) {
      this.params.logger.debug(
        `ClientAgent agentName=${this.params.agentName} _resurrectModel validation error: ${validation}`
      );
      const content = getPlaceholder();
      await this.params.history.push({
        agentName: this.params.agentName,
        role: "assistant",
        content,
      })
      return content;
    }

    ...

I also published it as a separate NPM module to simplify the agent swarm development with Ollama. Now Ollama can be easely connected to the Hono WebSocket server

import {
  addAgent,
  addCompletion,
  addSwarm,
  addTool,
  changeAgent,
  execute,
  session,
} from "agent-swarm-kit";

const NAVIGATE_TOOL = addTool({
  toolName: "navigate-tool",
  call: async (clientId, agentName, { to }) => {
    await changeAgent(to, clientId);
    await execute("Navigation complete. Notify the user", clientId);
  },
  validate: async () => true,
  type: "function",
  function: {
    name: "navigate-tool",
    description: "The tool for navigation",
    parameters: {
      type: "object",
      properties: {
        to: {
          type: "string",
          description: "The target agent for navigation",
        },
      },
      required: ["to"],
    },
  },
});

const ollama = new Ollama({ host: CC_OLLAMA_HOST });

const MOCK_COMPLETION = addCompletion({
  completionName: "navigate-completion",
  /**
   * Use whatever you want: NVIDIA NIM, OpenAI, GPT4All, Ollama or LM Studio
   * Even mock it for unit test of tool integration like it done in `test` folder
   * 
   * @see https://github.com/tripolskypetr/agent-swarm-kit/tree/master/test
   */
  getCompletion: async ({ messages, tools }) => {
    return ollama.chat({
      model: CC_OLLAMA_CHAT_MODEL,
      keep_alive: "1h",
      messages,
      tools,
    })
  },
});

const TRIAGE_AGENT = addAgent({
  agentName: "triage-agent",
  completion: MOCK_COMPLETION,
  prompt: "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents available: `sales-agent` and `refund-agent`",
  tools: [NAVIGATE_TOOL],
});

const SALES_AGENT = addAgent({
  agentName: "sales-agent",
  completion: MOCK_COMPLETION,
  prompt: "You are a sales agent that handles all actions related to placing the order to purchase an item.",
  tools: [NAVIGATE_TOOL],
});

const REDUND_AGENT = addAgent({
  agentName: "refund-agent",
  completion: MOCK_COMPLETION,
  prompt: "You are a refund agent that handles all actions related to refunds after a return has been processed.",
  tools: [NAVIGATE_TOOL],
});

const TEST_SWARM = addSwarm({
  agentList: [TRIAGE_AGENT, SALES_AGENT, REDUND_AGENT],
  defaultAgent: TRIAGE_AGENT,
  swarmName: "navigation-swarm",
});

...


app.get("/api/v1/session/:clientId", upgradeWebSocket((ctx) => {
  const clientId = ctx.req.param("clientId");

  const { complete, dispose } = session(clientId, TEST_SWARM)

  return {
    onMessage(event) {
      const message = event.data.toString();
      incomingSubject.next(await complete(message));
    },
    onClose: () => {
      await dispose();
    },
  }
}));

Link to the whole source code

@tripolskypetr commented on GitHub (Jan 25, 2025): In addition to the previous conversation I managed to create the algorithm which will rescue the model when it calling not existing tools, calling tools with invalid arguments, return the invalid text output which suppose to be the tool call ```typescript _resurrectModel = async (reason?: string) => { this.params.logger.debug( `ClientAgent agentName=${this.params.agentName} _resurrectModel` ); { await this.params.history.push({ role: "resque", agentName: this.params.agentName, content: reason || "Unknown error", }); await this.params.history.push({ role: "user", agentName: this.params.agentName, content: GLOBAL_CONFIG.CC_TOOL_CALL_EXCEPTION_PROMPT, }); } const message = await this.getCompletion(); const result = message.content; let validation: string | null = null; if (validation = await this.params.validate(result)) { this.params.logger.debug( `ClientAgent agentName=${this.params.agentName} _resurrectModel validation error: ${validation}` ); const content = getPlaceholder(); await this.params.history.push({ agentName: this.params.agentName, role: "assistant", content, }) return content; } ... ``` I also published it as a separate NPM module to simplify the agent swarm development with Ollama. Now Ollama can be easely connected to the [Hono WebSocket server](https://hono.dev/docs/helpers/websocket) ```typescript import { addAgent, addCompletion, addSwarm, addTool, changeAgent, execute, session, } from "agent-swarm-kit"; const NAVIGATE_TOOL = addTool({ toolName: "navigate-tool", call: async (clientId, agentName, { to }) => { await changeAgent(to, clientId); await execute("Navigation complete. Notify the user", clientId); }, validate: async () => true, type: "function", function: { name: "navigate-tool", description: "The tool for navigation", parameters: { type: "object", properties: { to: { type: "string", description: "The target agent for navigation", }, }, required: ["to"], }, }, }); const ollama = new Ollama({ host: CC_OLLAMA_HOST }); const MOCK_COMPLETION = addCompletion({ completionName: "navigate-completion", /** * Use whatever you want: NVIDIA NIM, OpenAI, GPT4All, Ollama or LM Studio * Even mock it for unit test of tool integration like it done in `test` folder * * @see https://github.com/tripolskypetr/agent-swarm-kit/tree/master/test */ getCompletion: async ({ messages, tools }) => { return ollama.chat({ model: CC_OLLAMA_CHAT_MODEL, keep_alive: "1h", messages, tools, }) }, }); const TRIAGE_AGENT = addAgent({ agentName: "triage-agent", completion: MOCK_COMPLETION, prompt: "You are to triage a users request, and call a tool to transfer to the right agent. There are two agents available: `sales-agent` and `refund-agent`", tools: [NAVIGATE_TOOL], }); const SALES_AGENT = addAgent({ agentName: "sales-agent", completion: MOCK_COMPLETION, prompt: "You are a sales agent that handles all actions related to placing the order to purchase an item.", tools: [NAVIGATE_TOOL], }); const REDUND_AGENT = addAgent({ agentName: "refund-agent", completion: MOCK_COMPLETION, prompt: "You are a refund agent that handles all actions related to refunds after a return has been processed.", tools: [NAVIGATE_TOOL], }); const TEST_SWARM = addSwarm({ agentList: [TRIAGE_AGENT, SALES_AGENT, REDUND_AGENT], defaultAgent: TRIAGE_AGENT, swarmName: "navigation-swarm", }); ... app.get("/api/v1/session/:clientId", upgradeWebSocket((ctx) => { const clientId = ctx.req.param("clientId"); const { complete, dispose } = session(clientId, TEST_SWARM) return { onMessage(event) { const message = event.data.toString(); incomingSubject.next(await complete(message)); }, onClose: () => { await dispose(); }, } })); ``` [Link to the whole source code](https://github.com/tripolskypetr/agent-swarm-kit/blob/master/src/client/ClientAgent.ts)

GiteaMirror commented

2026-05-04 10:05:37 -05:00

@rick-github commented on GitHub (Jan 25, 2025):

I'm glad that you agree that models are not 100% reliable and that model choice and good prompts are required for good results.

@rick-github commented on GitHub (Jan 25, 2025): I'm glad that you agree that models are not 100% reliable and that model choice and good prompts are required for good results.

Sign in to join this conversation.

Branches Tags

main

dhiltgen/ci

parth-launch-plan-gating

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#67359