github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6980] Tools support is not working right #4419

New Issue
Closed
opened 2026-04-12 15:21:26 -05:00 by GiteaMirror · 14 comments
GiteaMirror commented 2026-04-12 15:21:26 -05:00
Owner
Copy Link

Originally created by @acastry on GitHub (Sep 26, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6980

What is the issue?

Hi,
Tools support doens't work as expected i Guess. When activated it gets the right function to be called, but at the same time it doesn't return anymore normal response for any other phrases than tool support example.

I am copying last documentation example for tools support.

  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "What is the weather today in Paris?"
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            },
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'

Returns
{"model":"llama3.2","created_at":"2024-09-26T13:56:50.934137Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"celsius","location":"Paris"}}}]},"done_reason":"stop","done":true,"total_duration":1596672250,"load_duration":40646209,"prompt_eval_count":213,"prompt_eval_duration":1052689000,"eval_count":25,"eval_duration":500100000}

Let me just change the question:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "What is the value of pi ?"
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            },
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'

I cannot have normal response and it hallucinates :

{"model":"llama3.2","created_at":"2024-09-26T14:12:22.749298Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"pi","location":""}}}]},"done_reason":"stop","done":true,"total_duration":5842347542,"load_duration":4899628417,"prompt_eval_count":212,"prompt_eval_duration":480869000,"eval_count":23,"eval_duration":456247000}

What if i deactivate tools ?

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [
    {
      "role": "user",
      "content": "What is the value of pi ?"
    }
  ],
  "stream": false,
  "tools": [
  ]
}'

It works :
{"model":"llama3.2","created_at":"2024-09-26T14:16:03.899228Z","message":{"role":"assistant","content":"Pi (π) is an irrational number, which means it cannot be expressed as a finite decimal or fraction. The value of pi is approximately equal to 3.14159, but it goes on forever without repeating in a predictable pattern.\n\nIn fact, the first few digits of pi are:\n\n3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679\n\n...and so on!\n\nPi is a fundamental constant in mathematics and appears in many areas of geometry, trigonometry, calculus, and more. Its value is used to describe the circumference and area of circles, as well as other curved shapes.\n\nIf you need a more precise value, there are online calculators and mathematical libraries that can provide an infinite number of digits of pi!"},"done_reason":"stop","done":true,"total_duration":5318857625,"load_duration":41988708,"prompt_eval_count":32,"prompt_eval_duration":1744657000,"eval_count":173,"eval_duration":3530631000}

### Also tested with LLAMA3.1 and not working

curl http://localhost:11434/api/chat -d '{
  "model": "llama3.1",
  "messages": [
    {
      "role": "user",
      "content": "What is the value of pi ?"
    }
  ],
  "stream": false,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            },
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location", "format"]
        }
      }
    }
  ]
}'

{"model":"llama3.1","created_at":"2024-09-26T16:50:52.16426Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"celsius","location":"San Francisco, CA"}}}]},"done_reason":"stop","done":true,"total_duration":7556086500,"load_duration":40462833,"prompt_eval_count":214,"prompt_eval_duration":5782271000,"eval_count":48,"eval_duration":1731411000}

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.3.12

Originally created by @acastry on GitHub (Sep 26, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6980 ### What is the issue? Hi, Tools support doens't work as expected i Guess. When activated it gets the right function to be called, but at the same time it doesn't return anymore normal response for any other phrases than tool support example. ### **I am copying last documentation example for tools support.** ```curl http://localhost:11434/api/chat -d '{ "model": "llama3.2", "messages": [ { "role": "user", "content": "What is the weather today in Paris?" } ], "stream": false, "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the weather for, e.g. San Francisco, CA" }, "format": { "type": "string", "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'", "enum": ["celsius", "fahrenheit"] } }, "required": ["location", "format"] } } } ] }' ``` Returns ```{"model":"llama3.2","created_at":"2024-09-26T13:56:50.934137Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"celsius","location":"Paris"}}}]},"done_reason":"stop","done":true,"total_duration":1596672250,"load_duration":40646209,"prompt_eval_count":213,"prompt_eval_duration":1052689000,"eval_count":25,"eval_duration":500100000}``` **Let me just change the question:** ``` curl http://localhost:11434/api/chat -d '{ "model": "llama3.2", "messages": [ { "role": "user", "content": "What is the value of pi ?" } ], "stream": false, "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the weather for, e.g. San Francisco, CA" }, "format": { "type": "string", "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'", "enum": ["celsius", "fahrenheit"] } }, "required": ["location", "format"] } } } ] }' ``` ### **I cannot have normal response and it hallucinates :** `{"model":"llama3.2","created_at":"2024-09-26T14:12:22.749298Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"pi","location":""}}}]},"done_reason":"stop","done":true,"total_duration":5842347542,"load_duration":4899628417,"prompt_eval_count":212,"prompt_eval_duration":480869000,"eval_count":23,"eval_duration":456247000}` **What if i deactivate tools ?** ``` curl http://localhost:11434/api/chat -d '{ "model": "llama3.2", "messages": [ { "role": "user", "content": "What is the value of pi ?" } ], "stream": false, "tools": [ ] }' ``` It works : `{"model":"llama3.2","created_at":"2024-09-26T14:16:03.899228Z","message":{"role":"assistant","content":"Pi (π) is an irrational number, which means it cannot be expressed as a finite decimal or fraction. The value of pi is approximately equal to 3.14159, but it goes on forever without repeating in a predictable pattern.\n\nIn fact, the first few digits of pi are:\n\n3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679\n\n...and so on!\n\nPi is a fundamental constant in mathematics and appears in many areas of geometry, trigonometry, calculus, and more. Its value is used to describe the circumference and area of circles, as well as other curved shapes.\n\nIf you need a more precise value, there are online calculators and mathematical libraries that can provide an infinite number of digits of pi!"},"done_reason":"stop","done":true,"total_duration":5318857625,"load_duration":41988708,"prompt_eval_count":32,"prompt_eval_duration":1744657000,"eval_count":173,"eval_duration":3530631000}` **### Also tested with LLAMA3.1 and not working** ``` curl http://localhost:11434/api/chat -d '{ "model": "llama3.1", "messages": [ { "role": "user", "content": "What is the value of pi ?" } ], "stream": false, "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather for a location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The location to get the weather for, e.g. San Francisco, CA" }, "format": { "type": "string", "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'", "enum": ["celsius", "fahrenheit"] } }, "required": ["location", "format"] } } } ] }' ``` `{"model":"llama3.1","created_at":"2024-09-26T16:50:52.16426Z","message":{"role":"assistant","content":"","tool_calls":[{"function":{"name":"get_current_weather","arguments":{"format":"celsius","location":"San Francisco, CA"}}}]},"done_reason":"stop","done":true,"total_duration":7556086500,"load_duration":40462833,"prompt_eval_count":214,"prompt_eval_duration":5782271000,"eval_count":48,"eval_duration":1731411000} ` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.3.12
GiteaMirror added the bug label 2026-04-12 15:21:26 -05:00
GiteaMirror commented 2026-04-12 15:21:27 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 26, 2024):

https://github.com/ollama/ollama/issues/6127

<!-- gh-comment-id:2378118369 --> @rick-github commented on GitHub (Sep 26, 2024): https://github.com/ollama/ollama/issues/6127
GiteaMirror commented 2026-04-12 15:21:27 -05:00
Author
Owner
Copy Link

@pannous commented on GitHub (Sep 27, 2024):

I'm glad this ticket is still open because https://github.com/ollama/ollama/issues/6127 is closed and thus harder to find.

<!-- gh-comment-id:2379765890 --> @pannous commented on GitHub (Sep 27, 2024): I'm glad this ticket is still open because https://github.com/ollama/ollama/issues/6127 is closed and thus harder to find.
GiteaMirror commented 2026-04-12 15:21:28 -05:00
Author
Owner
Copy Link

@hassanallaham commented on GitHub (Nov 8, 2024):

I think the real problem is that Ollama trying to differentiate the models which support tool calling from other models. For example, if you created a model like llama3.2 from a gguf file downloaded from Hugging Face and you used a modelfile with a template that is not like what is in the template on the Ollama website the result will be: if you added any tool to the client it will tell you that the model does not support tool call. Ollama searches for something in the template in modelfile to classify it as a model which supports tool calling. This relation with the fact that the default template used by tool-calling models downloaded from the Ollama website forces the model to (always) call a function unless it receives the result of a function call (i.e. this template forces the model to always use tool-calling unless it received a message where the role is "tool". Either they need to update the template used to make the model decide when to use and when not to use tool calling (this may open a door for hallucination), Or (which is the best solution) they should remove this mechanism of classifying the model as a tools model based on that template in modelfile content. I believe that differentiation or classification of the models using the template in modelfile is a very bad idea since tool calling is a very important part of agentic workflows and since there are many models which can be used for this target even the system prompt is not written the way do Ollama needs to allow tool calling

<!-- gh-comment-id:2463472576 --> @hassanallaham commented on GitHub (Nov 8, 2024): I think the real problem is that Ollama trying to differentiate the models which support tool calling from other models. For example, if you created a model like llama3.2 from a gguf file downloaded from Hugging Face and you used a modelfile with a template that is not like what is in the template on the Ollama website the result will be: if you added any tool to the client it will tell you that the model does not support tool call. Ollama searches for something in the template in modelfile to classify it as a model which supports tool calling. This relation with the fact that the default template used by tool-calling models downloaded from the Ollama website forces the model to (always) call a function unless it receives the result of a function call (i.e. this template forces the model to always use tool-calling unless it received a message where the role is "tool". Either they need to update the template used to make the model decide when to use and when not to use tool calling (this may open a door for hallucination), Or (which is the best solution) they should remove this mechanism of classifying the model as a tools model based on that template in modelfile content. I believe that differentiation or classification of the models using the template in modelfile is a very bad idea since tool calling is a very important part of agentic workflows and since there are many models which can be used for this target even the system prompt is not written the way do Ollama needs to allow tool calling
GiteaMirror commented 2026-04-12 15:21:28 -05:00
Author
Owner
Copy Link

@hassanallaham commented on GitHub (Nov 8, 2024):

Here is the default system prompt used by Ollama for llama3.2:
Cutting Knowledge Date: December 2023. When you receive a tool call response, use the output to format an answer to the original user question.
You are a helpful assistant with tool calling capabilities.
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.

{"type":"function", "function":{
"name":"get_weather","description":"return the weather at a given location.
"parameters":{
"type":"object",
"required":["location"],
"properties":{
"location":{"type":"string","description":"The city name you want its weather."},
"temperature_unit":{"type":"string","description":"The temperature unit this can be "C" or "F""}}}}}

Now we can see that the directions forces the model to always call a function. And it can respond normally only in one case which is only when receiving the result of called tool. (That's why Openai has a parameters "tool_choice" to inject the possibilities of tool calling, for example: auto, none, or name of exact tool with The default behavior (tool_choice: "auto") is for the model to decide on its own whether to call a function and if so which function to call. )

<!-- gh-comment-id:2463521178 --> @hassanallaham commented on GitHub (Nov 8, 2024): Here is the default system prompt used by Ollama for llama3.2: Cutting Knowledge Date: December 2023. When you receive a tool call response, use the output to format an answer to the original user question. You are a helpful assistant with tool calling capabilities. Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables. {"type":"function", "function":{ "name":"get_weather","description":"return the weather at a given location. "parameters":{ "type":"object", "required":["location"], "properties":{ "location":{"type":"string","description":"The city name you want its weather."}, "temperature_unit":{"type":"string","description":\"The temperature unit this can be "C" or "F""}}}}} Now we can see that the directions forces the model to always call a function. And it can respond normally only in one case which is only when receiving the result of called tool. (That's why Openai has a parameters "tool_choice" to inject the possibilities of tool calling, for example: auto, none, or name of exact tool with The default behavior (tool_choice: "auto") is for the model to decide on its own whether to call a function and if so which function to call. )
GiteaMirror commented 2026-04-12 15:21:28 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Nov 8, 2024):

Yes, I also believe the template is very poor:

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

If the model follows the directions, it must return a tool call.

In contrast, consider the prompt from llama3-groq-tool-use which does not have this problem:

You are provided with function signatures within XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:

Who creates the Ollama templates? Are they from the Ollama developers, or from the model developers?

<!-- gh-comment-id:2464628687 --> @edmcman commented on GitHub (Nov 8, 2024): Yes, I also believe the template is very poor: > Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt. If the model follows the directions, it must return a tool call. In contrast, consider the [prompt](https://ollama.com/library/llama3-groq-tool-use/blobs/4d1abb94190b) from llama3-groq-tool-use which does not have this problem: > You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows: **Who creates the Ollama templates? Are they from the Ollama developers, or from the model developers?**
GiteaMirror commented 2026-04-12 15:21:29 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 8, 2024):

They are adapted from the chat_template created by the model developers.

<!-- gh-comment-id:2464634302 --> @rick-github commented on GitHub (Nov 8, 2024): They are adapted from the chat_template created by the model developers.
GiteaMirror commented 2026-04-12 15:21:29 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 8, 2024):

If you don't want the model to return a tool call, don't give it tools. If you give it tools, it's because you want the model to use a tool. #6127 discusses this. If you don't want to use the tool support in the ollama API, craft a system message as described here.

<!-- gh-comment-id:2464646967 --> @rick-github commented on GitHub (Nov 8, 2024): If you don't want the model to return a tool call, don't give it tools. If you give it tools, it's because you want the model to use a tool. #6127 discusses this. If you don't want to use the tool support in the ollama API, craft a system message as described [here](https://github.com/ollama/ollama/issues/5793#issuecomment-2238938100).
GiteaMirror commented 2026-04-12 15:21:30 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Nov 8, 2024):

@rick-github https://yourlogicalfallacyis.com/black-or-white

People want the tool_choice = "auto" behavior.

<!-- gh-comment-id:2464659106 --> @edmcman commented on GitHub (Nov 8, 2024): @rick-github https://yourlogicalfallacyis.com/black-or-white People want the tool_choice = "auto" behavior.
GiteaMirror commented 2026-04-12 15:21:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 8, 2024):

As described in #6127.

<!-- gh-comment-id:2464661678 --> @rick-github commented on GitHub (Nov 8, 2024): As described in #6127.
GiteaMirror commented 2026-04-12 15:21:30 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Nov 8, 2024):

Also, here is the "official" prompt template for Llama 3.2: https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2#-prompt-template-

system_prompt = """You are an expert in composing functions. You are given a question and a set of possible functions. 
Based on the question, you will need to make one or more function/tool calls to achieve the purpose. 
If none of the function can be used, point it out. If the given question lacks the parameters required by the function,
also point it out. You should only return the function call in tools call sections.

If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]\n
You SHOULD NOT include any other text in the response.

Here is a list of functions in JSON format that you can invoke.\n\n{functions}\n""".format(functions=function_definitions)

This is not the same as the Ollama prompt.

<!-- gh-comment-id:2464664779 --> @edmcman commented on GitHub (Nov 8, 2024): Also, here is the "official" prompt template for Llama 3.2: https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2#-prompt-template- ``` system_prompt = """You are an expert in composing functions. You are given a question and a set of possible functions. Based on the question, you will need to make one or more function/tool calls to achieve the purpose. If none of the function can be used, point it out. If the given question lacks the parameters required by the function, also point it out. You should only return the function call in tools call sections. If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)]\n You SHOULD NOT include any other text in the response. Here is a list of functions in JSON format that you can invoke.\n\n{functions}\n""".format(functions=function_definitions) ``` This is not the same as the Ollama prompt.
GiteaMirror commented 2026-04-12 15:21:31 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 8, 2024):

This is the official prompt as built in to the HF model:

{{- bos_token }}
{%- if custom_tools is defined %}
    {%- set tools = custom_tools %}
{%- endif %}
{%- if not tools_in_user_message is defined %}
    {%- set tools_in_user_message = true %}
{%- endif %}
{%- if not date_string is defined %}
    {%- if strftime_now is defined %}
        {%- set date_string = strftime_now("%d %b %Y") %}
    {%- else %}
        {%- set date_string = "26 Jul 2024" %}
    {%- endif %}
{%- endif %}
{%- if not tools is defined %}
    {%- set tools = none %}
{%- endif %}

{#- This block extracts the system message, so we can slot it into the right place. #}
{%- if messages[0]['role'] == 'system' %}
    {%- set system_message = messages[0]['content']|trim %}
    {%- set messages = messages[1:] %}
{%- else %}
    {%- set system_message = "" %}
{%- endif %}

{#- System message #}
{{- "<|start_header_id|>system<|end_header_id|>\n\n" }}
{%- if tools is not none %}
    {{- "Environment: ipython\n" }}
{%- endif %}
{{- "Cutting Knowledge Date: December 2023\n" }}
{{- "Today Date: " + date_string + "\n\n" }}
{%- if tools is not none and not tools_in_user_message %}
    {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }}
    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
    {{- "Do not use variables.\n\n" }}
    {%- for t in tools %}
        {{- t | tojson(indent=4) }}
        {{- "\n\n" }}
    {%- endfor %}
{%- endif %}
{{- system_message }}
{{- "<|eot_id|>" }}

{#- Custom tools are passed in a user message with some extra guidance #}
{%- if tools_in_user_message and not tools is none %}
    {#- Extract the first user message so we can plug it in here #}
    {%- if messages | length != 0 %}
        {%- set first_user_message = messages[0]['content']|trim %}
        {%- set messages = messages[1:] %}
    {%- else %}
        {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }}
{%- endif %}
    {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}}
    {{- "Given the following functions, please respond with a JSON for a function call " }}
    {{- "with its proper arguments that best answers the given prompt.\n\n" }}
    {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }}
    {{- "Do not use variables.\n\n" }}
    {%- for t in tools %}
        {{- t | tojson(indent=4) }}
        {{- "\n\n" }}
    {%- endfor %}
    {{- first_user_message + "<|eot_id|>"}}
{%- endif %}

{%- for message in messages %}
    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}
        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }}
    {%- elif 'tool_calls' in message %}
        {%- if not message.tool_calls|length == 1 %}
            {{- raise_exception("This model only supports single tool-calls at once!") }}
        {%- endif %}
        {%- set tool_call = message.tool_calls[0].function %}
        {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}}
        {{- '{"name": "' + tool_call.name + '", ' }}
        {{- '"parameters": ' }}
        {{- tool_call.arguments | tojson }}
        {{- "}" }}
        {{- "<|eot_id|>" }}
    {%- elif message.role == "tool" or message.role == "ipython" %}
        {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }}
        {%- if message.content is mapping or message.content is iterable %}
            {{- message.content | tojson }}
        {%- else %}
            {{- message.content }}
        {%- endif %}
        {{- "<|eot_id|>" }}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }}
{%- endif %}

No auto. If you want to adjust the template to meet your needs, please do so. As described in #6127.

<!-- gh-comment-id:2464669528 --> @rick-github commented on GitHub (Nov 8, 2024): This is the official prompt as built in to the HF model: ``` {{- bos_token }} {%- if custom_tools is defined %} {%- set tools = custom_tools %} {%- endif %} {%- if not tools_in_user_message is defined %} {%- set tools_in_user_message = true %} {%- endif %} {%- if not date_string is defined %} {%- if strftime_now is defined %} {%- set date_string = strftime_now("%d %b %Y") %} {%- else %} {%- set date_string = "26 Jul 2024" %} {%- endif %} {%- endif %} {%- if not tools is defined %} {%- set tools = none %} {%- endif %} {#- This block extracts the system message, so we can slot it into the right place. #} {%- if messages[0]['role'] == 'system' %} {%- set system_message = messages[0]['content']|trim %} {%- set messages = messages[1:] %} {%- else %} {%- set system_message = "" %} {%- endif %} {#- System message #} {{- "<|start_header_id|>system<|end_header_id|>\n\n" }} {%- if tools is not none %} {{- "Environment: ipython\n" }} {%- endif %} {{- "Cutting Knowledge Date: December 2023\n" }} {{- "Today Date: " + date_string + "\n\n" }} {%- if tools is not none and not tools_in_user_message %} {{- "You have access to the following functions. To call a function, please respond with JSON for a function call." }} {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }} {{- "Do not use variables.\n\n" }} {%- for t in tools %} {{- t | tojson(indent=4) }} {{- "\n\n" }} {%- endfor %} {%- endif %} {{- system_message }} {{- "<|eot_id|>" }} {#- Custom tools are passed in a user message with some extra guidance #} {%- if tools_in_user_message and not tools is none %} {#- Extract the first user message so we can plug it in here #} {%- if messages | length != 0 %} {%- set first_user_message = messages[0]['content']|trim %} {%- set messages = messages[1:] %} {%- else %} {{- raise_exception("Cannot put tools in the first user message when there's no first user message!") }} {%- endif %} {{- '<|start_header_id|>user<|end_header_id|>\n\n' -}} {{- "Given the following functions, please respond with a JSON for a function call " }} {{- "with its proper arguments that best answers the given prompt.\n\n" }} {{- 'Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.' }} {{- "Do not use variables.\n\n" }} {%- for t in tools %} {{- t | tojson(indent=4) }} {{- "\n\n" }} {%- endfor %} {{- first_user_message + "<|eot_id|>"}} {%- endif %} {%- for message in messages %} {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %} {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' }} {%- elif 'tool_calls' in message %} {%- if not message.tool_calls|length == 1 %} {{- raise_exception("This model only supports single tool-calls at once!") }} {%- endif %} {%- set tool_call = message.tool_calls[0].function %} {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' -}} {{- '{"name": "' + tool_call.name + '", ' }} {{- '"parameters": ' }} {{- tool_call.arguments | tojson }} {{- "}" }} {{- "<|eot_id|>" }} {%- elif message.role == "tool" or message.role == "ipython" %} {{- "<|start_header_id|>ipython<|end_header_id|>\n\n" }} {%- if message.content is mapping or message.content is iterable %} {{- message.content | tojson }} {%- else %} {{- message.content }} {%- endif %} {{- "<|eot_id|>" }} {%- endif %} {%- endfor %} {%- if add_generation_prompt %} {{- '<|start_header_id|>assistant<|end_header_id|>\n\n' }} {%- endif %} ``` No `auto`. If you want to adjust the template to meet your needs, please do so. As described in #6127.
GiteaMirror commented 2026-04-12 15:21:31 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Nov 8, 2024):

I don't really know the huggingface chat models well enough to know under which circumstances that is used. Aside: I found this template which is clearly a fairly heavily modified version of that template.

Anyway, I made an Ollama llama 3.2 3b model that uses the prompt format from this llama documentation.

It works much better. IMHO, the default Ollama llama 3.2 model should use something like this. Having a chat model that replies to "Hello" with a tool call is silly for most use cases (again, IMHO).

Anyway, props to Ollama team for allowing people to easily fork models and change the templates.

<!-- gh-comment-id:2464851699 --> @edmcman commented on GitHub (Nov 8, 2024): I don't really know the huggingface chat models well enough to know under which circumstances that is used. Aside: I found [this template](https://github.com/vllm-project/vllm/blob/main/examples/tool_chat_template_llama3.2_json.jinja) which is clearly a fairly heavily modified version of that template. Anyway, I made an [Ollama llama 3.2 3b model](https://ollama.com/ejschwar/llama3.2-autotool) that uses the prompt format from [this llama documentation](https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/text_prompt_format.md#zero-shot-function-calling). It works much better. IMHO, the default Ollama llama 3.2 model should use something like this. Having a chat model that replies to "Hello" with a tool call is silly for most use cases (again, IMHO). Anyway, props to Ollama team for allowing people to easily fork models and change the templates.
GiteaMirror commented 2026-04-12 15:21:31 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Nov 13, 2024):

A lot of relevant discussion on this thread:

https://github.com/meta-llama/llama-stack-apps/issues/36#issuecomment-2292193997

The HF prompts are probably not the authoritative source.

<!-- gh-comment-id:2474084226 --> @edmcman commented on GitHub (Nov 13, 2024): A lot of relevant discussion on this thread: https://github.com/meta-llama/llama-stack-apps/issues/36#issuecomment-2292193997 The HF prompts are probably not the authoritative source.
GiteaMirror commented 2026-04-12 15:21:32 -05:00
Author
Owner
Copy Link

@lionel319 commented on GitHub (Feb 7, 2025):

I noticed that as well.

I switched to Qwen2, and it works pretty well.

<!-- gh-comment-id:2642272951 --> @lionel319 commented on GitHub (Feb 7, 2025): I noticed that as well. I switched to Qwen2, and it works pretty well.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4419
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?