github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-13 17:34:40 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5863] function calling documentation #3655

New Issue
Closed
opened 2026-04-12 14:26:46 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 14:26:46 -05:00
Owner
Copy Link
Copy Source

Originally created by @UmutAlihan on GitHub (Jul 22, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5863

While searching the web I couldn't yet find a resource except function callilng through mistral_inference package.

I am curious whether anyone wrote a raw prompt in which function calling is working?

Originally created by @UmutAlihan on GitHub (Jul 22, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5863 While searching the web I couldn't yet find a resource except function callilng through mistral_inference package. I am curious whether anyone wrote a raw prompt in which function calling is working?
GiteaMirror added the documentation label 2026-04-12 14:26:46 -05:00
GiteaMirror commented 2026-04-12 14:26:50 -05:00
Author
Owner
Copy Link
Copy Source

@812781385 commented on GitHub (Jul 23, 2024):

Maybe you should try it：
https://github.com/812781385/ollama-webUI

<!-- gh-comment-id:2244487018 --> @812781385 commented on GitHub (Jul 23, 2024): Maybe you should try it： https://github.com/812781385/ollama-webUI
GiteaMirror commented 2026-04-12 14:26:51 -05:00
Author
Owner
Copy Link
Copy Source

@rick-github commented on GitHub (Jul 23, 2024):

An example of function calling documentation is https://docs.mistral.ai/capabilities/function_calling/

If you want to do function calling without using an LLM specific package, it can be done with direct HTTP calls, for example with the python requests package or using shell tools. For instance, if we take the mistral document from above, we can transpose it into a shell script.

First we set the model. This can be any tool-supporting model, but you asked about mistral-nemo so we'll use that:

model="mistral-nemo"

Then we create the tool list. This is copied from the mistral doc with some syntax changes to account for the difference between python and JSON structures:

tools='[
  {
        "type": "function",
        "function": {
            "name": "retrieve_payment_status",
            "description": "Get payment status of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_date",
            "description": "Get payment date of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    }
]
'

Next we construct the message that will be answered by the LLM using the tools:

messages='[
        {
            "content": "What'\''s the status of my transaction T1001?",
            "role": "user"
        }
    ]
'

Now we create the data that is sent to the LLM:

data='
  {
    "messages": '"$messages"',
    "tools": '"$tools"',
    "model": "'$model'",
    "temperature": 0.3
  }
'

Finally, send the query to ollama:

curl -s --location --request POST 'http://localhost:11434/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data "$data"

The result is:

{
  "id": "chatcmpl-956",
  "object": "chat.completion",
  "created": 1721723598,
  "model": "mistral-nemo",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_bst9zzpx",
            "type": "function",
            "function": {
              "name": "retrieve_payment_status",
              "arguments": "{\"transaction_id\":\"T1001\"}"
            }
          }
        ]
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 125,
    "completion_tokens": 27,
    "total_tokens": 152
  }
}

The script then needs to iterate over the tools calls and make the actual function call to retrieve the data:

$ resp=$(./function-call)
$ echo "$resp" | jq -c .choices[].message.tool_calls[]
{"id":"call_t6chpn4u","type":"function","function":{"name":"retrieve_payment_status","arguments":"{\"transaction_id\":\"T1001\"}"}}
$ for tool_call in $(echo "$resp" | jq -c .choices[].message.tool_calls[]) ; do
  function=$(echo "$tool_call" | jq -r .function.name)
  _=$(which $function) || { echo "function $function not found" ; break ; }
  args=$(echo "$tool_call" | jq -r .function.arguments)
  $function $args    # do the function
done

The python equivalent using requests is:

#!/usr/bin/env python3

import requests

model="mistral-nemo"

tools = [
  {
        "type": "function",
        "function": {
            "name": "retrieve_payment_status",
            "description": "Get payment status of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_date",
            "description": "Get payment date of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    }
]

messages = [
    {
        "content": "What'\''s the status of my transaction T1001?",
        "role": "user"
    }
]

data={
    "messages": messages,
    "tools": tools,
    "model": model,
    "temperature": 0.3
}

response = requests.post('http://localhost:11434/v1/chat/completions',
    headers={"Content-Type": "application/json"},
    json=data)

print(response.text)
<!-- gh-comment-id:2244684986 --> @rick-github commented on GitHub (Jul 23, 2024): An example of function calling documentation is https://docs.mistral.ai/capabilities/function_calling/ If you want to do function calling without using an LLM specific package, it can be done with direct HTTP calls, for example with the python `requests` package or using shell tools. For instance, if we take the mistral document from above, we can transpose it into a shell script. First we set the model. This can be any tool-supporting model, but you asked about mistral-nemo so we'll use that: ```sh model="mistral-nemo" ``` Then we create the tool list. This is copied from the mistral doc with some syntax changes to account for the difference between python and JSON structures: ```sh tools='[ { "type": "function", "function": { "name": "retrieve_payment_status", "description": "Get payment status of a transaction", "parameters": { "type": "object", "properties": { "transaction_id": { "type": "string", "description": "The transaction id." } }, "required": ["transaction_id"] } } }, { "type": "function", "function": { "name": "retrieve_payment_date", "description": "Get payment date of a transaction", "parameters": { "type": "object", "properties": { "transaction_id": { "type": "string", "description": "The transaction id." } }, "required": ["transaction_id"] } } } ] ' ``` Next we construct the message that will be answered by the LLM using the tools: ```sh messages='[ { "content": "What'\''s the status of my transaction T1001?", "role": "user" } ] ' ``` Now we create the data that is sent to the LLM: ```sh data=' { "messages": '"$messages"', "tools": '"$tools"', "model": "'$model'", "temperature": 0.3 } ' ``` Finally, send the query to ollama: ```sh curl -s --location --request POST 'http://localhost:11434/v1/chat/completions' \ --header 'Content-Type: application/json' \ --data "$data" ``` The result is: ```json { "id": "chatcmpl-956", "object": "chat.completion", "created": 1721723598, "model": "mistral-nemo", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "", "tool_calls": [ { "id": "call_bst9zzpx", "type": "function", "function": { "name": "retrieve_payment_status", "arguments": "{\"transaction_id\":\"T1001\"}" } } ] }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 125, "completion_tokens": 27, "total_tokens": 152 } } ``` The script then needs to iterate over the tools calls and make the actual function call to retrieve the data: ```sh $ resp=$(./function-call) $ echo "$resp" | jq -c .choices[].message.tool_calls[] {"id":"call_t6chpn4u","type":"function","function":{"name":"retrieve_payment_status","arguments":"{\"transaction_id\":\"T1001\"}"}} $ for tool_call in $(echo "$resp" | jq -c .choices[].message.tool_calls[]) ; do function=$(echo "$tool_call" | jq -r .function.name) _=$(which $function) || { echo "function $function not found" ; break ; } args=$(echo "$tool_call" | jq -r .function.arguments) $function $args # do the function done ``` The python equivalent using `requests` is: ```python #!/usr/bin/env python3 import requests model="mistral-nemo" tools = [ { "type": "function", "function": { "name": "retrieve_payment_status", "description": "Get payment status of a transaction", "parameters": { "type": "object", "properties": { "transaction_id": { "type": "string", "description": "The transaction id." } }, "required": ["transaction_id"] } } }, { "type": "function", "function": { "name": "retrieve_payment_date", "description": "Get payment date of a transaction", "parameters": { "type": "object", "properties": { "transaction_id": { "type": "string", "description": "The transaction id." } }, "required": ["transaction_id"] } } } ] messages = [ { "content": "What'\''s the status of my transaction T1001?", "role": "user" } ] data={ "messages": messages, "tools": tools, "model": model, "temperature": 0.3 } response = requests.post('http://localhost:11434/v1/chat/completions', headers={"Content-Type": "application/json"}, json=data) print(response.text) ```
GiteaMirror commented 2026-04-12 14:26:52 -05:00
Author
Owner
Copy Link
Copy Source

@ronneldavis commented on GitHub (Jul 24, 2024):

An example of function calling documentation is https://docs.mistral.ai/capabilities/function_calling/

If you want to do function calling without using an LLM specific package, it can be done with direct HTTP calls, for example with the python requests package or using shell tools. For instance, if we take the mistral document from above, we can transpose it into a shell script.

First we set the model. This can be any tool-supporting model, but you asked about mistral-nemo so we'll use that:

model="mistral-nemo"

Then we create the tool list. This is copied from the mistral doc with some syntax changes to account for the difference between python and JSON structures:

tools='[
  {
        "type": "function",
        "function": {
            "name": "retrieve_payment_status",
            "description": "Get payment status of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_date",
            "description": "Get payment date of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    }
]
'

Next we construct the message that will be answered by the LLM using the tools:

messages='[
        {
            "content": "What'\''s the status of my transaction T1001?",
            "role": "user"
        }
    ]
'

Now we create the data that is sent to the LLM:

data='
  {
    "messages": '"$messages"',
    "tools": '"$tools"',
    "model": "'$model'",
    "temperature": 0.3
  }
'

Finally, send the query to ollama:

curl -s --location --request POST 'http://localhost:11434/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data "$data"

The result is:

{
  "id": "chatcmpl-956",
  "object": "chat.completion",
  "created": 1721723598,
  "model": "mistral-nemo",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_bst9zzpx",
            "type": "function",
            "function": {
              "name": "retrieve_payment_status",
              "arguments": "{\"transaction_id\":\"T1001\"}"
            }
          }
        ]
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 125,
    "completion_tokens": 27,
    "total_tokens": 152
  }
}

The script then needs to iterate over the tools calls and make the actual function call to retrieve the data:

$ resp=$(./function-call)
$ echo "$resp" | jq -c .choices[].message.tool_calls[]
{"id":"call_t6chpn4u","type":"function","function":{"name":"retrieve_payment_status","arguments":"{\"transaction_id\":\"T1001\"}"}}
$ for tool_call in $(echo "$resp" | jq -c .choices[].message.tool_calls[]) ; do
  function=$(echo "$tool_call" | jq -r .function.name)
  _=$(which $function) || { echo "function $function not found" ; break ; }
  args=$(echo "$tool_call" | jq -r .function.arguments)
  $function $args    # do the function
done

The python equivalent using requests is:

#!/usr/bin/env python3

import requests

model="mistral-nemo"

tools = [
  {
        "type": "function",
        "function": {
            "name": "retrieve_payment_status",
            "description": "Get payment status of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "retrieve_payment_date",
            "description": "Get payment date of a transaction",
            "parameters": {
                "type": "object",
                "properties": {
                    "transaction_id": {
                        "type": "string",
                        "description": "The transaction id."
                    }
                },
                "required": ["transaction_id"]
            }
        }
    }
]

messages = [
    {
        "content": "What'\''s the status of my transaction T1001?",
        "role": "user"
    }
]

data={
    "messages": messages,
    "tools": tools,
    "model": model,
    "temperature": 0.3
}

response = requests.post('http://localhost:11434/v1/chat/completions',
    headers={"Content-Type": "application/json"},
    json=data)

print(response.text)

@rick-github This example uses the mistral client and not ollama. Ollama does not have native support for tool use as of yet.

<!-- gh-comment-id:2247118705 --> @ronneldavis commented on GitHub (Jul 24, 2024): > An example of function calling documentation is https://docs.mistral.ai/capabilities/function_calling/ > > If you want to do function calling without using an LLM specific package, it can be done with direct HTTP calls, for example with the python `requests` package or using shell tools. For instance, if we take the mistral document from above, we can transpose it into a shell script. > > First we set the model. This can be any tool-supporting model, but you asked about mistral-nemo so we'll use that: > > ```shell > model="mistral-nemo" > ``` > > Then we create the tool list. This is copied from the mistral doc with some syntax changes to account for the difference between python and JSON structures: > > ```shell > tools='[ > { > "type": "function", > "function": { > "name": "retrieve_payment_status", > "description": "Get payment status of a transaction", > "parameters": { > "type": "object", > "properties": { > "transaction_id": { > "type": "string", > "description": "The transaction id." > } > }, > "required": ["transaction_id"] > } > } > }, > { > "type": "function", > "function": { > "name": "retrieve_payment_date", > "description": "Get payment date of a transaction", > "parameters": { > "type": "object", > "properties": { > "transaction_id": { > "type": "string", > "description": "The transaction id." > } > }, > "required": ["transaction_id"] > } > } > } > ] > ' > ``` > > Next we construct the message that will be answered by the LLM using the tools: > > ```shell > messages='[ > { > "content": "What'\''s the status of my transaction T1001?", > "role": "user" > } > ] > ' > ``` > > Now we create the data that is sent to the LLM: > > ```shell > data=' > { > "messages": '"$messages"', > "tools": '"$tools"', > "model": "'$model'", > "temperature": 0.3 > } > ' > ``` > > Finally, send the query to ollama: > > ```shell > curl -s --location --request POST 'http://localhost:11434/v1/chat/completions' \ > --header 'Content-Type: application/json' \ > --data "$data" > ``` > > The result is: > > ```json > { > "id": "chatcmpl-956", > "object": "chat.completion", > "created": 1721723598, > "model": "mistral-nemo", > "system_fingerprint": "fp_ollama", > "choices": [ > { > "index": 0, > "message": { > "role": "assistant", > "content": "", > "tool_calls": [ > { > "id": "call_bst9zzpx", > "type": "function", > "function": { > "name": "retrieve_payment_status", > "arguments": "{\"transaction_id\":\"T1001\"}" > } > } > ] > }, > "finish_reason": "stop" > } > ], > "usage": { > "prompt_tokens": 125, > "completion_tokens": 27, > "total_tokens": 152 > } > } > ``` > > The script then needs to iterate over the tools calls and make the actual function call to retrieve the data: > > ```shell > $ resp=$(./function-call) > $ echo "$resp" | jq -c .choices[].message.tool_calls[] > {"id":"call_t6chpn4u","type":"function","function":{"name":"retrieve_payment_status","arguments":"{\"transaction_id\":\"T1001\"}"}} > $ for tool_call in $(echo "$resp" | jq -c .choices[].message.tool_calls[]) ; do > function=$(echo "$tool_call" | jq -r .function.name) > _=$(which $function) || { echo "function $function not found" ; break ; } > args=$(echo "$tool_call" | jq -r .function.arguments) > $function $args # do the function > done > ``` > > The python equivalent using `requests` is: > > ```python > #!/usr/bin/env python3 > > import requests > > model="mistral-nemo" > > tools = [ > { > "type": "function", > "function": { > "name": "retrieve_payment_status", > "description": "Get payment status of a transaction", > "parameters": { > "type": "object", > "properties": { > "transaction_id": { > "type": "string", > "description": "The transaction id." > } > }, > "required": ["transaction_id"] > } > } > }, > { > "type": "function", > "function": { > "name": "retrieve_payment_date", > "description": "Get payment date of a transaction", > "parameters": { > "type": "object", > "properties": { > "transaction_id": { > "type": "string", > "description": "The transaction id." > } > }, > "required": ["transaction_id"] > } > } > } > ] > > messages = [ > { > "content": "What'\''s the status of my transaction T1001?", > "role": "user" > } > ] > > data={ > "messages": messages, > "tools": tools, > "model": model, > "temperature": 0.3 > } > > response = requests.post('http://localhost:11434/v1/chat/completions', > headers={"Content-Type": "application/json"}, > json=data) > > print(response.text) > ``` @rick-github This example uses the mistral client and not ollama. Ollama does not have native support for tool use as of yet.
GiteaMirror commented 2026-04-12 14:26:53 -05:00
Author
Owner
Copy Link
Copy Source

@rick-github commented on GitHub (Jul 24, 2024):

@ronneldavis Tool support is evolving but ollama supports tool use with tool supporting models since https://github.com/ollama/ollama/pull/5284. This example code does not use the mistral client.

<!-- gh-comment-id:2247148780 --> @rick-github commented on GitHub (Jul 24, 2024): @ronneldavis Tool support is evolving but ollama supports tool use with tool supporting models since https://github.com/ollama/ollama/pull/5284. This example code does not use the mistral client.
GiteaMirror commented 2026-04-12 14:26:54 -05:00
Author
Owner
Copy Link
Copy Source

@ronneldavis commented on GitHub (Jul 24, 2024):

@rick-github Oh that's awesome! I believe the documentation hasn't been updated with the new addition so my apologies

<!-- gh-comment-id:2247440843 --> @ronneldavis commented on GitHub (Jul 24, 2024): @rick-github Oh that's awesome! I believe the documentation hasn't been updated with the new addition so my apologies
GiteaMirror commented 2026-04-12 14:26:54 -05:00
Author
Owner
Copy Link
Copy Source

@jmorganca commented on GitHub (Jul 26, 2024):

Hi there! Function calling is now supported in Ollama. See https://ollama.com/blog/tool-support or the api documentation: https://github.com/ollama/ollama/blob/main/docs/api.md#chat-request-with-tools

<!-- gh-comment-id:2251733340 --> @jmorganca commented on GitHub (Jul 26, 2024): Hi there! Function calling is now supported in Ollama. See https://ollama.com/blog/tool-support or the api documentation: https://github.com/ollama/ollama/blob/main/docs/api.md#chat-request-with-tools
GiteaMirror commented 2026-04-12 14:26:55 -05:00
Author
Owner
Copy Link
Copy Source

@UmutAlihan commented on GitHub (Jul 26, 2024):

thank you thank you thank you

<!-- gh-comment-id:2253353993 --> @UmutAlihan commented on GitHub (Jul 26, 2024): thank you thank you thank you
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels documentation
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3655