github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6155] Support Nested Parameters for Tools #3843

New Issue
Closed
opened 2026-04-12 14:40:55 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-12 14:40:55 -05:00
Owner
Copy Link

Originally created by @kirel on GitHub (Aug 3, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6155

What is the issue?

I have been trying to get ollama tool use to work with https://github.com/jekalmin/extended_openai_conversation but was getting errors. It looked like in the response from ollama tool_calls.0.function.aguments is not an object but a string containing the (sometimes correct) object with the params. I tried to debug where it's coming from and it seems to be an issue with ollama (or the models if they are so bad but I tried a few and it's consistent)

I put together a minimal example:

import requests
import json

# Define the API key and endpoint
api_key = 'ollama'
api_endpoint = 'http://localhost:11434/v1/chat/completions'

# Define the headers
headers = {
    'Content-Type': 'application/json',
    'Authorization': f'Bearer {api_key}',
}

tools = {
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "execute_services",
                "description": "Use this function to execute service of devices in Home Assistant.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "list": {
                            "type": "array",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "domain": {
                                        "type": "string",
                                        "description": "The domain of the service",
                                    },
                                    "service": {
                                        "type": "string",
                                        "description": "The service to be called",
                                    },
                                    "service_data": {
                                        "type": "object",
                                        "description": "The service data object to indicate what to control.",
                                        "properties": {
                                            "entity_id": {
                                                "type": "string",
                                                "description": "The entity_id retrieved from available devices. It must start with domain, followed by dot character.",
                                            }
                                        },
                                        "required": ["entity_id"],
                                    },
                                },
                                "required": ["domain", "service", "service_data"],
                            },
                        }
                    },
                },
            },
        }
    ],
    "tool_choice": "auto"
}
messages = [
    {
        "role": "system",
        "content": "Ich möchte, dass du als Smart Home Manager von Home Assistant agierst. Ich werde Informationen über das Smart Home zusammen mit einer Frage bereitstellen. Du wirst wahrheitsgemäß Korrekturen vornehmen oder die Frage in einem Satz in Alltagssprache anhand der bereitgestellten Informationen beantworten.\n\nAktuelle Zeit: 2024-08-03 22:11:35.920461+02:00\n\nVerfügbare Geräte:\n```csv\nentity_id,name,state,aliases\nlight.buro_deckenlampe_2,Kinderzimmer Deckenlampe,off\nDer aktuelle Zustand der Geräte ist in den verfügbaren Geräten angegeben. Verwende die execute_services-Funktion nur für angeforderte Aktionen, nicht für aktuelle Zustände. Benötigst keine Bestätigung für tool calls. Wiederhole oder lobe nicht, was der Benutzer sagt, sondern stelle eine kurze Anfrage.",
    },
    {
        "role": "user",
        "content": "Mach das Licht aus.",
        "name": "Daniel Kirsch",
    },
]

# Define the request payload
payload = {
    "model": "llama3.1:8b",
    "messages": messages,
    "max_tokens": 100,
    "temperature": 0.5
} | tools

# Make the POST request
response = requests.post(api_endpoint, headers=headers, data=json.dumps(payload))
response.json()
type(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments'])

I've tried

  • llama3.1:8b
  • llama3.1:70b
  • mistral-nemo
  • llama3-groq-tool-use

and they all suffer from the problem.

OS

macOS, Windows

GPU

Nvidia, Apple

CPU

AMD, Apple

Ollama version

0.3.3

Originally created by @kirel on GitHub (Aug 3, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6155 ### What is the issue? I have been trying to get ollama tool use to work with https://github.com/jekalmin/extended_openai_conversation but was getting errors. It looked like in the response from ollama `tool_calls.0.function.aguments` is not an object but a string containing the (sometimes correct) object with the params. I tried to debug where it's coming from and it seems to be an issue with ollama (or the models if they are so bad but I tried a few and it's consistent) I put together a minimal example: ``` import requests import json # Define the API key and endpoint api_key = 'ollama' api_endpoint = 'http://localhost:11434/v1/chat/completions' # Define the headers headers = { 'Content-Type': 'application/json', 'Authorization': f'Bearer {api_key}', } tools = { "tools": [ { "type": "function", "function": { "name": "execute_services", "description": "Use this function to execute service of devices in Home Assistant.", "parameters": { "type": "object", "properties": { "list": { "type": "array", "items": { "type": "object", "properties": { "domain": { "type": "string", "description": "The domain of the service", }, "service": { "type": "string", "description": "The service to be called", }, "service_data": { "type": "object", "description": "The service data object to indicate what to control.", "properties": { "entity_id": { "type": "string", "description": "The entity_id retrieved from available devices. It must start with domain, followed by dot character.", } }, "required": ["entity_id"], }, }, "required": ["domain", "service", "service_data"], }, } }, }, }, } ], "tool_choice": "auto" } messages = [ { "role": "system", "content": "Ich möchte, dass du als Smart Home Manager von Home Assistant agierst. Ich werde Informationen über das Smart Home zusammen mit einer Frage bereitstellen. Du wirst wahrheitsgemäß Korrekturen vornehmen oder die Frage in einem Satz in Alltagssprache anhand der bereitgestellten Informationen beantworten.\n\nAktuelle Zeit: 2024-08-03 22:11:35.920461+02:00\n\nVerfügbare Geräte:\n```csv\nentity_id,name,state,aliases\nlight.buro_deckenlampe_2,Kinderzimmer Deckenlampe,off\nDer aktuelle Zustand der Geräte ist in den verfügbaren Geräten angegeben. Verwende die execute_services-Funktion nur für angeforderte Aktionen, nicht für aktuelle Zustände. Benötigst keine Bestätigung für tool calls. Wiederhole oder lobe nicht, was der Benutzer sagt, sondern stelle eine kurze Anfrage.", }, { "role": "user", "content": "Mach das Licht aus.", "name": "Daniel Kirsch", }, ] # Define the request payload payload = { "model": "llama3.1:8b", "messages": messages, "max_tokens": 100, "temperature": 0.5 } | tools # Make the POST request response = requests.post(api_endpoint, headers=headers, data=json.dumps(payload)) response.json() type(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments']) ``` I've tried - llama3.1:8b - llama3.1:70b - mistral-nemo - llama3-groq-tool-use and they all suffer from the problem. ### OS macOS, Windows ### GPU Nvidia, Apple ### CPU AMD, Apple ### Ollama version 0.3.3
GiteaMirror added the feature requestapi labels 2026-04-12 14:40:55 -05:00
GiteaMirror commented 2026-04-12 14:40:56 -05:00
Author
Owner
Copy Link

@royjhan commented on GitHub (Aug 5, 2024):

https://platform.openai.com/docs/api-reference/chat/object

Screenshot 2024-08-05 at 11 08 52 AM

The openai response object returns arguments as a string, so the compatibility endpoint is maintaining this expectation

<!-- gh-comment-id:2269626727 --> @royjhan commented on GitHub (Aug 5, 2024): https://platform.openai.com/docs/api-reference/chat/object <img width="506" alt="Screenshot 2024-08-05 at 11 08 52 AM" src="https://github.com/user-attachments/assets/86b21f16-ca23-4658-bc7c-69e38609ecba"> The openai response object returns arguments as a string, so the compatibility endpoint is maintaining this expectation
GiteaMirror commented 2026-04-12 14:40:56 -05:00
Author
Owner
Copy Link

@kirel commented on GitHub (Aug 6, 2024):

Oh, then it looks like https://github.com/jekalmin/extended_openai_conversation is not implemented correctly? Thanks, then I'll close and check on that side of the source code.

<!-- gh-comment-id:2271225233 --> @kirel commented on GitHub (Aug 6, 2024): Oh, then it looks like https://github.com/jekalmin/extended_openai_conversation is not implemented correctly? Thanks, then I'll close and check on that side of the source code.
GiteaMirror commented 2026-04-12 14:40:57 -05:00
Author
Owner
Copy Link

@kirel commented on GitHub (Aug 6, 2024):

Ok, I now see what the issue is. There is double json encoding. Look at the response closely:

python> json.loads(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments'])['list']
'[\n    {\n        "service": "light.turn_off",\n        "entity_id": "light.buro_deckenlampe_2"\n    }\n]'

python> json.loads(json.loads(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments'])['list'])
[{'service': 'light.turn_off', 'entity_id': 'light.buro_deckenlampe_2'}]

Is it maybe an ollama bug after all? Or is the the model acting up?

<!-- gh-comment-id:2271286571 --> @kirel commented on GitHub (Aug 6, 2024): Ok, I now see what the issue is. There is double json encoding. Look at the response closely: ```python python> json.loads(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments'])['list'] '[\n {\n "service": "light.turn_off",\n "entity_id": "light.buro_deckenlampe_2"\n }\n]' python> json.loads(json.loads(response.json()['choices'][0]['message']['tool_calls'][0]['function']['arguments'])['list']) [{'service': 'light.turn_off', 'entity_id': 'light.buro_deckenlampe_2'}] ``` Is it maybe an ollama bug after all? Or is the the model acting up?
GiteaMirror commented 2026-04-12 14:40:57 -05:00
Author
Owner
Copy Link

@kirel commented on GitHub (Aug 6, 2024):

Looking into OLLAMA_DEBUG=1 logs I'm seeing

time=2024-08-06T23:43:17.183+02:00 level=DEBUG source=routes.go:1346 msg="chat request" images=0 prompt="<|start_header_id|>system<|end_header_id|>\n\nIc
       │ h möchte, dass du als Smart Home Manager von Home Assistant agierst. Ich werde Informationen über das Smart Home zusammen mit einer Frage bereitstellen. 
       │ Du wirst wahrheitsgemäß Korrekturen vornehmen oder die Frage in einem Satz in Alltagssprache anhand der bereitgestellten Informationen beantworten.\n\nAk
       │ tuelle Zeit: 2024-08-03 22:11:35.920461+02:00\n\nVerfügbare Geräte:\n```csv\nentity_id,name,state,aliases\nlight.buro_deckenlampe_2,Kinderzimmer Deckenla
       │ mpe,off\nDer aktuelle Zustand der Geräte ist in den verfügbaren Geräten angegeben. Verwende die execute_services-Funktion nur für angeforderte Aktionen, 
       │ nicht für aktuelle Zustände. Benötigst keine Bestätigung für tool calls. Wiederhole oder lobe nicht, was der Benutzer sagt, sondern stelle eine kurze Anf
       │ rage.\n\nYou are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the org
       │ inal use question.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nGiven the following functions, please respond with a JSON for a function call wit
       │ h its proper arguments that best answers the given prompt.\n\nRespond in the format {\"name\": function name, \"parameters\": dictionary of argument name
       │  and its value}. Do not use variables.\n\n[{\"type\":\"function\",\"function\":{\"name\":\"execute_services\",\"description\":\"Use this function to exec
       │ ute service of devices in Home Assistant.\",\"parameters\":{\"type\":\"object\",\"required\":null,\"properties\":{\"list\":{\"type\":\"array\",\"descript
       │ ion\":\"\"}}}}}]\n\nMach das Licht aus.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n"

which doesn't even include the full tool spec in the prompt. Does ollama only support shallow parameter list specs?

<!-- gh-comment-id:2272206683 --> @kirel commented on GitHub (Aug 6, 2024): Looking into OLLAMA_DEBUG=1 logs I'm seeing ``` time=2024-08-06T23:43:17.183+02:00 level=DEBUG source=routes.go:1346 msg="chat request" images=0 prompt="<|start_header_id|>system<|end_header_id|>\n\nIc │ h möchte, dass du als Smart Home Manager von Home Assistant agierst. Ich werde Informationen über das Smart Home zusammen mit einer Frage bereitstellen. │ Du wirst wahrheitsgemäß Korrekturen vornehmen oder die Frage in einem Satz in Alltagssprache anhand der bereitgestellten Informationen beantworten.\n\nAk │ tuelle Zeit: 2024-08-03 22:11:35.920461+02:00\n\nVerfügbare Geräte:\n```csv\nentity_id,name,state,aliases\nlight.buro_deckenlampe_2,Kinderzimmer Deckenla │ mpe,off\nDer aktuelle Zustand der Geräte ist in den verfügbaren Geräten angegeben. Verwende die execute_services-Funktion nur für angeforderte Aktionen, │ nicht für aktuelle Zustände. Benötigst keine Bestätigung für tool calls. Wiederhole oder lobe nicht, was der Benutzer sagt, sondern stelle eine kurze Anf │ rage.\n\nYou are a helpful assistant with tool calling capabilities. When you receive a tool call response, use the output to format an answer to the org │ inal use question.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nGiven the following functions, please respond with a JSON for a function call wit │ h its proper arguments that best answers the given prompt.\n\nRespond in the format {\"name\": function name, \"parameters\": dictionary of argument name │ and its value}. Do not use variables.\n\n[{\"type\":\"function\",\"function\":{\"name\":\"execute_services\",\"description\":\"Use this function to exec │ ute service of devices in Home Assistant.\",\"parameters\":{\"type\":\"object\",\"required\":null,\"properties\":{\"list\":{\"type\":\"array\",\"descript │ ion\":\"\"}}}}}]\n\nMach das Licht aus.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" ``` which doesn't even include the full tool spec in the prompt. Does ollama only support shallow parameter list specs?
GiteaMirror commented 2026-04-12 14:40:57 -05:00
Author
Owner
Copy Link

@kirel commented on GitHub (Aug 6, 2024):

Looking at
Bildschirmfoto 2024-08-07 um 00 16 12
and comparing with e04c7012c2/api/types.go (L165-L177) even with my limited Go knowledge I think I can confirm that "deep" parameters like nested objects and arrays are currently not supported.

I just tried with a simpler tool spec of

tools = {
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "execute_service",
                "description": "Use this function to execute service of devices in Home Assistant.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "domain": {
                            "type": "string",
                            "description": "The domain of the service",
                        },
                        "service": {
                            "type": "string",
                            "description": "The service to be called",
                        },
                        "entity_id": {
                            "type": "string",
                            "description": "The entity_id retrieved from available devices. It must start with domain, followed by dot character.",
                        },
                    },
                    "required": ["domain", "service", "entity_id"],
                },
            },
        }
    ],
    "tool_choice": "auto"
}

For that I'm getting the correct response.

So I'm assuming the double encoding in the response when I have a deep arguments object comes from something going wrong when the model tries to respond with a deep arguments object and ollama tries to parse that into it's "flat" struct and when it's encoded as string for the OpenAPI api we get the double encoding. Now that I'm writing it out it makes total sense.

Model responds "arguments": { "list": [...] } which ollama parses as ToolCallFunctionArguments{"list":"[...this is still JSON...]"} which 86b907f82a/openai/openai.go (L216) turns into "arguments": "{\"list\":\"[...now it's encoded twice...]\"}".

Then is this more like a feature request for deep argument specs (i.e. any JSON Schema spec)?!

<!-- gh-comment-id:2272261086 --> @kirel commented on GitHub (Aug 6, 2024): Looking at <img width="427" alt="Bildschirmfoto 2024-08-07 um 00 16 12" src="https://github.com/user-attachments/assets/1e2dbe69-c7a8-4fac-a305-bec860d41d83"> and comparing with https://github.com/ollama/ollama/blob/e04c7012c235d8972afe5538ff27802c77217b83/api/types.go#L165-L177 even with my limited Go knowledge I think I can confirm that "deep" parameters like nested objects and arrays are currently not supported. I just tried with a simpler tool spec of ``` tools = { "tools": [ { "type": "function", "function": { "name": "execute_service", "description": "Use this function to execute service of devices in Home Assistant.", "parameters": { "type": "object", "properties": { "domain": { "type": "string", "description": "The domain of the service", }, "service": { "type": "string", "description": "The service to be called", }, "entity_id": { "type": "string", "description": "The entity_id retrieved from available devices. It must start with domain, followed by dot character.", }, }, "required": ["domain", "service", "entity_id"], }, }, } ], "tool_choice": "auto" } ``` For that I'm getting the correct response. So I'm assuming the double encoding in the response when I have a deep arguments object comes from something going wrong when the model tries to respond with a deep arguments object and ollama tries to parse that into it's "flat" struct and when it's encoded as string for the OpenAPI api we get the double encoding. Now that I'm writing it out it makes total sense. Model responds `"arguments": { "list": [...] }` which ollama parses as `ToolCallFunctionArguments{"list":"[...this is still JSON...]"}` which https://github.com/ollama/ollama/blob/86b907f82ad1cc5eb16e919d6cb5830765d73be4/openai/openai.go#L216 turns into `"arguments": "{\"list\":\"[...now it's encoded twice...]\"}"`. Then is this more like a feature request for deep argument specs (i.e. any JSON Schema spec)?!
GiteaMirror commented 2026-04-12 14:40:58 -05:00
Author
Owner
Copy Link

@royjhan commented on GitHub (Aug 7, 2024):

Yes, that seems to be the case here. I'll edit this issue to reflect a feature case if that's okay with you!

<!-- gh-comment-id:2273909392 --> @royjhan commented on GitHub (Aug 7, 2024): Yes, that seems to be the case here. I'll edit this issue to reflect a feature case if that's okay with you!
GiteaMirror commented 2026-04-12 14:40:58 -05:00
Author
Owner
Copy Link

@kirel commented on GitHub (Aug 7, 2024):

Yes absolutely. Probably current docs should state the limitation and that tool calls on the openai API are not fully compatible.

<!-- gh-comment-id:2273920940 --> @kirel commented on GitHub (Aug 7, 2024): Yes absolutely. Probably current docs should state the limitation and that tool calls on the openai API are not fully compatible.
GiteaMirror commented 2026-04-12 14:40:59 -05:00
Author
Owner
Copy Link

@SGFGOV commented on GitHub (Feb 6, 2025):

Any movement on this issue ? Happy to help anyway way i can

<!-- gh-comment-id:2639856305 --> @SGFGOV commented on GitHub (Feb 6, 2025): Any movement on this issue ? Happy to help anyway way i can
GiteaMirror commented 2026-04-12 14:40:59 -05:00
Author
Owner
Copy Link

@jonigl commented on GitHub (Jul 16, 2025):

Hi there! I ran into the same issue and wanted to share my findings. I opened this issue, where I describe what I discovered and propose a possible solution.

To address the problem, I’ve also submitted two PRs to refactor the related behavior:

Would love any feedback or thoughts on it thanks!

<!-- gh-comment-id:3081590177 --> @jonigl commented on GitHub (Jul 16, 2025): Hi there! I ran into the same issue and wanted to share my findings. I opened [this issue](https://github.com/ollama/ollama/issues/11444), where I describe what I discovered and propose a possible solution. To address the problem, I’ve also submitted two PRs to refactor the related behavior: * [PR #11448 (ollama/ollama)](https://github.com/ollama/ollama/pull/11448) * [PR #542 (ollama/ollama-python)](https://github.com/ollama/ollama-python/pull/542)python/pull/542) Would love any feedback or thoughts on it thanks!
GiteaMirror commented 2026-04-12 14:41:00 -05:00
Author
Owner
Copy Link

@ParthSareen commented on GitHub (Dec 17, 2025):

Closed with https://github.com/ollama/ollama/pull/13508

<!-- gh-comment-id:3666934241 --> @ParthSareen commented on GitHub (Dec 17, 2025): Closed with https://github.com/ollama/ollama/pull/13508
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label api feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3843
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?