github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 21:55:43 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #8717] tools support seems broken for most models #31411

New Issue
Closed
opened 2026-04-22 11:50:22 -05:00 by GiteaMirror · 15 comments
GiteaMirror commented 2026-04-22 11:50:22 -05:00
Owner
Copy Link

Originally created by @Fastidius-au on GitHub (Jan 30, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8717

What is the issue?

phi4, qwen2.5 Deepseek hengwen/watt-tool-8B:latest Hammer2

all of these are the better functioncalling models atm and they show as not tools compatible for some reason in flowise ollama chat.......not sure where the issue is but it works sorta with 2.3 so it feels like Ollama hasnt got tools figured out properly....

hengwen/watt-tool-8B:latest and hammer2 are the berkel 8B models of highest rank. seems like you should be loading them into the library and making them work.

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @Fastidius-au on GitHub (Jan 30, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8717 ### What is the issue? phi4, qwen2.5 Deepseek hengwen/watt-tool-8B:latest Hammer2 all of these are the better functioncalling models atm and they show as not tools compatible for some reason in flowise ollama chat.......not sure where the issue is but it works sorta with 2.3 so it feels like Ollama hasnt got tools figured out properly.... hengwen/watt-tool-8B:latest and hammer2 are the berkel 8B models of highest rank. seems like you should be loading them into the library and making them work. ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-22 11:50:22 -05:00
GiteaMirror commented 2026-04-22 11:50:24 -05:00
Author
Owner
Copy Link

@Fastidius-au commented on GitHub (Jan 30, 2025):

llama 3.2 works sorry it just isnt very smart.....the hammer and watt tools are tool trained llama 3.2 I think so they SHOULD be working

<!-- gh-comment-id:2625963907 --> @Fastidius-au commented on GitHub (Jan 30, 2025): llama 3.2 works sorry it just isnt very smart.....the hammer and watt tools are tool trained llama 3.2 I think so they SHOULD be working
GiteaMirror commented 2026-04-22 11:50:24 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 31, 2025):

https://github.com/ollama/ollama/issues/8287#issuecomment-2581625140

<!-- gh-comment-id:2625987153 --> @rick-github commented on GitHub (Jan 31, 2025): https://github.com/ollama/ollama/issues/8287#issuecomment-2581625140
GiteaMirror commented 2026-04-22 11:50:25 -05:00
Author
Owner
Copy Link

@ParthSareen commented on GitHub (Jan 31, 2025):

Our llama3.2 template is not fully accurate as the model was trained to return python functions (and we expect JSON) which is why the reliability is a bit poorer. We don't have official support for tools on phi4 or deepseek yet but you can update the template or try this deepseek version out with the different template: https://ollama.com/michaelneale/deepseek-r1-goose

Ofcourse, YMMV as there isn't an official support for this. Look for the tools tag on the model page to see if there is support. Let me know how it goes!

<!-- gh-comment-id:2628475108 --> @ParthSareen commented on GitHub (Jan 31, 2025): Our llama3.2 template is not fully accurate as the model was trained to return python functions (and we expect JSON) which is why the reliability is a bit poorer. We don't have official support for tools on phi4 or deepseek yet but you can update the template or try this deepseek version out with the different template: https://ollama.com/michaelneale/deepseek-r1-goose Ofcourse, YMMV as there isn't an official support for this. Look for the `tools` tag on the model page to see if there is support. Let me know how it goes!
GiteaMirror commented 2026-04-22 11:50:25 -05:00
Author
Owner
Copy Link

@Fastidius-au commented on GitHub (Feb 2, 2025):

It goes badly. It’s always gone badly because it doesn’t say anything anywhere about any of what you just said. What template where. What screen or are you meaning in code?

Get Outlook for iOShttps://aka.ms/o0ukef

From: Parth Sareen @.>
Sent: Saturday, February 1, 2025 9:03:31 AM
To: ollama/ollama @.>
Cc: Fastidius-au @.>; Author @.>
Subject: Re: [ollama/ollama] tools support seems broken for most models (Issue #8717)

Closed #8717https://github.com/ollama/ollama/issues/8717 as completed.

—
Reply to this email directly, view it on GitHubhttps://github.com/ollama/ollama/issues/8717#event-16146259714, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT3GB7G3CLVIJNS26EPOET32NPXLHAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJWGE2DMMRVHE3TCNA.
You are receiving this because you authored the thread.Message ID: @.***>

<!-- gh-comment-id:2629582585 --> @Fastidius-au commented on GitHub (Feb 2, 2025): It goes badly. It’s always gone badly because it doesn’t say anything anywhere about any of what you just said. What template where. What screen or are you meaning in code? Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Parth Sareen ***@***.***> Sent: Saturday, February 1, 2025 9:03:31 AM To: ollama/ollama ***@***.***> Cc: Fastidius-au ***@***.***>; Author ***@***.***> Subject: Re: [ollama/ollama] tools support seems broken for most models (Issue #8717) Closed #8717<https://github.com/ollama/ollama/issues/8717> as completed. — Reply to this email directly, view it on GitHub<https://github.com/ollama/ollama/issues/8717#event-16146259714>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AT3GB7G3CLVIJNS26EPOET32NPXLHAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJWGE2DMMRVHE3TCNA>. You are receiving this because you authored the thread.Message ID: ***@***.***>
GiteaMirror commented 2026-04-22 11:50:26 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 2, 2025):

https://github.com/ollama/ollama/blob/main/docs/modelfile.md#template

<!-- gh-comment-id:2629587159 --> @rick-github commented on GitHub (Feb 2, 2025): https://github.com/ollama/ollama/blob/main/docs/modelfile.md#template
GiteaMirror commented 2026-04-22 11:50:28 -05:00
Author
Owner
Copy Link

@shruthiR-fauna commented on GitHub (Feb 24, 2025):

Trying to use watt-tool-8B to do function calling with langchain-ollama to make some comparisons across different models that support tool-calling and this is the error I see. How to make sense of this when these models claim to be designed to support tool-calling?

DEBUG:fauna.reasoning.go2_tools:Sending completion request with context: Go2CodePromptInput(asr='Hey robot, what time is it?', pose='unknown', ts=datetime.datetime(2025, 2, 24, 11, 45, 13, 987080))
INFO:httpx:HTTP Request: POST http://127.0.0.1:11434/api/chat "HTTP/1.1 400 Bad Request"
ERROR:fauna.reasoning.go2_tools:Failed to generate plan
Traceback (most recent call last):
  File "/home/rshruthi11/code/fauna2/projects/reasoning-engine/packages/reasoner-go2-tool-calling/reasoner_go2_tool_calling/reasoner.py", line 250, in reason_from_input
    await self.generate_plan(prompt_input)
  File "/home/rshruthi11/code/fauna2/projects/reasoning-engine/packages/reasoner-go2-tool-calling/reasoner_go2_tool_calling/reasoner.py", line 270, in generate_plan
    res = await self._llm_with_tools.ainvoke(prompt)
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 5364, in ainvoke
    return await self.bound.ainvoke(
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 305, in ainvoke
    llm_result = await self.agenerate_prompt(
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 870, in agenerate_prompt
    return await self.agenerate(
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 830, in agenerate
    raise exceptions[0]
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 998, in _agenerate_with_cache
    result = await self._agenerate(
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 788, in _agenerate
    final_chunk = await self._achat_stream_with_aggregation(
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 645, in _achat_stream_with_aggregation
    async for stream_resp in self._acreate_chat_stream(messages, stop, **kwargs):
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 575, in _acreate_chat_stream
    async for part in await self._async_client.chat(**chat_params):
  File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/ollama/_client.py", line 672, in inner
    raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: registry.ollama.ai/hengwen/watt-tool-8B:latest does not support tools (status code: 400)
<!-- gh-comment-id:2679144064 --> @shruthiR-fauna commented on GitHub (Feb 24, 2025): Trying to use `watt-tool-8B` to do function calling with langchain-ollama to make some comparisons across different models that support tool-calling and this is the error I see. How to make sense of this when these models claim to be designed to support tool-calling? ``` DEBUG:fauna.reasoning.go2_tools:Sending completion request with context: Go2CodePromptInput(asr='Hey robot, what time is it?', pose='unknown', ts=datetime.datetime(2025, 2, 24, 11, 45, 13, 987080)) INFO:httpx:HTTP Request: POST http://127.0.0.1:11434/api/chat "HTTP/1.1 400 Bad Request" ERROR:fauna.reasoning.go2_tools:Failed to generate plan Traceback (most recent call last): File "/home/rshruthi11/code/fauna2/projects/reasoning-engine/packages/reasoner-go2-tool-calling/reasoner_go2_tool_calling/reasoner.py", line 250, in reason_from_input await self.generate_plan(prompt_input) File "/home/rshruthi11/code/fauna2/projects/reasoning-engine/packages/reasoner-go2-tool-calling/reasoner_go2_tool_calling/reasoner.py", line 270, in generate_plan res = await self._llm_with_tools.ainvoke(prompt) File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/runnables/base.py", line 5364, in ainvoke return await self.bound.ainvoke( File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 305, in ainvoke llm_result = await self.agenerate_prompt( File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 870, in agenerate_prompt return await self.agenerate( File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 830, in agenerate raise exceptions[0] File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_core/language_models/chat_models.py", line 998, in _agenerate_with_cache result = await self._agenerate( File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 788, in _agenerate final_chunk = await self._achat_stream_with_aggregation( File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 645, in _achat_stream_with_aggregation async for stream_resp in self._acreate_chat_stream(messages, stop, **kwargs): File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/langchain_ollama/chat_models.py", line 575, in _acreate_chat_stream async for part in await self._async_client.chat(**chat_params): File "/home/rshruthi11/.cache/pypoetry/virtualenvs/reasoner-go2-tool-calling-BgWgfSIs-py3.10/lib/python3.10/site-packages/ollama/_client.py", line 672, in inner raise ResponseError(e.response.text, e.response.status_code) from None ollama._types.ResponseError: registry.ollama.ai/hengwen/watt-tool-8B:latest does not support tools (status code: 400) ```
GiteaMirror commented 2026-04-22 11:50:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 24, 2025):

A tool using model needs to have a template that describes how to use tools. Since watt-tool-8b is a fine tuned LLaMa-3.1-8B-Instruct, you can borrow the template from llama3.1:8b.

<!-- gh-comment-id:2679158246 --> @rick-github commented on GitHub (Feb 24, 2025): A tool using model needs to have a template that describes how to use tools. Since watt-tool-8b is a fine tuned LLaMa-3.1-8B-Instruct, you can borrow the template from [`llama3.1:8b`](https://ollama.com/library/llama3.1:8b-instruct-q4_K_M/blobs/948af2743fc7).
GiteaMirror commented 2026-04-22 11:50:29 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Feb 28, 2025):

The watt-tool-8b models are fine-tuned to use pythonic function calls, so they aren't compatible with Ollama right now as per @ParthSareen.

You might want to try my llama 3.2 models (see the blog post) but in my experience llama's tool calling doesn't work that well even when working around the Ollama-related issues.

<!-- gh-comment-id:2690721710 --> @edmcman commented on GitHub (Feb 28, 2025): The watt-tool-8b models are fine-tuned to use [pythonic function calls](https://huggingface.co/watt-ai/watt-tool-8B#how-to-use), so they aren't compatible with Ollama right now as per @ParthSareen. You might want to try my [llama 3.2 models](https://ollama.com/ejschwar/llama3.2-better-prompts) (see the [blog post](https://edmcman.github.io/blog/2025-02-21--lang-chain-and-ollama-make-building-local-tool-calling-agents-easy-it-s-a-shame-they-don-t-work-part-2/)) but in my experience llama's tool calling doesn't work that well even when working around the Ollama-related issues.
GiteaMirror commented 2026-04-22 11:50:30 -05:00
Author
Owner
Copy Link

@lasseedfast commented on GitHub (Mar 12, 2025):

Thanks for clarifying answer @edmcman! The watt models seem very good at function calling, so one wonder if there is a plan to make Ollama work with pythonic function calls?

What other models do you find best for function calling? I've tried llama's (works but not good) and phi4-mini (can't really make it answer with the function in the tool_calls field), but there might be others?

<!-- gh-comment-id:2719174370 --> @lasseedfast commented on GitHub (Mar 12, 2025): Thanks for clarifying answer @edmcman! The watt models seem very good at function calling, so one wonder if there is a plan to make Ollama work with pythonic function calls? What other models do you find best for function calling? I've tried llama's (works but not good) and phi4-mini (can't really make it answer with the function in the tool_calls field), but there might be others?
GiteaMirror commented 2026-04-22 11:50:31 -05:00
Author
Owner
Copy Link

@ParthSareen commented on GitHub (Mar 12, 2025):

@lasseedfast Yes! We plan to support other kinds as well. First on the list will be pythonic functions

<!-- gh-comment-id:2719316712 --> @ParthSareen commented on GitHub (Mar 12, 2025): @lasseedfast Yes! We plan to support other kinds as well. First on the list will be pythonic functions
GiteaMirror commented 2026-04-22 11:50:31 -05:00
Author
Owner
Copy Link

@edmcman commented on GitHub (Mar 13, 2025):

I've had pretty good luck with the qwen 2.5 models, but I've only tried them on llama.cpp.

<!-- gh-comment-id:2719525072 --> @edmcman commented on GitHub (Mar 13, 2025): I've had pretty good luck with the qwen 2.5 models, but I've only tried them on llama.cpp.
GiteaMirror commented 2026-04-22 11:50:32 -05:00
Author
Owner
Copy Link

@cdock1029 commented on GitHub (Mar 13, 2025):

@lasseedfast qwen2.5 works well at calling functions

<!-- gh-comment-id:2720053441 --> @cdock1029 commented on GitHub (Mar 13, 2025): @lasseedfast qwen2.5 works well at calling functions
GiteaMirror commented 2026-04-22 11:50:33 -05:00
Author
Owner
Copy Link

@Fastidius-au commented on GitHub (Mar 13, 2025):

Openweb ui has a native tool addin that addresses it via their gui. You think for the most part you probably should skip all this and just mcp call a seperate llm call for the tool call. Also hammer2 seems to have fixed my issues for the most part. I think the fix isn’t ollama but actually having toolcalls being done as mcp services for universal url based calling

Fasti

From: Conor Dockry @.>
Sent: Thursday, March 13, 2025 4:48:34 PM
To: ollama/ollama @.>
Cc: Fastidius-au @.>; Author @.>
Subject: Re: [ollama/ollama] tools support seems broken for most models (Issue #8717)

@lasseedfasthttps://github.com/lasseedfast qwen2.5 works well at calling functions

—
Reply to this email directly, view it on GitHubhttps://github.com/ollama/ollama/issues/8717#issuecomment-2720053441, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT3GB7BJ6YZNN5MBXPASEOT2UEL3FAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRQGA2TGNBUGE.
You are receiving this because you authored the thread.Message ID: @.***>

[cdock1029]cdock1029 left a comment (ollama/ollama#8717)https://github.com/ollama/ollama/issues/8717#issuecomment-2720053441

@lasseedfasthttps://github.com/lasseedfast qwen2.5 works well at calling functions

—
Reply to this email directly, view it on GitHubhttps://github.com/ollama/ollama/issues/8717#issuecomment-2720053441, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AT3GB7BJ6YZNN5MBXPASEOT2UEL3FAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRQGA2TGNBUGE.
You are receiving this because you authored the thread.Message ID: @.***>

<!-- gh-comment-id:2720066422 --> @Fastidius-au commented on GitHub (Mar 13, 2025): Openweb ui has a native tool addin that addresses it via their gui. You think for the most part you probably should skip all this and just mcp call a seperate llm call for the tool call. Also hammer2 seems to have fixed my issues for the most part. I think the fix isn’t ollama but actually having toolcalls being done as mcp services for universal url based calling Fasti ________________________________ From: Conor Dockry ***@***.***> Sent: Thursday, March 13, 2025 4:48:34 PM To: ollama/ollama ***@***.***> Cc: Fastidius-au ***@***.***>; Author ***@***.***> Subject: Re: [ollama/ollama] tools support seems broken for most models (Issue #8717) @lasseedfast<https://github.com/lasseedfast> qwen2.5 works well at calling functions — Reply to this email directly, view it on GitHub<https://github.com/ollama/ollama/issues/8717#issuecomment-2720053441>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AT3GB7BJ6YZNN5MBXPASEOT2UEL3FAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRQGA2TGNBUGE>. You are receiving this because you authored the thread.Message ID: ***@***.***> [cdock1029]cdock1029 left a comment (ollama/ollama#8717)<https://github.com/ollama/ollama/issues/8717#issuecomment-2720053441> @lasseedfast<https://github.com/lasseedfast> qwen2.5 works well at calling functions — Reply to this email directly, view it on GitHub<https://github.com/ollama/ollama/issues/8717#issuecomment-2720053441>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AT3GB7BJ6YZNN5MBXPASEOT2UEL3FAVCNFSM6AAAAABWGNO56WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMRQGA2TGNBUGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>
GiteaMirror commented 2026-04-22 11:50:35 -05:00
Author
Owner
Copy Link

@mirislam commented on GitHub (Mar 17, 2025):

qwen2.5:14b seems to work the best. It ofcourse is also slower than other smaller model. Would be great to get the watt-tool-8b working

<!-- gh-comment-id:2727758662 --> @mirislam commented on GitHub (Mar 17, 2025): qwen2.5:14b seems to work the best. It ofcourse is also slower than other smaller model. Would be great to get the watt-tool-8b working
GiteaMirror commented 2026-04-22 11:50:35 -05:00
Author
Owner
Copy Link

@uinstinct commented on GitHub (Sep 12, 2025):

Tool support should now be possible on most models with system message tools

<!-- gh-comment-id:3283840035 --> @uinstinct commented on GitHub (Sep 12, 2025): Tool support should now be possible on most models with [system message tools](https://docs.continue.dev/features/agent/model-setup#how-system-message-tools-work)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#31411
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?