github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 08:30:05 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #7544] Despite advertised, granite3-dense does not seem to support tools. #66855

New Issue
Closed
opened 2026-05-04 08:26:42 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-04 08:26:42 -05:00
Owner
Copy Link

Originally created by @chhu on GitHub (Nov 7, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7544

What is the issue?

granite3-dense
Gave bash as tool, but it is refusing to use it, other models work fine (qwen2.5 32b outshines all others for shell use).
Tool setup and sys prompt here: https://github.com/chhu/ollash/blob/main/index.js

asterope:~ >ask List file contents of current folder
Querying granite3-dense:8b-instruct-q8_0...

ls

asterope:~ >

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.1.40

Originally created by @chhu on GitHub (Nov 7, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7544 ### What is the issue? [granite3-dense](https://ollama.com/library/granite3-dense) Gave bash as tool, but it is refusing to use it, other models work fine (qwen2.5 32b outshines all others for shell use). Tool setup and sys prompt here: https://github.com/chhu/ollash/blob/main/index.js asterope:~ >ask List file contents of current folder Querying granite3-dense:8b-instruct-q8_0... ```bash ls ``` asterope:~ > ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.1.40
GiteaMirror added the bug label 2026-05-04 08:26:42 -05:00
GiteaMirror commented 2026-05-04 08:26:46 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Nov 7, 2024):

You need ollama version 0.3.14 or later to run granite3-dense.

<!-- gh-comment-id:2462173153 --> @rick-github commented on GitHub (Nov 7, 2024): You need ollama version 0.3.14 or later to run granite3-dense.
GiteaMirror commented 2026-05-04 08:26:53 -05:00
Author
Owner
Copy Link

@chhu commented on GitHub (Nov 8, 2024):

Sorrysorrysorry, messed that up, I am actually on 0.4.0...

<!-- gh-comment-id:2464310403 --> @chhu commented on GitHub (Nov 8, 2024): Sorrysorrysorry, messed that up, I am actually on 0.4.0...
GiteaMirror commented 2026-05-04 08:26:58 -05:00
Author
Owner
Copy Link

@kellyaa commented on GitHub (Nov 11, 2024):

Try adding this to your prompt (I did, and it worked!):

When calling a tool, respond in the format: <function_call> {"name": function name, "arguments": dictionary of argument name and its value}. Do not use variables.

We are investigating whether this means we need to update the prompt template.

<!-- gh-comment-id:2468661181 --> @kellyaa commented on GitHub (Nov 11, 2024): Try adding this to your prompt (I did, and it worked!): `When calling a tool, respond in the format: <function_call> {"name": function name, "arguments": dictionary of argument name and its value}. Do not use variables.` We are investigating whether this means we need to update the prompt template.
GiteaMirror commented 2026-05-04 08:27:01 -05:00
Author
Owner
Copy Link

@bjhargrave commented on GitHub (Nov 11, 2024):

See also https://www.ibm.com/granite/docs/models/granite/#tool-usefunction-calling-weather-scenario

I used the chat_template in the model.

tokenizer = AutoTokenizer.from_pretrained("ibm-granite/granite-3.0-8b-instruct")
tools = [get_stock_price, get_current_weather]
chat = [
    {"role":"system","content": "You are a helpful assistant with access to the following function calls. Your task is to produce a list of function calls necessary to generate response to the user utterance. Use the following function calls as required."},
    {"role": "user", "content": query }
]

prompt = tokenizer.apply_chat_template(conversation=chat, tools=tools, tokenize=False, add_generation_prompt=True)

I found that the tool schema format generated by the call from the functions (e.g. transformers.utils.get_json_schema) produced better results than the example schema on the link above. The tool schema generated by langchain_ibm.chat_models.convert_to_openai_tool also worked well.

<!-- gh-comment-id:2468789073 --> @bjhargrave commented on GitHub (Nov 11, 2024): See also https://www.ibm.com/granite/docs/models/granite/#tool-usefunction-calling-weather-scenario I used the chat_template in the model. ```python tokenizer = AutoTokenizer.from_pretrained("ibm-granite/granite-3.0-8b-instruct") tools = [get_stock_price, get_current_weather] chat = [ {"role":"system","content": "You are a helpful assistant with access to the following function calls. Your task is to produce a list of function calls necessary to generate response to the user utterance. Use the following function calls as required."}, {"role": "user", "content": query } ] prompt = tokenizer.apply_chat_template(conversation=chat, tools=tools, tokenize=False, add_generation_prompt=True) ``` I found that the tool schema format generated by the call from the functions (e.g. `transformers.utils.get_json_schema`) produced better results than the example schema on the link above. The tool schema generated by `langchain_ibm.chat_models.convert_to_openai_tool` also worked well.
GiteaMirror commented 2026-05-04 08:27:02 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 13, 2025):

As mentioned, granite3-dense requires extra encouragement to use tools. The template for granite3.1-dense has this, if you don't have any particular requirement for granite3, I suggest moving to the updated model. Closing this but feel free to re-open if you continue to have problems.

<!-- gh-comment-id:2586015636 --> @rick-github commented on GitHub (Jan 13, 2025): As mentioned, granite3-dense requires extra encouragement to use tools. The template for [granite3.1-dense](https://ollama.com/library/granite3.1-dense) has this, if you don't have any particular requirement for granite3, I suggest moving to the updated model. Closing this but feel free to re-open if you continue to have problems.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#66855
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?