github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #10338] Granite3.3 : thinking doesn't work if using tools #68848

New Issue
Closed
opened 2026-05-04 15:24:18 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-04 15:24:18 -05:00
Owner
Copy Link

Originally created by @lemassykoi on GitHub (Apr 18, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10338

What is the issue?

when submitting query to granite3.3 with enable thinking and tools, no thinking process

CODE:

import requests
import json
import base64
from utils import convert_nanoseconds
import colorama
BLUE = colorama.Fore.BLUE
YELLOW = colorama.Fore.YELLOW
RESET = colorama.Style.RESET_ALL

OLLAMA_MODEL = "granite3.3:8b"
OLLAMA_URL = "http://127.0.0.1:11434/api/chat"

############################
##   CHAT  WITHOUT TOOLS   #
############################
user_query = 'Hello, my name is Bob! What can you do?'

messages_without_thinking = [
    {
        "role": "user",
        "content": user_query
    }
]

messages_with_thinking = [
    {
        "role": "control",
        "content": "thinking"
    },
    {
        "role": "user",
        "content": user_query
    }
]


data = {
    "model": OLLAMA_MODEL,
    "messages": messages_with_thinking,
    "stream": False,
    "keep_alive": "1m",
    "options": {'temperature': 0.0, 'seed': 1234567890}
}
headers = {"Content-Type": "application/json"}
payload = json.dumps(data).encode("utf-8")
response = requests.post(OLLAMA_URL, headers=headers, data=payload, stream=False)

if response.status_code == 200:
    print(response.json())
    print(YELLOW + response.json()['message']['content'] + RESET)

else:
    print(f"Error: {response.status_code}")
    print(response.text)

print('\n')



#########################
##   CHAT  WITH TOOLS   #
#########################
user_query = 'Hello, my name is Bob! What can you do?'

messages_without_thinking = [
    {
        "role": "user",
        "content": user_query
    }
]

messages_with_thinking = [
    {
        "role": "control",
        "content": "thinking"
    },
    {
        "role": "user",
        "content": user_query
    }
]

tools = [
    {
        'type': 'function',
        'function': {
            'name': 'get_current_weather',
            'description': 'Get the current weather for a city',
            'parameters': {
                'type': 'object',
                'properties': {
                'city': {
                    'type': 'string',
                    'description': 'The name of the city',
                },
            },
            'required': ['city'],
            },
        },
    },
]

data = {
    "model": OLLAMA_MODEL,
    "messages": messages_with_thinking,
    "stream": False,
    "keep_alive": "1m",
    "tools": tools,
    "options": {'temperature': 0.0, 'seed': 1234567890}
}
headers = {"Content-Type": "application/json"}
payload = json.dumps(data).encode("utf-8")
response = requests.post(OLLAMA_URL, headers=headers, data=payload, stream=False)

if response.status_code == 200:
    print(response.json())
    print(YELLOW + response.json()['message']['content'] + RESET)

else:
    print(f"Error: {response.status_code}")
    print(response.text)

Relevant log output

## QUERY WITHOUT TOOLS
{'model': 'granite3.3:8b', 'created_at': '2025-04-18T14:40:18.277971846Z', 'message': {'role': 'assistant', 'content': "<think>Bob has asked a question to understand what I, as an AI, can assist with. Typically, an AI assistant like me can provide information, answer questions, help with tasks such as setting reminders or providing explanations on various topics, offer suggestions, and engage in conversation. \n\nTo address Bob's query comprehensively:\n\n1. **Information Provision**: I can give facts about a wide range of subjects including science, history, geography, technology, and more.\n2. **Q&A**: Answer questions to the best of my knowledge and ability, drawing from a vast dataset up to 2021.\n3. **Task Assistance**: Set reminders, provide step-by-step instructions for tasks, or suggest resources for further learning.\n4. **Conversational Engagement**: Participate in casual conversation, offer explanations, and clarify concepts.\n5. **Creative Tasks**: Generate simple texts like summaries, drafts, or brainstorm ideas within certain constraints.\n\nI am designed to be helpful, informative, and engaging while adhering to ethical guidelines.</think><response>Hello Bob! It's great to meet you. As an AI, I can assist in several ways:\n\n1. **Information Retrieval**: I can provide information on a broad spectrum of topics ranging from science and history to technology and more. Think of me as a digital encyclopedia with access to a vast amount of data up to 2021.\n\n2. **Answering Questions**: You can ask me questions, and I'll do my best to provide accurate and detailed responses based on the knowledge I've been trained on.\n\n3. **Task Support**: Need a reminder set or instructions for something? I can help with that. I can also guide you through processes or suggest resources for further exploration of a subject.\n\n4. **Conversational Interaction**: Engage in discussions, clarify concepts, or simply chat about various topics to pass the time.\n\n5. **Creative Assistance**: For tasks involving writing, like drafting an outline or summarizing information, I can offer assistance within given constraints.\n\nWhile I strive to be as helpful and accurate as possible, please keep in mind that my responses are based on patterns in the data I was trained on and don't represent real-time updates or personal experiences. \n\nFeel free to ask me anything you're curious about or need help with! How can I assist you today, Bob?</response>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 6983629209, 'load_duration': 42811848, 'prompt_eval_count': 185, 'prompt_eval_duration': 19407085, 'eval_count': 535, 'eval_duration': 6920214915}
<think>Bob has asked a question to understand what I, as an AI, can assist with. Typically, an AI assistant like me can provide information, answer questions, help with tasks such as setting reminders or providing explanations on various topics, offer suggestions, and engage in conversation. 

To address Bob's query comprehensively:

1. **Information Provision**: I can give facts about a wide range of subjects including science, history, geography, technology, and more.
2. **Q&A**: Answer questions to the best of my knowledge and ability, drawing from a vast dataset up to 2021.
3. **Task Assistance**: Set reminders, provide step-by-step instructions for tasks, or suggest resources for further learning.
4. **Conversational Engagement**: Participate in casual conversation, offer explanations, and clarify concepts.
5. **Creative Tasks**: Generate simple texts like summaries, drafts, or brainstorm ideas within certain constraints.

I am designed to be helpful, informative, and engaging while adhering to ethical guidelines.</think><response>Hello Bob! It's great to meet you. As an AI, I can assist in several ways:

1. **Information Retrieval**: I can provide information on a broad spectrum of topics ranging from science and history to technology and more. Think of me as a digital encyclopedia with access to a vast amount of data up to 2021.

2. **Answering Questions**: You can ask me questions, and I'll do my best to provide accurate and detailed responses based on the knowledge I've been trained on.

3. **Task Support**: Need a reminder set or instructions for something? I can help with that. I can also guide you through processes or suggest resources for further exploration of a subject.

4. **Conversational Interaction**: Engage in discussions, clarify concepts, or simply chat about various topics to pass the time.

5. **Creative Assistance**: For tasks involving writing, like drafting an outline or summarizing information, I can offer assistance within given constraints.

While I strive to be as helpful and accurate as possible, please keep in mind that my responses are based on patterns in the data I was trained on and don't represent real-time updates or personal experiences. 

Feel free to ask me anything you're curious about or need help with! How can I assist you today, Bob?</response>

## QUERY WITH TOOLS
{'model': 'granite3.3:8b', 'created_at': '2025-04-18T14:40:18.787832955Z', 'message': {'role': 'assistant', 'content': 'Hello Bob, I am an assistant that can provide information on various topics. For instance, I can give you the current weather in any city. How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 507598158, 'load_duration': 9873401, 'prompt_eval_count': 177, 'prompt_eval_duration': 14062721, 'eval_count': 37, 'eval_duration': 482543869}
Hello Bob, I am an assistant that can provide information on various topics. For instance, I can give you the current weather in any city. How can I assist you today?

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.6.5

Originally created by @lemassykoi on GitHub (Apr 18, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10338 ### What is the issue? when submitting query to granite3.3 with enable thinking and tools, no thinking process CODE: ```python import requests import json import base64 from utils import convert_nanoseconds import colorama BLUE = colorama.Fore.BLUE YELLOW = colorama.Fore.YELLOW RESET = colorama.Style.RESET_ALL OLLAMA_MODEL = "granite3.3:8b" OLLAMA_URL = "http://127.0.0.1:11434/api/chat" ############################ ## CHAT WITHOUT TOOLS # ############################ user_query = 'Hello, my name is Bob! What can you do?' messages_without_thinking = [ { "role": "user", "content": user_query } ] messages_with_thinking = [ { "role": "control", "content": "thinking" }, { "role": "user", "content": user_query } ] data = { "model": OLLAMA_MODEL, "messages": messages_with_thinking, "stream": False, "keep_alive": "1m", "options": {'temperature': 0.0, 'seed': 1234567890} } headers = {"Content-Type": "application/json"} payload = json.dumps(data).encode("utf-8") response = requests.post(OLLAMA_URL, headers=headers, data=payload, stream=False) if response.status_code == 200: print(response.json()) print(YELLOW + response.json()['message']['content'] + RESET) else: print(f"Error: {response.status_code}") print(response.text) print('\n') ######################### ## CHAT WITH TOOLS # ######################### user_query = 'Hello, my name is Bob! What can you do?' messages_without_thinking = [ { "role": "user", "content": user_query } ] messages_with_thinking = [ { "role": "control", "content": "thinking" }, { "role": "user", "content": user_query } ] tools = [ { 'type': 'function', 'function': { 'name': 'get_current_weather', 'description': 'Get the current weather for a city', 'parameters': { 'type': 'object', 'properties': { 'city': { 'type': 'string', 'description': 'The name of the city', }, }, 'required': ['city'], }, }, }, ] data = { "model": OLLAMA_MODEL, "messages": messages_with_thinking, "stream": False, "keep_alive": "1m", "tools": tools, "options": {'temperature': 0.0, 'seed': 1234567890} } headers = {"Content-Type": "application/json"} payload = json.dumps(data).encode("utf-8") response = requests.post(OLLAMA_URL, headers=headers, data=payload, stream=False) if response.status_code == 200: print(response.json()) print(YELLOW + response.json()['message']['content'] + RESET) else: print(f"Error: {response.status_code}") print(response.text) ``` ### Relevant log output ```shell ## QUERY WITHOUT TOOLS {'model': 'granite3.3:8b', 'created_at': '2025-04-18T14:40:18.277971846Z', 'message': {'role': 'assistant', 'content': "<think>Bob has asked a question to understand what I, as an AI, can assist with. Typically, an AI assistant like me can provide information, answer questions, help with tasks such as setting reminders or providing explanations on various topics, offer suggestions, and engage in conversation. \n\nTo address Bob's query comprehensively:\n\n1. **Information Provision**: I can give facts about a wide range of subjects including science, history, geography, technology, and more.\n2. **Q&A**: Answer questions to the best of my knowledge and ability, drawing from a vast dataset up to 2021.\n3. **Task Assistance**: Set reminders, provide step-by-step instructions for tasks, or suggest resources for further learning.\n4. **Conversational Engagement**: Participate in casual conversation, offer explanations, and clarify concepts.\n5. **Creative Tasks**: Generate simple texts like summaries, drafts, or brainstorm ideas within certain constraints.\n\nI am designed to be helpful, informative, and engaging while adhering to ethical guidelines.</think><response>Hello Bob! It's great to meet you. As an AI, I can assist in several ways:\n\n1. **Information Retrieval**: I can provide information on a broad spectrum of topics ranging from science and history to technology and more. Think of me as a digital encyclopedia with access to a vast amount of data up to 2021.\n\n2. **Answering Questions**: You can ask me questions, and I'll do my best to provide accurate and detailed responses based on the knowledge I've been trained on.\n\n3. **Task Support**: Need a reminder set or instructions for something? I can help with that. I can also guide you through processes or suggest resources for further exploration of a subject.\n\n4. **Conversational Interaction**: Engage in discussions, clarify concepts, or simply chat about various topics to pass the time.\n\n5. **Creative Assistance**: For tasks involving writing, like drafting an outline or summarizing information, I can offer assistance within given constraints.\n\nWhile I strive to be as helpful and accurate as possible, please keep in mind that my responses are based on patterns in the data I was trained on and don't represent real-time updates or personal experiences. \n\nFeel free to ask me anything you're curious about or need help with! How can I assist you today, Bob?</response>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 6983629209, 'load_duration': 42811848, 'prompt_eval_count': 185, 'prompt_eval_duration': 19407085, 'eval_count': 535, 'eval_duration': 6920214915} <think>Bob has asked a question to understand what I, as an AI, can assist with. Typically, an AI assistant like me can provide information, answer questions, help with tasks such as setting reminders or providing explanations on various topics, offer suggestions, and engage in conversation. To address Bob's query comprehensively: 1. **Information Provision**: I can give facts about a wide range of subjects including science, history, geography, technology, and more. 2. **Q&A**: Answer questions to the best of my knowledge and ability, drawing from a vast dataset up to 2021. 3. **Task Assistance**: Set reminders, provide step-by-step instructions for tasks, or suggest resources for further learning. 4. **Conversational Engagement**: Participate in casual conversation, offer explanations, and clarify concepts. 5. **Creative Tasks**: Generate simple texts like summaries, drafts, or brainstorm ideas within certain constraints. I am designed to be helpful, informative, and engaging while adhering to ethical guidelines.</think><response>Hello Bob! It's great to meet you. As an AI, I can assist in several ways: 1. **Information Retrieval**: I can provide information on a broad spectrum of topics ranging from science and history to technology and more. Think of me as a digital encyclopedia with access to a vast amount of data up to 2021. 2. **Answering Questions**: You can ask me questions, and I'll do my best to provide accurate and detailed responses based on the knowledge I've been trained on. 3. **Task Support**: Need a reminder set or instructions for something? I can help with that. I can also guide you through processes or suggest resources for further exploration of a subject. 4. **Conversational Interaction**: Engage in discussions, clarify concepts, or simply chat about various topics to pass the time. 5. **Creative Assistance**: For tasks involving writing, like drafting an outline or summarizing information, I can offer assistance within given constraints. While I strive to be as helpful and accurate as possible, please keep in mind that my responses are based on patterns in the data I was trained on and don't represent real-time updates or personal experiences. Feel free to ask me anything you're curious about or need help with! How can I assist you today, Bob?</response> ## QUERY WITH TOOLS {'model': 'granite3.3:8b', 'created_at': '2025-04-18T14:40:18.787832955Z', 'message': {'role': 'assistant', 'content': 'Hello Bob, I am an assistant that can provide information on various topics. For instance, I can give you the current weather in any city. How can I assist you today?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 507598158, 'load_duration': 9873401, 'prompt_eval_count': 177, 'prompt_eval_duration': 14062721, 'eval_count': 37, 'eval_duration': 482543869} Hello Bob, I am an assistant that can provide information on various topics. For instance, I can give you the current weather in any city. How can I assist you today? ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.6.5
GiteaMirror added the bug label 2026-05-04 15:24:18 -05:00
GiteaMirror commented 2026-05-04 15:24:21 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 18, 2025):

Thinking is enabled in the template only if tools and document processing is not specified.

    {{- /* Prompt without tools or documents */}}
    {{- if (and (not .Tools) (not $documents)) }}
        {{- $system = print $system " You are a helpful AI assistant."}}
        {{- if $thinking}}
            {{- $system = print $system "\nRespond to every user query in a
<!-- gh-comment-id:2815588939 --> @rick-github commented on GitHub (Apr 18, 2025): Thinking is enabled in the template only if tools and document processing is not specified. ``` {{- /* Prompt without tools or documents */}} {{- if (and (not .Tools) (not $documents)) }} {{- $system = print $system " You are a helpful AI assistant."}} {{- if $thinking}} {{- $system = print $system "\nRespond to every user query in a ```
GiteaMirror commented 2026-05-04 15:24:21 -05:00
Author
Owner
Copy Link

@lemassykoi commented on GitHub (Apr 18, 2025):

why? is it normal behaviour?

<!-- gh-comment-id:2815592216 --> @lemassykoi commented on GitHub (Apr 18, 2025): why? is it normal behaviour?
GiteaMirror commented 2026-05-04 15:24:21 -05:00
Author
Owner
Copy Link

@lemassykoi commented on GitHub (Apr 18, 2025):

Ok, I created a model file to bypass this restriction that I really don't understand

FROM granite3.3:8b

TEMPLATE """{{- /*

------ MESSAGE PARSING ------

*/}}
{{- /*
Declare the prompt structure variables to be filled in from messages
*/}}
{{- $system := "" }}
{{- $documents := "" }}
{{- $documentCounter := 0 }}
{{- $thinking := false }}
{{- $citations := false }}
{{- $hallucinations := false }}
{{- $length := "" }}
{{- $originality := "" }}

{{- /*
Loop over messages and look for a user-provided system message and documents
*/ -}}
{{- range .Messages }}

    {{- /* User defined system prompt(s) */}}
    {{- if (eq .Role "system")}}
        {{- if (ne $system "") }}
            {{- $system = print $system "\n\n" }}
        {{- end}}
        {{- $system = print $system .Content }}
    {{- end}}

    {{- /* Role specified controls */ -}}
    {{- if (and (ge (len .Role) 7) (eq (slice .Role 0 7) "control")) }}
        {{- if (eq .Content "thinking")}}{{- $thinking = true }}{{- end}}
        {{- if (eq .Content "citations")}}{{- $citations = true }}{{- end}}
        {{- if (eq .Content "hallucinations")}}{{- $hallucinations = true }}{{- end}}
        {{- if (and (ge (len .Content) 7) (eq (slice .Content 0 7) "length "))}}
            {{- $length = slice .Content 7 }}
        {{- end}}
        {{- if (and (ge (len .Content) 12) (eq (slice .Content 0 12) "originality "))}}
            {{- $originality = slice .Content 12 }}
        {{- end}}
    {{- end}}

    {{- /* Role specified document */ -}}
    {{- if (and (ge (len .Role) 8) (eq (slice .Role 0 8) "document")) }}
        {{- if (ne $documentCounter 0)}}
            {{- $documents = print $documents "\n\n"}}
        {{- end}}
        {{- $identifier := ""}}
        {{- if (ge (len .Role) 9) }}
            {{- $identifier = (slice .Role 9)}}
        {{- end}}
        {{- if (eq $identifier "") }}
            {{- $identifier := print $documentCounter}}
        {{- end}}
        {{- $documents = print $documents "<|start_of_role|>document {\"document_id\": \"" $identifier "\"}<|end_of_role|>\n" .Content "<|end_of_text|>"}}
        {{- $documentCounter = len (printf "a%*s" $documentCounter "")}}
    {{- end}}
{{- end}}

{{- /*
If no user message provided, build the default system message
*/ -}}
{{- if eq $system "" }}
    {{- $system = "Knowledge Cutoff Date: April 2024.\nYou are Granite, developed by IBM."}}

    {{- /* !!! MOVED THINKING BLOCK HERE !!! */}}
    {{- /* Add thinking instructions if requested, regardless of tools/docs */}}
    {{- if $thinking}}
        {{- $system = print $system "\nRespond to every user query in a comprehensive and detailed way. You can write down your thoughts and reasoning process before responding. In the thought process, engage in a comprehensive cycle of analysis, summarization, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. In the response section, based on various attempts, explorations, and reflections from the thoughts section, systematically present the final solution that you deem correct. The response should summarize the thought process. Write your thoughts between <think></think> and write your response between <response></response> for each user query."}}
    {{- end}}
    {{- /* !!! END OF MOVED BLOCK !!! */}}


    {{- /* Add Tools prompt */}}
    {{- if .Tools }}
        {{- $system = print $system " You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request." }}
    {{- end}}

    {{- /* Add documents prompt */}}
    {{- if $documents }}
        {{- if .Tools }}
            {{- $system = print $system "\n"}}
        {{- else }}
            {{- /* Add space only if no tools were mentioned */}}
            {{- $system = print $system " "}}
        {{- end}}
        {{- $system = print $system "Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data." }}
        {{- if $citations}}
            {{- $system = print $system "\nUse the symbols <|start_of_cite|> and <|end_of_cite|> to indicate when a fact comes from a document in the search result, e.g <|start_of_cite|> {document_id: 1}my fact <|end_of_cite|> for a fact from document 1. Afterwards, list all the citations with their corresponding documents in an ordered list."}}
        {{- end}}
        {{- if $hallucinations}}
            {{- $system = print $system "\nFinally, after the response is written, include a numbered list of sentences from the response with a corresponding risk value that are hallucinated and not based in the documents."}}
        {{- end}}
    {{- end}}

    {{- /* Prompt ONLY when no tools or documents were present */}}
    {{- /* We removed the thinking part from here */}}
    {{- if (and (not .Tools) (not $documents)) }}
        {{- $system = print $system " You are a helpful AI assistant."}}
        {{- /* Original thinking block was here, now removed */}}
    {{- end}}

{{- end}}
{{- /*

------ TEMPLATE EXPANSION ------

*/}}
{{- /* System Prompt */ -}}
<|start_of_role|>system<|end_of_role|>{{- $system }}<|end_of_text|>

{{- /* Tools */ -}}
{{- if .Tools }}
<|start_of_role|>available_tools<|end_of_role|>[
{{- range $index, $_ := .Tools }}
{{ . }}
{{- if and (ne (len (slice $.Tools $index)) 1) (gt (len $.Tools) 1) }},
{{- end}}
{{- end }}
]<|end_of_text|>
{{- end}}

{{- /* Documents */ -}}
{{- if $documents }}
{{ $documents }}
{{- end}}

{{- /* Standard Messages */}}
{{- range $index, $_ := .Messages }}
{{- if (and
    (ne .Role "system")
    (or (lt (len .Role) 7) (ne (slice .Role 0 7) "control"))
    (or (lt (len .Role) 8) (ne (slice .Role 0 8) "document"))
)}}
<|start_of_role|>
{{- if eq .Role "tool" }}tool_response
{{- else }}{{ .Role }}
{{- end }}<|end_of_role|>
{{- if .Content }}{{ .Content }}
{{- else if .ToolCalls }}<|tool_call|>
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}
{{- end }}
{{- if eq (len (slice $.Messages $index)) 1 }}
{{- if eq .Role "assistant" }}
{{- else }}<|end_of_text|>
<|start_of_role|>assistant
{{- if and (ne $length "") (ne $originality "") }} {"length": "{{ $length }}", "originality": "{{ $originality }}"}
{{- else if ne $length "" }} {"length": "{{ $length }}"}
{{- else if ne $originality "" }} {"originality": "{{ $originality }}"}
{{- end }}<|end_of_role|>
{{- end -}}
{{- else }}<|end_of_text|>
{{- end }}
{{- end }}
{{- end }}"""

# --- Ollama Parameters (Copied from original) ---
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|start_of_role|>"
PARAMETER stop "<|end_of_role|>"

Now, I have the same problem than with Granite3.2 and images with history : if I pass chat history, thinking is disabled

## CHAT WITH TOOLS AND HISTORY
{'model': 'granite3.3-think-tools:latest', 'created_at': '2025-04-18T15:17:16.952829421Z', 'message': {'role': 'assistant', 'content': 'I am designed to provide information and perform tasks based on a set of predefined functions. Currently, I can fetch the current weather of a city using an external weather API. Would you like to know the weather in a specific location?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 665282668, 'load_duration': 40344608, 'prompt_eval_count': 338, 'prompt_eval_duration': 12273921, 'eval_count': 47, 'eval_duration': 607266017}
I am designed to provide information and perform tasks based on a set of predefined functions. Currently, I can fetch the current weather of a city using an external weather API. Would you like to know the weather in a specific location?

## CHAT WITH TOOLS WITHOUT HISTORY
{'model': 'granite3.3-think-tools:latest', 'created_at': '2025-04-18T15:17:21.51231505Z', 'message': {'role': 'assistant', 'content': "<think>The user is introducing themselves and asking about my capabilities. I should provide a general overview of my functionalities without specific task requests. \n\nI am an AI assistant designed to help answer questions, provide information, and perform tasks such as:\n- Fetching data from the internet (with access to specified functions).\n- Summarizing texts or providing explanations on various topics.\n- Assisting with scheduling or reminders.\n- Offering basic calculations or conversions.\n\nFor this interaction, the user has not asked for a specific task yet, so I should keep my response general.</think><response>Hello Bob! I'm an AI assistant designed to help answer your questions and perform various tasks. Here are some things I can do:\n\n1. **Provide Information**: I can give you facts on a wide range of topics from science and history to general knowledge queries.\n2. **Web Data Fetching**: With access to specific functions, I can retrieve real-time data like weather updates, sports scores, or stock prices. For instance, I can tell you the current weather in any city if you provide me with the name.\n3. **Text Summarization and Explanation**: If you have a lengthy text or complex concept, I can help summarize it or explain it in simpler terms.\n4. **Task Management**: I can assist with setting reminders or managing your schedule.\n5. **Basic Calculations and Conversions**: Need to convert units or do some quick math? I'm here to help with that too!\n\nPlease let me know how you'd like to proceed or if you have a specific question or task in mind.</response>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 4557362810, 'load_duration': 8512238, 'prompt_eval_count': 307, 'prompt_eval_duration': 7425351, 'eval_count': 351, 'eval_duration': 4540118289}
<think>The user is introducing themselves and asking about my capabilities. I should provide a general overview of my functionalities without specific task requests. 

I am an AI assistant designed to help answer questions, provide information, and perform tasks such as:
- Fetching data from the internet (with access to specified functions).
- Summarizing texts or providing explanations on various topics.
- Assisting with scheduling or reminders.
- Offering basic calculations or conversions.

For this interaction, the user has not asked for a specific task yet, so I should keep my response general.</think><response>Hello Bob! I'm an AI assistant designed to help answer your questions and perform various tasks. Here are some things I can do:

1. **Provide Information**: I can give you facts on a wide range of topics from science and history to general knowledge queries.
2. **Web Data Fetching**: With access to specific functions, I can retrieve real-time data like weather updates, sports scores, or stock prices. For instance, I can tell you the current weather in any city if you provide me with the name.
3. **Text Summarization and Explanation**: If you have a lengthy text or complex concept, I can help summarize it or explain it in simpler terms.
4. **Task Management**: I can assist with setting reminders or managing your schedule.
5. **Basic Calculations and Conversions**: Need to convert units or do some quick math? I'm here to help with that too!

Please let me know how you'd like to proceed or if you have a specific question or task in mind.</response>
<!-- gh-comment-id:2815633715 --> @lemassykoi commented on GitHub (Apr 18, 2025): Ok, I created a model file to bypass this restriction that I really don't understand ``` FROM granite3.3:8b TEMPLATE """{{- /* ------ MESSAGE PARSING ------ */}} {{- /* Declare the prompt structure variables to be filled in from messages */}} {{- $system := "" }} {{- $documents := "" }} {{- $documentCounter := 0 }} {{- $thinking := false }} {{- $citations := false }} {{- $hallucinations := false }} {{- $length := "" }} {{- $originality := "" }} {{- /* Loop over messages and look for a user-provided system message and documents */ -}} {{- range .Messages }} {{- /* User defined system prompt(s) */}} {{- if (eq .Role "system")}} {{- if (ne $system "") }} {{- $system = print $system "\n\n" }} {{- end}} {{- $system = print $system .Content }} {{- end}} {{- /* Role specified controls */ -}} {{- if (and (ge (len .Role) 7) (eq (slice .Role 0 7) "control")) }} {{- if (eq .Content "thinking")}}{{- $thinking = true }}{{- end}} {{- if (eq .Content "citations")}}{{- $citations = true }}{{- end}} {{- if (eq .Content "hallucinations")}}{{- $hallucinations = true }}{{- end}} {{- if (and (ge (len .Content) 7) (eq (slice .Content 0 7) "length "))}} {{- $length = slice .Content 7 }} {{- end}} {{- if (and (ge (len .Content) 12) (eq (slice .Content 0 12) "originality "))}} {{- $originality = slice .Content 12 }} {{- end}} {{- end}} {{- /* Role specified document */ -}} {{- if (and (ge (len .Role) 8) (eq (slice .Role 0 8) "document")) }} {{- if (ne $documentCounter 0)}} {{- $documents = print $documents "\n\n"}} {{- end}} {{- $identifier := ""}} {{- if (ge (len .Role) 9) }} {{- $identifier = (slice .Role 9)}} {{- end}} {{- if (eq $identifier "") }} {{- $identifier := print $documentCounter}} {{- end}} {{- $documents = print $documents "<|start_of_role|>document {\"document_id\": \"" $identifier "\"}<|end_of_role|>\n" .Content "<|end_of_text|>"}} {{- $documentCounter = len (printf "a%*s" $documentCounter "")}} {{- end}} {{- end}} {{- /* If no user message provided, build the default system message */ -}} {{- if eq $system "" }} {{- $system = "Knowledge Cutoff Date: April 2024.\nYou are Granite, developed by IBM."}} {{- /* !!! MOVED THINKING BLOCK HERE !!! */}} {{- /* Add thinking instructions if requested, regardless of tools/docs */}} {{- if $thinking}} {{- $system = print $system "\nRespond to every user query in a comprehensive and detailed way. You can write down your thoughts and reasoning process before responding. In the thought process, engage in a comprehensive cycle of analysis, summarization, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. In the response section, based on various attempts, explorations, and reflections from the thoughts section, systematically present the final solution that you deem correct. The response should summarize the thought process. Write your thoughts between <think></think> and write your response between <response></response> for each user query."}} {{- end}} {{- /* !!! END OF MOVED BLOCK !!! */}} {{- /* Add Tools prompt */}} {{- if .Tools }} {{- $system = print $system " You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request." }} {{- end}} {{- /* Add documents prompt */}} {{- if $documents }} {{- if .Tools }} {{- $system = print $system "\n"}} {{- else }} {{- /* Add space only if no tools were mentioned */}} {{- $system = print $system " "}} {{- end}} {{- $system = print $system "Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data." }} {{- if $citations}} {{- $system = print $system "\nUse the symbols <|start_of_cite|> and <|end_of_cite|> to indicate when a fact comes from a document in the search result, e.g <|start_of_cite|> {document_id: 1}my fact <|end_of_cite|> for a fact from document 1. Afterwards, list all the citations with their corresponding documents in an ordered list."}} {{- end}} {{- if $hallucinations}} {{- $system = print $system "\nFinally, after the response is written, include a numbered list of sentences from the response with a corresponding risk value that are hallucinated and not based in the documents."}} {{- end}} {{- end}} {{- /* Prompt ONLY when no tools or documents were present */}} {{- /* We removed the thinking part from here */}} {{- if (and (not .Tools) (not $documents)) }} {{- $system = print $system " You are a helpful AI assistant."}} {{- /* Original thinking block was here, now removed */}} {{- end}} {{- end}} {{- /* ------ TEMPLATE EXPANSION ------ */}} {{- /* System Prompt */ -}} <|start_of_role|>system<|end_of_role|>{{- $system }}<|end_of_text|> {{- /* Tools */ -}} {{- if .Tools }} <|start_of_role|>available_tools<|end_of_role|>[ {{- range $index, $_ := .Tools }} {{ . }} {{- if and (ne (len (slice $.Tools $index)) 1) (gt (len $.Tools) 1) }}, {{- end}} {{- end }} ]<|end_of_text|> {{- end}} {{- /* Documents */ -}} {{- if $documents }} {{ $documents }} {{- end}} {{- /* Standard Messages */}} {{- range $index, $_ := .Messages }} {{- if (and (ne .Role "system") (or (lt (len .Role) 7) (ne (slice .Role 0 7) "control")) (or (lt (len .Role) 8) (ne (slice .Role 0 8) "document")) )}} <|start_of_role|> {{- if eq .Role "tool" }}tool_response {{- else }}{{ .Role }} {{- end }}<|end_of_role|> {{- if .Content }}{{ .Content }} {{- else if .ToolCalls }}<|tool_call|> {{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{- end }} {{- end }} {{- if eq (len (slice $.Messages $index)) 1 }} {{- if eq .Role "assistant" }} {{- else }}<|end_of_text|> <|start_of_role|>assistant {{- if and (ne $length "") (ne $originality "") }} {"length": "{{ $length }}", "originality": "{{ $originality }}"} {{- else if ne $length "" }} {"length": "{{ $length }}"} {{- else if ne $originality "" }} {"originality": "{{ $originality }}"} {{- end }}<|end_of_role|> {{- end -}} {{- else }}<|end_of_text|> {{- end }} {{- end }} {{- end }}""" # --- Ollama Parameters (Copied from original) --- PARAMETER stop "<|endoftext|>" PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>" PARAMETER stop "<|start_of_role|>" PARAMETER stop "<|end_of_role|>" ``` Now, I have the same problem than with Granite3.2 and images with history : if I pass chat history, thinking is disabled ``` ## CHAT WITH TOOLS AND HISTORY {'model': 'granite3.3-think-tools:latest', 'created_at': '2025-04-18T15:17:16.952829421Z', 'message': {'role': 'assistant', 'content': 'I am designed to provide information and perform tasks based on a set of predefined functions. Currently, I can fetch the current weather of a city using an external weather API. Would you like to know the weather in a specific location?'}, 'done_reason': 'stop', 'done': True, 'total_duration': 665282668, 'load_duration': 40344608, 'prompt_eval_count': 338, 'prompt_eval_duration': 12273921, 'eval_count': 47, 'eval_duration': 607266017} I am designed to provide information and perform tasks based on a set of predefined functions. Currently, I can fetch the current weather of a city using an external weather API. Would you like to know the weather in a specific location? ## CHAT WITH TOOLS WITHOUT HISTORY {'model': 'granite3.3-think-tools:latest', 'created_at': '2025-04-18T15:17:21.51231505Z', 'message': {'role': 'assistant', 'content': "<think>The user is introducing themselves and asking about my capabilities. I should provide a general overview of my functionalities without specific task requests. \n\nI am an AI assistant designed to help answer questions, provide information, and perform tasks such as:\n- Fetching data from the internet (with access to specified functions).\n- Summarizing texts or providing explanations on various topics.\n- Assisting with scheduling or reminders.\n- Offering basic calculations or conversions.\n\nFor this interaction, the user has not asked for a specific task yet, so I should keep my response general.</think><response>Hello Bob! I'm an AI assistant designed to help answer your questions and perform various tasks. Here are some things I can do:\n\n1. **Provide Information**: I can give you facts on a wide range of topics from science and history to general knowledge queries.\n2. **Web Data Fetching**: With access to specific functions, I can retrieve real-time data like weather updates, sports scores, or stock prices. For instance, I can tell you the current weather in any city if you provide me with the name.\n3. **Text Summarization and Explanation**: If you have a lengthy text or complex concept, I can help summarize it or explain it in simpler terms.\n4. **Task Management**: I can assist with setting reminders or managing your schedule.\n5. **Basic Calculations and Conversions**: Need to convert units or do some quick math? I'm here to help with that too!\n\nPlease let me know how you'd like to proceed or if you have a specific question or task in mind.</response>"}, 'done_reason': 'stop', 'done': True, 'total_duration': 4557362810, 'load_duration': 8512238, 'prompt_eval_count': 307, 'prompt_eval_duration': 7425351, 'eval_count': 351, 'eval_duration': 4540118289} <think>The user is introducing themselves and asking about my capabilities. I should provide a general overview of my functionalities without specific task requests. I am an AI assistant designed to help answer questions, provide information, and perform tasks such as: - Fetching data from the internet (with access to specified functions). - Summarizing texts or providing explanations on various topics. - Assisting with scheduling or reminders. - Offering basic calculations or conversions. For this interaction, the user has not asked for a specific task yet, so I should keep my response general.</think><response>Hello Bob! I'm an AI assistant designed to help answer your questions and perform various tasks. Here are some things I can do: 1. **Provide Information**: I can give you facts on a wide range of topics from science and history to general knowledge queries. 2. **Web Data Fetching**: With access to specific functions, I can retrieve real-time data like weather updates, sports scores, or stock prices. For instance, I can tell you the current weather in any city if you provide me with the name. 3. **Text Summarization and Explanation**: If you have a lengthy text or complex concept, I can help summarize it or explain it in simpler terms. 4. **Task Management**: I can assist with setting reminders or managing your schedule. 5. **Basic Calculations and Conversions**: Need to convert units or do some quick math? I'm here to help with that too! Please let me know how you'd like to proceed or if you have a specific question or task in mind.</response> ```
GiteaMirror commented 2026-05-04 15:24:23 -05:00
Author
Owner
Copy Link

@gabe-l-hart commented on GitHub (Apr 25, 2025):

Hi @lemassykoi thanks for bringing this issue up and for providing a clever option to work around it! As @rick-github pointed out, the chat template explicitly makes thinking mutually exclusive with both documents and tools. This was done by the model team because the tuning data for the different features did not overlap and therefore combining features is "undefined behavior." We've discussed internally whether the right answer to this is to forbid it, or to document that it's untested and let the user try anyway. I've considered making an updated version of the chat template (for both Ollama and transformers) that adds an extra control called unsafe which could be used to relax these restrictions. At the moment, however, the answer is that if you want to experiment with untested combinations of features, the onus falls on the client side to do the prompt composition.

<!-- gh-comment-id:2831521878 --> @gabe-l-hart commented on GitHub (Apr 25, 2025): Hi @lemassykoi thanks for bringing this issue up _and_ for providing a clever option to work around it! As @rick-github pointed out, the chat template explicitly makes `thinking` mutually exclusive with both `documents` and `tools`. This was done by the model team because the tuning data for the different features did not overlap and therefore combining features is "undefined behavior." We've discussed internally whether the right answer to this is to forbid it, or to document that it's untested and let the user try anyway. I've considered making an updated version of the chat template (for both Ollama and `transformers`) that adds an extra `control` called `unsafe` which could be used to relax these restrictions. At the moment, however, the answer is that if you want to experiment with untested combinations of features, the onus falls on the client side to do the prompt composition.
GiteaMirror commented 2026-05-04 15:24:24 -05:00
Author
Owner
Copy Link

@lemassykoi commented on GitHub (Aug 25, 2025):

I'm unhappy with ollama, performances, issues, staibility,

I then switched to LM Studio, and I got absolutely no errors with granite-3.3-8b-instruct

<!-- gh-comment-id:3219474303 --> @lemassykoi commented on GitHub (Aug 25, 2025): I'm unhappy with ollama, performances, issues, staibility, I then switched to LM Studio, and I got absolutely no errors with granite-3.3-8b-instruct
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#68848
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?