[PR #5915] [CLOSED] added tools when stream=true #58659

Closed
opened 2026-04-29 13:33:49 -05:00 by GiteaMirror · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/5915
Author: @vertrue
Created: 7/24/2024
Status: Closed

Base: mainHead: vertrue/fix-tools-stop-reason


📝 Commits (6)

  • 3134eba Add 'finish_reason': 'tool_calls' when tools are called
  • 20b7a56 Add 'finish_reason': 'tool_calls' when tools are called
  • 174e6c2 Merge branch 'main' into vertrue/fix-tools-stop-reason
  • 049eac2 Add ToolCalls and related FinishReason to the last chuck
  • a40ee23 Clean up debug
  • 3599753 Clean up debug

📊 Changes

2 files changed (+34 additions, -2 deletions)

View changed files

📝 openai/openai.go (+23 -2)
📝 server/routes.go (+11 -0)

📄 Description

fixes #5796

curl --location '127.0.0.1:11434/v1/chat/completions' \
--header 'Content-Type: application/json' \
--data '{
  "messages": [
    {
      "content": "You are a helpful AI assistant that can use tools.",
      "role": "system"
    },
    {
      "content": "What is function_1(10, 11)? use provided tools",
      "role": "user"
    }
  ],
  "model": "llama3-groq-tool-use:8b",
  "logprobs": false,
  "n": 1,
  "stream": true,
  "temperature": 0,
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "function_1",
        "description": "uses function function_1 for arguments a and b.",
        "parameters": {
          "type": "object",
          "properties": {
            "a": {
              "type": "integer"
            },
            "b": {
              "type": "integer"
            }
          },
          "required": [
            "a",
            "b"
          ]
        }
      }
    }
  ]
}'

chunks:

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\u003ctool_call\u003e"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"{\""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"id"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"0"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"name"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"function"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"_"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"1"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\","},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"arguments"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" {\""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"a"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"10"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"b"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"11"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"}}\n"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\u003c/tool_call\u003e"},"finish_reason":null}]}

data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":[{"id":"call_ejieksiz","type":"function","function":{"name":"function_1","arguments":"{\"a\":10,\"b\":11}"}}]},"finish_reason":"tool_calls"}]}

data: [DONE]

last chunk:

{
  "id": "chatcmpl-20",
  "object": "chat.completion.chunk",
  "created": 1721830177,
  "model": "llama3-groq-tool-use:8b",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "",
        "tool_calls": [
          {
            "id": "call_ejieksiz",
            "type": "function",
            "function": {
              "name": "function_1",
              "arguments": "{\"a\":10,\"b\":11}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

also works with /api/chat
also added "finish_reason": "tool_calls" for completions


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/5915 **Author:** [@vertrue](https://github.com/vertrue) **Created:** 7/24/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `vertrue/fix-tools-stop-reason` --- ### 📝 Commits (6) - [`3134eba`](https://github.com/ollama/ollama/commit/3134eba7590de55421d015a943bc515f573d98ab) Add 'finish_reason': 'tool_calls' when tools are called - [`20b7a56`](https://github.com/ollama/ollama/commit/20b7a564384830804e27ab3673244e7ddc4f7b61) Add 'finish_reason': 'tool_calls' when tools are called - [`174e6c2`](https://github.com/ollama/ollama/commit/174e6c2075041d46d7793aa0c99bd19a3cd3a2d2) Merge branch 'main' into vertrue/fix-tools-stop-reason - [`049eac2`](https://github.com/ollama/ollama/commit/049eac2f99d6a4065286c807a3efb52aa848c776) Add ToolCalls and related FinishReason to the last chuck - [`a40ee23`](https://github.com/ollama/ollama/commit/a40ee2375850f0ebdabd5d36e09a2b9b78705332) Clean up debug - [`3599753`](https://github.com/ollama/ollama/commit/359975345d7794cfef0016ad97ec8a81b41aab3e) Clean up debug ### 📊 Changes **2 files changed** (+34 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `openai/openai.go` (+23 -2) 📝 `server/routes.go` (+11 -0) </details> ### 📄 Description fixes #5796 ``` curl --location '127.0.0.1:11434/v1/chat/completions' \ --header 'Content-Type: application/json' \ --data '{ "messages": [ { "content": "You are a helpful AI assistant that can use tools.", "role": "system" }, { "content": "What is function_1(10, 11)? use provided tools", "role": "user" } ], "model": "llama3-groq-tool-use:8b", "logprobs": false, "n": 1, "stream": true, "temperature": 0, "tools": [ { "type": "function", "function": { "name": "function_1", "description": "uses function function_1 for arguments a and b.", "parameters": { "type": "object", "properties": { "a": { "type": "integer" }, "b": { "type": "integer" } }, "required": [ "a", "b" ] } } } ] }' ``` chunks: ``` data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\u003ctool_call\u003e"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\n"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"{\""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"id"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"0"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"name"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"function"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"_"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"1"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\","},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"arguments"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" {\""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"a"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830176,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"10"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" \""},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"b"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\":"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":" "},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"11"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"}}\n"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"\u003c/tool_call\u003e"},"finish_reason":null}]} data: {"id":"chatcmpl-20","object":"chat.completion.chunk","created":1721830177,"model":"llama3-groq-tool-use:8b","system_fingerprint":"fp_ollama","choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":[{"id":"call_ejieksiz","type":"function","function":{"name":"function_1","arguments":"{\"a\":10,\"b\":11}"}}]},"finish_reason":"tool_calls"}]} data: [DONE] ``` last chunk: ``` { "id": "chatcmpl-20", "object": "chat.completion.chunk", "created": 1721830177, "model": "llama3-groq-tool-use:8b", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "delta": { "role": "assistant", "content": "", "tool_calls": [ { "id": "call_ejieksiz", "type": "function", "function": { "name": "function_1", "arguments": "{\"a\":10,\"b\":11}" } } ] }, "finish_reason": "tool_calls" } ] } ``` also works with `/api/chat` also added `"finish_reason": "tool_calls"` for completions --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-29 13:33:49 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#58659