github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 06:03:26 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #12125] issue: Improve compatibility with OpenAI structured outputs json_schema response format #87217

New Issue
Closed
opened 2026-05-15 11:36:30 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-15 11:36:30 -05:00
Owner
Copy Link

Originally created by @SuperPat45 on GitHub (Mar 27, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12125

Check Existing Issues

  • I have searched the existing issues and discussions.
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.5.20

Ollama Version (if applicable)

0.6.2

Operating System

ubuntu 22.04

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have listed steps to reproduce the bug in detail.

Expected Behavior

What is the issue?

Can you improve the /chat/completions compatibility with OpenAI about the json_schema response_format?

I tested with the gemma3 model, and this OpenAI syntax is ignored if called through the /api/chat/completions OpenWebUI API whereas it works through the /ollama/v1/chat/completions API:

    "response_format": {,
        "type": "json_schema",
        "json_schema": {
            "name": "result",
            "schema": {
                "additionalProperties": false,,
                "type": "object",
                "required": [
                    "parm1"
                ],
                "properties": {
                    "parm1": {
....

Whereas this one work in all case:

            "format": {
                "type": "object",
                "required": [
                    "parm1"
                ],
                "properties": {
                    "parm1": {
....

It seems the response_format parameter is dropped before relaying to Ollama.

Actual Behavior

The response is not a json string unsing response_format, ok with format parameter.

Steps to Reproduce

Calling with /api/chat/completions, is not working:

curl -X 'POST' \
  'http:///XXX.XXX.XXX.XXX:3000/api/chat/completions' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer XXXXXXXXXXXXXX' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma3:12b",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful math tutor. Guide the user through the solution step by step."
      },
      {
        "role": "user",
        "content": "how can I solve 8x + 7 = -23"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "math_reasoning",
        "schema": {
          "type": "object",
          "properties": {
            "steps": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "explanation": { "type": "string" },
                  "output": { "type": "string" }
                },
                "required": ["explanation", "output"],
                "additionalProperties": false
              }
            },
            "final_answer": { "type": "string" }
          },
          "required": ["steps", "final_answer"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
}'

{
  "id": "gemma3:12b-6283b8f8-9dfa-463e-bb7a-6eef7d5a46bd",
  "created": 1743083500,
  "model": "gemma3:12b",
  "choices": [
    {
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop",
      "message": {
        "content": "Okay, let's tackle this equation together! Here's how to solve 8x + 7 = -23, breaking it down step-by-step:\n\n**Understanding the Goal**\n\nOur goal is to isolate 'x' on one side of the equation.  This means we want to get 'x' by itself. To do this, we'll use inverse operations.\n\n**Step 1: Isolate the term with 'x'**\n\n*   Notice that we have \"+ 7\" on the left side of the equation, next to the \"8x\". To undo the \"+ 7\", we need to subtract 7 from *both* sides of the equation.  This keeps the equation balanced.\n\n   8x + 7 - 7 = -23 - 7\n\n*   Simplify both sides:\n\n   8x = -30\n\n**Step 2: Solve for 'x'**\n\n*   Now we have 8x = -30. This means \"8 times x equals -30.\" To undo the multiplication by 8, we need to divide *both* sides of the equation by 8.\n\n   8x / 8 = -30 / 8\n\n*   Simplify both sides:\n\n    x = -30/8\n\n**Step 3: Simplify the fraction (optional)**\n\n*   The fraction -30/8 can be simplified. Both 30 and 8 are divisible by 2.\n\n   x = -15/4\n\n**Answer:**\n\nTherefore, x = -15/4.\n\n**Let's Recap**\n\n1.  **Subtract 7 from both sides:** This removed the \"+ 7\" from the left side.\n2.  **Divide both sides by 8:** This isolated 'x'.\n3.  **Simplify:** This gave us the final solution.\n\n**Do you want to try another example?**",
        "role": "assistant"
      }
    }
  ],
  "object": "chat.completion",
  "usage": {
    "response_token/s": 91.48,
    "prompt_token/s": 2516.11,
    "total_duration": 4532196164,
    "load_duration": 21152636,
    "prompt_eval_count": 45,
    "prompt_tokens": 45,
    "prompt_eval_duration": 17884726,
    "eval_count": 411,
    "completion_tokens": 411,
    "eval_duration": 4492640344,
    "approximate_total": "0h0m4s",
    "total_tokens": 456,
    "completion_tokens_details": {
      "reasoning_tokens": 0,
      "accepted_prediction_tokens": 0,
      "rejected_prediction_tokens": 0
    }
  }
}

Whereas calling with /ollama/v1/chat/completions, is ok:

curl -X 'POST' \
  'http://XXX.XXX.XXX.XXX:3000/ollama/v1/chat/completions' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer XXXXXXXXXXXXXX' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gemma3:12b",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful math tutor. Guide the user through the solution step by step."
      },
      {
        "role": "user",
        "content": "how can I solve 8x + 7 = -23"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "math_reasoning",
        "schema": {
          "type": "object",
          "properties": {
            "steps": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "explanation": { "type": "string" },
                  "output": { "type": "string" }
                },
                "required": ["explanation", "output"],
                "additionalProperties": false
              }
            },
            "final_answer": { "type": "string" }
          },
          "required": ["steps", "final_answer"],
          "additionalProperties": false
        },
        "strict": true
      }
    }
}'

{
  "id": "chatcmpl-154",
  "object": "chat.completion",
  "created": 1743083257,
  "model": "gemma3:12b",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n\"steps\": [\n{\"explanation\": \"Our goal is to isolate 'x' on one side of the equation. The first step is to get rid of the '+ 7' that's with the '8x' term. To do this, we need to subtract 7 from both sides of the equation. This keeps the equation balanced.\", \"output\": \"-7 from both sides: 8x + 7 - 7 = -23 - 7\"},\n{\"explanation\": \"Now let's simplify both sides of the equation.\", \"output\": \"Simplify: 8x = -30\"},\n{\"explanation\": \"Now we have 8x = -30. To isolate 'x', we need to divide both sides of the equation by the coefficient of x, which is 8.\", \"output\": \"Divide both sides by 8: 8x / 8 = -30 / 8\"},\n{\"explanation\": \"Now, simplify both sides again.\", \"output\": \"Simplify: x = -30/8\"},\n{\"explanation\": \"The fraction -30/8 can be simplified by dividing both the numerator and denominator by their greatest common divisor, which is 2.\", \"output\": \"Simplify the fraction: x = -15/4\"},\n{\"explanation\": \"So, the solution is x = -15/4.\", \"output\": \"Final Answer: The final answer is x = -15/4\"}\n]\n, \"final_answer\": \"x = -15/4\"\n}\n"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 45,
    "completion_tokens": 329,
    "total_tokens": 374
  }
}

Logs & Screenshots

n/a

Additional Information

No response

Originally created by @SuperPat45 on GitHub (Mar 27, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/12125 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.5.20 ### Ollama Version (if applicable) 0.6.2 ### Operating System ubuntu 22.04 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior ### What is the issue? Can you improve the /chat/completions compatibility with OpenAI about the json_schema response_format? I tested with the gemma3 model, and this OpenAI syntax is ignored if called through the /api/chat/completions OpenWebUI API whereas it works through the /ollama/v1/chat/completions API: ``` "response_format": {, "type": "json_schema", "json_schema": { "name": "result", "schema": { "additionalProperties": false,, "type": "object", "required": [ "parm1" ], "properties": { "parm1": { .... ``` Whereas this one work in all case: ``` "format": { "type": "object", "required": [ "parm1" ], "properties": { "parm1": { .... ``` It seems the response_format parameter is dropped before relaying to Ollama. ### Actual Behavior The response is not a json string unsing response_format, ok with format parameter. ### Steps to Reproduce Calling with /api/chat/completions, is not working: ``` curl -X 'POST' \ 'http:///XXX.XXX.XXX.XXX:3000/api/chat/completions' \ -H 'accept: application/json' \ -H 'Authorization: Bearer XXXXXXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "model": "gemma3:12b", "messages": [ { "role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step." }, { "role": "user", "content": "how can I solve 8x + 7 = -23" } ], "response_format": { "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": false } }, "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": false }, "strict": true } } }' ``` ``` { "id": "gemma3:12b-6283b8f8-9dfa-463e-bb7a-6eef7d5a46bd", "created": 1743083500, "model": "gemma3:12b", "choices": [ { "index": 0, "logprobs": null, "finish_reason": "stop", "message": { "content": "Okay, let's tackle this equation together! Here's how to solve 8x + 7 = -23, breaking it down step-by-step:\n\n**Understanding the Goal**\n\nOur goal is to isolate 'x' on one side of the equation. This means we want to get 'x' by itself. To do this, we'll use inverse operations.\n\n**Step 1: Isolate the term with 'x'**\n\n* Notice that we have \"+ 7\" on the left side of the equation, next to the \"8x\". To undo the \"+ 7\", we need to subtract 7 from *both* sides of the equation. This keeps the equation balanced.\n\n 8x + 7 - 7 = -23 - 7\n\n* Simplify both sides:\n\n 8x = -30\n\n**Step 2: Solve for 'x'**\n\n* Now we have 8x = -30. This means \"8 times x equals -30.\" To undo the multiplication by 8, we need to divide *both* sides of the equation by 8.\n\n 8x / 8 = -30 / 8\n\n* Simplify both sides:\n\n x = -30/8\n\n**Step 3: Simplify the fraction (optional)**\n\n* The fraction -30/8 can be simplified. Both 30 and 8 are divisible by 2.\n\n x = -15/4\n\n**Answer:**\n\nTherefore, x = -15/4.\n\n**Let's Recap**\n\n1. **Subtract 7 from both sides:** This removed the \"+ 7\" from the left side.\n2. **Divide both sides by 8:** This isolated 'x'.\n3. **Simplify:** This gave us the final solution.\n\n**Do you want to try another example?**", "role": "assistant" } } ], "object": "chat.completion", "usage": { "response_token/s": 91.48, "prompt_token/s": 2516.11, "total_duration": 4532196164, "load_duration": 21152636, "prompt_eval_count": 45, "prompt_tokens": 45, "prompt_eval_duration": 17884726, "eval_count": 411, "completion_tokens": 411, "eval_duration": 4492640344, "approximate_total": "0h0m4s", "total_tokens": 456, "completion_tokens_details": { "reasoning_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } } } ``` Whereas calling with /ollama/v1/chat/completions, is ok: ``` curl -X 'POST' \ 'http://XXX.XXX.XXX.XXX:3000/ollama/v1/chat/completions' \ -H 'accept: application/json' \ -H 'Authorization: Bearer XXXXXXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "model": "gemma3:12b", "messages": [ { "role": "system", "content": "You are a helpful math tutor. Guide the user through the solution step by step." }, { "role": "user", "content": "how can I solve 8x + 7 = -23" } ], "response_format": { "type": "json_schema", "json_schema": { "name": "math_reasoning", "schema": { "type": "object", "properties": { "steps": { "type": "array", "items": { "type": "object", "properties": { "explanation": { "type": "string" }, "output": { "type": "string" } }, "required": ["explanation", "output"], "additionalProperties": false } }, "final_answer": { "type": "string" } }, "required": ["steps", "final_answer"], "additionalProperties": false }, "strict": true } } }' ``` ``` { "id": "chatcmpl-154", "object": "chat.completion", "created": 1743083257, "model": "gemma3:12b", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "{\n\"steps\": [\n{\"explanation\": \"Our goal is to isolate 'x' on one side of the equation. The first step is to get rid of the '+ 7' that's with the '8x' term. To do this, we need to subtract 7 from both sides of the equation. This keeps the equation balanced.\", \"output\": \"-7 from both sides: 8x + 7 - 7 = -23 - 7\"},\n{\"explanation\": \"Now let's simplify both sides of the equation.\", \"output\": \"Simplify: 8x = -30\"},\n{\"explanation\": \"Now we have 8x = -30. To isolate 'x', we need to divide both sides of the equation by the coefficient of x, which is 8.\", \"output\": \"Divide both sides by 8: 8x / 8 = -30 / 8\"},\n{\"explanation\": \"Now, simplify both sides again.\", \"output\": \"Simplify: x = -30/8\"},\n{\"explanation\": \"The fraction -30/8 can be simplified by dividing both the numerator and denominator by their greatest common divisor, which is 2.\", \"output\": \"Simplify the fraction: x = -15/4\"},\n{\"explanation\": \"So, the solution is x = -15/4.\", \"output\": \"Final Answer: The final answer is x = -15/4\"}\n]\n, \"final_answer\": \"x = -15/4\"\n}\n" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 45, "completion_tokens": 329, "total_tokens": 374 } } ``` ### Logs & Screenshots n/a ### Additional Information _No response_
GiteaMirror added the bug label 2026-05-15 11:36:30 -05:00
GiteaMirror commented 2026-05-15 11:36:33 -05:00
Author
Owner
Copy Link

@tjbck commented on GitHub (Mar 28, 2025):

MIght be addressed with 5e720cd84b, testing wanted here.

<!-- gh-comment-id:2760014502 --> @tjbck commented on GitHub (Mar 28, 2025): MIght be addressed with 5e720cd84b4886594eac1ef8ac25cb61a4def85d, testing wanted here.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#87217
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?