github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-11-14 05:02:01 -06:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

bug: Streamed responses from OpenAI external source fall behind, then suddenly catch up when stream ends #248

New Issue
Closed
opened 2025-11-11 14:13:36 -06:00 by GiteaMirror · 3 comments
GiteaMirror commented 2025-11-11 14:13:36 -06:00
Owner
Copy Link
Copy Source

Originally created by @bjj on GitHub (Feb 1, 2024).

Bug Report

Description

Bug Summary:

When using a model via the external OpenAI connection, the text streams into the webui a bit slower than the model is streaming. Then, when the input stream is done, the last chunk of the response appears suddenly.

Steps to Reproduce:

Choose an OpenAI compatible external server. One widely available one is OpenRouter.

Choose a model that will stream relatively slowly (e.g. 10t/s).

Use a prompt with a long answer.

Expected Behavior:

The webui streams the response at the same speed as the source, ending at the same time as the source.

Actual Behavior:

The webui streams slightly slower than the source, dumping the backlog suddenly when the OpenAI connection closes.

Environment

  • Windows (ollama webui in docker)
  • Edge 121.0.2277.83

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I have reviewed the troubleshooting.md document.
  • I have included the browser console logs.
  • [x ] I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:

00:20:04.990  initNewChat
00:20:04.991  
00:20:04.991  Object
00:20:40.496  submitPrompt 
00:20:40.526  Array(0)
00:20:40.526  goliath
00:20:41.874  data: {"id":"exllamav2-1","choices":[{"delta":{"content":"Tom","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:41.874  Object
00:20:42.022  data: {"id":"exllamav2-1","choices":[{"delta":{"content":"atoes","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:42.022  Object
00:20:42.285  data: {"id":"exllamav2-1","choices":[{"delta":{"content":": Ju","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:42.285  Object
00:20:42.557  data: {"id":"exllamav2-1","choices":[{"delta":{"content":"icy,","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:42.557  Object
00:20:42.826  data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Nut","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:42.826  Object
00:20:43.089  data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ritious","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:43.089  Object
00:20:43.335  data: {"id":"exllamav2-1","choices":[{"delta":{"content":", and","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:20:43.335  Object
00:20:43.597  data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Versatile","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
...
00:21:35.284 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" add a","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:35.285 3.887d9769.js:7 Object
00:21:35.562 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" splash","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:35.562 3.887d9769.js:7 Object
00:21:35.836 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" of color","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:35.836 3.887d9769.js:7 Object
00:21:36.114 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" and tang","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:36.114 3.887d9769.js:7 Object
00:21:36.390 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"y flav","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:36.390 3.887d9769.js:7 Object
00:21:36.669 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"or.","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:36.669 3.887d9769.js:7 Object
00:21:36.949 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Blend","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:36.949 3.887d9769.js:7 Object
00:21:37.230 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" them","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:37.230 3.887d9769.js:7 Object
00:21:37.523 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" up,","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:37.523 3.887d9769.js:7 Object
00:21:37.797 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" and","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:37.797 3.887d9769.js:7 Object

// note timing becomes much tighter here>

00:21:38.113 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" you get","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.113 3.887d9769.js:7 Object
00:21:38.116 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" a del","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.116 3.887d9769.js:7 Object
00:21:38.117 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"icious glass","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.117 3.887d9769.js:7 Object
00:21:38.118 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" of Blo","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.118 3.887d9769.js:7 Object
00:21:38.119 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ody Mary","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.119 3.887d9769.js:7 Object
00:21:38.120 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" or gaz","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.120 3.887d9769.js:7 Object
00:21:38.121 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"pach","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.121 3.887d9769.js:7 Object
00:21:38.122 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"o soup","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.122 3.887d9769.js:7 Object
00:21:38.123 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":". Tom","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.123 3.887d9769.js:7 Object
00:21:38.123 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ato sau","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.123 3.887d9769.js:7 Object
00:21:38.124 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ce is","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.124 3.887d9769.js:7 Object
...
00:21:38.171 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" all the","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.171 3.887d9769.js:7 Object
00:21:38.171 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" better","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.171 3.887d9769.js:7 Object
00:21:38.172 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" for it","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.172 3.887d9769.js:7 Objectchoices: Array(1)0: delta: {content: ' for it', tool_calls: null, role: 'assistant'}finish_reason: nullindex: 1logprobs: null[[Prototype]]: Objectlength: 1[[Prototype]]: Array(0)created: 1706775639id: "exllamav2-1"model: "goliath"object: "chat.completion.chunk"system_fingerprint: "exllamav2"[[Prototype]]: Object
00:21:38.172 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":".","tool_calls":null,"role":"assistant"},"finish_reason":"stop","index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"}
00:21:38.172 3.887d9769.js:7 Objectchoices: Array(1)0: delta: {content: '.', tool_calls: null, role: 'assistant'}finish_reason: "stop"index: 1logprobs: null[[Prototype]]: Objectlength: 1[[Prototype]]: Array(0)created: 1706775639id: "exllamav2-1"model: "goliath"object: "chat.completion.chunk"system_fingerprint: "exllamav2"[[Prototype]]: Object
00:21:38.173 3.887d9769.js:7 data: [DONE]

Docker Container Logs:

2024-02-01 00:20:40 INFO:     172.17.0.1:47200 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
2024-02-01 00:20:40 INFO:     172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
2024-02-01 00:20:41 http://192.168.1.155:8001/v1/chat/completions none
2024-02-01 00:20:41 INFO:     172.17.0.1:47200 - "POST /openai/api/chat/completions HTTP/1.1" 200 OK
2024-02-01 00:21:38 INFO:     172.17.0.1:47200 - "POST /api/v1/chats/0e6c8876-fa4c-4884-8b86-96f20eb9835e HTTP/1.1" 200 OK
2024-02-01 00:21:38 INFO:     172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
2024-02-01 00:21:38 INFO:     172.17.0.1:47200 - "POST /api/v1/chats/0e6c8876-fa4c-4884-8b86-96f20eb9835e HTTP/1.1" 200 OK
2024-02-01 00:21:38 INFO:     172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
2024-02-01 00:21:38 INFO:     172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK

Screenshots (if applicable):
Skip forward to about 57 seconds:

https://github.com/ollama-webui/ollama-webui/assets/62541/6956b482-72ab-4da4-b493-bdb39ca80358

Installation Method

Docker container on Windows

Additional Information

I've verified multiple ways that the output of the OpenAI endpoint is smooth for the entire time. It just streams for the same period of time, slightly faster, and when it ends, the webui suddenly catches up.

Originally created by @bjj on GitHub (Feb 1, 2024). # Bug Report ## Description **Bug Summary:** When using a model via the external OpenAI connection, the text streams into the webui a bit slower than the model is streaming. Then, when the input stream is done, the last chunk of the response appears suddenly. **Steps to Reproduce:** Choose an OpenAI compatible external server. One widely available one is OpenRouter. Choose a model that will stream relatively slowly (e.g. 10t/s). Use a prompt with a long answer. **Expected Behavior:** The webui streams the response at the same speed as the source, ending at the same time as the source. **Actual Behavior:** The webui streams slightly slower than the source, dumping the backlog suddenly when the OpenAI connection closes. ## Environment - Windows (ollama webui in docker) - Edge 121.0.2277.83 ## Reproduction Details **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I have reviewed the troubleshooting.md document. - [x] I have included the browser console logs. - [x ] I have included the Docker container logs. ## Logs and Screenshots **Browser Console Logs:** ``` 00:20:04.990 initNewChat 00:20:04.991 00:20:04.991 Object 00:20:40.496 submitPrompt 00:20:40.526 Array(0) 00:20:40.526 goliath 00:20:41.874 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"Tom","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:41.874 Object 00:20:42.022 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"atoes","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:42.022 Object 00:20:42.285 data: {"id":"exllamav2-1","choices":[{"delta":{"content":": Ju","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:42.285 Object 00:20:42.557 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"icy,","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:42.557 Object 00:20:42.826 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Nut","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:42.826 Object 00:20:43.089 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ritious","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:43.089 Object 00:20:43.335 data: {"id":"exllamav2-1","choices":[{"delta":{"content":", and","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:20:43.335 Object 00:20:43.597 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Versatile","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} ... 00:21:35.284 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" add a","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:35.285 3.887d9769.js:7 Object 00:21:35.562 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" splash","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:35.562 3.887d9769.js:7 Object 00:21:35.836 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" of color","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:35.836 3.887d9769.js:7 Object 00:21:36.114 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" and tang","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:36.114 3.887d9769.js:7 Object 00:21:36.390 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"y flav","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:36.390 3.887d9769.js:7 Object 00:21:36.669 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"or.","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:36.669 3.887d9769.js:7 Object 00:21:36.949 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" Blend","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:36.949 3.887d9769.js:7 Object 00:21:37.230 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" them","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:37.230 3.887d9769.js:7 Object 00:21:37.523 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" up,","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:37.523 3.887d9769.js:7 Object 00:21:37.797 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" and","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:37.797 3.887d9769.js:7 Object // note timing becomes much tighter here> 00:21:38.113 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" you get","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.113 3.887d9769.js:7 Object 00:21:38.116 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" a del","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.116 3.887d9769.js:7 Object 00:21:38.117 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"icious glass","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.117 3.887d9769.js:7 Object 00:21:38.118 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" of Blo","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.118 3.887d9769.js:7 Object 00:21:38.119 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ody Mary","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.119 3.887d9769.js:7 Object 00:21:38.120 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" or gaz","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.120 3.887d9769.js:7 Object 00:21:38.121 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"pach","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.121 3.887d9769.js:7 Object 00:21:38.122 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"o soup","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.122 3.887d9769.js:7 Object 00:21:38.123 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":". Tom","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.123 3.887d9769.js:7 Object 00:21:38.123 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ato sau","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.123 3.887d9769.js:7 Object 00:21:38.124 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":"ce is","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.124 3.887d9769.js:7 Object ... 00:21:38.171 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" all the","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.171 3.887d9769.js:7 Object 00:21:38.171 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" better","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.171 3.887d9769.js:7 Object 00:21:38.172 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":" for it","tool_calls":null,"role":"assistant"},"finish_reason":null,"index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.172 3.887d9769.js:7 Objectchoices: Array(1)0: delta: {content: ' for it', tool_calls: null, role: 'assistant'}finish_reason: nullindex: 1logprobs: null[[Prototype]]: Objectlength: 1[[Prototype]]: Array(0)created: 1706775639id: "exllamav2-1"model: "goliath"object: "chat.completion.chunk"system_fingerprint: "exllamav2"[[Prototype]]: Object 00:21:38.172 3.887d9769.js:7 data: {"id":"exllamav2-1","choices":[{"delta":{"content":".","tool_calls":null,"role":"assistant"},"finish_reason":"stop","index":1,"logprobs":null}],"created":1706775639,"model":"goliath","system_fingerprint":"exllamav2","object":"chat.completion.chunk"} 00:21:38.172 3.887d9769.js:7 Objectchoices: Array(1)0: delta: {content: '.', tool_calls: null, role: 'assistant'}finish_reason: "stop"index: 1logprobs: null[[Prototype]]: Objectlength: 1[[Prototype]]: Array(0)created: 1706775639id: "exllamav2-1"model: "goliath"object: "chat.completion.chunk"system_fingerprint: "exllamav2"[[Prototype]]: Object 00:21:38.173 3.887d9769.js:7 data: [DONE] ``` **Docker Container Logs:** ``` 2024-02-01 00:20:40 INFO: 172.17.0.1:47200 - "POST /api/v1/chats/new HTTP/1.1" 200 OK 2024-02-01 00:20:40 INFO: 172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK 2024-02-01 00:20:41 http://192.168.1.155:8001/v1/chat/completions none 2024-02-01 00:20:41 INFO: 172.17.0.1:47200 - "POST /openai/api/chat/completions HTTP/1.1" 200 OK 2024-02-01 00:21:38 INFO: 172.17.0.1:47200 - "POST /api/v1/chats/0e6c8876-fa4c-4884-8b86-96f20eb9835e HTTP/1.1" 200 OK 2024-02-01 00:21:38 INFO: 172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK 2024-02-01 00:21:38 INFO: 172.17.0.1:47200 - "POST /api/v1/chats/0e6c8876-fa4c-4884-8b86-96f20eb9835e HTTP/1.1" 200 OK 2024-02-01 00:21:38 INFO: 172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK 2024-02-01 00:21:38 INFO: 172.17.0.1:47200 - "GET /api/v1/chats/ HTTP/1.1" 200 OK ``` **Screenshots (if applicable):** Skip forward to about 57 seconds: https://github.com/ollama-webui/ollama-webui/assets/62541/6956b482-72ab-4da4-b493-bdb39ca80358 ## Installation Method Docker container on Windows ## Additional Information I've verified multiple ways that the output of the OpenAI endpoint is smooth for the entire time. It just streams for the same period of time, slightly faster, and when it ends, the webui suddenly catches up.
GiteaMirror commented 2025-11-11 14:13:40 -06:00
Author
Owner
Copy Link
Copy Source

@tjbck commented on GitHub (Feb 1, 2024):

Hmm, is the webui being served over proxy of some sort? My guess is that internet speed/connection is the culprit here.

@tjbck commented on GitHub (Feb 1, 2024): Hmm, is the webui being served over proxy of some sort? My guess is that internet speed/connection is the culprit here.
GiteaMirror commented 2025-11-11 14:13:40 -06:00
Author
Owner
Copy Link
Copy Source

@justinh-rahb commented on GitHub (Feb 1, 2024):

Hmm, is the webui being served over proxy of some sort? My guess is that internet speed/connection is the culprit here.

Can confirm that if you put proxies in between ollama-webui and your ollama or other external APIs, you're going to notice things like this issue happening. It's kind of annoying but I don't believe there's much that could be done about it, certainly not from ollama-webui anyhow.

@justinh-rahb commented on GitHub (Feb 1, 2024): > Hmm, is the webui being served over proxy of some sort? My guess is that internet speed/connection is the culprit here. Can confirm that if you put proxies in between ollama-webui and your ollama or other external APIs, you're going to notice things like this issue happening. It's kind of annoying but I don't believe there's much that could be done about it, certainly not from ollama-webui anyhow.
GiteaMirror commented 2025-11-11 14:13:40 -06:00
Author
Owner
Copy Link
Copy Source

@bjj commented on GitHub (Feb 1, 2024):

Thanks. There's definitely a difference in behavior here between curl and ollama-webui, but I can work around it elsewhere. If I figure out what option is slightly different there I'll come back and add it to the issue.

@bjj commented on GitHub (Feb 1, 2024): Thanks. There's definitely a difference in behavior here between curl and ollama-webui, but I can work around it elsewhere. If I figure out what option is slightly different there I'll come back and add it to the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#248