github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-18 06:06:18 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5081] Timeout for long generation #65242

New Issue
Closed
opened 2026-05-03 20:08:06 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-05-03 20:08:06 -05:00
Owner
Copy Link

Originally created by @slavonnet on GitHub (Jun 16, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5081

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

https://github.com/ollama/ollama-js/issues/103

After 5 minutes of execution, 500 error is received. There is no way to increase the time. I use stream: false.
keepalive: 15m and env OLLAMA_KEEP_ALIVE=15m does not help

июн 16 17:43:34 ai ollama[3733311]: [GIN] 2024/06/16 - 17:43:34 | 500 |          5m1s |       127.0.0.1 | POST     "/api/chat"

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.1.45-rc1

Originally created by @slavonnet on GitHub (Jun 16, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5081 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? https://github.com/ollama/ollama-js/issues/103 After 5 minutes of execution, 500 error is received. There is no way to increase the time. I use `stream: false`. `keepalive: 15m` and env `OLLAMA_KEEP_ALIVE=15m` does not help ``` июн 16 17:43:34 ai ollama[3733311]: [GIN] 2024/06/16 - 17:43:34 | 500 | 5m1s | 127.0.0.1 | POST "/api/chat" ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.1.45-rc1
GiteaMirror added the bug label 2026-05-03 20:08:07 -05:00
GiteaMirror commented 2026-05-03 20:08:20 -05:00
Author
Owner
Copy Link

@slavonnet commented on GitHub (Jun 16, 2024):

i found parametr in llama.cpp

-to N, --timeout N: Server read/write timeout in seconds. Default 600
<!-- gh-comment-id:2171738014 --> @slavonnet commented on GitHub (Jun 16, 2024): i found parametr in llama.cpp ``` -to N, --timeout N: Server read/write timeout in seconds. Default 600 ```
GiteaMirror commented 2026-05-03 20:08:25 -05:00
Author
Owner
Copy Link

@hopperelec commented on GitHub (Jun 16, 2024):

600 seconds is 10 minutes, though, and I don't see anywhere where Ollama is overriding this

<!-- gh-comment-id:2171739999 --> @hopperelec commented on GitHub (Jun 16, 2024): 600 seconds is 10 minutes, though, and I don't see anywhere where Ollama is overriding this
GiteaMirror commented 2026-05-03 20:08:28 -05:00
Author
Owner
Copy Link

@slavonnet commented on GitHub (Jun 16, 2024):

OK, what are the options to fix the 5 minute limit?

<!-- gh-comment-id:2171741650 --> @slavonnet commented on GitHub (Jun 16, 2024): OK, what are the options to fix the 5 minute limit?
GiteaMirror commented 2026-05-03 20:08:31 -05:00
Author
Owner
Copy Link

@slavonnet commented on GitHub (Jun 16, 2024):

i have 3 x 4060ti 16gb and try run Mixtral 8x22b model (half run on gpu and half on cpu)

<!-- gh-comment-id:2171741995 --> @slavonnet commented on GitHub (Jun 16, 2024): i have 3 x 4060ti 16gb and try run Mixtral 8x22b model (half run on gpu and half on cpu)
GiteaMirror commented 2026-05-03 20:08:32 -05:00
Author
Owner
Copy Link

@slavonnet commented on GitHub (Jun 16, 2024):

89c79bec8c/llm/ext_server/httplib.h (L26)

<!-- gh-comment-id:2171744052 --> @slavonnet commented on GitHub (Jun 16, 2024): https://github.com/ollama/ollama/blob/89c79bec8cf7a7b04f761fcc5306d2edf47a4164/llm/ext_server/httplib.h#L26
GiteaMirror commented 2026-05-03 20:08:33 -05:00
Author
Owner
Copy Link

@slavonnet commented on GitHub (Jun 16, 2024):

https://github.com/ollama/ollama/pull/5084

<!-- gh-comment-id:2171746658 --> @slavonnet commented on GitHub (Jun 16, 2024): https://github.com/ollama/ollama/pull/5084
GiteaMirror commented 2026-05-03 20:08:34 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Nov 6, 2024):

This was most likely a load timeout. We added a new env var OLLAMA_LOAD_TIMEOUT to adjust this timeout for systems that take longer than 5m to load the model.

<!-- gh-comment-id:2458513769 --> @dhiltgen commented on GitHub (Nov 6, 2024): This was most likely a load timeout. We added a new env var `OLLAMA_LOAD_TIMEOUT` to adjust this timeout for systems that take longer than 5m to load the model.
GiteaMirror commented 2026-05-03 20:08:35 -05:00
Author
Owner
Copy Link

@SegFaulty commented on GitHub (Feb 24, 2025):

After investigating on this isseue, here a maybe helpfully comment:

If you run bigger models on consumer hardware, you will instant discover the hardcoded 5 Minute timeout for non streaming requests.

Our Solution to handle long running (like 10 minutes) requests:

  • stream=true,
  • collect all parts in the low level api request method (curl has a callback option (writefunction) to concate it relative easy)
  • return it like a one shot request response.
<!-- gh-comment-id:2678021167 --> @SegFaulty commented on GitHub (Feb 24, 2025): After investigating on this isseue, here a maybe helpfully comment: If you run bigger models on consumer hardware, you will instant discover the hardcoded 5 Minute timeout for non streaming requests. Our Solution to handle long running (like 10 minutes) requests: - stream=true, - collect all parts in the low level api request method (curl has a callback option (writefunction) to concate it relative easy) - return it like a one shot request response.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#65242
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?