github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #2176] Ollama instance stuck and hanging after few hours. #1244

New Issue
Closed
opened 2026-04-12 11:01:08 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 11:01:08 -05:00
Owner
Copy Link

Originally created by @jayouimet on GitHub (Jan 24, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2176

Originally assigned to: @dhiltgen on GitHub.

Hello,
We have an ollama instance that starts to hang after a few hours of use.
When using ctrl + c to stop the serve, we get a long stack trace resembling this, could be missing lines at the top as it is the maximum I can get from my ssh instance :

net/http/server.go:3086 +0x30 fp=0x140008e5fd0 sp=0x140008e5fa0 pc=0x104b90040
runtime.goexit()
        runtime/asm_arm64.s:1197 +0x4 fp=0x140008e5fd0 sp=0x140008e5fd0 pc=0x1049679f4
created by net/http.(*Server).Serve in goroutine 1
        net/http/server.go:3086 +0x4cc

goroutine 394 [sync.Mutex.Lock, 6 minutes]:
runtime.gopark(0x140008e2fc8?, 0x104953134?, 0xf0?, 0x88?, 0x140008e2fe8?)
        runtime/proc.go:398 +0xc8 fp=0x140008e2f90 sp=0x140008e2f70 pc=0x1049364e8
runtime.goparkunlock(...)
        runtime/proc.go:404
runtime.semacquire1(0x1055b2124, 0x7d?, 0x3, 0x1, 0x42?)
        runtime/sema.go:160 +0x208 fp=0x140008e2fe0 sp=0x140008e2f90 pc=0x104947b08
sync.runtime_SemacquireMutex(0x14000348450?, 0x0?, 0x0?)
        runtime/sema.go:77 +0x28 fp=0x140008e3020 sp=0x140008e2fe0 pc=0x104963248
sync.(*Mutex).lockSlow(0x1055b2120)
        sync/mutex.go:171 +0x174 fp=0x140008e3070 sp=0x140008e3020 pc=0x104972114
sync.(*Mutex).Lock(...)
        sync/mutex.go:90
github.com/jmorganca/ollama/server.GenerateHandler(0x140008fe200)
        github.com/jmorganca/ollama/server/routes.go:140 +0x90 fp=0x140008e3720 sp=0x140008e3070 pc=0x104e0ca60
github.com/gin-gonic/gin.(*Context).Next(...)
        github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0x140008fe200)
        github.com/jmorganca/ollama/server/routes.go:877 +0x78 fp=0x140008e3760 sp=0x140008e3720 pc=0x104e14dd8
github.com/gin-gonic/gin.(*Context).Next(...)
        github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0x140008fe200)
        github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x80 fp=0x140008e37b0 sp=0x140008e3760 pc=0x104df3900
github.com/gin-gonic/gin.(*Context).Next(...)
        github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.LoggerWithConfig.func1(0x140008fe200)
        github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xb0 fp=0x140008e3960 sp=0x140008e37b0 pc=0x104df2ca0
github.com/gin-gonic/gin.(*Context).Next(...)
        github.com/gin-gonic/gin@v1.9.1/context.go:174
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0x1400014fa00, 0x140008fe200)
        github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x524 fp=0x140008e3af0 sp=0x140008e3960 pc=0x104df1dd4
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0x1400014fa00, {0x1051bc230?, 0x140008f20e0}, 0x140008fe100)
        github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1a0 fp=0x140008e3b30 sp=0x140008e3af0 pc=0x104df1720
net/http.serverHandler.ServeHTTP({0x1051ba500?}, {0x1051bc230?, 0x140008f20e0?}, 0x6?)
        net/http/server.go:2938 +0xbc fp=0x140008e3b60 sp=0x140008e3b30 pc=0x104b8f92c
net/http.(*conn).serve(0x1400039e360, {0x1051bd7d8, 0x1400047a570})
        net/http/server.go:2009 +0x518 fp=0x140008e3fa0 sp=0x140008e3b60 pc=0x104b8bd28
net/http.(*Server).Serve.func3()
        net/http/server.go:3086 +0x30 fp=0x140008e3fd0 sp=0x140008e3fa0 pc=0x104b90040
runtime.goexit()
        runtime/asm_arm64.s:1197 +0x4 fp=0x140008e3fd0 sp=0x140008e3fd0 pc=0x1049679f4
created by net/http.(*Server).Serve in goroutine 1
        net/http/server.go:3086 +0x4cc

r0      0x458
r1      0xffffffffffffffff
r2      0x1
r3      0x1
r4      0x0
r5      0x1388
r6      0x34
r7      0x0
r8      0x3c
r9      0x1e6d2b9d0
r10     0x11
r11     0x0
r12     0x180
r13     0x170d8ef00
r14     0x181
r15     0x42
r16     0x18fa555f4
r17     0x1eff4e038
r18     0x0
r19     0x458
r20     0x0
r21     0x170d8ee80
r22     0x0
r23     0x17
r24     0x1388
r25     0x14000037798
r26     0x1051b4918
r27     0x820
r28     0x140006821a0
r29     0x170d8edb0
lr      0x987100018f989300
sp      0x170d8edb0
pc      0x18fa55600
fault   0x458

Memory looks good, usually generation times are in the range of a few seconds.
Tested version 0.1.17 and 0.1.20.
This is running on the Metal API.

Originally created by @jayouimet on GitHub (Jan 24, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2176 Originally assigned to: @dhiltgen on GitHub. Hello, We have an ollama instance that starts to hang after a few hours of use. When using ctrl + c to stop the serve, we get a long stack trace resembling this, could be missing lines at the top as it is the maximum I can get from my ssh instance : ``` net/http/server.go:3086 +0x30 fp=0x140008e5fd0 sp=0x140008e5fa0 pc=0x104b90040 runtime.goexit() runtime/asm_arm64.s:1197 +0x4 fp=0x140008e5fd0 sp=0x140008e5fd0 pc=0x1049679f4 created by net/http.(*Server).Serve in goroutine 1 net/http/server.go:3086 +0x4cc goroutine 394 [sync.Mutex.Lock, 6 minutes]: runtime.gopark(0x140008e2fc8?, 0x104953134?, 0xf0?, 0x88?, 0x140008e2fe8?) runtime/proc.go:398 +0xc8 fp=0x140008e2f90 sp=0x140008e2f70 pc=0x1049364e8 runtime.goparkunlock(...) runtime/proc.go:404 runtime.semacquire1(0x1055b2124, 0x7d?, 0x3, 0x1, 0x42?) runtime/sema.go:160 +0x208 fp=0x140008e2fe0 sp=0x140008e2f90 pc=0x104947b08 sync.runtime_SemacquireMutex(0x14000348450?, 0x0?, 0x0?) runtime/sema.go:77 +0x28 fp=0x140008e3020 sp=0x140008e2fe0 pc=0x104963248 sync.(*Mutex).lockSlow(0x1055b2120) sync/mutex.go:171 +0x174 fp=0x140008e3070 sp=0x140008e3020 pc=0x104972114 sync.(*Mutex).Lock(...) sync/mutex.go:90 github.com/jmorganca/ollama/server.GenerateHandler(0x140008fe200) github.com/jmorganca/ollama/server/routes.go:140 +0x90 fp=0x140008e3720 sp=0x140008e3070 pc=0x104e0ca60 github.com/gin-gonic/gin.(*Context).Next(...) github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/jmorganca/ollama/server.(*Server).GenerateRoutes.func1(0x140008fe200) github.com/jmorganca/ollama/server/routes.go:877 +0x78 fp=0x140008e3760 sp=0x140008e3720 pc=0x104e14dd8 github.com/gin-gonic/gin.(*Context).Next(...) github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0x140008fe200) github.com/gin-gonic/gin@v1.9.1/recovery.go:102 +0x80 fp=0x140008e37b0 sp=0x140008e3760 pc=0x104df3900 github.com/gin-gonic/gin.(*Context).Next(...) github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.LoggerWithConfig.func1(0x140008fe200) github.com/gin-gonic/gin@v1.9.1/logger.go:240 +0xb0 fp=0x140008e3960 sp=0x140008e37b0 pc=0x104df2ca0 github.com/gin-gonic/gin.(*Context).Next(...) github.com/gin-gonic/gin@v1.9.1/context.go:174 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0x1400014fa00, 0x140008fe200) github.com/gin-gonic/gin@v1.9.1/gin.go:620 +0x524 fp=0x140008e3af0 sp=0x140008e3960 pc=0x104df1dd4 github.com/gin-gonic/gin.(*Engine).ServeHTTP(0x1400014fa00, {0x1051bc230?, 0x140008f20e0}, 0x140008fe100) github.com/gin-gonic/gin@v1.9.1/gin.go:576 +0x1a0 fp=0x140008e3b30 sp=0x140008e3af0 pc=0x104df1720 net/http.serverHandler.ServeHTTP({0x1051ba500?}, {0x1051bc230?, 0x140008f20e0?}, 0x6?) net/http/server.go:2938 +0xbc fp=0x140008e3b60 sp=0x140008e3b30 pc=0x104b8f92c net/http.(*conn).serve(0x1400039e360, {0x1051bd7d8, 0x1400047a570}) net/http/server.go:2009 +0x518 fp=0x140008e3fa0 sp=0x140008e3b60 pc=0x104b8bd28 net/http.(*Server).Serve.func3() net/http/server.go:3086 +0x30 fp=0x140008e3fd0 sp=0x140008e3fa0 pc=0x104b90040 runtime.goexit() runtime/asm_arm64.s:1197 +0x4 fp=0x140008e3fd0 sp=0x140008e3fd0 pc=0x1049679f4 created by net/http.(*Server).Serve in goroutine 1 net/http/server.go:3086 +0x4cc r0 0x458 r1 0xffffffffffffffff r2 0x1 r3 0x1 r4 0x0 r5 0x1388 r6 0x34 r7 0x0 r8 0x3c r9 0x1e6d2b9d0 r10 0x11 r11 0x0 r12 0x180 r13 0x170d8ef00 r14 0x181 r15 0x42 r16 0x18fa555f4 r17 0x1eff4e038 r18 0x0 r19 0x458 r20 0x0 r21 0x170d8ee80 r22 0x0 r23 0x17 r24 0x1388 r25 0x14000037798 r26 0x1051b4918 r27 0x820 r28 0x140006821a0 r29 0x170d8edb0 lr 0x987100018f989300 sp 0x170d8edb0 pc 0x18fa55600 fault 0x458 ``` Memory looks good, usually generation times are in the range of a few seconds. Tested version 0.1.17 and 0.1.20. This is running on the Metal API.
GiteaMirror added the bug label 2026-04-12 11:01:08 -05:00
GiteaMirror commented 2026-04-12 11:01:10 -05:00
Author
Owner
Copy Link

@igorschlum commented on GitHub (Jan 25, 2024):

@jayouimet Thank you. Very interesting. Ollama could have an option to lock several LLMs in memory and handle a queue of requests to avoid setting this on your side.

<!-- gh-comment-id:1909262914 --> @igorschlum commented on GitHub (Jan 25, 2024): @jayouimet Thank you. Very interesting. Ollama could have an option to lock several LLMs in memory and handle a queue of requests to avoid setting this on your side.
GiteaMirror commented 2026-04-12 11:01:10 -05:00
Author
Owner
Copy Link

@jayouimet commented on GitHub (Feb 7, 2024):

@igorschlum It has been added in the last version, as a request parameter rather than an env variable. I remember seeing an issue that I can't find again saying Ollama would eventually hang after repeatedly sending the same prompt over and over. Could be a linked issue.
Will create another ticket or update this one if the problem persists.

<!-- gh-comment-id:1932677945 --> @jayouimet commented on GitHub (Feb 7, 2024): @igorschlum It has been added in the last version, as a request parameter rather than an env variable. I remember seeing an issue that I can't find again saying Ollama would eventually hang after repeatedly sending the same prompt over and over. Could be a linked issue. Will create another ticket or update this one if the problem persists.
GiteaMirror commented 2026-04-12 11:01:10 -05:00
Author
Owner
Copy Link

@koleshjr commented on GitHub (May 8, 2024):

This bug is so frustrating and it doesn't seem to be fixed even in the newer versions

<!-- gh-comment-id:2100373279 --> @koleshjr commented on GitHub (May 8, 2024): This bug is so frustrating and it doesn't seem to be fixed even in the newer versions
GiteaMirror commented 2026-04-12 11:01:11 -05:00
Author
Owner
Copy Link

@igorschlum commented on GitHub (May 8, 2024):

In the last version, you can now setup multiple instance of Ollama at the same time and keep a certain number of LLM in memory, it looks like a good solution for you and you don't need to wake up Ollama, as it can now be always up.

<!-- gh-comment-id:2101318169 --> @igorschlum commented on GitHub (May 8, 2024): In the last version, you can now setup multiple instance of Ollama at the same time and keep a certain number of LLM in memory, it looks like a good solution for you and you don't need to wake up Ollama, as it can now be always up.
GiteaMirror commented 2026-04-12 11:01:11 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (May 10, 2024):

Hanging is definitely a serious issue we that should be fixed quickly if not already fixed. Which model are you using? This will help reproduce the issue

<!-- gh-comment-id:2103671602 --> @jmorganca commented on GitHub (May 10, 2024): Hanging is definitely a serious issue we that should be fixed quickly if not already fixed. Which model are you using? This will help reproduce the issue
GiteaMirror commented 2026-04-12 11:01:11 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 1, 2024):

@jayouimet it sounds like you are no longer seeing the problem with newer versions.

I'm going to close the issue, but if you're still having problems @koleshjr make sure you're running the latest version, and if that doesn't resolve it, please share more information about your scenario and I'll reopen the issue. Are you using keep_alive set to -1 to keep the model loaded indefinitely? Are your clients hitting it periodically, or is it under constant requests? What sort of requests are the clients making (what model, which API, optional settings, example payload) that way we can try to repro to find the root cause.

<!-- gh-comment-id:2143570082 --> @dhiltgen commented on GitHub (Jun 1, 2024): @jayouimet it sounds like you are no longer seeing the problem with newer versions. I'm going to close the issue, but if you're still having problems @koleshjr make sure you're running the latest version, and if that doesn't resolve it, please share more information about your scenario and I'll reopen the issue. Are you using keep_alive set to -1 to keep the model loaded indefinitely? Are your clients hitting it periodically, or is it under constant requests? What sort of requests are the clients making (what model, which API, optional settings, example payload) that way we can try to repro to find the root cause.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1244
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?