github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 13:47:08 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15596] Gemma 4 /api/chat slows from 6s → 116s after few requests (v0.20.7) #87641

New Issue
Open
opened 2026-05-10 06:11:47 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-10 06:11:47 -05:00
Owner
Copy Link

Originally created by @deep1305 on GitHub (Apr 15, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15596

What is the issue?

CLI ollama run gemma4 slows down progressively after updating to v0.20.7.

The first ok response took about 6 seconds, but a later /api/chat request took 1m56s on the same session and hardware.

Before v0.20.7, the same setup typically answered in about 20–25 seconds even during a longer chat.

  • Model: gemma4:26b-a4b-it-q4_K_M
  • Thinking mode: on

Relevant log output

[GIN] 2026/04/14 - 22:58:34 | 200 |   10.8710184s |       127.0.0.1 | POST     "/api/generate"
[GIN] 2026/04/14 - 22:58:43 | 200 |    6.3127129s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2026/04/14 - 23:00:57 | 200 |         1m56s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2026/04/14 - 23:03:11 | 200 |         2m13s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2026/04/14 - 23:07:13 | 200 |          4m1s |       127.0.0.1 | POST     "/api/chat"

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

v0.20.7

Originally created by @deep1305 on GitHub (Apr 15, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15596 ### What is the issue? CLI `ollama run gemma4` slows down progressively after updating to v0.20.7. The first `ok` response took about 6 seconds, but a later `/api/chat` request took 1m56s on the same session and hardware. Before v0.20.7, the same setup typically answered in about 20–25 seconds even during a longer chat. - Model: gemma4:26b-a4b-it-q4_K_M - Thinking mode: on ### Relevant log output ```shell [GIN] 2026/04/14 - 22:58:34 | 200 | 10.8710184s | 127.0.0.1 | POST "/api/generate" [GIN] 2026/04/14 - 22:58:43 | 200 | 6.3127129s | 127.0.0.1 | POST "/api/chat" [GIN] 2026/04/14 - 23:00:57 | 200 | 1m56s | 127.0.0.1 | POST "/api/chat" [GIN] 2026/04/14 - 23:03:11 | 200 | 2m13s | 127.0.0.1 | POST "/api/chat" [GIN] 2026/04/14 - 23:07:13 | 200 | 4m1s | 127.0.0.1 | POST "/api/chat" ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version v0.20.7
GiteaMirror added the bug label 2026-05-10 06:11:47 -05:00
GiteaMirror commented 2026-05-10 06:11:49 -05:00
Author
Owner
Copy Link

@grepin commented on GitHub (Apr 15, 2026):

@deep1305 didn't make deep digging about kv-cache allocation in ollama (so, i could be wrong), but to diagnose the root cause, try to use tools like nvtop and determine (watching nvtop UI), which part of model layers + kvcache resides in VRAM, and which part resides in RAM. Most likely you will see "increasing RAM usage + increased CPU load (at some point) + reduced GPU load (at some point) + fixed VRAM usage" while increasing context ("long dialog"). And if it's so, "there is no good solution to optimize/improve this yet" in ollama. But there is always an option: you can "just buy better hardware" in terms of GPU VRAM.

Anyway, if you suppose regression in 0.20.7, it's reasonable to show two logs: for 0.20.7 (you already did it) and for previous version that had different values for the same use case.

<!-- gh-comment-id:4251857814 --> @grepin commented on GitHub (Apr 15, 2026): @deep1305 didn't make deep digging about kv-cache allocation in ollama (so, i could be wrong), but to diagnose the root cause, try to use tools like nvtop and determine (watching nvtop UI), which part of model layers + kvcache resides in VRAM, and which part resides in RAM. Most likely you will see "increasing RAM usage + increased CPU load (at some point) + reduced GPU load (at some point) + fixed VRAM usage" while increasing context ("long dialog"). And if it's so, "there is no good solution to optimize/improve this yet" in ollama. But there is always an option: you can "just buy better hardware" in terms of GPU VRAM. Anyway, if you suppose regression in 0.20.7, it's reasonable to show two logs: for 0.20.7 (you already did it) and for previous version that had different values for the same use case.
GiteaMirror commented 2026-05-10 06:11:51 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 15, 2026):

What's in the prompt?

$ for i in {1..10} ; do printf "%2d %4.2f\n" $i $(/usr/bin/time -f %e bash -c 'ollama-run.py gemma4 why is the sky blue --num_predict=128 >/dev/null' 2>&1)  ; done
 1 1.46
 2 1.47
 3 1.45
 4 1.45
 5 1.43
 6 1.38
 7 1.44
 8 1.45
 9 1.77
10 1.44
<!-- gh-comment-id:4252526470 --> @rick-github commented on GitHub (Apr 15, 2026): What's in the prompt? ```console $ for i in {1..10} ; do printf "%2d %4.2f\n" $i $(/usr/bin/time -f %e bash -c 'ollama-run.py gemma4 why is the sky blue --num_predict=128 >/dev/null' 2>&1) ; done 1 1.46 2 1.47 3 1.45 4 1.45 5 1.43 6 1.38 7 1.44 8 1.45 9 1.77 10 1.44 ```
GiteaMirror commented 2026-05-10 06:11:53 -05:00
Author
Owner
Copy Link

@Mati0kez commented on GitHub (Apr 16, 2026):

This progressive slowdown pattern suggests a memory leak, unbounded cache, or O(n²) behavior in the chat session handling. Looking at whether this is in the session state management or the model forward path.

<!-- gh-comment-id:4262406813 --> @Mati0kez commented on GitHub (Apr 16, 2026): This progressive slowdown pattern suggests a memory leak, unbounded cache, or O(n²) behavior in the chat session handling. Looking at whether this is in the session state management or the model forward path.
GiteaMirror commented 2026-05-10 06:11:55 -05:00
Author
Owner
Copy Link

@Mati0kez commented on GitHub (Apr 16, 2026):

Progressive slowdown pattern suggests a memory leak or O(n²) behavior in session handling. Looking at whether it's in session state management or KV cache eviction.

<!-- gh-comment-id:4262435189 --> @Mati0kez commented on GitHub (Apr 16, 2026): Progressive slowdown pattern suggests a memory leak or O(n²) behavior in session handling. Looking at whether it's in session state management or KV cache eviction.
GiteaMirror commented 2026-05-10 06:11:56 -05:00
Author
Owner
Copy Link

@PureBlissAK commented on GitHub (Apr 18, 2026):

🤖 Automated Triage & Analysis Report

Issue: #15596
Analyzed: 2026-04-18T18:19:25.273650

Analysis

  • Type: unknown
  • Severity: medium
  • Components: unknown

Implementation Plan

  • Effort: medium
  • Steps:

This issue has been triaged and marked for implementation.

<!-- gh-comment-id:4274304842 --> @PureBlissAK commented on GitHub (Apr 18, 2026): <!-- ollama-issue-orchestrator:v1 issue:15596 --> ## 🤖 Automated Triage & Analysis Report **Issue**: #15596 **Analyzed**: 2026-04-18T18:19:25.273650 ### Analysis - **Type**: unknown - **Severity**: medium - **Components**: unknown ### Implementation Plan - **Effort**: medium - **Steps**: *This issue has been triaged and marked for implementation.*
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87641
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?