github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 13:47:08 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #7152] Chat History persists with ollama run #82220

New Issue
Closed
opened 2026-05-09 13:29:39 -05:00 by GiteaMirror · 14 comments
GiteaMirror commented 2026-05-09 13:29:39 -05:00
Owner
Copy Link

Originally created by @hg0428 on GitHub (Oct 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7152

Originally assigned to: @jessegross on GitHub.

What is the issue?

image
See the image. The chat history is persisting for Falcon, but not any other models (at least that I've tried so far). I would expect it to not persist. It even persists after re-creating the model:
image

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.3.12

Originally created by @hg0428 on GitHub (Oct 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7152 Originally assigned to: @jessegross on GitHub. ### What is the issue? ![image](https://github.com/user-attachments/assets/b53c625b-0448-4d10-aa0b-63945f72c02e) See the image. The chat history is persisting for [Falcon](https://ollama.com/Hudson/falcon-mamba-instruct), but not any other models (at least that I've tried so far). I would expect it to not persist. It even persists after re-creating the model: ![image](https://github.com/user-attachments/assets/03cbe28e-9fbd-452a-b53b-503ec3c67d50) ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.3.12
GiteaMirror added the bug label 2026-05-09 13:29:39 -05:00
GiteaMirror commented 2026-05-09 13:29:42 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 9, 2024):

The problem happens in the API as well.
image

<!-- gh-comment-id:2402726983 --> @hg0428 commented on GitHub (Oct 9, 2024): The problem happens in the API as well. ![image](https://github.com/user-attachments/assets/1c38ffc5-a7be-4cbd-a3b8-cfa323cc2b44)
GiteaMirror commented 2026-05-09 13:29:43 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Oct 9, 2024):

What's in your Modelfile? What's the output of ollama show --modelfile falcon-mamba-instruct:7b-q4_0?

<!-- gh-comment-id:2402784042 --> @rick-github commented on GitHub (Oct 9, 2024): What's in your `Modelfile`? What's the output of `ollama show --modelfile falcon-mamba-instruct:7b-q4_0`?
GiteaMirror commented 2026-05-09 13:29:43 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 9, 2024):

What's in your Modelfile? What's the output of ollama show --modelfile falcon-mamba-instruct:7b-q4_0?

# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM falcon-mamba-instruct:7b-q4_0

FROM /Users/hudsongouge/.ollama/models/blobs/sha256-26e1d318e3c108d4f263056618afc3bf181fd518011451f3ea814eef03627157
TEMPLATE "{{- range .Messages }}<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{ end }}<|im_start|>assistant
"
SYSTEM You are a helpful AI assistant.
PARAMETER stop <|im_start|>
PARAMETER stop <|im_end|>
PARAMETER min_p 0.1
PARAMETER repeat_penalty 1
<!-- gh-comment-id:2402787363 --> @hg0428 commented on GitHub (Oct 9, 2024): > What's in your `Modelfile`? What's the output of `ollama show --modelfile falcon-mamba-instruct:7b-q4_0`? ``` # Modelfile generated by "ollama show" # To build a new Modelfile based on this, replace FROM with: # FROM falcon-mamba-instruct:7b-q4_0 FROM /Users/hudsongouge/.ollama/models/blobs/sha256-26e1d318e3c108d4f263056618afc3bf181fd518011451f3ea814eef03627157 TEMPLATE "{{- range .Messages }}<|im_start|>{{ .Role }} {{ .Content }}<|im_end|> {{ end }}<|im_start|>assistant " SYSTEM You are a helpful AI assistant. PARAMETER stop <|im_start|> PARAMETER stop <|im_end|> PARAMETER min_p 0.1 PARAMETER repeat_penalty 1 ```
GiteaMirror commented 2026-05-09 13:29:43 -05:00
Author
Owner
Copy Link

@victorb commented on GitHub (Oct 10, 2024):

To me this seems like a bug in the Python library you're using for Ollama, not in Ollama itself, or the API.

My own example:

    println!("Step #1");

    let messages = vec![
        client::Message::new("system", r#"You are a helpful assistant"#),
        client::Message::new("user", "x = 4 and y = 137. Remember that."),
    ];

    let response: LLMResponse = client::call_llm(&client, model, messages.clone())
        .await
        .unwrap();

    println!("Response:\n{:#?}", response.message.content);

    println!("Step #2");

    let messages = vec![
        client::Message::new("system", r#"You are a helpful assistant"#),
        client::Message::new("user", "What are x and y?"),
    ];

    let response: LLMResponse = client::call_llm(&client, model, messages.clone())
        .await
        .unwrap();

    println!("Response:\n{:#?}", response.message.content);

Outputs:

Step #1
Response:
"I've noted that you've set x to 4 and y to 137. If you need any calculations or further information related to these values, feel free to ask!"
Step #2
Response:
"To provide an answer, I need more information about what x and y represent. Could you please give me the context or equations involving x and y?"

Which seems to work like I expect it to.

Can you replicate this issue when you use the HTTP API directly without using the Python library you're using?

<!-- gh-comment-id:2404885338 --> @victorb commented on GitHub (Oct 10, 2024): To me this seems like a bug in the Python library you're using for Ollama, not in Ollama itself, or the API. My own example: ```rust println!("Step #1"); let messages = vec![ client::Message::new("system", r#"You are a helpful assistant"#), client::Message::new("user", "x = 4 and y = 137. Remember that."), ]; let response: LLMResponse = client::call_llm(&client, model, messages.clone()) .await .unwrap(); println!("Response:\n{:#?}", response.message.content); println!("Step #2"); let messages = vec![ client::Message::new("system", r#"You are a helpful assistant"#), client::Message::new("user", "What are x and y?"), ]; let response: LLMResponse = client::call_llm(&client, model, messages.clone()) .await .unwrap(); println!("Response:\n{:#?}", response.message.content); ``` Outputs: ``` Step #1 Response: "I've noted that you've set x to 4 and y to 137. If you need any calculations or further information related to these values, feel free to ask!" Step #2 Response: "To provide an answer, I need more information about what x and y represent. Could you please give me the context or equations involving x and y?" ``` Which seems to work like I expect it to. Can you replicate this issue when you use the HTTP API directly without using the Python library you're using?
GiteaMirror commented 2026-05-09 13:29:44 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 10, 2024):

To me this seems like a bug in the Python library you're using for Ollama, not in Ollama itself, or the API.

My own example:

    println!("Step #1");

    let messages = vec![
        client::Message::new("system", r#"You are a helpful assistant"#),
        client::Message::new("user", "x = 4 and y = 137. Remember that."),
    ];

    let response: LLMResponse = client::call_llm(&client, model, messages.clone())
        .await
        .unwrap();

    println!("Response:\n{:#?}", response.message.content);

    println!("Step #2");

    let messages = vec![
        client::Message::new("system", r#"You are a helpful assistant"#),
        client::Message::new("user", "What are x and y?"),
    ];

    let response: LLMResponse = client::call_llm(&client, model, messages.clone())
        .await
        .unwrap();

    println!("Response:\n{:#?}", response.message.content);

Outputs:

Step #1
Response:
"I've noted that you've set x to 4 and y to 137. If you need any calculations or further information related to these values, feel free to ask!"
Step #2
Response:
"To provide an answer, I need more information about what x and y represent. Could you please give me the context or equations involving x and y?"

Which seems to work like I expect it to.

Can you replicate this issue when you use the HTTP API directly without using the Python library you're using?

The issue is also in the Ollama cli via ollama run <model>, so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library.

<!-- gh-comment-id:2404934065 --> @hg0428 commented on GitHub (Oct 10, 2024): > To me this seems like a bug in the Python library you're using for Ollama, not in Ollama itself, or the API. > > My own example: > > ```rust > println!("Step #1"); > > let messages = vec![ > client::Message::new("system", r#"You are a helpful assistant"#), > client::Message::new("user", "x = 4 and y = 137. Remember that."), > ]; > > let response: LLMResponse = client::call_llm(&client, model, messages.clone()) > .await > .unwrap(); > > println!("Response:\n{:#?}", response.message.content); > > println!("Step #2"); > > let messages = vec![ > client::Message::new("system", r#"You are a helpful assistant"#), > client::Message::new("user", "What are x and y?"), > ]; > > let response: LLMResponse = client::call_llm(&client, model, messages.clone()) > .await > .unwrap(); > > println!("Response:\n{:#?}", response.message.content); > ``` > > Outputs: > > ``` > Step #1 > Response: > "I've noted that you've set x to 4 and y to 137. If you need any calculations or further information related to these values, feel free to ask!" > Step #2 > Response: > "To provide an answer, I need more information about what x and y represent. Could you please give me the context or equations involving x and y?" > ``` > > Which seems to work like I expect it to. > > Can you replicate this issue when you use the HTTP API directly without using the Python library you're using? The issue is also in the Ollama cli via `ollama run <model>`, so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library.
GiteaMirror commented 2026-05-09 13:29:45 -05:00
Author
Owner
Copy Link

@victorb commented on GitHub (Oct 10, 2024):

The issue is also in the Ollama cli via ollama run , so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library.

I can't seem to reproduce this issue with ollama run either, works like I expect it to too:

$ ./ollama run qwen2.5:32b "x = 4 and y = 137. Remember that."
Got it, you've specified that x = 4 and y = 137. If you need any calculations or further information involving these variables,
feel free to ask!

$ ./ollama run qwen2.5:32b "What are x and y?"
To provide an answer about what \(x\) and \(y\) are, I would need more context or information regarding the specific scenario
you're referring to, as \(x\) and \(y\) can represent different things depending on the situation:

1. **Mathematics**: In mathematics, \(x\)^C
<!-- gh-comment-id:2404969304 --> @victorb commented on GitHub (Oct 10, 2024): > The issue is also in the Ollama cli via ollama run <model>, so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library. I can't seem to reproduce this issue with `ollama run` either, works like I expect it to too: ```bash $ ./ollama run qwen2.5:32b "x = 4 and y = 137. Remember that." Got it, you've specified that x = 4 and y = 137. If you need any calculations or further information involving these variables, feel free to ask! $ ./ollama run qwen2.5:32b "What are x and y?" To provide an answer about what \(x\) and \(y\) are, I would need more context or information regarding the specific scenario you're referring to, as \(x\) and \(y\) can represent different things depending on the situation: 1. **Mathematics**: In mathematics, \(x\)^C ```
GiteaMirror commented 2026-05-09 13:29:45 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 10, 2024):

The issue is also in the Ollama cli via ollama run , so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library.

I can't seem to reproduce this issue with ollama run either, works like I expect it to too:

$ ./ollama run qwen2.5:32b "x = 4 and y = 137. Remember that."
Got it, you've specified that x = 4 and y = 137. If you need any calculations or further information involving these variables,
feel free to ask!

$ ./ollama run qwen2.5:32b "What are x and y?"
To provide an answer about what \(x\) and \(y\) are, I would need more context or information regarding the specific scenario
you're referring to, as \(x\) and \(y\) can represent different things depending on the situation:

1. **Mathematics**: In mathematics, \(x\)^C

If you see my original message, the issue ONLY occurs with Falcon Mamba. That's the weird part. Either way, it is a problem that needs to be fixed as that model is pretty much unusable for my user-case until this is fixed.

<!-- gh-comment-id:2404973990 --> @hg0428 commented on GitHub (Oct 10, 2024): > > The issue is also in the Ollama cli via ollama run , so I would expect it to be an issue with Ollama. The Python library I used was the official ollama python library. > > I can't seem to reproduce this issue with `ollama run` either, works like I expect it to too: > > ```shell > $ ./ollama run qwen2.5:32b "x = 4 and y = 137. Remember that." > Got it, you've specified that x = 4 and y = 137. If you need any calculations or further information involving these variables, > feel free to ask! > > $ ./ollama run qwen2.5:32b "What are x and y?" > To provide an answer about what \(x\) and \(y\) are, I would need more context or information regarding the specific scenario > you're referring to, as \(x\) and \(y\) can represent different things depending on the situation: > > 1. **Mathematics**: In mathematics, \(x\)^C > ``` If you see my original message, the issue ONLY occurs with Falcon Mamba. That's the weird part. Either way, it is a problem that needs to be fixed as that model is pretty much unusable for my user-case until this is fixed.
GiteaMirror commented 2026-05-09 13:29:46 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 10, 2024):

image

<!-- gh-comment-id:2404998200 --> @hg0428 commented on GitHub (Oct 10, 2024): ![image](https://github.com/user-attachments/assets/5ebd3327-a0d5-4922-9062-b5824aeef963)
GiteaMirror commented 2026-05-09 13:29:46 -05:00
Author
Owner
Copy Link

@YonTracks commented on GitHub (Oct 13, 2024):

I see something happening with the run cmd? runCmd := &cobra.Command{ Use: "run MODEL [PROMPT]", Short: "Run a model", Args: cobra.MinimumNArgs(1), PreRunE: checkServerHeartbeat, RunE: RunHandler, } also trying to learn the generateInteractive() mode bit? not sure? if I check the server.log I can see the chat endpoint being called? messages / context / cache / persistence issue there somewhere? I keep looking good luck.

<!-- gh-comment-id:2408778687 --> @YonTracks commented on GitHub (Oct 13, 2024): I see something happening with the run cmd? ``` runCmd := &cobra.Command{ Use: "run MODEL [PROMPT]", Short: "Run a model", Args: cobra.MinimumNArgs(1), PreRunE: checkServerHeartbeat, RunE: RunHandler, }``` also trying to learn the generateInteractive() mode bit? not sure? if I check the server.log I can see the chat endpoint being called? messages / context / cache / persistence issue there somewhere? I keep looking good luck.
GiteaMirror commented 2026-05-09 13:29:46 -05:00
Author
Owner
Copy Link

@YonTracks commented on GitHub (Oct 13, 2024):

I see, if I use ollama run llama3.1 what can you do? it uses the generate (love? seems more smart? powerful). but yes, if only ollama run llama3.1 or interactive mode? then it uses the chat with persisted messages / prompts (issue). good luck. I will still try see what I can find (chat and generate in a single function seems best) anyway good luck.

<!-- gh-comment-id:2408786323 --> @YonTracks commented on GitHub (Oct 13, 2024): I see, if I use `ollama run llama3.1 what can you do? ` it uses the generate (love? seems more smart? powerful). but yes, if only ` ollama run llama3.1` or interactive mode? then it uses the chat with persisted messages / prompts (issue). good luck. I will still try see what I can find (chat and generate in a single function seems best) anyway good luck.
GiteaMirror commented 2026-05-09 13:29:47 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Oct 16, 2024):

My guess here is the problem is with falcon-mamba and the kv cache. Llama3.1 and llama3.2 seem to be working correctly.

<!-- gh-comment-id:2415550465 --> @pdevine commented on GitHub (Oct 16, 2024): My guess here is the problem is with falcon-mamba and the kv cache. Llama3.1 and llama3.2 seem to be working correctly.
GiteaMirror commented 2026-05-09 13:29:47 -05:00
Author
Owner
Copy Link

@hg0428 commented on GitHub (Oct 16, 2024):

My guess here is the problem is with falcon-mamba and the kv cache. Llama3.1 and llama3.2 seem to be working correctly.

Yes, all other models I have tested (llama, Owen, gemma, etc...) work correctly.

<!-- gh-comment-id:2415560338 --> @hg0428 commented on GitHub (Oct 16, 2024): > My guess here is the problem is with falcon-mamba and the kv cache. Llama3.1 and llama3.2 seem to be working correctly. Yes, all other models I have tested (llama, Owen, gemma, etc...) work correctly.
GiteaMirror commented 2026-05-09 13:29:48 -05:00
Author
Owner
Copy Link

@jessegross commented on GitHub (Oct 16, 2024):

Yes, recurrent models such as Mamba do use the KV cache differently, so that is likely where the problem is.

<!-- gh-comment-id:2417533907 --> @jessegross commented on GitHub (Oct 16, 2024): Yes, recurrent models such as Mamba do use the KV cache differently, so that is likely where the problem is.
GiteaMirror commented 2026-05-09 13:29:48 -05:00
Author
Owner
Copy Link

@jessegross commented on GitHub (Oct 23, 2024):

I don't see this with the most recent 0.4.0-rc, can you see if it is still a problem there?

<!-- gh-comment-id:2433815648 --> @jessegross commented on GitHub (Oct 23, 2024): I don't see this with the most recent 0.4.0-rc, can you see if it is still a problem there?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#82220
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?