github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2731] [Issue] using gemma model as a chatbot #1641

New Issue
Closed
opened 2026-04-12 11:35:07 -05:00 by GiteaMirror · 13 comments
GiteaMirror commented 2026-04-12 11:35:07 -05:00
Owner
Copy Link

Originally created by @cosmo3769 on GitHub (Feb 24, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2731

I was using mistral model for my PDF chatbot. With the arrival of gemma model, I am trying to use this model. But it gives me an issue: After embedding external PDF document, when I ask question, it always gives me a response that it is not able to provide any information about the provided context.

Example of an issue:

If I uploaded ssl cookbook document, I ask a question: What is SSL?
In return the chatbot answers me with: The context does not provide any information about what SSL is, so I cannot answer this question from the provided context.

Tech stack involved

  • Using gemma:2b model. Also tried using gemma:7b (Will not use since this is running slow in local).
  • Using Xenova/all-MiniLM-L6-v2 embedding model from @xenova/transformers package.
  • Using Langchain.
  • Using Chroma as vectorstore.

Reproduce

It is a next.js application using langchain, chroma and transfomers.js.

  • Clone this repo: https://github.com/cosmo3769/PDFChatter/tree/gemma-model
  • Follow README.md setup guide.

The same code works for mistral and llama2:7b-chat but fails to work when using gemma:2b or gemma:7b. Any specific tweaks needed for this?

@jmorganca @mxyng

Originally created by @cosmo3769 on GitHub (Feb 24, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2731 I was using `mistral` model for my PDF chatbot. With the arrival of gemma model, I am trying to use this model. But it gives me an issue: ***After embedding external PDF document, when I ask question, it always gives me a response that it is not able to provide any information about the provided context.*** ## Example of an issue: If I uploaded `ssl cookbook` document, I ask a question: `What is SSL?` In return the chatbot answers me with: `The context does not provide any information about what SSL is, so I cannot answer this question from the provided context.` ## Tech stack involved * Using gemma:2b model. Also tried using gemma:7b (Will not use since this is running slow in local). * Using `Xenova/all-MiniLM-L6-v2` embedding model from `@xenova/transformers` package. * Using Langchain. * Using Chroma as vectorstore. ## Reproduce It is a next.js application using langchain, chroma and transfomers.js. * Clone this repo: `https://github.com/cosmo3769/PDFChatter/tree/gemma-model` * Follow `README.md` setup guide. The same code works for `mistral` and `llama2:7b-chat` but fails to work when using `gemma:2b` or `gemma:7b`. Any specific tweaks needed for this? @jmorganca @mxyng
GiteaMirror commented 2026-04-12 11:35:09 -05:00
Author
Owner
Copy Link

@XinyueZ commented on GitHub (Feb 25, 2024):

have you tried gemma:2b-instruct?

I have a related question https://github.com/ollama/ollama/issues/2743

<!-- gh-comment-id:1962925732 --> @XinyueZ commented on GitHub (Feb 25, 2024): have you tried gemma:2b-instruct? I have a related question https://github.com/ollama/ollama/issues/2743
GiteaMirror commented 2026-04-12 11:35:10 -05:00
Author
Owner
Copy Link

@cosmo3769 commented on GitHub (Feb 25, 2024):

I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong.

<!-- gh-comment-id:1963030568 --> @cosmo3769 commented on GitHub (Feb 25, 2024): I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong.
GiteaMirror commented 2026-04-12 11:35:11 -05:00
Author
Owner
Copy Link

@XinyueZ commented on GitHub (Feb 25, 2024):

https://ollama.com/library/gemma/tags

<!-- gh-comment-id:1963060739 --> @XinyueZ commented on GitHub (Feb 25, 2024): https://ollama.com/library/gemma/tags
GiteaMirror commented 2026-04-12 11:35:12 -05:00
Author
Owner
Copy Link

@cosmo3769 commented on GitHub (Feb 26, 2024):

Ohh how could I miss this. Well, I will give this a try and see if it works. Thanks!

<!-- gh-comment-id:1963487356 --> @cosmo3769 commented on GitHub (Feb 26, 2024): Ohh how could I miss this. Well, I will give this a try and see if it works. Thanks!
GiteaMirror commented 2026-04-12 11:35:13 -05:00
Author
Owner
Copy Link

@XinyueZ commented on GitHub (Feb 26, 2024):

btw, maybe you can post you code for your retrieval or chain.

<!-- gh-comment-id:1963863099 --> @XinyueZ commented on GitHub (Feb 26, 2024): btw, maybe you can post you code for your retrieval or chain.
GiteaMirror commented 2026-04-12 11:35:14 -05:00
Author
Owner
Copy Link

@cosmo3769 commented on GitHub (Feb 26, 2024):

The code for chain is included in worker.ts file under app directory.

<!-- gh-comment-id:1964907140 --> @cosmo3769 commented on GitHub (Feb 26, 2024): The code for chain is included in `worker.ts` file under `app` directory.
GiteaMirror commented 2026-04-12 11:35:15 -05:00
Author
Owner
Copy Link

@mvpbang commented on GitHub (Feb 27, 2024):

I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong.

upgrade ollama version 0.1.27, again try ollama run gemma:7b ok

<!-- gh-comment-id:1965848841 --> @mvpbang commented on GitHub (Feb 27, 2024): > I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong. upgrade ollama version 0.1.27, again try ollama run gemma:7b ok
GiteaMirror commented 2026-04-12 11:35:16 -05:00
Author
Owner
Copy Link

@Felictycf commented on GitHub (Feb 27, 2024):

I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong.

upgrade ollama version 0.1.27, again try ollama run gemma:7b ok

image
I have the same question. I keep asking him, but he keeps saying that the information provided by the context cannot be used. I used llama2 before, but my version is the latest 0.1.27

<!-- gh-comment-id:1966161743 --> @Felictycf commented on GitHub (Feb 27, 2024): > > I think ollama only provides gemma:2b and gemma:7b for now. Please correct me if I am wrong. > > upgrade ollama version 0.1.27, again try ollama run gemma:7b ok ![image](https://github.com/ollama/ollama/assets/90856766/56f94679-e187-4d77-b9a8-5a205ecc3c67) I have the same question. I keep asking him, but he keeps saying that the information provided by the context cannot be used. I used llama2 before, but my version is the latest 0.1.27
GiteaMirror commented 2026-04-12 11:35:17 -05:00
Author
Owner
Copy Link

@Felictycf commented on GitHub (Feb 27, 2024):

I was using mistral model for my PDF chatbot. With the arrival of gemma model, I am trying to use this model. But it gives me an issue: After embedding external PDF document, when I ask question, it always gives me a response that it is not able to provide any information about the provided context.

Example of an issue:

If I uploaded ssl cookbook document, I ask a question: What is SSL? In return the chatbot answers me with: The context does not provide any information about what SSL is, so I cannot answer this question from the provided context.

Tech stack involved

  • Using gemma:2b model. Also tried using gemma:7b (Will not use since this is running slow in local).
  • Using Xenova/all-MiniLM-L6-v2 embedding model from @xenova/transformers package.
  • Using Langchain.
  • Using Chroma as vectorstore.

Reproduce

It is a next.js application using langchain, chroma and transfomers.js.

  • Clone this repo: https://github.com/cosmo3769/PDFChatter/tree/gemma-model
  • Follow README.md setup guide.

The same code works for mistral and llama2:7b-chat but fails to work when using gemma:2b or gemma:7b. Any specific tweaks needed for this?

@jmorganca @mxyng
I have the same question. I keep asking him, but he keeps saying that the information provided by the context cannot be used. I used llama2 before, but my version is the latest 0.1.27
image

bro ,Have you found a solution?

<!-- gh-comment-id:1966163219 --> @Felictycf commented on GitHub (Feb 27, 2024): > I was using `mistral` model for my PDF chatbot. With the arrival of gemma model, I am trying to use this model. But it gives me an issue: _**After embedding external PDF document, when I ask question, it always gives me a response that it is not able to provide any information about the provided context.**_ > > ## Example of an issue: > If I uploaded `ssl cookbook` document, I ask a question: `What is SSL?` In return the chatbot answers me with: `The context does not provide any information about what SSL is, so I cannot answer this question from the provided context.` > > ## Tech stack involved > * Using gemma:2b model. Also tried using gemma:7b (Will not use since this is running slow in local). > * Using `Xenova/all-MiniLM-L6-v2` embedding model from `@xenova/transformers` package. > * Using Langchain. > * Using Chroma as vectorstore. > > ## Reproduce > It is a next.js application using langchain, chroma and transfomers.js. > > * Clone this repo: `https://github.com/cosmo3769/PDFChatter/tree/gemma-model` > * Follow `README.md` setup guide. > > The same code works for `mistral` and `llama2:7b-chat` but fails to work when using `gemma:2b` or `gemma:7b`. Any specific tweaks needed for this? > > @jmorganca @mxyng I have the same question. I keep asking him, but he keeps saying that the information provided by the context cannot be used. I used llama2 before, but my version is the latest 0.1.27 ![image](https://github.com/ollama/ollama/assets/90856766/fb52563d-9130-4a4b-9f85-5aa04f29bede) bro ,Have you found a solution?
GiteaMirror commented 2026-04-12 11:35:18 -05:00
Author
Owner
Copy Link

@cosmo3769 commented on GitHub (Feb 28, 2024):

@Felictycf I have not yet dived deep. I will get back to this in some time.

<!-- gh-comment-id:1968248186 --> @cosmo3769 commented on GitHub (Feb 28, 2024): @Felictycf I have not yet dived deep. I will get back to this in some time.
GiteaMirror commented 2026-04-12 11:35:19 -05:00
Author
Owner
Copy Link

@cosmo3769 commented on GitHub (Feb 28, 2024):

@mvpbang I am using latest version of ollama. Thank you.

<!-- gh-comment-id:1968249425 --> @cosmo3769 commented on GitHub (Feb 28, 2024): @mvpbang I am using latest version of ollama. Thank you.
GiteaMirror commented 2026-04-12 11:35:20 -05:00
Author
Owner
Copy Link

@epratik commented on GitHub (Feb 28, 2024):

I had issues with langchain, so made direct call to ollama api, but gemma2b has issues with chat, can someone help me on this ?

{ "model": "gemma:2b-instruct", "messages": [ { "role": "user", "content": "anything cheaper then that?" }, { "role": "assistant", "content": "The cheapest product by the context is the xyz with an offer price of 39.99." }, { "role": "user", "content": "give me chat summary till now" } ], "stream": false }

Response
{ "model": "gemma:2b-instruct", "created_at": "2024-02-28T05:09:01.385491Z", "message": { "role": "assistant", "content": "I am unable to provide a chat summary at this time. I do not have access to external chat data." }, "done": true, "total_duration": 651636333, "load_duration": 830500, "prompt_eval_duration": 103229000, "eval_count": 23, "eval_duration": 546308000 }

This works with gemma:7b model. I have tried different things, looks like it does not understand the conversation history.

<!-- gh-comment-id:1968282496 --> @epratik commented on GitHub (Feb 28, 2024): I had issues with langchain, so made direct call to ollama api, but gemma2b has issues with chat, can someone help me on this ? `{ "model": "gemma:2b-instruct", "messages": [ { "role": "user", "content": "anything cheaper then that?" }, { "role": "assistant", "content": "The cheapest product by the context is the xyz with an offer price of 39.99." }, { "role": "user", "content": "give me chat summary till now" } ], "stream": false }` Response `{ "model": "gemma:2b-instruct", "created_at": "2024-02-28T05:09:01.385491Z", "message": { "role": "assistant", "content": "I am unable to provide a chat summary at this time. I do not have access to external chat data." }, "done": true, "total_duration": 651636333, "load_duration": 830500, "prompt_eval_duration": 103229000, "eval_count": 23, "eval_duration": 546308000 }` This works with gemma:7b model. I have tried different things, looks like it does not understand the conversation history.
GiteaMirror commented 2026-04-12 11:35:21 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Jul 18, 2024):

Sorry for the slow response on this, peeps.

@cosmo3769 I think your question got answered before?

@epratik the Ollama API is stateless, so you'll have to keep track of the context yourself when you're calling it (this is both with the /api/generate endpoint as well as the /api/chat endpoint). The chat endpoint is probably easier because you can just maintain a list of messages for the context.

I'm going to go ahead and close the issue.

<!-- gh-comment-id:2237735588 --> @pdevine commented on GitHub (Jul 18, 2024): Sorry for the slow response on this, peeps. @cosmo3769 I think your question got answered before? @epratik the Ollama API is stateless, so you'll have to keep track of the context yourself when you're calling it (this is both with the `/api/generate` endpoint as well as the `/api/chat` endpoint). The chat endpoint is probably easier because you can just maintain a list of `messages` for the context. I'm going to go ahead and close the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1641
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?