github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-03-12 10:04:14 -05:00
Code Issues 308 Packages Projects Releases 100 Wiki Activity

feat: token counting according to model's context size #230

New Issue
Open
opened 2025-11-11 14:12:30 -06:00 by GiteaMirror · 4 comments
GiteaMirror commented 2025-11-11 14:12:30 -06:00
Owner
Copy Link

Originally created by @peperunas on GitHub (Jan 25, 2024).

Is your feature request related to a problem? Please describe.

I believe a crucial feature would be to have an idea of the context size of the conversation with a model. In this way, the user is aware of how much more the chat can continue.

As of now, it seems there is no "official" way to pull a model's context size from Ollama via its API. The issue has been raised and tracked here 1.

I guess this issue can track the advancement of the API endpoint on Ollama's side as well.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Originally created by @peperunas on GitHub (Jan 25, 2024). **Is your feature request related to a problem? Please describe.** I believe a crucial feature would be to have an idea of the context size of the conversation with a model. In this way, the user is aware of how much more the chat can continue. As of now, it seems there is no "official" way to pull a model's context size from Ollama via its API. The issue has been raised and tracked here [1]. I guess this issue can track the advancement of the API endpoint on Ollama's side as well. [1]: https://github.com/ollama/ollama/issues/1473 **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional context** Add any other context or screenshots about the feature request here.
GiteaMirror added the enhancementgood first issuehelp wantednon-core labels 2025-11-11 14:12:31 -06:00
GiteaMirror commented 2025-11-11 14:12:33 -06:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Feb 3, 2024):

You can sort of see this by clicking on the little "info" icon I think but agree it would be nice to have a clearer representation.

@jukofyork commented on GitHub (Feb 3, 2024): You can sort of see this by clicking on the little "info" icon I think but agree it would be nice to have a clearer representation.
GiteaMirror commented 2025-11-11 14:12:34 -06:00
Author
Owner
Copy Link

@justinh-rahb commented on GitHub (Feb 19, 2024):

should be a way to set "num_ctx" like when using ollama in code

Example of this:
ollamaLLM1 = Ollama(model="mistral:7b", temperature=0.9, num_ctx=8192)

It is already an option to set the "temperature", should be doable to add the "num_ctx" for manual context length.
or even automatic if that information can be pulled from the ollama api, but to my knowledge there is no way to do that, so it just defaults to 2048.

I'm not sure you've got this in the right place. This seems like a suggestion to make to the Ollama project, not here.

@justinh-rahb commented on GitHub (Feb 19, 2024): > should be a way to set "num_ctx" like when using ollama in code > > Example of this: > ollamaLLM1 = Ollama(model="mistral:7b", temperature=0.9, num_ctx=8192) > > It is already an option to set the "temperature", should be doable to add the "num_ctx" for manual context length. > or even automatic if that information can be pulled from the ollama api, but to my knowledge there is no way to do that, so it just defaults to 2048. I'm not sure you've got this in the right place. This seems like a suggestion to make to the Ollama project, not here.
GiteaMirror commented 2025-11-11 14:12:34 -06:00
Author
Owner
Copy Link

@scscgit commented on GitHub (Aug 7, 2024):

Cross-linking with issue https://github.com/open-webui/open-webui/discussions/4246#discussioncomment-10264834 which has been sadly turned into a discussion

@scscgit commented on GitHub (Aug 7, 2024): Cross-linking with issue https://github.com/open-webui/open-webui/discussions/4246#discussioncomment-10264834 which has been sadly turned into a discussion
GiteaMirror commented 2025-11-11 14:12:34 -06:00
Author
Owner
Copy Link

@evolutioned commented on GitHub (Jul 22, 2025):

i would suggest removing this as a "Good First Issue", as it requires a deep understanding of how to work with Ollama in the backend. It's a complex problem for a first timer.

@evolutioned commented on GitHub (Jul 22, 2025): i would suggest removing this as a "Good First Issue", as it requires a deep understanding of how to work with Ollama in the backend. It's a complex problem for a first timer.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label enhancement good first issue help wanted non-core
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#230
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?