SInce the ollama api returns the raw tokens of the context, counting these to display a raw context length in the generation info could be useful for those wanting to test large context models/see exactly how many tokens the model used.
Edit: Misunderstood Ollama api
Originally created by @notasquid1938 on GitHub (May 3, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1950
SInce the ollama api returns the raw tokens of the context, counting these to display a raw context length in the generation info could be useful for those wanting to test large context models/see exactly how many tokens the model used.
Edit: Misunderstood Ollama api
We already have something that seems to fit your description, if it's not feel free to elaborate more and let us know.
<!-- gh-comment-id:2094372696 -->
@tjbck commented on GitHub (May 4, 2024):
<img width="385" alt="image" src="https://github.com/open-webui/open-webui/assets/25473318/1cf8c343-9345-4e29-a0b6-a7f99a9ca6d2">
We already have something that seems to fit your description, if it's not feel free to elaborate more and let us know.
Are you referring to prompt_eval_count? I tried searching the repo but couldn't find any reference to what it means.
<!-- gh-comment-id:2094373497 -->
@notasquid1938 commented on GitHub (May 4, 2024):
Are you referring to prompt_eval_count? I tried searching the repo but couldn't find any reference to what it means.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @notasquid1938 on GitHub (May 3, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1950
SInce the ollama api returns the raw tokens of the context, counting these to display a raw context length in the generation info could be useful for those wanting to test large context models/see exactly how many tokens the model used.
Edit: Misunderstood Ollama api
@tjbck commented on GitHub (May 4, 2024):
We already have something that seems to fit your description, if it's not feel free to elaborate more and let us know.
@notasquid1938 commented on GitHub (May 4, 2024):
Are you referring to prompt_eval_count? I tried searching the repo but couldn't find any reference to what it means.
@tjbck commented on GitHub (May 4, 2024):
Ollama doc: https://github.com/ollama/ollama/blob/main/docs/api.md
@notasquid1938 commented on GitHub (May 4, 2024):
Thanks for the link