github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #4590] How i can understand, that model is not sure with answers ? (Get Entropy/Probas) #80561

New Issue
Closed
opened 2026-05-09 09:07:45 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-09 09:07:45 -05:00
Owner
Copy Link

Originally created by @antonbugaets on GitHub (May 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4590

i'm aware about temperature, top k/p parameters which i can configure using Ollama as serv. function for models inferencing,

But how do i can understand, that model is not sure about particular answer by my promts? While model inferences with Ollama.

I need to understand this to perform post-processing of 'low quality' answers.

Is it any possible ways to understand this ? Maybe by some optional parametr ? or get probas of tokens and calculate entropy.

Thanks!

Originally created by @antonbugaets on GitHub (May 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4590 i'm aware about temperature, top k/p parameters which i can configure using Ollama as serv. function for models inferencing, But how do i can understand, that model is not sure about particular answer by my promts? While model inferences with Ollama. I need to understand this to perform post-processing of 'low quality' answers. Is it any possible ways to understand this ? Maybe by some optional parametr ? or get probas of tokens and calculate entropy. Thanks!
GiteaMirror commented 2026-05-09 09:07:45 -05:00
Author
Owner
Copy Link

@Gomez12 commented on GitHub (May 23, 2024):

Do you have an algorithm or something like that for this?
Because it is possible to get back the probability of a token. But I wouldn't know where to go from there as I don't care about 50 -75% of the tokens of a response as long as it is readable. And tokens are not words etc.

The truthfulness of an answer hangs afaik mostly on just 1 or 2 words per sentence, the rest is just fluff to make it readable.

So first you would need to get all the statistics of all tokens (relatively easily done)
So you would need to find the most important words in an answer (you could use a lower model for this)
Then you have to translate the words to tokens, get the probabilities from that, and see what the probabilities of the keywords are, also taking into account all the alternatives with probabilities (a token can be unsure if it is for example single or plural, but that doesn't often change the truthfulness)

So I can see a way to make it work with something like crew-ai or basically multiple llms, but Ollama is (at least for me) just 1 model per question.

Another way to make it work for me is just simply ask a json response (I already need that) and just add a truthfulness grade from 1-10 into the response, I even ask it to add an explanation of the grade into the response.
And that works well for me, it will still sometimes hallucinate on the grade or the explanation or something like that, but if it gives a grade other than 1-10 then I just re-ask the question with a new context, which I also do if the model returns invalid json.
I have found that it is very important not to constrict the model to a grammar, as for me it seems like if the model goes off the road with the json prompting then it has a really high chance to go off the road with the answer

Or you could simulate json with just adding instructions for a start-word and a stop-word. If it doesn't obey those simple instructions, then it hasn't read the question good enough.

So I can see multiple ways around it with prompting, but no real way to put it in Ollama.

<!-- gh-comment-id:2127853814 --> @Gomez12 commented on GitHub (May 23, 2024): Do you have an algorithm or something like that for this? Because it is possible to get back the probability of a token. But I wouldn't know where to go from there as I don't care about 50 -75% of the tokens of a response as long as it is readable. And tokens are not words etc. The truthfulness of an answer hangs afaik mostly on just 1 or 2 words per sentence, the rest is just fluff to make it readable. So first you would need to get all the statistics of all tokens (relatively easily done) So you would need to find the most important words in an answer (you could use a lower model for this) Then you have to translate the words to tokens, get the probabilities from that, and see what the probabilities of the keywords are, also taking into account all the alternatives with probabilities (a token can be unsure if it is for example single or plural, but that doesn't often change the truthfulness) So I can see a way to make it work with something like crew-ai or basically multiple llms, but Ollama is (at least for me) just 1 model per question. Another way to make it work for me is just simply ask a json response (I already need that) and just add a truthfulness grade from 1-10 into the response, I even ask it to add an explanation of the grade into the response. And that works well for me, it will still sometimes hallucinate on the grade or the explanation or something like that, but if it gives a grade other than 1-10 then I just re-ask the question with a new context, which I also do if the model returns invalid json. I have found that it is very important not to constrict the model to a grammar, as for me it seems like if the model goes off the road with the json prompting then it has a really high chance to go off the road with the answer Or you could simulate json with just adding instructions for a start-word and a stop-word. If it doesn't obey those simple instructions, then it hasn't read the question good enough. So I can see multiple ways around it with prompting, but no real way to put it in Ollama.
GiteaMirror commented 2026-05-09 09:07:46 -05:00
Author
Owner
Copy Link

@antonbugaets commented on GitHub (May 24, 2024):

Hello @Gomez12 thanks for the advise.
I wanted to understand by some value and so on, to collect metrics and statistic.
i found one interesting article reg. my question https://medium.com/@miskelic/how-to-teach-your-llm-to-say-i-dont-know-8dc8beac1ea9

By the way i resolved my question by post processing. I found, if model tries to reflect to much, then model increases special symbols. It is accurate to my case. So, if there are to much symbols like "[" "(", e.t.c.
Model highlights alternative options by it, then model is not sure with the answer :)

Thanks!

<!-- gh-comment-id:2130229675 --> @antonbugaets commented on GitHub (May 24, 2024): Hello @Gomez12 thanks for the advise. I wanted to understand by some value and so on, to collect metrics and statistic. i found one interesting article reg. my question https://medium.com/@miskelic/how-to-teach-your-llm-to-say-i-dont-know-8dc8beac1ea9 By the way i resolved my question by post processing. I found, if model tries to reflect to much, then model increases special symbols. It is accurate to my case. So, if there are to much symbols like "[" "(", e.t.c. Model highlights alternative options by it, then model is not sure with the answer :) Thanks!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#80561
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?