github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #6794] Wrong response at math question! #81964

New Issue
Closed
opened 2026-05-09 12:52:43 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-09 12:52:43 -05:00
Owner
Copy Link

Originally created by @lsalamon on GitHub (Sep 13, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6794

What is the issue?

I try this question for llama 3.1:8b and he goes in hallucination at response:
Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3

OS Windows

GPU none

CPU AMD Ryzen 9 5900X 12-Core Processor

32 Gb memory

Ollama version ollama version is 0.3.10

Obs.:
I asked the same question for the solar model and the answer was wrong. There was no hallucination.

Originally created by @lsalamon on GitHub (Sep 13, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6794 ### What is the issue? I try this question for llama 3.1:8b and he goes in hallucination at response: Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3 ### OS Windows ### GPU none ### CPU AMD Ryzen 9 5900X 12-Core Processor ### 32 Gb memory ### Ollama version ollama version is 0.3.10 Obs.: I asked the same question for the solar model and the answer was wrong. There was no hallucination.
GiteaMirror added the bug label 2026-05-09 12:52:43 -05:00
GiteaMirror commented 2026-05-09 12:52:49 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 14, 2024):

LLMs are poor at math. If the prompt requires actually calculating anything, they will guess:

$ ollama run llama3.1:8b '1.34 ^ 10'
To calculate this, I'll raise 1.34 to the power of 10.

1.34^10 = 555,544.4

llama3.1:8b tries to evaluate your prompt by calculating the answer.

$ ollama run llama3.1:8b 'Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3'
The given equality `(SQR(2)*2 = (SQR(2))^3` isn't actually true. Let's break it down step by step.

First, let's evaluate `SQR(2)`. This is typically written as `√2`, which represents the square root of 2. The value of `√2` is approximately 1.41421 (though it's an irrational number).

Now, let's apply this to your original statement:

*   `SQR(2)*2`: We substitute `SQR(2)` with its value, getting `1.41421 * 2`, which equals `2.82843`.
*   `(SQR(2))^3`: Again substituting `SQR(2)` with its value and cubing the result, we get `(1.41421)^3 = 3.162277` (approximately).

These two expressions do not equal each other. The difference between them is quite noticeable.

It seems there was a misunderstanding in interpreting the given equality as true.

You can get better results by using a model that was trained in symbolic mathematics, because it can evaluate the equations without trying to calculate the results:

$ ollama run qwen2-math:7b-instruct-q4_K_M 'Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3'
 sure, let's break it down step by step.

First, let's define what SQR(2) means. SQR(2) is the square root of 2, which we can represent as √2. Now, let's substitute this into the equation:

The left side of the equation is:
SQR(2) * 2 = √2 * 2

The right side of the equation is:
(SQR(2))^3 = (√2)^3

Now, let's simplify both sides of the equation.

For the left side:
√2 * 2 = 2√2

For the right side:
(√2)^3 = √2 * √2 * √2

Since √2 * √2 = 2, we can substitute this into the equation:
√2 * √2 * √2 = 2 * √2

So, the right side simplifies to:
2√2

Now we see that both sides of the equation are equal:
2√2 = 2√2

Therefore, the original equation (SQR(2) * 2 = (SQR(2))^3) is true.
<!-- gh-comment-id:2350821356 --> @rick-github commented on GitHub (Sep 14, 2024): LLMs are poor at math. If the prompt requires actually calculating anything, they will guess: ```console $ ollama run llama3.1:8b '1.34 ^ 10' To calculate this, I'll raise 1.34 to the power of 10. 1.34^10 = 555,544.4 ``` llama3.1:8b tries to evaluate your prompt by calculating the answer. ```console $ ollama run llama3.1:8b 'Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3' The given equality `(SQR(2)*2 = (SQR(2))^3` isn't actually true. Let's break it down step by step. First, let's evaluate `SQR(2)`. This is typically written as `√2`, which represents the square root of 2. The value of `√2` is approximately 1.41421 (though it's an irrational number). Now, let's apply this to your original statement: * `SQR(2)*2`: We substitute `SQR(2)` with its value, getting `1.41421 * 2`, which equals `2.82843`. * `(SQR(2))^3`: Again substituting `SQR(2)` with its value and cubing the result, we get `(1.41421)^3 = 3.162277` (approximately). These two expressions do not equal each other. The difference between them is quite noticeable. It seems there was a misunderstanding in interpreting the given equality as true. ``` You can get better results by using a model that was trained in symbolic mathematics, because it can evaluate the equations without trying to calculate the results: ```console $ ollama run qwen2-math:7b-instruct-q4_K_M 'Can you explain why this mathematical equality is true: (SQR(2)*2 = (SQR(2))^3' sure, let's break it down step by step. First, let's define what SQR(2) means. SQR(2) is the square root of 2, which we can represent as √2. Now, let's substitute this into the equation: The left side of the equation is: SQR(2) * 2 = √2 * 2 The right side of the equation is: (SQR(2))^3 = (√2)^3 Now, let's simplify both sides of the equation. For the left side: √2 * 2 = 2√2 For the right side: (√2)^3 = √2 * √2 * √2 Since √2 * √2 = 2, we can substitute this into the equation: √2 * √2 * √2 = 2 * √2 So, the right side simplifies to: 2√2 Now we see that both sides of the equation are equal: 2√2 = 2√2 Therefore, the original equation (SQR(2) * 2 = (SQR(2))^3) is true. ```
GiteaMirror commented 2026-05-09 12:52:49 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Sep 17, 2024):

Hey @lsalamon , as @rick-github mentioned most LLMs aren't really great at this, but there are several (as he pointed out) which are better. I think this will change as more chain-of-thought type models come out, but we're not there yet.

I'm going to go ahead and close the issue since there's not much we can do until the model builders start building models which are better at math.

<!-- gh-comment-id:2356567407 --> @pdevine commented on GitHub (Sep 17, 2024): Hey @lsalamon , as @rick-github mentioned most LLMs aren't really great at this, but there are several (as he pointed out) which are better. I think this will change as more chain-of-thought type models come out, but we're not there yet. I'm going to go ahead and close the issue since there's not much we can do until the model builders start building models which are better at math.
GiteaMirror commented 2026-05-09 12:52:51 -05:00
Author
Owner
Copy Link

@lsalamon commented on GitHub (Sep 18, 2024):

Thanks for the recommendations.
I'm testing other versions and I've found that even the "llama3.1:70b" version doesn't answer the question correctly.
I thought it might be a mistake, but it is clear that we have a long way to go until we have a model that is capable of evaluating issues appropriately and in many areas of knowledge.

<!-- gh-comment-id:2359328424 --> @lsalamon commented on GitHub (Sep 18, 2024): Thanks for the recommendations. I'm testing other versions and I've found that even the "llama3.1:70b" version doesn't answer the question correctly. I thought it might be a mistake, but it is clear that we have a long way to go until we have a model that is capable of evaluating issues appropriately and in many areas of knowledge.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#81964
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?