github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #1805] which model to use for what's the root of 256256? #1029

New Issue
Closed
opened 2026-04-12 10:45:14 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 10:45:14 -05:00
Owner
Copy Link

Originally created by @dcasota on GitHub (Jan 5, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/1805

Originally created by @dcasota on GitHub (Jan 5, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/1805
GiteaMirror commented 2026-04-12 10:45:15 -05:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Jan 5, 2024):

Unless they have specially seen this result during training or they have access to an interpreter like ChatGPT has, there is no way they can calculate square roots of huge numbers.

They can give you a better answer if you guide them to use Bisection as they will often have seen the square roots of other values above and below. They can then use this to improve the bounds of other values they haven't been trained on and so on.

<!-- gh-comment-id:1878612555 --> @jukofyork commented on GitHub (Jan 5, 2024): Unless they have specially seen this result during training or they have access to an interpreter like ChatGPT has, there is no way they can calculate square roots of huge numbers. They can give you a better answer if you guide them to use Bisection as they will often have seen the square roots of other values above and below. They can then use this to improve the bounds of other values they haven't been trained on and so on.
GiteaMirror commented 2026-04-12 10:45:16 -05:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Jan 5, 2024):

Interestingly I've used the question "what is the square root of 1001" (or a similar number they've never seen before) to test the Wizard-Math-70b and Meta-Math-70b models and they get this hilariously wrong and reply with stuff like this:

"We know 15^2 is 225 and we know 16^2 is 256, so the Sqrt(1001) must lie between these 2 values. If we then... blah blah... 15.5ish"

But I reply that how can it be because we already know 20^2 is 400 so it must be much bigger and they both just don't get it and will die on their sword that it's 15.5ish.

The Llemma model on the other hand can use Bisection and get a reasonable answer, but because it's not been fine tuned for instruction or chat, it will give the answer then start hallucinating conversations between people on an imaginary message board discussing square roots and soon after go full on Battlestar Galactica "Hybrid" mode and start spouting pages of mathematical nonsense proofs! 🤣

The Mistral and Mixtral models are a little better but they also have lots of roots memorised and can often tell you the exact root of a 3-4 digit number to high precision.

My favourite question to ask is "How can you use Newton's Identities to efficiently calculate Elementary Symmetric Polynomials using Power Sums?". ChatGPT 4 can get this but it often has to use its Python interpreter to get the general formula. None of the open LLMs have ever got far and make a mess of it to variable degrees: some get confused straight away and start discussing something else Newton had his name on (like Newton's method, etc), some just memoize the e_1, e_2 and e_3 formulas straight off the wiki page. Others will try to use either the recursive formula or the direct formula that needs to use combinations and then proceed to make a complete word salad trying to rearrange the formulas. Tora-code tried to write a broken Python program and one even wrote a O(n!) algorithm that actually worked in theory and summed all possible subsets correctly. Again Llemma got the closest but then started hallucinating arXiv papers, message board discussions and email correspondences... 😢

I wish someone would fine tune Llemma properly as it does seem to be very strong.

I also suspect that Meta-Maths and Wizard-Math are somehow related as they get the same wrong answers often...

<!-- gh-comment-id:1878894040 --> @jukofyork commented on GitHub (Jan 5, 2024): Interestingly I've used the question "what is the square root of 1001" (or a similar number they've never seen before) to test the Wizard-Math-70b and Meta-Math-70b models and they get this hilariously wrong and reply with stuff like this: "We know 15^2 is 225 and we know 16^2 is 256, so the Sqrt(1001) must lie between these 2 values. If we then... blah blah... 15.5ish" But I reply that how can it be because we already know 20^2 is 400 so it must be much bigger and they both just don't get it and will die on their sword that it's 15.5ish. The Llemma model on the other hand can use Bisection and get a reasonable answer, but because it's not been fine tuned for instruction or chat, it will give the answer then start hallucinating conversations between people on an imaginary message board discussing square roots and soon after go full on Battlestar Galactica "Hybrid" mode and start spouting pages of mathematical nonsense proofs! 🤣 The Mistral and Mixtral models are a little better but they also have lots of roots memorised and can often tell you the exact root of a 3-4 digit number to high precision. My favourite question to ask is "How can you use Newton's Identities to efficiently calculate Elementary Symmetric Polynomials using Power Sums?". ChatGPT 4 can get this but it often has to use its Python interpreter to get the general formula. None of the open LLMs have ever got far and make a mess of it to variable degrees: some get confused straight away and start discussing something else Newton had his name on (like Newton's method, etc), some just memoize the e_1, e_2 and e_3 formulas straight off the wiki page. Others will try to use either the recursive formula or the direct formula that needs to use combinations and then proceed to make a complete word salad trying to rearrange the formulas. Tora-code tried to write a broken Python program and one even wrote a O(n!) algorithm that actually worked in theory and summed all possible subsets correctly. Again Llemma got the closest but then started hallucinating arXiv papers, message board discussions and email correspondences... 😢 I wish someone would fine tune Llemma properly as it does seem to be very strong. I also suspect that Meta-Maths and Wizard-Math are somehow related as they get the same wrong answers often...
GiteaMirror commented 2026-04-12 10:45:16 -05:00
Author
Owner
Copy Link

@umar-mq commented on GitHub (Jan 6, 2024):

Using LLMs for maths is always a bad idea. They are just not meant for that. Being able to do maths is a sign of super intelligence in AI, which unfortunately we have not yet reached.

If you want to find out the root of 256256, I would recommend to open up a calculator and type: sqrt(256256). Not only is it guaranteed to work, but it is also roughly 13 billion times faster.

Further resources

<!-- gh-comment-id:1879615017 --> @umar-mq commented on GitHub (Jan 6, 2024): Using LLMs for maths is always a bad idea. They are just not meant for that. Being able to do maths is a sign of super intelligence in AI, which unfortunately we have not yet reached. If you want to find out the root of 256256, I would recommend to open up a calculator and type: `sqrt(256256)`. Not only is it guaranteed to work, but it is also roughly 13 billion times faster. [Further resources](https://www.youtube.com/watch?v=lo82twBZT8Q)
GiteaMirror commented 2026-04-12 10:45:17 -05:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Jan 6, 2024):

I wish someone would fine tune Llemma properly as it does seem to be very strong.

Well I've actually got it somewhat working now. From:

https://old.reddit.com/r/learnmachinelearning/comments/17g7jof/why_does_it_keep_adding_random_text_after_it/

I think you need to change the prompt structure (before and after messages in the UI you’re using) to match the CodeLLama format.

System Prompt

You are an intelligent programming assistant.

User Message

Implement a linked list in C++

Assistant

and I then added "QED" and "█" as end tokens and it's working alot better now! It still likes to have imaginary conversation between "User" and "Assistant", but it's a lot more coherent and does actually stop now.

<!-- gh-comment-id:1879673681 --> @jukofyork commented on GitHub (Jan 6, 2024): > I wish someone would fine tune Llemma properly as it does seem to be very strong. Well I've actually got it somewhat working now. From: https://old.reddit.com/r/learnmachinelearning/comments/17g7jof/why_does_it_keep_adding_random_text_after_it/ > I think you need to change the prompt structure (before and after messages in the UI you’re using) to match the CodeLLama format. > >### System Prompt > >You are an intelligent programming assistant. > >### User Message > >Implement a linked list in C++ > >### Assistant and I then added "QED" and "█" as end tokens and it's working alot better now! It still likes to have imaginary conversation between "User" and "Assistant", but it's a lot more coherent and does actually stop now.
GiteaMirror commented 2026-04-12 10:45:17 -05:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Jan 6, 2024):

Using LLMs for maths is always a bad idea. They are just not meant for that. Being able to do maths is a sign of super intelligence in AI, which unfortunately we have not yet reached.

If you want to find out the root of 256256, I would recommend to open up a calculator and type: sqrt(256256). Not only is it guaranteed to work, but it is also roughly 13 billion times faster.

Further resources

I agree using for arithmetic type maths it's a terrible idea, but for some things it's not completely useless.

I remember watching a talk by Chris Bishop who is the author of one of the most important machine leaning textbooks, saying how even he couldn't understand how the Kalman Filter worked from the existing literature that was all about missile tracking, etc. Then one day it finally clicked and he realised the links between Hidden Markov Models and Kalman Filters and so on.

Now you can ask a LLM to explain stuff like this in terms to your own field and to fill in or guide you to any maths you are weak in to fully understand it. Making a total hash of the actual mathematical manipulation or arithmetic is not really a problem for things like this.

That talk was about 10-15 years ago and since then it's only got worse - especially in the machine learning literature, with people feeling obliged to embellish their papers with semi-irrelevant mathematics whilst simultaneously brushing important implementation details under the carpet.

<!-- gh-comment-id:1879685291 --> @jukofyork commented on GitHub (Jan 6, 2024): > Using LLMs for maths is always a bad idea. They are just not meant for that. Being able to do maths is a sign of super intelligence in AI, which unfortunately we have not yet reached. > > If you want to find out the root of 256256, I would recommend to open up a calculator and type: `sqrt(256256)`. Not only is it guaranteed to work, but it is also roughly 13 billion times faster. > > [Further resources](https://www.youtube.com/watch?v=lo82twBZT8Q) I agree using for arithmetic type maths it's a terrible idea, but for some things it's not completely useless. I remember watching a talk by Chris Bishop who is the author of one of the most important machine leaning textbooks, saying how even he couldn't understand how the Kalman Filter worked from the existing literature that was all about missile tracking, etc. Then one day it finally clicked and he realised the links between Hidden Markov Models and Kalman Filters and so on. Now you can ask a LLM to explain stuff like this in terms to your own field and to fill in or guide you to any maths you are weak in to fully understand it. Making a total hash of the actual mathematical manipulation or arithmetic is not really a problem for things like this. That talk was about 10-15 years ago and since then it's only got worse - especially in the machine learning literature, with people feeling obliged to embellish their papers with semi-irrelevant mathematics whilst simultaneously brushing important implementation details under the carpet.
GiteaMirror commented 2026-04-12 10:45:18 -05:00
Author
Owner
Copy Link

@mongolu commented on GitHub (Jan 6, 2024):

anyone knows about Wolfram?

https://www.wolframalpha.com/input?i=square+root+of+256256

<!-- gh-comment-id:1879736596 --> @mongolu commented on GitHub (Jan 6, 2024): anyone knows about [Wolfram](https://www.wolfram.com/wolfram-plugin-chatgpt/)? https://www.wolframalpha.com/input?i=square+root+of+256256
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1029
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?