github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #5731] SmolLM family #3568

New Issue
Closed
opened 2026-04-12 14:17:45 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-12 14:17:45 -05:00
Owner
Copy Link

Originally created by @DuckyBlender on GitHub (Jul 16, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5731

Originally assigned to: @BruceMacD on GitHub.

https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966

tried to add this myself, kept running into issues:

  • prompt template didn't work so I tried quantising with llama.cpp because maybe this is an issue with ollama auto-quantising
    here was my NOT WORKING prompt template for anyone trying:
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
  • llama.cpp didn't work because the model page doesn't have a tokenizer.model file
    Anyone more experienced than me, please help 😭

Also, the HF page recommends using the following parameters:

PARAMETER temperature 0.6
PARAMETER top_p 0.92
Originally created by @DuckyBlender on GitHub (Jul 16, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5731 Originally assigned to: @BruceMacD on GitHub. https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966 tried to add this myself, kept running into issues: - prompt template didn't work so I tried quantising with llama.cpp because maybe this is an issue with ollama auto-quantising here was my **NOT WORKING** prompt template for anyone trying: ``` <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant ``` - llama.cpp didn't work because the model page doesn't have a tokenizer.model file Anyone more experienced than me, please help :sob: Also, the HF page recommends using the following parameters: ``` PARAMETER temperature 0.6 PARAMETER top_p 0.92 ```
GiteaMirror added the model label 2026-04-12 14:17:45 -05:00
GiteaMirror commented 2026-04-12 14:17:48 -05:00
Author
Owner
Copy Link

@DuckyBlender commented on GitHub (Jul 17, 2024):

it seems that the HF inference on all of their model pages is broken...
image
This is roughly the same output I was getting with the chatMl prompt template (which I gave above)

<!-- gh-comment-id:2233197371 --> @DuckyBlender commented on GitHub (Jul 17, 2024): it seems that the HF inference on all of their model pages is broken... ![image](https://github.com/user-attachments/assets/3b160009-07ff-46ce-aa96-671ed9d687b8) This is roughly the same output I was getting with the chatMl prompt template (which I gave above)
GiteaMirror commented 2026-04-12 14:17:49 -05:00
Author
Owner
Copy Link

@DuckyBlender commented on GitHub (Jul 17, 2024):

This space seems to work. Maybe copy the parameters from here instead?

https://huggingface.co/spaces/vilarin/SmolLM-Instruct

<!-- gh-comment-id:2233206689 --> @DuckyBlender commented on GitHub (Jul 17, 2024): This space seems to work. Maybe copy the parameters from here instead? https://huggingface.co/spaces/vilarin/SmolLM-Instruct
GiteaMirror commented 2026-04-12 14:17:50 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Jul 17, 2024):

I'm looking at adding this model to the library at the moment. It can convert and run using the Ollama conversion. Here is how I ran it:

  1. Download the weights from HuggingFace.
  2. Create a Modelfile in the repo directory that I just downloaded:
FROM .
PARAMETER stop <|im_start|>
PARAMETER stop <|im_end|>
PARAMETER temperature 0.6
PARAMETER top_p 0.92

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>
"""
  1. Use the Ollama CLI to convert:
ollama create -f Modelfile smollm
  1. Run the model:
ollama run smollm

The model seems to be trained for very specific instructions in its instruct form:

> hello
Let $a,b \in (0,\infty)$ and $f$ be defined as: $$ f(x) = \begin{cases} x^{-1/2}\ln(1+ax^2) & \text{if } x>0 \\ 0 & \text{if } x=0 \end{cases}$$

The derivative of $f$ is given by: $$ f'(x) = \begin{cases} -\frac{a}{4}x^{-3/2}\ln(1+ax^2)+\frac{1}{2}(1+ax^2)\frac{2x}{1+ax^2} & \text{if } x>0 \\ 0 & \text{if } x=0 
\end{cases}$$

The second derivative of $f$ is: $$ f''(x) = \begin{cases} 
-\frac{3a}{8}\left(\ln(1+ax^2)-\frac{1}{ax^2}\right)+\frac{5}{4}(1+ax^2)\left(\frac{1-x^2}{ax^4}-2+\frac{7}{9}a\right) & \text{if } x>0 \\ 0 & \text{if } x=0 \end{cases}$$

The second derivative of $f$ is not equal to zero for all values of $x$. In fact, the second derivative has a local minimum at $x=-\frac{\ln(3)}{2a}$.

>>> List the steps to bake a chocolate cake from scratch.
Steps to bake a chocolate cake:

1. Gather ingredients (flour, sugar, cocoa powder, baking soda, salt, eggs, butter, milk, vanilla extract).
2. Preheat oven and prepare pans.
3. Combine dry ingredients in one bowl and wet ingredients in another.
4. Mix both mixtures together.
5. Pour batter into prepared pans.
6. Bake cake for 15-20 minutes or until toothpick inserted into center comes out clean.

Anyone have any tips or does this seem about right? Given the size of the model this is about what I'd expect.

<!-- gh-comment-id:2233890666 --> @BruceMacD commented on GitHub (Jul 17, 2024): I'm looking at adding this model to the library at the moment. It can convert and run using the Ollama conversion. Here is how I ran it: 1. Download the weights from HuggingFace. 2. Create a `Modelfile` in the repo directory that I just downloaded: ``` FROM . PARAMETER stop <|im_start|> PARAMETER stop <|im_end|> PARAMETER temperature 0.6 PARAMETER top_p 0.92 TEMPLATE """{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}{{ if .Prompt }}<|im_start|>user {{ .Prompt }}<|im_end|> {{ end }}<|im_start|>assistant {{ .Response }}<|im_end|> """ ``` 3. Use the Ollama CLI to convert: ``` ollama create -f Modelfile smollm ``` 4. Run the model: ``` ollama run smollm ``` --- The model seems to be trained for very specific instructions in its instruct form: ``` > hello Let $a,b \in (0,\infty)$ and $f$ be defined as: $$ f(x) = \begin{cases} x^{-1/2}\ln(1+ax^2) & \text{if } x>0 \\ 0 & \text{if } x=0 \end{cases}$$ The derivative of $f$ is given by: $$ f'(x) = \begin{cases} -\frac{a}{4}x^{-3/2}\ln(1+ax^2)+\frac{1}{2}(1+ax^2)\frac{2x}{1+ax^2} & \text{if } x>0 \\ 0 & \text{if } x=0 \end{cases}$$ The second derivative of $f$ is: $$ f''(x) = \begin{cases} -\frac{3a}{8}\left(\ln(1+ax^2)-\frac{1}{ax^2}\right)+\frac{5}{4}(1+ax^2)\left(\frac{1-x^2}{ax^4}-2+\frac{7}{9}a\right) & \text{if } x>0 \\ 0 & \text{if } x=0 \end{cases}$$ The second derivative of $f$ is not equal to zero for all values of $x$. In fact, the second derivative has a local minimum at $x=-\frac{\ln(3)}{2a}$. >>> List the steps to bake a chocolate cake from scratch. Steps to bake a chocolate cake: 1. Gather ingredients (flour, sugar, cocoa powder, baking soda, salt, eggs, butter, milk, vanilla extract). 2. Preheat oven and prepare pans. 3. Combine dry ingredients in one bowl and wet ingredients in another. 4. Mix both mixtures together. 5. Pour batter into prepared pans. 6. Bake cake for 15-20 minutes or until toothpick inserted into center comes out clean. ``` Anyone have any tips or does this seem about right? Given the size of the model this is about what I'd expect.
GiteaMirror commented 2026-04-12 14:17:51 -05:00
Author
Owner
Copy Link

@DuckyBlender commented on GitHub (Jul 17, 2024):

It can convert and run using the Ollama conversion.

I also ran it using the Ollama conversion but kept getting this output, so I thought maybe something is wrong? After all - ollama convertion doesn't work on everything.
image

<!-- gh-comment-id:2234335309 --> @DuckyBlender commented on GitHub (Jul 17, 2024): > It can convert and run using the Ollama conversion. I also ran it using the Ollama conversion but kept getting this output, so I thought maybe something is wrong? After all - ollama convertion doesn't work on everything. ![image](https://github.com/user-attachments/assets/c3fa7646-ba82-4bfe-9a12-f0e6517d6831)
GiteaMirror commented 2026-04-12 14:17:52 -05:00
Author
Owner
Copy Link

@tarbard commented on GitHub (Jul 23, 2024):

It's interesting, this model will answer all kinds of questions reasonably well but if you say "hi", "hello", "greetings" or anything like a normal conversation beginning then it responds with unrelated maths. I tried via system prompt to tell it to interact in a conversational style etc... but it still goes crazy when you say hi :-)

<!-- gh-comment-id:2244838629 --> @tarbard commented on GitHub (Jul 23, 2024): It's interesting, this model will answer all kinds of questions reasonably well but if you say "hi", "hello", "greetings" or anything like a normal conversation beginning then it responds with unrelated maths. I tried via system prompt to tell it to interact in a conversational style etc... but it still goes crazy when you say hi :-)
GiteaMirror commented 2026-04-12 14:17:53 -05:00
Author
Owner
Copy Link

@DuckyBlender commented on GitHub (Jul 29, 2024):

Added the SmolLM model on my own (with quants and all sizes)
https://ollama.com/duckyblender/smollm

Either this model is really strange or something is wrong. Maybe it's normal for such a small model.

<!-- gh-comment-id:2255805709 --> @DuckyBlender commented on GitHub (Jul 29, 2024): Added the SmolLM model on my own (with quants and all sizes) https://ollama.com/duckyblender/smollm Either this model is really strange or something is wrong. Maybe it's normal for such a small model.
GiteaMirror commented 2026-04-12 14:17:54 -05:00
Author
Owner
Copy Link

@ngxson commented on GitHub (Aug 8, 2024):

FYI, I've just joined HF and I asked my teammate the same question. Turns out, the dataset does not include normal daily talks, so it's normal for the model to behave like this. Hopefully there will be better data to train in the future.

<!-- gh-comment-id:2275280254 --> @ngxson commented on GitHub (Aug 8, 2024): FYI, I've just joined HF and I asked my teammate the same question. Turns out, the dataset does not include normal daily talks, so it's normal for the model to behave like this. Hopefully there will be better data to train in the future.
GiteaMirror commented 2026-04-12 14:17:55 -05:00
Author
Owner
Copy Link

@DuckyBlender commented on GitHub (Aug 20, 2024):

They released v0.2 which solves this issue. I'll be updating the model on my Ollama profile soon! (link above)

update: there is now an official ollama page https://ollama.com/library/smollm

<!-- gh-comment-id:2299134229 --> @DuckyBlender commented on GitHub (Aug 20, 2024): They released v0.2 which solves this issue. I'll be updating the model on my Ollama profile soon! (link above) update: there is now an official ollama page https://ollama.com/library/smollm
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3568
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?