github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 17:49:24 -05:00
Code Issues 14.5k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #775] How to create my own model with GGUF file?? #46880

New Issue
Closed
opened 2026-04-28 01:27:08 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-28 01:27:08 -05:00
Owner
Copy Link

Originally created by @wertyac on GitHub (Oct 13, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/775

I follow the instructions to create my own model. Bu failed.
The steps are as following:

  1. I have a own gguf file in /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf .

  2. And I create the Modefile in /opt/cllama2-13b-16k:
    ################################
    FROM /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf

  3. I create my own cllama2 with:
    root@144server:/opt/cllama2-13b-16k# ollama create cllama2-13b-16k -f ./Modelfile
    parsing modelfile
    looking for model
    pulling model file
    ⠧ pulling manifest Error: pull model manifest: model not found

How to create my own model?? It seems the instructions are not OK.

Originally created by @wertyac on GitHub (Oct 13, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/775 I follow the instructions to create my own model. Bu failed. The steps are as following: 1) I have a own gguf file in /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf . 2) And I create the Modefile in /opt/cllama2-13b-16k: ################################ FROM /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf 3) I create my own cllama2 with: root@144server:/opt/cllama2-13b-16k# ollama create cllama2-13b-16k -f ./Modelfile parsing modelfile looking for model pulling model file ⠧ pulling manifest Error: pull model manifest: model not found How to create my own model?? It seems the instructions are not OK.
GiteaMirror commented 2026-04-28 01:27:17 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Oct 13, 2023):

Based on the examples, I'm guessing you're using this model from Hugging Face?

I've tried your steps and it's worked for me. Here are the exact steps, as root:

  1. Create the directory /opt/cllama2-13b-16k as root
  2. cd /opt/cllama2-13b-16k
  3. Download the Q4_0 from Hugging Face: 
    # wget https://huggingface.co/shaowenchen/chinese-alpaca-2-13b-16k-gguf/resolve/main/chinese-alpaca-2-13b-16k.Q4_0.gguf
  4. Create the Modelfile 
    FROM /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf
  5. Call ollama create 
    # ollama create calpaca -f ./Modelfile 
parsing modelfile    
looking for model    
creating model layer    
creating config layer    
using already created layer sha256:81b2578c6456219373ddc8515bee1b970bf2391d534aaedecdca4de84f3d3063    
using already created layer sha256:b3bff570b0be7d987a67c0d09db81e4de564b91f3ca5efb29d6feed6b7d5714d    
writing manifest    
removing any unused layers    
success

I have a couple of follow up questions:

  1. What version of Ollama are you using? You can check with ollama -v
  2. How did you install Ollama?
  3. What are the permissions of /opt/cllama2-13b-16k
<!-- gh-comment-id:1762382948 --> @mxyng commented on GitHub (Oct 13, 2023): Based on the examples, I'm guessing you're using [this](https://huggingface.co/shaowenchen/chinese-alpaca-2-13b-16k-gguf) model from Hugging Face? I've tried your steps and it's worked for me. Here are the exact steps, as root: 1. Create the directory `/opt/cllama2-13b-16k` as root 2. `cd /opt/cllama2-13b-16k` 3. Download the Q4_0 from Hugging Face: ``` # wget https://huggingface.co/shaowenchen/chinese-alpaca-2-13b-16k-gguf/resolve/main/chinese-alpaca-2-13b-16k.Q4_0.gguf ``` 4. Create the Modelfile ``` FROM /opt/cllama2-13b-16k/chinese-alpaca-2-13b-16k.Q4_0.gguf ``` 5. Call `ollama create` ``` # ollama create calpaca -f ./Modelfile parsing modelfile looking for model creating model layer creating config layer using already created layer sha256:81b2578c6456219373ddc8515bee1b970bf2391d534aaedecdca4de84f3d3063 using already created layer sha256:b3bff570b0be7d987a67c0d09db81e4de564b91f3ca5efb29d6feed6b7d5714d writing manifest removing any unused layers success ``` I have a couple of follow up questions: 1. What version of Ollama are you using? You can check with `ollama -v` 2. How did you install Ollama? 3. What are the permissions of `/opt/cllama2-13b-16k`
GiteaMirror commented 2026-04-28 01:27:19 -05:00
Author
Owner
Copy Link

@saikatkumardey commented on GitHub (Oct 14, 2023):

I recently created a custom model and pushed it to the registry as well.

I wanted to play with tinyllama, a 1.1B parameter model.

First, I downloaded it on my local machine.

Then, I created a modelfile like this:

FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are a helpful assistant."""

Next, I ran the following command to create a custom model:

ollama create saikatkumardey/tinyllama:latest -f modelfile

I pushed the model to ollama.ai registry. You can find it here.

To use the model now, simply run:

ollama run saikatkumardey/tinyllama

<!-- gh-comment-id:1762965343 --> @saikatkumardey commented on GitHub (Oct 14, 2023): I recently created a custom model and pushed it to the registry as well. I wanted to play with [tinyllama](https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGUF/blob/main/tinyllama-1.1b-chat-v0.3.Q6_K.gguf), a 1.1B parameter model. First, I downloaded it on my local machine. Then, I created a modelfile like this: ``` FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf PARAMETER temperature 0.7 PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>" TEMPLATE """ <|im_start|>system {{ .System }}<|im_end|> <|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant """ SYSTEM """You are a helpful assistant.""" ``` Next, I ran the following command to create a custom model: `ollama create saikatkumardey/tinyllama:latest -f modelfile` I pushed the model to ollama.ai registry. You can find it [here](https://ollama.ai/saikatkumardey/tinyllama). To use the model now, simply run: `ollama run saikatkumardey/tinyllama`
GiteaMirror commented 2026-04-28 01:27:20 -05:00
Author
Owner
Copy Link

@LiuYang328 commented on GitHub (Oct 16, 2023):

I recently created a custom model and pushed it to the registry as well.

I wanted to play with tinyllama, a 1.1B parameter model.

First, I downloaded it on my local machine.

Then, I created a modelfile like this:

FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are a helpful assistant."""

Next, I ran the following command to create a custom model:

ollama create saikatkumardey/tinyllama:latest -f modelfile

I pushed the model to ollama.ai registry. You can find it here.

To use the model now, simply run:

ollama run saikatkumardey/tinyllama

the modelfie is too abstract, can you show me a more detailed example? (maybe use llama2)

<!-- gh-comment-id:1764233621 --> @LiuYang328 commented on GitHub (Oct 16, 2023): > I recently created a custom model and pushed it to the registry as well. > > I wanted to play with [tinyllama](https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGUF/blob/main/tinyllama-1.1b-chat-v0.3.Q6_K.gguf), a 1.1B parameter model. > > First, I downloaded it on my local machine. > > Then, I created a modelfile like this: > > ``` > FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf > PARAMETER temperature 0.7 > PARAMETER stop "<|im_start|>" > PARAMETER stop "<|im_end|>" > TEMPLATE """ > <|im_start|>system > {{ .System }}<|im_end|> > <|im_start|>user > {{ .Prompt }}<|im_end|> > <|im_start|>assistant > """ > SYSTEM """You are a helpful assistant.""" > ``` > > Next, I ran the following command to create a custom model: > > `ollama create saikatkumardey/tinyllama:latest -f modelfile` > > I pushed the model to ollama.ai registry. You can find it [here](https://ollama.ai/saikatkumardey/tinyllama). > > To use the model now, simply run: > > `ollama run saikatkumardey/tinyllama` the modelfie is too abstract, can you show me a more detailed example? (maybe use llama2)
GiteaMirror commented 2026-04-28 01:27:21 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Oct 24, 2023):

@wertyac thanks for the issue! Check out https://github.com/jmorganca/ollama/blob/main/docs/import.md#importing-gguf which walks through create a custom model from GGUF

<!-- gh-comment-id:1776278028 --> @jmorganca commented on GitHub (Oct 24, 2023): @wertyac thanks for the issue! Check out https://github.com/jmorganca/ollama/blob/main/docs/import.md#importing-gguf which walks through create a custom model from GGUF
GiteaMirror commented 2026-04-28 01:27:22 -05:00
Author
Owner
Copy Link

@Ross-Fan commented on GitHub (Apr 10, 2024):

I have met the same question
llamaproj % ollama create phi2 -f Modelfile transferring model data pulling model pulling manifest Error: pull model manifest: Get "https://./v2/llamaproj/ggml-model-q4_0.gguf/manifests/latest": dial tcp: lookup .: no such host
but when I checked the Modelfile, I found the path to the gguf is wrong, so ,change to the right path , then worked
llamaproj % ollama create phis2 -f Modelfile transferring model data creating model layer using already created layer sha256:fd506d24a4beexxxxxxxxxxxxxxxx7a74b467badc5727ca writing layer sha256:692e4d9xxxxxxe9e2xxxxxxxxxxxxxxxa78d4b99faf writing manifest success

<!-- gh-comment-id:2046657195 --> @Ross-Fan commented on GitHub (Apr 10, 2024): I have met the same question `llamaproj % ollama create phi2 -f Modelfile transferring model data pulling model pulling manifest Error: pull model manifest: Get "https://./v2/llamaproj/ggml-model-q4_0.gguf/manifests/latest": dial tcp: lookup .: no such host` but when I checked the Modelfile, I found the path to the gguf is wrong, so ,change to the right path , then worked `llamaproj % ollama create phis2 -f Modelfile transferring model data creating model layer using already created layer sha256:fd506d24a4beexxxxxxxxxxxxxxxx7a74b467badc5727ca writing layer sha256:692e4d9xxxxxxe9e2xxxxxxxxxxxxxxxa78d4b99faf writing manifest success`
GiteaMirror commented 2026-04-28 01:27:23 -05:00
Author
Owner
Copy Link

@whoamimi commented on GitHub (Sep 5, 2024):

I recently created a custom model and pushed it to the registry as well.
I wanted to play with tinyllama, a 1.1B parameter model.
First, I downloaded it on my local machine.
Then, I created a modelfile like this:

FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf
PARAMETER temperature 0.7
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are a helpful assistant."""

Next, I ran the following command to create a custom model:
ollama create saikatkumardey/tinyllama:latest -f modelfile
I pushed the model to ollama.ai registry. You can find it here.
To use the model now, simply run:
ollama run saikatkumardey/tinyllama

the modelfie is too abstract, can you show me a more detailed example? (maybe use llama2)

Regarding the defined template, is it the example to be returned or is it the default template being passed to the model?

<!-- gh-comment-id:2331854952 --> @whoamimi commented on GitHub (Sep 5, 2024): > > I recently created a custom model and pushed it to the registry as well. > > I wanted to play with [tinyllama](https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v0.3-GGUF/blob/main/tinyllama-1.1b-chat-v0.3.Q6_K.gguf), a 1.1B parameter model. > > First, I downloaded it on my local machine. > > Then, I created a modelfile like this: > > ``` > > FROM models/tinyllama-1.1b-chat-v0.3.Q6_K.gguf > > PARAMETER temperature 0.7 > > PARAMETER stop "<|im_start|>" > > PARAMETER stop "<|im_end|>" > > TEMPLATE """ > > <|im_start|>system > > {{ .System }}<|im_end|> > > <|im_start|>user > > {{ .Prompt }}<|im_end|> > > <|im_start|>assistant > > """ > > SYSTEM """You are a helpful assistant.""" > > ``` > > > > > > > > > > > > > > > > > > > > > > > > Next, I ran the following command to create a custom model: > > `ollama create saikatkumardey/tinyllama:latest -f modelfile` > > I pushed the model to ollama.ai registry. You can find it [here](https://ollama.ai/saikatkumardey/tinyllama). > > To use the model now, simply run: > > `ollama run saikatkumardey/tinyllama` > > the modelfie is too abstract, can you show me a more detailed example? (maybe use llama2) Regarding the defined template, is it the example to be returned or is it the default template being passed to the model?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#46880
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?