github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-05 23:53:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3998] Phi-3-mini-128k no load #2476

New Issue
Closed
opened 2026-04-12 12:48:29 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 12:48:29 -05:00
Owner
Copy Link

Originally created by @bambooqj on GitHub (Apr 28, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3998

model download: https://huggingface.co/PrunaAI/Phi-3-mini-128k-instruct-GGUF-Imatrix-smashed

modfile:

FROM ./Phi-3-mini-128k-instruct.Q4_K_M.gguf

PARAMETER num_ctx 65536
PARAMETER num_keep 4
PARAMETER stop <|user|>
PARAMETER stop <|assistant|>
PARAMETER stop <|system|>
PARAMETER stop <|end|>
PARAMETER stop <|endoftext|>

TEMPLATE """
{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
{{ end }}<|assistant|>
{{ .Response }}<|end|>
"""

error: Error: llama runner process no longer running: 3221226505

Originally created by @bambooqj on GitHub (Apr 28, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3998 model download: `https://huggingface.co/PrunaAI/Phi-3-mini-128k-instruct-GGUF-Imatrix-smashed` modfile: ``` FROM ./Phi-3-mini-128k-instruct.Q4_K_M.gguf PARAMETER num_ctx 65536 PARAMETER num_keep 4 PARAMETER stop <|user|> PARAMETER stop <|assistant|> PARAMETER stop <|system|> PARAMETER stop <|end|> PARAMETER stop <|endoftext|> TEMPLATE """ {{ if .System }}<|system|> {{ .System }}<|end|> {{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}<|end|> {{ end }}<|assistant|> {{ .Response }}<|end|> """ ``` error: `Error: llama runner process no longer running: 3221226505`
GiteaMirror added the model label 2026-04-12 12:48:29 -05:00
GiteaMirror commented 2026-04-12 12:48:30 -05:00
Author
Owner
Copy Link

@amonpaike commented on GitHub (Apr 28, 2024):

I had the same error the first time I tried creating my own model from gguf.
Then I tried this other modified version that someone else created, and it worked.
https://ollama.com/herald/phi3-128k

<!-- gh-comment-id:2081382224 --> @amonpaike commented on GitHub (Apr 28, 2024): I had the same error the first time I tried creating my own model from gguf. Then I tried this other modified version that someone else created, and it worked. https://ollama.com/herald/phi3-128k
GiteaMirror commented 2026-04-12 12:48:31 -05:00
Author
Owner
Copy Link

@taozhiyuai commented on GitHub (Apr 29, 2024):

I had the same error the first time I tried creating my own model from gguf. Then I tried this other modified version that someone else created, and it worked. https://ollama.com/herald/phi3-128k

herald/phi3-128k works on my MAC. but fail to import phi3 gguf.

% ollama run Phi-3-mini-128k-instruct.Q8_0:latest Error: llama runner process no longer running: -1 error:error loading model architecture: unknown model architecture: 'phi3'

`FROM /Users/taozhiyu/Downloads/M-GGUF/Phi-3-mini-128K-Instruct/Phi-3-mini-128K-Instruct_Q8_0.gguf
TEMPLATE """{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
{{ end }}<|assistant|>
{{ .Response }}<|end|>
"""

PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
PARAMETER stop "<|system|>"
PARAMETER stop "<|end|>"
PARAMETER stop "<|endoftext|>"`

<!-- gh-comment-id:2081706674 --> @taozhiyuai commented on GitHub (Apr 29, 2024): > I had the same error the first time I tried creating my own model from gguf. Then I tried this other modified version that someone else created, and it worked. https://ollama.com/herald/phi3-128k herald/phi3-128k works on my MAC. but fail to import phi3 gguf. ` % ollama run Phi-3-mini-128k-instruct.Q8_0:latest Error: llama runner process no longer running: -1 error:error loading model architecture: unknown model architecture: 'phi3'` `FROM /Users/taozhiyu/Downloads/M-GGUF/Phi-3-mini-128K-Instruct/Phi-3-mini-128K-Instruct_Q8_0.gguf TEMPLATE """{{ if .System }}<|system|> {{ .System }}<|end|> {{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}<|end|> {{ end }}<|assistant|> {{ .Response }}<|end|> """ PARAMETER stop "<|user|>" PARAMETER stop "<|assistant|>" PARAMETER stop "<|system|>" PARAMETER stop "<|end|>" PARAMETER stop "<|endoftext|>"`
GiteaMirror commented 2026-04-12 12:48:31 -05:00
Author
Owner
Copy Link

@coder543 commented on GitHub (Apr 29, 2024):

The underlying library llama.cpp does not support Phi-3-128k yet, so there's nothing ollama can do to support it yet.

<!-- gh-comment-id:2082858332 --> @coder543 commented on GitHub (Apr 29, 2024): The underlying library `llama.cpp` [does not support Phi-3-128k yet](https://github.com/ggerganov/llama.cpp/issues/6849#issuecomment-2074899603), so there's nothing `ollama` can do to support it yet.
GiteaMirror commented 2026-04-12 12:48:31 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 1, 2024):

Initial support for phi3 was added in 0.1.32, and conversion should be working in 0.1.33. Please give the latest RC a try and let us know if you're still having problems.

https://github.com/ollama/ollama/releases

<!-- gh-comment-id:2089207631 --> @dhiltgen commented on GitHub (May 1, 2024): Initial support for phi3 was added in 0.1.32, and conversion should be working in 0.1.33. Please give the latest RC a try and let us know if you're still having problems. https://github.com/ollama/ollama/releases
GiteaMirror commented 2026-04-12 12:48:32 -05:00
Author
Owner
Copy Link

@coder543 commented on GitHub (May 1, 2024):

@dhiltgen are you talking about Phi3, or Phi3-128k?

ollama mentions nothing about the 128k context model: https://ollama.com/library/phi3/tags

<!-- gh-comment-id:2089216838 --> @coder543 commented on GitHub (May 1, 2024): @dhiltgen are you talking about Phi3, or Phi3-128k? ollama mentions nothing about the 128k context model: https://ollama.com/library/phi3/tags
GiteaMirror commented 2026-04-12 12:48:32 -05:00
Author
Owner
Copy Link

@taozhiyuai commented on GitHub (May 4, 2024):

Modelfile generated by "ollama show"

To build a new Modelfile based on this one, replace the FROM line with:

FROM phi-3-mini-128K-Instruct_q8_0

FROM /Users/taozhiyu/Downloads/M-GGUF/Phi-3-mini-128K-Instruct/Phi-3-mini-128K-Instruct_Q8_0.gguf
TEMPLATE """{{ if .System }}<|system|>
{{ .System }}<|end|>
{{ end }}{{ if .Prompt }}<|user|>
{{ .Prompt }}<|end|>
{{ end }}<|assistant|>
{{ .Response }}<|end|>
"""
SYSTEM """你是一位中文智能助理,始终用中文回复消息."""
PARAMETER stop "<|user|>"
PARAMETER stop "<|assistant|>"
PARAMETER stop "<|system|>"
PARAMETER stop "<|end|>"
PARAMETER stop "<|endoftext|>"

'3e5f4a42f1 Q8 % ollama create phi-3-mini-128k-instruct-q8-0 -f modelfile
transferring model data
pulling model
pulling manifest
Error: pull model manifest: 400'

@dhiltgen

<!-- gh-comment-id:2094104785 --> @taozhiyuai commented on GitHub (May 4, 2024): # Modelfile generated by "ollama show" # To build a new Modelfile based on this one, replace the FROM line with: # FROM phi-3-mini-128K-Instruct_q8_0 FROM /Users/taozhiyu/Downloads/M-GGUF/Phi-3-mini-128K-Instruct/Phi-3-mini-128K-Instruct_Q8_0.gguf TEMPLATE """{{ if .System }}<|system|> {{ .System }}<|end|> {{ end }}{{ if .Prompt }}<|user|> {{ .Prompt }}<|end|> {{ end }}<|assistant|> {{ .Response }}<|end|> """ SYSTEM """你是一位中文智能助理,始终用中文回复消息.""" PARAMETER stop "<|user|>" PARAMETER stop "<|assistant|>" PARAMETER stop "<|system|>" PARAMETER stop "<|end|>" PARAMETER stop "<|endoftext|>" '3e5f4a42f1 Q8 % ollama create phi-3-mini-128k-instruct-q8-0 -f modelfile transferring model data pulling model pulling manifest Error: pull model manifest: 400' @dhiltgen
GiteaMirror commented 2026-04-12 12:48:33 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Jul 5, 2024):

Hi @taozhiyuai are you still seeing this issue? If not, I will close for now, but do let me know and I can help out here

<!-- gh-comment-id:2210088170 --> @jmorganca commented on GitHub (Jul 5, 2024): Hi @taozhiyuai are you still seeing this issue? If not, I will close for now, but do let me know and I can help out here
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#2476
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?