github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #6327] [MERGED] convert safetensor adapters into GGUF #22621

New Issue
Closed
opened 2026-04-19 16:26:39 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-19 16:26:39 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/6327
Author: @pdevine
Created: 8/12/2024
Status: ✅ Merged
Merged: 8/23/2024
Merged by: @pdevine

Base: main ← Head: pdevine/gguf-adapters

📝 Commits (10+)

📊 Changes

16 files changed (+697 additions, -101 deletions)

View changed files

📝 cmd/cmd.go (+6 -0)
📝 convert/convert.go (+97 -14)
📝 convert/convert_bert.go (+9 -9)
📝 convert/convert_gemma.go (+9 -9)
📝 convert/convert_gemma2.go (+6 -6)
➕ convert/convert_gemma2_adapter.go (+91 -0)
📝 convert/convert_llama.go (+8 -8)
➕ convert/convert_llama_adapter.go (+169 -0)
📝 convert/convert_mixtral.go (+8 -8)
📝 convert/convert_phi3.go (+7 -7)
📝 convert/convert_test.go (+238 -24)
📝 convert/reader.go (+2 -0)
📝 llm/ggml.go (+8 -0)
📝 server/images.go (+6 -5)
📝 server/model.go (+30 -8)
📝 server/model_test.go (+3 -3)

📄 Description

This change converts a Safetensors based LoRA into GGUF and ties it w/ a base model. Only llama2/llama3/mistral/gemma2 will work initially. You can create the Modelfile to look like:

FROM llama3
ADAPTER /path/to/my/safetensor/adapter/directory

I'll add in some tests, but wanted to get this out so people could try it out.

Replaces #5524

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/6327 **Author:** [@pdevine](https://github.com/pdevine) **Created:** 8/12/2024 **Status:** ✅ Merged **Merged:** 8/23/2024 **Merged by:** [@pdevine](https://github.com/pdevine) **Base:** `main` ← **Head:** `pdevine/gguf-adapters` --- ### 📝 Commits (10+) - [`68b27b1`](https://github.com/ollama/ollama/commit/68b27b1e0797847c9b1380d8670f334a015c66bb) convert safetensor adapters into GGUF - [`484227d`](https://github.com/ollama/ollama/commit/484227db2de30dfa93d65a6e2c836b29bf7ea583) fix unittests - [`7f39bca`](https://github.com/ollama/ollama/commit/7f39bcae88397e40bb32ce01f146029637ac291c) comments - [`2ebb43b`](https://github.com/ollama/ollama/commit/2ebb43beee67543ee68c6eb28f688083c9b1ff14) feed the linter - [`6980456`](https://github.com/ollama/ollama/commit/69804565b35d3607158b9209c8d9b92745f99262) comments - [`75133d1`](https://github.com/ollama/ollama/commit/75133d15fa55a80d870f06f48c9c3d618f9a7122) swap out the converters - [`32abdce`](https://github.com/ollama/ollama/commit/32abdce43404ebc8c6c6fa6bb50f9c19aac135d1) gofumpt the llama converter - [`84a2154`](https://github.com/ollama/ollama/commit/84a2154dff63baa75399df9ead0e1ecfc0add8e5) add unittest + pass KV instead of baselayer - [`008e168`](https://github.com/ollama/ollama/commit/008e1682171bd06be4dbdd5827042af9ec395714) add adapter testdata - [`08c8d57`](https://github.com/ollama/ollama/commit/08c8d57dbe02a65a712bc9fcbf5953ce48fa2301) smaller test file ### 📊 Changes **16 files changed** (+697 additions, -101 deletions) <details> <summary>View changed files</summary> 📝 `cmd/cmd.go` (+6 -0) 📝 `convert/convert.go` (+97 -14) 📝 `convert/convert_bert.go` (+9 -9) 📝 `convert/convert_gemma.go` (+9 -9) 📝 `convert/convert_gemma2.go` (+6 -6) ➕ `convert/convert_gemma2_adapter.go` (+91 -0) 📝 `convert/convert_llama.go` (+8 -8) ➕ `convert/convert_llama_adapter.go` (+169 -0) 📝 `convert/convert_mixtral.go` (+8 -8) 📝 `convert/convert_phi3.go` (+7 -7) 📝 `convert/convert_test.go` (+238 -24) 📝 `convert/reader.go` (+2 -0) 📝 `llm/ggml.go` (+8 -0) 📝 `server/images.go` (+6 -5) 📝 `server/model.go` (+30 -8) 📝 `server/model_test.go` (+3 -3) </details> ### 📄 Description This change converts a Safetensors based LoRA into GGUF and ties it w/ a base model. Only llama2/llama3/mistral/gemma2 will work initially. You can create the Modelfile to look like: ``` FROM llama3 ADAPTER /path/to/my/safetensor/adapter/directory ``` I'll add in some tests, but wanted to get this out so people could try it out. Replaces #5524 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-19 16:26:39 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#22621
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?