github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 17:49:24 -05:00
Code Issues 14.6k Packages Projects Releases 122 Wiki Activity

[PR #11508] fix: quantization on non-macos systems #24100

New Issue
Open
opened 2026-04-19 17:23:07 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-19 17:23:07 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/11508
Author: @mxyng
Created: 7/23/2025
Status: 🔄 Open

Base: main ← Head: mxyng/quant

📝 Commits (3)

📊 Changes

4 files changed (+95 additions, -33 deletions)

View changed files

📝 ml/backend/ggml/ggml/src/ggml.go (+1 -0)
📝 ml/backend/ggml/quantization.go (+27 -29)
📝 server/quantization.go (+13 -4)
📝 server/quantization_test.go (+54 -0)

📄 Description

This fixes and adds a test for quantization on non-Darwin system. Previously quantizing on Linux and Window would produce corrupt files due to values being mapped incorrectly.

Also update Quantize to return a iter.Seq[[]byte]. This allows portions of the tensor to be quantized at a time (which it was already doing before) which reduces overall memory usage

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/11508 **Author:** [@mxyng](https://github.com/mxyng) **Created:** 7/23/2025 **Status:** 🔄 Open **Base:** `main` ← **Head:** `mxyng/quant` --- ### 📝 Commits (3) - [`2a03498`](https://github.com/ollama/ollama/commit/2a03498bbb1d41fa413ef404cd106032118f879e) iter quant - [`362bf09`](https://github.com/ollama/ollama/commit/362bf0901f444977c323cef246af6a82c680702d) cleanup quantization - [`c8b1f9e`](https://github.com/ollama/ollama/commit/c8b1f9e1a1595ea96bc61649bcbe10be513b563a) fix quantization ### 📊 Changes **4 files changed** (+95 additions, -33 deletions) <details> <summary>View changed files</summary> 📝 `ml/backend/ggml/ggml/src/ggml.go` (+1 -0) 📝 `ml/backend/ggml/quantization.go` (+27 -29) 📝 `server/quantization.go` (+13 -4) 📝 `server/quantization_test.go` (+54 -0) </details> ### 📄 Description This fixes and adds a test for quantization on non-Darwin system. Previously quantizing on Linux and Window would produce corrupt files due to values being mapped incorrectly. Also update `Quantize` to return a `iter.Seq[[]byte]`. This allows portions of the tensor to be quantized at a time (which it was already doing before) which reduces overall memory usage --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-19 17:23:07 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#24100
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?