github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #4322] [MERGED] Add support for IQ quantizarions #73750

New Issue
Closed
opened 2026-05-05 05:41:11 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-05 05:41:11 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/4322
Author: @BruceMacD
Created: 5/10/2024
Status: ✅ Merged
Merged: 5/23/2024
Merged by: @BruceMacD

Base: main ← Head: brucemacd/iq-quants

📝 Commits (1)

  • 643abf7 Add support for IQ1_S, IQ3_S, IQ2_S, IQ4_XS. IQ4_NL

📊 Changes

2 files changed (+72 additions, -10 deletions)

View changed files

📝 llm/filetype.go (+45 -5)
📝 llm/ggml.go (+27 -5)

📄 Description

This change allows importing IQ type gguf quantization with ollama create.

This change carries the commit from #3657 while moving its changes around to the refactored project structure.

❯ ./ollama create nous-hermes-2-mistral:IQ_4XS -f /Users/bruce/models/nous-hermes-2-mistral/Modelfile
transferring model data 
using existing layer sha256:737258efad6ba5cf7232de66715a26cadba67b0e4bdace5cf03cf49d1e4864a0 
creating new layer sha256:d7285065edcb87b4852f1144dd090812df1b00ade49f74e234066ea9407a14bc 
creating new layer sha256:d8ba2f9a17b3bbdeb5690efaa409b3fcb0b56296a777c7a69c78aa33bbddf182 
creating new layer sha256:b2c4ee0a7317771fcbe7413c369d72ea911c63e6f52b2b0d6298a5a14c8e4983 
writing manifest 
success 

❯ ./ollama run nous-hermes-2-mistral:IQ_4XS
>>> write some python

def print_fruits(fruits):
    for fruit in fruits:
        print(fruit)

Tested with:
IQ1_S
IQ1_M
IQ2_M
IQ3_XXS
IQ3_XS
IQ3_S
IQ4_NL
IQ4_XS

resolves #3622

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/4322 **Author:** [@BruceMacD](https://github.com/BruceMacD) **Created:** 5/10/2024 **Status:** ✅ Merged **Merged:** 5/23/2024 **Merged by:** [@BruceMacD](https://github.com/BruceMacD) **Base:** `main` ← **Head:** `brucemacd/iq-quants` --- ### 📝 Commits (1) - [`643abf7`](https://github.com/ollama/ollama/commit/643abf7ae86da7a71a1ab6ca2a283ecd65ab6956) Add support for IQ1_S, IQ3_S, IQ2_S, IQ4_XS. IQ4_NL ### 📊 Changes **2 files changed** (+72 additions, -10 deletions) <details> <summary>View changed files</summary> 📝 `llm/filetype.go` (+45 -5) 📝 `llm/ggml.go` (+27 -5) </details> ### 📄 Description This change allows importing `IQ` type gguf quantization with `ollama create`. This change carries the commit from #3657 while moving its changes around to the refactored project structure. ```bash ❯ ./ollama create nous-hermes-2-mistral:IQ_4XS -f /Users/bruce/models/nous-hermes-2-mistral/Modelfile transferring model data using existing layer sha256:737258efad6ba5cf7232de66715a26cadba67b0e4bdace5cf03cf49d1e4864a0 creating new layer sha256:d7285065edcb87b4852f1144dd090812df1b00ade49f74e234066ea9407a14bc creating new layer sha256:d8ba2f9a17b3bbdeb5690efaa409b3fcb0b56296a777c7a69c78aa33bbddf182 creating new layer sha256:b2c4ee0a7317771fcbe7413c369d72ea911c63e6f52b2b0d6298a5a14c8e4983 writing manifest success ❯ ./ollama run nous-hermes-2-mistral:IQ_4XS >>> write some python def print_fruits(fruits): for fruit in fruits: print(fruit) ``` Tested with: `IQ1_S` `IQ1_M` `IQ2_M` `IQ3_XXS` `IQ3_XS` `IQ3_S` `IQ4_NL` `IQ4_XS` resolves #3622 --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-05 05:41:11 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#73750
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?