github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-23 14:13:13 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[PR #10807] [MERGED] fix: mllama quality #44620

New Issue
Closed
opened 2026-04-25 00:13:24 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-25 00:13:24 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/10807
Author: @mxyng
Created: 5/21/2025
Status: ✅ Merged
Merged: 5/22/2025
Merged by: @mxyng

Base: main ← Head: mxyng/mllama

📝 Commits (2)

📊 Changes

2 files changed (+40 additions, -37 deletions)

View changed files

📝 convert/convert_mllama.go (+37 -18)
📝 model/models/mllama/model_vision.go (+3 -19)

📄 Description

this change contains a series of fixes for mllama impacting both the model itself and the graph

model changes

  • attn_gate and ffn_gate did not have tanh applied to them
  • attn_q and attn_k tensors for the vision model did not have their attention heads swapped as is tradition for llama models

graph changes

  • during refactoring, ffn_gate was moved out of mllama.VisionMLP into its parent but was not applied in the correct spot
  • remove mllama.VisionSelfAttention.Gate which is unused
  • use nn.Attention

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/10807 **Author:** [@mxyng](https://github.com/mxyng) **Created:** 5/21/2025 **Status:** ✅ Merged **Merged:** 5/22/2025 **Merged by:** [@mxyng](https://github.com/mxyng) **Base:** `main` ← **Head:** `mxyng/mllama` --- ### 📝 Commits (2) - [`9f050e7`](https://github.com/ollama/ollama/commit/9f050e7c531c0fed68e0e9fcd5c64580175d53a2) fix mllama convert - [`5b976a8`](https://github.com/ollama/ollama/commit/5b976a8e340ae7e5da9cb132d15ec1215addfccd) fix mllama ### 📊 Changes **2 files changed** (+40 additions, -37 deletions) <details> <summary>View changed files</summary> 📝 `convert/convert_mllama.go` (+37 -18) 📝 `model/models/mllama/model_vision.go` (+3 -19) </details> ### 📄 Description this change contains a series of fixes for mllama impacting both the model itself and the graph model changes - `attn_gate` and `ffn_gate` did not have `tanh` applied to them - `attn_q` and `attn_k` tensors for the vision model did not have their attention heads swapped as is tradition for llama models graph changes - during refactoring, ffn_gate was moved out of `mllama.VisionMLP` into its parent but was not applied in the correct spot - remove `mllama.VisionSelfAttention.Gate` which is unused - use `nn.Attention` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-25 00:13:24 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#44620
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?