[GH-ISSUE #1730] MLX backend #63022

New Issue

GiteaMirror · 2026-05-03T11:17:44-05:00

GiteaMirror commented

2026-05-03 11:17:44 -05:00

Originally created by @ageorgios on GitHub (Dec 27, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/1730

Can ollama be converted to use MLX from Apple as backend for the models ?

Originally created by @ageorgios on GitHub (Dec 27, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/1730 Can ollama be converted to use MLX from Apple as backend for the models ?

GiteaMirror added the feature request label 2026-05-03 11:17:45 -05:00

GiteaMirror commented

2026-05-03 11:17:48 -05:00

@Josecodesalot commented on GitHub (Dec 31, 2023):

This Please!

@Josecodesalot commented on GitHub (Dec 31, 2023): This Please!

GiteaMirror commented

2026-05-03 11:17:49 -05:00

@easp commented on GitHub (Jan 2, 2024):

What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet.

@easp commented on GitHub (Jan 2, 2024): What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet.

GiteaMirror commented

2026-05-03 11:17:49 -05:00

@KernelBypass commented on GitHub (Jan 10, 2024):

Found these benchmarks:
https://medium.com/@andreask_75652/benchmarking-apples-mlx-vs-llama-cpp-bbbebdc18416

Seems like MLX is indeed slower than the llama.cpp masterpiece, at least for now. I did not verify though.

@KernelBypass commented on GitHub (Jan 10, 2024): Found these benchmarks: https://medium.com/@andreask_75652/benchmarking-apples-mlx-vs-llama-cpp-bbbebdc18416 Seems like MLX is indeed slower than the llama.cpp masterpiece, at least for now. I did not verify though.

GiteaMirror commented

2026-05-03 11:17:50 -05:00

@Edu126 commented on GitHub (Jan 23, 2024):

This would be very nice!
and not only for text generation, Image/Multimodal would be boosted too.

@Edu126 commented on GitHub (Jan 23, 2024): This would be very nice! and not only for text generation, Image/Multimodal would be boosted too.

GiteaMirror commented

2026-05-03 11:17:51 -05:00

@JimmyLv commented on GitHub (Apr 20, 2024):

someone made this https://github.com/kspviswa/PyOMlx

@JimmyLv commented on GitHub (Apr 20, 2024): someone made this https://github.com/kspviswa/PyOMlx

GiteaMirror commented

2026-05-03 11:17:52 -05:00

@magnusviri commented on GitHub (May 4, 2024):

Ollama is awesome and does so many things and some of us want to play with mlx models.

@magnusviri commented on GitHub (May 4, 2024): Ollama is awesome and does so many things and some of us want to play with mlx models.

GiteaMirror commented

2026-05-03 11:17:53 -05:00

@angelo-cortez commented on GitHub (May 30, 2024):

bump

@angelo-cortez commented on GitHub (May 30, 2024): bump

GiteaMirror commented

2026-05-03 11:17:56 -05:00

@mxyng commented on GitHub (May 30, 2024):

Commenting here to say we're aware of MLX. I've been working on a prototype but I can't give an ETA at for MLX support at this time

@mxyng commented on GitHub (May 30, 2024): Commenting here to say we're aware of MLX. I've been working on a prototype but I can't give an ETA at for MLX support at this time

GiteaMirror commented

2026-05-03 11:17:59 -05:00

@qdrddr commented on GitHub (Jun 28, 2024):

Related to this Apple CoreML support to utilize Apple Neural Engine (ANE) alongside GPU & CPU:
https://github.com/ollama/ollama/issues/3898

@qdrddr commented on GitHub (Jun 28, 2024): Related to this Apple CoreML support to utilize Apple Neural Engine (ANE) alongside GPU & CPU: https://github.com/ollama/ollama/issues/3898

GiteaMirror commented

2026-05-03 11:18:02 -05:00

@ibehnam commented on GitHub (Aug 20, 2024):

Any updates on this? MLX is now faster than Llama.cpp on Mac.

@ibehnam commented on GitHub (Aug 20, 2024): Any updates on this? MLX is now faster than Llama.cpp on Mac.

GiteaMirror commented

2026-05-03 11:18:03 -05:00

@garhbod commented on GitHub (Aug 21, 2024):

Any progress @mxyng? Is this a seperate project that other could contribute to?

@garhbod commented on GitHub (Aug 21, 2024): Any progress @mxyng? Is this a seperate project that other could contribute to?

GiteaMirror commented

2026-05-03 11:18:03 -05:00

@nicarq commented on GitHub (Aug 29, 2024):

it would be awesome. MLX is moving really fast and it would make sense that it would be the best tool long-term to run models on Apple's hardware.

@nicarq commented on GitHub (Aug 29, 2024): it would be awesome. MLX is moving really fast and it would make sense that it would be the best tool long-term to run models on Apple's hardware.

GiteaMirror commented

2026-05-03 11:18:04 -05:00

@parthpat12 commented on GitHub (Sep 9, 2024):

Please add support for MLX! Any update @mxyng?

@parthpat12 commented on GitHub (Sep 9, 2024): Please add support for MLX! Any update @mxyng?

GiteaMirror commented

2026-05-03 11:18:05 -05:00

@ivanfioravanti commented on GitHub (Sep 15, 2024):

MLX support would be awesome!!!

@ivanfioravanti commented on GitHub (Sep 15, 2024): MLX support would be awesome!!!

GiteaMirror commented

2026-05-03 11:18:05 -05:00

@czzarr commented on GitHub (Sep 15, 2024):

indeed it would be!

@czzarr commented on GitHub (Sep 15, 2024): indeed it would be!

GiteaMirror commented

2026-05-03 11:18:06 -05:00

@vietvudanh commented on GitHub (Oct 2, 2024):

MLX would support vision models too.

@vietvudanh commented on GitHub (Oct 2, 2024): MLX would support vision models too.

GiteaMirror commented

2026-05-03 11:18:06 -05:00

@Bigsy commented on GitHub (Oct 9, 2024):

Seems the new MLX backend in LMStudio is providing some real benefits especially in regards to memory consumption. Would be great to get support in Ollama.

@Bigsy commented on GitHub (Oct 9, 2024): Seems the new MLX backend in LMStudio is providing some real benefits especially in regards to memory consumption. Would be great to get support in Ollama.

GiteaMirror commented

2026-05-03 11:18:08 -05:00

@hg0428 commented on GitHub (Oct 10, 2024):

I have been testing the MLX backend in LM Studio, and I have found it to be on average 40% faster for inference than Ollama using the same exact settings with the same model at the same precision.
I am using M3 Max 36GB memory.

@hg0428 commented on GitHub (Oct 10, 2024): I have been testing the MLX backend in LM Studio, and I have found it to be on average 40% faster for inference than Ollama using the same exact settings with the same model at the same precision. I am using M3 Max 36GB memory.

GiteaMirror commented

2026-05-03 11:18:10 -05:00

@robbiemu commented on GitHub (Oct 18, 2024):

I have been testing the MLX backend in LM Studio, and I have found it to be on average 40% faster for inference than Ollama using the same exact settings with the same model at the same precision. I am using M3 Max 36GB memory.

I've seen numbers, admittedly a couple months ago, around 20% faster. Can you share a bit more - what models/context/settings? what iogpu.wired_limit_mb? etc

20% is still 20% more than Im doing currently :D

I'm not sure how good of an idea it is to have Ollama add a lot of features only available to some people .. but it does have some NVIDIA exclusive (or nvidia vs cpu only) stuff at least.

@robbiemu commented on GitHub (Oct 18, 2024): > I have been testing the MLX backend in LM Studio, and I have found it to be on average 40% faster for inference than Ollama using the same exact settings with the same model at the same precision. I am using M3 Max 36GB memory. I've seen numbers, admittedly a couple months ago, around 20% faster. Can you share a bit more - what models/context/settings? what iogpu.wired_limit_mb? etc 20% is still 20% more than Im doing currently :D I'm not sure how good of an idea it is to have Ollama add a lot of features only available to some people .. but it does have some NVIDIA exclusive (or nvidia vs cpu only) stuff at least.

GiteaMirror commented

2026-05-03 11:18:11 -05:00

@CharafChnioune commented on GitHub (Oct 20, 2024):

So any plans to add mlx like lmstudio? Mlx supports multi model and is faster now. Llama.cpp is sort of death since the stoped vision

@CharafChnioune commented on GitHub (Oct 20, 2024): So any plans to add mlx like lmstudio? Mlx supports multi model and is faster now. Llama.cpp is sort of death since the stoped vision

GiteaMirror commented

2026-05-03 11:18:12 -05:00

@twalderman commented on GitHub (Oct 23, 2024):

Given that there are less options with MLX for models but the ones that are available are good workhorses, it would be a huge benefit to have Ollama support mlx. As others have stated, LM studio supports MLX and the performance is great however Ollama still supports a wider range of templates and potentially upcoming support for more sampler options. Having one solution is ideal for Apple Silicon.

@twalderman commented on GitHub (Oct 23, 2024): Given that there are less options with MLX for models but the ones that are available are good workhorses, it would be a huge benefit to have Ollama support mlx. As others have stated, LM studio supports MLX and the performance is great however Ollama still supports a wider range of templates and potentially upcoming support for more sampler options. Having one solution is ideal for Apple Silicon.

GiteaMirror commented

2026-05-03 11:18:14 -05:00

@ice6 commented on GitHub (Oct 30, 2024):

it is good to support this :) most important keep ollama steady and fast!

@ice6 commented on GitHub (Oct 30, 2024): it is good to support this :) most important keep `ollama` steady and fast!

GiteaMirror commented

2026-05-03 11:18:16 -05:00

@nercone-dev commented on GitHub (Oct 31, 2024):

On the MacBook Air (Apple M3 Normal), it is now faster.
For example, it was faster even when running 13B Codellama.
This is probably a technology that can be optimized for Apple silicon (especially M3 and later).
So it would be better to implement it.

https://github.com/user-attachments/assets/6aadc898-1c62-43c8-91d6-8c2308db603c

https://github.com/user-attachments/assets/e65e373f-a2bd-4089-82d0-66c3cbab4db5

Model

MacBook Air 13-inch (2024, M3)

CPU

Apple M3

Memory (RAM)

16GB

@nercone-dev commented on GitHub (Oct 31, 2024): On the MacBook Air (Apple M3 Normal), it is now **faster**. For example, it was **faster even when running 13B Codellama**. This is probably a technology that can be optimized for Apple silicon (especially M3 and later). So it would be better to implement it. https://github.com/user-attachments/assets/6aadc898-1c62-43c8-91d6-8c2308db603c https://github.com/user-attachments/assets/e65e373f-a2bd-4089-82d0-66c3cbab4db5 ## Model MacBook Air 13-inch (2024, M3) ## CPU Apple M3 ## Memory (RAM) 16GB

GiteaMirror commented

2026-05-03 11:18:19 -05:00

@hg0428 commented on GitHub (Oct 31, 2024):

On my hardware, MLX runs an average of 40% faster than Llama.cpp (actual percentage varies from 38%-42%. 40% is the average over many tests).

@hg0428 commented on GitHub (Oct 31, 2024): On my hardware, MLX runs an average of 40% faster than Llama.cpp (actual percentage varies from 38%-42%. 40% is the average over many tests).

GiteaMirror commented

2026-05-03 11:18:20 -05:00

@twalderman commented on GitHub (Oct 31, 2024):

What are people using for mlx local serving? What implementation is best and worth implementing in ollama?

@twalderman commented on GitHub (Oct 31, 2024): What are people using for mlx local serving? What implementation is best and worth implementing in ollama?

GiteaMirror commented

2026-05-03 11:18:21 -05:00

@hg0428 commented on GitHub (Oct 31, 2024):

What are people using for mlx local serving? What implementation is best and worth implementing in ollama?

The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX.

https://github.com/lmstudio-ai/mlx-engine

@hg0428 commented on GitHub (Oct 31, 2024): > What are people using for mlx local serving? What implementation is best and worth implementing in ollama? The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX. https://github.com/lmstudio-ai/mlx-engine

GiteaMirror commented

2026-05-03 11:18:21 -05:00

@ahmetkca commented on GitHub (Nov 9, 2024):

What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet.

I hav just tried LM Studio's new MLX backend and you can see 11+ tokens per second improvement for same model. The model in question was qwen2.5:7b-instruct-q8_0 from 70~ tokens per second to 81~ tokens per second

@ahmetkca commented on GitHub (Nov 9, 2024): > What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet. I hav just tried LM Studio's new MLX backend and you can see 11+ tokens per second improvement for same model. The model in question was qwen2.5:7b-instruct-q8_0 from 70~ tokens per second to 81~ tokens per second

GiteaMirror commented

2026-05-03 11:18:22 -05:00

@ahmetkca commented on GitHub (Nov 9, 2024):

What are people using for mlx local serving? What implementation is best and worth implementing in ollama?

The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX.

https://github.com/lmstudio-ai/mlx-engine

https://github.com/ollama/ollama/issues/1730#issuecomment-2466166189

@ahmetkca commented on GitHub (Nov 9, 2024): > > What are people using for mlx local serving? What implementation is best and worth implementing in ollama? > > The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX. > > https://github.com/lmstudio-ai/mlx-engine https://github.com/ollama/ollama/issues/1730#issuecomment-2466166189

GiteaMirror commented

2026-05-03 11:18:23 -05:00

@ahmetkca commented on GitHub (Nov 9, 2024):

What are people using for mlx local serving? What implementation is best and worth implementing in ollama?

The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX.

https://github.com/lmstudio-ai/mlx-engine

This is actually wrapper around MLX's mlx-lm python package. So perhaps, Ollama team can do better by completely bypassing python?

@ahmetkca commented on GitHub (Nov 9, 2024): > > What are people using for mlx local serving? What implementation is best and worth implementing in ollama? > > The developers of LM Studio have created a wrapper around MLX that makes it super simple. They used it to transition LM Studio from supporting only Llama.cpp as a backend to being able to support both Llama.cpp and MLX. > > https://github.com/lmstudio-ai/mlx-engine This is actually wrapper around MLX's mlx-lm python package. So perhaps, Ollama team can do better by completely bypassing python?

GiteaMirror commented

2026-05-03 11:18:23 -05:00

@logiota commented on GitHub (Nov 10, 2024):

What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet.

llama3.2 is at least 64% faster with MLX on M4! Just got mine :)

@logiota commented on GitHub (Nov 10, 2024): > What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet. llama3.2 is at least 64% faster with MLX on M4! Just got mine :)

GiteaMirror commented

2026-05-03 11:18:24 -05:00

@Idmon commented on GitHub (Nov 18, 2024):

Are we gonna see MLX support?

I think now with the new M4 Max chips it's a great time to support it.

@Idmon commented on GitHub (Nov 18, 2024): Are we gonna see MLX support? I think now with the new M4 Max chips it's a great time to support it.

GiteaMirror commented

2026-05-03 11:18:25 -05:00

@ljgeneral commented on GitHub (Nov 19, 2024):

Followed, looking forward to ollama support, I will try LM Studio first

@ljgeneral commented on GitHub (Nov 19, 2024): Followed, looking forward to ollama support, I will try LM Studio first

GiteaMirror commented

2026-05-03 11:18:25 -05:00

@bhupesh-sf commented on GitHub (Nov 22, 2024):

Want to follow to see its support natively

@bhupesh-sf commented on GitHub (Nov 22, 2024): Want to follow to see its support natively

GiteaMirror commented

2026-05-03 11:18:27 -05:00

@NashvilleBrandon commented on GitHub (Nov 27, 2024):

The people need this.

@NashvilleBrandon commented on GitHub (Nov 27, 2024): The people need this.

GiteaMirror commented

2026-05-03 11:18:29 -05:00

@bakaburg1 commented on GitHub (Dec 4, 2024):

Up!

@bakaburg1 commented on GitHub (Dec 4, 2024): Up!

GiteaMirror commented

2026-05-03 11:18:32 -05:00

@baoduy commented on GitHub (Dec 5, 2024):

1 more vote for this month be 👍👍👍

@baoduy commented on GitHub (Dec 5, 2024): 1 more vote for this month be 👍👍👍

GiteaMirror commented

2026-05-03 11:18:34 -05:00

@elfriscia commented on GitHub (Dec 5, 2024):

1 more vote for this month be 👍👍👍

Wish granted. One more 👍
This feature is native on Lm Studio and there’s a big difference.

@elfriscia commented on GitHub (Dec 5, 2024): > 1 more vote for this month be 👍👍👍 Wish granted. One more 👍 This feature is native on Lm Studio and there’s a big difference.

GiteaMirror commented

2026-05-03 11:18:36 -05:00

@vietvudanh commented on GitHub (Dec 6, 2024):

Well, I ended up using MLX directly. Just wish they supports json output mode.

@vietvudanh commented on GitHub (Dec 6, 2024): Well, I ended up using MLX directly. Just wish they supports json output mode.

GiteaMirror commented

2026-05-03 11:18:37 -05:00

@smsmatt commented on GitHub (Dec 7, 2024):

Would appear to be a big win for Ollama to put this feature in.

@smsmatt commented on GitHub (Dec 7, 2024): Would appear to be a big win for Ollama to put this feature in.

GiteaMirror commented

2026-05-03 11:18:37 -05:00

@cryptedx commented on GitHub (Dec 8, 2024):

Well, this would be a big win for Ollama!

@cryptedx commented on GitHub (Dec 8, 2024): Well, this would be a big win for Ollama!

GiteaMirror commented

2026-05-03 11:18:38 -05:00

@hg0428 commented on GitHub (Dec 8, 2024):

Llama.cpp is exploring Apple Silicon ANE support, which not even MLX has. If implemented properly, that could make Llama.cpp significantly faster than MLX.

@hg0428 commented on GitHub (Dec 8, 2024): Llama.cpp is exploring Apple Silicon ANE support, which not even MLX has. If implemented properly, that could make Llama.cpp significantly faster than MLX.

GiteaMirror commented

2026-05-03 11:18:40 -05:00

@ivanfioravanti commented on GitHub (Dec 8, 2024):

All models must be created adhoc for ANE. Moreover ANE is faster if you have few GPU cores otherwise noSent from the road.Il giorno 8 dic 2024, alle ore 19:44, Hudson Gouge @.***> ha scritto:
Llama.cpp is exploring Apple Silicon ANE support, which not even MLX has. If implemented properly, that could make Llama.cpp significantly faster than MLX.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

@ivanfioravanti commented on GitHub (Dec 8, 2024): All models must be created adhoc for ANE. Moreover ANE is faster if you have few GPU cores otherwise noSent from the road.Il giorno 8 dic 2024, alle ore 19:44, Hudson Gouge ***@***.***> ha scritto: Llama.cpp is exploring Apple Silicon ANE support, which not even MLX has. If implemented properly, that could make Llama.cpp significantly faster than MLX. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

GiteaMirror commented

2026-05-03 11:18:40 -05:00

@hg0428 commented on GitHub (Dec 8, 2024):

All models must be created adhoc for ANE. Moreover ANE is faster if you have few GPU cores otherwise no.

The ANE is like the GPU, but specialized for AI. The model files themselves need not be changed. You can run anything on it just like the GPU, thanks the new API that Apple released. Previously, you had to use CoreML; now, you can access it directly. The ANE alone can get performance roughly equivalent to the M4 Max GPU, which is quite good. Combined with GPU, it can result in a significant performance boost.

@hg0428 commented on GitHub (Dec 8, 2024): > All models must be created adhoc for ANE. Moreover ANE is faster if you have few GPU cores otherwise no. The ANE is like the GPU, but specialized for AI. The model files themselves need not be changed. You can run anything on it just like the GPU, thanks the new API that Apple released. Previously, you had to use CoreML; now, you can access it directly. The ANE alone can get performance roughly equivalent to the M4 Max GPU, which is quite good. Combined with GPU, it can result in a significant performance boost.

GiteaMirror commented

2026-05-03 11:18:41 -05:00

@elfriscia commented on GitHub (Dec 8, 2024):

The troll downvoting is frustrated for not being able to have a MacBook and will reply anything to contradict this statement.

@elfriscia commented on GitHub (Dec 8, 2024): The troll downvoting is frustrated for not being able to have a MacBook and will reply anything to contradict this statement.

GiteaMirror commented

2026-05-03 11:18:42 -05:00

@vietvudanh commented on GitHub (Dec 9, 2024):

Well, soon I guess: Ollama's post

@vietvudanh commented on GitHub (Dec 9, 2024): Well, soon I guess: [Ollama's post](https://x.com/ollama/status/1865238754052485293)

GiteaMirror commented

2026-05-03 11:18:45 -05:00

@iamhenry commented on GitHub (Dec 9, 2024):

@iamhenry commented on GitHub (Dec 9, 2024): ![image](https://github.com/user-attachments/assets/2d0b000d-a651-4ca3-8653-33546c2f66d3)

GiteaMirror commented

2026-05-03 11:18:49 -05:00

@integrate-your-mind commented on GitHub (Dec 11, 2024):

If this happens by Monday I will be so happy.

@integrate-your-mind commented on GitHub (Dec 11, 2024): If this happens by Monday I will be so happy.

GiteaMirror commented

2026-05-03 11:18:53 -05:00

@rheinardkorf commented on GitHub (Dec 17, 2024):

It did not happen Monday. 😅

@rheinardkorf commented on GitHub (Dec 17, 2024): It did not happen Monday. 😅

GiteaMirror commented

2026-05-03 11:18:55 -05:00

@loganyc1 commented on GitHub (Dec 18, 2024):

Did this ship?

@loganyc1 commented on GitHub (Dec 18, 2024): Did this ship?

GiteaMirror commented

2026-05-03 11:18:59 -05:00

@MS00-GitIt commented on GitHub (Dec 18, 2024):

Reading all this has been fun. I can't wait to see this issue a marked as "Closed".

@MS00-GitIt commented on GitHub (Dec 18, 2024): Reading all this has been fun. I can't wait to see this issue a marked as "Closed".

GiteaMirror commented

2026-05-03 11:19:04 -05:00

@zhaopengme commented on GitHub (Dec 28, 2024):

Waiting.⏳(๑˃̵ᴗ˂̵)

@zhaopengme commented on GitHub (Dec 28, 2024): Waiting.⏳(๑˃̵ᴗ˂̵)

GiteaMirror commented

2026-05-03 11:19:10 -05:00

@dalisoft commented on GitHub (Dec 30, 2024):

Reading all this has been fun. I can't wait to see this issue a marked as "Closed".

Better if Closed as completed not as Closed as not planned 😄

@dalisoft commented on GitHub (Dec 30, 2024): > Reading all this has been fun. I can't wait to see this issue a marked as "Closed". Better if Closed as completed not as Closed as not planned 😄

GiteaMirror commented

2026-05-03 11:19:11 -05:00

@Swich1987 commented on GitHub (Jan 4, 2025):

Looking forward for that support 🤞

@Swich1987 commented on GitHub (Jan 4, 2025): Looking forward for that support 🤞

GiteaMirror commented

2026-05-03 11:19:13 -05:00

@jeffhaskin commented on GitHub (Jan 16, 2025):

What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet.

Update 1 year later:

MLX is 25.02% faster than GGUF on my M1 Air 16G
MLX is 41.01% faster than GGUF on my M2 Pro 32G.

Model: Llama 3.1 8b Instruct Q4
Platform: LM Studio

On the M1 Air, even Phi4-14b is getting usable speeds (7.06 tok/sec MLX vs 5 tok/sec GGUF), which puts it just into the usability range for me.

On the M2 Pro 32G, I'm getting similar speeds with Qwen2.5 32B (mlx), which is great.

@jeffhaskin commented on GitHub (Jan 16, 2025): > What do you hope to gain from this? I don't think MLX is faster for inference, at least not yet. Update 1 year later: - MLX is 25.02% faster than GGUF on my M1 Air 16G - MLX is 41.01% faster than GGUF on my M2 Pro 32G. Model: Llama 3.1 8b Instruct Q4 Platform: LM Studio On the M1 Air, even **Phi4-14b** is getting usable speeds (7.06 tok/sec MLX vs 5 tok/sec GGUF), which puts it just into the usability range for me. On the M2 Pro 32G, I'm getting similar speeds with Qwen2.5 32B (mlx), which is great.

GiteaMirror commented

2026-05-03 11:19:15 -05:00

@iamhenry commented on GitHub (Jan 16, 2025):

monday is coming up 😅

@iamhenry commented on GitHub (Jan 16, 2025): monday is coming up 😅

GiteaMirror commented

2026-05-03 11:19:17 -05:00

@schneipk commented on GitHub (Jan 22, 2025):

I'm so stoked ! OllaMLXa <3

@schneipk commented on GitHub (Jan 22, 2025): I'm so stoked ! OllaMLXa <3

GiteaMirror commented

2026-05-03 11:19:19 -05:00

@Unicorndy commented on GitHub (Jan 24, 2025):

Is it happening soon? 🤞

@Unicorndy commented on GitHub (Jan 24, 2025): Is it happening soon? 🤞

GiteaMirror commented

2026-05-03 11:19:22 -05:00

@Byte1122 commented on GitHub (Jan 31, 2025):

I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate!

@Byte1122 commented on GitHub (Jan 31, 2025): I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate!

GiteaMirror commented

2026-05-03 11:19:23 -05:00

@bhupesh-sf commented on GitHub (Feb 1, 2025):

here we go https://github.com/ollama/ollama/pull/8490

@bhupesh-sf commented on GitHub (Feb 1, 2025): here we go [https://github.com/ollama/ollama/pull/8490](https://github.com/ollama/ollama/pull/8490)

GiteaMirror commented

2026-05-03 11:19:24 -05:00

@TheurgicDuke771 commented on GitHub (Feb 14, 2025):

I see the PR https://github.com/ollama/ollama/pull/8490 is now closed. Can we expect this in next stable release?

@TheurgicDuke771 commented on GitHub (Feb 14, 2025): I see the PR https://github.com/ollama/ollama/pull/8490 is now closed. Can we expect this in next stable release?

GiteaMirror commented

2026-05-03 11:19:25 -05:00

@kconner commented on GitHub (Feb 18, 2025):

#8490 was superseded by #9118, which seems worth watching for progress.

@kconner commented on GitHub (Feb 18, 2025): #8490 was superseded by #9118, which seems worth watching for progress.

GiteaMirror commented

2026-05-03 11:19:26 -05:00

@neolee commented on GitHub (Feb 22, 2025):

So do we have any plan and/or schedule for MLX support?

@neolee commented on GitHub (Feb 22, 2025): So do we have any plan and/or schedule for MLX support?

GiteaMirror commented

2026-05-03 11:19:27 -05:00

@HadiCherkaoui commented on GitHub (Feb 23, 2025):

Has it arrived?

@HadiCherkaoui commented on GitHub (Feb 23, 2025): Has it arrived?

GiteaMirror commented

2026-05-03 11:19:28 -05:00

@robwilkes commented on GitHub (Feb 26, 2025):

Models converted to MLX format are ~20% faster than the same models in GGUF format on my M4 MacBook Pro

@robwilkes commented on GitHub (Feb 26, 2025): Models converted to MLX format are ~20% faster than the same models in GGUF format on my M4 MacBook Pro

GiteaMirror commented

2026-05-03 11:19:29 -05:00

@Caojunisstudying commented on GitHub (Mar 14, 2025):

I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate!

However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results.

@Caojunisstudying commented on GitHub (Mar 14, 2025): > I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate! However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results.

GiteaMirror commented

2026-05-03 11:19:29 -05:00

@robwilkes commented on GitHub (Mar 14, 2025):

I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate!

However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results.

Are you sure you're comparing the same quantization, context size, etc, like for like?

The memory usage is the same for me, maybe even a tiny bit better as Mac can manage the memory slightly better with MLX, however it's mostly negligible / identical.

The filesizes are roughly equivalent and therefore the memory utilisation is roughly equivalent.

@robwilkes commented on GitHub (Mar 14, 2025): > > I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate! > > However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results. Are you sure you're comparing the same quantization, context size, etc, like for like? The memory usage is the same for me, maybe even a tiny bit better as Mac can manage the memory slightly better with MLX, however it's mostly negligible / identical. The filesizes are roughly equivalent and therefore the memory utilisation is roughly equivalent.

GiteaMirror commented

2026-05-03 11:19:30 -05:00

@Caojunisstudying commented on GitHub (Mar 14, 2025):

I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate!

However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results.

Are you sure you're comparing the same quantization, context size, etc, like for like?

The memory usage is the same for me, maybe even a tiny bit better as Mac can manage the memory slightly better with MLX, however it's mostly negligible / identical.

The filesizes are roughly equivalent and therefore the memory utilisation is roughly equivalent.

like qwq-32b Q4 on ollama vs qwq-32b 4bit on LMs, my macmini has just 24G memory which of very high load in such scenario, the result is that ollama run hard but finish the answer, the LMs lead my mac to crash and have to restart. I can use ollama and docker, ragflow normally, but crash with LMs because of ram out again on 14b models.
so I think ollama in running on cpu and ram of 24G, but when mlx model use gpu to accelerate, part of the ram must be allocated to gpu, so the ram gets not enought。

@Caojunisstudying commented on GitHub (Mar 14, 2025): > > > I am using LM Studio and Ollama. The MLX models on LM Studio are way faster. I hope I can switch entirely to Ollama, but with LM Studio. Anyway, love to support this. If Ollama need donations or what so ever, love to donate! > > > > > > However, when I use the MLX model, my Macmini takes up much more memory, so I have to abandon MLX, because if cpu performance is good enough，I would rather spend memory on building models with higher precision , I don’t know how Ollama will be running the mlx model in the future，hope better results. > > Are you sure you're comparing the same quantization, context size, etc, like for like? > > The memory usage is the same for me, maybe even a tiny bit better as Mac can manage the memory slightly better with MLX, however it's mostly negligible / identical. > > The filesizes are roughly equivalent and therefore the memory utilisation is roughly equivalent. like qwq-32b Q4 on ollama vs qwq-32b 4bit on LMs, my macmini has just 24G memory which of very high load in such scenario, the result is that ollama run hard but finish the answer, the LMs lead my mac to crash and have to restart. I can use ollama and docker, ragflow normally, but crash with LMs because of ram out again on 14b models. so I think ollama in running on cpu and ram of 24G, but when mlx model use gpu to accelerate, part of the ram must be allocated to gpu, so the ram gets not enought。

GiteaMirror commented

2026-05-03 11:19:31 -05:00

@arty-hlr commented on GitHub (Mar 14, 2025):

MLX does not need to preallocate memory for the context size, so you should be able to run models with their full context size with better memory utilisation. @Caojunisstudying the whole 24G are not available to you/to the GPU, only 2/3 of those by default. Did you change the limit with sudo sysctl iogpu.wired_limit_mb=XXXXX?

@arty-hlr commented on GitHub (Mar 14, 2025): MLX does not need to preallocate memory for the context size, so you should be able to run models with their full context size with better memory utilisation. @Caojunisstudying the whole 24G are not available to you/to the GPU, only 2/3 of those by default. Did you change the limit with `sudo sysctl iogpu.wired_limit_mb=XXXXX`?

GiteaMirror commented

2026-05-03 11:19:32 -05:00

@Caojunisstudying commented on GitHub (Mar 14, 2025):

MLX does not need to preallocate memory for the context size, so you should be able to run models with their full context size with better memory utilisation. @Caojunisstudying the whole 24G are not available to you/to the GPU, only 2/3 of those by default. Did you change the limit with sudo sysctl iogpu.wired_limit_mb=XXXXX?

that may be the key, My LMs keep using the default settings, I will change it and try again, Thanks a lot for your suggestion.

@Caojunisstudying commented on GitHub (Mar 14, 2025): > MLX does not need to preallocate memory for the context size, so you should be able to run models with their full context size with better memory utilisation. [@Caojunisstudying](https://github.com/Caojunisstudying) the whole 24G are not available to you/to the GPU, only 2/3 of those by default. Did you change the limit with `sudo sysctl iogpu.wired_limit_mb=XXXXX`? that may be the key, My LMs keep using the default settings, I will change it and try again, Thanks a lot for your suggestion.

GiteaMirror commented

2026-05-03 11:19:33 -05:00

@anurmatov commented on GitHub (Mar 16, 2025):

this gpu mem limit parameter is a game changer, although it's worth keeping in mind that it's a runtime thing and resets on restarts. i've created a script that automatically sets the desired value even after restarts, might be helpful especially for headless setups

@anurmatov commented on GitHub (Mar 16, 2025): this gpu mem limit parameter is a game changer, although it's worth keeping in mind that it's a runtime thing and resets on restarts. i've created a [script](https://github.com/anurmatov/mac-studio-server) that automatically sets the desired value even after restarts, might be helpful especially for headless setups

GiteaMirror commented

2026-05-03 11:19:35 -05:00

@easp commented on GitHub (Mar 16, 2025):

man sysctl.conf

@easp commented on GitHub (Mar 16, 2025): `man sysctl.conf`

GiteaMirror commented

2026-05-03 11:19:38 -05:00

@zengqingfu1442 commented on GitHub (Mar 23, 2025):

Add mlx-vlm backend also.

@zengqingfu1442 commented on GitHub (Mar 23, 2025): Add mlx-vlm backend also.

GiteaMirror commented

2026-05-03 11:19:39 -05:00

@matthieuHenocque commented on GitHub (May 9, 2025):

Lack of MLX support is the only reason I don't use Ollama. In some cases MLX Q8 is 20% faster than GGUF, and memory usage is better handled.

So please, add MLX support to Ollama

@matthieuHenocque commented on GitHub (May 9, 2025): Lack of MLX support is the only reason I don't use Ollama. In some cases MLX Q8 is 20% faster than GGUF, and memory usage is better handled. So please, add MLX support to Ollama

GiteaMirror commented

2026-05-03 11:19:41 -05:00

@vdbwim commented on GitHub (Jun 3, 2025):

Is there a target date to get MLX supported?

@vdbwim commented on GitHub (Jun 3, 2025): Is there a target date to get MLX supported?

GiteaMirror commented

2026-05-03 11:19:43 -05:00

@sxy-trans-n commented on GitHub (Jun 4, 2025):

https://github.com/Trans-N-ai/swama

High-performance MLX-based LLM inference engine for macOS with native Swift implementation

@sxy-trans-n commented on GitHub (Jun 4, 2025): https://github.com/Trans-N-ai/swama High-performance MLX-based LLM inference engine for macOS with native Swift implementation

GiteaMirror commented

2026-05-03 11:19:44 -05:00

@FelikZ commented on GitHub (Jun 23, 2025):

https://github.com/Trans-N-ai/swama

It's a next level fast indeed. Tried yesterday with Qwen3 30B, mind blowing. Would be nice if it get support for Mistral models too.

benchmarks are crazy:

@FelikZ commented on GitHub (Jun 23, 2025): > https://github.com/Trans-N-ai/swama It's a next level fast indeed. Tried yesterday with Qwen3 30B, mind blowing. Would be nice if it get support for Mistral models too. [benchmarks are crazy](https://github.com/Trans-N-ai/swama/issues/15#issuecomment-2961338474): ![Image](https://github.com/user-attachments/assets/e924d4d2-e95b-4fdf-ae23-5f3e36bef748)

GiteaMirror commented

2026-05-03 11:19:46 -05:00

@openSourcerer9000 commented on GitHub (Jul 19, 2025):

Seems ollamas mac userbase is withering away along with this PR
Draft MLX go backend for new engine by dhiltgen · Pull Request #9118 · ollama/ollama https://share.google/snsqjHmkYOqBecvLf

@openSourcerer9000 commented on GitHub (Jul 19, 2025): Seems ollamas mac userbase is withering away along with this PR Draft MLX go backend for new engine by dhiltgen · Pull Request #9118 · ollama/ollama https://share.google/snsqjHmkYOqBecvLf

GiteaMirror commented

2026-05-03 11:19:46 -05:00

@zhaopengme commented on GitHub (Aug 4, 2025):

hi.Have there been any recent developments?

@zhaopengme commented on GitHub (Aug 4, 2025): hi.Have there been any recent developments?

GiteaMirror commented

2026-05-03 11:19:48 -05:00

@HKiOnline commented on GitHub (Aug 5, 2025):

I'm using LM Studio at the moment as it has a MLX backend. The performance differences are very clear. I would love to use Ollama instead.

@HKiOnline commented on GitHub (Aug 5, 2025): I'm using LM Studio at the moment as it has a MLX backend. The performance differences are very clear. I would love to use Ollama instead.

GiteaMirror commented

2026-05-03 11:19:50 -05:00

@BradKML commented on GitHub (Aug 15, 2025):

@HKiOnline got anything similar that is FOSS AND MLX-compatible?

@BradKML commented on GitHub (Aug 15, 2025): @HKiOnline got anything similar that is FOSS AND MLX-compatible?

GiteaMirror commented

2026-05-03 11:19:54 -05:00

@CharafChnioune commented on GitHub (Aug 15, 2025):

GTA 6 or mlx support? Bets are on

Verzonden vanuit Outlook voor iOShttps://aka.ms/o0ukef

Van: Brad @.>
Verzonden: Friday, August 15, 2025 1:09:09 PM
Aan: ollama/ollama @.>
CC: Charaf @.>; Comment @.>
Onderwerp: Re: [ollama/ollama] MLX backend (Issue #1730)

[https://avatars.githubusercontent.com/u/58927531?s=20&v=4]BradKML left a comment (ollama/ollama#1730)https://github.com/ollama/ollama/issues/1730#issuecomment-3191266779

@HKiOnlinehttps://github.com/HKiOnline got anything similar that is FOSS AND MLX-compatible?

—
Reply to this email directly, view it on GitHubhttps://github.com/ollama/ollama/issues/1730#issuecomment-3191266779, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A45DQFIFM6HFXWEDL2FHSVD3NW5VLAVCNFSM6AAAAABBEW4KFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOJRGI3DMNZXHE.
You are receiving this because you commented.Message ID: @.***>

@CharafChnioune commented on GitHub (Aug 15, 2025): GTA 6 or mlx support? Bets are on Verzonden vanuit Outlook voor iOS<https://aka.ms/o0ukef> ________________________________ Van: Brad ***@***.***> Verzonden: Friday, August 15, 2025 1:09:09 PM Aan: ollama/ollama ***@***.***> CC: Charaf ***@***.***>; Comment ***@***.***> Onderwerp: Re: [ollama/ollama] MLX backend (Issue #1730) [https://avatars.githubusercontent.com/u/58927531?s=20&v=4]BradKML left a comment (ollama/ollama#1730)<https://github.com/ollama/ollama/issues/1730#issuecomment-3191266779> @HKiOnline<https://github.com/HKiOnline> got anything similar that is FOSS AND MLX-compatible? — Reply to this email directly, view it on GitHub<https://github.com/ollama/ollama/issues/1730#issuecomment-3191266779>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A45DQFIFM6HFXWEDL2FHSVD3NW5VLAVCNFSM6AAAAABBEW4KFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCOJRGI3DMNZXHE>. You are receiving this because you commented.Message ID: ***@***.***>

GiteaMirror commented

2026-05-03 11:19:56 -05:00

@mkozjak commented on GitHub (Aug 15, 2025):

@HKiOnline got anything similar that is FOSS AND MLX-compatible?

mlx-omni-server, as some say, might be the best option for us.

@mkozjak commented on GitHub (Aug 15, 2025): > [@HKiOnline](https://github.com/HKiOnline) got anything similar that is FOSS AND MLX-compatible? mlx-omni-server, as some say, might be the best option for us.

GiteaMirror commented

2026-05-03 11:19:59 -05:00

@curious-boy-007 commented on GitHub (Sep 14, 2025):

@ageorgios @mkozjak @Josecodesalot
You might want to take a look at this MLX + GGUF compatible CLI tool:

https://github.com/NexaAI/nexa-sdk

@curious-boy-007 commented on GitHub (Sep 14, 2025): @ageorgios @mkozjak @Josecodesalot You might want to take a look at this MLX + GGUF compatible CLI tool: - https://github.com/NexaAI/nexa-sdk

GiteaMirror commented

2026-05-03 11:20:02 -05:00

@mkozjak commented on GitHub (Sep 14, 2025):

@ageorgios @mkozjak @Josecodesalot You might want to take a look at this MLX + GGUF compatible CLI tool:

https://github.com/NexaAI/nexa-sdk

Doesn't work at all in combination with Zed for me.

@mkozjak commented on GitHub (Sep 14, 2025): > [@ageorgios](https://github.com/ageorgios) [@mkozjak](https://github.com/mkozjak) [@Josecodesalot](https://github.com/Josecodesalot) You might want to take a look at this MLX + GGUF compatible CLI tool: > > * https://github.com/NexaAI/nexa-sdk Doesn't work at all in combination with Zed for me.

GiteaMirror commented

2026-05-03 11:20:04 -05:00

@curious-boy-007 commented on GitHub (Sep 14, 2025):

@mkozjak Please try this version for macOS installer
https://github.com/NexaAI/nexa-sdk?tab=readme-ov-file#macos
Also woud you please let me know the error log for Zed editor? In macOS build-in terminal, should work.

@curious-boy-007 commented on GitHub (Sep 14, 2025): @mkozjak Please try this version for macOS installer https://github.com/NexaAI/nexa-sdk?tab=readme-ov-file#macos Also woud you please let me know the error log for Zed editor? In macOS build-in terminal, should work.

GiteaMirror commented

2026-05-03 11:20:08 -05:00

@ericcurtin commented on GitHub (Oct 13, 2025):

In Docker Model Runner we've put effort into putting all our code in one central place to make it easier for people to contribute. Please star, fork and contribute (especially please contribute an mlx backend):

https://github.com/docker/model-runner

We have vulkan support. You can pull models from Docker Hub, Huggingface or any other OCI registry. You can also push models to Docker Hub or any other OCI registry.

@ericcurtin commented on GitHub (Oct 13, 2025): In Docker Model Runner we've put effort into putting all our code in one central place to make it easier for people to contribute. Please star, fork and contribute (especially please contribute an mlx backend): https://github.com/docker/model-runner We have vulkan support. You can pull models from Docker Hub, Huggingface or any other OCI registry. You can also push models to Docker Hub or any other OCI registry.

GiteaMirror commented

2026-05-03 11:20:10 -05:00

@Globolo001 commented on GitHub (Dec 20, 2025):

Any updates on this feature?
Very surprised it takes THIS long as LMStudio been having it for a while now (but I guess thats just the curse of OpenSource)

@Globolo001 commented on GitHub (Dec 20, 2025): Any updates on this feature? Very surprised it takes THIS long as LMStudio been having it for a while now (but I guess thats just the curse of OpenSource)

GiteaMirror commented

2026-05-03 11:20:13 -05:00

@jeffhaskin commented on GitHub (Jan 17, 2026):

Haven't touched ollama in more than a year because it doesn't support mlx. use to be the core of my system.

mlx is significantly faster on my M2 macbook than gguyf.

Our love was sweet, but it is ended. Farewell, sweet Juliette.

@jeffhaskin commented on GitHub (Jan 17, 2026): Haven't touched ollama in more than a year because it doesn't support mlx. use to be the core of my system. mlx is _significantly_ faster on my M2 macbook than gguyf. Our love was sweet, but it is ended. Farewell, sweet Juliette.

GiteaMirror commented

2026-05-03 11:20:14 -05:00

@TomLucidor commented on GitHub (Jan 19, 2026):

My current choice, they seemed to be moving very fast https://github.com/cubist38/mlx-openai-server
(as for something more conventional, Ramalama or Jan seemed good as a FOSS universal adapter compared to LMStudio?)

@TomLucidor commented on GitHub (Jan 19, 2026): My current choice, they seemed to be moving very fast https://github.com/cubist38/mlx-openai-server (as for something more conventional, Ramalama or Jan seemed good as a FOSS universal adapter compared to LMStudio?)

GiteaMirror commented

2026-05-03 11:20:15 -05:00

@Globolo001 commented on GitHub (Jan 19, 2026):

Awwwww maaaannnn.
I feel like in theory Claude Opus should just be able look at the mlx docs. Check the open source LMStudio MLX adapter
for reference and implement it in an afternoon.

Is it because just because a lack of trying, a promising work in progress, just proofing to be way harder than I naively think.
Or has an architectural roadblock been hit that effectively does not allow OLLAMA to use anything other than llama cpp

I could find two active branches. But no info what they actually try to achieve. Apart from the MLX in the branch name. (Also why don’t branches have a feature description attribute in git. So annoying just trying to figure out the intention from branch and commit names)
https://github.com/ollama/ollama/tree/mlx-gpu-cd

https://github.com/ollama/ollama/tree/mxyng/next-mlx

Also the main readme seems to allow for a custom build with mlx.
Building with MLX (experimental)

However not sure wether this is also for pulling mlx models. Or only building models using mlx.
But only did very shallow digging. If there is a better discussion I'd love to get some insights :)

@Globolo001 commented on GitHub (Jan 19, 2026): Awwwww maaaannnn. I feel like in theory Claude Opus should just be able look at the mlx docs. Check the open source [LMStudio MLX adapter](https://github.com/lmstudio-ai/mlx-engine) for reference and implement it in an afternoon. Is it because just because a lack of trying, a promising work in progress, just proofing to be way harder than I naively think. Or has an architectural roadblock been hit that effectively does not allow OLLAMA to use anything other than llama cpp I could find two active branches. But no info what they actually try to achieve. Apart from the MLX in the branch name. (Also why don’t branches have a feature description attribute in git. So annoying just trying to figure out the intention from branch and commit names) https://github.com/ollama/ollama/tree/mlx-gpu-cd https://github.com/ollama/ollama/tree/mxyng/next-mlx Also the main readme seems to allow for a custom build with mlx. _Building with MLX (experimental)_ However not sure wether this is also for pulling mlx models. Or only building models using mlx. But only did very shallow digging. If there is a better discussion I'd love to get some insights :)

GiteaMirror commented

2026-05-03 11:20:17 -05:00

@Jingyuan-Zheng commented on GitHub (Mar 4, 2026):

Is it currently supported to run MLX models in Ollama? I haven't found a configuration method or an option to enable it.

@Jingyuan-Zheng commented on GitHub (Mar 4, 2026): Is it currently supported to run MLX models in Ollama? I haven't found a configuration method or an option to enable it.

GiteaMirror commented

2026-05-03 11:20:19 -05:00

@qdrddr commented on GitHub (Mar 4, 2026):

I think it's supported now. But I can't find documentationon how to use it.
https://github.com/ollama/ollama/pull/13648

@qdrddr commented on GitHub (Mar 4, 2026): I think it's supported now. But I can't find documentationon how to use it. https://github.com/ollama/ollama/pull/13648

GiteaMirror commented

2026-05-03 11:20:20 -05:00

@jackbravo commented on GitHub (Mar 4, 2026):

And seems it is just for some usecases:

Yes, our goal is to add text and embedding models. We're starting with imagegen since that's a brand new capability for Ollama, and once we get it working well, then we'll start tackling other models. We're expecting a lot of churn in the backend code as we flesh things out and refine how we want the engine to work.

@jackbravo commented on GitHub (Mar 4, 2026): And seems it is just for some usecases: > Yes, our goal is to add text and embedding models. We're starting with imagegen since that's a brand new capability for Ollama, and once we get it working well, then we'll start tackling other models. We're expecting a lot of churn in the backend code as we flesh things out and refine how we want the engine to work.

GiteaMirror commented

2026-05-03 11:20:27 -05:00

@Byte1122 commented on GitHub (Mar 4, 2026):

Yes it is supported, guys just use ai to discover it. Not long ago this was added, also you need to enable it in the confit or modelfile

33ee7168ba

@Byte1122 commented on GitHub (Mar 4, 2026): Yes it is supported, guys just use ai to discover it. Not long ago this was added, also you need to enable it in the confit or modelfile https://github.com/ollama/ollama/commit/33ee7168ba1e16c813b52dc2c9417efa1e2e9f20

GiteaMirror commented

2026-05-03 11:20:32 -05:00

@huyz commented on GitHub (Mar 5, 2026):

Is MLX support still experimental and does it require special compilation?
I'm looking for out-of-the-box support

@huyz commented on GitHub (Mar 5, 2026): Is MLX support still experimental and does it require special compilation? I'm looking for out-of-the-box support

Sign in to join this conversation.

Branches Tags

main

hoyyeva/anthropic-local-image-path

dhiltgen/ci

dhiltgen/llama-runner

parth-remove-claude-desktop-launch

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#63022