[GH-ISSUE #6006] Docker version of Ollama model loads slowly #29516

Closed
opened 2026-04-22 08:28:12 -05:00 by GiteaMirror · 14 comments
Owner

Originally created by @billowssun on GitHub (Jul 27, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6006

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

image

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @billowssun on GitHub (Jul 27, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6006 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? ![image](https://github.com/user-attachments/assets/7897f2a4-3a33-4fc7-9ca5-eac8b1839431) ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the wslperformancewindows labels 2026-04-22 08:28:12 -05:00
Author
Owner

@billowssun commented on GitHub (Jul 27, 2024):

image

<!-- gh-comment-id:2253731673 --> @billowssun commented on GitHub (Jul 27, 2024): ![image](https://github.com/user-attachments/assets/9c3476e1-da62-4c4e-a079-c6b04c8eb7ff)
Author
Owner

@mxmp210 commented on GitHub (Jul 27, 2024):

Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file.

<!-- gh-comment-id:2254101529 --> @mxmp210 commented on GitHub (Jul 27, 2024): Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file.
Author
Owner

@billowssun commented on GitHub (Aug 4, 2024):

Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file.

I found the root of the problem. It's because mapping the local directory causes the large model to load slowly. If file path mapping is not used, loading materials would be very fast

<!-- gh-comment-id:2267281230 --> @billowssun commented on GitHub (Aug 4, 2024): > Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file. I found the root of the problem. It's because mapping the local directory causes the large model to load slowly. If file path mapping is not used, loading materials would be very fast
Author
Owner

@mxmp210 commented on GitHub (Aug 4, 2024):

Yes, it works because docker volumes load from Linux mapped filesystem, and it has no extra system call overhead. So, it's not a bug, this behaviour is expected.

It is the technical limitation of WSL2 filesystem architecture.

<!-- gh-comment-id:2267369827 --> @mxmp210 commented on GitHub (Aug 4, 2024): Yes, it works because docker volumes load from Linux mapped filesystem, and it has no extra system call overhead. So, it's not a bug, this behaviour is expected. It is the technical limitation of WSL2 filesystem architecture.
Author
Owner

@jmorganca commented on GitHub (Sep 4, 2024):

Hi @billowssun, yes, while WSL2's filesystem mapping is very fast, it still may have some performance constraints that running on Windows directly may not have, including not being able to memory map chunks of the model into memory.

<!-- gh-comment-id:2327884061 --> @jmorganca commented on GitHub (Sep 4, 2024): Hi @billowssun, yes, while WSL2's filesystem mapping is very fast, it still may have some performance constraints that running on Windows directly may not have, including not being able to memory map chunks of the model into memory.
Author
Owner

@dhiltgen commented on GitHub (Sep 5, 2024):

I think we can close this one. When using WSL2, I/O back to the NTFS filesystem can be slow, so using the native linux filesystem on the vhd will be much faster.

<!-- gh-comment-id:2332553137 --> @dhiltgen commented on GitHub (Sep 5, 2024): I think we can close this one. When using WSL2, I/O back to the NTFS filesystem can be slow, so using the native linux filesystem on the vhd will be much faster.
Author
Owner

@agibson-fl commented on GitHub (Oct 16, 2024):

Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file.

@mxmp210 I'm having the same issue as reported in this issue. However, I'm not exactly following this guidance to get around the issue.

When I look under Resources in Docker Desktop, it shows WSL 2 backed and that the disk image location is in C:\Users\owner\AppData\Local\Docker\wsl

I have the following volumes setup on my docker run command.

-v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama `
-v C:\Users\owner\Documents\docker\ollama\data:/app/backend/data `

Where do I need to move the ollama and data folders? Can you be more explicit?

<!-- gh-comment-id:2417764864 --> @agibson-fl commented on GitHub (Oct 16, 2024): > Is docker running on WSL2 backend? If then move the model in WSL2's HDD space some where in your user folder, it's technically a VM extension and filesystem is slower due to system calls being translated between two kernels. If not please provide further information on environment, OS, system specs, GPU etc. Attach ollama log file. @mxmp210 I'm having the same issue as reported in this issue. However, I'm not exactly following this guidance to get around the issue. When I look under Resources in Docker Desktop, it shows WSL 2 backed and that the disk image location is in `C:\Users\owner\AppData\Local\Docker\wsl` I have the following volumes setup on my docker run command. ``` -v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama ` -v C:\Users\owner\Documents\docker\ollama\data:/app/backend/data ` ``` Where do I need to move the ollama and data folders? Can you be more explicit?
Author
Owner

@mxmp210 commented on GitHub (Oct 17, 2024):

Where do I need to move the ollama and data folders? Can you be more explicit?

Anything in your home folder /home and root directory / except /mnt/<drive_letter> resides inside virtual disk, other devices in /mnt are mounted via qemu compatibility layer. That is the core reason why other paths are slower as the layer is responsible to translate Linux system calls to Windows compatible system calls.

Use user's home directory and it will have native like performance. Or you can use volume declaration without bind mount, that will store volumes inside wsl filesystem.

Hope that helps!

<!-- gh-comment-id:2418414710 --> @mxmp210 commented on GitHub (Oct 17, 2024): > Where do I need to move the ollama and data folders? Can you be more explicit? Anything in your home folder `/home` and root directory `/` except `/mnt/<drive_letter>` resides inside virtual disk, other devices in `/mnt` are mounted via qemu compatibility layer. That is the core reason why other paths are slower as the layer is responsible to translate Linux system calls to Windows compatible system calls. Use user's home directory and it will have native like performance. Or you can use volume declaration without bind mount, that will store volumes inside wsl filesystem. Hope that helps!
Author
Owner

@agibson-fl commented on GitHub (Oct 17, 2024):

@mxmp210 The default location for models inside the container seems to be /root/.ollama. So I setup my volume as such:

-v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama `

However, when i remove this, I have 0 problems running the models. As a cost of making this change though, my downloaded models won't persist which is annoying since they're quite large. I'm STILL not following what you're saying to change to fix this. Can I add back the volume but a different windows path that would be faster and allow the models to load? All your suggestions refer to Linux paths I'm presuming inside the container. Do you have an example volume mount where I can persist my models beyond destroying/recreating the container AND have the models work when loaded, for example?

<!-- gh-comment-id:2418422160 --> @agibson-fl commented on GitHub (Oct 17, 2024): @mxmp210 The default location for models inside the container seems to be /root/.ollama. So I setup my volume as such: ``` -v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama ` ``` However, when i remove this, I have 0 problems running the models. As a cost of making this change though, my downloaded models won't persist which is annoying since they're quite large. I'm STILL not following what you're saying to change to fix this. Can I add back the volume but a different windows path that would be faster and allow the models to load? All your suggestions refer to Linux paths I'm presuming inside the container. Do you have an example volume mount where I can persist my models beyond destroying/recreating the container AND have the models work when loaded, for example?
Author
Owner

@mxmp210 commented on GitHub (Oct 17, 2024):

-v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama `

This translates to /mnt/c/Users/owner/Documents/docker/ollama/ollama:/root/.ollama

What you need is to create an explicit volume via docker volume create ollama_data and then reference it like -v ollama_data:/root/.ollama that ensures /root/.ollama path resides at /var/lib/docker/volumes/ which is the default location for the volumes managed by docker in most linux distros. This location happens to be outside /mnt/<location> thus gives the performance.

If you are using docker-compose :

services:
  ollama:
    ...
    volumes:
      - ollama_data:/root/.ollama
 
# Declare volume here - docker will create it / reference it & reuse it as long as it exists.      
volumes:
  ollama_data:
    #external: true # Only needed if volume already exists from external sources

Declaring volume keeps the data on disk as long as volume exists with docker. It will stay persistent unlike temporary volumes that are created on-the-go basis.

Please read docker volumes documentation for more details, if you are still having trouble understanding how docker volumes work.

<!-- gh-comment-id:2418482110 --> @mxmp210 commented on GitHub (Oct 17, 2024): > ``` > -v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama ` > ``` This translates to `/mnt/c/Users/owner/Documents/docker/ollama/ollama:/root/.ollama` What you need is to create an explicit volume via `docker volume create ollama_data` and then reference it like `-v ollama_data:/root/.ollama` that ensures `/root/.ollama` path resides at `/var/lib/docker/volumes/` which is the default location for the volumes managed by docker in most linux distros. This location happens to be outside `/mnt/<location>` thus gives the performance. If you are using docker-compose : ```yaml services: ollama: ... volumes: - ollama_data:/root/.ollama # Declare volume here - docker will create it / reference it & reuse it as long as it exists. volumes: ollama_data: #external: true # Only needed if volume already exists from external sources ``` Declaring volume keeps the data on disk as long as volume exists with docker. It will stay persistent unlike temporary volumes that are created on-the-go basis. Please read [docker volumes documentation](https://docs.docker.com/engine/storage/volumes/) for more details, if you are still having trouble understanding how docker volumes work.
Author
Owner

@agibson-fl commented on GitHub (Oct 17, 2024):

@mxmp210 I follow this. Thank you for the more clear, direct example!

<!-- gh-comment-id:2418486209 --> @agibson-fl commented on GitHub (Oct 17, 2024): @mxmp210 I follow this. Thank you for the more clear, direct example!
Author
Owner

@Milor123 commented on GitHub (Dec 19, 2024):

-v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama `

This translates to /mnt/c/Users/owner/Documents/docker/ollama/ollama:/root/.ollama

What you need is to create an explicit volume via docker volume create ollama_data and then reference it like -v ollama_data:/root/.ollama that ensures /root/.ollama path resides at /var/lib/docker/volumes/ which is the default location for the volumes managed by docker in most linux distros. This location happens to be outside /mnt/<location> thus gives the performance.

If you are using docker-compose :

services:
  ollama:
    ...
    volumes:
      - ollama_data:/root/.ollama
 
# Declare volume here - docker will create it / reference it & reuse it as long as it exists.      
volumes:
  ollama_data:
    #external: true # Only needed if volume already exists from external sources

Declaring volume keeps the data on disk as long as volume exists with docker. It will stay persistent unlike temporary volumes that are created on-the-go basis.

Please read docker volumes documentation for more details, if you are still having trouble understanding how docker volumes work.

Ohh excellent, I came from linux and now I'm on windows, a lot of things have happened to me, and thanks to your comment I could understand and solve the problem.

My ollama was very slow, but since I moved all my files to the volume inside WSL then they load very fast as in linux.
Thanks guys!❤️

<!-- gh-comment-id:2552526235 --> @Milor123 commented on GitHub (Dec 19, 2024): > > ``` > > -v C:\Users\owner\Documents\docker\ollama\ollama:/root/.ollama ` > > ``` > > This translates to `/mnt/c/Users/owner/Documents/docker/ollama/ollama:/root/.ollama` > > What you need is to create an explicit volume via `docker volume create ollama_data` and then reference it like `-v ollama_data:/root/.ollama` that ensures `/root/.ollama` path resides at `/var/lib/docker/volumes/` which is the default location for the volumes managed by docker in most linux distros. This location happens to be outside `/mnt/<location>` thus gives the performance. > > If you are using docker-compose : > > ```yaml > services: > ollama: > ... > volumes: > - ollama_data:/root/.ollama > > # Declare volume here - docker will create it / reference it & reuse it as long as it exists. > volumes: > ollama_data: > #external: true # Only needed if volume already exists from external sources > ``` > > Declaring volume keeps the data on disk as long as volume exists with docker. It will stay persistent unlike temporary volumes that are created on-the-go basis. > > Please read [docker volumes documentation](https://docs.docker.com/engine/storage/volumes/) for more details, if you are still having trouble understanding how docker volumes work. Ohh excellent, I came from linux and now I'm on windows, a lot of things have happened to me, and thanks to your comment I could understand and solve the problem. My ollama was very slow, but since I moved all my files to the volume inside WSL then they load very fast as in linux. Thanks guys!❤️
Author
Owner

@gitnlsn commented on GitHub (Aug 11, 2025):

Hi.

I'm running llama3:1b in ollama docker. It seems that after some idle time the next call takes longer. Why I ask? Is it docker configuration or should I run ollama without docker?

I'm not expert at docker performance.

<!-- gh-comment-id:3174495143 --> @gitnlsn commented on GitHub (Aug 11, 2025): Hi. I'm running llama3:1b in ollama docker. It seems that after some idle time the next call takes longer. Why I ask? Is it docker configuration or should I run ollama without docker? I'm not expert at docker performance.
Author
Owner

@TafadzwaD commented on GitHub (Sep 1, 2025):

Hi.

I'm running llama3:1b in ollama docker. It seems that after some idle time the next call takes longer. Why I ask? Is it docker configuration or should I run ollama without docker?

I'm not expert at docker performance.

@gitnlsn You can specify OLLAMA_KEEP_ALIVE, environment variable, when you run your ollama to stop the model from being offloaded from memory when idle. This will solve your cold start issue, the downside is that ollama ends up holding memory it is not using when idle.

<!-- gh-comment-id:3241476023 --> @TafadzwaD commented on GitHub (Sep 1, 2025): > Hi. > > I'm running llama3:1b in ollama docker. It seems that after some idle time the next call takes longer. Why I ask? Is it docker configuration or should I run ollama without docker? > > I'm not expert at docker performance. @gitnlsn You can specify [`OLLAMA_KEEP_ALIVE`](https://github.com/ollama/ollama/pull/3094), environment variable, when you run your ollama to stop the model from being offloaded from memory when idle. This will solve your cold start issue, the downside is that ollama ends up holding memory it is not using when idle.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#29516