github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15948] Ollama leaving orphaned files after safetensors import #87843

New Issue
Closed
opened 2026-05-10 06:26:40 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-05-10 06:26:40 -05:00
Owner
Copy Link

Originally created by @levicki on GitHub (May 3, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15948

What is the issue?

This issue has been split from #15925 so it can be addressed separately.

Import of safetensors models leaves orphaned files in the blobs directory after conversion to GGUF is complete (success or failure, doesn't matter). Those files take a lot of space and user has no way of knowing which files are not used in order to delete them manually.

I propose using a FILE_FLAG_DELETE_ON_CLOSE when creating those files (at least on Windows, not sure what Linux / Mac equivalent is).

Relevant log output

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.22.1

Originally created by @levicki on GitHub (May 3, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15948 ### What is the issue? This issue has been split from #15925 so it can be addressed separately. Import of safetensors models leaves orphaned files in the `blobs` directory after conversion to `GGUF` is complete (success or failure, doesn't matter). Those files take a lot of space and user has no way of knowing which files are not used in order to delete them manually. I propose using a `FILE_FLAG_DELETE_ON_CLOSE` when creating those files (at least on Windows, not sure what Linux / Mac equivalent is). ### Relevant log output ```shell ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.22.1
GiteaMirror added the bug label 2026-05-10 06:26:40 -05:00
GiteaMirror commented 2026-05-10 06:26:42 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 3, 2026):

Don't manually delete them, a server restart will perform a housekeeping run which will remove unused blobs.

<!-- gh-comment-id:4367146777 --> @rick-github commented on GitHub (May 3, 2026): Don't manually delete them, a server restart will perform a housekeeping run which will remove unused blobs.
GiteaMirror commented 2026-05-10 06:26:42 -05:00
Author
Owner
Copy Link

@levicki commented on GitHub (May 3, 2026):

Don't manually delete them, a server restart will perform a housekeeping run which will remove unused blobs.

  1. I haven't noticed it doing that so far (I had leftover files staying between restarts), and I don't think that's documented behavior.
  2. Having to wait for (or force a) server restart after importing several large safetensors models to reclaim disk space is... inefficient to say the least. There's no good excuse not to clean up when you're done.

With that said, copying those files into blobs in order to import them is a massive waste of time and NVME write capacity to begin with — you should either read them from where they are or symlink them temporarily if they must be in that specific folder for import to work. But to me using model registry meant for permanent model storage (the folder user has configured for this) as a temporary folder for import is a design stink.

Housekeeping run can't address any of that.

<!-- gh-comment-id:4367438068 --> @levicki commented on GitHub (May 3, 2026): > Don't manually delete them, a server restart will perform a housekeeping run which will remove unused blobs. 1. I haven't noticed it doing that so far (I had leftover files staying between restarts), and I don't think that's documented behavior. 2. Having to wait for (or force a) server restart after importing several large safetensors models to reclaim disk space is... inefficient to say the least. There's no good excuse not to clean up when you're done. With that said, copying those files into `blobs` in order to import them is a massive waste of time and NVME write capacity to begin with &mdash; you should either read them from where they are or symlink them temporarily if they must be in that specific folder for import to work. But to me using model registry meant for permanent model storage (the folder user has configured for this) as a temporary folder for import is a design stink. Housekeeping run can't address any of that.
GiteaMirror commented 2026-05-10 06:26:43 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 5, 2026):

Ollama has two modes during ollama create;

  • the first (and original) mode is client/server based where you run a local server which the ollama CLI client pushes all of the files it needs to the local server in order to perform whatever work needs to be done. This is the mode that you're using right now.
  • there is a second newer mode which only works for the MLX runner on MacOS which runs in the client and does not push everything to the server (you use the --experimental flag to trigger this right now).

What we're moving to is a blend of the two where you can specify to run it on the sever, or you can run it locally without copying all of the files through the client. That won't be ready for a little while though unfortunately as we transition the various runners.

@rick-github is correct that the server does have a garbage collection mode and will remove any blobs that aren't in use. For now you can just restart the server and it will remove any unused file. I'm going to go ahead and close the issue.

<!-- gh-comment-id:4383387781 --> @pdevine commented on GitHub (May 5, 2026): Ollama has two modes during `ollama create`; - the first (and original) mode is client/server based where you run a local server which the ollama CLI client pushes all of the files it needs to the local server in order to perform whatever work needs to be done. This is the mode that you're using right now. - there is a second newer mode which only works for the MLX runner on MacOS which runs in the client and does not push everything to the server (you use the --experimental flag to trigger this right now). What we're moving to is a blend of the two where you can specify to run it on the sever, or you can run it locally without copying all of the files through the client. That won't be ready for a little while though unfortunately as we transition the various runners. @rick-github is correct that the server does have a garbage collection mode and will remove any blobs that aren't in use. For now you can just restart the server and it will remove any unused file. I'm going to go ahead and close the issue.
GiteaMirror commented 2026-05-10 06:26:43 -05:00
Author
Owner
Copy Link

@levicki commented on GitHub (May 5, 2026):

@pdevine

I am getting a vibe that you haven't even read what I wrote here?

First, I am not using MacOS so I don't see how an experimental MLX runner for MacOS helps me?

Second regarding this:

What we're moving to is a blend of the two where you can specify to run it on the sever, or you can run it locally without copying all of the files through the client. That won't be ready for a little while though unfortunately as we transition the various runners.

This isn't about running a model at all, it's about import from safetensors into model registry and only about that.

Third, even if server has garbage collection it doesn't need to trash 116 GB of NVME writes just to import a model when you can symlink the original files temporarily even on windows. Not to mention that ollama server process has elevated rights and symlink privilege enabled and also impersonation so it can access the files wherever they are (maybe web server can't but server process itself can).

So it's all about the current architectural cludge and resulting massive waste of resources, but if you are saying there's nothing to be fixed I'll go away.

<!-- gh-comment-id:4383688320 --> @levicki commented on GitHub (May 5, 2026): @pdevine I am getting a vibe that you haven't even read what I wrote here? First, I am not using MacOS so I don't see how an experimental MLX runner for MacOS helps me? Second regarding this: > What we're moving to is a blend of the two where you can specify to run it on the sever, or you can run it locally without copying all of the files through the client. That won't be ready for a little while though unfortunately as we transition the various runners. This isn't about running a model at all, it's about import from safetensors into model registry and only about that. Third, even if server has garbage collection it doesn't need to trash 116 GB of NVME writes just to import a model when you can symlink the original files temporarily even on windows. Not to mention that ollama server process has elevated rights and symlink privilege enabled and also impersonation so it can access the files wherever they are (maybe web server can't but server process itself can). So it's all about the current architectural cludge and resulting massive waste of resources, but if you are saying there's nothing to be fixed I'll go away.
GiteaMirror commented 2026-05-10 06:26:43 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 5, 2026):

@levicki yes, I read what you wrote. I wasn't referring to running a model on the server, I was referring to running the convert process on the server. Ollama by its very nature is client/server based (it's why there is an OLLAMA_HOST environment setting for changing what the client connects to).

We've been changing the architecture and we're kind of in an "in-between" state right now (with the two modes) as we work through the differences.

<!-- gh-comment-id:4383705408 --> @pdevine commented on GitHub (May 5, 2026): @levicki yes, I read what you wrote. I wasn't referring to running a model on the server, I was referring to running the _convert process_ on the server. Ollama by its very nature is client/server based (it's why there is an OLLAMA_HOST environment setting for changing what the client connects to). We've been changing the architecture and we're kind of in an "in-between" state right now (with the two modes) as we work through the differences.
GiteaMirror commented 2026-05-10 06:26:44 -05:00
Author
Owner
Copy Link

@levicki commented on GitHub (May 5, 2026):

@pdevine Ok then, I understand that there's a client server split of course but I still don't understand why files havve to be copied (or as you say "uploaded") in order to be converted. Is there any ETA for avoiding pointless copies? That's all I am interested in. Also if ollama supported separate mmproj file instead of just merged text+vision I wouldn't bother to import safetensors through it — I'd just import already available and quantized GGUF files from HF hub.

<!-- gh-comment-id:4383729117 --> @levicki commented on GitHub (May 5, 2026): @pdevine Ok then, I understand that there's a client server split of course but I still don't understand why files havve to be copied (or as you say "uploaded") in order to be converted. Is there any ETA for avoiding pointless copies? That's all I am interested in. Also if ollama supported separate mmproj file instead of just merged text+vision I wouldn't bother to import safetensors through it &mdash; I'd just import already available and quantized GGUF files from HF hub.
GiteaMirror commented 2026-05-10 06:26:44 -05:00
Author
Owner
Copy Link

@levicki commented on GitHub (May 5, 2026):

To clarify, because of inability to use separate mmproj in ollama I currently have to use the following workflow:

  1. Download 60 GB BF16 safetensors instead of pre-quantized 20 GB weights and 1.2 GB mmproj GGUF files
  2. Let ollama write 60 GB safetensors to blobs on import + 60 GB GGUF it creates because text+vision model has to be a single file and llama.cpp can't produce such file
  3. Quantize using llama-quantize because despite llama.cpp backend supporting Q5_K_M ollama doesn't support it
  4. Import quantized GGUF into ollama (another 22 GB of writes)

If there is a better way feel free to let me know because the waste of resources involved is almost painful.

<!-- gh-comment-id:4383762783 --> @levicki commented on GitHub (May 5, 2026): To clarify, because of inability to use separate mmproj in ollama I currently have to use the following workflow: 1. Download 60 GB BF16 safetensors instead of pre-quantized 20 GB weights and 1.2 GB mmproj GGUF files 2. Let ollama write 60 GB safetensors to blobs on import + 60 GB GGUF it creates because text+vision model has to be a single file and llama.cpp can't produce such file 3. Quantize using llama-quantize because despite llama.cpp backend supporting Q5_K_M ollama doesn't support it 4. Import quantized GGUF into ollama (another 22 GB of writes) If there is a better way feel free to let me know because the waste of resources involved is almost painful.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87843
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?