github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 05:36:16 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #5876] Allow file-only access #65702

New Issue
Closed
opened 2026-05-03 22:17:48 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-03 22:17:48 -05:00
Owner
Copy Link

Originally created by @QuickHare on GitHub (Jul 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5876

I would like to be able to use a model direct from its file without first loading it into memory.

This seems counter-productive, as in-memory would be much faster, but there are a few use cases for this.

  1. It works on low-memory systems.
  2. It allows for testing of a model.
  3. Two or more instances could use the same model file without using more space in memory, especially if the file is located in a tmpfs filesystem (in a ram disk, which is possible in Windows, Linux and MacOS).

If there are transformation for optimisation purposes upon load, can this be achieved whilst creating the model with a Modelfile?

I was looking to see if this was possible and, as part of my enquiries, turned to CoPilot to see if it knew of how to do it. It suggested there was a possible configuration file written in JSON that could do it with {load_in_memory: false} - but I don't see /etc/ollama/ollama.conf having been created. It is possible CoPilot was confused, as AI isn't always accurate.

Originally created by @QuickHare on GitHub (Jul 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5876 I would like to be able to use a model direct from its file without first loading it into memory. This seems counter-productive, as in-memory would be much faster, but there are a few use cases for this. 1. It works on low-memory systems. 2. It allows for testing of a model. 3. Two or more instances could use the same model file without using more space in memory, especially if the file is located in a tmpfs filesystem (in a ram disk, which is possible in Windows, Linux and MacOS). If there are transformation for optimisation purposes upon load, can this be achieved whilst creating the model with a Modelfile? I was looking to see if this was possible and, as part of my enquiries, turned to CoPilot to see if it knew of how to do it. It suggested there was a possible configuration file written in JSON that could do it with {load_in_memory: false} - but I don't see /etc/ollama/ollama.conf having been created. It is possible CoPilot was confused, as AI isn't always accurate.
GiteaMirror added the feature request label 2026-05-03 22:17:48 -05:00
GiteaMirror commented 2026-05-03 22:17:52 -05:00
Author
Owner
Copy Link

@igorschlum commented on GitHub (Aug 10, 2024):

@QuickHare, I understand your interest in using a model directly from its file without first loading it into memory. While this approach could be beneficial in certain scenarios, such as low-memory systems or for testing purposes, there are some important considerations.

Accessing the model directly from the file for each word output could be significantly slower compared to loading the model into memory. This is because the app would need to read and process data from the file repeatedly, which can introduce considerable latency. For instance, when Ollama previously allowed running models on macOS without checking memory availability, the output performance was notably slow, with delays of several seconds per character.

In terms of optimization, if there are transformations applied during the model loading process, these might not be feasible with direct file access. These transformations are typically designed to enhance performance and are applied when the model is initially loaded into memory.

If you're working with limited memory, it is generally more efficient to use a model that fits within your system's memory constraints. Alternatively, you might consider using Ollama on a shared server where multiple users can access the model, or leveraging an AI tool accessible via API. However, keep in mind that using an API may have implications for privacy and can incur costs based on token usage.

Regarding the JSON configuration file suggested by CoPilot, it seems there might have been some confusion. Currently, there is no /etc/ollama/ollama.conf file that supports a {load_in_memory: false} configuration for this purpose.

Given that this request does not seem feasible with the current capabilities and objectives of our project, we should focus on tracking issues related to bugs or realizable requests. Therefore, it would be best to close this issue.

<!-- gh-comment-id:2280751721 --> @igorschlum commented on GitHub (Aug 10, 2024): @QuickHare, I understand your interest in using a model directly from its file without first loading it into memory. While this approach could be beneficial in certain scenarios, such as low-memory systems or for testing purposes, there are some important considerations. Accessing the model directly from the file for each word output could be significantly slower compared to loading the model into memory. This is because the app would need to read and process data from the file repeatedly, which can introduce considerable latency. For instance, when Ollama previously allowed running models on macOS without checking memory availability, the output performance was notably slow, with delays of several seconds per character. In terms of optimization, if there are transformations applied during the model loading process, these might not be feasible with direct file access. These transformations are typically designed to enhance performance and are applied when the model is initially loaded into memory. If you're working with limited memory, it is generally more efficient to use a model that fits within your system's memory constraints. Alternatively, you might consider using Ollama on a shared server where multiple users can access the model, or leveraging an AI tool accessible via API. However, keep in mind that using an API may have implications for privacy and can incur costs based on token usage. Regarding the JSON configuration file suggested by CoPilot, it seems there might have been some confusion. Currently, there is no /etc/ollama/ollama.conf file that supports a {load_in_memory: false} configuration for this purpose. Given that this request does not seem feasible with the current capabilities and objectives of our project, we should focus on tracking issues related to bugs or realizable requests. Therefore, it would be best to close this issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#65702
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?