github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2418] What are the system requirements? #63448

New Issue
Closed
opened 2026-05-03 13:33:36 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-05-03 13:33:36 -05:00
Owner
Copy Link

Originally created by @worikgh on GitHub (Feb 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2418

It would be very useful to have a section on system requirements in the README.md

Nothing too detailed, but:

  • Disc space required
  • Main ram
  • Video/Compute card requirements

Keep up the good work!

Originally created by @worikgh on GitHub (Feb 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2418 It would be very useful to have a section on system requirements in the README.md Nothing too detailed, but: * Disc space required * Main ram * Video/Compute card requirements Keep up the good work!
GiteaMirror commented 2026-05-03 13:33:40 -05:00
Author
Owner
Copy Link

@Dax911 commented on GitHub (Feb 9, 2024):

Minimal as for the software, but it is entirely dependent on what kind of model you are trying to run. In theory it is more about what your hardware can support than any minimum specs they are building for.

<!-- gh-comment-id:1935227360 --> @Dax911 commented on GitHub (Feb 9, 2024): Minimal as for the software, but it is entirely dependent on what kind of model you are trying to run. In theory it is more about what your hardware can support than any minimum specs they are building for.
GiteaMirror commented 2026-05-03 13:33:41 -05:00
Author
Owner
Copy Link

@davidthewatson commented on GitHub (Feb 13, 2024):

I concur with @worikgh while realizing the depth in what @Dax911 has stated.

A simple table of models to be used as a quick binary or ternary (yes=green, no=red, maybe=yellow) heuristic to choose deployment platforms and their requirements as to GPU may be helpful. For instance, a table that listed columns such as Model, CPU, and GPU would enable users to make decisions before downloading as to what hardware to target for successful deployment, i.e. where it's likely that 2-3 combinations of deployment hardware parameters have predictable success at deployment-time, where others may be more edge case due to their emergence or complexity. Having deployed Docker on WSL2 on NVidia, I've seen that complexity first-hand.

A quick search of "GPU" in issues gives a rough idea of the implied complexity of deployment, with the top 3 being in the last 5 days, like this issue, and also covering AMD, Nvidia, and Docker, and WSL2:

https://github.com/ollama/ollama/issues?q=is%3Aissue+is%3Aopen+gpu

This would make it much simpler to just know which hardware combo to choose for deployment in a home or research lab, where desktop, mobile, cloud, and embedded environments up to and including clusters of AMD, ARM, Apple, Intel, and NVidia can be deployed to support platforms like ollama.

Thanks!

<!-- gh-comment-id:1942266540 --> @davidthewatson commented on GitHub (Feb 13, 2024): I concur with @worikgh while realizing the depth in what @Dax911 has stated. A simple table of models to be used as a quick binary or ternary (yes=green, no=red, maybe=yellow) heuristic to choose deployment platforms and their requirements as to GPU may be helpful. For instance, a table that listed columns such as Model, CPU, and GPU would enable users to make decisions before downloading as to what hardware to target for successful deployment, i.e. where it's likely that 2-3 combinations of deployment hardware parameters have predictable success at deployment-time, where others may be more edge case due to their emergence or complexity. Having deployed Docker on WSL2 on NVidia, I've seen that complexity first-hand. A quick search of "GPU" in issues gives a rough idea of the implied complexity of deployment, with the top 3 being in the last 5 days, like this issue, and also covering AMD, Nvidia, and Docker, and WSL2: https://github.com/ollama/ollama/issues?q=is%3Aissue+is%3Aopen+gpu This would make it much simpler to just know which hardware combo to choose for deployment in a home or research lab, where desktop, mobile, cloud, and embedded environments up to and including clusters of AMD, ARM, Apple, Intel, and NVidia can be deployed to support platforms like ollama. Thanks!
GiteaMirror commented 2026-05-03 13:33:44 -05:00
Author
Owner
Copy Link

@Dax911 commented on GitHub (Feb 13, 2024):

I concur with @worikgh while realizing the depth in what @Dax911 has stated.

A simple table of models to be used as a quick binary or ternary (yes=green, no=red, maybe=yellow) heuristic to choose deployment platforms and their requirements as to GPU may be helpful. For instance, a table that listed columns such as Model, CPU, and GPU would enable users to make decisions before downloading as to what hardware to target for successful deployment, i.e. where it's likely that 2-3 combinations of deployment hardware parameters have predictable success at deployment-time, where others may be more edge case due to their emergence or complexity. Having deployed Docker on WSL2 on NVidia, I've seen that complexity first-hand.

A quick search of "GPU" in issues gives a rough idea of the implied complexity of deployment, with the top 3 being in the last 5 days, like this issue, and also covering AMD, Nvidia, and Docker, and WSL2:

https://github.com/ollama/ollama/issues?q=is%3Aissue+is%3Aopen+gpu

This would make it much simpler to just know which hardware combo to choose for deployment in a home or research lab, where desktop, mobile, cloud, and embedded environments up to and including clusters of AMD, ARM, Apple, Intel, and NVidia can be deployed to support platforms like ollama.

Thanks!

Not the place for it

The suggestion to incorporate hardware compatibility benchmarking within Ollama overlooks the inherent complexity and variability involved in assessing model performance across diverse hardware configurations. While Ollama excels in facilitating the deployment of AI models, expecting it to encompass benchmarking functionalities places undue burden on the devs. Benchmarking involves rigorous testing and validation processes, including performance optimization and comparison across multiple hardware setups.

Furthermore, the responsibility for benchmarking and determining supported hardware configurations primarily lies with the model developers. They possess the necessary expertise and domain knowledge to optimize their models for specific hardware environments. Expecting Ollama to provide exhaustive support for all possible hardware configurations is impractical and unfeasible.

Ultimately, users should collaborate closely with model developers to assess performance and compatibility across different hardware setups. This collaborative approach ensures that users receive tailored recommendations and support based on their specific deployment requirements.

1. Lack of Feasibility

While acknowledging the complexity of hardware configurations and deployment environments, it's important to note that Ollama already runs for any given model with a specific file type. The assertion that extending this functionality to include hardware specifications is unfeasible. Additionally, comparing the complexity of managing hardware configurations to managing AI models is not entirely applicable, as Ollama primarily deals with the latter.

2. Proposal for a Quick Reference Table

The proposal for a quick reference table to aid users in selecting deployment platforms based on hardware requirements is commendable. However, it's essential to recognize that such a table would only serve as a general heuristic and may not encompass all possible deployment scenarios accurately. Hardware compatibility often depends on various factors beyond just CPU and GPU specifications, such as driver compatibility, firmware versions, and underlying software dependencies. Therefore, while a reference table could be useful as a starting point, it should not be relied upon as the sole determinant of deployment success.

3. Linux can't even do this for all its distros you expect ollama to?

The assertion that Ollama should provide certainty regarding hardware compatibility across different system distributions overlooks the inherent variability and complexity within the Mac, Linux and Windows ecosystems. With numerous distributions, each offering unique kernel versions, package managers, and configurations, guaranteeing compatibility with all hardware configurations is virtually impossible. Even major distributions like Ubuntu, Fedora, and CentOS may exhibit differences in hardware support depending on factors such as kernel version and driver availability. We can't even guarantee a specific graphics card will work with a given distro or version and yet you're expecting Ollama to provide definitive guidance on hardware compatibility across a majority of distributions? This is unrealistic and impractical.

4. Testing Requirements

Implementing hardware compatibility checks within Ollama would necessitate extensive testing across a diverse range of hardware configurations, including CPUs, GPUs, and other peripherals. This testing process would be resource-intensive and time-consuming, requiring continuous updates and validation to ensure accuracy and reliability. While the benefits of such functionality are evident, it's essential to consider the trade-offs in terms of development resources and project priorities. Prioritizing features that directly contribute to Ollama's core functionality and user experience may be more beneficial in the short term.

What you are asking for is closer to a service like PC Benchmarking services. Not something the ollama team wants to do. By all means I think such a service would be kick ass, but this is not the place for it.

TL;DR

The suggestion to incorporate hardware compatibility benchmarking within Ollama is impractical and places undue burden on the developers. Such benchmarking involves complex testing processes and is primarily the responsibility of model developers. Additionally, guaranteeing compatibility across various system distributions, including Linux, is unrealistic given the inherent variability within these ecosystems. Implementing hardware compatibility checks would require extensive testing and resources, which may not align with Ollama's core mission. Instead, users should collaborate with model developers to assess performance and compatibility across different hardware setups.

<!-- gh-comment-id:1942401516 --> @Dax911 commented on GitHub (Feb 13, 2024): > I concur with @worikgh while realizing the depth in what @Dax911 has stated. > > A simple table of models to be used as a quick binary or ternary (yes=green, no=red, maybe=yellow) heuristic to choose deployment platforms and their requirements as to GPU may be helpful. For instance, a table that listed columns such as Model, CPU, and GPU would enable users to make decisions before downloading as to what hardware to target for successful deployment, i.e. where it's likely that 2-3 combinations of deployment hardware parameters have predictable success at deployment-time, where others may be more edge case due to their emergence or complexity. Having deployed Docker on WSL2 on NVidia, I've seen that complexity first-hand. > > A quick search of "GPU" in issues gives a rough idea of the implied complexity of deployment, with the top 3 being in the last 5 days, like this issue, and also covering AMD, Nvidia, and Docker, and WSL2: > > https://github.com/ollama/ollama/issues?q=is%3Aissue+is%3Aopen+gpu > > This would make it much simpler to just know which hardware combo to choose for deployment in a home or research lab, where desktop, mobile, cloud, and embedded environments up to and including clusters of AMD, ARM, Apple, Intel, and NVidia can be deployed to support platforms like ollama. > > Thanks! ### Not the place for it The suggestion to incorporate hardware compatibility benchmarking within Ollama overlooks the inherent complexity and variability involved in assessing model performance across diverse hardware configurations. While Ollama excels in facilitating the deployment of AI models, expecting it to encompass benchmarking functionalities places undue burden on the devs. Benchmarking involves rigorous testing and validation processes, including performance optimization and comparison across multiple hardware setups. Furthermore, the responsibility for benchmarking and determining supported hardware configurations primarily lies with the model developers. They possess the necessary expertise and domain knowledge to optimize their models for specific hardware environments. Expecting Ollama to provide exhaustive support for all possible hardware configurations is impractical and unfeasible. Ultimately, users should collaborate closely with model developers to assess performance and compatibility across different hardware setups. This collaborative approach ensures that users receive tailored recommendations and support based on their specific deployment requirements. #### 1. Lack of Feasibility While acknowledging the complexity of hardware configurations and deployment environments, it's important to note that Ollama already runs for any given model with a specific file type. The assertion that extending this functionality to include hardware specifications is unfeasible. Additionally, comparing the complexity of managing hardware configurations to managing AI models is not entirely applicable, as Ollama primarily deals with the latter. #### 2. Proposal for a Quick Reference Table The proposal for a quick reference table to aid users in selecting deployment platforms based on hardware requirements is commendable. However, it's essential to recognize that such a table would only serve as a general heuristic and may not encompass all possible deployment scenarios accurately. Hardware compatibility often depends on various factors beyond just CPU and GPU specifications, such as driver compatibility, firmware versions, and underlying software dependencies. Therefore, while a reference table could be useful as a starting point, it should not be relied upon as the sole determinant of deployment success. #### 3. Linux can't even do this for all its distros you expect ollama to? The assertion that Ollama should provide certainty regarding hardware compatibility across different system distributions overlooks the inherent variability and complexity within the Mac, Linux and Windows ecosystems. With numerous distributions, each offering unique kernel versions, package managers, and configurations, guaranteeing compatibility with all hardware configurations is virtually impossible. Even major distributions like Ubuntu, Fedora, and CentOS may exhibit differences in hardware support depending on factors such as kernel version and driver availability. We can't even guarantee a specific graphics card will work with a given distro or version and yet you're expecting Ollama to provide definitive guidance on hardware compatibility across a majority of distributions? This is unrealistic and impractical. #### 4. Testing Requirements Implementing hardware compatibility checks within Ollama would necessitate extensive testing across a diverse range of hardware configurations, including CPUs, GPUs, and other peripherals. This testing process would be resource-intensive and time-consuming, requiring continuous updates and validation to ensure accuracy and reliability. While the benefits of such functionality are evident, it's essential to consider the trade-offs in terms of development resources and project priorities. Prioritizing features that directly contribute to Ollama's core functionality and user experience may be more beneficial in the short term. What you are asking for is closer to a service like [PC Benchmarking](https://www.userbenchmark.com/Software) services. Not something the ollama team wants to do. By all means I think such a service would be kick ass, but this is not the place for it. #### TL;DR The suggestion to incorporate hardware compatibility benchmarking within Ollama is impractical and places undue burden on the developers. Such benchmarking involves complex testing processes and is primarily the responsibility of model developers. Additionally, guaranteeing compatibility across various system distributions, including Linux, is unrealistic given the inherent variability within these ecosystems. Implementing hardware compatibility checks would require extensive testing and resources, which may not align with Ollama's core mission. Instead, users should collaborate with model developers to assess performance and compatibility across different hardware setups.
GiteaMirror commented 2026-05-03 13:33:46 -05:00
Author
Owner
Copy Link

@worikgh commented on GitHub (Feb 18, 2024):

Yes. Point taken.

<!-- gh-comment-id:1951034161 --> @worikgh commented on GitHub (Feb 18, 2024): Yes. Point taken.
GiteaMirror commented 2026-05-03 13:33:49 -05:00
Author
Owner
Copy Link

@mavwolverine commented on GitHub (Jul 10, 2024):

Sorry to comment on a closed issue, but since it is related asking it here. Is there any way to figure out minimum hardware requirements for a model?

For ex I tried codestral on g4dn.xlarge and it could not complete the inference "pu VRAM usage didn't recover within timeout".

Thanks

<!-- gh-comment-id:2220618208 --> @mavwolverine commented on GitHub (Jul 10, 2024): Sorry to comment on a closed issue, but since it is related asking it here. Is there any way to figure out minimum hardware requirements for a model? For ex I tried codestral on g4dn.xlarge and it could not complete the inference "pu VRAM usage didn't recover within timeout". Thanks
GiteaMirror commented 2026-05-03 13:33:49 -05:00
Author
Owner
Copy Link

@Dax911 commented on GitHub (Jul 12, 2024):

Sorry to comment on a closed issue, but since it is related asking it here. Is there any way to figure out minimum hardware requirements for a model?

For ex I tried codestral on g4dn.xlarge and it could not complete the inference "pu VRAM usage didn't recover within timeout".

Thanks

Check the HuggingFace or dev site (the source of truth for ur model) where ever it was originally published should have information on min specs if not SOL.

<!-- gh-comment-id:2226288098 --> @Dax911 commented on GitHub (Jul 12, 2024): > Sorry to comment on a closed issue, but since it is related asking it here. Is there any way to figure out minimum hardware requirements for a model? > > For ex I tried codestral on g4dn.xlarge and it could not complete the inference "pu VRAM usage didn't recover within timeout". > > Thanks Check the HuggingFace or dev site (the source of truth for ur model) where ever it was originally published should have information on min specs if not SOL.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#63448
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?