github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #973] nix flake #475

New Issue
Open
opened 2026-04-12 10:09:20 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 10:09:20 -05:00
Owner
Copy Link

Originally created by @ck3mp3r on GitHub (Nov 2, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/973

Would be great to have a nix flake to be able to consume it via nix flakes on flakehub.com

Originally created by @ck3mp3r on GitHub (Nov 2, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/973 Would be great to have a nix flake to be able to consume it via nix flakes on flakehub.com
GiteaMirror added the feature request label 2026-04-12 10:09:20 -05:00
GiteaMirror commented 2026-04-12 10:09:21 -05:00
Author
Owner
Copy Link

@DrRuhe commented on GitHub (Nov 3, 2023):

ollama is already packaged in nixpkgs (currently only on version 0.0.17, but a version bump is in the works)

The benefit of a flake would be the ability to expose a library that allows building the models from nix. Something like this maybe:

{
    
    buildModelfile = {
        from,
        parameters ? {},
        template ? null,
        system ? "",
        adapters ? [],
        license ? null
    }:let
        l = pkgs.lib // builtins;
        parametersString = (l.concatStringSep "\n" (l.mapAttrs (param: val: "PARAMETER ${param} ${value}") parameters));
        templateString = if (builtins.isString template) then ''TEMPLATE """${template}"""'' else "";
        systemString = if (builtins.isString template) then ''SYSTEM """${template}"""'' else "";
        licenseString = if (builtins.isString template) then ''LICENSE """${template}"""'' else "";
        adaptersString = (l.concatStringSep "\n" (l.map (adapter: "ADAPTER ${adapter}") adapters));
    in pkgs.writeTextFile "Modulefile" ''
        FROM ${from}
        ${parametersString}
        ${templateString}
        ${systemString}
        ${adapters}
        ${licenseString}
    '';
}
<!-- gh-comment-id:1792250526 --> @DrRuhe commented on GitHub (Nov 3, 2023): ollama is[ already packaged in nixpkgs](https://github.com/NixOS/nixpkgs/blob/611d4f4ebe5c0ecbf584e554681d57920e968f50/pkgs/tools/misc/ollama/default.nix#L9) (currently only on version 0.0.17, but a [version bump](https://github.com/NixOS/nixpkgs/pull/257760) is in the works) The benefit of a flake would be the ability to expose a library that allows building the models from nix. Something like this maybe: ```nix { buildModelfile = { from, parameters ? {}, template ? null, system ? "", adapters ? [], license ? null }:let l = pkgs.lib // builtins; parametersString = (l.concatStringSep "\n" (l.mapAttrs (param: val: "PARAMETER ${param} ${value}") parameters)); templateString = if (builtins.isString template) then ''TEMPLATE """${template}"""'' else ""; systemString = if (builtins.isString template) then ''SYSTEM """${template}"""'' else ""; licenseString = if (builtins.isString template) then ''LICENSE """${template}"""'' else ""; adaptersString = (l.concatStringSep "\n" (l.map (adapter: "ADAPTER ${adapter}") adapters)); in pkgs.writeTextFile "Modulefile" '' FROM ${from} ${parametersString} ${templateString} ${systemString} ${adapters} ${licenseString} ''; } ```
GiteaMirror commented 2026-04-12 10:09:22 -05:00
Author
Owner
Copy Link

@ck3mp3r commented on GitHub (Nov 3, 2023):

Having a flake would also just a allow nix run github:jmorganca/ollama which doesn't require changing to unstable just to be able to install it...

<!-- gh-comment-id:1792373078 --> @ck3mp3r commented on GitHub (Nov 3, 2023): Having a flake would also just a allow `nix run github:jmorganca/ollama` which doesn't require changing to unstable just to be able to install it...
GiteaMirror commented 2026-04-12 10:09:22 -05:00
Author
Owner
Copy Link

@nixinator commented on GitHub (Dec 13, 2023):

The version in nixpkgs doesn't seem to work with CUDA, because it uses the llama-cpp package from nixpkgs.

This handy flake fixes the problem. It works well, and GPU acceleration works too!

It avoids the ollama bring in llama-cpp as a submodule too, which IMHO is a good thing.

https://github.com/havaker/ollama-nix

I'm not sure if @havaker would contribute their flake to the repo, but it that is the case it does open up a whole raft of end to end integration tests that can be run in nix ci and using the nixos testing framework.

<!-- gh-comment-id:1854153302 --> @nixinator commented on GitHub (Dec 13, 2023): The version in nixpkgs doesn't seem to work with CUDA, because it uses the llama-cpp package from nixpkgs. This handy flake fixes the problem. It works well, and GPU acceleration works too! It avoids the ollama bring in llama-cpp as a submodule too, which IMHO is a good thing. https://github.com/havaker/ollama-nix I'm not sure if @havaker would contribute their flake to the repo, but it that is the case it does open up a whole raft of end to end integration tests that can be run in nix ci and using the nixos testing framework.
GiteaMirror commented 2026-04-12 10:09:22 -05:00
Author
Owner
Copy Link

@sbatial commented on GitHub (Mar 5, 2024):

@ck3mp3r You can just as well do nix run nixpkgs/nixos-unstable#ollama.
That won't change it globally.

(Or do nix run github:nixos/nixpkgs/nixos-unstable#ollama for that matter.. but that's kind of unnecessary I'd say)

That does not address other problems that package might have of course.. but you are pretty free in mixing-and-matching with 'native' nix!

More examples on https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake#flake-references

<!-- gh-comment-id:1979808206 --> @sbatial commented on GitHub (Mar 5, 2024): @ck3mp3r You can just as well do `nix run nixpkgs/nixos-unstable#ollama`. That won't change it globally. (Or do `nix run github:nixos/nixpkgs/nixos-unstable#ollama` for that matter.. but that's kind of unnecessary I'd say) That does not address other problems that package might have of course.. but you are pretty free in mixing-and-matching with 'native' nix! More examples on https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake#flake-references
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#475
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?