github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #258] Ollama running in Dockerfile #109

New Issue
Closed
opened 2026-04-12 09:38:43 -05:00 by GiteaMirror · 14 comments
GiteaMirror commented 2026-04-12 09:38:43 -05:00
Owner
Copy Link

Originally created by @osamanatouf2 on GitHub (Aug 2, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/258

@jmorganca @mxyng I got ./ollama serve to work in docker. The only issue is that I am not able to pull down the files for other models like llama2 via the commad ./ollama pull llama2. I have tested the same configuration on ubuntu and works fine. Just inside the docker I get the following issue: Error: Post "http://127.0.0.1:11434/api/pull": dial tcp 127.0.0.1:11434: connect: connection refused
Here is my Dockerfile:

ROM ubuntu:latest

RUN apt-get update

RUN apt-get install -y git

WORKDIR /home

RUN (cd /home; git clone https://github.com/jmorganca/ollama.git)
RUN apt-get install -y wget
RUN wget https://golang.org/dl/go1.20.7.linux-amd64.tar.gz -O go.tar.gz
RUN apt-get install -y gcc
RUN apt-get install -y g++
RUN tar -C /usr/local -xzf go.tar.gz
ENV PATH=$PATH:/usr/local/go/bin
RUN go version
RUN rm go.tar.gz
WORKDIR /home/ollama
RUN go build .
RUN ./ollama pull llama2
EXPOSE 11434
RUN ./ollama serve &

and it only fails on the line RUN ./ollama pull llama2 and I have no clue where the issue is coming from:

Originally created by @osamanatouf2 on GitHub (Aug 2, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/258 @jmorganca @mxyng I got ./ollama serve to work in docker. The only issue is that I am not able to pull down the files for other models like llama2 via the commad ./ollama pull llama2. I have tested the same configuration on ubuntu and works fine. Just inside the docker I get the following issue: ```Error: Post "http://127.0.0.1:11434/api/pull": dial tcp 127.0.0.1:11434: connect: connection refused``` Here is my Dockerfile: ``` ROM ubuntu:latest RUN apt-get update RUN apt-get install -y git WORKDIR /home RUN (cd /home; git clone https://github.com/jmorganca/ollama.git) RUN apt-get install -y wget RUN wget https://golang.org/dl/go1.20.7.linux-amd64.tar.gz -O go.tar.gz RUN apt-get install -y gcc RUN apt-get install -y g++ RUN tar -C /usr/local -xzf go.tar.gz ENV PATH=$PATH:/usr/local/go/bin RUN go version RUN rm go.tar.gz WORKDIR /home/ollama RUN go build . RUN ./ollama pull llama2 EXPOSE 11434 RUN ./ollama serve & ``` and it only fails on the line RUN ./ollama pull llama2 and I have no clue where the issue is coming from:
GiteaMirror added the question label 2026-04-12 09:38:43 -05:00
GiteaMirror commented 2026-04-12 09:38:44 -05:00
Author
Owner
Copy Link

@steventkrawczyk commented on GitHub (Aug 2, 2023):

Hi @jmorganca Could you take a look? @osamanatouf2 is helping us integrate with Ollama and he ran into some issues

<!-- gh-comment-id:1662511977 --> @steventkrawczyk commented on GitHub (Aug 2, 2023): Hi @jmorganca Could you take a look? @osamanatouf2 is helping us integrate with Ollama and he ran into some issues
GiteaMirror commented 2026-04-12 09:38:45 -05:00
Author
Owner
Copy Link

@BruceMacD commented on GitHub (Aug 2, 2023):

The issue here is that the ollama CLI sends a pull request to the server, so in this case when ollama pull llama2 is run it fails because ollama serve hasn't been run yet to start the server.

Maybe this will work:

FROM ubuntu:latest

RUN apt-get update

RUN apt-get install -y git

WORKDIR /home

RUN (cd /home; git clone https://github.com/jmorganca/ollama.git)
RUN apt-get install -y wget
RUN wget https://golang.org/dl/go1.20.7.linux-amd64.tar.gz -O go.tar.gz
RUN apt-get install -y gcc
RUN apt-get install -y g++
RUN tar -C /usr/local -xzf go.tar.gz
ENV PATH=$PATH:/usr/local/go/bin
RUN go version
RUN rm go.tar.gz
WORKDIR /home/ollama
RUN go build .
EXPOSE 11434
RUN ./ollama serve &
RUN ./ollama pull llama2
<!-- gh-comment-id:1662666455 --> @BruceMacD commented on GitHub (Aug 2, 2023): The issue here is that the ollama CLI sends a pull request to the server, so in this case when `ollama pull llama2` is run it fails because `ollama serve` hasn't been run yet to start the server. Maybe this will work: ``` FROM ubuntu:latest RUN apt-get update RUN apt-get install -y git WORKDIR /home RUN (cd /home; git clone https://github.com/jmorganca/ollama.git) RUN apt-get install -y wget RUN wget https://golang.org/dl/go1.20.7.linux-amd64.tar.gz -O go.tar.gz RUN apt-get install -y gcc RUN apt-get install -y g++ RUN tar -C /usr/local -xzf go.tar.gz ENV PATH=$PATH:/usr/local/go/bin RUN go version RUN rm go.tar.gz WORKDIR /home/ollama RUN go build . EXPOSE 11434 RUN ./ollama serve & RUN ./ollama pull llama2 ```
GiteaMirror commented 2026-04-12 09:38:45 -05:00
Author
Owner
Copy Link

@osamanatouf2 commented on GitHub (Aug 2, 2023):

@BruceMacD Does not seem to be the issue. Were you successful in running the build?

<!-- gh-comment-id:1662709674 --> @osamanatouf2 commented on GitHub (Aug 2, 2023): @BruceMacD Does not seem to be the issue. Were you successful in running the build?
GiteaMirror commented 2026-04-12 09:38:45 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Aug 2, 2023):

There's a fundamental problem with this Dockerfile. It's not good practice to run background processes like since a Dockerfile is a recipe to generating a Docker image and is not equivalent to the runtime of the container.

An example Dockerfile exists in the repo. This give you a container that runs ollama serve by default:

docker build -t ollama -f Dockerfile .
docker run -d -p 11434:11434 -v $HOME/.ollama:/home/ollama/.ollama ollama
ollama pull llama2

@osamanatouf2 can you elaborate on what you're trying to achieve so we can understand the problem?

<!-- gh-comment-id:1662730988 --> @mxyng commented on GitHub (Aug 2, 2023): There's a fundamental problem with this Dockerfile. It's not good practice to run background processes like since a Dockerfile is a recipe to generating a Docker image and is not equivalent to the runtime of the container. An example Dockerfile exists in the repo. This give you a container that runs `ollama serve` by default: ``` docker build -t ollama -f Dockerfile . docker run -d -p 11434:11434 -v $HOME/.ollama:/home/ollama/.ollama ollama ollama pull llama2 ``` @osamanatouf2 can you elaborate on what you're trying to achieve so we can understand the problem?
GiteaMirror commented 2026-04-12 09:38:46 -05:00
Author
Owner
Copy Link

@osamanatouf2 commented on GitHub (Aug 2, 2023):

@mxyng I am trying to get ollama as a stand-alone service that we ship with a docker container and integrate it into another library promptstool library in this case. ollama does not work across all systems and that is the reason why I choose docker. I got it working on Ubuntu locally, but does not seem to be working in Dockerfile env.

<!-- gh-comment-id:1662893937 --> @osamanatouf2 commented on GitHub (Aug 2, 2023): @mxyng I am trying to get ollama as a stand-alone service that we ship with a docker container and integrate it into another library promptstool library in this case. ollama does not work across all systems and that is the reason why I choose docker. I got it working on Ubuntu locally, but does not seem to be working in Dockerfile env.
GiteaMirror commented 2026-04-12 09:38:46 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Aug 2, 2023):

I see. In that case you have two options:

  1. Build a Docker image using the Dockerfile in the repo as a starting point and COPY in a local model. Keep in mind docker build can only COPY files in its context so you'll need to link ~/.ollama/models to your docker build context. This creates a large container (GBs) where both ollama server and llama2 exist in a single Docker image.
  2. Build a Docker image using the Dockerfile in the repo and integrate it into your library. At runtime, configure the model which will pull it from the registry. The container will be small but the end user will still need to pull the image.

There's no functional difference between either methods. Either way, the end user will need to pull both ollama and model.

I suggest option 2 because the container can be reused for other models, i.e. if the end user wants to use something other than llama2.

<!-- gh-comment-id:1663032510 --> @mxyng commented on GitHub (Aug 2, 2023): I see. In that case you have two options: 1. Build a Docker image using the Dockerfile in the repo as a starting point and `COPY` in a local model. Keep in mind `docker build` can only `COPY` files in its context so you'll need to link `~/.ollama/models` to your `docker build` context. This creates a large container (GBs) where both `ollama server` and `llama2` exist in a single Docker image. 2. Build a Docker image using the Dockerfile in the repo and integrate it into your library. At runtime, configure the model which will pull it from the registry. The container will be small but the end user will still need to pull the image. There's no functional difference between either methods. Either way, the end user will need to pull both ollama and model. I suggest option 2 because the container can be reused for other models, i.e. if the end user wants to use something other than llama2.
GiteaMirror commented 2026-04-12 09:38:46 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Aug 3, 2023):

Hi @osamanatouf2 @steventkrawczyk

As @mxyng mentioned there's a Dockerfile in this repo that's a great starting point: https://github.com/jmorganca/ollama/blob/main/Dockerfile

With this you can run Ollama in a container:

docker build . -t example
docker run -p 11434:11434 example

From there you'll be able to connect to it to pull a model and run it!

curl -X POST http://localhost:11434/api/pull -d '{"name": "llama2"}'
curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2", "prompt":"Why is the sky blue?"}'

Would this solve your problem? @mxyng's last comment also provides some more options to create a container with models downloaded ahead of time, although we'll be making this easier.

While I wouldn't recommend this long term, one thing you could do is add a line to the Dockerfile:

RUN ollama serve & ollama pull llama2b

This would make sure your image is ready to go with llama2b - but again, this isn't the ideal experience

<!-- gh-comment-id:1663115029 --> @jmorganca commented on GitHub (Aug 3, 2023): Hi @osamanatouf2 @steventkrawczyk As @mxyng mentioned there's a `Dockerfile` in this repo that's a great starting point: https://github.com/jmorganca/ollama/blob/main/Dockerfile With this you can run Ollama in a container: ``` docker build . -t example docker run -p 11434:11434 example ``` From there you'll be able to connect to it to pull a model and run it! ``` curl -X POST http://localhost:11434/api/pull -d '{"name": "llama2"}' curl -X POST http://localhost:11434/api/generate -d '{"model": "llama2", "prompt":"Why is the sky blue?"}' ``` Would this solve your problem? @mxyng's last comment also provides some more options to create a container with models downloaded ahead of time, although we'll be making this easier. While I wouldn't recommend this long term, one thing you could do is add a line to the Dockerfile: ``` RUN ollama serve & ollama pull llama2b ``` This would make sure your image is ready to go with `llama2b` - but again, this isn't the ideal experience
GiteaMirror commented 2026-04-12 09:38:47 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Aug 3, 2023):

To add: generating in Docker may not always be the best option because of performance.

For example on Mac, the virtualization framework (where Docker runs) unfortunately does not support GPU acceleration. How are you looking to run Ollama so that it works with prompttools on all platforms?

<!-- gh-comment-id:1663121257 --> @jmorganca commented on GitHub (Aug 3, 2023): To add: generating in Docker may not always be the best option because of performance. For example on Mac, the virtualization framework (where Docker runs) unfortunately does not support GPU acceleration. How are you looking to run Ollama so that it works with `prompttools` on all platforms?
GiteaMirror commented 2026-04-12 09:38:47 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Aug 9, 2023):

Checking in here folks! @osamanatouf2 @steventkrawczyk how is the integration with Ollama coming along? Were you able to get up and running? Happy to help :-)

<!-- gh-comment-id:1670643665 --> @jmorganca commented on GitHub (Aug 9, 2023): Checking in here folks! @osamanatouf2 @steventkrawczyk how is the integration with Ollama coming along? Were you able to get up and running? Happy to help :-)
GiteaMirror commented 2026-04-12 09:38:47 -05:00
Author
Owner
Copy Link

@osamanatouf2 commented on GitHub (Aug 9, 2023):

@jmorganca I got ollama working in docker and it is working fine. Just need some extra work to get it ready to use in prompttools repo.

<!-- gh-comment-id:1670755521 --> @osamanatouf2 commented on GitHub (Aug 9, 2023): @jmorganca I got ollama working in docker and it is working fine. Just need some extra work to get it ready to use in prompttools repo.
GiteaMirror commented 2026-04-12 09:38:48 -05:00
Author
Owner
Copy Link

@priamai commented on GitHub (Aug 9, 2023):

I was just fiddling with this, we should also specify which volumes needs to be mounted so that once you download the model and if the container is restarted you don't have to download the models again.

<!-- gh-comment-id:1672262384 --> @priamai commented on GitHub (Aug 9, 2023): I was just fiddling with this, we should also specify which volumes needs to be mounted so that once you download the model and if the container is restarted you don't have to download the models again.
GiteaMirror commented 2026-04-12 09:38:48 -05:00
Author
Owner
Copy Link

@priamai commented on GitHub (Aug 9, 2023):

Something like this?

version: "3.3"
services:
  ollama:
    build:
      context: .
    env_file: .env
    ports:
      - "${HOST_API_PORT:-11434}:${CONTAINER_API_PORT:-11434}"
    stdin_open: true
    tty: true
    volumes:
      - ./models:/app/models

We just need to know where the folder "/app/models" is actually located.

<!-- gh-comment-id:1672267022 --> @priamai commented on GitHub (Aug 9, 2023): Something like this? ``` version: "3.3" services: ollama: build: context: . env_file: .env ports: - "${HOST_API_PORT:-11434}:${CONTAINER_API_PORT:-11434}" stdin_open: true tty: true volumes: - ./models:/app/models ``` We just need to know where the folder "/app/models" is actually located.
GiteaMirror commented 2026-04-12 09:38:48 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Sep 7, 2023):

There's now a Dockerfile that's kept up to date: https://github.com/jmorganca/ollama/blob/main/Dockerfile

For volumes, you can mount -v ./ollama:/home/ollama/.ollama to keep persistence between restarts

Will close this issue for now but feel free to post more or drop in the discord to talk more Docker x Ollama: https://discord.gg/ollama – some of the maintainers are ex-Docker engineers and so we love to chat about this stuff 😊

<!-- gh-comment-id:1710159922 --> @jmorganca commented on GitHub (Sep 7, 2023): There's now a `Dockerfile` that's kept up to date: https://github.com/jmorganca/ollama/blob/main/Dockerfile For volumes, you can mount `-v ./ollama:/home/ollama/.ollama` to keep persistence between restarts Will close this issue for now but feel free to post more or drop in the discord to talk more Docker x Ollama: https://discord.gg/ollama – some of the maintainers are ex-Docker engineers and so we love to chat about this stuff 😊
GiteaMirror commented 2026-04-12 09:38:48 -05:00
Author
Owner
Copy Link

@airtonix commented on GitHub (Dec 12, 2023):

possibly useful comment in other related ticket: https://github.com/jmorganca/ollama/issues/546#issuecomment-1722759440

<!-- gh-comment-id:1852863581 --> @airtonix commented on GitHub (Dec 12, 2023): possibly useful comment in other related ticket: https://github.com/jmorganca/ollama/issues/546#issuecomment-1722759440
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#109
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?