github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #387] Client can't connect to server #175

New Issue
Closed
opened 2026-04-12 09:42:18 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-12 09:42:18 -05:00
Owner
Copy Link

Originally created by @freeqaz on GitHub (Aug 20, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/387

Following the readme on my Arch linux setup yields the following error:

$ ./ollama run llama2
Error: could not connect to ollama server, run 'ollama serve' to start it

Steps to reproduce:

git clone git@github.com:jmorganca/ollama.git
cd ollama
git build .
./ollama serve &
./ollama run llama2

The output from the serve command is the following:

$ ./ollama serve
Couldn't find '/home/<USER>/.ollama/id_ed25519'. Generating new private key.
Your new public key is: 

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMQh86qSVLsOKQASDF123/FpS123/ASDF123ADg0uHka

[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /                         --> github.com/jmorganca/ollama/server.Serve.func1 (4 handlers)
[GIN-debug] HEAD   /                         --> github.com/jmorganca/ollama/server.Serve.func2 (4 handlers)
[GIN-debug] POST   /api/pull                 --> github.com/jmorganca/ollama/server.PullModelHandler (4 handlers)
[GIN-debug] POST   /api/generate             --> github.com/jmorganca/ollama/server.GenerateHandler (4 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/jmorganca/ollama/server.EmbeddingHandler (4 handlers)
[GIN-debug] POST   /api/create               --> github.com/jmorganca/ollama/server.CreateModelHandler (4 handlers)
[GIN-debug] POST   /api/push                 --> github.com/jmorganca/ollama/server.PushModelHandler (4 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/jmorganca/ollama/server.CopyModelHandler (4 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/jmorganca/ollama/server.ListModelsHandler (4 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/jmorganca/ollama/server.DeleteModelHandler (4 handlers)
2023/08/19 17:12:15 routes.go:437: Listening on 127.0.0.1:11434
[GIN] 2023/08/19 - 17:12:51 | 400 |     389.591µs |       127.0.0.1 | POST     "/api/generate"

Searching for the error in code doesn't show me which line in the Golang is the issue. I have tried passing in --verbose and any other flags while reading the source code as well as trying to set OLLAMA_HOST="localhost:11434" since maybe I figure it listening on 127.0.0.1:11434 is the issue. Alas, no success.

I also checked my firewall and I don't think that's the issue because curl works.

$ curl -X POST http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt":"Why is the sky blue?"
}'
{"error":"stat /home/<USER>/.ollama/models/manifests/registry.ollama.ai/library/llama2/latest: no such file or directory"

I wish that I could make this report more actionable for y'all. I just know that I'm not the only user that's going to get hit by this and want to make an issue to track this. Thanks for the help -- hopefully we can find a solution that's easy and be able to document for users how to debug this in the future.

Cheers!

Originally created by @freeqaz on GitHub (Aug 20, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/387 Following the readme on my Arch linux setup yields the following error: ```sh $ ./ollama run llama2 Error: could not connect to ollama server, run 'ollama serve' to start it ``` Steps to reproduce: ```sh git clone git@github.com:jmorganca/ollama.git cd ollama git build . ./ollama serve & ./ollama run llama2 ``` The output from the serve command is the following: ```sh $ ./ollama serve Couldn't find '/home/<USER>/.ollama/id_ed25519'. Generating new private key. Your new public key is: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMQh86qSVLsOKQASDF123/FpS123/ASDF123ADg0uHka [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] GET / --> github.com/jmorganca/ollama/server.Serve.func1 (4 handlers) [GIN-debug] HEAD / --> github.com/jmorganca/ollama/server.Serve.func2 (4 handlers) [GIN-debug] POST /api/pull --> github.com/jmorganca/ollama/server.PullModelHandler (4 handlers) [GIN-debug] POST /api/generate --> github.com/jmorganca/ollama/server.GenerateHandler (4 handlers) [GIN-debug] POST /api/embeddings --> github.com/jmorganca/ollama/server.EmbeddingHandler (4 handlers) [GIN-debug] POST /api/create --> github.com/jmorganca/ollama/server.CreateModelHandler (4 handlers) [GIN-debug] POST /api/push --> github.com/jmorganca/ollama/server.PushModelHandler (4 handlers) [GIN-debug] POST /api/copy --> github.com/jmorganca/ollama/server.CopyModelHandler (4 handlers) [GIN-debug] GET /api/tags --> github.com/jmorganca/ollama/server.ListModelsHandler (4 handlers) [GIN-debug] DELETE /api/delete --> github.com/jmorganca/ollama/server.DeleteModelHandler (4 handlers) 2023/08/19 17:12:15 routes.go:437: Listening on 127.0.0.1:11434 [GIN] 2023/08/19 - 17:12:51 | 400 | 389.591µs | 127.0.0.1 | POST "/api/generate" ``` Searching for the error in code doesn't show me which line in the Golang is the issue. I have tried passing in `--verbose` and any other flags while reading the source code as well as trying to set `OLLAMA_HOST="localhost:11434"` since maybe I figure it listening on `127.0.0.1:11434` is the issue. Alas, no success. I also checked my firewall and I don't think that's the issue because `curl` works. ```sh $ curl -X POST http://localhost:11434/api/generate -d '{ "model": "llama2", "prompt":"Why is the sky blue?" }' {"error":"stat /home/<USER>/.ollama/models/manifests/registry.ollama.ai/library/llama2/latest: no such file or directory" ``` I wish that I could make this report more actionable for y'all. I just know that I'm not the only user that's going to get hit by this and want to make an issue to track this. Thanks for the help -- hopefully we can find a solution that's easy and be able to document for users how to debug this in the future. Cheers!
GiteaMirror commented 2026-04-12 09:42:20 -05:00
Author
Owner
Copy Link

@iliyanvelchev commented on GitHub (Aug 21, 2023):

I am experiencing the same issue.
However, the REST API seems to be working fine, so I pulled a model using it

curl -X POST http://localhost:11434/api/pull -d '{
  "name": "llama2:7b"
}'

I am also using this Chrome extension as a UI until there is a fix.

<!-- gh-comment-id:1685993543 --> @iliyanvelchev commented on GitHub (Aug 21, 2023): I am experiencing the same issue. However, the REST API seems to be working fine, so I pulled a model using it ``` curl -X POST http://localhost:11434/api/pull -d '{ "name": "llama2:7b" }' ``` I am also using [this](https://chrome.google.com/webstore/detail/ollama-ui/cmgdpmlhgjhoadnonobjeekmfcehffco) Chrome extension as a UI until there is a fix.
GiteaMirror commented 2026-04-12 09:42:21 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Aug 21, 2023):

So you have ollama serve running on one terminal, and then when running a model in a different terminal, it says it can't reach the server?

Did the .ollama directory get created?

<!-- gh-comment-id:1687019982 --> @technovangelist commented on GitHub (Aug 21, 2023): So you have ollama serve running on one terminal, and then when running a model in a different terminal, it says it can't reach the server? Did the .ollama directory get created?
GiteaMirror commented 2026-04-12 09:42:21 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Aug 22, 2023):

We haven't heard back from you. Was your issue resolved? Let us know if you need help. Or try the Discord at Discord

<!-- gh-comment-id:1688657348 --> @technovangelist commented on GitHub (Aug 22, 2023): We haven't heard back from you. Was your issue resolved? Let us know if you need help. Or try the Discord at [Discord](https://discord.com/invite/ollama)
GiteaMirror commented 2026-04-12 09:42:21 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Aug 22, 2023):

I'll go ahead and close this. If its still an issue, please reopen the issue and we can continue.

<!-- gh-comment-id:1688969023 --> @technovangelist commented on GitHub (Aug 22, 2023): I'll go ahead and close this. If its still an issue, please reopen the issue and we can continue.
GiteaMirror commented 2026-04-12 09:42:22 -05:00
Author
Owner
Copy Link

@vRobM commented on GitHub (Oct 5, 2023):

I have the same issue, where it doesn't work from the command line. Just says cannot connect to server.

what made it work was setting the HOME and OLLAMA_HOST env vars before running the binary.

<!-- gh-comment-id:1749403807 --> @vRobM commented on GitHub (Oct 5, 2023): I have the same issue, where it doesn't work from the command line. Just says cannot connect to server. what made it work was setting the HOME and OLLAMA_HOST env vars before running the binary.
GiteaMirror commented 2026-04-12 09:42:23 -05:00
Author
Owner
Copy Link

@wifiuk commented on GitHub (Oct 6, 2023):

I have the same issue, where it doesn't work from the command line. Just says cannot connect to server.

what made it work was setting the HOME and OLLAMA_HOST env vars before running the binary.

can you share more information or docs on this?

<!-- gh-comment-id:1750096341 --> @wifiuk commented on GitHub (Oct 6, 2023): > I have the same issue, where it doesn't work from the command line. Just says cannot connect to server. > > what made it work was setting the HOME and OLLAMA_HOST env vars before running the binary. can you share more information or docs on this?
GiteaMirror commented 2026-04-12 09:42:23 -05:00
Author
Owner
Copy Link

@AnteFil commented on GitHub (Oct 30, 2023):

Faced the same problem. It throws an error at startup. Error: could not connect to ollama server, run 'ollama serve' to start it

<!-- gh-comment-id:1784679493 --> @AnteFil commented on GitHub (Oct 30, 2023): Faced the same problem. It throws an error at startup. Error: could not connect to ollama server, run 'ollama serve' to start it
GiteaMirror commented 2026-04-12 09:42:24 -05:00
Author
Owner
Copy Link

@seccodingguy commented on GitHub (Jan 8, 2024):

I, too, faced the same issue. My issue is centered around creating a Docker image that uses Ollama to interact with a separately created vector db created in Chroma. I started the serve and executed the pull in a shell script. The serve was started in a background process as follows:

nohup ollama serve > /dev/null 2>&1 &

I then executed the pull after starting the serve as follows:

ollama pull llama2-uncensored

I was then receiving the Error: Unable to connect to host message. After trying a few tricks, I was able to resolve the error by placing a separate command to execute between the ollama serve and the ollama pull commands. Here is how I solved the error message in the context of my flow:

...
nohup ollama serve > /dev/null 2>&1 &
ps -ef | grep "ollama" | grep -v grep
ollama pull llama2-uncensored
...

The Dockerfile contents are as follows:

FROM python:3.11-slim-bookworm

RUN apt-get update

Install required libraries

RUN apt-get install -y --no-install-recommends ffmpeg libsm6 libxext6 gcc make cmake cifs-utils nano curl wget

#Packages required by Python modules
RUN apt-get install -y build-essential libxml2-dev libxslt1-dev zlib1g-dev g++ golang
RUN apt-get install -y pandoc poppler-utils libleptonica-dev tesseract-ocr libtesseract-dev python3-pil tesseract-ocr-eng tesseract-ocr-script-latn

A hack to get around the error when building tesseract-ocr

COPY dependencies/longintrepr.h /usr/local/include/python3.11/longintrepr.h

Install required Python modules

RUN pip install --upgrade pip
RUN pip install --upgrade wheel
RUN pip install -U scikit-learn
RUN pip install torch==2.1.2+cpu -f https://download.pytorch.org/whl/torch_stable.html
RUN pip install torchvision==0.16.2+cpu -f https://download.pytorch.org/whl/torch_stable.html
RUN pip install transformers==4.36.2 --only-binary=:all:
RUN pip install langchain==0.0.353 --only-binary=:all:
RUN pip install -U sentence-transformers
RUN pip install huggingface_hub
RUN pip install PyPDF2
RUN pip install uvicorn
RUN pip install chromadb==0.4.22
RUN pip install qdrant-client
RUN pip install fastapi
RUN pip install gpt4all==1.0.8
RUN pip install llama-cpp-python
RUN pip install urllib3
RUN pip install PyMuPDF
RUN pip install python-dotenv
RUN pip install extract-msg
RUN pip install tabulate
RUN pip install pandoc
RUN pip install pypandoc
RUN pip install tqdm

Required for getting page count for PDF files that are image based

RUN pip install pdf2image
RUN pip install unstructured
RUN pip install unstructured[pdf]
RUN pip install unstructured[epub]
RUN pip install tesseract

RUN pip install Cython
RUN pip install pytesseract

RUN curl https://ollama.ai/install.sh | sh
RUN curl -L https://ollama.ai/download/ollama-linux-amd64 -o /usr/bin/ollama
RUN chmod +x /usr/bin/ollama

Make the directories for storing files and data

RUN mkdir /home/app
RUN mkdir /home/app/data

Copy the required scripts

COPY python/private_chat.py /home/app/private_chat.py
COPY scripts/private_chat.sh /home/app/private_chat.sh

RUN chmod +x /home/app/private_chat.sh

CMD ["./home/app/private_chat.sh"]

And the private_chat shell script is as follows:

#!/bin/bash
OLLAMA_HOST=0.0.0.0
export OLLAMA_HOST
PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH
export PATH
LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
export LD_LIBRARY_PATH
NVIDIA_DRIVER_CAPABILITIES=compute,utility
export NVIDIA_DRIVER_CAPABILITIES

echo "Mounting the samambashare drive locations.\n"
mount -t cifs -o username='someuser',password='somepassword' //1.1.1.1/sambashare/Literature /mnt/docs
mount -t cifs -o username='someuser',password='somepassword' //1.1.1.1/sambashare/qdrant/ollama /mnt/ollama

#ollama serve &>/dev/null &
nohup ollama serve > /dev/null 2>&1 &
ps -ef | grep "ollama" | grep -v grep
ollama pull llama2-uncensored

python3 /home/app/private_chat.py

<!-- gh-comment-id:1881551261 --> @seccodingguy commented on GitHub (Jan 8, 2024): I, too, faced the same issue. My issue is centered around creating a Docker image that uses Ollama to interact with a separately created vector db created in Chroma. I started the serve and executed the pull in a shell script. The serve was started in a background process as follows: nohup ollama serve > /dev/null 2>&1 & I then executed the pull after starting the serve as follows: ollama pull llama2-uncensored I was then receiving the Error: Unable to connect to host message. After trying a few tricks, I was able to resolve the error by placing a separate command to execute between the ollama serve and the ollama pull commands. Here is how I solved the error message in the context of my flow: ... nohup ollama serve > /dev/null 2>&1 & ps -ef | grep "ollama" | grep -v grep ollama pull llama2-uncensored ... The Dockerfile contents are as follows: FROM python:3.11-slim-bookworm RUN apt-get update # Install required libraries RUN apt-get install -y --no-install-recommends ffmpeg libsm6 libxext6 gcc make cmake cifs-utils nano curl wget #Packages required by Python modules RUN apt-get install -y build-essential libxml2-dev libxslt1-dev zlib1g-dev g++ golang RUN apt-get install -y pandoc poppler-utils libleptonica-dev tesseract-ocr libtesseract-dev python3-pil tesseract-ocr-eng tesseract-ocr-script-latn # A hack to get around the error when building tesseract-ocr COPY dependencies/longintrepr.h /usr/local/include/python3.11/longintrepr.h # Install required Python modules RUN pip install --upgrade pip RUN pip install --upgrade wheel RUN pip install -U scikit-learn RUN pip install torch==2.1.2+cpu -f https://download.pytorch.org/whl/torch_stable.html RUN pip install torchvision==0.16.2+cpu -f https://download.pytorch.org/whl/torch_stable.html RUN pip install transformers==4.36.2 --only-binary=:all: RUN pip install langchain==0.0.353 --only-binary=:all: RUN pip install -U sentence-transformers RUN pip install huggingface_hub RUN pip install PyPDF2 RUN pip install uvicorn RUN pip install chromadb==0.4.22 RUN pip install qdrant-client RUN pip install fastapi RUN pip install gpt4all==1.0.8 RUN pip install llama-cpp-python RUN pip install urllib3 RUN pip install PyMuPDF RUN pip install python-dotenv RUN pip install extract-msg RUN pip install tabulate RUN pip install pandoc RUN pip install pypandoc RUN pip install tqdm # Required for getting page count for PDF files that are image based RUN pip install pdf2image RUN pip install unstructured RUN pip install unstructured[pdf] RUN pip install unstructured[epub] RUN pip install tesseract RUN pip install Cython RUN pip install pytesseract RUN curl https://ollama.ai/install.sh | sh RUN curl -L https://ollama.ai/download/ollama-linux-amd64 -o /usr/bin/ollama RUN chmod +x /usr/bin/ollama # Make the directories for storing files and data RUN mkdir /home/app RUN mkdir /home/app/data # Copy the required scripts COPY python/private_chat.py /home/app/private_chat.py COPY scripts/private_chat.sh /home/app/private_chat.sh RUN chmod +x /home/app/private_chat.sh CMD ["./home/app/private_chat.sh"] And the private_chat shell script is as follows: #!/bin/bash OLLAMA_HOST=0.0.0.0 export OLLAMA_HOST PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$PATH export PATH LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 export LD_LIBRARY_PATH NVIDIA_DRIVER_CAPABILITIES=compute,utility export NVIDIA_DRIVER_CAPABILITIES echo "Mounting the samambashare drive locations.\n" mount -t cifs -o username='someuser',password='somepassword' //1.1.1.1/sambashare/Literature /mnt/docs mount -t cifs -o username='someuser',password='somepassword' //1.1.1.1/sambashare/qdrant/ollama /mnt/ollama #ollama serve &>/dev/null & nohup ollama serve > /dev/null 2>&1 & ps -ef | grep "ollama" | grep -v grep ollama pull llama2-uncensored python3 /home/app/private_chat.py
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#175
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?