github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #1845] Ollama from remote #1051

New Issue
Closed
opened 2026-04-12 10:47:18 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 10:47:18 -05:00
Owner
Copy Link

Originally created by @HAL9KKK on GitHub (Jan 7, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/1845

Ollama is using always localhost.

I have 2 colab istances:
Colab1 (server)

# Set LD_LIBRARY_PATH so the system NVIDIA library 
import os
import asyncio
os.environ.update({'LD_LIBRARY_PATH': '/usr/lib64-nvidia'})

async def run_process(cmd):
  print('>>> starting', *cmd)
  p = await asyncio.subprocess.create_subprocess_exec(
      *cmd,
      stdout=asyncio.subprocess.PIPE,
      stderr=asyncio.subprocess.PIPE,
  )

  async def pipe(lines):
    async for line in lines:
      print(line.strip().decode('utf-8'))

  await asyncio.gather(
      pipe(p.stdout),
      pipe(p.stderr),
  )

await asyncio.gather(
    run_process(['ollama', 'serve']),
    run_process(['ngrok', 'http', '--log', 'stderr', '11434']),
)

>>> starting ollama serve
>>> starting ngrok http --log stderr 11434
2024/01/07 18:10:03 routes.go:929: Listening on 127.0.0.1:11434 (version 0.1.18)
t=2024-01-07T18:10:03+0000 lvl=info msg="started tunnel" obj=tunnels name=command_line addr=http://localhost:11434/ url=https://7b8c-34-83-27-150.ngrok-free.app/

Colab2 (client)

import os
os.environ["OLLAMA_HOST"]="https://7b8c-34-83-27-150.ngrok-free.app"

import subprocess
pr= subprocess.Popen(['ollama', 'run', 'openhermes'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)

After running the subprocess "ollama run openhermes" the server start running the model, so the connection client server is working thanks to the OLLAMA_HOST variable

The problem is when I run ollama from langchain

from langchain.llms import Ollama
ollama_llm = Ollama(model="openhermes")
ollama_llm.generate(["hello"])

ConnectionError: HTTPConnectionPool(host='localhost', port=11434)

Why OLLAMA_HOST is not working with langchain?

Originally created by @HAL9KKK on GitHub (Jan 7, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/1845 Ollama is using always localhost. I have 2 colab istances: **Colab1 (server)** ``` # Set LD_LIBRARY_PATH so the system NVIDIA library import os import asyncio os.environ.update({'LD_LIBRARY_PATH': '/usr/lib64-nvidia'}) async def run_process(cmd): print('>>> starting', *cmd) p = await asyncio.subprocess.create_subprocess_exec( *cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, ) async def pipe(lines): async for line in lines: print(line.strip().decode('utf-8')) await asyncio.gather( pipe(p.stdout), pipe(p.stderr), ) await asyncio.gather( run_process(['ollama', 'serve']), run_process(['ngrok', 'http', '--log', 'stderr', '11434']), ) ``` ``` >>> starting ollama serve >>> starting ngrok http --log stderr 11434 2024/01/07 18:10:03 routes.go:929: Listening on 127.0.0.1:11434 (version 0.1.18) t=2024-01-07T18:10:03+0000 lvl=info msg="started tunnel" obj=tunnels name=command_line addr=http://localhost:11434/ url=https://7b8c-34-83-27-150.ngrok-free.app/ ``` **Colab2 (client)** ``` import os os.environ["OLLAMA_HOST"]="https://7b8c-34-83-27-150.ngrok-free.app" import subprocess pr= subprocess.Popen(['ollama', 'run', 'openhermes'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) ``` After running the subprocess "ollama run openhermes" the server start running the model, so the connection client server is working thanks to the OLLAMA_HOST variable The problem is when I run ollama from langchain ``` from langchain.llms import Ollama ollama_llm = Ollama(model="openhermes") ollama_llm.generate(["hello"]) ``` ConnectionError: HTTPConnectionPool(host='localhost', port=11434) Why OLLAMA_HOST is not working with langchain?
GiteaMirror commented 2026-04-12 10:47:19 -05:00
Author
Owner
Copy Link

@wrapss commented on GitHub (Jan 7, 2024):

Why OLLAMA_HOST is not working with langchain?

try with base_url as shown in this tutorial

<!-- gh-comment-id:1880183695 --> @wrapss commented on GitHub (Jan 7, 2024): > Why OLLAMA_HOST is not working with langchain? try with base_url as shown in this [tutorial ](https://github.com/jmorganca/ollama/blob/main/docs/tutorials/langchainpy.md)
GiteaMirror commented 2026-04-12 10:47:20 -05:00
Author
Owner
Copy Link

@prusnak commented on GitHub (Jan 7, 2024):

Yes, you should use the following code because langchain does not use OLLAMA_HOST variable:

ollama_llm = Ollama(base_url="https://your_url:11434", model="llama2")
<!-- gh-comment-id:1880186910 --> @prusnak commented on GitHub (Jan 7, 2024): Yes, you should use the following code because langchain does not use OLLAMA_HOST variable: ``` python ollama_llm = Ollama(base_url="https://your_url:11434", model="llama2") ```
GiteaMirror commented 2026-04-12 10:47:21 -05:00
Author
Owner
Copy Link

@HAL9KKK commented on GitHub (Jan 7, 2024):

Thank you so much for your prompt reply @wrapss and @prusnak tomorrow I will try but I am quite sure you are right!!!

<!-- gh-comment-id:1880190415 --> @HAL9KKK commented on GitHub (Jan 7, 2024): Thank you so much for your prompt reply @wrapss and @prusnak tomorrow I will try but I am quite sure you are right!!!
GiteaMirror commented 2026-04-12 10:47:22 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Jan 8, 2024):

As others have mentioned, ollama serves on localhost by default. If you want to change this, set OLLAMA_HOST. Please see the FAQ for details

<!-- gh-comment-id:1881677938 --> @mxyng commented on GitHub (Jan 8, 2024): As others have mentioned, ollama serves on localhost by default. If you want to change this, set `OLLAMA_HOST`. Please see the [FAQ](https://github.com/jmorganca/ollama/blob/main/docs/faq.md#how-can-i-expose-ollama-on-my-network) for details
GiteaMirror commented 2026-04-12 10:47:23 -05:00
Author
Owner
Copy Link

@kzos commented on GitHub (May 16, 2024):

@HAL9KKK By any chance , have you hosted this code somewhere or mind sharing it to me, I am trying to setup similar thing. if you could share the entire setup , I shall start working on actual study work.

Host/serve llama on server
and access it in multiple clients.

thanks in advance.

<!-- gh-comment-id:2115223076 --> @kzos commented on GitHub (May 16, 2024): @HAL9KKK By any chance , have you hosted this code somewhere or mind sharing it to me, I am trying to setup similar thing. if you could share the entire setup , I shall start working on actual study work. Host/serve llama on server and access it in multiple clients. thanks in advance.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1051
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?