github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-21 13:11:39 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #888] How to deploy on k8s #429

New Issue
Closed
opened 2026-04-12 10:05:09 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-12 10:05:09 -05:00
Owner
Copy Link

Originally created by @xinmans on GitHub (Oct 24, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/888

Originally assigned to: @mxyng on GitHub.

Anyone can share some deploymen.yaml ?

Originally created by @xinmans on GitHub (Oct 24, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/888 Originally assigned to: @mxyng on GitHub. Anyone can share some deploymen.yaml ?
GiteaMirror added the documentationfeature request labels 2026-04-12 10:05:09 -05:00
GiteaMirror commented 2026-04-12 10:05:11 -05:00
Author
Owner
Copy Link

@Syulin7 commented on GitHub (Oct 26, 2023):

@xinmans This is a example I used for testing.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: llm
  name: ollama
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: llm
  template:
    metadata:
      labels:
        app.kubernetes.io/name: llm
    spec:
      containers:
        - image: ollama/ollama
          imagePullPolicy: IfNotPresent
          name: llm
          ports:
            - containerPort: 11434
              protocol: TCP
          resources:
            limits:
              nvidia.com/gpu: "1"
            requests:
              nvidia.com/gpu: "1"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: llm
  name: ollama
spec:
  ports:
    - name: "11434"
      port: 11434
      targetPort: 11434
  selector:
    app.kubernetes.io/name: llm
  type: ClusterIP
<!-- gh-comment-id:1780293721 --> @Syulin7 commented on GitHub (Oct 26, 2023): @xinmans This is a example I used for testing. ``` apiVersion: apps/v1 kind: Deployment metadata: labels: app.kubernetes.io/name: llm name: ollama spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: llm template: metadata: labels: app.kubernetes.io/name: llm spec: containers: - image: ollama/ollama imagePullPolicy: IfNotPresent name: llm ports: - containerPort: 11434 protocol: TCP resources: limits: nvidia.com/gpu: "1" requests: nvidia.com/gpu: "1" --- apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/name: llm name: ollama spec: ports: - name: "11434" port: 11434 targetPort: 11434 selector: app.kubernetes.io/name: llm type: ClusterIP ```
GiteaMirror commented 2026-04-12 10:05:11 -05:00
Author
Owner
Copy Link

@xinmans commented on GitHub (Oct 28, 2023):

@xinmans This is a example I used for testing.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/name: llm
  name: ollama
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: llm
  template:
    metadata:
      labels:
        app.kubernetes.io/name: llm
    spec:
      containers:
        - image: ollama/ollama
          imagePullPolicy: IfNotPresent
          name: llm
          ports:
            - containerPort: 11434
              protocol: TCP
          resources:
            limits:
              nvidia.com/gpu: "1"
            requests:
              nvidia.com/gpu: "1"
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: llm
  name: ollama
spec:
  ports:
    - name: "11434"
      port: 11434
      targetPort: 11434
  selector:
    app.kubernetes.io/name: llm
  type: ClusterIP

thanks very much, Warning: GPU support may not enabled, check you have installed install GPU drivers: nvidia-smi command failed

Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
2023-10-28T10:40:10.783453287+08:00 Your new public key is:
2023-10-28T10:40:10.783455847+08:00
2023-10-28T10:40:10.783457668+08:00 ssh-ed25519 xxx/xxx
2023-10-28T10:40:10.783458898+08:00
2023-10-28T10:40:10.783623603+08:00 2023/10/28 02:40:10 images.go:828: total blobs: 0
2023-10-28T10:40:10.783656858+08:00 2023/10/28 02:40:10 images.go:835: total unused blobs removed: 0
2023-10-28T10:40:10.783790279+08:00 2023/10/28 02:40:10 routes.go:662: Listening on [::]:11434 (version 0.1.6)
2023-10-28T10:40:10.784148934+08:00 2023/10/28 02:40:10 routes.go:682: Warning: GPU support may not enabled, check you have installed install GPU drivers: nvidia-smi command failed

<!-- gh-comment-id:1783672581 --> @xinmans commented on GitHub (Oct 28, 2023): > @xinmans This is a example I used for testing. > > ``` > apiVersion: apps/v1 > kind: Deployment > metadata: > labels: > app.kubernetes.io/name: llm > name: ollama > spec: > replicas: 1 > selector: > matchLabels: > app.kubernetes.io/name: llm > template: > metadata: > labels: > app.kubernetes.io/name: llm > spec: > containers: > - image: ollama/ollama > imagePullPolicy: IfNotPresent > name: llm > ports: > - containerPort: 11434 > protocol: TCP > resources: > limits: > nvidia.com/gpu: "1" > requests: > nvidia.com/gpu: "1" > --- > apiVersion: v1 > kind: Service > metadata: > labels: > app.kubernetes.io/name: llm > name: ollama > spec: > ports: > - name: "11434" > port: 11434 > targetPort: 11434 > selector: > app.kubernetes.io/name: llm > type: ClusterIP > ``` thanks very much, Warning: GPU support may not enabled, check you have installed install GPU drivers: nvidia-smi command failed Couldn't find '/root/.ollama/id_ed25519'. Generating new private key. 2023-10-28T10:40:10.783453287+08:00 Your new public key is: 2023-10-28T10:40:10.783455847+08:00 2023-10-28T10:40:10.783457668+08:00 ssh-ed25519 xxx/xxx 2023-10-28T10:40:10.783458898+08:00 2023-10-28T10:40:10.783623603+08:00 2023/10/28 02:40:10 images.go:828: total blobs: 0 2023-10-28T10:40:10.783656858+08:00 2023/10/28 02:40:10 images.go:835: total unused blobs removed: 0 2023-10-28T10:40:10.783790279+08:00 2023/10/28 02:40:10 routes.go:662: Listening on [::]:11434 (version 0.1.6) 2023-10-28T10:40:10.784148934+08:00 2023/10/28 02:40:10 routes.go:682: Warning: GPU support may not enabled, check you have installed install GPU drivers: nvidia-smi command failed
GiteaMirror commented 2026-04-12 10:05:12 -05:00
Author
Owner
Copy Link

@Syulin7 commented on GitHub (Nov 3, 2023):

@xinmans Following the issue https://github.com/jmorganca/ollama/issues/797

<!-- gh-comment-id:1791928544 --> @Syulin7 commented on GitHub (Nov 3, 2023): @xinmans Following the issue https://github.com/jmorganca/ollama/issues/797
GiteaMirror commented 2026-04-12 10:05:12 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Nov 3, 2023):

There's a basic Kubernetes example in #959 which also describes how to deploy with GPU

<!-- gh-comment-id:1792651098 --> @mxyng commented on GitHub (Nov 3, 2023): There's a basic Kubernetes example in #959 which also describes how to deploy with GPU
GiteaMirror commented 2026-04-12 10:05:12 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Nov 12, 2023):

It looks like your pod does not have DNS configured? Adding registry.ollama.ai only allows the registry to be resolved but the download is actually somewhere else (Cloudflare)

<!-- gh-comment-id:1806981165 --> @mxyng commented on GitHub (Nov 12, 2023): It looks like your pod does not have DNS configured? Adding `registry.ollama.ai` only allows the registry to be resolved but the download is actually somewhere else (Cloudflare)
GiteaMirror commented 2026-04-12 10:05:13 -05:00
Author
Owner
Copy Link

@xinmans commented on GitHub (Nov 12, 2023):

It looks like your pod does not have DNS configured? Adding registry.ollama.ai only allows the registry to be resolved but the download is actually somewhere else (Cloudflare)

network issue ,have fix. thanks.

<!-- gh-comment-id:1806981370 --> @xinmans commented on GitHub (Nov 12, 2023): > It looks like your pod does not have DNS configured? Adding `registry.ollama.ai` only allows the registry to be resolved but the download is actually somewhere else (Cloudflare) network issue ,have fix. thanks.
GiteaMirror commented 2026-04-12 10:05:14 -05:00
Author
Owner
Copy Link

@technovangelist commented on GitHub (Dec 4, 2023):

It looks like this issue is resolved. I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.

<!-- gh-comment-id:1839637603 --> @technovangelist commented on GitHub (Dec 4, 2023): It looks like this issue is resolved. I will go ahead and close it now. If you think there is anything we left out, reopen and we can address. Thanks for being part of this great community.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label documentation feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#429
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?