github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #1063] Failed to verify certificate: x509: certificate signed by unknown authority #47034

New Issue
Closed
opened 2026-04-28 02:48:47 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-28 02:48:47 -05:00
Owner
Copy Link

Originally created by @marcellodesales on GitHub (Nov 9, 2023).
Original GitHub issue: https://github.com/ollama/ollama/issues/1063

I started using Ollama in the last 12hrs and I'm loving it... Why? Because I come from the CloudNative space, I've been working with Docker/Kubernetes Engineering for a while... I love the concept from Ollama and I can't wait until the Modelfile works well :)

🚨 Problem

  • While pulling models, we get failures on the pull of models
    • form the CLI
    • to the API endpoints

🐳 Docker client

So, some clues on this:

  • According to https://github.com/kubernetes/kubernetes/issues/43924#issuecomment-290905127, this error occurs when a docker client tries to pull docker images from an insecure Docker Registry...
  • Considering Ollama uses a docker registry to implement the model repository, I would say it's possible ollama's backend is actually a Docker Registry whose TLS certs were self-signed...
    • Meanwhile, ollama's CLI client runs a client that connects to the docker daemon to pull the Models...
  • I don't get the same error running from my local machine, but I get it when running in a Kubernetes cluster...
    • My local machine has all the bypass and lower security configuration while the Kubernetes cluster doesn't

At least I know registry.ollama.ai is a docker registry ;), which indicates the suspicions above...

$ docker pull registry.ollama.ai/library/llama2
Using default tag: latest
latest: Pulling from library/llama2
unsupported media type application/vnd.ollama.image.model

👽 Using the API

  • This is similar to the bug reported at https://github.com/jmorganca/ollama/issues/823, which I think it was prematurely closed...
curl -i http://localhost:11434/api/pull -d '{"name": "llama2"}'
HTTP/1.1 200 OK
Content-Type: application/x-ndjson
Date: Thu, 0[9](jobs/1414268#step:5:10) Nov 2023 20:22:16 GMT
Transfer-Encoding: chunked

{"status":"pulling manifest"}
{"error":"pull model manifest: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority"}
  • Also the Json objects returned by the server is returning those extra/unescaped "
$ echo '{"error":"pull model manifest: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority"}' | jq
parse error: Invalid numeric literal at line 1, column 42

🔊 Server Logs

Print service container logs: c470f383b37b44b6b05555572e49de37_dockerhubdockerartifactorycomollamaollama_c9a05e
/usr/local/bin/docker logs --details bcc88cf81ef4ff93[2](https://git.companycom/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:2)[3](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:3)50a2522c4596f0c7f01ee52432558f98b350be6319695d
 Couldn't find '/root/.ollama/id_ed25519'. Generating new private key.
 2023/11/09 22:16:[4](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:4)6 images.go:824: total blobs: 0
 2023/11/09 22:16:46 images.go:831: total unused blobs removed: 0
 2023/11/09 22:16:46 routes.go:680: Listening on [::]:11434 (version 0.1.8)
 Your new public key is: 
 
 ssh-ed2[5](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:5)519 AAAAC3NzaC1lZDI1NTE5AAAAID4[6](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:6)z8kD0XvZfSsZnSogyAdTu/06A0e0YvpxrRlSfIXA
 
 2023/11/09 22:16:46 routes.go:[7](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:7)00: Warning: GPU support may not be enabled, check you have installed GPU drivers: nvidia-smi command failed
 2023/11/09 22:16:47 images.go:1172: couldn't start upload: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority
 2023/11/09 22:16:4[8](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:8) images.go:1172: couldn't start upload: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x50[9](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:9): certificate signed by unknown authority
 [GIN] 2023/[11](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:11)/09 - 22:16:47 | 200 |   93.842[15](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:15)9ms |      172.18.0.1 | POST     "/api/pull"
 [GIN] 2023/11/09 - 22:16:48 | 200 |    51.96959ms |      172.18.0.1 | POST     "/api/pull"
 [GIN] 2023/11/09 - 22:[16](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:16):48 | 404 |     160.559µs |      172.18.0.1 | POST     "/api/generate"

❓ Approach to the problem

  • I think the Ollama CLI and server must have settings to by-pass this security setting on the docker client against an "insecure" docker registry
    • I know the intent is good to host registry.ollama.ai, but what if users deploy their own ollama registries on their enterprises?
  • If we are to trust this registry, or any other deployed by anyone, I would think it's safe to say that the same toggles implemented by the Docker and Kuberentes communities should be added to ollama...
  • Docker daemon can be configured to support accept insecure registries https://docs.docker.com/engine/reference/commandline/dockerd/#insecure-registries
    • Users provide --insecure-registry registry.company.myollama.genai param (if I were to deploy my own)

🤔 Possible solution

// InsecureSkipVerify controls whether a client verifies the server's
// certificate chain and host name. If InsecureSkipVerify is true, crypto/tls
// accepts any certificate presented by the server and any host name in that
// certificate. In this mode, TLS is susceptible to machine-in-the-middle
// attacks unless custom verification is used. This should be used only for
// testing or in combination with VerifyConnection or VerifyPeerCertificate.

       client := http.Client{
		Transport: &http.Transport{
			Proxy: http.ProxyURL(proxyURL),
		},
	}

+	// Check if the registry TLS should be skipped
+	if skip, ok := os.LookupEnv("MODELS_REGISTRY_SKIP_TLS_CHECK"); ok && (skip == "true" || skip == "0") {
+		// If so, add an unverified TLS configuration to the HTTP client
+		client.Transport = &http.Transport{
+			TLSClientConfig: &tls.Config{
+				InsecureSkipVerify: true,
+			},
+			Proxy: http.ProxyURL(proxyURL),
+		}
+	}
Originally created by @marcellodesales on GitHub (Nov 9, 2023). Original GitHub issue: https://github.com/ollama/ollama/issues/1063 I started using Ollama in the last 12hrs and I'm loving it... Why? Because I come from the CloudNative space, I've been working with Docker/Kubernetes Engineering for a while... I love the concept from Ollama and I can't wait until the `Modelfile` works well :) # 🚨 Problem * While pulling models, we get failures on the pull of models * form the CLI * to the API endpoints # 🐳 Docker client So, some clues on this: * According to https://github.com/kubernetes/kubernetes/issues/43924#issuecomment-290905127, this error occurs when a docker client tries to pull docker images from an insecure Docker Registry... * Considering Ollama uses a docker registry to implement the model repository, I would say it's possible ollama's backend is actually a Docker Registry whose TLS certs were self-signed... * Meanwhile, ollama's CLI client runs a client that connects to the docker daemon to pull the Models... * I don't get the same error running from my local machine, but I get it when running in a Kubernetes cluster... * My local machine has all the bypass and lower security configuration while the Kubernetes cluster doesn't At least I know registry.ollama.ai is a docker registry ;), which indicates the suspicions above... ```console $ docker pull registry.ollama.ai/library/llama2 Using default tag: latest latest: Pulling from library/llama2 unsupported media type application/vnd.ollama.image.model ``` # 👽 Using the API * This is similar to the bug reported at https://github.com/jmorganca/ollama/issues/823, which I think it was prematurely closed... ```console curl -i http://localhost:11434/api/pull -d '{"name": "llama2"}' HTTP/1.1 200 OK Content-Type: application/x-ndjson Date: Thu, 0[9](jobs/1414268#step:5:10) Nov 2023 20:22:16 GMT Transfer-Encoding: chunked ``` ```json {"status":"pulling manifest"} {"error":"pull model manifest: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority"} ``` > * Also the Json objects returned by the server is returning those extra/unescaped `"` ```console $ echo '{"error":"pull model manifest: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority"}' | jq parse error: Invalid numeric literal at line 1, column 42 ``` # 🔊 Server Logs * According to the server logs, the error is printed at https://github.com/jmorganca/ollama/blob/main/server/images.go#L1170 * At least we should consider having the settings on `makeRequest` to take into account the certificate instructions * https://github.com/jmorganca/ollama/blob/main/server/images.go#L1208 * If we want to do something like the docker or kubernetes approach, consider adding the the insecure tls option in the client * https://github.com/jmorganca/ollama/blob/main/server/images.go#L1245-L1251 ```console Print service container logs: c470f383b37b44b6b05555572e49de37_dockerhubdockerartifactorycomollamaollama_c9a05e /usr/local/bin/docker logs --details bcc88cf81ef4ff93[2](https://git.companycom/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:2)[3](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:3)50a2522c4596f0c7f01ee52432558f98b350be6319695d Couldn't find '/root/.ollama/id_ed25519'. Generating new private key. 2023/11/09 22:16:[4](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:4)6 images.go:824: total blobs: 0 2023/11/09 22:16:46 images.go:831: total unused blobs removed: 0 2023/11/09 22:16:46 routes.go:680: Listening on [::]:11434 (version 0.1.8) Your new public key is: ssh-ed2[5](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:5)519 AAAAC3NzaC1lZDI1NTE5AAAAID4[6](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:6)z8kD0XvZfSsZnSogyAdTu/06A0e0YvpxrRlSfIXA 2023/11/09 22:16:46 routes.go:[7](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:7)00: Warning: GPU support may not be enabled, check you have installed GPU drivers: nvidia-smi command failed 2023/11/09 22:16:47 images.go:1172: couldn't start upload: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x509: certificate signed by unknown authority 2023/11/09 22:16:4[8](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:8) images.go:1172: couldn't start upload: Get "https://registry.ollama.ai/v2/library/llama2/manifests/latest": tls: failed to verify certificate: x50[9](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:9): certificate signed by unknown authority [GIN] 2023/[11](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:11)/09 - 22:16:47 | 200 | 93.842[15](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:15)9ms | 172.18.0.1 | POST "/api/pull" [GIN] 2023/11/09 - 22:16:48 | 200 | 51.96959ms | 172.18.0.1 | POST "/api/pull" [GIN] 2023/11/09 - 22:[16](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-prreviewer/actions/runs/504830/jobs/1415211#step:13:16):48 | 404 | 160.559µs | 172.18.0.1 | POST "/api/generate" ``` # ❓ Approach to the problem * I think the Ollama CLI and server must have settings to by-pass this security setting on the docker client against an "insecure" docker registry * I know the intent is good to host `registry.ollama.ai`, but what if users deploy their own ollama registries on their enterprises? * If we are to trust this registry, or any other deployed by anyone, I would think it's safe to say that the same toggles implemented by the Docker and Kuberentes communities should be added to ollama... * Docker daemon can be configured to support accept insecure registries https://docs.docker.com/engine/reference/commandline/dockerd/#insecure-registries * Users provide `--insecure-registry registry.company.myollama.genai` param (if I were to deploy my own) # 🤔 Possible solution * According to https://pkg.go.dev/github.com/docker/docker/client#section-readme, > // InsecureSkipVerify controls whether a client verifies the server's // certificate chain and host name. If InsecureSkipVerify is true, crypto/tls // accepts any certificate presented by the server and any host name in that // certificate. In this mode, TLS is susceptible to machine-in-the-middle // attacks unless custom verification is used. This should be used only for // testing or in combination with VerifyConnection or VerifyPeerCertificate. * Then, a possible patch for this problem could be as follows: * Adding the patch below to https://github.com/jmorganca/ollama/blob/main/server/images.go#L1245-L1251 * If we want to trust the configured MODELS_REGISTRY, then look for an env var `MODELS_REGISTRY_SKIP_TLS_CHECK` that specifies that... ```diff client := http.Client{ Transport: &http.Transport{ Proxy: http.ProxyURL(proxyURL), }, } + // Check if the registry TLS should be skipped + if skip, ok := os.LookupEnv("MODELS_REGISTRY_SKIP_TLS_CHECK"); ok && (skip == "true" || skip == "0") { + // If so, add an unverified TLS configuration to the HTTP client + client.Transport = &http.Transport{ + TLSClientConfig: &tls.Config{ + InsecureSkipVerify: true, + }, + Proxy: http.ProxyURL(proxyURL), + } + } ```
GiteaMirror commented 2026-04-28 02:48:52 -05:00
Author
Owner
Copy Link

@65a commented on GitHub (Nov 10, 2023):

This seems like you're just missing ca-certificates in your cluster or container images

<!-- gh-comment-id:1805083883 --> @65a commented on GitHub (Nov 10, 2023): This seems like you're just missing ca-certificates in your cluster or container images
GiteaMirror commented 2026-04-28 02:48:58 -05:00
Author
Owner
Copy Link

@marcellodesales commented on GitHub (Nov 10, 2023):

@65a that's the same docker image I've been running on my mac... That should be shipped with the image... But in any case, I would think we should support the same case as the docker client, given the models are hosted in a Docker Registry... I particularly will deploy the same infra to host the models in isolation...

<!-- gh-comment-id:1805222997 --> @marcellodesales commented on GitHub (Nov 10, 2023): @65a that's the same docker image I've been running on my mac... That should be shipped with the image... But in any case, I would think we should support the same case as the docker client, given the models are hosted in a Docker Registry... I particularly will deploy the same infra to host the models in isolation...
GiteaMirror commented 2026-04-28 02:49:01 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Nov 10, 2023):

@65a is likely correctly. The ollama library has a certificate signed by Google. The official docker container explicitly installs ca-certificates into an Ubuntu container image.

Another possibility you have a forward proxy with a self signed certificate that's not installed as a system certificate. In that case, you'll need to install the self signed certificate as well as set HTTPS_PROXY in order for ollama to reach the internet

<!-- gh-comment-id:1805955571 --> @mxyng commented on GitHub (Nov 10, 2023): @65a is likely correctly. The ollama library has a certificate signed by Google. The official docker container explicitly installs ca-certificates into an Ubuntu container image. Another possibility you have a forward proxy with a self signed certificate that's not installed as a system certificate. In that case, you'll need to install the self signed certificate as well as set HTTPS_PROXY in order for ollama to reach the internet
GiteaMirror commented 2026-04-28 02:49:02 -05:00
Author
Owner
Copy Link

@marcellodesales commented on GitHub (Nov 10, 2023):

@mxyng Just looking for it... and Yes, the certificate is in the container...

  • What I can see now is that we have an egress rule going through a firewall with cillium, which I wasn't aware of... :S

So, I could only understand the problem by disabling the certificate verification in code with the patch I suggested... that way I think we can give others the same benefit... I suggest just having the skip setting as an env var...

🔊 Logs for the ollama container cert

  • Logs for the verification of the certs described...
$ docker run --entrypoint bash -v $(pwd):$(pwd) -w $(pwd) ollama/ollama -c "openssl s_client -connect registry.ollama.ai:443 -showcerts </dev/null"</summary>


depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
verify return:1
depth=1 C = US, O = Google Trust Services LLC, CN = GTS CA 1D4
verify return:1
depth=0 CN = ollama.ai
verify return:1
DONE
CONNECTED(00000003)
---
Certificate chain
 0 s:CN = ollama.ai
   i:C = US, O = Google Trust Services LLC, CN = GTS CA 1D4
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Oct 27 11:55:06 2023 GMT; NotAfter: Jan 25 12:48:40 2024 GMT
-----BEGIN CERTIFICATE-----
MIIFgjCCBGqgAwIBAgIQc70da3L+ed4QYvXhE8fpVjANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExM
QzETMBEGA1UEAxMKR1RTIENBIDFENDAeFw0yMzEwMjcxMTU1MDZaFw0yNDAxMjUx
MjQ4NDBaMBQxEjAQBgNVBAMTCW9sbGFtYS5haTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBALApuMJBxFiSIS42O04cLo18/rnNMQAJZ43luT08MLNxyE9B
YCs8iEEucfNg8b7ORbVlAcHx6v+ujnme8bVZ++7yMOcTSzLnJkUydZ9JY/mAIGFM
E1onacM1TY0uL+DbGwoI+1b3ZL586ktHWs5KpN7EepO7tTtLqt0RGYyIiDO1xbYt
ckjeFgD4kSElo+4NqpBFdYL2tDU9Ewv4pbub9h+GQQuyWT+GrBga931FN6IUEwJX
9yuMJctBWABo1hRmn7n5NcqdycdgGXZqEGC7ycnsoCumTydqHkvSiyFUVF3wB8mI
6LRXeUI0byzP8xZTUQms7+njXiGTquv1WzwUNFsCAwEAAaOCApwwggKYMA4GA1Ud
DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0G
A1UdDgQWBBQZjVZZ6jHzir6KpT/u0HEnw+y3pTAfBgNVHSMEGDAWgBQl4hgOsleR
lCrl1F2GkIPeU7O4kjB4BggrBgEFBQcBAQRsMGowNQYIKwYBBQUHMAGGKWh0dHA6
Ly9vY3NwLnBraS5nb29nL3MvZ3RzMWQ0L1h5aElWN2FZV0lzMDEGCCsGAQUFBzAC
hiVodHRwOi8vcGtpLmdvb2cvcmVwby9jZXJ0cy9ndHMxZDQuZGVyMEEGA1UdEQQ6
MDiCCW9sbGFtYS5haYINd3d3Lm9sbGFtYS5haYIScmVnaXN0cnkub2xsYW1hLmFp
gghvbGFtYS5haTAhBgNVHSAEGjAYMAgGBmeBDAECATAMBgorBgEEAdZ5AgUDMDwG
A1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmxzLnBraS5nb29nL2d0czFkNC9uOWli
bjExM2swTS5jcmwwggEDBgorBgEEAdZ5AgQCBIH0BIHxAO8AdgB2/4g/Crb7lVHC
Ycz1h7o0tKTNuyncaEIKn+ZnTFo6dAAAAYtxM+I0AAAEAwBHMEUCIQCk354F0/8n
q0BoXc0Px1FyjVLJcY8BQ2VRJLlOdTrioAIgBMefyBKZi2hx+g+UErhgdsogt+EW
eUfEWyWJiJROCxIAdQBIsONr2qZHNA/lagL6nTDrHFIBy1bdLIHZu7+rOdiEcwAA
AYtxM+InAAAEAwBGMEQCIF32wr65UmHvO/U+O6br0ezCFm+v1C6wRr3rmxQH2vpN
AiAKIxvSFND/hxuPvlKRJ0p+pUqHGH8Hnicy8HpIgqdMKTANBgkqhkiG9w0BAQsF
AAOCAQEAPP2fva3bP4lNW0HpNITMHS4eVjw6XEqbcBKRQBUXPl61xYhFnvpoOTHb
e+YPsBRZsAVW1VTXASXGSAfV00CCjNBmFGPoWiR2xEFgfkQ2YrxhKHf/zOBuB6ZC
hW4WLMwnhw/g/8hQ3G2HETu6vSkHE1XhRhxyoIQvzki0dB7Yj5T68haa9H6UtjPV
t25tFYObwt+on/pa3aYLya1SDgBnw/DjVwzL6tE80RAlPA7DGTdvnEAGG2+qLcMf
oOrCIb2FdroBYA0NmM7JuJP8JALmn2D3dA2z9GDwRKkx0Yi/Uohu/y4SlnOoLEf5
jEt5ej3RmUwFRb2Ef0t2VPuWUb51eA==
-----END CERTIFICATE-----
 1 s:C = US, O = Google Trust Services LLC, CN = GTS CA 1D4
   i:C = US, O = Google Trust Services LLC, CN = GTS Root R1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug 13 00:00:42 2020 GMT; NotAfter: Sep 30 00:00:42 2027 GMT
-----BEGIN CERTIFICATE-----
MIIFjDCCA3SgAwIBAgINAgCOsgIzNmWLZM3bmzANBgkqhkiG9w0BAQsFADBHMQsw
CQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExMQzEU
MBIGA1UEAxMLR1RTIFJvb3QgUjEwHhcNMjAwODEzMDAwMDQyWhcNMjcwOTMwMDAw
MDQyWjBGMQswCQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZp
Y2VzIExMQzETMBEGA1UEAxMKR1RTIENBIDFENDCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAKvAqqPCE27l0w9zC8dTPIE89bA+xTmDaG7y7VfQ4c+mOWhl
UebUQpK0yv2r678RJExK0HWDjeq+nLIHN1Em5j6rARZixmyRSjhIR0KOQPGBMUld
saztIIJ7O0g/82qj/vGDl//3t4tTqxiRhLQnTLXJdeB+2DhkdU6IIgx6wN7E5NcU
H3Rcsejcqj8p5Sj19vBm6i1FhqLGymhMFroWVUGO3xtIH91dsgy4eFKcfKVLWK3o
2190Q0Lm/SiKmLbRJ5Au4y1euFJm2JM9eB84Fkqa3ivrXWUeVtye0CQdKvsY2Fka
zvxtxvusLJzLWYHk55zcRAacDA2SeEtBbQfD1qsCAwEAAaOCAXYwggFyMA4GA1Ud
DwEB/wQEAwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwEgYDVR0T
AQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQUJeIYDrJXkZQq5dRdhpCD3lOzuJIwHwYD
VR0jBBgwFoAU5K8rJnEaK0gnhS9SZizv8IkTcT4waAYIKwYBBQUHAQEEXDBaMCYG
CCsGAQUFBzABhhpodHRwOi8vb2NzcC5wa2kuZ29vZy9ndHNyMTAwBggrBgEFBQcw
AoYkaHR0cDovL3BraS5nb29nL3JlcG8vY2VydHMvZ3RzcjEuZGVyMDQGA1UdHwQt
MCswKaAnoCWGI2h0dHA6Ly9jcmwucGtpLmdvb2cvZ3RzcjEvZ3RzcjEuY3JsME0G
A1UdIARGMEQwCAYGZ4EMAQIBMDgGCisGAQQB1nkCBQMwKjAoBggrBgEFBQcCARYc
aHR0cHM6Ly9wa2kuZ29vZy9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAgEA
IVToy24jwXUr0rAPc924vuSVbKQuYw3nLflLfLh5AYWEeVl/Du18QAWUMdcJ6o/q
FZbhXkBH0PNcw97thaf2BeoDYY9Ck/b+UGluhx06zd4EBf7H9P84nnrwpR+4GBDZ
K+Xh3I0tqJy2rgOqNDflr5IMQ8ZTWA3yltakzSBKZ6XpF0PpqyCRvp/NCGv2KX2T
uPCJvscp1/m2pVTtyBjYPRQ+QuCQGAJKjtN7R5DFrfTqMWvYgVlpCJBkwlu7+7KY
3cTIfzE7cmALskMKNLuDz+RzCcsYTsVaU7Vp3xL60OYhqFkuAOOxDZ6pHOj9+OJm
YgPmOT4X3+7L51fXJyRH9KfLRP6nT31D5nmsGAOgZ26/8T9hsBW1uo9ju5fZLZXV
VS5H0HyIBMEKyGMIPhFWrlt/hFS28N1zaKI0ZBGD3gYgDLbiDT9fGXstpk+Fmc4o
lVlWPzXe81vdoEnFbr5M272HdgJWo+WhT9BYM0Ji+wdVmnRffXgloEoluTNcWzc4
1dFpgJu8fF3LG0gl2ibSYiCi9a6hvU0TppjJyIWXhkJTcMJlPrWx1VytEUGrX2l0
JDwRjW/656r0KVB02xHRKvm2ZKI03TglLIpmVCK3kBKkKNpBNkFt8rhafcCKOb9J
x/9tpNFlQTl7B39rJlJWkR17QnZqVptFePFORoZmFzM=
-----END CERTIFICATE-----
 2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R1
   i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jun 19 00:00:42 2020 GMT; NotAfter: Jan 28 00:00:42 2028 GMT
-----BEGIN CERTIFICATE-----
MIIFYjCCBEqgAwIBAgIQd70NbNs2+RrqIQ/E8FjTDTANBgkqhkiG9w0BAQsFADBX
MQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEQMA4GA1UE
CxMHUm9vdCBDQTEbMBkGA1UEAxMSR2xvYmFsU2lnbiBSb290IENBMB4XDTIwMDYx
OTAwMDA0MloXDTI4MDEyODAwMDA0MlowRzELMAkGA1UEBhMCVVMxIjAgBgNVBAoT
GUdvb2dsZSBUcnVzdCBTZXJ2aWNlcyBMTEMxFDASBgNVBAMTC0dUUyBSb290IFIx
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAthECix7joXebO9y/lD63
ladAPKH9gvl9MgaCcfb2jH/76Nu8ai6Xl6OMS/kr9rH5zoQdsfnFl97vufKj6bwS
iV6nqlKr+CMny6SxnGPb15l+8Ape62im9MZaRw1NEDPjTrETo8gYbEvs/AmQ351k
KSUjB6G00j0uYODP0gmHu81I8E3CwnqIiru6z1kZ1q+PsAewnjHxgsHA3y6mbWwZ
DrXYfiYaRQM9sHmklCitD38m5agI/pboPGiUU+6DOogrFZYJsuB6jC511pzrp1Zk
j5ZPaK49l8KEj8C8QMALXL32h7M1bKwYUH+E4EzNktMg6TO8UpmvMrUpsyUqtEj5
cuHKZPfmghCN6J3Cioj6OGaK/GP5Afl4/Xtcd/p2h/rs37EOeZVXtL0m79YB0esW
CruOC7XFxYpVq9Os6pFLKcwZpDIlTirxZUTQAs6qzkm06p98g7BAe+dDq6dso499
iYH6TKX/1Y7DzkvgtdizjkXPdsDtQCv9Uw+wp9U7DbGKogPeMa3Md+pvez7W35Ei
Eua++tgy/BBjFFFy3l3WFpO9KWgz7zpm7AeKJt8T11dleCfeXkkUAKIAf5qoIbap
sZWwpbkNFhHax2xIPEDgfg1azVY80ZcFuctL7TlLnMQ/0lUTbiSw1nH69MG6zO0b
9f6BQdgAmD06yK56mDcYBZUCAwEAAaOCATgwggE0MA4GA1UdDwEB/wQEAwIBhjAP
BgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBTkrysmcRorSCeFL1JmLO/wiRNxPjAf
BgNVHSMEGDAWgBRge2YaRQ2XyolQL30EzTSo//z9SzBgBggrBgEFBQcBAQRUMFIw
JQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnBraS5nb29nL2dzcjEwKQYIKwYBBQUH
MAKGHWh0dHA6Ly9wa2kuZ29vZy9nc3IxL2dzcjEuY3J0MDIGA1UdHwQrMCkwJ6Al
oCOGIWh0dHA6Ly9jcmwucGtpLmdvb2cvZ3NyMS9nc3IxLmNybDA7BgNVHSAENDAy
MAgGBmeBDAECATAIBgZngQwBAgIwDQYLKwYBBAHWeQIFAwIwDQYLKwYBBAHWeQIF
AwMwDQYJKoZIhvcNAQELBQADggEBADSkHrEoo9C0dhemMXoh6dFSPsjbdBZBiLg9
NR3t5P+T4Vxfq7vqfM/b5A3Ri1fyJm9bvhdGaJQ3b2t6yMAYN/olUazsaL+yyEn9
WprKASOshIArAoyZl+tJaox118fessmXn1hIVw41oeQa1v1vg4Fv74zPl6/AhSrw
9U5pCZEt4Wi4wStz6dTZ/CLANx8LZh1J7QJVj2fhMtfTJr9w4z30Z209fOU0iOMy
+qduBmpvvYuR7hZL6Dupszfnw0Skfths18dG9ZKb59UhvmaSGZRVbNQpsg3BZlvi
d0lIKO2d1xozclOzgjXPYovJJIultzkMu34qQb9Sz/yilrbCgj8=
-----END CERTIFICATE-----
---
Server certificate
subject=CN = ollama.ai
issuer=C = US, O = Google Trust Services LLC, CN = GTS CA 1D4
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 4720 bytes and written 400 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
<!-- gh-comment-id:1806005631 --> @marcellodesales commented on GitHub (Nov 10, 2023): @mxyng Just looking for it... and Yes, the certificate is in the container... * What I can see now is that we have an egress rule going through a firewall with cillium, which I wasn't aware of... :S * I did patch the app to by-pass the certificate verification and I was able to connect to the service, but for that the firewall blocked the call https://github.com/jmorganca/ollama/issues/1072#issue-1987255223 So, I could only understand the problem by disabling the certificate verification in code with the patch I suggested... that way I think we can give others the same benefit... I suggest just having the skip setting as an env var... # 🔊 Logs for the ollama container cert * Logs for the verification of the certs described... ```console $ docker run --entrypoint bash -v $(pwd):$(pwd) -w $(pwd) ollama/ollama -c "openssl s_client -connect registry.ollama.ai:443 -showcerts </dev/null"</summary> depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1 verify return:1 depth=1 C = US, O = Google Trust Services LLC, CN = GTS CA 1D4 verify return:1 depth=0 CN = ollama.ai verify return:1 DONE CONNECTED(00000003) --- Certificate chain 0 s:CN = ollama.ai i:C = US, O = Google Trust Services LLC, CN = GTS CA 1D4 a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256 v:NotBefore: Oct 27 11:55:06 2023 GMT; NotAfter: Jan 25 12:48:40 2024 GMT -----BEGIN CERTIFICATE----- MIIFgjCCBGqgAwIBAgIQc70da3L+ed4QYvXhE8fpVjANBgkqhkiG9w0BAQsFADBG MQswCQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExM QzETMBEGA1UEAxMKR1RTIENBIDFENDAeFw0yMzEwMjcxMTU1MDZaFw0yNDAxMjUx MjQ4NDBaMBQxEjAQBgNVBAMTCW9sbGFtYS5haTCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBALApuMJBxFiSIS42O04cLo18/rnNMQAJZ43luT08MLNxyE9B YCs8iEEucfNg8b7ORbVlAcHx6v+ujnme8bVZ++7yMOcTSzLnJkUydZ9JY/mAIGFM E1onacM1TY0uL+DbGwoI+1b3ZL586ktHWs5KpN7EepO7tTtLqt0RGYyIiDO1xbYt ckjeFgD4kSElo+4NqpBFdYL2tDU9Ewv4pbub9h+GQQuyWT+GrBga931FN6IUEwJX 9yuMJctBWABo1hRmn7n5NcqdycdgGXZqEGC7ycnsoCumTydqHkvSiyFUVF3wB8mI 6LRXeUI0byzP8xZTUQms7+njXiGTquv1WzwUNFsCAwEAAaOCApwwggKYMA4GA1Ud DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMB0G A1UdDgQWBBQZjVZZ6jHzir6KpT/u0HEnw+y3pTAfBgNVHSMEGDAWgBQl4hgOsleR lCrl1F2GkIPeU7O4kjB4BggrBgEFBQcBAQRsMGowNQYIKwYBBQUHMAGGKWh0dHA6 Ly9vY3NwLnBraS5nb29nL3MvZ3RzMWQ0L1h5aElWN2FZV0lzMDEGCCsGAQUFBzAC hiVodHRwOi8vcGtpLmdvb2cvcmVwby9jZXJ0cy9ndHMxZDQuZGVyMEEGA1UdEQQ6 MDiCCW9sbGFtYS5haYINd3d3Lm9sbGFtYS5haYIScmVnaXN0cnkub2xsYW1hLmFp gghvbGFtYS5haTAhBgNVHSAEGjAYMAgGBmeBDAECATAMBgorBgEEAdZ5AgUDMDwG A1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmxzLnBraS5nb29nL2d0czFkNC9uOWli bjExM2swTS5jcmwwggEDBgorBgEEAdZ5AgQCBIH0BIHxAO8AdgB2/4g/Crb7lVHC Ycz1h7o0tKTNuyncaEIKn+ZnTFo6dAAAAYtxM+I0AAAEAwBHMEUCIQCk354F0/8n q0BoXc0Px1FyjVLJcY8BQ2VRJLlOdTrioAIgBMefyBKZi2hx+g+UErhgdsogt+EW eUfEWyWJiJROCxIAdQBIsONr2qZHNA/lagL6nTDrHFIBy1bdLIHZu7+rOdiEcwAA AYtxM+InAAAEAwBGMEQCIF32wr65UmHvO/U+O6br0ezCFm+v1C6wRr3rmxQH2vpN AiAKIxvSFND/hxuPvlKRJ0p+pUqHGH8Hnicy8HpIgqdMKTANBgkqhkiG9w0BAQsF AAOCAQEAPP2fva3bP4lNW0HpNITMHS4eVjw6XEqbcBKRQBUXPl61xYhFnvpoOTHb e+YPsBRZsAVW1VTXASXGSAfV00CCjNBmFGPoWiR2xEFgfkQ2YrxhKHf/zOBuB6ZC hW4WLMwnhw/g/8hQ3G2HETu6vSkHE1XhRhxyoIQvzki0dB7Yj5T68haa9H6UtjPV t25tFYObwt+on/pa3aYLya1SDgBnw/DjVwzL6tE80RAlPA7DGTdvnEAGG2+qLcMf oOrCIb2FdroBYA0NmM7JuJP8JALmn2D3dA2z9GDwRKkx0Yi/Uohu/y4SlnOoLEf5 jEt5ej3RmUwFRb2Ef0t2VPuWUb51eA== -----END CERTIFICATE----- 1 s:C = US, O = Google Trust Services LLC, CN = GTS CA 1D4 i:C = US, O = Google Trust Services LLC, CN = GTS Root R1 a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256 v:NotBefore: Aug 13 00:00:42 2020 GMT; NotAfter: Sep 30 00:00:42 2027 GMT -----BEGIN CERTIFICATE----- MIIFjDCCA3SgAwIBAgINAgCOsgIzNmWLZM3bmzANBgkqhkiG9w0BAQsFADBHMQsw CQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZpY2VzIExMQzEU MBIGA1UEAxMLR1RTIFJvb3QgUjEwHhcNMjAwODEzMDAwMDQyWhcNMjcwOTMwMDAw MDQyWjBGMQswCQYDVQQGEwJVUzEiMCAGA1UEChMZR29vZ2xlIFRydXN0IFNlcnZp Y2VzIExMQzETMBEGA1UEAxMKR1RTIENBIDFENDCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBAKvAqqPCE27l0w9zC8dTPIE89bA+xTmDaG7y7VfQ4c+mOWhl UebUQpK0yv2r678RJExK0HWDjeq+nLIHN1Em5j6rARZixmyRSjhIR0KOQPGBMUld saztIIJ7O0g/82qj/vGDl//3t4tTqxiRhLQnTLXJdeB+2DhkdU6IIgx6wN7E5NcU H3Rcsejcqj8p5Sj19vBm6i1FhqLGymhMFroWVUGO3xtIH91dsgy4eFKcfKVLWK3o 2190Q0Lm/SiKmLbRJ5Au4y1euFJm2JM9eB84Fkqa3ivrXWUeVtye0CQdKvsY2Fka zvxtxvusLJzLWYHk55zcRAacDA2SeEtBbQfD1qsCAwEAAaOCAXYwggFyMA4GA1Ud DwEB/wQEAwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwEgYDVR0T AQH/BAgwBgEB/wIBADAdBgNVHQ4EFgQUJeIYDrJXkZQq5dRdhpCD3lOzuJIwHwYD VR0jBBgwFoAU5K8rJnEaK0gnhS9SZizv8IkTcT4waAYIKwYBBQUHAQEEXDBaMCYG CCsGAQUFBzABhhpodHRwOi8vb2NzcC5wa2kuZ29vZy9ndHNyMTAwBggrBgEFBQcw AoYkaHR0cDovL3BraS5nb29nL3JlcG8vY2VydHMvZ3RzcjEuZGVyMDQGA1UdHwQt MCswKaAnoCWGI2h0dHA6Ly9jcmwucGtpLmdvb2cvZ3RzcjEvZ3RzcjEuY3JsME0G A1UdIARGMEQwCAYGZ4EMAQIBMDgGCisGAQQB1nkCBQMwKjAoBggrBgEFBQcCARYc aHR0cHM6Ly9wa2kuZ29vZy9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAgEA IVToy24jwXUr0rAPc924vuSVbKQuYw3nLflLfLh5AYWEeVl/Du18QAWUMdcJ6o/q FZbhXkBH0PNcw97thaf2BeoDYY9Ck/b+UGluhx06zd4EBf7H9P84nnrwpR+4GBDZ K+Xh3I0tqJy2rgOqNDflr5IMQ8ZTWA3yltakzSBKZ6XpF0PpqyCRvp/NCGv2KX2T uPCJvscp1/m2pVTtyBjYPRQ+QuCQGAJKjtN7R5DFrfTqMWvYgVlpCJBkwlu7+7KY 3cTIfzE7cmALskMKNLuDz+RzCcsYTsVaU7Vp3xL60OYhqFkuAOOxDZ6pHOj9+OJm YgPmOT4X3+7L51fXJyRH9KfLRP6nT31D5nmsGAOgZ26/8T9hsBW1uo9ju5fZLZXV VS5H0HyIBMEKyGMIPhFWrlt/hFS28N1zaKI0ZBGD3gYgDLbiDT9fGXstpk+Fmc4o lVlWPzXe81vdoEnFbr5M272HdgJWo+WhT9BYM0Ji+wdVmnRffXgloEoluTNcWzc4 1dFpgJu8fF3LG0gl2ibSYiCi9a6hvU0TppjJyIWXhkJTcMJlPrWx1VytEUGrX2l0 JDwRjW/656r0KVB02xHRKvm2ZKI03TglLIpmVCK3kBKkKNpBNkFt8rhafcCKOb9J x/9tpNFlQTl7B39rJlJWkR17QnZqVptFePFORoZmFzM= -----END CERTIFICATE----- 2 s:C = US, O = Google Trust Services LLC, CN = GTS Root R1 i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256 v:NotBefore: Jun 19 00:00:42 2020 GMT; NotAfter: Jan 28 00:00:42 2028 GMT -----BEGIN CERTIFICATE----- MIIFYjCCBEqgAwIBAgIQd70NbNs2+RrqIQ/E8FjTDTANBgkqhkiG9w0BAQsFADBX MQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEQMA4GA1UE CxMHUm9vdCBDQTEbMBkGA1UEAxMSR2xvYmFsU2lnbiBSb290IENBMB4XDTIwMDYx OTAwMDA0MloXDTI4MDEyODAwMDA0MlowRzELMAkGA1UEBhMCVVMxIjAgBgNVBAoT GUdvb2dsZSBUcnVzdCBTZXJ2aWNlcyBMTEMxFDASBgNVBAMTC0dUUyBSb290IFIx MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAthECix7joXebO9y/lD63 ladAPKH9gvl9MgaCcfb2jH/76Nu8ai6Xl6OMS/kr9rH5zoQdsfnFl97vufKj6bwS iV6nqlKr+CMny6SxnGPb15l+8Ape62im9MZaRw1NEDPjTrETo8gYbEvs/AmQ351k KSUjB6G00j0uYODP0gmHu81I8E3CwnqIiru6z1kZ1q+PsAewnjHxgsHA3y6mbWwZ DrXYfiYaRQM9sHmklCitD38m5agI/pboPGiUU+6DOogrFZYJsuB6jC511pzrp1Zk j5ZPaK49l8KEj8C8QMALXL32h7M1bKwYUH+E4EzNktMg6TO8UpmvMrUpsyUqtEj5 cuHKZPfmghCN6J3Cioj6OGaK/GP5Afl4/Xtcd/p2h/rs37EOeZVXtL0m79YB0esW CruOC7XFxYpVq9Os6pFLKcwZpDIlTirxZUTQAs6qzkm06p98g7BAe+dDq6dso499 iYH6TKX/1Y7DzkvgtdizjkXPdsDtQCv9Uw+wp9U7DbGKogPeMa3Md+pvez7W35Ei Eua++tgy/BBjFFFy3l3WFpO9KWgz7zpm7AeKJt8T11dleCfeXkkUAKIAf5qoIbap sZWwpbkNFhHax2xIPEDgfg1azVY80ZcFuctL7TlLnMQ/0lUTbiSw1nH69MG6zO0b 9f6BQdgAmD06yK56mDcYBZUCAwEAAaOCATgwggE0MA4GA1UdDwEB/wQEAwIBhjAP BgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBTkrysmcRorSCeFL1JmLO/wiRNxPjAf BgNVHSMEGDAWgBRge2YaRQ2XyolQL30EzTSo//z9SzBgBggrBgEFBQcBAQRUMFIw JQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnBraS5nb29nL2dzcjEwKQYIKwYBBQUH MAKGHWh0dHA6Ly9wa2kuZ29vZy9nc3IxL2dzcjEuY3J0MDIGA1UdHwQrMCkwJ6Al oCOGIWh0dHA6Ly9jcmwucGtpLmdvb2cvZ3NyMS9nc3IxLmNybDA7BgNVHSAENDAy MAgGBmeBDAECATAIBgZngQwBAgIwDQYLKwYBBAHWeQIFAwIwDQYLKwYBBAHWeQIF AwMwDQYJKoZIhvcNAQELBQADggEBADSkHrEoo9C0dhemMXoh6dFSPsjbdBZBiLg9 NR3t5P+T4Vxfq7vqfM/b5A3Ri1fyJm9bvhdGaJQ3b2t6yMAYN/olUazsaL+yyEn9 WprKASOshIArAoyZl+tJaox118fessmXn1hIVw41oeQa1v1vg4Fv74zPl6/AhSrw 9U5pCZEt4Wi4wStz6dTZ/CLANx8LZh1J7QJVj2fhMtfTJr9w4z30Z209fOU0iOMy +qduBmpvvYuR7hZL6Dupszfnw0Skfths18dG9ZKb59UhvmaSGZRVbNQpsg3BZlvi d0lIKO2d1xozclOzgjXPYovJJIultzkMu34qQb9Sz/yilrbCgj8= -----END CERTIFICATE----- --- Server certificate subject=CN = ollama.ai issuer=C = US, O = Google Trust Services LLC, CN = GTS CA 1D4 --- No client certificate CA names sent Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 4720 bytes and written 400 bytes Verification: OK --- New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 2048 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 0 (ok) ---
GiteaMirror commented 2026-04-28 02:49:03 -05:00
Author
Owner
Copy Link

@marcellodesales commented on GitHub (Nov 11, 2023):

testing with 0.1.8 without the patch

  • I'm getting this from calling the pull command, it it seems like it is trying to pull the llama2 image from dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/ sha256/22/22f7f8ef5f
    • Could this be the problem with the certificate issue?
Run docker run --network host  -v $(pwd):$(pwd) -w $(pwd) -v $HOME/.ollama:/root/.ollama dockerhub.docker.artifactory.company.com/ollama/ollama:0.1.8 pull llama2 || true
pulling manifest
Error: Head "https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/
sha256/22/22f7f8ef5f[4](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm-
prreviewer/actions/runs/508408/jobs/1423518#step:6:5)c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb8
0a983a2/data?X-Amz-Algorithm=AWS4-HMAC-SHA2[5](https://git.company.com/vionix-platform/devsecops-ai-llm-prreviewer/actions/runs/508408/jobs/1423518#step:6:6)6&X-Amz-
Credential=66040c77ac1b787c3af820529859349a%!F(MISSING)20231111%!F(MISSING)auto%!F(MISSING)
s3%!F(MISSING)aws4_request&X-Amz-Date=20231111T003348Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=7b502e76cdb2848abe1b7541d47e9c2c4ffe5cd9df29c7d91fbdd89
4169c5dcd": tls: failed to verify certificate: x509: certificate signed by unknown authority

testing 0.1.8 With the patch

  • The patch does make it work by skipping the TLS verification from the pull request
  • For some reason, the 403 result below is similar to the current issue #676
Run curl -s -i --no-buffer http://localhost:11434/api/pull -d '{"name": "llama2"}'
HTTP/1.1 200 OK
Content-Type: application/x-ndjson
Date: Sat, 11 Nov 2023 00:46:23 GMT
Transfer-Encoding: chunked

{"status":"pulling manifest"}
{"error":"403: "}
{"status":"pulling manifest"}
pulling manifest
{"error":"pull model manifest: file does not exist"}
<!-- gh-comment-id:1806602648 --> @marcellodesales commented on GitHub (Nov 11, 2023): # testing with 0.1.8 without the patch * I'm getting this from calling the pull command, it it seems like it is trying to pull the llama2 image from `dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/ sha256/22/22f7f8ef5f` * Could this be the problem with the certificate issue? ```console Run docker run --network host -v $(pwd):$(pwd) -w $(pwd) -v $HOME/.ollama:/root/.ollama dockerhub.docker.artifactory.company.com/ollama/ollama:0.1.8 pull llama2 || true pulling manifest Error: Head "https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/ sha256/22/22f7f8ef5f[4](https://git.company.com/seceng-devsecops-platform/devsecops-ai-llm- prreviewer/actions/runs/508408/jobs/1423518#step:6:5)c791c1b03d7eb414399294764d7cc82c7e94aa81a1feb8 0a983a2/data?X-Amz-Algorithm=AWS4-HMAC-SHA2[5](https://git.company.com/vionix-platform/devsecops-ai-llm-prreviewer/actions/runs/508408/jobs/1423518#step:6:6)6&X-Amz- Credential=66040c77ac1b787c3af820529859349a%!F(MISSING)20231111%!F(MISSING)auto%!F(MISSING) s3%!F(MISSING)aws4_request&X-Amz-Date=20231111T003348Z&X-Amz-Expires=1200&X-Amz-SignedHeaders=host&X-Amz-Signature=7b502e76cdb2848abe1b7541d47e9c2c4ffe5cd9df29c7d91fbdd89 4169c5dcd": tls: failed to verify certificate: x509: certificate signed by unknown authority ``` # testing 0.1.8 With the patch * The patch does make it work by skipping the TLS verification from the pull request * For some reason, the 403 result below is similar to the current issue #676 ```console Run curl -s -i --no-buffer http://localhost:11434/api/pull -d '{"name": "llama2"}' HTTP/1.1 200 OK Content-Type: application/x-ndjson Date: Sat, 11 Nov 2023 00:46:23 GMT Transfer-Encoding: chunked {"status":"pulling manifest"} {"error":"403: "} {"status":"pulling manifest"} pulling manifest {"error":"pull model manifest: file does not exist"} ```
GiteaMirror commented 2026-04-28 02:49:05 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Nov 17, 2023):

It's highly possible the certificate, while in the container, may not be configured. Here's an example of using a self-signed certificate in Docker

  1. First start a proxy. In this case, I'm using mitmproxy because it's very easy to setup. This starts mitmproxy in a container and mounts its home directory, giving us access to the self-signed certificates it created. 
    docker run -d -v ./mitmproxy:/home/mitmproxy/.mitmproxy -p 61107:8080 --name mitmproxy mitmproxy/mitmproxy mitmdump
  2. Install the mitmproxy CA certificate into the ollama container 
    FROM ollama/ollama
COPY ./mitmproxy/mitmproxy-ca.pem /usr/local/share/ca-certificates/mitmproxy-ca.crt
RUN update-ca-certificates
  3. Build this new image 
    docker build -t ollama-with-ca .
  4. Start Ollama in Docker with HTTPS_PROXY set. This assumes Docker Desktop which sets the magic hostname host.docker.internal 
    docker run -d -p 11434:11434 -e HTTPS_PROXY=http://host.docker.internal:61107 --name ollama ollama-with-ca
  5. Pull an image 
    docker exec -it ollama ollama pull orca-mini

Note 2 very important steps:

  1. Setting HTTPS_PROXY when starting the Ollama container
  2. Installing the self-signed certificate into /usr/local/share/ca-certificates and calling update-ca-certificates

Without 1, you will (very likely) get blocked by the network.
Without 2, you will get failed to verify certificate: x509: certificate signed by unknown authority

<!-- gh-comment-id:1815546695 --> @mxyng commented on GitHub (Nov 17, 2023): It's highly possible the certificate, while in the container, may not be configured. Here's an example of using a self-signed certificate in Docker 1. First start a proxy. In this case, I'm using [mitmproxy](https://mitmproxy.org/) because it's very easy to setup. This starts mitmproxy in a container and mounts its home directory, giving us access to the self-signed certificates it created. ``` docker run -d -v ./mitmproxy:/home/mitmproxy/.mitmproxy -p 61107:8080 --name mitmproxy mitmproxy/mitmproxy mitmdump ``` 2. Install the mitmproxy CA certificate into the ollama container ```dockerfile FROM ollama/ollama COPY ./mitmproxy/mitmproxy-ca.pem /usr/local/share/ca-certificates/mitmproxy-ca.crt RUN update-ca-certificates ``` 3. Build this new image ``` docker build -t ollama-with-ca . ``` 4. Start Ollama in Docker with HTTPS_PROXY set. This assumes Docker Desktop which sets the magic hostname `host.docker.internal` ``` docker run -d -p 11434:11434 -e HTTPS_PROXY=http://host.docker.internal:61107 --name ollama ollama-with-ca ``` 5. Pull an image ``` docker exec -it ollama ollama pull orca-mini ``` Note 2 very important steps: 1. Setting `HTTPS_PROXY` when starting the Ollama container 2. Installing the self-signed certificate into `/usr/local/share/ca-certificates` and calling `update-ca-certificates` Without 1, you will (very likely) get blocked by the network. Without 2, you will get `failed to verify certificate: x509: certificate signed by unknown authority`
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#47034
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?