github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 08:59:13 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #5912] [CLOSED] Server tls 3203 #22501

New Issue
Closed
opened 2026-04-19 16:23:12 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-19 16:23:12 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/5912
Author: @gabe-l-hart
Created: 7/24/2024
Status: ❌ Closed

Base: main ← Head: ServerTLS-3203

📝 Commits (9)

  • cdc5e69 feat: Add config plumbing for TLS server options
  • 04ebafc feat: Add validation logic for host scheme vs TLS config
  • c4a6b1a feat: Set up serving with (m)TLS
  • f42d8d1 test: Unit tests for TLS config
  • 0cc7891 feat: Add mTLS config parsing and usage to the client side
  • ed1bb03 feat: Split blocking server.Serve into non-blocking and blocking
  • 4d4d4f5 test: Move TLS test data gen into a test package
  • 17604d1 test: Unit tests for client/server in all flavors of (m)TLS config
  • 5358d8e fix: Move to debug logging

📊 Changes

7 files changed (+526 additions, -22 deletions)

View changed files

📝 api/client.go (+18 -2)
📝 cmd/cmd.go (+19 -1)
📝 envconfig/config.go (+111 -0)
📝 envconfig/config_test.go (+68 -0)
➕ envconfig/configtest/tls_helpers.go (+99 -0)
➕ server/mtls_test.go (+152 -0)
📝 server/routes.go (+59 -19)

📄 Description

Disclaimer!

This PR started as a small feature addition and resulted in some significant scope creep when I added the unit tests. I'm certainly open to trying to remove some of that refactoring for ServerNonBlocking if that's preferred, but figured it was worth presenting as-is to start the discussion.

Issues

This issue supports https://github.com/ollama/ollama/issues/3203 by adding encryption between client and server. It does not fully address the issue since the core feature request is for auth.

Description

This PR adds support for running the primary ollama server/client interactions using TLS and Mutual TLS (mTLS). It does not address encryption between the ollama server and the individual model servers.

Changes

The changes in this PR are grouped as follows:

New envconfig variables:

  • To boot an (m)TLS server:
    • OLLAMA_HOST: If the scheme is https://, the server will attempt to boot with TLS or mTLS based on the presence of the below variables
    • OLLAMA_TLS_SERVER_KEY: File with the private key (required for TLS)
    • OLLAMA_TLS_SERVER_CERT: File with the public cert (required for TLS)
    • OLLAMA_TLS_CLIENT_CA: File with the CA cert for the key/cert pair the client will use (required for mTLS)
  • To connect a client to an (m)TLS server:
    • OLLAMA_HOST: If the scheme is https://, the client will attempt to connect to a (m)TLS server depending on the presence of the below variables
    • OLLAMA_TLS_SERVER_CA: File with the CA cert for the server's key/cert pair (required for TLS if the server's key/cert pair is signed by a non-system CA. If not given, but the scheme is https://, the system CAs will be used.)
    • OLLAMA_TLS_CLIENT_KEY: File with the private key for the client when connecting to an mTLS server (required for mTLS)
    • OLLAMA_TLS_CLIENT_CERT: File with the public cert for the client when connecting to an mTLS server (required for mTLS)

Config parsing:

  • In envconfig, there is a new getTlsConfig function which parses all of the TLS-related variables for both client and server
  • getTlsConfig is used to populate envconfig.ServerTlsConfig and envconfig.ClientTlsconfig with tls.Config objects if configured to use (m)TLS
  • If not configured for TLS, these objects will remain nil which is the indicator elsewhere in the code that TLS is not enabled

Server Setup

The primary change in routes.go is to add a conditional around calling the ServeTLS function on the http.Server object based on the value of envconfig.ServerTlsConfig.

The rest of the changes there were all made in support of helping to make the server easier to boot in unit tests. For that, I split the Serve function into two parts: ServeNonBlocking which returns an instance of the server.Server struct, and Serve which uses ServeNonBlocking and then blocks on the server terminating.

Client Setup

In client.go, the change is to look at envconfig.ClientTlsConfig and instantiate the http.Client accordingly

Unit Testing

  • Add a new testing package envconfig/configtest that holds helpers for dynamically generating TLS data
  • Extend the envconfig tests to test the parsing of config data
  • Add a new server/mtls_test.go test suite that tests the server/client communication with no TLS, standard TLS, and mTLS

Testing

In addition to the unit tests, I've also verified that the communication works separately using scripts I have from other projects for generating self-signed mTLS data. Here are the steps I used:

gen_mtls_test_files.sh 
#!/usr/bin/env bash

## Config ######################################################################

# Optional additional SANs can be set with SANS
SANS=${SANS:-""}

# CN can be overloaded
CN=${CN:-"foo.bar.com"}

set -eo pipefail

# Set up additional SANs block
IFS=' ' read -r -a sans_arr <<< "$SANS"
extra_sans=""
counter="1"
for san in "${sans_arr[@]}"
do
    counter=$(expr "$counter" "+" "1")
    echo "Adding SAN DNS.$counter [$san]"
    extra_sans="$extra_sans\nDNS.$counter = $san"
done

root_name="ca"
root_key="$root_name.key.pem"
root_crt="$root_name.cert.pem"

server_name="server"
server_key="$server_name.key.pem"
server_crt="$server_name.cert.pem"

client_name="client"
client_key="$client_name.key.pem"
client_crt="$client_name.cert.pem"

common_config='
[req]
default_bits       = 4096
default_keyfile    = server.key.pem
distinguished_name = req_distinguished_name
x509_extensions    = x509_ext
string_mask        = utf8only

[req_distinguished_name]
countryName                 = Country Name (2 letter code)
countryName_default         = US
stateOrProvinceName         = State or Province Name (full name)
stateOrProvinceName_default = Denver
localityName                = Locality Name (eg, city)
localityName_default        = Denver
organizationName            = Organization Name (eg, company)
organizationName_default    = Gabe Inc
commonName                  = Common Name (eg, YOUR name)
commonName_max              = 64
'

ca_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always, issuer
basicConstraints       = critical, CA:TRUE, pathlen:1
keyUsage               = keyCertSign, cRLSign
"

derived_config="
$common_config

[x509_ext]
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid,issuer
basicConstraints       = CA:FALSE
keyUsage               = digitalSignature, keyEncipherment
subjectAltName         = @alt_names

[alt_names]
DNS.1 = localhost
$extra_sans
IP.1  = '127.0.0.1'
"

# use wild card in subject, not all clients accept that, but Java grpc client does
# we also have subject alternative names 127.0.0.1 and localhost in our openssl.cnf file (used when creating the server crt)
SUBJ="/C=US/ST=Denver/L=Denver/O=Gabe Inc/CN=$CN"

# Set the expiration for 10 years
expiration_days=3650

## Root ########################################################################


# Create the root key
echo "[Creating root key]"
openssl genrsa -out $root_key 2048

# create root key and cert
echo "[Creating root cert]"
openssl req \
    -config <(echo -e "$ca_config") \
    -x509 \
    -new \
    -nodes \
    -key $root_key \
    -sha256 \
    -subj "$SUBJ" \
    -out $root_crt

## Server ######################################################################

# create a new server key and certificate signing request
echo "[Creating server key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $server_key -out $server_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign server cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $server_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $server_crt -days $expiration_days

# write out server key in pkcs8 format, required by grpc
echo "[Convert server key to pkcs8]"
cp $server_key $server_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $server_key.tmp -out $server_key

## Client ######################################################################

# create a new server key and certificate signing request
echo "[Creating client key and signing request]"
openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $client_key -out $client_name.csr -newkey rsa:2048 -subj "$SUBJ"

# sign the request with our root cert key
echo "[Sign client cert]"
openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $client_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $client_crt -days $expiration_days

# write out client key in pkcs8 format, required by grpc
echo "[Convert client key to pkcs8]"
cp $client_key $client_key.tmp
openssl pkcs8 -topk8 -nocrypt -in $client_key.tmp -out $client_key

# Clean up
rm *.tmp
rm *.csr
rm *.srl

# Using the above gen_mtls_test_files.sh
gen_mtls_test_files.sh

# Build it
go build .

# Boot the server with mTLS enabled
OLLAMA_HOST="https://localhost:54321" \
OLLAMA_TLS_SERVER_KEY=server.key.pem \
OLLAMA_TLS_SERVER_CERT=server.cert.pem \
OLLAMA_TLS_CLIENT_CA=ca.cert.pem \
./ollama serve

# Run a client command with mTLS enabled (separate terminal or background the server)
OLLAMA_HOST="https://127.0.0.1:54321" \
OLLAMA_TLS_SERVER_CA=ca.cert.pem \
OLLAMA_TLS_CLIENT_KEY=client.key.pem \
OLLAMA_TLS_CLIENT_CERT=client.cert.pem  \
./ollama ls

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/5912 **Author:** [@gabe-l-hart](https://github.com/gabe-l-hart) **Created:** 7/24/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `ServerTLS-3203` --- ### 📝 Commits (9) - [`cdc5e69`](https://github.com/ollama/ollama/commit/cdc5e69704131db61a370bf82b351c0dd9cf89cd) feat: Add config plumbing for TLS server options - [`04ebafc`](https://github.com/ollama/ollama/commit/04ebafc6f41696784cac6177920c89dde762fe2d) feat: Add validation logic for host scheme vs TLS config - [`c4a6b1a`](https://github.com/ollama/ollama/commit/c4a6b1a11e326791a7f1f3fd9a8e4d19d57f40ae) feat: Set up serving with (m)TLS - [`f42d8d1`](https://github.com/ollama/ollama/commit/f42d8d1dd78fdc6c4c0eab0ac44da1105ee42946) test: Unit tests for TLS config - [`0cc7891`](https://github.com/ollama/ollama/commit/0cc78919f03cafa21f78e61a63454aceecc6f848) feat: Add mTLS config parsing and usage to the client side - [`ed1bb03`](https://github.com/ollama/ollama/commit/ed1bb0366084860ca2f1e5b639326b82527a57a3) feat: Split blocking server.Serve into non-blocking and blocking - [`4d4d4f5`](https://github.com/ollama/ollama/commit/4d4d4f53e1a8aaead82614c3d6e964cf56130e06) test: Move TLS test data gen into a test package - [`17604d1`](https://github.com/ollama/ollama/commit/17604d1c5d201422f6016419e8e86acadee4d617) test: Unit tests for client/server in all flavors of (m)TLS config - [`5358d8e`](https://github.com/ollama/ollama/commit/5358d8e88b87065d2a686c628b740a4d1a765ce1) fix: Move to debug logging ### 📊 Changes **7 files changed** (+526 additions, -22 deletions) <details> <summary>View changed files</summary> 📝 `api/client.go` (+18 -2) 📝 `cmd/cmd.go` (+19 -1) 📝 `envconfig/config.go` (+111 -0) 📝 `envconfig/config_test.go` (+68 -0) ➕ `envconfig/configtest/tls_helpers.go` (+99 -0) ➕ `server/mtls_test.go` (+152 -0) 📝 `server/routes.go` (+59 -19) </details> ### 📄 Description **Disclaimer!** This PR started as a small feature addition and resulted in some significant scope creep when I added the unit tests. I'm certainly open to trying to remove some of that refactoring for `ServerNonBlocking` if that's preferred, but figured it was worth presenting as-is to start the discussion. ## Issues This issue supports https://github.com/ollama/ollama/issues/3203 by adding encryption between client and server. It does not fully address the issue since the core feature request is for auth. ## Description This PR adds support for running the primary `ollama` server/client interactions using TLS and Mutual TLS (mTLS). It does not address encryption between the `ollama` server and the individual model servers. ## Changes The changes in this PR are grouped as follows: ### New `envconfig` variables: * To boot an (m)TLS server: * `OLLAMA_HOST`: If the `scheme` is `https://`, the server will attempt to boot with TLS or mTLS based on the presence of the below variables * `OLLAMA_TLS_SERVER_KEY`: File with the private key (required for TLS) * `OLLAMA_TLS_SERVER_CERT`: File with the public cert (required for TLS) * `OLLAMA_TLS_CLIENT_CA`: File with the CA cert for the key/cert pair the client will use (required for mTLS) * To connect a client to an (m)TLS server: * `OLLAMA_HOST`: If the `scheme` is `https://`, the client will attempt to connect to a (m)TLS server depending on the presence of the below variables * `OLLAMA_TLS_SERVER_CA`: File with the CA cert for the server's key/cert pair (required for TLS if the server's key/cert pair is signed by a non-system CA. If not given, but the `scheme` is `https://`, the system CAs will be used.) * `OLLAMA_TLS_CLIENT_KEY`: File with the private key for the client when connecting to an mTLS server (required for mTLS) * `OLLAMA_TLS_CLIENT_CERT`: File with the public cert for the client when connecting to an mTLS server (required for mTLS) ### Config parsing: * In `envconfig`, there is a new `getTlsConfig` function which parses all of the TLS-related variables for both client and server * `getTlsConfig` is used to populate `envconfig.ServerTlsConfig` and `envconfig.ClientTlsconfig` with [tls.Config](https://pkg.go.dev/crypto/tls#Config) objects if configured to use (m)TLS * If not configured for TLS, these objects will remain `nil` which is the indicator elsewhere in the code that TLS is not enabled ### Server Setup The primary change in `routes.go` is to add a conditional around calling the `ServeTLS` function on the `http.Server` object based on the value of `envconfig.ServerTlsConfig`. The rest of the changes there were all made in support of helping to make the server easier to boot in unit tests. For that, I split the `Serve` function into two parts: `ServeNonBlocking` which returns an instance of the `server.Server` struct, and `Serve` which uses `ServeNonBlocking` and then blocks on the server terminating. ### Client Setup In `client.go`, the change is to look at `envconfig.ClientTlsConfig` and instantiate the `http.Client` accordingly ### Unit Testing * Add a new testing package `envconfig/configtest` that holds helpers for dynamically generating TLS data * Extend the `envconfig` tests to test the parsing of config data * Add a new `server/mtls_test.go` test suite that tests the server/client communication with no TLS, standard TLS, and mTLS ## Testing In addition to the unit tests, I've also verified that the communication works separately using scripts I have from other projects for generating self-signed mTLS data. Here are the steps I used: <details> <summary>gen_mtls_test_files.sh</summary> ```sh #!/usr/bin/env bash ## Config ###################################################################### # Optional additional SANs can be set with SANS SANS=${SANS:-""} # CN can be overloaded CN=${CN:-"foo.bar.com"} set -eo pipefail # Set up additional SANs block IFS=' ' read -r -a sans_arr <<< "$SANS" extra_sans="" counter="1" for san in "${sans_arr[@]}" do counter=$(expr "$counter" "+" "1") echo "Adding SAN DNS.$counter [$san]" extra_sans="$extra_sans\nDNS.$counter = $san" done root_name="ca" root_key="$root_name.key.pem" root_crt="$root_name.cert.pem" server_name="server" server_key="$server_name.key.pem" server_crt="$server_name.cert.pem" client_name="client" client_key="$client_name.key.pem" client_crt="$client_name.cert.pem" common_config=' [req] default_bits = 4096 default_keyfile = server.key.pem distinguished_name = req_distinguished_name x509_extensions = x509_ext string_mask = utf8only [req_distinguished_name] countryName = Country Name (2 letter code) countryName_default = US stateOrProvinceName = State or Province Name (full name) stateOrProvinceName_default = Denver localityName = Locality Name (eg, city) localityName_default = Denver organizationName = Organization Name (eg, company) organizationName_default = Gabe Inc commonName = Common Name (eg, YOUR name) commonName_max = 64 ' ca_config=" $common_config [x509_ext] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always, issuer basicConstraints = critical, CA:TRUE, pathlen:1 keyUsage = keyCertSign, cRLSign " derived_config=" $common_config [x509_ext] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer basicConstraints = CA:FALSE keyUsage = digitalSignature, keyEncipherment subjectAltName = @alt_names [alt_names] DNS.1 = localhost $extra_sans IP.1 = '127.0.0.1' " # use wild card in subject, not all clients accept that, but Java grpc client does # we also have subject alternative names 127.0.0.1 and localhost in our openssl.cnf file (used when creating the server crt) SUBJ="/C=US/ST=Denver/L=Denver/O=Gabe Inc/CN=$CN" # Set the expiration for 10 years expiration_days=3650 ## Root ######################################################################## # Create the root key echo "[Creating root key]" openssl genrsa -out $root_key 2048 # create root key and cert echo "[Creating root cert]" openssl req \ -config <(echo -e "$ca_config") \ -x509 \ -new \ -nodes \ -key $root_key \ -sha256 \ -subj "$SUBJ" \ -out $root_crt ## Server ###################################################################### # create a new server key and certificate signing request echo "[Creating server key and signing request]" openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $server_key -out $server_name.csr -newkey rsa:2048 -subj "$SUBJ" # sign the request with our root cert key echo "[Sign server cert]" openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $server_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $server_crt -days $expiration_days # write out server key in pkcs8 format, required by grpc echo "[Convert server key to pkcs8]" cp $server_key $server_key.tmp openssl pkcs8 -topk8 -nocrypt -in $server_key.tmp -out $server_key ## Client ###################################################################### # create a new server key and certificate signing request echo "[Creating client key and signing request]" openssl req -config <(echo -e "$derived_config") -new -nodes -sha256 -keyout $client_key -out $client_name.csr -newkey rsa:2048 -subj "$SUBJ" # sign the request with our root cert key echo "[Sign client cert]" openssl x509 -req -sha256 -extfile <(echo -e "$derived_config") -extensions x509_ext -in $client_name.csr -CA $root_crt -CAkey $root_key -CAcreateserial -out $client_crt -days $expiration_days # write out client key in pkcs8 format, required by grpc echo "[Convert client key to pkcs8]" cp $client_key $client_key.tmp openssl pkcs8 -topk8 -nocrypt -in $client_key.tmp -out $client_key # Clean up rm *.tmp rm *.csr rm *.srl ``` </details> ```sh # Using the above gen_mtls_test_files.sh gen_mtls_test_files.sh # Build it go build . # Boot the server with mTLS enabled OLLAMA_HOST="https://localhost:54321" \ OLLAMA_TLS_SERVER_KEY=server.key.pem \ OLLAMA_TLS_SERVER_CERT=server.cert.pem \ OLLAMA_TLS_CLIENT_CA=ca.cert.pem \ ./ollama serve # Run a client command with mTLS enabled (separate terminal or background the server) OLLAMA_HOST="https://127.0.0.1:54321" \ OLLAMA_TLS_SERVER_CA=ca.cert.pem \ OLLAMA_TLS_CLIENT_KEY=client.key.pem \ OLLAMA_TLS_CLIENT_CERT=client.cert.pem \ ./ollama ls ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-19 16:23:12 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#22501
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?