HTTP 400 in fresh nginx reverse proxy deployment #12462

Closed
opened 2025-11-02 10:10:32 -06:00 by GiteaMirror · 8 comments
Owner

Originally created by @bavarialogy on GitHub (Feb 9, 2024).

Description

Hello there,
I'm trying to finish my second deployment of Gitea 1.21.4 using docker-compose in combination with nginx-proxy by nginxproxy. This is my compose file for gitea:

`version: "3"

volumes:
gitea-data:
gitea-db:

services:
server:
image: gitea/gitea:1.21.4
environment:
- GITEA__database__DB_TYPE=postgres
- GITEA__database__HOST=db:5432
- GITEA__database__USER=gitea
- GITEA__database__PASSWD=supersecret
- VIRTUAL_HOST=git.domain.tld
- VIRTUAL_PORT=3000
- LETSENCRYPT_HOST=git.domain.tld
- LETSENCRYPT_EMAIL=push@domain.tld
restart: unless-stopped
networks:
- nginx-proxy-backend
- gitea-internal
volumes:
- gitea-data:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
depends_on:
- db
db:
image: postgres:14
restart: unless-stopped
environment:
- POSTGRES_USER=gitea
- POSTGRES_PASSWORD=supersecret
- POSTGRES_DB=gitea
networks:
- gitea-internal
volumes:
- gitea-db:/var/lib/postgresql/data

networks:
gitea-internal:
nginx-proxy-backend:
external: true
name: nginx-proxy-backend`

This is the corresponding nginxproxy/nginx-proxy docker-compose.yml file (for reference):

`version: '2'

volumes:
vhostd:
acme:
certs:
nginxhtml:
dhparam:

networks:
nginx-proxy-backend:
external: true
name: nginx-proxy-backend

services:
nginx-proxy:
restart: always
image: nginxproxy/nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- ./customconf:/etc/nginx/conf.d
- "/var/run/docker.sock:/tmp/docker.sock:ro"
- vhostd:/etc/nginx/vhost.d
- nginxhtml:/usr/share/nginx/html
- certs:/etc/nginx/certs:ro
- dhparam:/etc/nginx/dhparam
networks:
- nginx-proxy-backend
logging:
driver: "json-file"
options:
max-size: "10240m"

letsencrypt-nginx-proxy-companion:
restart: always
image: nginxproxy/acme-companion
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- certs:/etc/nginx/certs:rw
- acme:/etc/acme.sh
volumes_from:
- "nginx-proxy"
logging:
driver: "json-file"
options:
max-size: "10240m"`

The reverse proxy config that is generated by nginx-proxy looks like this:

# git.domain.tld/ upstream git.domain.tld { # Container: domaingit-server-1 # networks: # domaingit_gitea-internal (unreachable) # nginx-proxy-backend (reachable) # IP address: 172.16.128.4 # exposed ports: 22/tcp 3000/tcp # default port: 80 # using port: 3000 server 172.16.128.4:3000; } server { server_name git.domain.tld; access_log /var/log/nginx/access.log vhost; listen 80 ; # Do not HTTPS redirect Let's Encrypt ACME challenge location ^~ /.well-known/acme-challenge/ { auth_basic off; auth_request off; allow all; root /usr/share/nginx/html; try_files $uri =404; break; } location / { return 301 https://$host$request_uri; } } server { server_name git.domain.tld; access_log /var/log/nginx/access.log vhost; http2 on; listen 443 ssl ; ssl_session_timeout 5m; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_certificate /etc/nginx/certs/git.domain.tld.crt; ssl_certificate_key /etc/nginx/certs/git.domain.tld.key; ssl_dhparam /etc/nginx/certs/git.domain.tld.dhparam.pem; ssl_stapling on; ssl_stapling_verify on; ssl_trusted_certificate /etc/nginx/certs/git.domain.tld.chain.pem; set $sts_header ""; if ($https) { set $sts_header "max-age=31536000"; } add_header Strict-Transport-Security $sts_header always; include /etc/nginx/vhost.d/default; location / { proxy_pass http://git.domain.tld; set $upstream_keepalive false; }

I have tried turning logging up to "trace" in the app.ini file, but got no information. I even strace'd the nginx-worker and got this:

strace: Process 2037606 attached epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 61826) = 1 read(3, "\27\3\3\0000\345@\272y\264\206(\10\243\30\37\225\r&2\36/b\26!F\17\2Y\221\7\364"..., 16709) = 53 read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) getsockname(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("172.16.128.2")}, [112->16]) = 0 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 15 ioctl(15, FIONBIO, [1]) = 0 epoll_ctl(42, EPOLL_CTL_ADD, 15, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=155833576, u64=139771276547304}}) = 0 connect(15, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("172.16.128.4")}, 16) = -1 EINPROGRESS (Operation now in progress) epoll_wait(42, [{EPOLLOUT, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 getsockopt(15, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 writev(15, [{iov_base="GET / HTTP/1.1\r\nHost: git.domain"..., iov_len=1215}], 1) = 1215 epoll_wait(42, [{EPOLLIN|EPOLLOUT, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 recvfrom(15, "HTTP/1.1 400 Bad Request\r\nConten"..., 4096, 0, NULL, NULL) = 103 epoll_wait(42, [{EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 readv(15, [{iov_base="", iov_len=3993}], 1) = 0 close(15) = 0 write(3, "\27\3\3\0\212\34\273\246\241\213b<\326\274\0337\37\3103#\2\315!\336\346|\341wd\210\202\33"..., 143) = 143 write(12, "git.domain.tld 84.160.71.23 "..., 227) = 227 epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 65000) = 1 read(3, "\27\3\3\0\36H>'\303\341\t\303\r\260eU\340\343\260\374pdu\361\365\31\354\f\271\303\217\311"..., 16709) = 35 read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 64981) = 1 read(3, "\27\3\3\08\342\364)\268GJ\346\344d\320T\326YWk\225\360\344\222\211xh\340\215g\350"..., 16709) = 61 read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) getsockname(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("172.16.128.2")}, [112->16]) = 0 socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 15 ioctl(15, FIONBIO, [1]) = 0 epoll_ctl(42, EPOLL_CTL_ADD, 15, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=155833577, u64=139771276547305}}) = 0 connect(15, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("172.16.128.4")}, 16) = -1 EINPROGRESS (Operation now in progress) epoll_wait(42, [{EPOLLOUT, {u32=155833577, u64=139771276547305}}], 512, 360000) = 1 getsockopt(15, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 writev(15, [{iov_base="GET /favicon.ico HTTP/1.1\r\nHost:"..., iov_len=1153}], 1) = 1153 epoll_wait(42, [{EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=155833577, u64=139771276547305}}], 512, 360000) = 1 recvfrom(15, "HTTP/1.1 400 Bad Request\r\nConten"..., 4096, 0, NULL, NULL) = 103 readv(15, [{iov_base="", iov_len=3993}], 1) = 0 close(15) = 0 write(3, "\27\3\3\0\212\215fM\220\266x\240\271\202\357\0303\21S\207<8\t~\227\323\210e\275\3312."..., 143) = 143 write(12, "git.domain.tld 84.160.71.23 "..., 264) = 264 epoll_wait(42, ^Cstrace: Process 2037606 detached <detached ...>

I've spent a lot of time trying to debug the situation but I'm at the end. Does anyone have an idea what could be the problem?

Gitea Version

1.21.4

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

Linux/Docker

How are you running Gitea?

Docker (w/ compose)

Database

PostgreSQL

Originally created by @bavarialogy on GitHub (Feb 9, 2024). ### Description Hello there, I'm trying to finish my second deployment of Gitea 1.21.4 using docker-compose in combination with nginx-proxy by nginxproxy. This is my compose file for gitea: > `version: "3" > > volumes: > gitea-data: > gitea-db: > > services: > server: > image: gitea/gitea:1.21.4 > environment: > - GITEA__database__DB_TYPE=postgres > - GITEA__database__HOST=db:5432 > - GITEA__database__USER=gitea > - GITEA__database__PASSWD=supersecret > - VIRTUAL_HOST=git.domain.tld > - VIRTUAL_PORT=3000 > - LETSENCRYPT_HOST=git.domain.tld > - LETSENCRYPT_EMAIL=push@domain.tld > restart: unless-stopped > networks: > - nginx-proxy-backend > - gitea-internal > volumes: > - gitea-data:/data > - /etc/timezone:/etc/timezone:ro > - /etc/localtime:/etc/localtime:ro > depends_on: > - db > db: > image: postgres:14 > restart: unless-stopped > environment: > - POSTGRES_USER=gitea > - POSTGRES_PASSWORD=supersecret > - POSTGRES_DB=gitea > networks: > - gitea-internal > volumes: > - gitea-db:/var/lib/postgresql/data > > networks: > gitea-internal: > nginx-proxy-backend: > external: true > name: nginx-proxy-backend` This is the corresponding nginxproxy/nginx-proxy docker-compose.yml file (for reference): > `version: '2' > > volumes: > vhostd: > acme: > certs: > nginxhtml: > dhparam: > > networks: > nginx-proxy-backend: > external: true > name: nginx-proxy-backend > > services: > nginx-proxy: > restart: always > image: nginxproxy/nginx-proxy > ports: > - "80:80" > - "443:443" > volumes: > - ./customconf:/etc/nginx/conf.d > - "/var/run/docker.sock:/tmp/docker.sock:ro" > - vhostd:/etc/nginx/vhost.d > - nginxhtml:/usr/share/nginx/html > - certs:/etc/nginx/certs:ro > - dhparam:/etc/nginx/dhparam > networks: > - nginx-proxy-backend > logging: > driver: "json-file" > options: > max-size: "10240m" > > letsencrypt-nginx-proxy-companion: > restart: always > image: nginxproxy/acme-companion > volumes: > - "/var/run/docker.sock:/var/run/docker.sock:ro" > - certs:/etc/nginx/certs:rw > - acme:/etc/acme.sh > volumes_from: > - "nginx-proxy" > logging: > driver: "json-file" > options: > max-size: "10240m"` The reverse proxy config that is generated by nginx-proxy looks like this: > `# git.domain.tld/ > upstream git.domain.tld { > # Container: domaingit-server-1 > # networks: > # domaingit_gitea-internal (unreachable) > # nginx-proxy-backend (reachable) > # IP address: 172.16.128.4 > # exposed ports: 22/tcp 3000/tcp > # default port: 80 > # using port: 3000 > server 172.16.128.4:3000; > } > server { > server_name git.domain.tld; > access_log /var/log/nginx/access.log vhost; > listen 80 ; > # Do not HTTPS redirect Let's Encrypt ACME challenge > location ^~ /.well-known/acme-challenge/ { > auth_basic off; > auth_request off; > allow all; > root /usr/share/nginx/html; > try_files $uri =404; > break; > } > location / { > return 301 https://$host$request_uri; > } > } > server { > server_name git.domain.tld; > access_log /var/log/nginx/access.log vhost; > http2 on; > listen 443 ssl ; > ssl_session_timeout 5m; > ssl_session_cache shared:SSL:50m; > ssl_session_tickets off; > ssl_certificate /etc/nginx/certs/git.domain.tld.crt; > ssl_certificate_key /etc/nginx/certs/git.domain.tld.key; > ssl_dhparam /etc/nginx/certs/git.domain.tld.dhparam.pem; > ssl_stapling on; > ssl_stapling_verify on; > ssl_trusted_certificate /etc/nginx/certs/git.domain.tld.chain.pem; > set $sts_header ""; > if ($https) { > set $sts_header "max-age=31536000"; > } > add_header Strict-Transport-Security $sts_header always; > include /etc/nginx/vhost.d/default; > location / { > proxy_pass http://git.domain.tld; > set $upstream_keepalive false; > }` I have tried turning logging up to "trace" in the app.ini file, but got no information. I even strace'd the nginx-worker and got this: > `strace: Process 2037606 attached > epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 61826) = 1 > read(3, "\27\3\3\0000\345@\272y\264\206(\10\243\30\37\225\r&2\36/b\26!F\17\2Y\221\7\364"..., 16709) = 53 > read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) > getsockname(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("172.16.128.2")}, [112->16]) = 0 > socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 15 > ioctl(15, FIONBIO, [1]) = 0 > epoll_ctl(42, EPOLL_CTL_ADD, 15, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=155833576, u64=139771276547304}}) = 0 > connect(15, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("172.16.128.4")}, 16) = -1 EINPROGRESS (Operation now in progress) > epoll_wait(42, [{EPOLLOUT, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 > getsockopt(15, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 > writev(15, [{iov_base="GET / HTTP/1.1\r\nHost: git.domain"..., iov_len=1215}], 1) = 1215 > epoll_wait(42, [{EPOLLIN|EPOLLOUT, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 > recvfrom(15, "HTTP/1.1 400 Bad Request\r\nConten"..., 4096, 0, NULL, NULL) = 103 > epoll_wait(42, [{EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=155833576, u64=139771276547304}}], 512, 360000) = 1 > readv(15, [{iov_base="", iov_len=3993}], 1) = 0 > close(15) = 0 > write(3, "\27\3\3\0\212\34\273\246\241\213b<\326\274\0337\37\3103#\2\315!\336\346|\341wd\210\202\33"..., 143) = 143 > write(12, "git.domain.tld 84.160.71.23 "..., 227) = 227 > epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 65000) = 1 > read(3, "\27\3\3\0\36H>'\303\341\t\303\r\260eU\340\343\260\374pdu\361\365\31\354\f\271\303\217\311"..., 16709) = 35 > read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) > epoll_wait(42, [{EPOLLIN, {u32=155833080, u64=139771276546808}}], 512, 64981) = 1 > read(3, "\27\3\3\08\342\364)\268GJ\346\344d\320T\326YWk\225\360\344\222\211xh\340\215g\350"..., 16709) = 61 > read(3, 0x5583fdfb7ae3, 16709) = -1 EAGAIN (Resource temporarily unavailable) > getsockname(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("172.16.128.2")}, [112->16]) = 0 > socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 15 > ioctl(15, FIONBIO, [1]) = 0 > epoll_ctl(42, EPOLL_CTL_ADD, 15, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=155833577, u64=139771276547305}}) = 0 > connect(15, {sa_family=AF_INET, sin_port=htons(3000), sin_addr=inet_addr("172.16.128.4")}, 16) = -1 EINPROGRESS (Operation now in progress) > epoll_wait(42, [{EPOLLOUT, {u32=155833577, u64=139771276547305}}], 512, 360000) = 1 > getsockopt(15, SOL_SOCKET, SO_ERROR, [0], [4]) = 0 > writev(15, [{iov_base="GET /favicon.ico HTTP/1.1\r\nHost:"..., iov_len=1153}], 1) = 1153 > epoll_wait(42, [{EPOLLIN|EPOLLOUT|EPOLLRDHUP, {u32=155833577, u64=139771276547305}}], 512, 360000) = 1 > recvfrom(15, "HTTP/1.1 400 Bad Request\r\nConten"..., 4096, 0, NULL, NULL) = 103 > readv(15, [{iov_base="", iov_len=3993}], 1) = 0 > close(15) = 0 > write(3, "\27\3\3\0\212\215fM\220\266x\240\271\202\357\0303\21S\207<8\t~\227\323\210e\275\3312."..., 143) = 143 > write(12, "git.domain.tld 84.160.71.23 "..., 264) = 264 > epoll_wait(42, ^Cstrace: Process 2037606 detached > <detached ...>` I've spent a lot of time trying to debug the situation but I'm at the end. Does anyone have an idea what could be the problem? ### Gitea Version 1.21.4 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System Linux/Docker ### How are you running Gitea? Docker (w/ compose) ### Database PostgreSQL
GiteaMirror added the issue/needs-feedback label 2025-11-02 10:10:32 -06:00
Author
Owner

@davegermiquet commented on GitHub (Feb 11, 2024):

Hi,

I'm new here and I'd love to try to help. This issue might be a configuration issue, not a bug in gitea.

Do you have logs?

I'd first try running docker logs on the individual components.

But even before that I'd try to connect through it without letsencrypt.

So First thing:

  • Does gitea boot up properly on port 3000?

  • Can you reverse proxy on regular http port without lets encrypt?

  • Whats gitea logs showing if its not starting up?

  • User the docker compose logs command to see whats happening in the individual instances.

@davegermiquet commented on GitHub (Feb 11, 2024): Hi, I'm new here and I'd love to try to help. This issue might be a configuration issue, not a bug in gitea. Do you have logs? I'd first try running docker logs on the individual components. But even before that I'd try to connect through it without letsencrypt. So First thing: - Does gitea boot up properly on port 3000? - Can you reverse proxy on regular http port without lets encrypt? - Whats gitea logs showing if its not starting up? - User the docker compose logs command to see whats happening in the individual instances.
Author
Owner

@bavarialogy commented on GitHub (Feb 12, 2024):

Hi,

thanks for your reply! I appreciate your will to help me out. The bad thing is that there isn't any relevant information on request handling logged by the gitea app server (not even with log set to "trace"). The gitea log does not show anything new once I follow it and reproduce the HTTP 400 error. That's why I created the strace of nginx, to see if the 400 comes from the application server (which it apparently does).

Direct access to gitea on port 3000 works through HTTP. Even curl'ing the application server from the reverse proxy container using the address from the reverse proxy config works, I get the gitea setup page on the fresh install and, once that has been completed using direct access (via tcp/3000), the login page. All of that, as mentioned, from inside the nginx container. From what I interpret out of the strace output (which is from the nginx worker process), nginx connects to the application server and gets back the 400 (which it then forwards to the user)..

I do agree that my issue is most likely a configuration issue, will remain thankful for any hints.

@bavarialogy commented on GitHub (Feb 12, 2024): Hi, thanks for your reply! I appreciate your will to help me out. The bad thing is that there isn't any relevant information on request handling logged by the gitea app server (not even with log set to "trace"). The gitea log does not show anything new once I follow it and reproduce the HTTP 400 error. That's why I created the strace of nginx, to see if the 400 comes from the application server (which it apparently does). Direct access to gitea on port 3000 works through HTTP. Even curl'ing the application server from the reverse proxy container using the address from the reverse proxy config works, I get the gitea setup page on the fresh install and, once that has been completed using direct access (via tcp/3000), the login page. All of that, as mentioned, from inside the nginx container. From what I interpret out of the strace output (which is from the nginx worker process), nginx connects to the application server and gets back the 400 (which it then forwards to the user).. I do agree that my issue is most likely a configuration issue, will remain thankful for any hints.
Author
Owner

@davegermiquet commented on GitHub (Feb 12, 2024):

Hi,

When I get home I'll do a ngrep/tcpdump for localhost to see and compare traffic. What headers are being sent over to the HTTP to port 3000. You'll need to filter only the nginx-proxy data and see what traffic is being sent over by nginx.

I'd suggest for you to do that as well, see whats being sent for that bad request to happen over tcpdump/ngrep i'll give you the exact commands later if you don't know how to do it in the evening.

@davegermiquet commented on GitHub (Feb 12, 2024): Hi, When I get home I'll do a ngrep/tcpdump for localhost to see and compare traffic. What headers are being sent over to the HTTP to port 3000. You'll need to filter only the nginx-proxy data and see what traffic is being sent over by nginx. I'd suggest for you to do that as well, see whats being sent for that bad request to happen over tcpdump/ngrep i'll give you the exact commands later if you don't know how to do it in the evening.
Author
Owner

@davegermiquet commented on GitHub (Feb 13, 2024):

In order to sniff the traffic you need to do the following:

this is base on configuration you sent:

Install the ngrep package on gitea:

docker compose exec server /bin/bash
apk add ngrep
ping nginx-proxy 

(you'll get an ip)
example:

64 bytes from 192.168.129.4: seq=0 ttl=64 time=0.069 ms

ifconfig -a

look at the interface that the ip is on

(if its eth0) probably is

for the command below you might want to add 404 or 400 as well to see the error condition.

Try OK first.

ngrep -d eth0  -q 'GET|HTTP/1.1|OK' port 3000

you'll see traffic like this:

interface: eth0 (192.168.129.0/255.255.240.0)
filter: ( port 3000 ) and ((ip || ip6) || (vlan && (ip || ip6)))
match: GET|HTTP/1.1|OK

T 192.168.129.4:54674 -> 192.168.129.2:3000 [AP] #4
  GET / HTTP/1.0..HOST: git.redacted.net..X-Forwarded-Proto: https..X-Real-IP: XX.XXX.XXX.XXX..X-Forwarded-For: XX.XXX.XXX.XXX..Connection: close..Cache-Control: max-age=0..sec-ch-ua: "Not A(Brand";v="99", "Google Chrome";v="121
  ", "Chromium";v="121"..sec-ch-ua-mobile: ?0..sec-ch-ua-platform: "Linux"..Upgrade-Insecure-Requests: 1..User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..Accept: text
  /html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7..Sec-Fetch-Site: none..Sec-Fetch-Mode: navigate..Sec-Fetch-User: ?1..Sec-Fetch-Dest: document..Acce
  pt-Encoding: gzip, deflate, br..Accept-Language: en-GB,en-US;q=0.9,en;q=0.8..Cookie: i_like_gitea=9e31d894655c1f76; _csrf=i99JEgCzM1_8WZTOlRekZJ9uLC86MTcwNzc4NDIzNDc1MjA5NDk2NA....                                                   

T 192.168.129.2:3000 -> 192.168.129.4:54674 [AP] #6
  HTTP/1.0 200 OK..Cache-Control: max-age=0, private, must-revalidate, no-transform..Content-Type: text/html; charset=utf-8..X-Frame-Options: SAMEORIGIN..Date: Tue, 13 Feb 2024 00:36:16 GMT....<!DOCTYPE html>.<html lang="en-US" class
  ="theme-auto">.<head>..<meta name="viewport" content="width=device-width, initial-scale=1">..<title>Gitea: Git with a cup of tea</title>..<link rel="manifest" href="data:application/json;base64,eyJuYW1lIjoiR2l0ZWE6IEdpdCB3aXRoIGEgY

You can then see what headers are being passed if any are missing and what actual gitea is returning back.

There are other ways to get the ip and interface this is just an example.

@davegermiquet commented on GitHub (Feb 13, 2024): In order to sniff the traffic you need to do the following: this is base on configuration you sent: Install the ngrep package on gitea: ``` docker compose exec server /bin/bash apk add ngrep ping nginx-proxy ``` (*you'll get an ip*) example: 64 bytes from 192.168.129.4: seq=0 ttl=64 time=0.069 ms ``` ifconfig -a ``` look at the interface that the ip is on (if its eth0) probably is for the command below you might want to add 404 or 400 as well to see the error condition. Try OK first. ``` ngrep -d eth0 -q 'GET|HTTP/1.1|OK' port 3000 ``` you'll see traffic like this: ```ngrep -d eth0 -q 'GET|HTTP/1.1|OK' port 3000 interface: eth0 (192.168.129.0/255.255.240.0) filter: ( port 3000 ) and ((ip || ip6) || (vlan && (ip || ip6))) match: GET|HTTP/1.1|OK T 192.168.129.4:54674 -> 192.168.129.2:3000 [AP] #4 GET / HTTP/1.0..HOST: git.redacted.net..X-Forwarded-Proto: https..X-Real-IP: XX.XXX.XXX.XXX..X-Forwarded-For: XX.XXX.XXX.XXX..Connection: close..Cache-Control: max-age=0..sec-ch-ua: "Not A(Brand";v="99", "Google Chrome";v="121 ", "Chromium";v="121"..sec-ch-ua-mobile: ?0..sec-ch-ua-platform: "Linux"..Upgrade-Insecure-Requests: 1..User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..Accept: text /html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7..Sec-Fetch-Site: none..Sec-Fetch-Mode: navigate..Sec-Fetch-User: ?1..Sec-Fetch-Dest: document..Acce pt-Encoding: gzip, deflate, br..Accept-Language: en-GB,en-US;q=0.9,en;q=0.8..Cookie: i_like_gitea=9e31d894655c1f76; _csrf=i99JEgCzM1_8WZTOlRekZJ9uLC86MTcwNzc4NDIzNDc1MjA5NDk2NA.... T 192.168.129.2:3000 -> 192.168.129.4:54674 [AP] #6 HTTP/1.0 200 OK..Cache-Control: max-age=0, private, must-revalidate, no-transform..Content-Type: text/html; charset=utf-8..X-Frame-Options: SAMEORIGIN..Date: Tue, 13 Feb 2024 00:36:16 GMT....<!DOCTYPE html>.<html lang="en-US" class ="theme-auto">.<head>..<meta name="viewport" content="width=device-width, initial-scale=1">..<title>Gitea: Git with a cup of tea</title>..<link rel="manifest" href="data:application/json;base64,eyJuYW1lIjoiR2l0ZWE6IEdpdCB3aXRoIGEgY ``` You can then see what headers are being passed if any are missing and what actual gitea is returning back. There are other ways to get the ip and interface this is just an example.
Author
Owner

@bavarialogy commented on GitHub (Feb 16, 2024):

Hi there,

i'm pretty impressed by the level of detail you're providing to help me. THANK YOU!

This is what I got going with the steps:

d2b72772e3a9:/# ngrep -d eth1 -q 'GET|HTTP1.1|OK' port 3000
interface: eth1 (172.16.128.0/255.255.240.0)
filter: ( port 3000 ) and ((ip || ip6) || (vlan && (ip || ip6)))
match: GET|HTTP1.1|OK

T 172.16.128.2:53424 -> 172.16.128.4:3000 [AP] #4
GET / HTTP/1.1..Host: git.domain.tld..Connection: close..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Host: git.domain.tld..X-Forwarded-Proto: h
ttps..X-Forwarded-Ssl: on..X-Forwarded-Port: 443..X-Original-URI: /..Host: git.domain.tld..X-Original-URL: https://git.domain.tld/..X-Forwarded-Proto: https..X-Forward
ed-Host: git.domain.tld..X-Forwarded-Uri: /..X-Forwarded-Ssl: on..X-Forwarded-For: REDACTED-REAL-IP..X-Real-IP: REDACTED-REAL-IP..Host: git.domain.tld..X-Real-IP: REDACTED-REAL-IP
..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Proto: https..cache-control: max-age=0..upgrade-insecure-requests: 1..user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleW
ebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,appli
cation/signed-exchange;v=b3;q=0.7..sec-fetch-site: none..sec-fetch-mode: navigate..sec-fetch-user: ?1..sec-fetch-dest: document..sec-ch-ua: "Not A(Brand";v="99", "Google Chrom
e";v="121", "Chromium";v="121"..sec-ch-ua-mobile: ?0..sec-ch-ua-platform: "Windows"..accept-encoding: gzip, deflate, br..accept-language: en-US,en;q=0.9,de;q=0.8....

T 172.16.128.2:53428 -> 172.16.128.4:3000 [AP] #14
GET /favicon.ico HTTP/1.1..Host: git.domain.tld..Connection: close..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Host: git.domain.tld..X-Forward
ed-Proto: https..X-Forwarded-Ssl: on..X-Forwarded-Port: 443..X-Original-URI: /favicon.ico..Host: git.domain.tld..X-Original-URL: https://git.domain.tld/favicon.ico..X-
Forwarded-Proto: https..X-Forwarded-Host: git.domain.tld..X-Forwarded-Uri: /favicon.ico..X-Forwarded-Ssl: on..X-Forwarded-For: REDACTED-REAL-IP..X-Real-IP: REDACTED-REAL-IP..Host:
git.domain.tld..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Proto: https..sec-ch-ua: "Not A(Brand";v="99", "Google Chrome";v="121", "Chromium";v="
121"..sec-ch-ua-mobile: ?0..user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..sec-ch-ua-platform: "W
indows"..accept: image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8..sec-fetch-site: same-origin..sec-fetch-mode: no-cors..sec-fetch-dest: image..referer: https:
//git.domain.tld/..accept-encoding: gzip, deflate, br..accept-language: en-US,en;q=0.9,de;q=0.8....

I'm not experienced enough to tell if there are headers missing, I can only see that some seem to be double. Could this be a problem?

@bavarialogy commented on GitHub (Feb 16, 2024): Hi there, i'm pretty impressed by the level of detail you're providing to help me. THANK YOU! This is what I got going with the steps: > d2b72772e3a9:/# ngrep -d eth1 -q 'GET|HTTP1.1|OK' port 3000 > interface: eth1 (172.16.128.0/255.255.240.0) > filter: ( port 3000 ) and ((ip || ip6) || (vlan && (ip || ip6))) > match: GET|HTTP1.1|OK > > T 172.16.128.2:53424 -> 172.16.128.4:3000 [AP] #4 > GET / HTTP/1.1..Host: git.domain.tld..Connection: close..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Host: git.domain.tld..X-Forwarded-Proto: h > ttps..X-Forwarded-Ssl: on..X-Forwarded-Port: 443..X-Original-URI: /..Host: git.domain.tld..X-Original-URL: https://git.domain.tld/..X-Forwarded-Proto: https..X-Forward > ed-Host: git.domain.tld..X-Forwarded-Uri: /..X-Forwarded-Ssl: on..X-Forwarded-For: REDACTED-REAL-IP..X-Real-IP: REDACTED-REAL-IP..Host: git.domain.tld..X-Real-IP: REDACTED-REAL-IP > ..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Proto: https..cache-control: max-age=0..upgrade-insecure-requests: 1..user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleW > ebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,appli > cation/signed-exchange;v=b3;q=0.7..sec-fetch-site: none..sec-fetch-mode: navigate..sec-fetch-user: ?1..sec-fetch-dest: document..sec-ch-ua: "Not A(Brand";v="99", "Google Chrom > e";v="121", "Chromium";v="121"..sec-ch-ua-mobile: ?0..sec-ch-ua-platform: "Windows"..accept-encoding: gzip, deflate, br..accept-language: en-US,en;q=0.9,de;q=0.8.... > > T 172.16.128.2:53428 -> 172.16.128.4:3000 [AP] #14 > GET /favicon.ico HTTP/1.1..Host: git.domain.tld..Connection: close..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Host: git.domain.tld..X-Forward > ed-Proto: https..X-Forwarded-Ssl: on..X-Forwarded-Port: 443..X-Original-URI: /favicon.ico..Host: git.domain.tld..X-Original-URL: https://git.domain.tld/favicon.ico..X- > Forwarded-Proto: https..X-Forwarded-Host: git.domain.tld..X-Forwarded-Uri: /favicon.ico..X-Forwarded-Ssl: on..X-Forwarded-For: REDACTED-REAL-IP..X-Real-IP: REDACTED-REAL-IP..Host: > git.domain.tld..X-Real-IP: REDACTED-REAL-IP..X-Forwarded-For: REDACTED-REAL-IP..X-Forwarded-Proto: https..sec-ch-ua: "Not A(Brand";v="99", "Google Chrome";v="121", "Chromium";v=" > 121"..sec-ch-ua-mobile: ?0..user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36..sec-ch-ua-platform: "W > indows"..accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8..sec-fetch-site: same-origin..sec-fetch-mode: no-cors..sec-fetch-dest: image..referer: https: > //git.domain.tld/..accept-encoding: gzip, deflate, br..accept-language: en-US,en;q=0.9,de;q=0.8.... > I'm not experienced enough to tell if there are headers missing, I can only see that some seem to be double. Could this be a problem?
Author
Owner

@davegermiquet commented on GitHub (Feb 16, 2024):

Can you try this ngrep instead of the previous? To see if we can see what the bad request is as well:

ngrep -d eth1  -q 'GET|HTTP/1.1|200|400' port 3000

Let me see those headers .I'll look at them.

Maybe gitea isn't listening to that port. Can you try this:

change in your nginx config to point to the internal name instead of the ip?:

Example in my config i've got this:

server {
    # Listen for requests on your domain/IP address.
    listen       443 ssl;
    ssl_certificate /etc/davessl/fullchain.pem;
    ssl_certificate_key /etc/davessl/privkey.pem;
    server_name git.redacted.net;

    root /var/www/html;

    location / {
        # Proxy all requests to Gitea running on port 3000
        proxy_pass http://server:3000;
        # Pass on information about the requests to the proxied service using headers
        proxy_set_header HOST $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server:
image: gitea/gitea:latest
environment:
- USER_UID=1000
- USER_GID=1000
......
Where my docker compose looks like this:

I've seen this issue at work with my colleague, and when i changed the dns name it was locating to (LOCALHOST as he wasn't using docker it fixed the issue)

@davegermiquet commented on GitHub (Feb 16, 2024): Can you try this ngrep instead of the previous? To see if we can see what the bad request is as well: ``` ngrep -d eth1 -q 'GET|HTTP/1.1|200|400' port 3000 ``` Let me see those headers .I'll look at them. Maybe gitea isn't listening to that port. Can you try this: change in your nginx config to point to the internal name instead of the ip?: Example in my config i've got this: ``` server { # Listen for requests on your domain/IP address. listen 443 ssl; ssl_certificate /etc/davessl/fullchain.pem; ssl_certificate_key /etc/davessl/privkey.pem; server_name git.redacted.net; root /var/www/html; location / { # Proxy all requests to Gitea running on port 3000 proxy_pass http://server:3000; # Pass on information about the requests to the proxied service using headers proxy_set_header HOST $host; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } ``` server: image: gitea/gitea:latest environment: - USER_UID=1000 - USER_GID=1000 ...... Where my docker compose looks like this: I've seen this issue at work with my colleague, and when i changed the dns name it was locating to (LOCALHOST as he wasn't using docker it fixed the issue)
Author
Owner

@davegermiquet commented on GitHub (Feb 16, 2024):

You mentioned you see no logs, have you tried this:

docker compose logs server

this is where the logs come out of.

You need to be in the folder where docker-compose.yml is.

Another example you can do is before trying to log in do:

docker compose logs server -f

I think this line is the problem:

proxy_pass http://git.domain.tld;

Is that going to the Public IP?

id modify it and change it to the local ip. I guess i said it above change it to the name of the service in docker-compose.yml

@davegermiquet commented on GitHub (Feb 16, 2024): You mentioned you see no logs, have you tried this: docker compose logs server this is where the logs come out of. You need to be in the folder where docker-compose.yml is. Another example you can do is before trying to log in do: docker compose logs server -f I think this line is the problem: proxy_pass http://git.domain.tld; Is that going to the Public IP? id modify it and change it to the local ip. I guess i said it above change it to the name of the service in docker-compose.yml
Author
Owner

@GiteaBot commented on GitHub (Mar 28, 2024):

We close issues that need feedback from the author if there were no new comments for a month. 🍵

@GiteaBot commented on GitHub (Mar 28, 2024): We close issues that need feedback from the author if there were no new comments for a month. :tea:
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#12462