HEAD to /robots.txt returns 404 error #12885

New Issue

GiteaMirror · 2025-11-02T10:23:56-06:00

GiteaMirror commented

2025-11-02 10:23:56 -06:00

Originally created by @osfrickler on GitHub (Apr 19, 2024).

Description

When accessing the /robots.txt location with curl -I, a 404 error is returned. Instead a proper 200 response is expected. Other locations seem not to be affected.

$ curl -I https://try.gitea.io/robots.txt
HTTP/2 404 
alt-svc: h3=":443"; ma=2592000
cache-control: max-age=0, private, must-revalidate, no-transform
content-type: text/plain;charset=utf-8
date: Fri, 19 Apr 2024 12:24:58 GMT
server: Caddy
set-cookie: i_like_gitea=0b00704e9048a57e; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _csrf=6C6usmqRx9Q9v3UExz4aJICNcMU6MTcxMzUyOTQ5ODYwMDU0MTEzNg; Path=/; Max-Age=86400; HttpOnly; Secure; SameSite=Lax
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
content-length: 11

$ curl -s -i https://try.gitea.io/robots.txt | head
HTTP/2 200 
accept-ranges: bytes
alt-svc: h3=":443"; ma=2592000
cache-control: private, max-age=21600
content-type: text/plain; charset=utf-8
date: Fri, 19 Apr 2024 12:25:08 GMT
last-modified: Thu, 16 Sep 2021 04:52:52 GMT
server: Caddy
content-length: 108
...

Gitea Version

1.23.0+dev-172-geb24d973b0

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

using try.gitea.io, same issue on https://opendev.org

Database

MySQL/MariaDB

Originally created by @osfrickler on GitHub (Apr 19, 2024). ### Description When accessing the /robots.txt location with `curl -I`, a 404 error is returned. Instead a proper 200 response is expected. Other locations seem not to be affected. ``` $ curl -I https://try.gitea.io/robots.txt HTTP/2 404 alt-svc: h3=":443"; ma=2592000 cache-control: max-age=0, private, must-revalidate, no-transform content-type: text/plain;charset=utf-8 date: Fri, 19 Apr 2024 12:24:58 GMT server: Caddy set-cookie: i_like_gitea=0b00704e9048a57e; Path=/; HttpOnly; Secure; SameSite=Lax set-cookie: _csrf=6C6usmqRx9Q9v3UExz4aJICNcMU6MTcxMzUyOTQ5ODYwMDU0MTEzNg; Path=/; Max-Age=86400; HttpOnly; Secure; SameSite=Lax x-content-type-options: nosniff x-frame-options: SAMEORIGIN content-length: 11 $ curl -s -i https://try.gitea.io/robots.txt | head HTTP/2 200 accept-ranges: bytes alt-svc: h3=":443"; ma=2592000 cache-control: private, max-age=21600 content-type: text/plain; charset=utf-8 date: Fri, 19 Apr 2024 12:25:08 GMT last-modified: Thu, 16 Sep 2021 04:52:52 GMT server: Caddy content-length: 108 ... ``` ### Gitea Version 1.23.0+dev-172-geb24d973b0 ### Can you reproduce the bug on the Gitea demo site? Yes ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System _No response_ ### How are you running Gitea? using try.gitea.io, same issue on https://opendev.org ### Database MySQL/MariaDB

GiteaMirror added the type/bug label 2025-11-02 10:23:56 -06:00

GiteaMirror closed this issue

2025-11-02 10:23:58 -06:00

GiteaMirror commented

2025-11-02 10:23:59 -06:00

@wxiaoguang commented on GitHub (Apr 19, 2024):

-> Fix HEAD method for robots.txt #30603

@wxiaoguang commented on GitHub (Apr 19, 2024): -> Fix HEAD method for robots.txt #30603

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#12885