[PR #242] [MERGED] fix robots.txt #5498

New Issue

GiteaMirror · 2026-04-16T00:49:51-05:00

GiteaMirror commented

2026-04-16 00:49:51 -05:00

📋 Pull Request Information

Original PR: https://github.com/imputnet/cobalt/pull/242
Author: @dumbmoron
Created: 11/10/2023
Status: ✅ Merged
Merged: 11/11/2023
Merged by: @wukko

Base: current ← Head: patch-1

📝 Commits (1)

d936dd7 fix robots.txt

📊 Changes

1 file changed (+4 additions, -1 deletions)

View changed files

📝 src/front/robots.txt (+4 -1)

📄 Description

currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css"

(an example path that would be accepted (and therefore disallowed on scrapers) by this regex would be https://cobalt.tools/icons/ /fonts/ bla.js .css, which is obviously nonsense & useless)

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/imputnet/cobalt/pull/242 **Author:** [@dumbmoron](https://github.com/dumbmoron) **Created:** 11/10/2023 **Status:** ✅ Merged **Merged:** 11/11/2023 **Merged by:** [@wukko](https://github.com/wukko) **Base:** `current` ← **Head:** `patch-1` --- ### 📝 Commits (1) - [`d936dd7`](https://github.com/imputnet/cobalt/commit/d936dd73fe50119dc6e486d63d583fcb64b91000) fix robots.txt ### 📊 Changes **1 file changed** (+4 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `src/front/robots.txt` (+4 -1) </details> ### 📄 Description currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css" (an example path that would be accepted (and therefore disallowed on scrapers) by this regex would be `https://cobalt.tools/icons/ /fonts/ bla.js .css`, which is obviously nonsense & useless) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-16 00:49:51 -05:00

GiteaMirror closed this issue

2026-04-16 00:49:51 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cobalt#5498