currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css"
(an example path that would be accepted (and therefore disallowed on scrapers) by this regex would be https://cobalt.tools/icons/ /fonts/ bla.js .css, which is obviously nonsense & useless)
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/imputnet/cobalt/pull/242
**Author:** [@dumbmoron](https://github.com/dumbmoron)
**Created:** 11/10/2023
**Status:** ✅ Merged
**Merged:** 11/11/2023
**Merged by:** [@wukko](https://github.com/wukko)
**Base:** `current` ← **Head:** `patch-1`
---
### 📝 Commits (1)
- [`d936dd7`](https://github.com/imputnet/cobalt/commit/d936dd73fe50119dc6e486d63d583fcb64b91000) fix robots.txt
### 📊 Changes
**1 file changed** (+4 additions, -1 deletions)
<details>
<summary>View changed files</summary>
📝 `src/front/robots.txt` (+4 -1)
</details>
### 📄 Description
currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css"
(an example path that would be accepted (and therefore disallowed on scrapers) by this regex would be `https://cobalt.tools/icons/ /fonts/ bla.js .css`, which is obviously nonsense & useless)
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/imputnet/cobalt/pull/242
Author: @dumbmoron
Created: 11/10/2023
Status: ✅ Merged
Merged: 11/11/2023
Merged by: @wukko
Base:
current← Head:patch-1📝 Commits (1)
d936dd7fix robots.txt📊 Changes
1 file changed (+4 additions, -1 deletions)
View changed files
📝
src/front/robots.txt(+4 -1)📄 Description
currently the robots.txt file is useless because it's interpreted as one path "/icons/ /fonts/ *.js *.css"
(an example path that would be accepted (and therefore disallowed on scrapers) by this regex would be
https://cobalt.tools/icons/ /fonts/ bla.js .css, which is obviously nonsense & useless)🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.