[Bug] Some symbols not encoded, stripped by browser on navigation #14287

New Issue

GiteaMirror · 2025-11-02T11:08:42-06:00

GiteaMirror commented

2025-11-02 11:08:42 -06:00

Originally created by @OdinVex on GitHub (Mar 21, 2025).

Description

I've got some files that require the @ symbol in their name. Attempting to browse to them via the web UI results in the browser stripping @ onward (trims entire string), resulting in a 404 unless I copy the link and use %40 to encode the URL, then it works.

Example File: example@example.ext becomes example when clicking on the link. The href and title are output raw with @ shown. Perhaps browsers might suspect it's an attempt to login or something.

Gitea Version

Forgejo

Can you reproduce the bug on the Gitea demo site?

Yes

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

Docker.

Database

MySQL/MariaDB

Originally created by @OdinVex on GitHub (Mar 21, 2025). ### Description I've got some files that require the `@` symbol in their name. Attempting to browse to them via the web UI results in the browser stripping `@` onward (trims entire string), resulting in a `404` unless I copy the link and use `%40` to encode the URL, then it works. Example File: `example@example.ext` becomes `example` **when clicking on the link**. The `href` and `title` are output raw with `@` shown. Perhaps browsers might suspect it's an attempt to login or something. ### Gitea Version Forgejo ### Can you reproduce the bug on the Gitea demo site? Yes ### Log Gist _No response_ ### Screenshots _No response_ ### Git Version _No response_ ### Operating System _No response_ ### How are you running Gitea? Docker. ### Database MySQL/MariaDB

GiteaMirror added the issue/needs-feedback label 2025-11-02 11:08:42 -06:00

GiteaMirror closed this issue

2025-11-02 11:08:42 -06:00

GiteaMirror commented

2025-11-02 11:08:43 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

What browser your are using? I can't reproduce it on my side.

@wxiaoguang commented on GitHub (Mar 21, 2025): What browser your are using? I can't reproduce it on my side. ![Image](https://github.com/user-attachments/assets/e9dc0974-553b-4d37-b70c-550edf436443)

GiteaMirror commented

2025-11-02 11:08:43 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

LibreWolf. Not sure if it matters, but it's in a subfolder.

@OdinVex commented on GitHub (Mar 21, 2025): LibreWolf. Not sure if it matters, but it's in a subfolder.

GiteaMirror commented

2025-11-02 11:08:43 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

There is no problem for sub folder, see the screenshot

And by the way, @ is a valid char for URL path, it doesn't need to be encoded, see:

https://stackoverflow.com/questions/4669692/valid-characters-for-directory-part-of-a-url-for-short-links

So maybe it is the browser's bug?

@wxiaoguang commented on GitHub (Mar 21, 2025): There is no problem for sub folder, see the screenshot <details> ![Image](https://github.com/user-attachments/assets/8f9b0976-524c-4534-9d31-a05cfc732413) </details> And by the way, `@` is a valid char for URL path, it doesn't need to be encoded, see: https://stackoverflow.com/questions/4669692/valid-characters-for-directory-part-of-a-url-for-short-links So maybe it is the browser's bug?

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

"Although these are 79 characters in total that can be used in a path segment literally, some user agents do encode some of these characters as well (e.g. %7E instead of ~). That’s why many use just the 62 alphanumeric characters (i.e. A–Z, a–z, 0–9) or the Base 64 Encoding with URL and Filename Safe Alphabet (i.e. A–Z, a–z, 0–9, -, _)." From the first 'answer' in that Stackoverflow link.

@OdinVex commented on GitHub (Mar 21, 2025): "Although these are 79 characters in total that can be used in a path segment literally, some user agents do encode some of these characters as well (e.g. %7E instead of ~). That’s why many use just the 62 alphanumeric characters (i.e. A–Z, a–z, 0–9) or the [Base 64 Encoding with URL and Filename Safe Alphabet](https://datatracker.ietf.org/doc/html/rfc4648#section-5) (i.e. A–Z, a–z, 0–9, -, _)." From the first 'answer' in that Stackoverflow link.

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

So that's the user agent (browser)'s bug, it should follow the standard to accept @

@wxiaoguang commented on GitHub (Mar 21, 2025): So that's the user agent (browser)'s bug, it should follow the standard to accept `@`

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

"Although these are 79 characters in total that can be used in a path segment literally, some user agents do encode some of these characters as well (e.g. %7E instead of ~). That’s why many use just the 62 alphanumeric characters (i.e. A–Z, a–z, 0–9) or the Base 64 Encoding with URL and Filename Safe Alphabet (i.e. A–Z, a–z, 0–9, -, _)." From the first 'answer' in that Stackoverflow link.

That answer just means: any char could be encoded, for example: a could also be encoded, and all user agents are able to decode it.

But it doesn't mean that a user agent should be unable to handle the standard chars like @.

@wxiaoguang commented on GitHub (Mar 21, 2025): > "Although these are 79 characters in total that can be used in a path segment literally, some user agents do encode some of these characters as well (e.g. %7E instead of ~). That’s why many use just the 62 alphanumeric characters (i.e. A–Z, a–z, 0–9) or the [Base 64 Encoding with URL and Filename Safe Alphabet](https://datatracker.ietf.org/doc/html/rfc4648#section-5) (i.e. A–Z, a–z, 0–9, -, _)." From the first 'answer' in that Stackoverflow link. That answer just means: any char could be encoded, for example: `a` could also be encoded, and all user agents are able to decode it. But it doesn't mean that a user agent should be unable to handle the standard chars like `@`.

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

So that's the user agent (browser)'s bug, it should follow the standard to accept @

https://www.rfc-editor.org/rfc/rfc3986.txt

      reserved    = gen-delims / sub-delims

      gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"

      sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
                  / "*" / "+" / "," / ";" / "="

   The purpose of reserved characters is to provide a set of delimiting
   characters that are distinguishable from other data within a URI.
   URIs that differ in the replacement of a reserved character with its
   corresponding percent-encoded octet are not equivalent.  Percent-
   encoding a reserved character, or decoding a percent-encoded octet
   that corresponds to a reserved character, will change how the URI is
   interpreted by most applications.  Thus, characters in the reserved
   set are protected from normalization and are therefore safe to be
   used by scheme-specific and producer-specific algorithms for
   delimiting data subcomponents within a URI.

"Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications."

So it can be interpreted differently, perhaps it should be unambiguous in the interpretation by encoding it so it's seen as data rather than a part of scheme? Maybe I'm missing something, but it appears that way. I'll try asking LibreWolf about it, maybe they can upstream a fix, but I don't see any harm in encoding it.

@OdinVex commented on GitHub (Mar 21, 2025): > So that's the user agent (browser)'s bug, it should follow the standard to accept `@` https://www.rfc-editor.org/rfc/rfc3986.txt ``` reserved = gen-delims / sub-delims gen-delims = ":" / "/" / "?" / "#" / "[" / "]" / "@" sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "=" The purpose of reserved characters is to provide a set of delimiting characters that are distinguishable from other data within a URI. URIs that differ in the replacement of a reserved character with its corresponding percent-encoded octet are not equivalent. Percent- encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications. Thus, characters in the reserved set are protected from normalization and are therefore safe to be used by scheme-specific and producer-specific algorithms for delimiting data subcomponents within a URI. ``` "Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications." So it can be interpreted differently, perhaps it should be unambiguous in the interpretation by encoding it so it's seen as data rather than a part of scheme? Maybe I'm missing something, but it appears that way. I'll try asking LibreWolf about it, maybe they can upstream a fix, but I don't see any harm in encoding it.

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

That part doesn't really apply to the "path encoding"

And you can see Golang also follows the standard: no need to encode @.

func main() {
	a := url.PathEscape("a?b@.txt")
	println(a)
}
//output: a%3Fb@.txt

We always use the official and standard libraries, so if there is a bug, the bug should be reported to Golang

@wxiaoguang commented on GitHub (Mar 21, 2025): That part doesn't really apply to the "path encoding" And you can see Golang also follows the standard: no need to encode `@`. ```go func main() { a := url.PathEscape("a?b@.txt") println(a) } //output: a%3Fb@.txt ``` We always use the official and standard libraries, so if there is a bug, the bug should be reported to Golang

GiteaMirror commented

2025-11-02 11:08:44 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

That part doesn't really apply to the "path encoding"

And you can see Golang also follows the standard: no need to encode @.

func main() {
a := url.PathEscape("a?b@.txt")
println(a)
}
//output: a%3Fb@.txt

We always use the official and standard libraries, so if there is a bug, the bug should be reported to Golang

The fact it can be interpreted between scheme and application suggests it's up to the implementation, at least what I'm concluding from the RFC. Maybe I'm wrong, I'll try to find out where/who it needs to go. I've wrote a ViolentMonkey script to rewrite the URIs on every page to encode @ for now.

Edit: Not sure if it matters, but the actual file path was https://.../src/branch/lineage-22.1/proprietary/odm/bin/hw/android.hardware.drm@1.3-service.widevine and raw HTML (relative link with stripped user/repo): .../src/branch/lineage-22.1/proprietary/odm/bin/hw/android.hardware.drm@1.3-service.widevine, a cloned repo of an Android proprietary-bin dump for ROM development.

@OdinVex commented on GitHub (Mar 21, 2025): > That part doesn't really apply to the "path encoding" > > And you can see Golang also follows the standard: no need to encode `@`. > > func main() { > a := url.PathEscape("a?b@.txt") > println(a) > } > //output: a%3Fb@.txt > > We always use the official and standard libraries, so if there is a bug, the bug should be reported to Golang The fact it can be interpreted between scheme and application suggests it's up to the implementation, at least what I'm concluding from the RFC. Maybe I'm wrong, I'll try to find out where/who it needs to go. I've wrote a ViolentMonkey script to rewrite the URIs on *every* page to encode `@` for now. Edit: Not sure if it matters, but the actual file path was `https://.../src/branch/lineage-22.1/proprietary/odm/bin/hw/android.hardware.drm@1.3-service.widevine` and raw HTML (*relative link with stripped user/repo*): `.../src/branch/lineage-22.1/proprietary/odm/bin/hw/android.hardware.drm@1.3-service.widevine`, a cloned repo of an Android proprietary-bin dump for ROM development.

GiteaMirror commented

2025-11-02 11:08:45 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

Let's take a look at more examples.

The widely used JS site: https://www.unpkg.com/

If your browser is unable to handle @ in a URL path, then it will have problems on various sites.

@wxiaoguang commented on GitHub (Mar 21, 2025): Let's take a look at more examples. The widely used JS site: https://www.unpkg.com/ ![Image](https://github.com/user-attachments/assets/1fe821ec-47c6-4c2a-910e-6e1c63ae6a31) If your browser is unable to handle `@` in a URL path, then it will have problems on various sites.

GiteaMirror commented

2025-11-02 11:08:45 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

It worked there. I guess now I need to figure out why my browser is stripping it but only for this.

@OdinVex commented on GitHub (Mar 21, 2025): It worked there. I guess now I need to figure out why my browser is stripping it but only for this.

GiteaMirror commented

2025-11-02 11:08:45 -06:00

@wxiaoguang commented on GitHub (Mar 21, 2025):

And one more exmaple: https://www.jsdelivr.com/ . No site really needs to encode @ (but it is fine to encode).

Then the @ encoding isn't the root case.

No idea why some links doesn't work on your side.

@wxiaoguang commented on GitHub (Mar 21, 2025): And one more exmaple: https://www.jsdelivr.com/ . No site really needs to encode `@` (but it is fine to encode). Then the `@` encoding isn't the root case. No idea why some links doesn't work on your side.

GiteaMirror commented

2025-11-02 11:08:45 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

I'll try to figure out why, but encoding it does "fix" for this. I wonder if it's some kind of security/header or something specific to my instance.

@OdinVex commented on GitHub (Mar 21, 2025): I'll try to figure out why, but encoding it does "fix" for this. I wonder if it's some kind of security/header or something specific to my instance.

GiteaMirror commented

2025-11-02 11:08:45 -06:00

@OdinVex commented on GitHub (Mar 21, 2025):

Figured it out. Buggy extension that doesn't report even seeing the URI. I've spent an hour trying to figure this out and that extension would normally show a notification it's doing something but didn't. Redirector silently borked the link, so I'll take the issue to Redirector. Apologies.

@OdinVex commented on GitHub (Mar 21, 2025): Figured it out. Buggy extension that doesn't report even seeing the URI. I've spent an hour trying to figure this out and that extension would normally show a notification it's doing something but didn't. `Redirector` silently borked the link, so I'll take the issue to Redirector. Apologies.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/gitea#14287