Incorrect report of ambiguous unicode characters #10343

Closed
opened 2025-11-02 09:04:49 -06:00 by GiteaMirror · 15 comments
Owner

Originally created by @sergeych on GitHub (Feb 25, 2023).

Description

The wiki page viewer (and editor) show a warring on ambiguous unicode chararcters I tried for most non-latin1 characters. Please remove it at all. It is hardly useful even would it be functional (and it is not). It is not possible to make a good guess on which unicode is OK for a project with many localisations. The locale of the developer (reporter, wiki writer) computer/browser often will be different from the locale that is used to report a problem (in a localised text for example) or describe a test case described in the wiki page.

Gitea Version

1.18.5

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

Screenshot 2023-02-25 at 15 06 10

Git Version

1.18.5

Operating System

Linux ** 4.19.0-21-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64

How are you running Gitea?

downloaded your binaries

Database

PostgreSQL

Originally created by @sergeych on GitHub (Feb 25, 2023). ### Description The wiki page viewer (and editor) show a warring on ambiguous unicode chararcters I tried for most non-latin1 characters. Please remove it at all. It is hardly useful even would it be functional (and it is not). It is not possible to make a good guess on which unicode is OK for a project with many localisations. The locale of the developer (reporter, wiki writer) computer/browser often will be different from the locale that is used to report a problem (in a localised text for example) or describe a test case described in the wiki page. ### Gitea Version 1.18.5 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist _No response_ ### Screenshots <img width="823" alt="Screenshot 2023-02-25 at 15 06 10" src="https://user-images.githubusercontent.com/300065/221361266-60b3ad8c-a92a-4e53-9208-0bce6fb274ca.png"> ### Git Version 1.18.5 ### Operating System Linux ** 4.19.0-21-amd64 #1 SMP Debian 4.19.249-2 (2022-06-30) x86_64 ### How are you running Gitea? downloaded your binaries ### Database PostgreSQL
GiteaMirror added the topic/wikitype/bug labels 2025-11-02 09:04:49 -06:00
Author
Owner

@sergeych commented on GitHub (Feb 25, 2023):

looks like it is fixed to not yet included in the try.gitea version (1.20...). Hope it is fixed ;)

@sergeych commented on GitHub (Feb 25, 2023): looks like it is fixed to not yet included in the try.gitea version (1.20...). Hope it is fixed ;)
Author
Owner

@lunny commented on GitHub (Mar 5, 2023):

I think this is a known issue.

@lunny commented on GitHub (Mar 5, 2023): I think this is a known issue.
Author
Owner

@wxiaoguang commented on GitHub (Mar 5, 2023):

Known issues have been fixed. This could be a new issue.

If you could provide a reproducible page on try.gitea.io, maybe it would help some maintainers to take a look at it.

@wxiaoguang commented on GitHub (Mar 5, 2023): Known issues have been fixed. This could be a new issue. If you could provide a reproducible page on try.gitea.io, maybe it would help some maintainers to take a look at it.
Author
Owner

@wxiaoguang commented on GitHub (Mar 5, 2023):

More or less related to this one, but not the same one:

image

@wxiaoguang commented on GitHub (Mar 5, 2023): More or less related to this one, but not the same one: * https://try.gitea.io/DarrenPIngram/test/src/branch/main/test * https://github.com/go-gitea/gitea/issues/20451 ![image](https://user-images.githubusercontent.com/2114189/222962470-162f2b16-e7b1-4a77-b090-29744654a0f5.png)
Author
Owner

@sergeych commented on GitHub (Mar 6, 2023):

Known issues have been fixed. This could be a new issue.

If you could provide a reproducible page on try.gitea.io, maybe it would help some maintainers to take a look at it.

I was not able to reproduce it on try.gitea - but the version there is a beta, while I'm using the latest stable. Hope it is fixed in beta (but strange it is not yet fixed in fresh new stable), but it is very likely its just a new "feature" not yet ported to the try.gitea. I sincerely hate the idea of trying to analyze unicode in the text field... Let browser do it. Practically no reason to spend effort to just make the life of gitea user just more complicated.

@sergeych commented on GitHub (Mar 6, 2023): > Known issues have been fixed. This could be a new issue. > > If you could provide a reproducible page on try.gitea.io, maybe it would help some maintainers to take a look at it. I was not able to reproduce it on try.gitea - but the version there is a beta, while I'm using the latest stable. Hope it is fixed in beta (but strange it is not yet fixed in fresh new stable), but it is very likely its just a new "feature" not yet ported to the try.gitea. I sincerely hate the idea of trying to analyze unicode in the text field... Let browser do it. Practically no reason to spend effort to just make the life of gitea user just more complicated.
Author
Owner

@wxiaoguang commented on GitHub (Mar 6, 2023):

I was not able to reproduce it on try.gitea

Could you share the text contents of your screenshot? Wrap them here by ```

while I'm using the latest stable

Do you mean 1.18.x? If there is a chance, could you try 1.19.0-rc1? Most changes in try.gita.io are also in 1.19, but not in 1.18.

@wxiaoguang commented on GitHub (Mar 6, 2023): > I was not able to reproduce it on try.gitea Could you share the text contents of your screenshot? Wrap them here by ```` ``` ```` > while I'm using the latest stable Do you mean 1.18.x? If there is a chance, could you try 1.19.0-rc1? Most changes in try.gita.io are also in 1.19, but not in 1.18.
Author
Owner

@ARNik commented on GitHub (Mar 22, 2023):

Have the same issue with 1.19.0

@ARNik commented on GitHub (Mar 22, 2023): Have the same issue with 1.19.0
Author
Owner

@sergeych commented on GitHub (Mar 31, 2023):

So any plan to fix it? Just remove the clumsy and useless check.

@sergeych commented on GitHub (Mar 31, 2023): So any plan to fix it? Just remove the clumsy and useless check.
Author
Owner

@wxiaoguang commented on GitHub (Apr 1, 2023):

Could you share the text contents of your screenshot? Wrap them here by ```


How about introducing a global config option to disable the check for all repositories?

@wxiaoguang commented on GitHub (Apr 1, 2023): Could you share the text contents of your screenshot? Wrap them here by ```` ``` ```` ---- How about introducing a global config option to disable the check for all repositories?
Author
Owner

@lunny commented on GitHub (Apr 1, 2023):

I like to add a global option to disable it but keep it open default as before.

@lunny commented on GitHub (Apr 1, 2023): I like to add a global option to disable it but keep it open default as before.
Author
Owner

@sergeych commented on GitHub (Apr 1, 2023):

with due respect what is the reason to leave it on by default if it just
does not works false reporting absolutely normal sequences?

On Sat, Apr 1, 2023 at 4:13 AM Lunny Xiao @.***> wrote:

I like to add a global option to disable it but keep it open default as
before.


Reply to this email directly, view it on GitHub
https://github.com/go-gitea/gitea/issues/23149#issuecomment-1492812414,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACJIIITP53YHHTBTHRSFADW66MNDANCNFSM6AAAAAAVH4EK3E
.
You are receiving this because you authored the thread.Message ID:
@.***>

@sergeych commented on GitHub (Apr 1, 2023): with due respect what is the reason to leave it on by default if it just does not works false reporting absolutely normal sequences? On Sat, Apr 1, 2023 at 4:13 AM Lunny Xiao ***@***.***> wrote: > I like to add a global option to disable it but keep it open default as > before. > > — > Reply to this email directly, view it on GitHub > <https://github.com/go-gitea/gitea/issues/23149#issuecomment-1492812414>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AACJIIITP53YHHTBTHRSFADW66MNDANCNFSM6AAAAAAVH4EK3E> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
Author
Owner

@wxiaoguang commented on GitHub (Apr 1, 2023):

For security:

ps: Could you share the text contents of your screenshot? Wrap them here by ``` ? At least, it's easier for developer to reproduce your issue and do improvements in the future.

@wxiaoguang commented on GitHub (Apr 1, 2023): For security: * https://www.securityweek.com/trojan-source-attack-abuses-unicode-inject-vulnerabilities-code/ * https://threatpost.com/trojan-source-invisible-bugs-source-code/175891/ * GitHub also warns on ambiguous unicode characters, while it does better ps: Could you share the text contents of your screenshot? Wrap them here by ```` ``` ```` ? At least, it's easier for developer to reproduce your issue and do improvements in the future.
Author
Owner

@sergeych commented on GitHub (Apr 1, 2023):

The text was:

> Важно: право на операцию импорта теперь имеют только те пользователи, у
которых имя (не логин!) кончаются на "$"

On Sat, Apr 1, 2023 at 4:36 AM wxiaoguang @.***> wrote:

For security:

https://www.securityweek.com/trojan-source-attack-abuses-unicode-inject-vulnerabilities-code/

https://threatpost.com/trojan-source-invisible-bugs-source-code/175891/

  • GitHub also warns on ambiguous unicode characters, while it does
    better

ps: Could you share the text contents of your screenshot? Wrap them here
by ``` ? At least, it's easier for developer to reproduce your issue and
do improvements in the future.


Reply to this email directly, view it on GitHub
https://github.com/go-gitea/gitea/issues/23149#issuecomment-1492816801,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACJIIMOYEB3Q3WDVMX6MKTW66PEBANCNFSM6AAAAAAVH4EK3E
.
You are receiving this because you authored the thread.Message ID:
@.***>

@sergeych commented on GitHub (Apr 1, 2023): The text was: ~~~ > Важно: право на операцию импорта теперь имеют только те пользователи, у которых имя (не логин!) кончаются на "$" ~~~ On Sat, Apr 1, 2023 at 4:36 AM wxiaoguang ***@***.***> wrote: > For security: > > - > https://www.securityweek.com/trojan-source-attack-abuses-unicode-inject-vulnerabilities-code/ > - > https://threatpost.com/trojan-source-invisible-bugs-source-code/175891/ > - GitHub also warns on ambiguous unicode characters, while it does > better > > ps: Could you share the text contents of your screenshot? Wrap them here > by ``` ? At least, it's easier for developer to reproduce your issue and > do improvements in the future. > > — > Reply to this email directly, view it on GitHub > <https://github.com/go-gitea/gitea/issues/23149#issuecomment-1492816801>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AACJIIMOYEB3Q3WDVMX6MKTW66PEBANCNFSM6AAAAAAVH4EK3E> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
Author
Owner

@silverwind commented on GitHub (Apr 30, 2023):

I assume wiki needs the same treatment as https://github.com/go-gitea/gitea/pull/22016, cc @zeripath.

@silverwind commented on GitHub (Apr 30, 2023): I assume wiki needs the same treatment as https://github.com/go-gitea/gitea/pull/22016, cc @zeripath.
Author
Owner

@silverwind commented on GitHub (Apr 30, 2023):

This should be an easy fix. Hide the warning on wiki pages via template condition, similar to what https://github.com/go-gitea/gitea/pull/22016 did for file render view.

@silverwind commented on GitHub (Apr 30, 2023): This should be an easy fix. Hide the warning on wiki pages via template condition, similar to what https://github.com/go-gitea/gitea/pull/22016 did for file render view.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#10343