Dashes in Wiki page titles are replaced with spaces #3647

@lunny commented on GitHub (Jul 23, 2019):

It seems we already have a PR to fix this.

@lunny commented on GitHub (Jul 23, 2019): It seems we already have a PR to fix this.

GiteaMirror commented

@qwertfisch commented on GitHub (Jul 24, 2019):

@lunny In the list of open PRs I could not find anything related. Can you reference the PR in mind?

@qwertfisch commented on GitHub (Jul 24, 2019): @lunny In the list of open PRs I could not find anything related. Can you reference the PR in mind?

GiteaMirror commented

@qwertfisch commented on GitHub (Jul 24, 2019):

@zeripath Sorry, that does not work either. The replacement of a dash character to a space seems to be executed right after URI decoding, but there is no double decoding.

I made an overview table to see the different behaviour. The last column means that a page can be loaded on the given URL, which does not work in the second and third case. It is only visible in the list of all pages.

Description	Filename	Resulting URL	Display	Page is found
Normal URI encoded chars	`colon%3A-try-it.md`	`colon%3A-try-it`	colon: try it	✔
Space URI encoded	`space%20test.md`	`space-test`	space test	✘
Dash URI encoded	`page-with-one%2Ddash.md`	`page-with-one-dash`	page with one dash	✘
Dash double URI encoded	`page-with-one%252Ddash.md`	`page-with-one%2Ddash`	page with one%2Ddash	✔

The second and third rows are kind of confusing. It looks as if %20 and %2D are URI decoded from the filename, then there is the default dash/space conversion to create a page title. But when creating the URL, it seems as if it’s not created from filename, but from the (previously decoded) page title. Which of course results in - characters for all spaces (and no dash at all), so neither the correct file with %2D nor %20 can be found.

Suggestion

Can’t we just remove skip the errornous space/dash conversion and store the page with URI encoding? This would solve every problem case, with the slight disadvantage of spaces encoded %20 in the URL …

@qwertfisch commented on GitHub (Jul 24, 2019): @zeripath Sorry, that does not work either. The replacement of a dash character to a space seems to be executed right after URI decoding, but there is no double decoding. I made an overview table to see the different behaviour. The last column means that a page can be loaded on the given URL, which does not work in the second and third case. It is only visible in the list of all pages. | Description | Filename | Resulting URL | Display | Page is found | | --- | --- | --- | --- | ---: | | Normal URI encoded chars | `colon%3A-try-it.md` | `colon%3A-try-it` | colon: try it | ✔ | | Space URI encoded | `space%20test.md` | `space-test` | space test | ✘ | | Dash URI encoded | `page-with-one%2Ddash.md` | `page-with-one-dash` | page with one dash | ✘ | | Dash double URI encoded | `page-with-one%252Ddash.md` | `page-with-one%2Ddash` | page with one%2Ddash | ✔ | The second and third rows are kind of confusing. It looks as if %20 and %2D are URI decoded from the filename, then there is the default dash/space conversion to create a page title. But when creating the URL, it seems as if it’s not created from filename, but from the (previously decoded) page title. Which of course results in `-` characters for all spaces (and no dash at all), so neither the correct file with `%2D` nor `%20` can be found. ## Suggestion Can’t we just remove skip the errornous space/dash conversion and store the page with URI encoding? This would solve every problem case, with the slight disadvantage of spaces encoded `%20` in the URL …

GiteaMirror commented

d4667a4949/models/wiki.go (L43-L53)

@mrsdizzie commented on GitHub (Jul 24, 2019):

Haven't looked too close but this is probably happening here:

So its first unescaping the file name and then running it through something that just replaces - with " " which matches described behavior.

I agree the filenames should have always been stored encoded but unfortunately they weren't and changing that would put dashes in the title of everyones page where they weren't there before. Would be a breaking change to consider.

@mrsdizzie commented on GitHub (Jul 24, 2019): Haven't looked too close but this is probably happening here: https://github.com/go-gitea/gitea/blob/d4667a4949729ad73b7dc0c633bd6daca4508073/models/wiki.go#L43-L53 So its first unescaping the file name and then running it through something that just replaces - with " " which matches described behavior. I agree the filenames should have always been stored encoded but unfortunately they weren't and changing that would put dashes in the title of everyones page where they weren't there before. Would be a breaking change to consider.

GiteaMirror commented

@zeripath commented on GitHub (Jul 24, 2019):

Make it a per repo configurable?

@zeripath commented on GitHub (Jul 24, 2019): Make it a per repo configurable?

GiteaMirror commented

@mrsdizzie commented on GitHub (Jul 24, 2019):

I hesitate to add yet more config options, particularly to keep a behavior that is not good (current replacing of space with dash).

I'd rather try and have some sort of code to handle legacy cases, or even just consider all literal dashes as spaces (breaking the less common case of a dash in the title) and then escape everything going forward and handle that properly.

I guess that would maybe look like first replacing all literal dashes and then escaping the filename. It would only break existing titles with a dash in them which already seem to not work properly anyways per this issue

@mrsdizzie commented on GitHub (Jul 24, 2019): I hesitate to add yet more config options, particularly to keep a behavior that is not good (current replacing of space with dash). I'd rather try and have some sort of code to handle legacy cases, or even just consider all literal dashes as spaces (breaking the less common case of a dash in the title) and then escape everything going forward and handle that properly. I guess that would maybe look like first replacing all literal dashes and then escaping the filename. It would only break existing titles with a dash in them which already seem to not work properly anyways per this issue

GiteaMirror commented

@zeripath commented on GitHub (Jul 24, 2019):

Hmm what happens to + ?

@zeripath commented on GitHub (Jul 24, 2019): Hmm what happens to + ?

GiteaMirror commented

@mrsdizzie commented on GitHub (Jul 24, 2019):

In what situation? I think in the situation of saving a new filename we should make sure to encode + to %2B and not leave it as +

Are you asking if somebody already has a + in their filename? If so that should just still work since any unencoding wouldn't mess with it.

@mrsdizzie commented on GitHub (Jul 24, 2019): In what situation? I think in the situation of saving a new filename we should make sure to encode + to %2B and not leave it as + Are you asking if somebody already has a + in their filename? If so that should just still work since any unencoding wouldn't mess with it.

GiteaMirror commented

@mrsdizzie commented on GitHub (Jul 24, 2019):

Probably the real problem to solve would be preserving current links or making sure they still work somehow

@mrsdizzie commented on GitHub (Jul 24, 2019): Probably the real problem to solve would be preserving current links or making sure they still work somehow

GiteaMirror commented

@qwertfisch commented on GitHub (Jul 25, 2019):

@zeripath + and ? are escaped and can be used properly. In fact every ASCII special character is usable. Only -._~ are not encoded but put directly in the filename / URL. (And of course the space is converted to dash before storing as file. This should be %20 normally.)

page title: chartest !"#$%&'()*+, ./:;<=>?@[]^_`{|}~§
filename: chartest-%21%22%23%24%25%26%27%28%29%2A%2B%2C-.%2F%3A%3B%3C%3D%3E%3F%40%5B%5C%5D%5E_%60%7B%7C%7D~%C2%A7.md

@qwertfisch commented on GitHub (Jul 25, 2019): @zeripath `+` and `?` are escaped and can be used properly. In fact every ASCII special character is usable. Only `-._~` are not encoded but put directly in the filename / URL. (And of course the space is converted to dash before storing as file. This should be `%20` normally.) - page title: chartest !"#$%&'()*+, ./:;<=>?@[\]^_`{|}~§ - filename: `chartest-%21%22%23%24%25%26%27%28%29%2A%2B%2C-.%2F%3A%3B%3C%3D%3E%3F%40%5B%5C%5D%5E_%60%7B%7C%7D~%C2%A7.md`

GiteaMirror commented

@zeripath commented on GitHub (Jul 25, 2019):

@mrsdizzie @qwertfisch I was meaning that a plain '+' in an url should map to ' ' by convention, and through my testing it appears that this doesn't get mapped to '-' but rather retains the ' ' when passed through - so you should be able to reach a file with spaces in that way.

Yeesh this is so broken. I've been ignoring the wiki, as like the diff page, as I have been suspicious that it needs a thorough overhaul. This proves my fears.

@mrsdizzie I agree that adding configuration should be avoided as much as is possible. If we don't want to add configuration another option is to from 1.10 do things correctly but fallback to the previous behaviour if the old way would have a different file?

@zeripath commented on GitHub (Jul 25, 2019): @mrsdizzie @qwertfisch I was meaning that a plain '+' in an url should map to ' ' by convention, and through my testing it appears that this doesn't get mapped to '-' but rather retains the ' ' when passed through - so you should be able to reach a file with spaces in that way. Yeesh this is so broken. I've been ignoring the wiki, as like the diff page, as I have been suspicious that it needs a thorough overhaul. This proves my fears. @mrsdizzie I agree that adding configuration should be avoided as much as is possible. If we don't want to add configuration another option is to from 1.10 do things correctly but fallback to the previous behaviour if the old way would have a different file?

GiteaMirror commented

@mrsdizzie commented on GitHub (Jul 26, 2019):

Perhaps -- probably something to think about if redesigning this. I'll note that it appears Gitea has just copied this same exact behavior from Github, which does the same thing with dashes and titles/filenames and you can't create a page title with dashes in the title there either.

@mrsdizzie commented on GitHub (Jul 26, 2019): Perhaps -- probably something to think about if redesigning this. I'll note that it appears Gitea has just copied this same exact behavior from Github, which does the same thing with dashes and titles/filenames and you can't create a page title with dashes in the title there either.

GiteaMirror commented

@icetiger1974 commented on GitHub (May 19, 2021):

hello
Is this issue solved?
Or remain in the same situation?

@icetiger1974 commented on GitHub (May 19, 2021): hello Is this issue solved? Or remain in the same situation?

GiteaMirror commented

@unchaynd commented on GitHub (Dec 17, 2022):

You can't put a dash in a Wiki page title?... Seriously??

@unchaynd commented on GitHub (Dec 17, 2022): You can't put a dash in a Wiki page title?... Seriously??

GiteaMirror commented