mirror of
https://github.com/semver/semver.git
synced 2026-07-11 03:53:53 -05:00
[GH-ISSUE #121] Unicode? #4323
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/semver#4323
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @beatgammit on GitHub (Jun 18, 2013).
Original GitHub issue: https://github.com/semver/semver/issues/121
I don't understand why only ASCII is supported, and I can understand developers from around the world wanting unicode prerelease/build parts.
Can this be loosened to allow UTF-8 strings as well? It's a bit more difficult to implement in languages without good unicode support, but is this a good enough reason to restrict it to ASCII?
@denis-sokolov commented on GitHub (Sep 5, 2013):
What are real world use cases for non-ASCII requirement? This is not an end-user UI, this is something people type in their shells. Version numbers can easily find their way to file/directory names.
Don't get me wrong, I strongly dislike English-centered view of the world in programming, but in this case this is deep technical information. I wouldn't expect a motherboard serial number to contain non-ASCII characters, and neither would I expect to see a build number.
@kostrse commented on GitHub (Sep 9, 2013):
Version numbers should be culture-neutral and understandable by people of any culture. Version number suffixes in Russian or Chinese don't make a lot of sense for non-Russian or Chinese speakers.
Everything should go toward simplification of communication between people, not otherwise.
So, I'm against Unicode.
@ShimmerFairy commented on GitHub (Nov 28, 2013):
One easy situation is if someone wants to use
Instead of
Which I at least don't think is so far-fetched.
@denis-sokolov I know this is different for everybody, but at least in my situation it's not that hard to type and otherwise interact with Unicode characters. For me specifically this is a non-issue, and I'd be surprised if a general-purpose computer from the last decade or so was incapable of handling Unicode filenames. And even then, I would imagine software targeting such incapable systems would refrain from using Unicode filenames and such in any situation.
@kostrse I'm not sure that's a valid argument. I'm sure there are Russian and Chinese developers who'd like to use Cyrillic or Hanzi in their version numbers, regardless of whether foreigners can understand them. Not every piece of software is made for a global audience; thus I see little point in presuming such.
So, to be clear, I don't buy the practical reasons for the limit, nor do I see many technical reasons. Also to keep in mind is that this won't force you to use Unicode, it just keeps you from violating the specification because you decided you prefer lowercase greek letters to spelling out their names in English :) .
EDIT: Ah, almost forgot another easy case: Chinese, Japanese and Korean developers could easily be using fullwidth characters instead of the typical ASCII: https://en.wikipedia.org/wiki/Fullwidth#Fullwidth_form
Granted, you could argue that they should just switch to the original ASCII letters, but at least I think
Isn't too rare a thing to disallow.
@denis-sokolov commented on GitHub (Nov 28, 2013):
@lue, what are the practical advantages of
1.2.3-αor1.2.3-ａｌｐｈａover1.2.3-alpha?Use cases should have some measurable benefit to be worthy of considering. I see no advantage that a developer with whatever background would have when using one of these provided examples.
Please provide a use case, because I don't see any proof for this statement.
Nobody argues that more features allow for more possibilities.
But it's also clear that adding more features brings more problems.
The decision to implement a feature should weigh the benefit against the risks.
I see risks very clearly. I still don't see any real benefits.
@ShimmerFairy commented on GitHub (Nov 28, 2013):
I don't have any real-world examples onhand. My thoughts are based on the idea that "just because I'm not going to need it, doesn't mean there's no point to it". I also wanted to comment because all the replies thus far to this bug were strictly anti-Unicode, so I felt it important to log a pro-Unicode view :) .
I do recognize the technical difficulties of Unicode support coming from ASCII-only. I just think in this day and age you need a very good technical reason to use ASCII instead of Unicode, beyond relatively common implementation hurdles, and I haven't seen them yet.
I understand the idea the SemVar should require the least number of not strictly necessary things as possible. I just happen to think Unicode is the rule, not the exception, these days, which makes this limitation more arbitrary than others. I'm also of the opinion that a system without the ability to handle a Unicode-using version string will also have much larger programs in running the offending program.
Of course, my views are likely horribly biased by the fact that I've yet to work on such ancient or strange hardware that can't handle Unicode. For me though, even though I'll never likely go beyond ASCII in a version string (except for α and related), I don't take that as reason enough to implement the restriction.
(There's also the larger issue of forcing developers to abide by any one character encoding to use a not-inherently-encoding-dependent version number, but that's another can of worms perhaps.)
And yes, there's no practical value in
1.2.3-α, I just think it's more fun :) .@denis-sokolov commented on GitHub (Nov 29, 2013):
@lue, this was a very well expressed argument. 👍
I do still think that for SemVer it's useful to limit ourselves.
Note, that SemVer does not allow ASCII set of characters, instead it's only
[A-Za-z0-9].Among other things it allows people to reason about the sort order (SemVer chapter 11).
@ShimmerFairy commented on GitHub (Nov 29, 2013):
Now that you pointed out the alphanumeric limit, I think I see how this issue was caused. It was the use of the term "ASCII" where it isn't the best choice (assuming that SemVar doesn't care which encoding you store your version string in, be it ASCII, UTF-8, EBCDIC, etc.).
would better read
In chapters 9 and 10 . ("ASCII sort order" in Chapter 11 seems fine enough though.) Perhaps such a change would keep people (myself included) from seeing this as an encoding limitation, when really it's just your ordinary alphanumerics limitation.
And in the context of sorting, it does get tricky with multiple scripts. So I'm fine with the alphanumerics limitation. It was just that it seemed like an encoding limitation.
@beatgammit, you can use UTF-8 encoding on your version string already. Just not the full range of Unicode characters, for keep-sorting-sane reasons. This same thing would come up if we were, say, talking about CP-437 strings, or any other kind of encoding with characters beyond alphanumerics and the hyphen.
(Of course, you could just say you sort by codepoint and leave it at that, but... ☺)
@bessarabov commented on GitHub (Nov 30, 2013):
Here is the problem that we will face if we will allow unicode in SemVer.
Can you find the difference with this 2 verions (the hint: the letters look the same, but actually that are different letters):
One more example:
I think that this a huge problem and it is so huge that only because of this problem we should not allow the unicode symbols in the SemVer.
@beatgammit commented on GitHub (Nov 30, 2013):
@bessarabov - This is the same argument as
1.2.3-0vs1.2.3-O(0 vs capital O). On some fonts, they render the same. There are also fonts where there is no distinction between capitals and lower case.Ok, here's a list of the arguments so far against allowing Unicode:
This is also something that end-users see on a
Downloadspage for a project. For example, if I have a list of downloadable versions, and I support Japanese, English and German, I may build separate version for each locale. I could use locale specifiers (en_US, en_UK, jp, de), but that doesn't solve the problem of build modifiers.For example, if I have separate builds with different features, I may want the features specified in the target locale. For example, consider this list of builds:
Specifying features in the target language
While I'm arging for this, I would also like to have
_available (specifying locales would be much nicer withen_US). This is a separate issue if Unicode characters are not to be allowed.And version number suffixes in English don't make a lot of sense for non-English speakers. This is very similar to my above comment about builds for a specific locale. I do not expect all users of my software to speak enough english to know the English names for all special builds in their language. If it's not supported in the standard, I have two options:
First off, according to the standard (for build metadata):
So, the answer is no, I may not use anything besides Latin alphanumerics and the hyphen if I want my version strings to be recognized by all SemVer-compliant parsers.
As for the argument (sane sorting), build metadata does not affect sorting:
I can accept only allowing ASCII on pre-release versions if sorting really is that ambiguous, but even then, there are typically libraries for a locale-specific, unicode-aware sort. For example, in JavaScript, there's
String.localeCompare, which I take to mean there's an unambiguous sort for unicode strings.My main argument is:
Should SemVer decide on a common set of unambiguous characters that may be used for all projects?
I don't think so. Why should SemVer define what a project can and cannot use?
@ShimmerFairy commented on GitHub (Nov 30, 2013):
@beatgammit about your comment
That's exactly what I said. You can use UTF-8 encoding, but not its characters. I think you misread my statement there. (It's the same case with ISO-8859-13, CP-437, and UTF-32BE).
I do agree with the other arguments you made though, and have said as much :) .
Now that I see where the confusion arose from, I'm totally OK with the restriction, although I do agree that the underscore should be allowed within the current
[A-Za-z0-9-_]range.That's why I proposed the
s/ASCII/Latin/change above, in order to not bring encoding-based terms where they don't belong. Although looking further, a character restriction on the+stuffportion is wrong, considering it apparently doesn't matter. Use whatever characters the encoding you store the version string in allows. (After all, if it has no bearing on version sorting, why care?)So, to put it another way, in Chapter 9, replace "ASCII" with "Latin" or other sufficiently encoding-neutral term. Additionally allow use of the underscore, considering it equivalent to the hyphen.
In Chapter 10, strike
and replace with
https://en.wikipedia.org/wiki/Graphic_character should give you an idea of what "graphical characters" means. It's basically there to disallow use of, say, the BEL character or other control codes in the string. This is of course dependent on your chosen encoding's definition of a character.
@beatgammit commented on GitHub (Dec 1, 2013):
This is definitely progress, so I'll take it, assuming that your proposed changes to Chapter 10 would also be included. This would satisfy my use cases, and keep sorting simple. I still think that unicode can be handled elegantly, since given a locale, sort order is pretty well-defined.
However, I also don't think it's worth arguing about, since the pre-release version should only contain information about the stage of development, not anything the end-user needs to read to know about the contents of the package.
@majewsky commented on GitHub (Nov 4, 2015):
"Latin alphanumerics" is an incorrect name for
[A-Za-z0-9]. "Latin" includes stuff like "Éæø". The technically correct name is "Basic Latin alphanumerics" as far as I can see.@hofmand commented on GitHub (May 17, 2017):
If a build number were added to the spec (MAJOR.MINOR.PATCH.BUILD), it could be used for sorting in place of the pre-release/build metadata. Then ASCII vs. Unicode wouldn't matter.
@jwdonahue commented on GitHub (Dec 2, 2017):
@beatgammit
For the range of allowed characters in the current version (2.0.0) of the spec, the ASCII and UTF-8 encodings are identical. This is a very old open issue that does not appear to have much chance of being resolved in favor of any other unicode encodings or range of characters. Please close this issue at your earliest possible convenience.
@silkentrance commented on GitHub (Dec 7, 2017):
@jwdonahue you called it. please close right now.
@jwdonahue commented on GitHub (Dec 7, 2017):
Alas, I cannot, or if I can, it is not obvious to me how. But even if I could, I would give @beatgammit a few more days to chime in or close it themselves.
@beatgammit commented on GitHub (Jan 21, 2018):
Which is why I'm proposing changing that.
I want to use graphical UTF-8 characters (e.g. Japanese Kanji) in at least the build portion of a semver string. For sorting purposes, implementations can already ignore everything after the
+if they don't need it. I would also like to have it in the pre-release part as well, but I understand that there are legitimate reasons why that would be undesirable.@jwdonahue I respectfully decline.
I think this issue has merit. Leaving it open does not hurt anyone or anything, so I see no reason to close it.
Adding unicode to the build portion of a semver does absolutely nothing in terms of spec-compliant parsers since they can just discard it, and users can ignore any funky font rendering for data in that part of the string.
@jwdonahue commented on GitHub (Jan 22, 2018):
@Haacked, this issue seems worthy of consideration for the version 3 plan.
I would personally support UTF-8 encodings for ranges of characters in other languages that are effectively equivalent, in those languages, to alpha-numeric and hyphens (with periods delimiting fields), provided that translations between language encodings can be automated and mechanism exists to provide proper-sort ordering. I think we need to keep an extremely tight reign on allowed field separator characters (currently just periods). I am too ignorant of localization issues to understand most of the implications however. This would definitely be a breaking change to the spec, so we'd have to get buy-in from stake-holders and be certain we don't over-complicate the specification, or the myriad applications already using SemVer.
Demand for a larger character set has not been high, as far as I know. Only six of us have chimed in on this thread over the course of the last three and half years or so, and I only visited it earlier in an effort to reduce the noise in the open issues list. Are there any examples of packaging tools that support a broader range of UTF-8 characters other than Latin alpha-numerics in their version strings? If you could show that SemVer was at risk of being marginalized due to common best practices, you might have a chance at instigating some change.
I for one, think that the current alpha-numeric sorting of the prerelease field is a design weakness. My personal preference would be some form of version schema, that would define labels with explicit sort ordering at each field level in the version string. Only pure numeric fields should sort numerically. The current scheme sorts -Dev higher than -alpha or -beta, which just seems wrong to me and leads to prerelease strings of the form -a.dev or -a.experiment.
I personally think that version 3 should be a more-or-less complete make-over that uses a version meta tag (URI) to identify the version schema in use, and allowing developers to define their semantics explicitly rather than implicitly. We could easily default to a schema that effectively defines SemVer 2.0.0 when no version meta tag is supplied, making the existing scheme forward compatible. A schema would make it much easier to automate sorting, and allow pre-defined tags along with their sort order. Add translatable descriptions to those tags and developers could more easily grok the intentions/semantics behind them. Something along these lines would make it easier for tool makers to provide support for a more inclusive range of UTF-8.
@jwdonahue commented on GitHub (Jan 11, 2020):
@beatgammit, it's been a year since the last post. I don't think I've seen any PR's related to this thread, which is ~6 years old at this point. I think it's time to close this thread.
@jwdonahue commented on GitHub (Jun 9, 2020):
@alexandrtovmach, can we put a "Punt to v3" tag on this and close it please? There is no reason to keep it open, but it does merit consideration if we ever come up with an acceptable version disambiguation scheme and guidelines for a v3 round of changes.
@alexandrtovmach commented on GitHub (Jun 10, 2020):
@jwdonahue sure, I'll go through this repo issues soon. Thanks for mention 👍
@KyNorthstar commented on GitHub (Jun 13, 2020):
I forked this repo and started going down the rabbit hole figuring out what it'd take to let you use arbitrary Unicode in a SemVer... Wow is it a lot. It might end up changing the regexes so they define what NOT to put in an identifier. Something like
\d+\.\d+\.\d+(?:-[^\.\+]+(?:\.[^\.\+]+)*)(?:+[^\.\+]+(?:\.[^\.\+]+)*)?@dbinette commented on GitHub (Nov 20, 2020):
Assuming that arbitrary encoding can be encoded into SemVer identifiers by an embedded app isn't likely to be difficult, it might be difficult for a small app to render its own SemVer and to portray its codepage.
It is likely that he SemVer is most useful for entities outside of the app itself, i.e. entities that are more prepared to deal with internationalization, it does impose a significant burden above that which is required for handling alpha-numerics in order to provide a capability beyond a normal need.
For an app that is manipulating its own SemVer ; the issue of dealing with extensive charset can be avoided, but dealing with external identifiers containing UTF-8 can be a problem.
I would support limiting identifiers to [A-Za-z0-9]
@DaveMcEwan commented on GitHub (Jan 23, 2023):
Also worth noting (explicitly) that ASCII
[A-Za-z0-9]is a subset of UTF8, so the argument isn't really about changing from ASCII to UTF8 - the argument is about extending the range of usable UTF8 characters.IMO, the existing range
[A-Za-z0-9]is perfectly good enough because it's the lowest common denominator amongst everybody who interacts with computer code. It's a fundamental requirement that version numbers should be a regular language, i.e. you can match them with a regex. A regex to match all the different graphical characters in UTF8, but avoiding homoglyphs, is unworkably complex; especially in comparison to the simplicity of[A-Za-z0-9](also known as[:alnum:]in POSIX).