[GH-ISSUE #163] Backporting #7080

Closed
opened 2026-06-20 16:45:55 -05:00 by GiteaMirror · 21 comments
Owner

Originally created by @gergelypolonkai on GitHub (Sep 16, 2013).
Original GitHub issue: https://github.com/semver/semver/issues/163

When a minor (Y) or major (X) version is released that also introduces a bug fix (which normally would be a patch level release (Z), that bug fix SHOULD be backported to the last minor release (thus releasing 1.1 after 1.0.5 SHOULD “create” a new 1.0.6 release, or releasing 2.0 after 1.1.8 SHOULD “create” a 1.1.9 release, too). This way if the users depend on 1.0.*, they will not have to transition to the new release, but still have the bug fix. (Although this would have real meaning when a new major version comes out. Also, this is not really a versioning, but a development issue.)

Originally created by @gergelypolonkai on GitHub (Sep 16, 2013). Original GitHub issue: https://github.com/semver/semver/issues/163 When a minor (Y) or major (X) version is released that also introduces a bug fix (which normally would be a patch level release (Z), that bug fix SHOULD be backported to the last minor release (thus releasing 1.1 after 1.0.5 SHOULD “create” a new 1.0.6 release, or releasing 2.0 after 1.1.8 SHOULD “create” a 1.1.9 release, too). This way if the users depend on 1.0.*, they will not have to transition to the new release, but still have the bug fix. (Although this would have real meaning when a new major version comes out. Also, this is not really a versioning, but a development issue.)
Author
Owner

@dumblob commented on GitHub (Mar 5, 2016):

Actually, backporting is a bigger problem than that. In practice, a serious security bug appears in software ABC. ABC uses linear (or non-linear) development with semver. The security fix needs to be ported to all types of versions running on production servers all over the world.

ABC version history looks like this:

1.0.0
        1.0.1 - in production
        1.0.2
    1.1.0 - in production
        1.1.1
    1.2.0
2.0.0 - in production
    2.1.0
    2.2.0 - in production
        2.2.1
    2.3.0 - security bug found, but applicable to the whole version history

Now the question is, how should we force production machines to update to a version which contains the version they're currently running, but with the backported security bug fix.

Note, that the production machines must not (for stability reasons or missing official certifications etc.) just switch to a newer version of ABC (disregarding if it's the Z or Y or X version subtree).

Note also, that these production versions with this security fix applied will need to be fixed again and then again etc. So the solution is not to add another number field meaning a "security fix version" (a patch type), because there might be any number of other types of patches, which will be important. Thus in the production environment, only a subset of these types of patches will not be ignored when updating ABC.

I was thinking about some dynamic approach with semver being the forward-looking basis and then any number of specified extensions following e.g. a concept of TLV (i.e. not literally, but just the conceptual idea of an infinite opportunity of adding, without the need to preserve order - therefore only a central online register of the most common TLV extensions would be needed).

The first TLV extension might be just "-sec1" (producing 1.0.1-sec1 1.1.0-sec1 2.0.0-sec1 2.2.0-sec1) with the meaning: this version contains security fix 1 where 1 would be a growing integer ID starting at 1 (an implicit Length is hidden in the type, which is sec as security fix; type has to have minimum length of 3 characters) and will be equivalent to -sec1:1.0.0 (semantic versioning of the patch). Another advantage would be the possibility to list all versions which are solving the security bug number 1 (I'm not saying applying the same security patch, because that might differ). In case the security patch wasn't correct, but already in production, a simple semver can be added after the security fix ID, like this: 1.0.1-sec1:1.0.1 and only the affected production versions will be changed.

The additive capability of TLV would allow e.g. something like 1.0.1-sec1-test1 reading version 1.0.1 as in semver with security fix 1 applied and being a testing version 1. Note, that 1.0.1-sec1-test1 is the same as 1.0.1-test1-sec1.

<!-- gh-comment-id:192662245 --> @dumblob commented on GitHub (Mar 5, 2016): Actually, backporting is a **bigger** problem than that. In practice, a serious security bug appears in software ABC. ABC uses linear (or [non-linear](https://github.com/mojombo/semver/issues/259)) development with semver. The security fix needs to be ported to all types of versions running on production servers all over the world. ABC version history looks like this: ``` 1.0.0 1.0.1 - in production 1.0.2 1.1.0 - in production 1.1.1 1.2.0 2.0.0 - in production 2.1.0 2.2.0 - in production 2.2.1 2.3.0 - security bug found, but applicable to the whole version history ``` Now the question is, how should we force production machines to update to a version which contains the version they're currently running, but with the backported security bug fix. Note, that the production machines must not (for stability reasons or missing official certifications etc.) just switch to a newer version of ABC (disregarding if it's the Z or Y or X version subtree). Note also, that these production versions with this security fix applied will need to be fixed again and then again etc. So the solution is not to add another number field meaning a "security fix version" (a patch _type_), because there might be any number of other _types_ of patches, which will be important. Thus in the production environment, only a subset of these _types_ of patches will not be ignored when updating ABC. I was thinking about some dynamic approach with semver being the forward-looking basis and then any number of specified extensions following e.g. a concept of [TLV](https://en.wikipedia.org/wiki/Type-length-value) (i.e. not literally, but just the conceptual idea of an infinite opportunity of adding, without the need to preserve order - therefore only a central online register of the most common TLV extensions would be needed). The first TLV extension might be just "-sec1" (producing `1.0.1-sec1` `1.1.0-sec1` `2.0.0-sec1` `2.2.0-sec1`) with the meaning: this version contains _security fix 1_ where `1` would be a growing integer ID starting at 1 (an implicit [Length](https://en.wikipedia.org/wiki/Type-length-value) is hidden in the type, which is `sec` as _security fix_; type has to have minimum length of 3 characters) and will be equivalent to `-sec1:1.0.0` (semantic versioning of the patch). Another advantage would be the possibility to list all versions which are solving the security bug number `1` (I'm not saying _applying the same security patch_, because that might differ). In case the security patch wasn't correct, but already in production, a simple semver can be added after the security fix ID, like this: `1.0.1-sec1:1.0.1` and only the affected production versions will be changed. The additive capability of TLV would allow e.g. something like `1.0.1-sec1-test1` reading _version 1.0.1 as in semver with security fix 1 applied and being a testing version 1_. Note, that `1.0.1-sec1-test1` is the same as `1.0.1-test1-sec1`.
Author
Owner

@dumblob commented on GitHub (Mar 29, 2016):

Also related to https://github.com/mojombo/semver/issues/256 , https://github.com/mojombo/semver/issues/242 , and a little bit to https://github.com/mojombo/semver/issues/213 .

<!-- gh-comment-id:203065706 --> @dumblob commented on GitHub (Mar 29, 2016): Also related to https://github.com/mojombo/semver/issues/256 , https://github.com/mojombo/semver/issues/242 , and a little bit to https://github.com/mojombo/semver/issues/213 .
Author
Owner

@dumblob commented on GitHub (Aug 1, 2016):

With regards to the (non)development of Semver in the past years and for clarity reasons, I would advocate for the TLV extensions being at least 3 characters long (e.g. -sec1 instead of -s1). I updated the TLV extension proposal above to mention this.

These TLV extensions should then also encompass other currently scattered and weirdly non-unified ad-hoc capabalities of Semver 2.0.0 (e.g. pre-release denotation with -pre1, build metadata denotation with -build1, etc.).

<!-- gh-comment-id:236510229 --> @dumblob commented on GitHub (Aug 1, 2016): With regards to the (non)development of Semver in the past years and for clarity reasons, I would advocate for the TLV extensions being **at least 3 characters** long (e.g. `-sec1` instead of `-s1`). I updated the TLV extension proposal above to mention this. These TLV extensions should then also encompass other currently scattered and weirdly non-unified ad-hoc capabalities of Semver 2.0.0 (e.g. pre-release denotation with `-pre1`, build metadata denotation with `-build1`, etc.).
Author
Owner

@FichteFoll commented on GitHub (Aug 1, 2016):

You should separate the number from the letters with a dot so they will be properly ordered. E.g. -pre.1.

<!-- gh-comment-id:236564668 --> @FichteFoll commented on GitHub (Aug 1, 2016): You should separate the number from the letters with a dot so they will be properly ordered. E.g. `-pre.1`.
Author
Owner

@dumblob commented on GitHub (Aug 1, 2016):

@FichteFoll well, I don't see any issue with ordering with either of -pre1 and -pre.1, but I'm not opposed to the suggestion. In my initial proposal I just didn't want to separate the number from the type just because it seems unnecessary and thus saves one character space.

<!-- gh-comment-id:236573116 --> @dumblob commented on GitHub (Aug 1, 2016): @FichteFoll well, I don't see any issue with ordering with either of `-pre1` and `-pre.1`, but I'm not opposed to the suggestion. In my initial proposal I just didn't want to separate the number from the _type_ just because it seems unnecessary and thus saves one character space.
Author
Owner

@FichteFoll commented on GitHub (Aug 1, 2016):

@dumblob, you will start facing problems when you reach numbers with more than one digit. -pre12 has lower precedence than -pre2 for example.

<!-- gh-comment-id:236643056 --> @FichteFoll commented on GitHub (Aug 1, 2016): @dumblob, you will start facing problems when you reach numbers with more than one digit. `-pre12` has lower precedence than `-pre2` for example.
Author
Owner

@dumblob commented on GitHub (Aug 1, 2016):

@FichteFoll in case we'll want to use lexicographical comparison, then yes. In case e.g. the pull request https://github.com/mojombo/semver/pull/206 will get merged in one form or another, then no.

Therefore I would leave it as an open question to be solved for the next major release (3.0.0?) of Semver.

<!-- gh-comment-id:236682481 --> @dumblob commented on GitHub (Aug 1, 2016): @FichteFoll in case we'll want to use lexicographical comparison, then yes. In case e.g. the pull request https://github.com/mojombo/semver/pull/206 will get merged in one form or another, then no. Therefore I would leave it as an open question to be solved for the next major release (3.0.0?) of Semver.
Author
Owner

@ameenross commented on GitHub (Oct 13, 2016):

Wouldn't someone who is on 1.0.1 just update to 1.0.3, skipping 1.0.2, when 1.0.3 is marked as a security release and an announcement made in a security bulletin? I don't get the need for extra stuff to be encoded in the version number, it's just overkill in this case.

<!-- gh-comment-id:253443316 --> @ameenross commented on GitHub (Oct 13, 2016): Wouldn't someone who is on `1.0.1` just update to `1.0.3`, skipping `1.0.2`, when `1.0.3` is marked as a security release and an announcement made in a security bulletin? I don't get the need for extra stuff to be encoded in the version number, it's just overkill in this case.
Author
Owner

@dumblob commented on GitHub (Oct 17, 2016):

@ameenross the reason for the overhead is that there are more use cases for version diversification (backporting, security fixes, non-linear development, HW-architecture-dependent versions of other mainline versions, released testing versions, etc.). If we used version skipping, we would end up with tens of different skipping/encoding schemes just to accommodate all the use cases we don't even know about yet. In other words, we would end up with arbitrary-sized hashes (unique identification would be a concatenation of the version + bulletin explanation paragraph), i.e. lengthy, cluttered and cofusing human-unfriendly versioning. And that is in direct contrast to why SemVer exists - i.e. solely for humans to immediately see what they are working with (for machines it's enough to have any unique hash).

All the stuff I described in my posts in this thread is used (in different forms) on a daily basis for multi-project handling on many places - Linux distributions packaging, npm, CPAN, RubyGems, PyPI, Maven Central Repository, GNU Guix, etc. All these have pretty complicated global versioning schemes developed over decades just to accommodate all the existing and future variations.

Keep in mind SemVer is not targetted only at one project, because we certainly don't want to have inconsistent versioning for upstream, for distribution of upstream packages (think of Fedora, Debian, etc. - all have strict rules about patching which is not related to upstreams), for deployments (e.g. your very own admins in your corporation patch already patched packages and build their own derived versions), for internal testing teams, for external testing teams, etc. All this must be fully consistent and extensible to be human-friendly and clear.

Just study for example all the details of the Debian packaging and try to maintain few non-trivial and important bigger packages for two years or more and you'll understand exactly what I'm talking about.

<!-- gh-comment-id:254178498 --> @dumblob commented on GitHub (Oct 17, 2016): @ameenross the reason for the overhead is that there are more use cases for version diversification (backporting, security fixes, non-linear development, HW-architecture-dependent versions of other mainline versions, released testing versions, etc.). If we used version skipping, we would end up with tens of different skipping/encoding schemes just to accommodate all the use cases we don't even know about yet. In other words, we would end up with arbitrary-sized hashes (unique identification would be a concatenation of the version + bulletin explanation paragraph), i.e. lengthy, cluttered and cofusing human-unfriendly versioning. And that is in direct contrast to why SemVer exists - i.e. solely for humans to immediately see what they are working with (for machines it's enough to have any unique hash). All the stuff I described in my posts in this thread is used (in different forms) on a daily basis for multi-project handling on many places - Linux distributions packaging, npm, CPAN, RubyGems, PyPI, Maven Central Repository, GNU Guix, etc. All these have pretty **complicated** global versioning schemes developed over decades just to accommodate all the existing and future variations. Keep in mind SemVer is not targetted only at one project, because we certainly don't want to have inconsistent versioning for upstream, for distribution of upstream packages (think of Fedora, Debian, etc. - all have strict rules about patching which is not related to upstreams), for deployments (e.g. your very own admins in your corporation patch already patched packages and build their own derived versions), for internal testing teams, for external testing teams, etc. All this must be fully consistent and extensible to be human-friendly and clear. Just study for example all the details of the Debian packaging and try to maintain few non-trivial and important bigger packages for two years or more and you'll understand exactly what I'm talking about.
Author
Owner

@ameenross commented on GitHub (Oct 17, 2016):

@dumblob I'm not sure how you interpreted skipping, but I was referring to your example where someone was on 1.0.1 where 1.0.2 has already been released.

Debian packaging

Right, so that "someone" who is on 1.0.1 in your example is the Debian package. Upstream creates a security release, 1.0.3 or whatever, and Debian doesn't want to update to that version, but only patch their version 1.0.1, so they need to backport the patch. I know this happens in practice, but those projects don't really use Semver to begin with. If they did, they wouldn't need to hold off shipping 1.0.3 (which in Semver would be perfectly backward compatible) which simply includes the security patch, unless for some reason they don't trust that version.

for machines it's enough to have any unique hash

Not necessarily. Dependency management (>= 1.4.3) can't work with hashes.

<!-- gh-comment-id:254189516 --> @ameenross commented on GitHub (Oct 17, 2016): @dumblob I'm not sure how you interpreted skipping, but I was referring to your example where someone was on `1.0.1` where `1.0.2` has already been released. > Debian packaging Right, so that "someone" who is on `1.0.1` in your example is the Debian package. Upstream creates a security release, `1.0.3` or whatever, and Debian doesn't want to update to that version, but only patch their version `1.0.1`, so they need to backport the patch. I know this happens in practice, but those projects don't really use Semver to begin with. If they did, they wouldn't need to hold off shipping `1.0.3` (which in Semver would be perfectly backward compatible) which simply includes the security patch, unless for some reason they don't trust that version. > for machines it's enough to have any unique hash Not necessarily. Dependency management (`>= 1.4.3`) can't work with hashes.
Author
Owner

@dumblob commented on GitHub (Oct 17, 2016):

@dumblob I'm not sure how you interpreted skipping, but I was referring to your example where someone was on 1.0.1 where 1.0.2 has already been released.

This will not work at all e.g. because of planning. The software is developed according to some plan - new features are added in (from our point of view) random order, these future versions are internally tested being labeled as the future version and suddenly the security bug is found. Noone will change the release plans and versioning and clear the whole infrastructure from the current new versions just because of some security bug which will need the version numbers.

It's also nearly impossible to track information about application of the security bug across all releases (especially those in production). Another issue with skipping is the inability to use just simple operators < >= > >= for dependency specification, because of no option to specify which version ranges to skip. And this is extremely important for production releases which usually specify the highest compatible version as dependency. These dependencies are then allowed to receive security fixes, but are not allowed to increase the version number, because of too high risk, that there will be somewhere the version 1.0.2 available instead of 1.0.3 (e.g. because of delay in propagation of newly pushed packages to all mirrors) and this version 1.0.2 is dangerous for whatever reason.

This ad-hoc solution of skipping not-in-production versions and just releasing a newer version with the security fix greatly complicates all the versioning even if it doesn't look like so at the first glance.

I know this happens in practice, but those projects don't really use Semver to begin with. If they did, they wouldn't need to hold off shipping 1.0.3 (which in Semver would be perfectly backward compatible) which simply includes the security patch, unless for some reason they don't trust that version.

This is theory. In practice noone cares about upstream in these cases, because they're simply not trusted in general (sorry to be so explicit). Disregarding who follows (and how quickly) SemVer, the end user (admin, packager, etc.) will do it the way I described above - will specify maximum version allowed including the last PATCH part.

Backporting, security fixing, etc. happens mainly on long running systems (e.g. 4 years without restarting) and these systems are not updated to any newer versions (disregarding if it's a backwards compatible release or not). It's just a rule of thumb - anything can go wrong and admins care extremely to have as minimal updates as possible. It's really a different world than the forward-looking development nowadays for which SemVer was originally designed.

Not necessarily. Dependency management (>= 1.4.3) can't work with hashes.

See e.g. the Continuity property of hashes, Extendible hashing or any other hashing with hierarchy.

<!-- gh-comment-id:254205949 --> @dumblob commented on GitHub (Oct 17, 2016): > @dumblob I'm not sure how you interpreted skipping, but I was referring to your example where someone was on 1.0.1 where 1.0.2 has already been released. This will not work at all e.g. because of planning. The software is developed according to some plan - new features are added in (from our point of view) random order, these future versions are internally tested being labeled as the future version and suddenly the security bug is found. Noone will change the release plans and versioning and clear the whole infrastructure from the current new versions just because of some security bug which will need the version numbers. It's also nearly impossible to track information about application of the security bug across all releases (especially those in production). Another issue with skipping is the inability to use just simple operators `<` `>=` `>` `>=` for dependency specification, because of no option to specify which version ranges to skip. And this is extremely important for production releases which usually specify the highest compatible version as dependency. These dependencies are then allowed to receive security fixes, but are not allowed to increase the version number, because of too high risk, that there will be somewhere the version `1.0.2` available instead of `1.0.3` (e.g. because of delay in propagation of newly pushed packages to all mirrors) and this version `1.0.2` is dangerous for whatever reason. This ad-hoc solution of skipping not-in-production versions and just releasing a newer version with the security fix greatly complicates all the versioning even if it doesn't look like so at the first glance. > I know this happens in practice, but those projects don't really use Semver to begin with. If they did, they wouldn't need to hold off shipping 1.0.3 (which in Semver would be perfectly backward compatible) which simply includes the security patch, unless for some reason they don't trust that version. This is theory. In practice noone cares about upstream in these cases, because they're simply not trusted in general (sorry to be so explicit). Disregarding who follows (and how quickly) SemVer, the end user (admin, packager, etc.) will do it the way I described above - will specify maximum version allowed including the last PATCH part. Backporting, security fixing, etc. happens mainly on long running systems (e.g. 4 years without restarting) and these systems are not updated to any newer versions (disregarding if it's a backwards compatible release or not). It's just a rule of thumb - anything can go wrong and admins care extremely to have as minimal updates as possible. It's really a different world than the forward-looking development nowadays for which SemVer was originally designed. > Not necessarily. Dependency management (>= 1.4.3) can't work with hashes. See e.g. the [Continuity](https://en.wikipedia.org/wiki/Hash_function#Continuity) property of hashes, [Extendible hashing](https://en.wikipedia.org/wiki/Extendible_hashing) or any other hashing with hierarchy.
Author
Owner

@ameenross commented on GitHub (Oct 17, 2016):

@dumblob Thanks, that makes the situation you were describing much clearer. Though I wonder, for long running systems aren't there already methods to hot-patch software, including the Linux kernel itself? Wouldn't that normally be a more desirable situation than having to manually patch your entire software stack (I get that that is a business case, but that's besides the point)?

<!-- gh-comment-id:254252423 --> @ameenross commented on GitHub (Oct 17, 2016): @dumblob Thanks, that makes the situation you were describing much clearer. Though I wonder, for long running systems aren't there already methods to hot-patch software, including the Linux kernel itself? Wouldn't that normally be a more desirable situation than having to manually patch your entire software stack (I get that that is a business case, but that's besides the point)?
Author
Owner

@crazedsanity commented on GitHub (Oct 18, 2016):

@dumblob It seems like you're trying to find a way to circumvent rules put in place by people/organizations that are unwilling to update regularly. It seems to me to be less of a development issue and more of an understanding and/or acceptance issue.

Planning major (X) and even minor (Y) version number releases in advance is fine. The patch version (Z) should be reserved for security and bug fixes, which gets updated only when there's something to fix.

Applying this to your situation:

1.0.0
        1.0.1 - in production
        1.0.2
        1.0.3 - X1 fixed ***
    1.1.0 - in production
        1.1.1
        1.1.2 - X1 fixed ***
    1.2.0
        1.2.1 - X1 fixed ***
2.0.0 - in production
    2.1.0
       2.1.1 - X1 fixed ***
    2.2.0 - in production
        2.2.1
        2.2.2 - X1 fixed ***
    2.3.0 - security bug found ("X1"), but applicable to the whole version history
        2.3.1 - X1 fixed ***

For every single MAJOR + MINOR version supported, a new release is created with an incremented PATCH version, wherein "X1" (a particular security bug) is fixed.

If you don't want to have that many versions fixed, drop support for them. My preference in these situations is to fully support the very latest MAJOR version up to the last two MINOR releases: in this case, 2.2.x and 2.3.x. The lastest MAJOR+MINOR release of the last version gets security fixes, so 1.2.x.

If you are unable to use the PATCH version for what it is meant for, you'll probably have to resort to doing hot fixes, as indicated by @ameenross

<!-- gh-comment-id:254618376 --> @crazedsanity commented on GitHub (Oct 18, 2016): @dumblob It seems like you're trying to find a way to circumvent rules put in place by people/organizations that are unwilling to update regularly. It seems to me to be less of a development issue and more of an understanding and/or acceptance issue. Planning major (X) and even minor (Y) version number releases in advance is fine. The patch version (Z) should be reserved for security and bug fixes, which gets updated only when there's something to fix. Applying this to your situation: ``` 1.0.0 1.0.1 - in production 1.0.2 1.0.3 - X1 fixed *** 1.1.0 - in production 1.1.1 1.1.2 - X1 fixed *** 1.2.0 1.2.1 - X1 fixed *** 2.0.0 - in production 2.1.0 2.1.1 - X1 fixed *** 2.2.0 - in production 2.2.1 2.2.2 - X1 fixed *** 2.3.0 - security bug found ("X1"), but applicable to the whole version history 2.3.1 - X1 fixed *** ``` For every single MAJOR + MINOR version supported, a new release is created with an incremented PATCH version, wherein "`X1`" (a particular security bug) is fixed. If you don't want to have that many versions fixed, drop support for them. My preference in these situations is to fully support the very latest MAJOR version up to the last two MINOR releases: in this case, `2.2.x` and `2.3.x`. The lastest MAJOR+MINOR release of the last version gets security fixes, so `1.2.x`. If you are unable to use the PATCH version for what it is meant for, you'll probably have to resort to doing hot fixes, as indicated by @ameenross
Author
Owner

@dumblob commented on GitHub (Nov 17, 2016):

@dumblob It seems like you're trying to find a way to circumvent rules put in place by people/organizations that are unwilling to update regularly. It seems to me to be less of a development issue and more of an understanding and/or acceptance issue.

It's not circumventing. It's purely technically convenient solution greatly minimizing the overall risk, budget, time, etc. (two patches at once generate exponentially more test cases and potential risk combinations than one patch). Using your own words "it seems to me to be less of a development issue and more of an understanding and/or acceptance issue of SemVer people, that there is also a backwards-looking world, which basically runs and backs up the whole forward-looking world".

The patch version (Z) should be reserved for security and bug fixes, which gets updated only when there's something to fix
...
If you are unable to use the PATCH version for what it is meant for, you'll probably have to resort to doing hot fixes, as indicated by @ameenross

First, your example did not satisfy the conditions I outlined. Namely, you provided the production environments not only with the one particular fix for the "X1" security bug. But a bunch of others, unknown, untested (in the particular production environments), and unnecessary patches being added to preceding versions (e.g. to 1.0.2 1.1.1 2.2.1). I also can't see you provided a version for a fixed 2.0.0 (did you omit it for some reason?).

@ameenross what do you mean by hot fixing with regards to versioning? According to my experience and knowledge, hot fixing exactly matches the issue I outlined in the example above and therefore hot fixing is a subject to the same SemVer TLV extension I proposed.

I think we should more focus on finding universal and KISS solutions to existing problems which are expected to stay with us for another few decades rather than trying to teach the existing world how to avoid such problems in the first place.

Do you have any other ideas how to match the current wide spread requirements (which I tried to describe throughout my posts) with extending SemVer? If so, put them here for discussion.

<!-- gh-comment-id:261215328 --> @dumblob commented on GitHub (Nov 17, 2016): @dumblob It seems like you're trying to find a way to circumvent rules put in place by people/organizations that are unwilling to update regularly. It seems to me to be less of a development issue and more of an understanding and/or acceptance issue. It's not circumventing. It's purely technically convenient solution greatly minimizing the overall risk, budget, time, etc. (two patches at once generate exponentially more test cases and potential risk combinations than one patch). Using your own words "it seems to me to be less of a development issue and more of an understanding and/or acceptance issue of SemVer people, that there is also a backwards-looking world, which basically runs and backs up the whole forward-looking world". > The patch version (Z) should be reserved for security and bug fixes, which gets updated only when there's something to fix > ... > If you are unable to use the PATCH version for what it is meant for, you'll probably have to resort to doing hot fixes, as indicated by @ameenross First, your example did not satisfy the conditions I outlined. Namely, you provided the production environments not only with the one particular fix for the "X1" security bug. But a bunch of others, unknown, untested (in the particular production environments), and unnecessary patches being added to preceding versions (e.g. to `1.0.2` `1.1.1` `2.2.1`). I also can't see you provided a version for a fixed `2.0.0` (did you omit it for some reason?). @ameenross what do you mean by hot fixing with regards to versioning? According to my experience and knowledge, hot fixing exactly matches the issue I outlined in the [example above](https://github.com/mojombo/semver/issues/163#issuecomment-192662245) and therefore hot fixing is a subject to the same SemVer TLV extension I proposed. I think we should more focus on finding universal and KISS solutions to existing problems which are expected to stay with us for another few decades rather than trying to teach the existing world how to avoid such problems in the first place. Do you have any other ideas how to match the current wide spread requirements (which I tried to describe throughout my posts) with extending SemVer? If so, put them here for discussion.
Author
Owner

@ameenross commented on GitHub (Nov 17, 2016):

two patches at once generate exponentially more test cases and potential risk combinations than one patch

Sure, but who is the one doing the patching? The upstream developer or the package maintainer / distro provider? If it's the upstream developer, they should just develop patches and release versions of their software they choose to maintain and support.
If it's the package maintainer / distro, then do you even care what versioning scheme is used by upstream if you're that conservative? They could be using some custom versioning scheme (yay, 1.1.4.12.9.3.5) and you would still do the same thing: manually applying single patches. So from that perspective, IMHO, this doesn't seem like a problem that Semver should fix.

@ameenross what do you mean by hot fixing with regards to versioning?

I was talking about hot-patching, not hot-fixing. You were talking about long-running systems, I interpreted that to mean continuously running without reboot. Not sure if that's what you meant though.

I think we should more focus on finding universal and KISS solutions to existing problems which are expected to stay with us for another few decades rather than trying to teach the existing world how to avoid such problems in the first place.

Nobody can disagree with that. But we need to clearly identify the use-case. I think the workflow you're describing is not limited to Semver-versioned upstream. So while I agree that there should be a clear convention, I'm not convinced that this should be covered by Semver.

By the way, isn't there already a convention? For example, I see packages versioned 2.1.4-2.1+deb8u2 all the time. I don't know if these conventions are practical or anything though. Also: http://serverfault.com/a/604549/98184

<!-- gh-comment-id:261239871 --> @ameenross commented on GitHub (Nov 17, 2016): > two patches at once generate exponentially more test cases and potential risk combinations than one patch Sure, but who is the one doing the patching? The upstream developer or the package maintainer / distro provider? If it's the upstream developer, they should just develop patches and release versions of their software _they_ choose to maintain and support. If it's the package maintainer / distro, then do you even care what versioning scheme is used by upstream if you're that conservative? _They_ could be using some custom versioning scheme (yay, `1.1.4.12.9.3.5`) and _you_ would still do the same thing: manually applying single patches. So from that perspective, IMHO, this doesn't seem like a problem that Semver should fix. > @ameenross what do you mean by hot fixing with regards to versioning? I was talking about [hot-patching](https://en.wikipedia.org/wiki/Dynamic_software_updating), not hot-fixing. You were talking about long-running systems, I interpreted that to mean continuously running without reboot. Not sure if that's what you meant though. > I think we should more focus on finding universal and KISS solutions to existing problems which are expected to stay with us for another few decades rather than trying to teach the existing world how to avoid such problems in the first place. Nobody can disagree with that. But we need to clearly identify the use-case. I think the workflow you're describing is not limited to Semver-versioned upstream. So while I agree that there should be a clear convention, I'm not convinced that this should be covered by Semver. By the way, isn't there already a convention? For example, I see packages versioned [`2.1.4-2.1+deb8u2`](https://packages.debian.org/jessie/vcs/git) all the time. I don't know if these conventions are practical or anything though. Also: http://serverfault.com/a/604549/98184
Author
Owner

@dumblob commented on GitHub (Nov 22, 2016):

Sure, but who is the one doing the patching? The upstream developer or the package maintainer / distro provider? If it's the upstream developer, they should just develop patches and release versions of their software they choose to maintain and support.
If it's the package maintainer / distro, then do you even care what versioning scheme is used by upstream if you're that conservative? They could be using some custom versioning scheme (yay, 1.1.4.12.9.3.5) and you would still do the same thing: manually applying single patches. So from that perspective, IMHO, this doesn't seem like a problem that Semver should fix.

Of course, different SW distributions (not talking just about Linux distributions, but distribution of SW in general - including CPAN, PyPI, npm, Docker Hub, PkgBase, anything...) use different strategies of versioning. Some SW distributions are closer to or even "the same" as the upstream developers (developer develops "directly" in the user-facing repository, i.e. pushes there stable releases), some are way too far from upstream SW distribution. Then there are often overlapping repositories (upstreams provide and maintain few releases for different SW distribution systems, but not for all, which results in having 2 or more same versions available, but with potentially different content). And so on. Inconsistency, ambiguity, etc. everywhere.

I agree, that Semver was originally meant just for forward-looking upstream development. With the advent of the SW distribution systems, which are directly facing consumers, the need for consistency between upstream versioning and consumer-facing versioning scheme significantly increased and is still rapidly growing.

So, the question seems to be: "How much does Semver want to be compatible with current and future SW world instead of being just another versioning scheme to be skipped by SW distributions due to it's large incompatibility with the consumer needs (even though satisfying developer needs)?"

I was talking about hot-patching, not hot-fixing. You were talking about long-running systems, I interpreted that to mean continuously running without reboot. Not sure if that's what you meant though.

Yes, I meant hot-patching as well. The fact, that it's a patched running version needs to be designated somewhere in runtime though - usually with a version showing other meaningful value than the originally non-patched one.

So while I agree that there should be a clear convention, I'm not convinced that this should be covered by Semver.

Let's talk about this as it seems to be the very core of all the proposals trying to extend Semver. In my opinion, Semver should either

  1. directly define the semantics as I demonstrated in the extension proposal https://github.com/mojombo/semver/issues/163#issuecomment-192662245 covering all possible cases (and thus maintain an official database of keywords of particular TLV extensions as mentioned in https://github.com/mojombo/semver/issues/163#issuecomment-236510229 )

  2. or explicitly state what and why is not covered in Semver with a reference to a Semver-compatible initiative covering these extensions (which might or might not be based on the proposal https://github.com/mojombo/semver/issues/163#issuecomment-192662245 ) to ensure Semver counts on consumer-facing SW distribution, but wants to stay minimal through focusing solely on forward-looking upstream development

By the way, isn't there already a convention? For example, I see packages versioned 2.1.4-2.1+deb8u2 all the time. I don't know if these conventions are practical or anything though. Also: http://serverfault.com/a/604549/98184

No, this is very specific convention of one of the many existing SW distribution systems covering many, but definitely not all the needed cases.

<!-- gh-comment-id:262185925 --> @dumblob commented on GitHub (Nov 22, 2016): >Sure, but who is the one doing the patching? The upstream developer or the package maintainer / distro provider? If it's the upstream developer, they should just develop patches and release versions of their software they choose to maintain and support. >If it's the package maintainer / distro, then do you even care what versioning scheme is used by upstream if you're that conservative? They could be using some custom versioning scheme (yay, 1.1.4.12.9.3.5) and you would still do the same thing: manually applying single patches. So from that perspective, IMHO, this doesn't seem like a problem that Semver should fix. Of course, different SW distributions (not talking just about Linux distributions, but distribution of SW in general - including CPAN, PyPI, npm, Docker Hub, PkgBase, anything...) use different strategies of versioning. Some SW distributions are closer to or even "the same" as the upstream developers (developer develops "directly" in the user-facing repository, i.e. pushes there stable releases), some are way too far from upstream SW distribution. Then there are often overlapping repositories (upstreams provide and maintain few releases for different SW distribution systems, but not for all, which results in having 2 or more same versions available, but with potentially different content). And so on. Inconsistency, ambiguity, etc. everywhere. I agree, that Semver was originally meant just for forward-looking upstream development. With the advent of the SW distribution systems, which are directly facing consumers, the need for consistency between upstream versioning and consumer-facing versioning scheme significantly increased and is still rapidly growing. So, the question seems to be: "How much does Semver want to be compatible with current and future SW world instead of being just another versioning scheme to be skipped by SW distributions due to it's large incompatibility with the consumer needs (even though satisfying developer needs)?" >I was talking about hot-patching, not hot-fixing. You were talking about long-running systems, I interpreted that to mean continuously running without reboot. Not sure if that's what you meant though. Yes, I meant hot-patching as well. The fact, that it's a patched running version needs to be designated somewhere in runtime though - usually with a version showing other meaningful value than the originally non-patched one. >So while I agree that there should be a clear convention, I'm not convinced that this should be covered by Semver. Let's talk about this as it seems to be the very core of **all** the proposals trying to extend Semver. In my opinion, Semver should either 1. directly define the semantics as I demonstrated in the extension proposal https://github.com/mojombo/semver/issues/163#issuecomment-192662245 covering all possible cases (and thus maintain an official database of keywords of particular TLV extensions as mentioned in https://github.com/mojombo/semver/issues/163#issuecomment-236510229 ) 1. or explicitly state what and why is not covered in Semver with a reference to a Semver-compatible initiative covering these extensions (which might or might not be based on the proposal https://github.com/mojombo/semver/issues/163#issuecomment-192662245 ) to ensure Semver counts on consumer-facing SW distribution, but wants to stay minimal through focusing solely on forward-looking upstream development >By the way, isn't there already a convention? For example, I see packages versioned 2.1.4-2.1+deb8u2 all the time. I don't know if these conventions are practical or anything though. Also: http://serverfault.com/a/604549/98184 No, this is very specific convention of one of the many existing SW distribution systems covering many, but definitely not all the needed cases.
Author
Owner

@ameenross commented on GitHub (Nov 23, 2016):

No, this is very specific convention of one of the many existing SW distribution systems covering many, but definitely not all the needed cases.

Do you expect 1.2.3-sec1 distributed by, say, Debian to be the same version as 1.2.3-sec1 distributed by, say, Arch? I don't see how you could enforce that of even have a reasonable expectation of that. It seems to me that your suggestion to solve the inconsistency you're describing just leads to other inconsistencies.

<!-- gh-comment-id:262465103 --> @ameenross commented on GitHub (Nov 23, 2016): > No, this is very specific convention of one of the many existing SW distribution systems covering many, but definitely not all the needed cases. Do you expect `1.2.3-sec1` distributed by, say, Debian to be the same version as `1.2.3-sec1` distributed by, say, Arch? I don't see how you could enforce that of even have a reasonable expectation of that. It seems to me that your suggestion to solve the inconsistency you're describing just leads to other inconsistencies.
Author
Owner

@dumblob commented on GitHub (Dec 22, 2016):

Do you expect 1.2.3-sec1 distributed by, say, Debian to be the same version as 1.2.3-sec1 distributed by, say, Arch?

Of course I do (actually rather 1.2.3-sec1-deb2:2.2.0 and 1.2.3-sec1-arch2016:2016.9.3). That's the ultimate goal for the future!

All major BSD and Linux distributions are trying hard to have all builds reproducible (basically by splitting the package into two parts - application and metadata). This is another huge step towards unified versions (in this case even bit-perfect).

For versioning of user-facing SW packages it means, the "main" version should be just the version of the main package target (usually an application, but might be arbitrary data) as versioned by upstream. And there should be also a version of the distribution-specific metadata (e.g. -deb2:2.2.0 arch2016:2016.9.3 from above; note the metadata version should be on par with the distribution release for which the package was prepared).

But for making it crystal-clear to users what is any particular package all about, the most basic foundation - versioning - needs to be laid for even allowing this initiative to start in a large scale and not only on per-repository basis like e.g. the reproducible builds currently.

So, now back to the question about the convention/standard as mentioned in https://github.com/mojombo/semver/issues/163#issuecomment-261239871 and https://github.com/mojombo/semver/issues/163#issuecomment-262185925 . Any (other) ideas?

<!-- gh-comment-id:268788636 --> @dumblob commented on GitHub (Dec 22, 2016): >Do you expect 1.2.3-sec1 distributed by, say, Debian to be the same version as 1.2.3-sec1 distributed by, say, Arch? Of course I do (actually rather [`1.2.3-sec1-deb2:2.2.0`](https://www.debian.org/releases/ ) and [`1.2.3-sec1-arch2016:2016.9.3`](https://www.archlinux.org/releng/releases/ )). That's the ultimate goal for the future! All major BSD and Linux distributions are trying hard to have all [builds reproducible](https://reproducible-builds.org/ ) (basically by splitting the package into two parts - application and metadata). This is another huge step towards unified versions (in this case even bit-perfect). For versioning of user-facing SW packages it means, the "main" version should be just the version of the main package target (usually an application, but might be arbitrary data) as versioned by upstream. And there should be also a version of the distribution-specific metadata (e.g. `-deb2:2.2.0` `arch2016:2016.9.3` from above; note the metadata version should be on par with the distribution release for which the package was prepared). But for making it crystal-clear to users what is any particular package all about, the most basic foundation - versioning - needs to be laid for even allowing this initiative to start in a large scale and not only on per-repository basis like e.g. the reproducible builds currently. So, now back to the question about the convention/standard as mentioned in https://github.com/mojombo/semver/issues/163#issuecomment-261239871 and https://github.com/mojombo/semver/issues/163#issuecomment-262185925 . Any (other) ideas?
Author
Owner

@ameenross commented on GitHub (Dec 22, 2016):

So this just keeps getting more and more confusing. You said:

How much does Semver want to be compatible with current and future SW world instead of being just another versioning scheme to be skipped by SW distributions

So you want Semver to NOT be wrapped (with intrinsics ignored) by distributors. You want Semver to be something that won't be ignored. It seems to me that this would require the entire world to switch to Semver, but I digress. You contradict yourself because you said previously:

production machines must not (for stability reasons or missing official certifications etc.) just switch to a newer version of ABC (disregarding if it's the Z or Y or X version subtree)

If Semver is already followed, then Semver states that 1.1.3 is perfectly compatible with 1.1.0. So you are, at the same time, advocating ignoring and not ignoring Semver. Which is it?

<!-- gh-comment-id:268822216 --> @ameenross commented on GitHub (Dec 22, 2016): So this just keeps getting more and more confusing. You said: > How much does Semver want to be compatible with current and future SW world instead of being just another versioning scheme to be skipped by SW distributions So you want Semver to NOT be wrapped (with intrinsics ignored) by distributors. You want Semver to be something that won't be ignored. It seems to me that this would require the entire world to switch to Semver, but I digress. You contradict yourself because you said previously: > production machines must not (for stability reasons or missing official certifications etc.) just switch to a newer version of ABC (disregarding if it's the Z or Y or X version subtree) If Semver is already followed, then Semver states that `1.1.3` is perfectly compatible with `1.1.0`. So you are, at the same time, advocating *ignoring* and *not ignoring* Semver. Which is it?
Author
Owner

@jwdonahue commented on GitHub (Dec 3, 2017):

@gergelypolonkai, support policies for old versions are not in the remit of SemVer. SemVer specifies what semantic version strings look like and when to increment major, minor and patch. If you policy is back-port some bug fixes to earlier versions that still supported, bump the patch level.

Unless you intend to issue a PR for the proposed changes, please close this issue.

<!-- gh-comment-id:348732599 --> @jwdonahue commented on GitHub (Dec 3, 2017): @gergelypolonkai, support policies for old versions are not in the remit of SemVer. SemVer specifies what semantic version strings look like and when to increment major, minor and patch. If you policy is back-port some bug fixes to earlier versions that still supported, bump the patch level. Unless you intend to issue a PR for the proposed changes, please close this issue.
Author
Owner

@gergelypolonkai commented on GitHub (Dec 3, 2017):

Yes, you are right, just as I said in the last sentence, it's more like a developer policy. Thanks for clearing that up.

<!-- gh-comment-id:348742374 --> @gergelypolonkai commented on GitHub (Dec 3, 2017): Yes, you are right, just as I said in the last sentence, it's more like a developer policy. Thanks for clearing that up.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/semver#7080