Using 3rd party (non-semver) content in semver environments #385

Closed
opened 2026-02-17 11:58:49 -06:00 by GiteaMirror · 36 comments
Owner

Originally created by @ilg-ul on GitHub (Jun 17, 2019).

I know that the question seems a bit stupid, but I'm facing a problem when trying to integrate existing 3rd party content into a semver environment, and I thought that maybe others faced similar issues; perhaps an authorized advice from the gurus around may shed some light and possibly lead to some additions to future versions of the specs.

A real world example of this problem is a binary package that installs the ARM GCC toolchain:

https://github.com/gnu-mcu-eclipse/arm-none-eabi-gcc-xpack

The actual releases are:

https://www.npmjs.com/package/@gnu-mcu-eclipse/arm-none-eabi-gcc

The meaning of the version string (8.2.1-1.7.1) is:

  • 8.2.1 = the original FSF version
  • 8.2.1-1 = the first ARM release based on FSF 8.2.1
  • 8.2.1-1.7 = the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1
  • 8.2.1-1.7.1 = the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1 (ufff...)

What I basically did was to use the 'pre-release' part of the semver definition to encode additional version numbers.

As far as I understood the specs, this is syntactically correct, but may be considered abusive and thus I'm not very comfortable with it.

However I could not find a better solution. :-(

From a strictly semver compliant point of view, probably the solution would be to assign new semver versions and ignore the version used in the original packages, but I think this would simply increase confusion.

The problem can be more generally re-phrased: "how to assign semver versions when publishing 3rd party content (libraries/binaries/etc), which may or may not use semver themselves, and when more release numbers are needed to fully identify the release?".

In the above example, assuming I start numbering with FSF 8.2.1, ARM 1st release, one possible sequence would be:

  • 1.0.0 for 8.2.1-1.1.1 (the first NPM package based on the first GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1)
  • 1.0.1 for 8.2.1-1.1.2 (the second NPM release of the same 8.2.1-1.1)
  • 1.0.2 for 8.2.1-1.2.1 (the first NPM package based on the second GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1)
  • ...
  • 1.0.13 for 8.2.1-1.7.1 (the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1)
  • 1.1.0 for 8.2.1-2.1.1
  • 1.2.0 for 8.3.0-1.1.1
  • ...
  • 2.0.0 for 9.1.0-1.1.1
  • ...

Personally, if I had to use such a scheme for the well known GCC, I'd be very confused.

One possible way to reduce the confusion would be to append the original versions as 'build metadata', for example:

  • 1.0.0+8.2.1
  • 1.0.1+8.2.1
  • 1.0.2+8.2.1
  • ...
  • 1.0.13+8.2.1
  • 1.1.0+8.2.1
  • 1.2.0+8.3.0
  • ...
  • 2.0.0+9.1.0

However some of the information is lost (the last 3 digits in the initial scheme), and adding them to the already very long string would make it too long.

Personally I'm not happy with this solution either.

Any suggestions will be highly appreciated.

Liviu

Originally created by @ilg-ul on GitHub (Jun 17, 2019). I know that the question seems a bit stupid, but I'm facing a problem when trying to integrate existing 3rd party content into a semver environment, and I thought that maybe others faced similar issues; perhaps an authorized advice from the gurus around may shed some light and possibly lead to some additions to future versions of the specs. A real world example of this problem is a binary package that installs the ARM GCC toolchain: https://github.com/gnu-mcu-eclipse/arm-none-eabi-gcc-xpack The actual releases are: https://www.npmjs.com/package/@gnu-mcu-eclipse/arm-none-eabi-gcc The meaning of the version string (8.2.1-1.7.1) is: - 8.2.1 = the original FSF version - 8.2.1-1 = the first ARM release based on FSF 8.2.1 - 8.2.1-1.7 = the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1 - 8.2.1-1.7.1 = the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1 (ufff...) What I basically did was to use the 'pre-release' part of the semver definition to encode additional version numbers. As far as I understood the specs, this is syntactically correct, but may be considered abusive and thus I'm not very comfortable with it. However I could not find a better solution. :-( From a strictly semver compliant point of view, probably the solution would be to assign new semver versions and ignore the version used in the original packages, but I think this would simply increase confusion. The problem can be more generally re-phrased: "how to assign semver versions when publishing 3rd party content (libraries/binaries/etc), which may or may not use semver themselves, and when more release numbers are needed to fully identify the release?". In the above example, assuming I start numbering with FSF 8.2.1, ARM 1st release, one possible sequence would be: - 1.0.0 for 8.2.1-1.1.1 (the first NPM package based on the first GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1) - 1.0.1 for 8.2.1-1.1.2 (the second NPM release of the same 8.2.1-1.1) - 1.0.2 for 8.2.1-1.2.1 (the first NPM package based on the second GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1) - ... - 1.0.13 for 8.2.1-1.7.1 (the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1) - 1.1.0 for 8.2.1-2.1.1 - 1.2.0 for 8.3.0-1.1.1 - ... - 2.0.0 for 9.1.0-1.1.1 - ... Personally, if I had to use such a scheme for the well known GCC, I'd be very confused. One possible way to reduce the confusion would be to append the original versions as 'build metadata', for example: - 1.0.0+8.2.1 - 1.0.1+8.2.1 - 1.0.2+8.2.1 - ... - 1.0.13+8.2.1 - 1.1.0+8.2.1 - 1.2.0+8.3.0 - ... - 2.0.0+9.1.0 However some of the information is lost (the last 3 digits in the initial scheme), and adding them to the already very long string would make it too long. Personally I'm not happy with this solution either. Any suggestions will be highly appreciated. Liviu
GiteaMirror added the question label 2026-02-17 11:58:49 -06:00
Author
Owner

@ilg-ul commented on GitHub (Jun 19, 2019):

Any thoughts?

(later edit)

to summarize here are some further points that came during the discussion:

  • for 3rd party content I do not expect consumers to use semVer with advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax.
  • the semVer syntax is needed because when publishing packages in a semVer environment (like the npm registry) it is to be expected that packages with a non SemVer version field (in package.json for npm) will be rejected.
  • the original version string that the publisher released is not enough to identify the package (for example it is highly possible that ARM will release a 2nd package based on FSF 8.2.1, I may release subsequent packages if bugs are identified in ARMs release, and I may discover that I missed something in my npm packages and release new ones to fix this)
@ilg-ul commented on GitHub (Jun 19, 2019): Any thoughts? (later edit) to summarize here are some further points that came during the discussion: - for 3rd party content I do not expect consumers to use semVer with advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax. - the semVer syntax is needed because when publishing packages in a semVer environment (like the npm registry) it is to be expected that packages with a non SemVer version field (in package.json for npm) will be rejected. - the original version string that the publisher released is not enough to identify the package (for example it is highly possible that ARM will release a 2nd package based on FSF 8.2.1, I may release subsequent packages if bugs are identified in ARMs release, and I may discover that I missed something in my npm packages and release new ones to fix this)
Author
Owner

@raghukiranp commented on GitHub (Jul 16, 2019):

We also have a similar requirement where we want to certify one of the modules to a specific service which does not follow semVer.

what is the best way / recommended way to denote my package version 1.2.4 is certified against 20.1.2.3 version of the service... is metadata is the way or just README.md documentation?

@raghukiranp commented on GitHub (Jul 16, 2019): We also have a similar requirement where we want to certify one of the modules to a specific service which does not follow semVer. what is the best way / recommended way to denote my package version 1.2.4 is certified against 20.1.2.3 version of the service... is metadata is the way or just README.md documentation?
Author
Owner

@klehelley commented on GitHub (Jul 16, 2019):

While SemVer is very useful for a lot of common cases, it is not a one-size-fits-all. So I think in both your cases that you may be at the limit of what SemVer has to offer. There are potential ways to keep using SemVer however, that I'll try to explore here.

But the first question should be "Is SemVer a good fit for my project? What does it bring?". @ilg-ul I am not aware of the versioning strategy in use for GCC, but is it SemVer-compatible to begin with? That is, does it make sense that moving from 8.2.1 to 8.2.2 should be painless, while moving from 8.2.1 to 9.0.0 should require an analysis of the incompatibilities and time to perform the migration? If not, even ignoring the added information in the pre-release part of the version string, it is not really following SemVer. Which is not necessarily a bad thing if it has good reasons not to.

Now, if you really want to follow SemVer and you want to be able to release at a different rhythm than GCC does, the possible sequence you explore in the middle of your message would be the way to go. But like you said, it might be confusing to some users. One way to alleviate that would be to document in a very obvious manner which version of GCC, which ARM release, which MCU Eclipse release and which NPM package it corresponds to. Think something like putting a table or a list at the very to of the README of the project, on the download page, in the changelog and maybe even a summary of the versions used throughout the releases in a wiki or something. It may sound like work, but using SemVer does not mean one shouldn't also provide good enough documentation for a project.

If you keep the first versioning scheme you presented, you can document that is is based on SemVer, and add that it uses the pre-release part in a specific way for final releases, how the version string for actual pre-releases would look like...


@raghukiranp your case to me seems a little simpler to reason about. Putting the certification information in a prominent way in the documentation looks like the way to go to me. Also, could it be that a given version of your package can be certified against several versions of the service you are talking about at some point? If that is the case, encoding that information in the version string is definitely not the way to go in my opinion.

@klehelley commented on GitHub (Jul 16, 2019): While SemVer is very useful for a lot of common cases, it is not a one-size-fits-all. So I think in both your cases that you may be at the limit of what SemVer has to offer. There are potential ways to keep using SemVer however, that I'll try to explore here. But the first question should be "Is SemVer a good fit for my project? What does it bring?". @ilg-ul I am not aware of the versioning strategy in use for GCC, but is it SemVer-compatible to begin with? That is, does it make sense that moving from `8.2.1` to `8.2.2` should be painless, while moving from `8.2.1` to `9.0.0` should require an analysis of the incompatibilities and time to perform the migration? If not, even ignoring the added information in the pre-release part of the version string, it is not really following SemVer. Which is not necessarily a bad thing if it has good reasons not to. Now, if you really want to follow SemVer and you want to be able to release at a different rhythm than GCC does, the possible sequence you explore in the middle of your message would be the way to go. But like you said, it might be confusing to some users. One way to alleviate that would be to document in a very obvious manner which version of GCC, which ARM release, which MCU Eclipse release and which NPM package it corresponds to. Think something like putting a table or a list at the very to of the README of the project, on the download page, in the changelog and maybe even a summary of the versions used throughout the releases in a wiki or something. It may sound like work, but using SemVer does not mean one shouldn't also provide good enough documentation for a project. If you keep the first versioning scheme you presented, you can document that is is based on SemVer, and add that it uses the pre-release part in a specific way for final releases, how the version string for actual pre-releases would look like... --- @raghukiranp your case to me seems a little simpler to reason about. Putting the certification information in a prominent way in the documentation looks like the way to go to me. Also, could it be that a given version of your package can be certified against several versions of the service you are talking about at some point? If that is the case, encoding that information in the version string is definitely not the way to go in my opinion.
Author
Owner

@ilg-ul commented on GitHub (Jul 16, 2019):

Hi Kevin @klehelley,

Thank you for the explanation.

the first question should be "Is SemVer a good fit for my project? What does it bring?"

Yes, this is a good question.

In my opinion SemVer is a smart and elegant solution, and, generally speaking, I think that designing a packaging system that uses it is a fair choice. npm is probably the most successful such packaging system that uses SemVer, but let's keep the discussion generic, the same problem occurs with any packaging system that uses SemVer. Let's call this fictious packaging system xPack.

As long as original content is published using this scheme, everything is fine, the existing SemVer syntax and semantics have no problem to define the dependencies and the definitions are reasonably complex.

The problem occurs as soon as we try to publish 3rd party content, and there are several possible cases:

  • the 3rd party content already uses SemVer
  • the 3rd party content does not use SemVer, but uses a 2-3 number system, following a proprietary semantic
  • the 3rd party content does not use SemVer, and uses non-numeric version ids (like 4.12d).

The third case is relatively rare, and, to simplify things, I would ignore it for now.

The first case seems simple, but it is not; republishing 3rd party content requires adding at least a new number and SemVer allows only 3 of them.

The second case is somehow similar, syntactically the original numbers (let's assume there are 3 of them, the most common case) can be represented by the 3 SemVer numbers, but the problem is the same, we need at least a 4th number.

To make things worse, republishing 3rd party content can be chained, and cases like the 1st xPack package based on the 7th GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1 can be encountered.

So, coming back to your question, assuming the fictious xPack system already uses SemVer, it is equivalent to is it a good fit for 3rd party content?. Or, going up on the design chain, is SemVer a good choice for xPack? If not, what would be the alternate solutions?

The long term question would be is it possible to extend SemVer to cover such cases?

The short term question is, assuming the packaging system already exists and uses SemVer, how to accomodate the extra numbers?

As per my initial message, I could identify 2 possible workarounds:

  • use the 'pre-release' part
  • assign completely new numbers and append the original numbers as 'build metadata'

Both have drawbacks, the question is which is the least bad choice.

I am not aware of the versioning strategy in use for GCC

It is somehow similar to SemVer, but I would not go into specific details, it is just an example, and I would keep the discussion into the framework of the generic use case.

Now, if you really want to follow SemVer and you want to be able to release at a different rhythm than GCC does, the possible sequence you explore in the middle of your message would be the way to go. ... putting a table or a list at the very top of the README of the project

If I got it right, you mean using completely new numbers.

I seriously considered this choice, but, after a long analysis, I concluded that the confusion it adds is too high. In practical terms, it requires someone who looks at a list of dependencies to refer to the translation tables for each dependent project, which is not practical.

My initial workaround was to use the 'build metadata' to store this translation, but I rejected it, since the result would be a very long version string, like 1.2.3+8.2.1.1.7.1, which is a bit too much.

If you keep the first versioning scheme you presented, you can document that is is based on SemVer, and add that it uses the pre-release part in a specific way for final releases, how the version string for actual pre-releases would look like...

Yes, this was my conclusion too, using this scheme requires a small training for the users, to know how to refer to the versions when maintaining the list of dependencies, but I think this is relatively straightforward.

I just reorganised my projects and, if there are no further suggestions, I'll proceed and republish all my packages using the 'pre-release' workaround.

@ilg-ul commented on GitHub (Jul 16, 2019): Hi Kevin @klehelley, Thank you for the explanation. > the first question should be "Is SemVer a good fit for my project? What does it bring?" Yes, this is a good question. In my opinion SemVer is a smart and elegant solution, and, generally speaking, I think that designing a packaging system that uses it is a fair choice. npm is probably the most successful such packaging system that uses SemVer, but let's keep the discussion generic, the same problem occurs with any packaging system that uses SemVer. Let's call this fictious packaging system _xPack_. As long as original content is published using this scheme, everything is fine, the existing SemVer syntax and semantics have no problem to define the dependencies and the definitions are reasonably complex. The problem occurs as soon as we try to publish 3rd party content, and there are several possible cases: - the 3rd party content already uses SemVer - the 3rd party content does not use SemVer, but uses a 2-3 number system, following a proprietary semantic - the 3rd party content does not use SemVer, and uses non-numeric version ids (like 4.12d). The third case is relatively rare, and, to simplify things, I would ignore it for now. The first case seems simple, but it is not; republishing 3rd party content requires adding at least a new number and SemVer allows only 3 of them. The second case is somehow similar, syntactically the original numbers (let's assume there are 3 of them, the most common case) can be represented by the 3 SemVer numbers, but the problem is the same, we need at least a 4th number. To make things worse, republishing 3rd party content can be chained, and cases like _the 1st xPack package based on the 7th GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1_ can be encountered. So, coming back to your question, assuming the fictious xPack system already uses SemVer, it is equivalent to _is it a good fit for 3rd party content?_. Or, going up on the design chain, _is SemVer a good choice for xPack?_ If not, what would be the alternate solutions? The long term question would be _is it possible to extend SemVer to cover such cases?_ The short term question is, assuming the packaging system already exists and uses SemVer, _how to accomodate the extra numbers?_ As per my initial message, I could identify 2 possible workarounds: - use the 'pre-release' part - assign completely new numbers and append the original numbers as 'build metadata' Both have drawbacks, the question is which is the least bad choice. > I am not aware of the versioning strategy in use for GCC It is somehow similar to SemVer, but I would not go into specific details, it is just an example, and I would keep the discussion into the framework of the generic use case. > Now, if you really want to follow SemVer and you want to be able to release at a different rhythm than GCC does, the possible sequence you explore in the middle of your message would be the way to go. ... putting a table or a list at the very top of the README of the project If I got it right, you mean using completely new numbers. I seriously considered this choice, but, after a long analysis, I concluded that the confusion it adds is too high. In practical terms, it requires someone who looks at a list of dependencies to refer to the translation tables for each dependent project, which is not practical. My initial workaround was to use the 'build metadata' to store this translation, but I rejected it, since the result would be a very long version string, like `1.2.3+8.2.1.1.7.1`, which is a bit too much. > If you keep the first versioning scheme you presented, you can document that is is based on SemVer, and add that it uses the pre-release part in a specific way for final releases, how the version string for actual pre-releases would look like... Yes, this was my conclusion too, using this scheme requires a small training for the users, to know how to refer to the versions when maintaining the list of dependencies, but I think this is relatively straightforward. I just reorganised my projects and, if there are no further suggestions, I'll proceed and republish all my packages using the 'pre-release' workaround.
Author
Owner

@klehelley commented on GitHub (Jul 17, 2019):

Both workarounds seemed OK to me. Ultimately, it looks like you have very good reasons to choose the "pre-release" on so go for it. :) The way that is simpler to reason about for the users of your package (without being too simplistic) is in my opinion the way to go.

The long term question would be is it possible to extend SemVer to cover such cases?

In my opinion, it definitely is possible. However when that's the case, the changes or additions should be well-documented. That way the people reading and interpreting the version numbers do not have to make potentially wrong assumptions.

Both have drawbacks, the question is which is the least bad choice.

I'd turn that sentence on its head and instead ask: are they both "good enough"? Is one of them "good enough"? I think this way of thinking helps making that kind of choice without necessarily taking a lot of time to do so, not being paralysed when having to choose between what would be after analysis somewhat equal, valid choices. Obviously the difficulty here is defining what "good enough" is. As a bonus if for a given choice from a limited set of options none fits, it can also help drive the decision-taking process towards exploring other possibilities, instead of ending up settling for a solution that does not work in the end.

@klehelley commented on GitHub (Jul 17, 2019): Both workarounds seemed OK to me. Ultimately, it looks like you have very good reasons to choose the "pre-release" on so go for it. :) The way that is simpler to reason about for the users of your package (without being too simplistic) is in my opinion the way to go. > The long term question would be _is it possible to extend SemVer to cover such cases?_ In my opinion, it definitely is possible. However when that's the case, the changes or additions should be well-documented. That way the people reading and interpreting the version numbers do not have to make potentially wrong assumptions. > Both have drawbacks, the question is which is the least bad choice. I'd turn that sentence on its head and instead ask: are they both "good enough"? Is one of them "good enough"? I think this way of thinking helps making that kind of choice without necessarily taking a lot of time to do so, not being paralysed when having to choose between what would be after analysis somewhat equal, valid choices. Obviously the difficulty here is defining what "good enough" is. As a bonus if for a given choice from a limited set of options none fits, it can also help drive the decision-taking process towards exploring other possibilities, instead of ending up settling for a solution that does not work in the end.
Author
Owner

@ilg-ul commented on GitHub (Jul 17, 2019):

it looks like you have very good reasons to choose the "pre-release" on so go for it.

thank you for confirming this, I'll go ahead with it.

... extend SemVer to cover such cases? In my opinion, it definitely is possible. However when that's the case, the changes or additions should be well-documented.

any thoughts on how this might look like? simply removing the limitation to 3 numbers would be enough?

@ilg-ul commented on GitHub (Jul 17, 2019): > it looks like you have very good reasons to choose the "pre-release" on so go for it. thank you for confirming this, I'll go ahead with it. > ... extend SemVer to cover such cases? In my opinion, it definitely is possible. However when that's the case, the changes or additions should be well-documented. any thoughts on how this might look like? simply removing the limitation to 3 numbers would be enough?
Author
Owner

@klehelley commented on GitHub (Jul 18, 2019):

Something along the lines of:

Versioning scheme

This projects derives its version numbers from those of the GNU GCC project. It uses a versioning scheme derived from SemVer 2.0.0, with the following differences:

  • (differences between SemVer and GCC regarding the management of the first part of the version string)
  • The part following the hyphen (defined in SemVer 2.0.0, item 9 as denoting pre-release, potentially unstable version) is used in final release of this project with the following format : <ARM release>.<GNU MCU release>.<NPM package>. This project do not release packages with what the SemVer 2.0 specification defines as "normal version number".

As a consequence of these differences, SemVer-aware tools can be used to determine precedence between two different versions of this project. However, they should not be relied on to determine whether a version is stable (as they will report all versions of this project as unstable). (+ other caveats as applicable.)

Ideally that part of the documentation should stay short and to the point. I wonder if it wouldn't be simpler to actually mostly reference the versioning scheme used by GNU GCC, add a description of the way what's behind the hyphen is built, and in a single paragraph focus on the relationship between this versioning scheme and SemVer (similar to the last one in my example above, though slightly expanded).

@klehelley commented on GitHub (Jul 18, 2019): Something along the lines of: > ## Versioning scheme > > This projects derives its version numbers from those of the GNU GCC project. It uses a versioning scheme derived from [SemVer 2.0.0](https://semver.org/spec/v2.0.0.html), with the following differences: > > - _(differences between SemVer and GCC regarding the management of the first part of the version string)_ > - The part following the hyphen (defined in [SemVer 2.0.0, item 9](https://semver.org/spec/v2.0.0.html#spec-item-9) as denoting pre-release, potentially unstable version) is used in final release of this project with the following format : `<ARM release>.<GNU MCU release>.<NPM package>`. This project do not release packages with what the SemVer 2.0 specification defines as "normal version number". > > As a consequence of these differences, SemVer-aware tools can be used to determine precedence between two different versions of this project. However, they should not be relied on to determine whether a version is stable (as they will report all versions of this project as unstable). _(+ other caveats as applicable.)_ Ideally that part of the documentation should stay short and to the point. I wonder if it wouldn't be simpler to actually mostly reference the versioning scheme used by GNU GCC, add a description of the way what's behind the hyphen is built, and in a single paragraph focus on the relationship between this versioning scheme and SemVer (similar to the last one in my example above, though slightly expanded).
Author
Owner

@ilg-ul commented on GitHub (Jul 18, 2019):

Ah, thank you for the above text, it is useful for that GCC derived specific project, but it seems that my question was confusing and you misunderstood it.

The question actually was about a possible way to extend the SemVer specs themselves to better support projects with 3rd party content. For example by removing the limitation of having only 3 numbers (major.minor.patch -> major.minor.patch.fourth.fifth. ... .n-th). This would be a better solution that using the pre-release numbers as extra version numbers. Do you think that such an extension to the SemVer specs would be possible in the future? If so, what would be such a possible new definition?

As for the ARM GCC distribution mentioned in the initial message, it was just an example (a real life one, though), I would prefer not to take it as reference, since there are many other such projects in this situation, I am already publishing a RISC-V GCC, OpenOCD, QEMU ARM, and more are planned, both binary xPacks (like GNU GCC, GNU GDB, GNU binutils, GNU coreutils, make, CMake, Ninja Build) and source xPacks (like FreeRTOS, Chan FatFS, plus many more), and all will be published as NPM packages with SemVer versions, so there are many different versioning schemes that should somehow be accomodated.

As reference, the (just published, preliminary) web site of the xPack project is https://xpack.github.io.

@ilg-ul commented on GitHub (Jul 18, 2019): Ah, thank you for the above text, it is useful for that GCC derived specific project, but it seems that my question was confusing and you misunderstood it. The question actually was about a possible way to extend the SemVer specs themselves to better support projects with 3rd party content. For example by removing the limitation of having only 3 numbers (major.minor.patch -> major.minor.patch.fourth.fifth. ... .n-th). This would be a better solution that using the pre-release numbers as extra version numbers. Do you think that such an extension to the SemVer specs would be possible in the future? If so, what would be such a possible new definition? As for the ARM GCC distribution mentioned in the initial message, it was just an example (a real life one, though), I would prefer not to take it as reference, since there are many other such projects in this situation, I am already publishing a RISC-V GCC, OpenOCD, QEMU ARM, and more are planned, both binary xPacks (like GNU GCC, GNU GDB, GNU binutils, GNU coreutils, make, CMake, Ninja Build) and source xPacks (like FreeRTOS, Chan FatFS, plus many more), and all will be published as NPM packages with SemVer versions, so there are many different versioning schemes that should somehow be accomodated. As reference, the (just published, preliminary) web site of the xPack project is https://xpack.github.io.
Author
Owner

@klehelley commented on GitHub (Jul 19, 2019):

Ah. OK it wasn't clear to me that it actually was a question regarding a potential change request for the specification.

In that case, while I am pretty interested in this project I am probably not the best person to ask. The project maintainers have a much better view of the current strengths and limits of the current specification, so it is probably better for one of them to answer.

However, from my point of view, if your request were to be accepted and a new SemVer specification produced that would take it into account, I believe it would result in a more complex, harder to understand result. In the current specification, the biggest point is IMO the semantics associated with each number. If the possibility is added to add as many numbers as one sees fit, should the same semantics be applied to the first three numbers? Or would it be up to the developer/packager to decide what each number means and how it can evolve? You can see that in the latter case waters down the specification, forcing all its users to add specific instruction in their documentation, and in the end making SemVer less useful. Especially considering that there are tools designed to use these semantics (see npm update for example).

IMO, your specific use case is not a good fit for SemVer. However it seems common enough that it could be useful to have a versioning specification designed so that it can be reused by multiple projects. Actually, something that doesn't rely on the packaged/recompiled projects versioning scheme using SemVer would be more useful.

Some useful comments/discussions regarding the addition of numbers to SemVer:

I think at some point there was a discussion somewhere regarding the versioning constraints of package managers (DEB, RPM...), that are the same as yours it seems, but I have been unable to find the conversation.

@klehelley commented on GitHub (Jul 19, 2019): Ah. OK it wasn't clear to me that it actually was a question regarding a potential change request for the specification. In that case, while I am pretty interested in this project I am probably not the best person to ask. The project maintainers have a much better view of the current strengths and limits of the current specification, so it is probably better for one of them to answer. However, from my point of view, if your request were to be accepted and a new SemVer specification produced that would take it into account, I believe it would result in a more complex, harder to understand result. In the current specification, the biggest point is IMO the semantics associated with each number. If the possibility is added to add as many numbers as one sees fit, should the same semantics be applied to the first three numbers? Or would it be up to the developer/packager to decide what each number means and how it can evolve? You can see that in the latter case waters down the specification, forcing all its users to add specific instruction in their documentation, and in the end making SemVer less useful. Especially considering that there are tools designed to use these semantics (see [`npm update`](https://docs.npmjs.com/cli/update) for example). IMO, your specific use case is not a good fit for SemVer. **However it seems common enough that it could be useful to have a versioning specification designed so that it can be reused by multiple projects.** Actually, something that doesn't rely on the packaged/recompiled projects versioning scheme using SemVer would be more useful. Some useful comments/discussions regarding the addition of numbers to SemVer: * [Feedback: what SemVer is _not_ for](https://github.com/semver/semver/issues/323) * [Version of language bindings, api wrappers, etc](https://github.com/semver/semver/issues/352) (similar to your issue) I think at some point there was a discussion somewhere regarding the versioning constraints of package managers (DEB, RPM...), that are the same as yours it seems, but I have been unable to find the conversation.
Author
Owner

@jwdonahue commented on GitHub (Aug 10, 2019):

@ilg-ul, why exactly are you repackaging third party packages?

@jwdonahue commented on GitHub (Aug 10, 2019): @ilg-ul, why exactly are you repackaging third party packages?
Author
Owner

@ilg-ul commented on GitHub (Aug 10, 2019):

@jwdonahue, good question. I guess to reuse existing packages, but in a strict version controlled environment.

I started to add some documentation to the project: https://xpack.github.io

any thoughts on this?

@ilg-ul commented on GitHub (Aug 10, 2019): @jwdonahue, good question. I guess to reuse existing packages, but in a strict version controlled environment. I started to add some documentation to the project: https://xpack.github.io any thoughts on this?
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

There is no one-size fits all mapping from non-SemVer version strings and SemVer version strings. In many cases, there can be no meaningful mapping, so conversion isn't possible in those cases. SemVer is just one of a handful of versioning schemes that even attempts to publicly define its own semantics. Many are essentially arbitrary, meaning semantics == gibberish, and out of the few that do define them, they are simply not compatible because they contain no information regarding breaking changes and non-breaking changes.

The real difficulties in reusing existing packages "in a strict version controlled environment", is they; may not have been produced with any meaningful version control or tracking to start with, and the apparent history, implied by your view of the publicly available packages, may be incomplete. In some cases, publishers re-release the same version number multiple times, with/without announcing any changes.

All is not lost however, if you think you can get your hands a complete enough collection of packages. I see several ways forward:

  1. Attempt to use release notes, publicly available consumer feedback, testing, measures of changes to available unit tests and code reviews (where possible), to make the determinations required to re-sequence all or an important fraction of the available packages, using SemVer.
  2. Make no claims of knowledge of the semantics of any non-compliant versions, and simply label them all 0.1.0 through 0.n.0 (ie; unlabeled prerelease).
  3. 1.0.0 through n.0.0. Just a counter that says any change may have been a breaking change.

#1 implies taking on a lot of work, and option #2 says you know nothing about the release quality of any of the packages, and #3 says you think they are all "release" quality events, not just patches or non-breaking upgrades, but the difference between them is probably breaking or you simply don't know whether they are breaking changes.

What you have to consider when taking on something like this, is that SemVer states the semantic rules for publishers, but it also implies rules for consumers. What you want to do is prevent risk averse consumers, from lightly, or automatically, taking updates that might break their builds or their products. If you convey the wrong information, your resulting series of repacks will be no better than the original publisher's own scheme(s).

@jwdonahue commented on GitHub (Aug 16, 2019): There is no one-size fits all mapping from non-SemVer version strings and SemVer version strings. In many cases, there can be no meaningful mapping, so conversion isn't possible in those cases. SemVer is just one of a handful of versioning schemes that even attempts to publicly define its own semantics. Many are essentially arbitrary, meaning semantics == gibberish, and out of the few that do define them, they are simply not compatible because they contain no information regarding breaking changes and non-breaking changes. The real difficulties in reusing existing packages "in a strict version controlled environment", is they; may not have been produced with any meaningful version control or tracking to start with, and the apparent history, implied by your view of the publicly available packages, may be incomplete. In some cases, publishers re-release the same version number multiple times, with/without announcing any changes. All is not lost however, if you think you can get your hands a complete enough collection of packages. I see several ways forward: 1. Attempt to use release notes, publicly available consumer feedback, testing, measures of changes to available unit tests and code reviews (where possible), to make the determinations required to re-sequence all or an important fraction of the available packages, using SemVer. 2. Make no claims of knowledge of the semantics of any non-compliant versions, and simply label them all 0.1.0 through 0.n.0 (ie; unlabeled prerelease). 3. 1.0.0 through n.0.0. Just a counter that says any change may have been a breaking change. #1 implies taking on a lot of work, and option #2 says you know nothing about the release quality of any of the packages, and #3 says you think they are all "release" quality events, not just patches or non-breaking upgrades, but the difference between them is probably breaking or you simply don't know whether they are breaking changes. What you have to consider when taking on something like this, is that SemVer states the semantic rules for publishers, but it also implies rules for consumers. What you want to do is prevent risk averse consumers, from lightly, or automatically, taking updates that might break their builds or their products. If you convey the wrong information, your resulting series of repacks will be no better than the original publisher's own scheme(s).
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

I would add that no packaging tool or build environment is ever going to completely replace existing tools. We all have to live with heterogeneous solutions if we are working on anything non-trivial. If your new packaging tool offers superior properties to those that are already in use, it may claim a significant share of the market, but it's never going to be compelling enough to warrant the effort required to convert the existing body of packages, or even a small percentage of them.

@jwdonahue commented on GitHub (Aug 16, 2019): I would add that no packaging tool or build environment is ever going to completely replace existing tools. We all have to live with heterogeneous solutions if we are working on anything non-trivial. If your new packaging tool offers superior properties to those that are already in use, it may claim a significant share of the market, but it's never going to be compelling enough to warrant the effort required to convert the existing body of packages, or even a small percentage of them.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

I'm afraid you misunderstood what I expect from semver when used for 3rd party packages.

I do not expect consumers to use advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax.

your new packaging tool

if you take some time and try to understand what xPack is, you'll notice that there is no new packaging tool, it is using regular npm packages and for binary packages compressed archives.

the trick is that once packed as xPacks, either with original content or with 3rd party content, packages can be automatically integrated and used in a project.

in other words, if a project refers to a specific version of a toolchian, a specific version of the builder, specific versions of the libraries, etc, running the build will produce the same result regardless of the toolchain, builder, libraries, etc available on the development machine, and this should remain true for a reasonable period of time; plus that it easily allow for different projects to use different versions of the tools/libraries/etc.

another advantage of a strict version controlled environment is that the 'configure' step looses most of its relevance, and, if perfomed once successfully, it is expected to always be successful (and produce the same result).

it is nothing new, npm uses this mechanism for JavaScript and does an excelent job. xPacks tries to do the same for C/C++ (actually the mechanism is generic, I guess it can be used for any language or development environments).

@ilg-ul commented on GitHub (Aug 16, 2019): I'm afraid you misunderstood what I expect from semver when used for 3rd party packages. I do not expect consumers to use advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax. > your new packaging tool if you take some time and try to understand what xPack is, you'll notice that there is no new packaging tool, it is using regular npm packages and for binary packages compressed archives. the trick is that once packed as xPacks, either with original content or with 3rd party content, packages can be automatically integrated and used in a project. in other words, if a project refers to a specific version of a toolchian, a specific version of the builder, specific versions of the libraries, etc, running the build will produce the same result regardless of the toolchain, builder, libraries, etc available on the development machine, and this should remain true for a reasonable period of time; plus that it easily allow for different projects to use different versions of the tools/libraries/etc. another advantage of a strict version controlled environment is that the 'configure' step looses most of its relevance, and, if perfomed once successfully, it is expected to always be successful (and produce the same result). it is nothing new, npm uses this mechanism for JavaScript and does an excelent job. xPacks tries to do the same for C/C++ (actually the mechanism is generic, I guess it can be used for any language or development environments).
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

I do not expect consumers to use advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax.

That doesn't scale well in DevOps environments, but good luck with your project.

@jwdonahue commented on GitHub (Aug 16, 2019): > I do not expect consumers to use advanced semantics at all, I expect their dependencies to use full and exact version strings, no "approximate", "compatible" or other such elaborate syntax. That doesn't scale well in DevOps environments, but good luck with your project.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

And if you are always going to be explicit, then why not just use the original version string that the publisher released? It's just a tag that should be unique enough when tupled with the package name. Semantics are all about communication between two or more entities. They are pointless if you are just barking into the ether.

It's precisely because there's someone on the other end of that communication who needs more than just a unique tag to decide whether to take one version of a package or another, that we apply semantics to versioning. If you don't expect your consumers to actually use the information embedded in the version string, then why go to the trouble of converting the original publishers version string to SemVer? In some cases, you'd actually be removing information. You might as well apply a GUID.

@jwdonahue commented on GitHub (Aug 16, 2019): And if you are always going to be explicit, then why not just use the original version string that the publisher released? It's just a tag that should be unique enough when tupled with the package name. Semantics are all about communication between two or more entities. They are pointless if you are just barking into the ether. It's precisely because there's someone on the other end of that communication who needs more than just a unique tag to decide whether to take one version of a package or another, that we apply semantics to versioning. If you don't expect your consumers to actually use the information embedded in the version string, then why go to the trouble of converting the original publishers version string to SemVer? In some cases, you'd actually be removing information. You might as well apply a GUID.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

That doesn't scale well in DevOps environments

can you further elaborate on this? traditional devops environments have very little or no control over the versioning; I think that having limited semantics dependencies (with exact versions) is better than having no dependencies at all and relying on the versions installed in the system.

@ilg-ul commented on GitHub (Aug 16, 2019): > That doesn't scale well in DevOps environments can you further elaborate on this? traditional devops environments have very little or no control over the versioning; I think that having limited semantics dependencies (with exact versions) is better than having no dependencies at all and relying on the versions installed in the system.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

why not just use the original version string that the publisher released?

because it is not enough to identify the package.

take the above example:

  • 8.2.1-1.7.1 = the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1

it is highly possible that ARM will release a 2nd package based on FSF 8.2.1, I may release subsequent packages if bugs are identified in ARMs release, and I may discover that I missed something in my npm packages and release new ones to fix this.

see https://www.npmjs.com/package/@gnu-mcu-eclipse/arm-none-eabi-gcc?activeTab=versions for a real life example.

why go to the trouble of converting the original publishers version string to SemVer? In some cases, you'd actually be removing information. You might as well apply a GUID.

because I expect the npm registry to reject packages with a non SemVer version field in package.json.

@ilg-ul commented on GitHub (Aug 16, 2019): > why not just use the original version string that the publisher released? because it is not enough to identify the package. take the above example: - 8.2.1-1.7.1 = the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1 it is highly possible that ARM will release a 2nd package based on FSF 8.2.1, I may release subsequent packages if bugs are identified in ARMs release, and I may discover that I missed something in my npm packages and release new ones to fix this. see https://www.npmjs.com/package/@gnu-mcu-eclipse/arm-none-eabi-gcc?activeTab=versions for a real life example. > why go to the trouble of converting the original publishers version string to SemVer? In some cases, you'd actually be removing information. You might as well apply a GUID. because I expect the npm registry to reject packages with a non SemVer version field in package.json.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

can you further elaborate on this?

I am sorry (LOL). Actually there's lots of DevOps systems out there that automate versioning. I know of tens of thousands of build agents, that pull-in packages based on version ranges, build and kick off unit and higher level test runs. Even the choice to publish the resulting product can be made automatically. Usually, the decision to publish to the general public (a GA release), does involve human intervention, but the deciders have information from automated build/test and flight test (alpha, beta, etc.) results to work with.

I know of systems that track automated bug reports from automated testing systems, that are used to decide which of the various combinations of versions of software and hardware components will be locked-in for the GA release.

@jwdonahue commented on GitHub (Aug 16, 2019): > can you further elaborate on this? I am sorry (LOL). Actually there's lots of DevOps systems out there that automate versioning. I know of tens of thousands of build agents, that pull-in packages based on version ranges, build and kick off unit and higher level test runs. Even the choice to publish the resulting product can be made automatically. Usually, the decision to publish to the general public (a GA release), does involve human intervention, but the deciders have information from automated build/test and flight test (alpha, beta, etc.) results to work with. I know of systems that track automated bug reports from automated testing systems, that are used to decide which of the various combinations of versions of software and hardware components will be locked-in for the GA release.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

because I expect the npm registry to reject packages with a non SemVer version field in package.json.

Ah. So we come full circle. You need SemVer, implying that consumers care about semantic content, therefore you must find an adequate mapping from some foreign scheme to SemVer, which is a known impossibility for the vast majority of foreign schemes out there, not to mention the even larger set that have no discernible semantics to translate from. I think the best you can do in most cases, is re-sequence the series using either a prerelease scheme or major version sequence, as I stated earlier.

@jwdonahue commented on GitHub (Aug 16, 2019): > because I expect the npm registry to reject packages with a non SemVer version field in package.json. Ah. So we come full circle. You need SemVer, implying that consumers care about semantic content, therefore you must find an adequate mapping from some foreign scheme to SemVer, which is a known impossibility for the vast majority of foreign schemes out there, not to mention the even larger set that have no discernible semantics to translate from. I think the best you can do in most cases, is re-sequence the series using either a prerelease scheme or major version sequence, as I stated earlier.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

there's lots of DevOps systems out there that automate versioning

sure, just that none that I know are generic enough, like the npm dependencies mechanism, that works on all platforms and with a very wide range of applications.

with xPack all you have to do is run 'xpm install` in the application folder and all dependencies (tools/libraries/etc) are automatically satisfied. compare this to the complexity and specificity of the DevOps you know and you'll see there is a very big difference.

I think the best you can do in most cases, is re-sequence the series using either a prerelease scheme

that's exactly the solution that I came to. thank you for confirming.

@ilg-ul commented on GitHub (Aug 16, 2019): > there's lots of DevOps systems out there that automate versioning sure, just that none that I know are generic enough, like the npm dependencies mechanism, that works on all platforms and with a very wide range of applications. with xPack all you have to do is run 'xpm install` in the application folder and all dependencies (tools/libraries/etc) are automatically satisfied. compare this to the complexity and specificity of the DevOps you know and you'll see there is a very big difference. > I think the best you can do in most cases, is re-sequence the series using either a prerelease scheme that's exactly the solution that I came to. thank you for confirming.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

You might also consider moving some information from the original version string over to the package name itself. A package P that has versions for several different environments might be renamed P.environment or P.environment.compiler, then the actual version string might be easier to translate. For instance, many four digit version schemes, simply have a build number in the fourth field. In cases where you see a lot of 1.0.0.#### kind of version strings, all you know about the #### is it is another build. It might be identical to the any number of previous builds or could have breaking changes. These would map easily into 0.1.#### and you then preserve any prerelease or meta tags from the original by mapping those directly onto the SemVer prerelease and meta tags.

@jwdonahue commented on GitHub (Aug 16, 2019): You might also consider moving some information from the original version string over to the package name itself. A package P that has versions for several different environments might be renamed P.environment or P.environment.compiler, then the actual version string might be easier to translate. For instance, many four digit version schemes, simply have a build number in the fourth field. In cases where you see a lot of 1.0.0.#### kind of version strings, all you know about the #### is it is another build. It might be identical to the any number of previous builds or could have breaking changes. These would map easily into 0.1.#### and you then preserve any prerelease or meta tags from the original by mapping those directly onto the SemVer prerelease and meta tags.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

BTW, I have worked with systems that use various combinations of drop folders, nuget, NPM, zip and other schemes all in the same build system.

@jwdonahue commented on GitHub (Aug 16, 2019): BTW, I have worked with systems that use various combinations of drop folders, nuget, NPM, zip and other schemes all in the same build system.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

I have worked with systems that use various combinations of drop folders, nuget, NPM, zip and other schemes all in the same build system.

I have no doubt that many dependency management schemes can be done with existing solutions, just that all of them imply significantly more work than issuing a single xpm install command.

for example here is the Travis file for an xPack project:

os:
  - linux

dist: trusty

language: node_js
node_js:
  - "node" 

install:
  - npm install --global xpm

script:
  - xpm install
  - xpm run test

and a dependency list may look like:

  "dependencies": {
    "@micro-os-plus/diag-trace": "~1.0.6",
    "@sifive/hifive1-board": "~1.0.3"
  },
  "devDependencies": {
    "@xpack/xpbuild": "~1.2.3",
    "@xpack-dev-tools/arm-none-eabi-gcc": "8.2.1-1.7.1"  <-- 3rd party, exact match
  }

in my opinion this is quite easy to use (and elegant, btw), I don't think you can simplify it much further.

I agree that it is a small overhead to pack a 3rd party package as an xPack, but once it is done, it can be easily used in multiple projects (instead of dealing with complicated solutions in each project) thus the effort may be well justified.

@ilg-ul commented on GitHub (Aug 16, 2019): > I have worked with systems that use various combinations of drop folders, nuget, NPM, zip and other schemes all in the same build system. I have no doubt that many dependency management schemes can be done with existing solutions, just that all of them imply significantly more work than issuing a single `xpm install` command. for example here is the Travis file for an xPack project: ``` os: - linux dist: trusty language: node_js node_js: - "node" install: - npm install --global xpm script: - xpm install - xpm run test ``` and a dependency list may look like: ``` "dependencies": { "@micro-os-plus/diag-trace": "~1.0.6", "@sifive/hifive1-board": "~1.0.3" }, "devDependencies": { "@xpack/xpbuild": "~1.2.3", "@xpack-dev-tools/arm-none-eabi-gcc": "8.2.1-1.7.1" <-- 3rd party, exact match } ``` in my opinion this is quite easy to use (and elegant, btw), I don't think you can simplify it much further. I agree that it is a small overhead to pack a 3rd party package as an xPack, but once it is done, it can be easily used in multiple projects (instead of dealing with complicated solutions in each project) thus the effort may be well justified.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

The "8.2.1-1.7.1" version string contains a prerelease tag, adding transitive dependency issues for any consumers of that package, who would explicitly rule out prerelease software for inclusion in their products. Like I said, semantic versioning caries information to the consumer. Your example probably takes a release version of whatever is in xpack-dev-tools/arm-none-eabi-gcc and makes a prerelease version of it, effectively changing the semantics from whatever the original publisher intended, to something different.

Any complete solution, must be at least complex enough to satisfy the complexity of the underlying requirements. In your example, you have not met the requirements bar. If you want to preserve the non-prerelease information embedded in the original version string, you would need a scheme that does not include a prerelease tag. Syntactic correctness is not sufficient. You must provide a semantically correct SemVer string or everyone that uses an xPack would have to deal with the additional complexity of having separate rules for handling them, as they already do for the original packages.

@jwdonahue commented on GitHub (Aug 16, 2019): The "8.2.1-1.7.1" version string contains a prerelease tag, adding transitive dependency issues for any consumers of that package, who would explicitly rule out prerelease software for inclusion in their products. Like I said, semantic versioning caries information to the consumer. Your example probably takes a release version of whatever is in xpack-dev-tools/arm-none-eabi-gcc and makes a prerelease version of it, effectively changing the semantics from whatever the original publisher intended, to something different. Any complete solution, must be at least complex enough to satisfy the complexity of the underlying requirements. In your example, you have not met the requirements bar. If you want to preserve the non-prerelease information embedded in the original version string, you would need a scheme that does not include a prerelease tag. Syntactic correctness is not sufficient. You must provide a semantically correct SemVer string or everyone that uses an xPack would have to deal with the additional complexity of having separate rules for handling them, as they already do for the original packages.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

So I am basically saying that mapping from any non-semver PV to a Psemver, requires capturing any known semantics of PV and mapping those onto some combination of the package name and Psemver. It is not a purely mechanical/syntactic process. If the semantics of PV are unknown, then treat it as a prerelease and encourage the original publisher to change their ways, or ask the consumers of that package how they interpret them. Perhaps everything is a release from the publisher of PV, if that's the case, re-sequence them as N.0.0 releases.

Have you discussed your proposals with the NPM community? I think they can probably reveal more of the corner cases you'd have to consider than I can.

@jwdonahue commented on GitHub (Aug 16, 2019): So I am basically saying that mapping from any non-semver PV to a Psemver, requires capturing any known semantics of PV and mapping those onto some combination of the package name and Psemver. It is not a purely mechanical/syntactic process. If the semantics of PV are unknown, then treat it as a prerelease and encourage the original publisher to change their ways, or ask the consumers of that package how they interpret them. Perhaps everything is a release from the publisher of PV, if that's the case, re-sequence them as N.0.0 releases. Have you discussed your proposals with the NPM community? I think they can probably reveal more of the corner cases you'd have to consider than I can.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

adding transitive dependency issues for any consumers of that package, who would explicitly rule out prerelease software for inclusion in their products.

well, this was the point of the topic, how to integrate 3rd party packages into semVer environments. if the extra versiong numbers are encoded as prerelease, it is obvious that parent packages cannot rule out the use of prelease software. this can be explicitly mentioned in the package README file.

I understand that this is a compromise, but I could not find a better one.

You must provide a semantically correct SemVer string or everyone that uses an xPack would have to deal with the additional complexity of having separate rules for handling them, as they already do for the original packages.

that would be nice, but unfortunatelly for some packages there is not even possible to tell if the two releases are compatible or not without thouroughly inspecting the change log (take QEMU, which decided to make yearly releases regardless of the changes, and refused to consider any semantic versioning, or OpenOCD, which after years still continues to use numbers like 0.x.0, maintaining the same minor for years, and also refused to use any semantic versioning); so renumbering the versions using semVer semantics would probably lead to strings like 1.0, 2.0, 3.0, etc, totally uncorelated with original versions, which I think is very confusing for the users, since they always have to use a conversion table. not to mention that sometimes increasing the major does not come with incompatibilities, just that it is hard to tell so, so users will gamble when updating dependencies.

@ilg-ul commented on GitHub (Aug 16, 2019): > adding transitive dependency issues for any consumers of that package, who would explicitly rule out prerelease software for inclusion in their products. well, this was the point of the topic, how to integrate 3rd party packages into semVer environments. if the extra versiong numbers are encoded as prerelease, it is obvious that parent packages cannot rule out the use of prelease software. this can be explicitly mentioned in the package README file. I understand that this is a compromise, but I could not find a better one. > You must provide a semantically correct SemVer string or everyone that uses an xPack would have to deal with the additional complexity of having separate rules for handling them, as they already do for the original packages. that would be nice, but unfortunatelly for some packages there is not even possible to tell if the two releases are compatible or not without thouroughly inspecting the change log (take QEMU, which decided to make yearly releases regardless of the changes, and refused to consider any semantic versioning, or OpenOCD, which after years still continues to use numbers like 0.x.0, maintaining the same minor for years, and also refused to use any semantic versioning); so renumbering the versions using semVer semantics would probably lead to strings like 1.0, 2.0, 3.0, etc, totally uncorelated with original versions, which I think is very confusing for the users, since they always have to use a conversion table. not to mention that sometimes increasing the major does not come with incompatibilities, just that it is hard to tell so, so users will gamble when updating dependencies.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

You can usually always append the original version string to the meta tag btw. It seems to me your interested in providing SemVer's sort ordering and you should be interested in preserving any of the available semantics. You're versioning a package, NOT the API contained within it, that was already done by the publisher and it seems you actually do want to preserve the publisher's version string.

@jwdonahue commented on GitHub (Aug 16, 2019): You can usually always append the original version string to the meta tag btw. It seems to me your interested in providing SemVer's sort ordering and you should be interested in preserving any of the available semantics. You're versioning a package, NOT the API contained within it, that was already done by the publisher and it seems you actually do want to preserve the publisher's version string.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

mapping from any non-semver PV to a Psemver, requires capturing any known semantics of PV and mapping those onto some combination of the package name and Psemver

it is not only about capturing the package semantics, it is also about encoding additional numbers (like the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1).

altering the package name with versioning info pollutes the scope with countless packages, I don't think that registry maintainers would appreciate this.

Have you discussed your proposals with the NPM community?

I don't think there are many legacy JavaScript packages to be treated as package with 3rd party content, so I expect all their packages to have original content and so easily follow the semVer semantics.

always append the original version string to the meta tag

yes, this was one of the proposed solutions in the initial message in this topic.

It seems to me your interested in providing SemVer's sort ordering

yes, that is preferred.

You're versioning a package, NOT the API contained within it, that was already done by the publisher

as in the GCC example, there might be 3 publishers in line before actually publishing the package on NPM.

and it seems you actually do want to preserve the publisher's version string.

yes, as much as possible, but in some cases it is not realistic, for example ARM named their 1st release based on FSF 8.2.1 GCC as 8-2018-q4-major, the second one as 8-2019-q3-update, and so on. they too considered that semantic versioning does not fit their needs and came with this unusual scheme.

as I said many times, there is no perfect solution, only compromises.

@ilg-ul commented on GitHub (Aug 16, 2019): > mapping from any non-semver PV to a Psemver, requires capturing any known semantics of PV and mapping those onto some combination of the package name and Psemver it is not only about capturing the package semantics, it is also about encoding additional numbers (like _the first NPM package based on the sevenths GNU MCU Eclipse release based on ARM 1st release based on FSF 8.2.1_). altering the package name with versioning info pollutes the scope with countless packages, I don't think that registry maintainers would appreciate this. > Have you discussed your proposals with the NPM community? I don't think there are many legacy JavaScript packages to be treated as package with 3rd party content, so I expect all their packages to have original content and so easily follow the semVer semantics. > always append the original version string to the meta tag yes, this was one of the proposed solutions in the initial message in this topic. > It seems to me your interested in providing SemVer's sort ordering yes, that is preferred. > You're versioning a package, NOT the API contained within it, that was already done by the publisher as in the GCC example, there might be 3 publishers in line before actually publishing the package on NPM. > and it seems you actually do want to preserve the publisher's version string. yes, as much as possible, but in some cases it is not realistic, for example ARM named their 1st release based on FSF 8.2.1 GCC as `8-2018-q4-major`, the second one as `8-2019-q3-update`, and so on. they too considered that semantic versioning does not fit their needs and came with this unusual scheme. as I said many times, there is no perfect solution, only compromises.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

Some organizations do not allow prerelease components in the their release software. By down-grading the status of every third party product provided in an xPack, you are ruling it out for these organizations. Most, but not all of them, do have processes to get exemptions for this rule, but that's an offsetting complication.

I am just not convinced you are making anything easier for the consumers of these products. Consider that these producers have been supplying customers, potentially for years. Those customers have already adapted their processes to accommodate these producers. Many of them, never use NPM.

I also think you have to be careful what it is you are packaging and what the producer's license says about that. They might not be very thrilled to find that you are simply repacking their product and potentially confusing their consumers without providing any added value. Perhaps if they had a real market to provide NPM's or use SemVer they would do exactly that. Even if they have an MIT or similar license, they might refuse support to anyone not using their official releases.

@jwdonahue commented on GitHub (Aug 16, 2019): Some organizations do not allow prerelease components in the their release software. By down-grading the status of every third party product provided in an xPack, you are ruling it out for these organizations. Most, but not all of them, do have processes to get exemptions for this rule, but that's an offsetting complication. I am just not convinced you are making anything easier for the consumers of these products. Consider that these producers have been supplying customers, potentially for years. Those customers have already adapted their processes to accommodate these producers. Many of them, never use NPM. I also think you have to be careful what it is you are packaging and what the producer's license says about that. They might not be very thrilled to find that you are simply repacking their product and potentially confusing their consumers without providing any added value. Perhaps if they had a real market to provide NPM's or use SemVer they would do exactly that. Even if they have an MIT or similar license, they might refuse support to anyone not using their official releases.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

I don't claim xPack can solve all possible problems of the world, binary xPacks are actually portable archives which can also be installed manually, and source xPacks are usually snapshots of GitHub projects, that can also be installed in traditional ways, so those who do not want to automate their builds with xpm can use traditional ways.

and I personally plan to use xPacks mainly for packages with original content, so semVer should be fine.

but there are a number of existing 3rd party binary tools (like toolchains and other build tools) that will be needed; I know that using the prerelease field is not great, but in my opinion is an acceptable compromise. if there will be complains about this, and a renumbering scheme will be preferred, I'll reevaluate, it is always possible to increase the major and start afresh.

also if future versions of semVer will accept more than 3 numbers in the main part, I'll use them and get rid of the prerelease part.

thank you for your feedback, I highly appreciate it.

@ilg-ul commented on GitHub (Aug 16, 2019): I don't claim xPack can solve all possible problems of the world, binary xPacks are actually portable archives which can also be installed manually, and source xPacks are usually snapshots of GitHub projects, that can also be installed in traditional ways, so those who do not want to automate their builds with `xpm` can use traditional ways. and I personally plan to use xPacks mainly for packages with original content, so semVer should be fine. but there are a number of existing 3rd party binary tools (like toolchains and other build tools) that will be needed; I know that using the prerelease field is not great, but in my opinion is an acceptable compromise. if there will be complains about this, and a renumbering scheme will be preferred, I'll reevaluate, it is always possible to increase the major and start afresh. also if future versions of semVer will accept more than 3 numbers in the main part, I'll use them and get rid of the prerelease part. thank you for your feedback, I highly appreciate it.
Author
Owner

@jwdonahue commented on GitHub (Aug 16, 2019):

Have a look at VersionSchema and VersionMeta. I was just starting to work on tooling when Microsoft sucked me into their vortex for an 18 month contract. The goal is a human readable version string that can also be reasoned over by machines. There are many producers who do not think SemVer is appropriate for them, and I can think of many cases where it really is completely inadequate.

@jwdonahue commented on GitHub (Aug 16, 2019): Have a look at [VersionSchema](http://versionschema.org) and [VersionMeta](http:versionmeta.org). I was just starting to work on tooling when Microsoft sucked me into their vortex for an 18 month contract. The goal is a human readable version string that can also be reasoned over by machines. There are many producers who do not think SemVer is appropriate for them, and I can think of many cases where it really is completely inadequate.
Author
Owner

@ilg-ul commented on GitHub (Aug 16, 2019):

Have a look at VersionSchema and VersionMeta.

yes, interresting, I'm glad that semVer is not the last word and future solutions are considered.

however in my case, since xPacks are extensions of npm packages, I am bound to the limits of the npm tool and npm registries.

for the momen I'm using the official npmjs.com registry, but I might migrate to the GitHub NPM registry, when available.

the xPack project is still in its infancy, only recently I created its web site and finalised the details of xPack contents and procedures to publish.

but even so, with a very young and experimental project, it should be noted that the few existing xPacks were relatively well received, according to npmjs.com analytics, xpm was downloaded for 23k times, the ARM toolchain xPack for 9.5k times, the RISC-V toolchain for 2.1k times, OpenOCD for 4.9k times, QEMU for 3.1k times, which I would consider an encouraging start.

with xPack support planned for Visual Studio Code, I expect usage to increase.

I'll update the documentation to clarify the possible administrative issues when using packages with prerelease version strings, and also explicitly mention that the original licenses apply.

@ilg-ul commented on GitHub (Aug 16, 2019): > Have a look at VersionSchema and VersionMeta. yes, interresting, I'm glad that semVer is not the last word and future solutions are considered. however in my case, since xPacks are extensions of npm packages, I am bound to the limits of the npm tool and npm registries. for the momen I'm using the official npmjs.com registry, but I might migrate to the GitHub NPM registry, when available. the xPack project is still in its infancy, only recently I created its web site and finalised the details of xPack contents and procedures to publish. but even so, with a very young and experimental project, it should be noted that the few existing xPacks were relatively well received, according to npmjs.com analytics, **xpm** was downloaded for 23k times, the ARM toolchain xPack for 9.5k times, the RISC-V toolchain for 2.1k times, OpenOCD for 4.9k times, QEMU for 3.1k times, which I would consider an encouraging start. with xPack support planned for Visual Studio Code, I expect usage to increase. I'll update the documentation to clarify the possible administrative issues when using packages with prerelease version strings, and also explicitly mention that the original licenses apply.
Author
Owner

@jwdonahue commented on GitHub (Aug 17, 2019):

Glad I could be of some assistance. I think we've probably run this one to ground at this point. Time pop open a beer and veg.

@jwdonahue commented on GitHub (Aug 17, 2019): Glad I could be of some assistance. I think we've probably run this one to ground at this point. Time pop open a beer and veg.
Author
Owner

@jwdonahue commented on GitHub (Aug 27, 2019):

altering the package name with versioning info pollutes the scope with countless packages, I don't think that registry maintainers would appreciate this.

Have you asked them? What about your customer base?

If you think about it, you've encountered a weakness in the NPM design. They may have been confronted with this problem in the past. They just might have better guidance for you on this topic. Since you are completely degrading the semantics by way of translation, you might as well use a non-semver string. I know they recommend using semver, but I am also pretty sure they support other flavors, more or less gracefully.

@jwdonahue commented on GitHub (Aug 27, 2019): > altering the package name with versioning info pollutes the scope with countless packages, I don't think that registry maintainers would appreciate this. Have you asked them? What about your customer base? If you think about it, you've encountered a weakness in the NPM design. They may have been confronted with this problem in the past. They just might have better guidance for you on this topic. Since you are completely degrading the semantics by way of translation, you might as well use a non-semver string. I know they recommend using semver, but I am also pretty sure they support other flavors, more or less gracefully.
Author
Owner

@jwdonahue commented on GitHub (Aug 27, 2019):

My final advice on this topic is don't pretend to apply semver if you are not actually semver compliant. If you are never going to release any N.0.0 stable releases, then there really isn't any point of using semver.

Unless you have further questions, please close this issue at your earliest possible convenience.

@jwdonahue commented on GitHub (Aug 27, 2019): My final advice on this topic is **don't pretend to apply semver if you are not actually semver compliant**. If you are never going to release any N.0.0 stable releases, then there really isn't any point of using semver. Unless you have further questions, please close this issue at your earliest possible convenience.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/semver#385