mirror of
https://github.com/semver/semver.git
synced 2026-07-10 19:50:47 -05:00
[GH-ISSUE #516] Semantic version scheme for embedded devices #7356
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @sjlongland on GitHub (May 17, 2019).
Original GitHub issue: https://github.com/semver/semver/issues/516
Hi,
I figured I'd throw this up here as an idea to others. Sometimes you want to have a version number in some simple format that a microcontroller can easily deal with. The thinking was to use a subset of Semver 2.0, and encode the major/minor/patch/release type/revision as a 32-bit unsigned integer in a manner that allows for direct comparisons to know if something is newer/older/same.
I'm not expecting this to necessarily be adopted as is, but floating this as a concept.
The following is the encoding scheme I came up with.
The components of this are:
Major,Minor,Patch: the Semantic Version (2.0) version number of the firmware image. These are encoded as unsigned 8-bit integers with a valid range of 0-255.T: The release type, an enumeration:T=0: Alpha releaseT=1: Beta releaseT=2: Release candidateT=3: Production release (revision MUST be 0)Revision: This in combination with theTfield define which alpha/beta/release candidate this is.The numbers can be directly compared using integer arithmetic operators in a manner that respects the semantic versioning standard.
Example version codes:
17.9.3317.9.33-rc.517.9.33-alphaRegards,
Stuart Longland
@runeimp commented on GitHub (May 18, 2019):
I like it as a concept but there are a few gotchas to work out.
TandRevisionencoding for the pre-release version also needs to be at minimum a 6-bit and 10-bit values respectively. Probably larger to be as inclusive as possible.Your example would probably work, as is, for like 90% of the software projects out there. But there are other API systems and software projects that have very different needs for the pre-release labels. There is also the possibility of needing to consider the build number. While it should rarely be considered for sorting there are times it's valid and not currently part of the example. One way to include build but easily avoid it is the encode the values as a 64-bit float. With 48 bits for the integer part (8-bit Major, 8-bit Minor, 16-bit Patch, 16-bit Pre-release [6-bit T and 10-bit Revision]) and 16 bit remainder. That way just convert to int or floor the value to not consider build in the comparison.
But really that could easily fall short for many projects and falls into the same problems as using a fixed field database. Some data just wont fit. It really needs to be a structured value store like SQL table with integer and large text fields, CSV, JSON or the string that is already the spec. 🤔
@sjlongland commented on GitHub (May 19, 2019):
Ahh okay, wasn't sure how "big" the
major/minor/patchnumbers could get, Semver 2.0 doesn't really say.majorcould get arbitrarily large, but I haven't seen too many projects where any component exceeded the limit of auint8_t.As for the
revision, I assumed this would follow the same pattern as the other fields. I deliberately did not consider build metadata as that is not used in comparing version numbers and it won't fit in an integer anyway.I'll admit I was targetting a FPU-less 32-bit microcontroller (Cortex M3) so that's why I gravitated towards a
uint32_tas the basis for the implementation. It's also not "huge" for an 8-bit microcontroller, a comparison can be theoretically done in under 10 instructions depending on the CPU used.Using IEEE-754 isn't something I considered, largely because versions aren't technically "numbers" in the mathematical sense. I worry about what floating-point rounding error might do to the interpretation of these numbers, and it does complicate display (
newlib-nanodoesn't give you float-capableprintfunless you ask for it, and will add significant amounts of code to your build if you do).A
uint64_twould be fine though: you could double the size of themajor/minor/patchfields and use the proposed 6-bittype+ 10-bitrevisionif you needed the full gamut of release versions. Then to drop the build number, it's a simple AND operation, which on most CPUs is a dedicated instruction.I guess that's why I suggested the original proposal as a "subset" -- it's enough to be useful in embedded devices without being onerous on computation or storage requirements.
@sjlongland commented on GitHub (May 19, 2019):
For reference, this is what code that prints the version as it stands now, looks like:
@sjlongland commented on GitHub (May 20, 2019):
Right, so doing some pen and paper work… and reading the the spec more closely, it seems this is the real sticking point:
The strings are arbitrary (this I missed), but for the purpose of encoding, we need to put a limitation on what values are permitted so that we can avoid having to store and parse strings (which can be a royal pain to do in C and assembly). I played around with some ideas on paper.
At first I looked at ways those 6-bit values could be used to encode the schemes, initially toying with just zero-padding my original 2-bit type field format, then looking at how to encode the initial character. 7-bits is needed for an ASCII character.
If it weren't for case-sensitivity, we could just use ITA-2 (with the additional bit being used to select letters/figures) or the Radix-50 variant used on the DEC PDP-6, however SemVer does consider case, so we need to encode upper and lower case. Given this, I propose two variants on the suggested
uint64_tformat.7-bit ASCII-inspired format
This started out as encoding the first letter of this release type field, left-justifying it to leave 9 least significant bits for the revision. After some thought, I realised that if I tweak things a little from the ASCII standard, we can gain a 11-bit range for the revision whilst still being somewhat true to the ASCII standard which yielded the 5-bit type format I describe next. The following is that tweaked 7-bit version:
uint16_t(0-65535)uint16_t(0-65535)uint16_t(0-65535)0x00-0x3f: Pre-alphas, encoded in decimal perhaps?X.Y.Z-0.REVthrough toX.Y.Z-63.REV0x40: Alpha versions:X.Y.Z-alpha.REV0x41-0x5a: Project-specific releases, encoded asX.Y.Z-A.REVthrough toX.Y.Z-Z.REV0x5b-0x5f: Reserved for future use0x60: Beta versions:X.Y.Z-beta.REV0x61-0x6f: Project-specific releases, encoded asX.Y.Z-a.REVthrough toX.Y.Z-o.REV0x70-0x73: Release Candidates, the least significant bits expand the revision range to 0-20470x74-0x7a: Project-specific releases, encoded asX.Y.Z-t.REVthrough toX.Y.Z-z.REV0x7b: Reserved for future use0x7c-0x7f: Production release versionsThe release type MUST either be one of:
alpha,beta,rc; a major.minor value where major is in the range 0-63, OR a word uniquely identified by its first letter in either case; or omitted completely for a production release. The revision is IGNORED for production releases.5-bit type format
uint16_t(0-65535)uint16_t(0-65535)uint16_t(0-65535)0x00-0x0f: Pre-alphas… encoded as decimal:X.Y.Z-0.REVthough toX.Y.Z-15.REV0x10: Alpha versions… encoded asX.Y.Z-alpha.R0x11-0x17: Custom release versions, encoded asX.Y.Z-D.REVthrough toX.Y.Z-Y.REV0x18: Beta versions0x19-0x1b: Custom release versions, encoded asX.Y.Z-d.REVthrough toX.Y.Z-l.REV0x1c: Release Candidates… encoded asX.Y.Z-rc.REV0x1d-0x1e: Custom release versions, encoded asX.Y.Z-t.REVthrough toX.Y.Z-x.REV0x1f: Production release versions… encoded asX.Y.Z, revision is ignored.The release type MUST either be one of:
alpha,beta,rc; a major.minor value where major is in the range 0-15, OR a word uniquely identified by its first letter starting with one of the following letters:D,H,L,P,T,X,Y,d,h,l,t,x; or omitted completely for a production release. The revision is IGNORED for production releases.Compatibility of the two formats
Both of these match in the assignment of the major/minor/patch versions. In the 7-bit version, I tweaked the numbers a bit to make use of "invalid" ASCII characters (not in the set 0-9 A-Z a-z) to represent the alpha, beta and production versions.
00000xxR RRRRRRRRX.Y.Z-0.RX.Y.Z-0.R00001xxR RRRRRRRRX.Y.Z-4.RX.Y.Z-1.R01111xxR RRRRRRRRX.Y.Z-60.RX.Y.Z-15.R10000xxR RRRRRRRRX.Y.Z-alpha.RX.Y.Z-alpha.R10001xxR RRRRRRRRX.Y.Z-D.RX.Y.Z-D.R10010xxR RRRRRRRRX.Y.Z-H.RX.Y.Z-H.R10111xxR RRRRRRRRX.Y.Z-Y.RX.Y.Z-Y.R11000xxR RRRRRRRRX.Y.Z-beta.RX.Y.Z-beta.R11001xxR RRRRRRRRX.Y.Z-d.RX.Y.Z-d.R11011xxR RRRRRRRRX.Y.Z-l.RX.Y.Z-l.R11100xxR RRRRRRRRX.Y.Z-rc.RX.Y.Z-rc.R11101xxR RRRRRRRRX.Y.Z-t.RX.Y.Z-t.R11110xxR RRRRRRRRX.Y.Z-x.RX.Y.Z-x.R11111xxR RRRRRRRRX.Y.Z(release ignored)X.Y.Z(release ignored)@sjlongland commented on GitHub (May 20, 2019):
… sigh and one Big Problem with a
uint64_tformat is that NodeJS and JavaScript are 64-bit challenged:NodeJS 10.4 apparently has support for
BigInt, but not v8 and earlier. On this basis, I think the 32-bit version should be considered as in IoT applications, one may want compatibility between a NodeJS application and a small embedded device running C/C++-based firmware.So that makes 3 suggested formats:
uint64_tformat with 5-bit typeuint64_tformat with a 7-bit type@runeimp commented on GitHub (May 20, 2019):
@sjlongland Be sure to check out https://www.versionmeta.org/ and http://versionschema.org/ which are probably but necessary for making something like this really useful.
@sjlongland commented on GitHub (May 21, 2019):
Yeah, I'm not proposing we use XML here… remember the use case for this embedded subset: microcontrollers. Some of which are 32-bit, many of which are 8-bit, and a good number of them have 64kB of RAM or less.
Wireless networks are even more constrained, IEEE 802.15.4 supports a maximum native MTU of 128 bytes, Nordic's nRF24L01+ has a MTU of 32 bytes.
255.255.255-alpha.63occupies 22 bytes, or slightly over ⅓ of the payload. Contrast that with0xffffff3f, which occupies just 4, or would be 5 bytes sent as CBOR. There isn't room for an XML schema like VersionSchema.The use case in my situation is knowing whether a firmware update is newer than the one we're running now; and perhaps in the future to know if it's a breaking change so we can avoid automatically applying it without manual intervention (so we don't accidentally brick a whole fleet of wireless mesh devices installed in obscure and hard-to-reach places). That's exactly what SemVer is for. In my case, I want to get to something where, to check for a breaking change, I do (obviously using properly defined constants rather than magic numbers):
While I recognise SemVer's goals as primarily being a human-readable format, what I am trying to achieve is a compatible "binary representation" which will be possible to manage in constrained contexts.
This "eSemVer 2.0" (for want of a better name) version standard, when represented in canonical text format, would be a 100% valid and compliant SemVer 2.0 string -- BUT could then be losslessly encoded to a compact 4 or 8 byte representation which can then be directly compared and managed using only bit-wise and integer arithmetic in a small number of instructions.
@jwdonahue commented on GitHub (Oct 3, 2019):
@sjlongland, separate your concerns. Human readable version strings do not need to be embedded in your tiny products if you can perform a lookup.
Embed a unique N bit identifier in your "version" field. Use SemVer to version the packages you distribute. Then say 1.0.0 might have an Id of 100 or whatever number of iterations it took to get to 1.0.0. The Id is associated with the SemVer version tag in your revision control system or other database.
All you really need to decide whether to take an update, are two (maybe three bits) bits worth of flags and whatever number of bits you have left over for an Ident. One bit for breaking change and another bit for priority bug fix (as-in critical security fix). Use the remaining bits to determine whether the device is above or below the offered update.
But seriously think about this for a moment. If you're really that constrained for space, how are you ever going to secure the bloody thing? Should such a device ever take an update? Perhaps a secure brick command would be more appropriate? Not sure you'd have room to implement that either.
@jwdonahue commented on GitHub (Oct 3, 2019):
@sjlongland wrote:
Well it would be VersionMeta actually, and ya, the addition of a tag or URL would take up more space. You bring up an important issue however that I haven't even thought to address for VersionMeta/Schema. Specification of binary formats might be useful. The topic of smaller version foot-prints, for embedded devices, has come up at least several times since this site went online. In today's world, it's not exactly a corner case. Pretty sure I left the door open to extend it however:
@sjlongland commented on GitHub (Oct 3, 2019):
Ahh okay, point taken: answer is "write your firmware once, to cater for all possible use cases the product will ever support, and never make a mistake that causes a security or stability issue."
@KyNorthstar commented on GitHub (Oct 14, 2019):
99.99.999-Alpha.999+9999is about the length of the longest semantic version I've encountered in the wild. It's 24 bytes long in ASCII. Is your embedded device really that constrained on memory/drive that it can't handle a 24-byte string? I think SemVer is probably lightweight enough as-is and this optimization isn't necessary...@jwdonahue commented on GitHub (Oct 14, 2019):
@sjlongland, nobody really cares how you encode the information on your device. It's not like your going to be sharing your code with anybody, so you don't really need SemVer for that. Just use a single 16, 32 or 64 bit unsigned integer and be done with it. Your proposed scheme is interesting, but not relevant to the SemVer spec.
Unless you're going to issue a PR with proposed changes to the specification, please close this thread at your earliest possible convenience.
@jwdonahue commented on GitHub (Oct 14, 2019):
Oh, sorry, I see you did close it. Thank you!