SemVer, NuGet, and Nightly Builds

Oct 24, 2011 versioning semver nuget suggest edit

Recently, a group of covert ninjas within my organization started to investigate what it would take to change our internal build and continuous integration systems (CI) to take advantage of NuGet for many of our products, and I need your input!

Ok, they’re not really covert ninjas, that just sounds much cooler than a team of slightly pudgy software developers. Ok, they’ve told me to speak for myself, they’re in great shape!

In response to popular demand, we changed our minds and decided to support Semantic Versioning (SemVer) as the means to specify pre-release packages for the next version of NuGet (1.6).

In part, this is the cause of the delay for this release as it required extensive changes to NuGet and the NuGet.org gallery. I will write a blog post later that covers this in more detail, but for now, you can read our spec on it which is mostly up to date. I hope.

I’m really excited to change our own build to use NuGet because it will force us to eat our own dogfood and feel the pain that many of you feel with NuGet in such scenarios. Until we feel that pain, we won’t have a great solution to the pain.

A really brief intro to SemVer

You can read the SemVer spec here, but in case you’re lazy, I’ll provide a brief summary.

SemVer is a convention for versioning your public APIs that gives meaning to the version number. Each version has three parts, Major.Minor.Patch.

In brief, these correspond to:

Major: Breaking changes.
Minor: New features, but backwards compatible.
Patch: Backwards compatible bug fixes only.

Additionally, pre-release versions of your API can be denoted by appending a dash and an arbitrary string after the Patch number. For example:

1.0.1-alpha
1.0.1-beta
1.0.1-Fizzlebutt

When you’re ready to release, you just remove the pre-release part and that version is considered “higher” than all the pre-release versions. The pre-release versions are given precedence in alphabetical order (well technically lexicographic ASCII sort order).

Therefore, the following is an example from lowest to highest versions of a package.

1.0.1-alpha
1.0.1-alpha2
1.0.1-beta
1.0.1-rc
1.0.1-zeeistalmostdone
1.0.1

How NuGet uses SemVer

As I mentioned before, I’ll write up a longer blog post about how SemVer figures into your package. For now, I just want to make it clear that if you’re using 4-part version numbers today, your packages will still work and behave as before.

It’s only when you specify a 3-part version with a version string that NuGet gets strict about SemVer. For example, NuGet allows 1.0.1-beta but does not allow 1.0.1.234-beta.234.

How to deal with nightly builds?

So the question I have is, how do we deal with nightly (or continous) builds?

For example, suppose I start work on what will become 1.0.1-beta. Internally, I may post nightly builds of 1.0.1-beta for others in my team to use. Then at some point, I’ll stamp a release as the official 1.0.1-beta for public consumption.

The problem is, each of those builds need to have the package version incremented. This ensures that folks can revert back to a last-known-good nightly build if a problem comes up. SemVer doesn’t seem to address how to handle internal nightly (or continuous) builds. It’s really focused on public releases.

Note, we’re thinking about this for our internal setup, not for the public gallery. I’ll address that question later.

We had a few ideas in mind.

Stick with the previous version number and change labels just before release

The idea here is that when we’re working on 1.0.1beta, we version the packages using the alpha label and increment it with a build number.

1.0.1-alpha (public release)
1.0.1-alpha.1 (internal build)
1.0.1-alpha.2 (internal build)

A variant of this approach is to append the date (and counter) in number format.

1.0.1-alpha (public release)**
1.0.1-alpha.20101025001 (internal build)
1.0.1-alpha.20101026001 (internal build on the next day)
1.0.1-alpha.20101026002 (another internal build on the same day)

With this approach, when we’re ready to cut the release, we simply change the package to be 1.0.1-beta and release it.

The downside of this approach is that it’s not completely clear that these are internal nightly builds of what will be 1.0.1-beta. They could be builds of 1.0.1-alpha2.

Yet another variant of this approach is to name our public releases with an even Minor or Patch number and our internal releases with an odd one. So when we’re ready to work on 1.0.2-beta, we’d version the package as 1.0.1-beta. When we’re ready to release, we change it to 1.0.2-beta.

Have a separate feed with its own versioning scheme.

Another thought was to simply have a completely separate feed with its own versioning scheme. So you can choose to grab packages from the stable feed, or the nightly feed.

In the nightly feed, the package version might just be the date.

2010.10.25001
2010.10.25002
2010.10.25003

The downside of this approach is that it’s not clear at all what release version these will apply to. Also, when you’re ready to promote one to the stable feed, you have to move it in there and completely change the version number.

Support an optional Build number for Semantic Versions

For NuGet 1.6, you can still use a four-part version number. But NuGet is strict if the version is clearly a SemVer version. For example, if you specify a pre-release string, such as 1.0.1-alpha, NuGet will not allow a fourth version part.

But we had the idea that we could extend SemVer to support this concept of a build number. This might be off by default in the public NuGet.org gallery, but could be something you could turn on individually. What it would allow you to do is continue to push new builds of 1.0.1alpha with an incrementing build number. For example:

1.0.1-beta.0001 (nightly)
1.0.1-beta.0002 (nightly)
1.0.1-beta.0003 (nightly)
1.0.1-beta (public)

Note that unlike a standard 4-part version, 1.0.1-beta is a higher version than 1.0.1-beta.0003.

While I’m hesitant to suggest a custom extension to SemVer, it makes a lot of sense to me. After all, this would be a convention applied to internal builds and not for public releases.

Question for you

So, do you like any of these approaches? Do you have a better approach?

While I love comments on my blog, I would like to direct discussion to the NuGet Discussions page. I look forward to hearing your advice on how to handle this situation. Whatever we decide, we want to bake in first-class support into NuGet to make this sort of thing easier.

Found a typo or mistake in the post? suggest edit

Comments

26 responses

Chris • October 25th, 2011
Hate going against a public spec, but we would prefer the following. Just seems more intuitive:
1.0.1beta.0001 (nightly)
1.0.1beta.0002 (nightly)
1.0.1beta.0003 (nightly)
1.0.1beta (public)
Robert McLaws • October 25th, 2011
This right here is actually why I dislike SemVer. It keeps things simple for simple scenarios, but falls apart when you have anything more complicated.
Personally, I like date-based build numbers a lot better. My build numbers work as follows:
MajorVersion.YYYY.MMDD.TTTT. Then you know everything you need to know about when the build was released.
You might be thinking "wow, you know I really hate it when people reply to a problem with a solution that does not apply/won't get adopted." And I would understand. But supporting a different versioning semantic is not what I'm proposing. Well, not entirely.
.NEt assemblies support two different versions: The Logical version and the File Version. Each supports separate scenarios. I suggest that NuGet continue to work the way it currently does with versioning, and do not change anything. Instead, support additional attributes, like "IsPreRelease" or "IsNightly" flags, s well as an arbitrary version string, to allow multiple versions to be in the feed without stepping on each other's toes.
In this case, the default public feed would show everything where those flags are off. The Package Manager UI would let you specify if you wanted to include PreRelease or Nightly builds in the feed. This would also have the benefit of being queryable, since the underlying feed is OData.
Otherwise, I think trying to cram too much into one version string will just cause more problems further down the line. This way, there can be a version string used for sort order, and a completely separate one for display only.
Brad Wilson • October 25th, 2011
Re: your examples, SemVer strictly only A-Z, a-z, 0-9, and dash (-) in the special version number. Your examples used underscores, which would not be legal.
Ken Egozi • October 25th, 2011
A slight change that I like is to use odd minor versions as "pre release" or "bleeding edge" that is followed by an even number for "RTM"
e.g. - version 1.5, 1.5.1, ..., 1.5.n while stabilizing the 1.6 version, calling it 1.6 when it's done, then 1.6.1 is bugfixes, and 1.7.x starts to act as the pre-release for 1.8 version (or a future 2.0)
this way you do not need the strings (beta, alpha, whateva), which is good imo - you do not waste time and cpu cycles on "is it a beta2 or an rc?" just increment patch-level for the odd version, and go even when you're ready.
plus number=only versions are easier on the AssemblyVersion thing
haacked • October 25th, 2011
@Brad, thanks! Fixed!
Betty • October 25th, 2011
I don't like the idea of separate versioning schemes, unless you can upgrade from a nightly to a stable with little effort.
I would drop the ability to have arbitrary pre-release names and stick with the understood stable, RC, beta, alpha, and nightly. Allowing users to pick the min quality they allow in the packages.config.
Along with Major.Minor.Patch.Build where patch can be the date in any mangled form you like (eg 20111026 or days since 2000) as long as it's an incremented integer.
nuspec: <version quality="stable|rc|alpha|beta|nightly">4.1.10715.0</version> packages.config <package id="blah" version="4.1.10715.0" quality="alpha" />
haacked • October 25th, 2011
@Chris Yeah, I'm hesitant to go against the spec, but since the spec only refers to "public" APIs, maybe it's fine to change it for private internal builds. Anybody know anyone responsible for SemVer? Maybe we can propose an amendment to it.
@Betty if we choose to follow SemVer, which has a lot of momentum behind it, we want to follow the spec as strictly as possible. However, within specific organizations, there's no reason you couldn't limit yourself to a standard convention.
In fact, I think we'll recommend that you use well known strings.
Brad Wilson • October 25th, 2011
By the way, my vote is for "Stick with the previous version number and change labels just before release" because it follows SemVer's semantics. Going against the grain seems like a bad idea. Either support SemVer properly, or don't support it at all.
Betty • October 25th, 2011
@Haacked, my only real issue with SemVer is that Nightly > Alpha/Beta which isn't the case. It's a fairly basic case and fairly core to support CI (imo), surely it must be covered by the standard somehow...
Damian Powell • October 25th, 2011
At our company we have 'extended' SemVer with a fourth digit which is a psuedo build-number. It's actually the number of revisions since the last SemVer tag in our repository. It's a shame that our approach doesn't fit with either SemVer or any of your propsoed changes. It's even more of a shame that the current SemVer strategy can't be used with [assembly: AssemblyVersion(...)] attributes.
If you want to officially extend the spec, then you need to contact Tom Preston Warner (his details are at the bottom of the SemVer.org page).
Damian Powell • October 25th, 2011
Obviously, I meant Tom Preston-Werner! Sorry, Tom.
Chris Chilvers • October 25th, 2011
One thing I think is worth noting is that package version does not have to equal the assembly versions. This is important because I might release build 1.0.1.546 as a release candidate, that works perfectly so I promote that build to release.
If the version number of the assembly had to match the package then to promote to release I'd have to re-build, but if I do that I have a new, untested, assembly that should really have to go back through being a release candidate.
The only other option is to look at a tool for re-writing an assemblies version numbers (both win32 and .net) without re-compiling. Though said tool will also have to take in to account debugging symbols and be able to re-write them as well as update the appropriate GUIDs and checksums used for debugging symbols, so this could end up a bit of a mess.
Rob • October 25th, 2011
I'm totally digging the idea of
1.0.1-alpha.###
The dash separation is perfectly legal, improves readability a bit, and makes a pretty nice separator between the version and the pre-release parts. It may even help in determining you have a prerelease because you can just scan the version for "-".
Showing an ordering:
1.0.0-alpha.001
1.0.0-apha
1.0.0-alpha1.001
1.0.0-alpha1.002
1.0.0-apha1
1.0.0-beta
1.0.0-rc
1.0.0

Thoughts?
dotnetchris • October 25th, 2011
1.0.1beta.0001 (nightly)
1.0.1beta.0002 (nightly)
1.0.1beta.0003 (nightly)
1.0.1beta (public)
This is fine.
I've also advised a few people that wish to use SemVer for real products that they want to follow
ProductCycle.MajorVer.MinorVer.Patch.
SemVer is totally awesome and required when you build packages with Nuget to avoid replacing DLL Hell with Version Hell. The only downside to SemVer is you will need to increment major version ALOT MORE frequently than you would realize if you don't follow SemVer.
Seeing as many people like to tie major versions with paid upgrades, having to increment the major version because you renamed 1 property is a bit stiff.
This is also potentially something you could evaluate for nuget, to use 4 version #s (matches .NET better too). And just increment the patch # for every CI build. By definition a nightly build should just be a patch increase, otherwise you've already introduced changes that should require a version increment further than a patch.
Tom Dane • October 25th, 2011
It seems to me semantic versioning is overrated. By releasing internal builds and alpha builds you're basically pushing half finished products on your users, basically forcing them to be your QA.
Responsible teams would not publish internal, alpha - or even beta - builds, but only ship the stable ones. And the old fashioned versioning perfectly supports this, why complicate it more?
Rob • October 26th, 2011
More thinking about this, I like the idea of reserving the forth number for packages to a package fix version. Or allow us to fix the package.
So
1.0.0-alpha~001
1.0.0-alpha
1.0.0-alpha.1
1.0.0-alpha1~001
1.0.0-alpha1~002
1.0.0-apha1
1.0.0-beta
1.0.0-rc
1.0.0-rc.1
1.0.0-rc.2
1.0.0
haacked • October 26th, 2011
BTW, great discussion on this happening over at our CodePlex site: http://nuget.codeplex.com/discussions/277189
Frode Rosand • October 27th, 2011
I support a strict semantic versioning. Two of the alternatives support that:
1.1.1alpha-001
1.1.1alpha-001
1.1.1alpha
or the odd/even minor version variant:
1.1.0 pre-release version
1.1.1 pre-release version
1.2.0 release version
Some of the products using the latter model does so strictly for officially available versions (nightly builds, unstable betas and so on), but it could be used regardless of public or private release. Not every release going public must follow the previous version sequentially. If you walk through 1.1.0 through 1.1.112 before you release 1.2.0 then that's fine.
Personally I like the cleanliness of the odd/even approach but would be fine with a non-breaking SemVer compatible 1.1.1alpha-001 structure.
Rafael Teixeira • October 31st, 2011
I second Chris Marisic, in that SemVer is not helpful when dealing with marketing decisions on versioning.
Most of our libraries (in nugets) are in sync with mandated product Major.Minor versions so only Build/Release numbers are available for semver so where Nuget is heading isn't going to be much helpful...
Probably we will need to evolve to something like Chris suggestion, over time, but pure SemVer is not a foreseeable scenario for us...
Stoyan • November 3rd, 2011
Looks like a lot of effort will be put into making NuGet something closer to http://maven.apache.org.
Why couldn't you guys just put your effort into http://incubator.apache.org/npanday/ instead of trying to re-invent the wheel every time :(
Our team was struggling with NuGet (was NuPack at the time), just as your colleagues would be - it's simply good enough for LOB web devs and single lib writers and for no one else. We're now in the process of moving to NPanday (too late with the semantic versioning + too far from Maven).
Koen • November 9th, 2011
@Stoyan I completely hate maven as a distribution/dependency management (Maven is a lot more: the ant-like build "system" is nice on the other hand)
A lot of companies loose money over Maven but don't realize it, or don't want to realize (yes there is experience behind that statement). But telling a java developer that they waste time with Maven is like telling ...
And telling a .Net developer the same about NuGet is almost the same... "Microsoft endorses it so it must be good".
So yeah I hear you, Nuget is great for solo developers, script-kiddies and the like, ...
Koen • November 9th, 2011
hmm first post seamed to have been lost so here attempt nr 2:
Besides SemVer I would like to add there are other "Semantic Versioning"-standards like:
www.osgi.org/.../SemanticVersioning.pdf
Which actually allow for a beta, rc2, ... etc. appendix.
Johannes Hansen • December 30th, 2011
The scenario you describe seems to be covered by SemVer 2.0.0-rc.1. Which as you can see from SemVer's own version allows dots after the string.
Section 11 in SemVer 2.0.0-rc.1 says the following about build versions.

A build version MAY be denoted by appending a plus sign and a series of dot separated identifiers immediately following the patch version or pre-release version. Identifiers MUST be comprised of only ASCII alphanumerics and dash [0-9A-Za-z-]. Build versions satisfy and have a higher precedence than the associated normal version. Examples: 1.0.0+build.1, 1.3.7+build.11.e0f985a.

Note that this allows the usage of "+" for build versions.
Section 12 goes on to describe the precedence calculation:

Precedence MUST be calculated by separating the version into major, minor, patch, pre-release, and build identifiers in that order. Major, minor, and patch versions are always compared numerically. Pre-release and build version precedence MUST be determined by comparing each dot separated identifier as follows: identifiers consisting of only digits are compared numerically and identifiers with letters or dashes are compared lexically in ASCII sort order. Numeric identifiers always have lower precedence than non-numeric identifiers. Example: 1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0-rc.1+build.1 < 1.0.0 < 1.0.0+0.3.7 < 1.3.7+build < 1.3.7+build.2.b8f12d7 < 1.3.7+build.11.e0f985a.
Johannes Hansen • December 30th, 2011
Oh, and I think you should adhere to SemVer 2.0.0-rc.1, not SemVer 1.0.0
Jason • January 15th, 2012
So let's think about the patch versioning for a moment. 1.2.3beta is what will *become* 1.2.3. So why would you add build numbers to the previous patch version? Shouldn't you be adding them to what the build will become?
If you just released 1.2.3alpha, then the next build should be 1.2.3beta-{buildstamp}. In this manner, the buildstamps are still ordered lexicographically so they still function correctly as a patch number. Additionally, when the pre-release is ready for public consumption, the buildstamp is simply dropped. Now the pre-release version is still higher-order lexicographically so it's in the proper order. Also, the method of incrementing from pre-release to release is the same as nightly to public. Simply drop the buildstamp to go public; similarly drop the pre-release (alpha, beta, etc) to release the patch.
Jakub Konecki • February 7th, 2012
I wonder if someone could answer my question on SO:
stackoverflow.com/...
I'm using a custom NuGet feed for deploying own NuGet packages. I'm using semver so my CI server is generating a deploying new prerelease packages on every build. Those prerelease packages are obviously not visible by default in Package Manager.
Is there a way to specify on package / feed / global level that prerelease packages should appear under Updates tab in Package Manager?
If I install a prerelease package by for example editing packages.config file my package is clearly marked in Package Manager with red 'Prerelease' label, so the manager understands versioning properly.