Sign in Subscribe
uportal

uPortal should adopt Semantic Versioning

Andrew Petro

• 16 min read
uPortal should adopt Semantic Versioning

uPortal should adopt Semantic Versioning. uPortal should also adopt some other practices supported and improved by Semantic Versioning.

This is mostly just moving a decimal point, but it is the right move for the decimal point to make at this juncture.

Semantic Versioning is about nomenclature and what changes go where and are called what. It isn't itself anything about how often uPortal releases or what changes uPortal adopts how fast or how you as an adopter adopt, extend, develop to, or enhance uPortal. Semantic Versioning will expose some pain points that already exist and provide standard terms of art for talking about this, but it will not cause any new pain.

Edits

March 2016 :

  • Added JSON web services as an example of an exposed API to be semantically versioned.
  • Adjusted examples to contemplate adopting Semantic Versioning for a 5.0 release.
  • Copy edits

This post

  • What is Semantic Versioning?
  • Why?
  • How this relates to the current versioning scheme
  • How uPortal could make the transition
  • How this relates to other ways to make uPortal better
  • Examples of projects with experiences with Semantic Versioning
  • Maven automation of Semantic Versioning compliance checking

What is Semantic Versioning?

Semantic Versioning is a standard for the meaning (semantics) conveyed by version numbering.

Here's the summary from the Semantic Versioning definition:

Summary

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes,

MINOR version when you add functionality in a backwards-compatible manner,

and PATCH version when you make backwards-compatible bug fixes.

There's more thorough discussion beyond the summary on the Semantic Versioning website from whence that definition is drawn.

Why?

uPortal should adopt Semantic Versioning because it

  • Provides a common, intuitive language to talk about versions
  • reduces risk to adopters in keeping up with patch releases
  • clarifies the meaning of minor releases and reduces risk to adopters in keeping up with minor releases
  • Supports developing external plugins for uPortal
  • improves feasibility of documentation and upgrade path development
  • reduces the friction of merging changes across multiple live branches
  • makes uPortal progress more marketable

Common language for talking about versions

Semantic Versioning allows adopters and participants to readily understand how versions relate to one another.

Shawn: What do uPortal version numbers mean?

Dave: Well, starting with uPortal 5, uPortal uses Semantic Versioning.

Shawn: Great!

and if you're not already familiar, well, it's well defined elsewhere.

Jason: What do uPortal version numbers mean?

Dave: Well, starting with uPortal 5, uPortal uses Semantic Versioning.

Shawn: I don't know what that means.

Dave: It's easy to learn -- read over www.semver.org a couple times and then we can talk over what uPortal considers to be "API" and thus meaningfully "backwards-compatible" across MINOR releases.

Using Semantic Versioning is analogous to using the Apache 2 license. If you want to explain to someone what Apache 2 is and how it works and why it's a good idea, well, that ends up being a long explanation like this. But if you want to accurately, succinctly communicate uPortal's licensing strategy, you can say "uPortal is licensed under Apache 2.", and that fits in a tweet.

Likewise, if you want to explain to someone what Semantic Versioning is and why uPortal ought to adopt it, well, I guess you end up with a long writeup like this, as well as the actual Semantic Versioning definition. But if you want to accurately, succinctly communicate uPortal's versioning strategy once uPortal adopts Semantic Versioning, you could just say "uPortal uses Semantic Versioning.", and that too fits in a tweet.

Maven Central suggests using Semantic Versioning.

we strongly suggest that you [use] semantic versioning (http://semver.org) to assist your users in their version choice.

Reduced risk in keeping up with patch versions

Semantic Versioning defines a very conservative, bugfix-only, backwards compatible patch series. This would reduce risk and analysis load on adopters in keeping up with patch releases within a Minor version (e.g., upgrading from 5.1.1 to 5.1.2) since the changeset between the two would be exclusively bugfixes on functionality included in 5.1.1.

Shawn: What's uPortal 5.1.2?

Dave: It's uPortal 5.1.0 with some bugs squished. Even more bugs fixed than were fixed in 5.1.1!

Shawn: You mean there's no new features or non-bugfix enhancements I need to understand in contemplating and executing on this upgrade?

Dave: Not a one.

Shawn: Groovy! I'll upgrade straightaway then.

Focusing the maintenance branch only on bugfixes makes the branch less noisy and less change-ful, which should reduce the level of effort required of adopters in keeping up with the patch releases.

Reduced risk in keeping up with minor releases

Semantic Versioning defines a conservative, backwards-compatible minor version series. This would reduce risk and friction on adopters in keeping up with minor releases within a Major version.

So, an adopter of 5.2.1 could look to upgrade to 5.3.3 (to pick up the new features in the 5.3 version) confident that 5.3 version is "backwards compatible" with the currently adopted version (5.2.1).

Semantic Versioning is about APIs. uPortal doesn't currently do a great job of differentiating APIs (which ought to be kept compatible within a Major version) from implementation that is not APIs (which could change within a Major version). The current adoption model of locally forking the code and locally hacking away isn't helping this, since it enables treating the whole shebang as malleable. So there's some challenge here and uPortal will have to work out what "backwards compatible" means in its context. uPortal also might work to become more rigorous about differentiating API from implementation, and the implicit assumption that most things are API until proven otherwise might help drive that differentiation.

So, what's an API? Maybe all these things:

  • Public interfaces in the uPortal codebase, to which adopters might have written implementations.
  • Public non-final classes in the uPortal codebase, to which adopters might have written sub-classes.
  • Values exposed to the XSLTs and the flags and switches adopters can set to affect the XSLT
  • JSON web services exposed
  • The effective API exposed to skinning, such that skins would keep working across minor upgrades
  • Configuration files (the interface whereby adopters configure the portal), including Spring configuration and properties files.

Or maybe we can countenance some change in those things. Might it be acceptable to require adopters to add a new portal.properties property on upgrading between minor releases, so long as care is taken with decent release notes that call out these changes? I think so. Might it be acceptable to require adopters to adjust Spring context configuration on these upgrades? I think yes on that too, again so long as lovingly documented in release notes accompanying the minor release. I'd say it's backwards compatible if the adopter can reasonably achieve what they were achieving with their prior configuration, even if they need to make configuration changes that are straightforward and well-documented (and preferably automated in an upgrade script).

You might think differently. Perhaps you see requiring adding a new property or adjusting some XML configuration as not-backwards-compatible change. Perhaps your view is that it's only backwards compatible if literally exactly the adopters's existing configuration files will work unchanged.

Fair enough -- coming to consensus about what backwards-compatible means will be valuable for uPortal, and is something the project at least implicitly had to define in the current regime of backwards-compatible PATCH releases that afford improvements and new features. Semantic Versioning doesn't create the problem of needing to understand what is meant by backwards-compatible in uPortal, it just reveals and reminds that this is important.

But wherever the project falls on the questions of just what is meant by "backwards-compatible" minor releases, this will be less upgrade pain across minor releases than the status quo of accepting some amount of not-backwards-compatible change on minor releases.

Shawn: What's uPortal 5.2?

Dave: It's uPortal 5.1 with some backwards-compatible improvements and new features added.

Shawn: You mean there's no non-backwards-compatible change that will add friction to my upgrade?

Dave: Not a one. Backwards-compatible is kind of a nuanced concept. uPortal is maturing as to how API-flavored it considers properties files and configuration, so you may need to update those. All this is well-documented in the release notes. And you should probably consider whether you'd like to make use of the new features (also well-documented in the release notes). So this is more hassle than a patch release upgrade, but still, you can think of it as a feasible upgrade project.

Shawn: Hmm. That sounds like a little pain, but I bet I can manage it. I'll upgrade soon.

Dave: It probably won't be too bad. Dig into those release notes and see how rough the upgrade steps seem relative to your uPortal implementation.

Enables developing plugins

This is perhaps the most important advantage of Semantic Versioning. It sets the stage for defining APIs and versioning them in such a way that they remain viable throughout the remainder of the applicable Major version of the product.

Currently, MINOR releases are not backwards compatible, and so if you write a DLM Audience Evaluator for uPortal 4.1.4, it may not work in 4.2.0. If you package the new evaluator directly in the uPortal codebase, then it will not magically keep up-to-date with breaking API changes, but its brokenness will become apparent more readily, and so there's pressure towards a monolithic codebase. A bucket of code out in Jasig/uPortal/uportal-war/src/main/java . A bucket of code in that directory in your local fork of uPortal. There isn't a lot of feasibility in packaging your new DLM Audience Evaluator as an add-able .jar file in a separate repository, versioned and published to Maven Central even. Part of why that is not feasible is that if you did manage to do that, it would reliably work only within a MINOR release under the current approach, whereas it would work across the entire remainder of a MAJOR release under Semantic Versioning.

Semantic Versioning is by no means the only improvement necessary to better enable writing externalized plugins, but it is a piece of the puzzle.

Improves project ability to document and support

Under Semantic Versioning, uPortal 5.3.0 will be a thing with a fixed featureset. The features don't vary with more goodies coming in at points along the 5.3.1, 5.3.2, etc. patch series. This improves the project's ability to document both what 5.3 is and to document or even automate how to upgrade from it to 5.4.

Reduces friction in implementing improvements and new features

Under Semantic Versioning, new features and improvements do not go into the patches branch for the minor release under maintenance, so there isn't this constant drag of, for each new feature or improvement realized in master, considering and executing on also merging that feature into the patches branch for the current release. There isn't that judgment call or discussion about whether the feature belongs in that patches branch. There is not that opportunity to, in an understandable urge to go faster, just go ahead and cherry pick it over without testing it. The answer is just no. There is one fewer place to merge it, one fewer place to test it, and less noise on a branch that is about stability and bugfixes. This should allow the product to develop faster since adding an improvement or feature requires touching fewer active product maintenance branches.

Makes uPortal progress more marketable

If I told you that more than two years elapsed between the 4.0 release and the 4.1 release, you might assume that uPortal as a product shipped no new features for many months on end. That is not actually so, since all sorts of new features were showing up in 4.1.x releases in that interim, but releasing 4.0.14 just sounds a lot less momentous than does releasing 4.1, because other products one encounters don't typically hide their light under a bushel like this.

uPortal is developing and shipping new features. The project should take credit for that by calling the releases that include those new features what they are: minor releases, not patch releases.

Makes uPortal release engineering more feasible

Since there's less change, only bugfixes, in a patch release, it should be easier to validate and ship a patch release. No new features to verify and test, just bugfixes.

How does this relate to how uPortal currently does versioning?

The very short version is that adopting Semantic Versioning is just moving a decimal point.

Current Regime

uPortal currently allows backwards-compatible enhancements and new features in patch releases, besides bugfixes.

So, there had been fifteen patch releases on uPortal 4.0 as of this writing. 4.0.1. 4.0.2. all the way up to 4.0.15. Most of these introduced new features and enhancements besides bugfixes.

uPortal currently allows non-backwards-compatible changes that do not inflict too much pain on upgraders in minor releases. How much pain is allowed has varied all the way up to database schema changes and needing to run export-import of data.

Major releases have been reserved for revolutionary change, like dropping support for IChannel.

Mapping the Current Regime to Semantic Versioning

Semantic Versioning moves the decimal point one place, roughly.

So Semantic Versioning invents something uPortal doesn't have currently, PATCH releases that exclusively provide backwards-compatible bugfixes.

uPortal's current PATCH releases including backwards-compatible improvements and new features as well as bugfixes become MINOR releases under Semantic Versioning.

uPortal's current MINOR releases including non-backwards-compatible change become MAJOR releases under Semantic Versioning.

uPortal's current MAJOR releases including revolutionary change also become MAJOR releases under Semantic Versioning. Semantic Versioning doesn't differentiate between breaking an API a little bit or a lot -- a change is either a non-backwards-compatible change giving rise to a MAJOR release or it is not.

How would uPortal adopt Semantic Versioning?

In order to implement Semantic Versioning, uPortal as a project needs to discuss and define and come to consensus on what "backwards compatible" means to uPortal, so that the rules between MINOR releases will be well understood.

If uPortal were to adopt Semantic Versioning for the 5.0 release, uPortal would:

  • close the rel-5-0-patches branch to changes that are not bugfixes.
  • close the master branch to changes that are not backwards-compatible.
  • welcome development of necessarily non-backwards-compatible enhancements and new features on feature branches. When sufficient value of such feature branches accumulates to outweigh the cost of breaking backwards compatibility, the developers agree that it's time for a MAJOR release, merge these to master, and cut a uPortal 6.0.0 release from master rather than a 5.x release.

How does this relate to other ways to make uPortal better?

Semantic Versioning is itself just about nomenclature -- about what changes are made in what source control branches resulting in releases with what labels, how long versions are maintained, etc. Semantic Versioning does not in itself compel anything in particular about release schedule, product architecture, or anything else, really.

Semantic versioning doesn't compel adopters to be more conservative or more bleeding-edge. It does make that choice clearer -- the patch release path is most conservative; grabbing random proposed new-features branches is most bleeding-edge.

Semantic Versioning does support and enable some follow-on practices and policies that might also help.

Rapid predictable release cadence

I think uPortal needs a more rapid and predictable release cadence. I envision something like this:

  • Monthly patch releases of the latest minor release. So, after uPortal 5.0.0 is released, (exclusively) bugfixes go into the rel-5-0-patches branch yielding monthly patch releases of 5.0.1, 5.0.2.
  • Quarterly minor releases of master. So, after uPortal 5.0 is released, master becomes the branch for developing backwards-compatible enhancements and new features towards a 5.1 release. Three months or so later, 5.1.0 is released.
  • Annual or so major releases. So, after 5.0 is released, expect a 5.1 three months later, a 5.2 three months after that, a 5.3 three months after that, and then instead of a 5.4, a uPortal 6.0 release which merges in necessarily-not-backwards-compatible feature branches into master.

Semantic Versioning supports more rapid release cadence by making the relationships of the releases more apparent from their version number labels.

Closing out older versions' maintenance branches

It would be best to minimize the number of branches being maintained, so I envision three main active branches at any given time:

  • The previous release's maintenance branch. So, after 5.3.0 releases, the rel-5-2-patches branch becomes more conservative, accepting only important security fixes.
  • The current release's maintenance branch. So, after 5.3.0 releases, rel-5-3-patches branch is for bugfixes on 5.3.0 and accepts bugfixes.
  • The master branch. So, after 5.3.0 releases, master accepts backwards-compatible enhancements and backwards-compatible new features towards uPortal 5.4.

So at any given time, there is master marching along with backwards-compatible enhancements, there the current minor release maintaining with bugfixes, and there is one further minor release back maintaining with security bugfixes only.

Combined with a more rapid release cadence, this might be too aggressive for some adopters to keep up. After all, un-tempered, this would mean that uPortal 5.3 would stop receiving bugfixes three months after 6.0 releases and would stop receiving security fixes three months after that. The project could address this concern by inventing a Long Term Support Release strategy akin to that of Ubuntu Linux. Say, the second Minor release of any given Major release (e.g., 4.2, 5.2, etc.) becomes the Long Term Support release, with its patches branch remaining open and cutting bugfixes for a very long time, say one year past the release of the next Long Term Support release. (That'd be roughly 2 years) Adopters seeking a very conservative adoption and upgrade strategy where they intend to hang out on a Major version for a while would leapfrog from Long Term Support version to Long Term Support version, upgrading every 18 months or so, say.

That incurs the pain of a longer-running additional patches branch to keep patching and releasing, but at least it focuses that effort to get maximal value from keeping around such a conservative line of maintenance with fewer such long-lived maintenance branches.

I would favor adding that complexity only in response to adopter complaints that the release cadence and versioning strategy is causing pain by moving too quickly.

Semantic Versioning supports closing out older versions' maintenance branches by defining what sorts of changes are allowable in a minor release and so discouraging unbearable impediments to upgrading. Adopters keeping up with the releases do not miss anything by older maintenance branches closing on versions they are no longer using.

Attention to the upgrade path

It would be great to apply renewed attention to the upgrade path, upgrade path documentation, and upgrade tooling, so that adopters are more able to keep up with releases in a timely manner so that the uPortal community remains more focused on the progress in recent releases and so that adopters realize value from new releases sooner.

Semantic Versioning supports attention to the upgrade path by constraining how much a given Minor version can change across its patch releases and so creating a tighter from-to upgrade path story to document in the release notes of the minor release.

Semantic Versioning will also force determining what it means for Minor releases to be backwards compatible, which is in part attention to the upgrade experience.

Move the manual into source control co-located with the product source code

Semantic Versioning should result in more frequent Minor releases of uPortal since that becomes the way the product ships enhancements and new features to adopters. uPortal's current strategy of forking a manual Confluence wiki space for each minor release will yield another wiki space for each of those releases. It might be healthier to move that documentation into source control co-located with the portal codebase so that it naturally branches with the releases.

Separate API from implementation

Semantic Versioning is about API compatibility. It's the API that must be backwards compatible between MINOR releases, and the ability to revise APIs that makes MAJOR releases momentous.

uPortal does not currently do a great job of clearly communicating what will be treated as an API and so kept backwards-compatible between minor releases and what is not an API but is just an implementation detail and so may evolve in non-backwards-compatible ways between minor releases.

There's a lot of potential value in defining APIs and Semantic Versioning will pay dividends in enabling integration with and developing to those APIs.

For example. Suppose I develop a nifty new group store. Let's call it the "Nifty Group Store" for illustrative purposes. Suppose you're on uPortal 4.2.1.

Well, under Semantic Versioning, it's not permissible to include this new feature in uPortal 4.2.2, so you can't get it by upgrading to the next patch release. Sorry, that's just too much noise and complexity in the patches branch.

But, under Semantic Versioning, the APIs uPortal exposes must be backwards compatible, and so if uPortal defined and exposed a suitable API for adding this plugin, then I could compose it as a separate project dependent upon uPortal's API, and it would work in whatever major version of uPortal I targeted and all later minor and patch releases of that major version. Semantic Versioning allows exposing APIs with constraints on how those APIs change so there's a sufficient window of opportunity for a plugin to remain compatible with the product and thus valuable.

This technique allows you, the adopter, to pick and choose which plugins you want in your portal, just as you choose portlets, as well as allowing the product to pick up the feature when it's mature enough through dependency in a MINOR release.

Getting more of the code out of the uPortal repository would be a good thing. Even if uPortal decides the Nifty Group Store ought to ship in the product, the source code for the Nifty Group Store needn't be hanging out in one massive monolithic source directory with the rest of the codebase and its unit tests needn't be run every time uPortal is built, needn't contribute to the build time.

Separating API from implementation, writing to those APIs, and decoupling could be healthy and bear many fruits, and is a separate consideration from what features ought to ship in the product, since this allows a strategy for including these features without having to include their source code directly.

Semantic Versioning makes creating and maintaining plugins more feasible because APIs hold constant across MINOR releases and so a lovely plugin can be written to 5.0.3's APIs, say, and expect to remain working on all uPortal 5 versions (including, say, 5.2.1).

Enable adoption models that don't require forking the code

Semantic Versioning relates somehow to enabling adoption models that don't involve forking and modifying the codebase of the uPortal product itself. If you are going to have separate code that works with and relates to the uPortal product code, then you need APIs to write to and those APIs need to be stable long enough to be useful.

Enabling adoption models where the uPortal product is a binary dependency to which local configuration and special sauce code is added could be a very healthy evolution for uPortal in that it would allow adopters to experience it more as a product, to treat it is a product, and would involve greatly easing the adoption and upgrade pathways.

Of course, Semantic Versioning does not in itself enable such models. Much other engineering work will be required work that out. But Semantic Versioning is a supporting policy, one small step down the road to treating uPortal more like a product.

Other projects that have or have not adopted Semantic Versioning

These are not all great reads -- these are raw examples I read in thinking about Semantic Versioning and uPortal.

It's not all rainbows and butterflies -- there are experiences to learn from.

Directions for future research

I came across a Maven plugin that enforces semantic versioning. Of course, agreeing on and documenting policy is a wonderful thing, but automating enforcement of that policy is even better since it reduces the vigilance load on the committers and release engineers and avoids putting humans in the position of having to be a nag about things best checked by robots.

Cover image : XKCD #1296 : CC-BY-NC.

Sign up for more like this.

Enter your email
Subscribe