Base Action URL

Vision for Productized uPortal

Andrew Petro -

In which I articulate a vision for a uPortal that feels much more like a product.

Executive Summary

The next generation of uPortal should be more like a product and less like a construction set.

Most adopters will adopt pre-fabricated ready-to-go artifacts (think: Docker containers). They will point it at a database, edit some CSS and add their own .jars and JSON sources to fulfill well defined product APIs, and the more of that they do, the richer the product will become.

Got

  • Office365 / Google Apps for Education calendars / Bedework / some other calendar
  • a way to marshal course, grades, or exam locations as JSON
  • middleware for reading and storing personal profile information
  • W2s as PDFs you’d like to distribute to employees

?

Great! Point the product at those things, and those features become available.

Don’t have some or any of those things? No worries. The product will gracefully degrade such that an adopter can implement any sub-set of the APIs and the product will provide the best available experience with what it has to work with.

Most adopters will not edit a local copy of the source code for most modules of the product and will not build the product from source. They may build their own add-ons and plugins from source, but will regard most of the product as static from the perspective of their local implementation. (Oh, and those add-ons may not be written in Java.)

Why???

So that:

  • the product is radically more adoptable.
  • it is radically more feasible to keep up with ugprades
  • effort moves from local customization to engineering the shared product itself
  • multiple institutions can genuinely be deploying and collaborating on the very same open source software
  • the return-on-investment in open source collaboration in this domain improves greatly for all concerned
  • cloud adoption models are better enabled. Adopting uPortal in the cloud via a SaaS vendor should not be your only adoption option, but it should be an option, and a compelling one at that.

Disclaimers

Everybody chill out.

This is teeing up a vision for discussion. It’s not forking uPortal, and it’s not unilaterally saying where uPortal must go. Maybe these ideas go nowhere.

Better is better. Make progress now.

This vision is not cause for you to put your campus IT implementations on hold and hold your breath waiting for the Rapture. That tends to go badly. Serve your users now with what is as best you can. :) Do not sit on your hands waiting for the Next Great Thing.

I speak only for myself

In this post, as always in this blog, I speak only for myself. These are not necessarily the opinions and visions of my colleagues, employer, of the uPortal project, of Apereo, etc.

I have scribbled on a whiteboard a few times. To the extent that these are good visions, they are informed by valuable conversations with colleagues.

Be part of the visioning

Be a part of envisioning and working towards The Next Great Thing.

Maybe these ideas or ideas leading from them lead to something wonderful to deliver excellent portal experiences to campus users and to enable a next generation of productive collaboration and collegiality among higher education IT professionals. That would be neat.

Grab a cup of coffee

This is long form. I have done my best to summarize, to organize with headings, and to highlight for skimming.

If you just read the highlights, maybe you could just slam an espresso and be about your day.

Vision

In this vision, the next generation uPortal will be:

  • A product. You grab a ready-to-deploy image, you plug in your sources of interesting data, you edit some CSS to skin it, you edit some configuration, and 80% of adopters are ready to turn it on in production to serve their constituents.
  • A responsive, usable, accessible single-page web application that never disrupts the user with a slow, annoying request-response full page refresh.
  • An able microservices container serving up lightweight JSON web services feeding those rich UIs, and optionally feeding rich native applications for mobile devices and serving other applications.
  • A set of well defined, semantically versioned JSON web services and Java APIs for the domain of higher education portals (APIs about courses, paystubs, benefits, leave balances, registration processes, personal information, emergency contact information, web conferencing scheduling, …)
  • Sets of re-usable, semantically versioned implementations of those APIs for integrations with local data sources and commonly adopted systems
  • And yes, a platform for delivering additional experiences not provided by the product itself through the addition of local custom code — but this last and least, not first and foremost.

In this vision, the next generation uPortal will have a more sustainable and efficient product development model.

Being a product

Maybe the best way to articulate what this means is to describe how 80% of adopters ought to experience this product.

Adopting the product for the first time

Suppose an adopter wants to adopt uPortal 17.1, going from nothing to a portal.

The adopter goes to apereo.org , downloads the uPortal 17.1 product, sticks it on a server (it’s probably a Docker container or so), and turns it on. It displays a message saying “Yo, uPortal needs a database. See the readme!”

Okay. The adopter reads the readme, sets up a suitable RDBMS schema, and maybe edits a text file or so to tell the product where this database is.

Now the adopter tries the product again in a browser. It says “Yo! I think I am a new deployment, because I see an empty database. If this is so, let’s go through this wizard for initial setup.”

At the end of that wizard, the configuration is in that database.

There’s a well-defined skinning API involving fulfilling semantically defined CSS classes and providing logo and background images. In the implementation process the adopter customizes and uploads a skin to make the product largely comport with the institutional brand.

Implementing the product

Product implementation involves understanding and fulfilling the APIs the product is able to consume.

So, one feature the product has is it can give employees access to their W2s. In order to make the product actually do this, the adopter needs to fulfill a well-defined Java API representing a source of information about, for a given user, what W2s are known and from what URLs can they be viewed or downloaded.

Suppose an adopter has such W2s and wants to go ahead and make those available through the portal.

Well, the adopter could write a simple Java class or two fulfilling this API, compile it into a .jar, and drop it into the product. The product would discover this, activate it, and try to use it to fulfill this API. If it worked, the feature would appear for supported users; if not, no worries, the feature would give an appropriate degraded error experience and a clear warning would appear in the administrative UI and logs about the feature seeming to be intended to be fulfilled but this failing.

But wait, there’s more. The product also comes with an adequate bridges-to-JSON implementation of this API that simply reads a well-defined JSON web service to fulfill this information. So instead of writing any custom Java, the adopter can instead, through an administrative UI, configure the URL at which this web service is fulfilled and some metadata about how the portal ought to authenticate to it.

So the adopter does not actually have to write any Java — the process fulfilling the API could be a .NET application, or a lovingly handcrafted PHP thing, or could be a batch process that drops plain old text files in the right format with the right names in the right directories on a plain old httpd server.

The point is, the product incrementally meets the adopter where the adopter is at. The more APIs you fulfill, the more the product does, but no worries if you’re not ready to fulfill an API or even if you fulfill it imperfectly.

Upgrading

Progress progresses, as it does.

A new version of uPortal, uPortal 17.2, is released.

Being semantically versioned, uPortal 17.2 is perfectly backwards compatible with 17.1, which is to say, it is a drop in replacement. It is 17.1 except less buggy and more featureful.

You grab the new artifact, point it at your current configuration, and you're upgraded. Done. No worries.

In 17.2, that W2s API improved such that you can still give the portal the URL of a W2, but you can also give the portal the actual W2 data and the portal will go ahead and display the data attractively on the screen and even generate a downloadable PDF if you like. This is fed by a new version of the W2s API with a new version of the W2s JSON web services.

But it is backwards compatible. So, point it at the boring old 17.1 version of that JSON web service, and it just works, as it did before. Point it at the new version of the API, and it does more.

The adopter reads the release notes, identifies what relevant APIs improved, determines business value in feeding better data to the portal, and queues local efforts to fulfill the better APIs. Or not, if you liked your portal just fine the way it was thanks.

The point is, upgrades are low hassle for adopters, so adopters do them, and like doing them.

Developing

Ah.

Here’s the fundamental idea in this vision:

  • Product implementation is local and trivial.
  • Product development is shared and rigorous.

So, how did that W2s API improve between version 17.1 and 17.2? Where did the code implementing that new better experience come from, and who wrote it, and how was that effort marshalled?

Well. Some adopter needed that feature and applied development effort through the uPortal product community software development process to achieve the changeset. The feature was

  • pitched to the uPortal adopter community for input and feedback and co-development of the idea, was
  • well-designed with user-centeredness, responsiveness, and accessibility firmly in mind. Maybe there was a design sprint . Certainly there is radical openness to feedback.
  • pitched to the uPortal developer and committer community for input and feedback and co-development of the implementation
  • developed using test-driven development practices achieved prescribed test coverages and comporting to Google’s rigorous code style guide
  • carefully semantically versioned so as to not break backwards compatibility
  • developed according to an opinionated uPortal product style guide so that it looks and feels and behaves like other uPortal modules so that the product feels like a cohesive whole,
  • thoroughly respectfully optional so that existing adopter user experiences are not inappropriately disrupted.

That’s a lot of work and care! But by doing that work in the uPortal product itself, when that 17.2 version released, the university then locally had a seamless and pleasant adoption process of the very feature it developed (or got a consultant to develop).

Development is in the product, in the open source project.
Adoption is local. We have more in common than we have locally different, so much so that we can design and build a shared open source product that differs locally only through configuration and through what integration APIs we choose to fulfill.

By moving rigor into the product development process, we move hassle out of the product adoption process.

We spend less local effort mucking about with Maven overlays on overlays and with git forks and merges and rebases oh my and instead spend effort building shared semantically versioned modules that can be locally adopted without such complexity.

Thoughts on how to achieve this vision

This post is primarily about the vision.

Nonetheless, an unachievable vision isn’t worth much.

I think this vision is quite achievable, however. I think we have progress towards it. In some ways it’s a different take on things we already do, with much more panache.

So, what do we need to do to achieve this vision?

Semantic versioning

Semantic versioning, and some good nuances of what that means, are essential to easing the upgrade process. Stable APIs or at least clear API versioning expectations make adopters more able to choose to write to and rely on an API without exerting direct source code control over the uPortal component consuming that API.

Well-defined semantically versioned higher education domain JSON APIs

We already do some of this, in options for how data is fed to e.g. the Courses Portlet. This is doing that more carefully in a less coupled way.

Well-defined semantically versioned framework APIs

Key to making it sustainable to write implementations to the APIs is having well-defined, semantically versioned such APIs such that a plugin for a given uportal product version just works for that version series.

Compelling features

The product should do great things.

Notifications might be a fine place to dig in.

Radically better code quality

Test coverage, Google code style adoption, looser coupling, better definition, and more smaller modules well-versioned.

Style guidance and design panache

An opinionated style guide that’s good and with those opinions adhered to in the product is essential to making these UI components look good together and to achieving the magic of no matter how the product gracefully degrades the remaining experience looks whole.

Maybe the MyUW style guide (source) would be a decent starting point.

Predictable release cycle

If you know there’s at most a month lag from your getting your new feature into the product and a binary release of the product including your feature, that should reduce the pressure to locally fork to get that feature sooner.

Better, more modular technologies

While the product is in some ways envisioned as a monolith, a big, static, complete solution that adapts to available APIs and degrades gracefully, that is, it is adopted as a convenient whole….

… on the development side, that product does not have to actually be a monolith. It should consist of small, well defined, cohesive, semantically versioned modules that are orchestrated together into the whole that adopters regard as The uPortal Product.

So. Docker. AngularJS. Directives. Partials. JSON. Spring Cloud Netflix.

Beyond portlets and beyond portlet containers.

Acknowledgement

Photo credit : zabowski on flickr, via CC-BY-ND-2.0.