Base Action URL

Recovering from a broken staged Sonatype Maven repository

Andrew Petro -

The uPortal 4.1.1 release process staged a broken Maven repository in Sonatype. I only noticed this after pushing the 4.1.1 release commits and tags to GitHub. This is the story of how I recovered.

What went wrong

In the uPortal release process, the release engineer runs

mvn release:prepare

and then

mvn release:perform

Among other things, these commands update local metadata files associated with the Maven build (pom.xmls), commit those changes to local Git, and push built artifacts and signatures on those artifacts up to the Sonatype staging repository for preparing open source software releases for publication to Maven Central.

When I released uPortal 4.1.1, I ran those commands, and they seemed to be just fine. The release process called for locally building the quickstart and testing, which I did, and then pushing the commits up to GitHub, which I did, and then Closing and Releasing the staged Maven repository from Sonatype, which I tried to do.

Oops.

Apparently the staging process can fail to push some of the artifacts up to the Sonatype repository such that the repository is incomplete. In this particular case an md5 checksum file was outright missing. When i tried to close the repository, the repository failed the Central-suitability checks.

By the time I realized this, I had mvn clean'ed my local environment and no longer had copies of those signed artifacts.

Things that didn't work in fixing it

So, now I had a GitHub state that reflected the uPortal 4.1.1 release commits and tag, but I didn't have a proper staged Sonatype repo to close and release.

mvn deploy

So, mvn package locally builds the stuff that your Maven project is supposed to build. .jars and .wars and .ears, oh my. And mvn install further installs that stuff into your local Maven repository (e.g. that .m2/repository/ directory.

But mvn deploy, that's supposed to deploy your stuff out to the remote Maven repository, right?

Eh. Naive invocation didn't work. In releasing open source stuff up to Maven Central, one likes to cryptographically sign the artifacts. Apparently naive invocation of mvn deploy doesn't do that, which is one reason it's not sufficient.

Release the tag

So I tried naively checking out the published uportal-4.1.1 tag and mvn release:prepare, mvn release:perform ing on that.

That didn't work either. The release plugin likes to work from a SNAPSHOT state, not a tagged version state.

Manually generate the one missing signature

It probably would have worked to manually generate the one missing signature and use it to fix the exiting staged repository or to build a new correct repository. I didn't look into this option because the solution I'm about to describe worked first.

Of course, had I kept around that .md5 file then fixing the repo to include the needed file would have been a great way to get un-stuck.

What actually worked

I checked out the commit prior to when i ran mvn release:prepare for the 4.1.1 release. This allowed me to locally do the release again against a detached HEAD that is the code base exactly as it was when I originally did the release. (I could and should have done this in a local branch, since the detached HEAD made the local commits that mvn release:prepare release:perform wants to do fail, but as it turns out those failed commits don't matter.)

I told the Maven release to use the target SCM tag dup-uportal-4.1.1 so it didn't conflict with the existing already-pushed uportal-4.1.1 tag.

And hey, it worked. It staged a uPortal 4.1.1 Maven repository with Sonatype that successfully Closed and successfully Released. So now the 4.1.1 artifacts are up on Maven Central.

The dup-uportal-4.1.1 String does show up in the uportal-parent pom.xml as the project --> scm --> tag ,

<project ...>
  ...
  <scm>
    ...
    <tag>dup-uportal-4.1.1</tag>
    ...

but that doesn't seem to matter.

Fail fast

I've updated the uPortal release process checklist so that the Sonatype repository Close step comes before pushing the release commits and tag up to GitHub. The next time Maven / Sonatype get confused about staging the Maven artifacts for release, this should be discovered sooner.

Fail loudly

When the upload of artifacts to Sonatype fails, that should, of course, fail the mvn:release step, rather than failing silently. It matters to the release engineer, of course, that the mvn release command didn't do all the things it needed to do, like actually upload the generated release artifacts.

I haven't done anything with this insight so far, but that would be the deeper fix.

Failing silently isn't cool.

Gradle?

I'm told I'll love Gradle. Maybe it doesn't fail in these ways. These experiences with Maven do remind me of The Maven.

Cover photo: Model of MAVEN spacecraft : CC-BY-NC.