Sonatype release upload improvements

[chrisrueger]

Bnd 7.2.0 was published to Maven Central via the new Sonatype Publisher Portal and support for it was added to bnd itself (yes bnd uses bnd to deploy itself).

• Update to latest Sonatype/Maven portal deployment #6931

Problems

Currently sonatype expects that we upload each artifact all together and at the end there is an additional finalization HTTP call.

This "at the end" is a problem, because it requires to do something "at the end" or with the "last artifact".
This is different from the existing mental model of bnd's MavenBndRepository and Releaser which did not have this "last artifact" / "at the end" concept. In 7.2.0 we forced a square peg into a round hole and made it work, but it turned out that this caused other issues with releasing bnd's gradle and maven plugins, because they are not using bnd's Releaser.

This new function which considers "at the end" needs to be triggered from multiple places (CLI, Eclipse Release, Gradle publish task, maven publish task). But for example in gradle there is no concept to find out if we are at the "last plugin" (at the end). Gradle does not have a concept of "last" , because everything could in theorie also be parallelized where there is no order of things. So for gradle I built a hack to to determine "am I in the last artifact".

Long story short: What we did in 7.2.0 is not optimial and we should look for ways to improve this.

Idea

Separate building and Sonatype deployment.

1. build: bnd workspace, gradle-plugins and maven-plugins build their artifacts to a shared dedicated "sonatype-release" folder
2. Upload process: after all above are done: a new Sonatype release process uploads this folder to sonatype

flowchart TD
    A[bnd workspace] -->|1. build| E[folder sonatype-release]
    B[gradle-plugins] --> |1. build| E
    C[maven-plugins] -->|1. build| E
    D[bndtools-eclipse] -->|1. build| E
    E --> |2. Upload process| F[Sonatype Publisher Portal]
  

Loading

That way there is only a "release" of each artifact to a local folder and no keeping track of "last artifact". There is only an "after" (deploy the folder to sonatype), and this should be quite possible with all tools like gradle, maven, bnd.

Currently we are doing this process already, but more in a "workaround" kind of way in ci-publish.sh, by publishing in that order:

• publish gradle-plugins -> dist/bundles and jfrog
• publish maven-plugins -> dist/bundles
• publish maven-plugins -> again, but to jfrog
• copy all from dist/bundles-> cnf/cache/sonatype-release/
• publish bnd-workspace via ./gradlew --no-daemon -Dmaven.repo.local=dist/m2 :publish "$@" which uploads the folder cnf/cache/sonatype-release/ to Sonatype

Problem is the last step (publish bnd-workspace) which implicitly uploads gradle-plugins and maven-plugins to sonatype too, although they actually were not part of the bnd-workspace bundles.
So the goal is to make this process less hacky and more explicit, like this:

• publish gradle-plugins -> dist/bundles
• publish maven-plugins -> dist/bundles
• publish bnd-workspace -> dist/bundles
• release the above to Sonatype (This part is our TODO and should to be separate from the building)
• release to JFrog ? (also separate and obvious)

Alternatives

Let's also look into external tools like:

• https://gradleup.com/nmcp/
• https://github.com/maveniverse/njord

[chrisrueger]

There is still the idea around of extracting the bnd maven and gradle plugins out of bnd to a new home / maintainer (e.g. to just separate repo, or Eclipse / Tycho (@laeubi ), OSGi (@stbischof )). See https://github.com/bndtools/bnd/wiki/WIP-ideas-%E2%80%90-bnd-8.0.0-breaking-changes#separate--extract-maven-plugins-and-gradle-plugins

Just mentioning this, as it would impact and maybe simplify the idea above.