incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <>
Subject Re: Airflow voting on release artifacts
Date Tue, 25 Apr 2017 06:38:54 GMT
Hi John,

Typically one handles a “release candidate” equal to a release, except for the fact that
it hasn’t been fully tested and reviewed yet. In case everyone (all entities) is satisfied
with the RC it is then just rebranded to Release. Unfortunately, in the python ecosystem the
version information is contained within the (source) package and as metadata by means of the
filename. These two need to be in sync, otherwise package installers will get confused.

So imagine, if we started voting on “apache-airflow-1.8.1+incubating”, but it does *not*
pass the vote due to a blocker bug or a license issue. We would need to restart the vote with
an updated package but with the same name. Semantically, this is already confusing as you
would need to examine the signatures to see if the version has changed. It is also risky for
the users/testers and therefore for the developers: the new version will not trigger an upgrade
with package installers and a new installation would need to be forced (contrary to how maven
handles things for example). This means, we could receive bug reports on “1.8.1” which
actually are “1.8.1-RC1” or “1.8.1-RC2” etc. 

In Apache we vote on artefacts. A rebrand delivers us two different artefacts: RC + Release.
For us, as a podling, that would mean 4 votes: 2 at the PMC and 2 at the IPMC. This is very
time consuming for a very minor change.

Anecdotical evidence from two other projects shows that Apache Beam voted on 0.6.0-RC2 as
a release and then rebranded and updated packages after the vote (please correct me if I am
wrong, just did a mailinglist scan and a comparison on the trees). Apache Libcloud (a pure
python project) votes twice, but they seem to handle RCs differently, as there are large time
gaps between the votes.

So we are seeking advice how to make efficient use of the Apache process, given the limitations
of the ecosystem. I prefer what Apache Beam seems to be doing. I can also imagine to combine
two artefacts in one vote: pointing everyone to the RC including it as a tar ball, but also
including the tar ball and signatures for the *potential* rebranded version to release. Somehow
we would need to hide the *potential* release as much from plain sight as possible, as we
would have no way to distinguish between the actual release and the potential.


> On 25 Apr 2017, at 02:11, John D. Ament <> wrote:
> Do you need to cut a vote with something named rc?  Why can't you just use
> the version #?
> On Mon, Apr 24, 2017 at 4:18 PM Chris Riccomini <>
> wrote:
>> Hey all,
>> We've had a question arise in the Airflow project. We're currently cutting
>> release candidates (RCs), and voting on those. These RCs contain an
>> artifact with the suffix -rc0, -rc1, etc. The problem with this is that the
>> final RC that passes the vote still has an -rcX in its version number. To
>> get around this, the release manager has just been rebuilding the .asc,
>> .sha, etc, and publishing this new artifact as the final release.
>> Hitesh recently raised the issue that the artifact that passes the vote
>> MUST be the one that we actually release. The problem with this is that it
>> would force us to publish multiple RCs with the exact same version number.
>> This is a terrible experience from a developer and user point of view.
>> Airflow is a Python project, and many users pip install various versions of
>> Airflow to test things out. Having multiple RCs with the same version
>> number is going to make it really difficult to keep things straight, both
>> during the VOTE, and afterwards.
>> Based on a cursory look, it seems that other projects handle this problem
>> in one of three ways:
>> * Double vote (the final RC gets voted AGAIN, this time, with proper
>> naming)
>> * Rename/rebuild after final vote (This is what Airflow is doing, and Beam
>> does this, I believe)
>> * All RCs have the exact same version number
>> Most Java projects are just publishing multiple RCs with the exact same
>> version number. This seems bad from a general software engineering
>> perspective, since it's going to cause build cache issues (e.g. .mvn
>> already contains version 1.2.3, even though a new RC with the same version
>> should be installed). Beam does seem to be using -SNAPSHOT releases in Java
>> for RCs, which is CLOSER to what we've been doing in Airflow (-rc0, -rc1,
>> etc), and they seem to be renaming after the release, just like we were.
>> What's the guidance here? We seem to have two requirements that are at odds
>> with eachtother:
>> 1) The RC artifact that we vote on is the one that we release
>> 2) We have real version numbers for RCs which are independent of one
>> another so build systems, dependency managements systems, and package
>> management systems can properly handle them.
>> Any help or thoughts would be appreciated.
>> Cheers,
>> Chris

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message