incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Wendell <>
Subject Re: [VOTE] Release Apache Spark 0.8.0-incubating (rc4)
Date Sun, 15 Dec 2013 05:24:25 GMT
Hey Marvin,

Is this policy actually written up anywhere (along with best practices
on how to deal with this issue if you indeed have third party
dependencies)? I'm just asking because I don't see an obvious "fix"
for this based on the way Spark is built.

Second - this issue was not brought to our attention before - and in
particular was not raised during our 0.8.0 major release through the
incubator. Since this (0.8.1) release is a maintenance release, doing
a large change to the build system is not possible. It seems to me a
bit much to ask us to completely re-tool this entire project in order
for a simple maintenance release to pass. Especially since other top
level projects are clearly still employing this practice (not that
they are in-the-right, but just that this is a policy which it seems
is still being shaped).

We are planning to do our next major release (Spark 0.9.0) while still
under incubation in the next few weeks. Could I propose that we create
a parallel discussion about how we might re-tool or build process with
the aim towards satisfying whatever policy exists in that release?
We'll probably need guidance on that from people at Apache since,
again, there is no documented guidelines about what is allowed and
what isn't.

- Patrick

On Sat, Dec 14, 2013 at 8:20 PM, Marvin Humphrey <> wrote:
> On Sat, Dec 14, 2013 at 3:43 PM, Patrick Wendell <> wrote:
>>> However they both contain binaries, which is not good.
>>> Third party jars should *not* be included in SCM nor in source releases.
>> These are not binary artifacts containing our project's code. They are
>> our build tool and immediate dependencies that are not published in
>> maven. I've looked around to find TL projects that also use sbt and
>> they also include the sbt jar in the source release. For instance
>> Apache Kafka does the same thing:
> I appreciate your doing the research, and I understand why you might think
> following Kafka's example is a reasonable approach.  However, that binary is a
> problem for Kafka.  If Kafka's releases were like that when they graduated,
> it's a failure of the Incubator as well.
> Please read these messages from ASF Board member Roy Fielding:
> This has to be fixed.  If some TLP PMCs have not been made aware that binary
> dependencies may not be bundled in source releases, the Incubator must not
> compound the problem by failing to educate current podlings.
> Marvin Humphrey
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message