incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wangda Tan <w...@cloudera.com.INVALID>
Subject Re: [DISCUSS] YuniKorn Proposal
Date Sat, 14 Dec 2019 01:04:46 GMT
Hi Justin,

Thanks for digging into this and giving feedback, let me try to answer
these questions:


On Thu, Dec 12, 2019 at 6:35 PM Justin Mclean <justin@classsoftware.com>
wrote:

> Hi,
>
> Sounds like a good proposal.
>
> I can see it was opened source back in March but you haven’t seemed to
> have made any releases or added any new committers since then. An I correct
> in that assumption? Is the any reason for that? I note in the proposal that
> you have people from other organisations that you say are already
> collaborating with you on Github, but I cannot see much if any activity
> from them. The core repo has 5 main committers (and two others). The other
> repos are similar. As you note not all contribution are code, so I may be
> missing something. Can you explain how the initial committers were chosen
> and provide a bit more detail on their relationship to the project?
>

We added a committer Tao Yang after the project started. (See contribution
from Tao:
https://github.com/cloudera/yunikorn-core/commits?author=TaoYang526)

We actually did some releases, but on Docker Hub:
https://hub.docker.com/r/yunikorn/yunikorn-scheduler-k8s/tags. We didn't
put it to Github's release page.

The reason the number of contributors not significantly grow is the project
is hosted under cloudera.com, contributors from other companies are not
comfortable with contributing to other company's repo and even hesitate to
try that.

To answer what is the philosophy behind non-code committers we added.
- The initial committers are from several relevant Apache projects who are
interested in YuniKorn and committed to giving guidance, suggestions and
participant in the future if possible.

- Following folks are from YARN community (see following folks), we
collaborated on YARN scheduler for years and they have lots of domain
knowledge about scheduling. They're willing to give feedbacks and
suggestions to YuniKorn. And we already got tons of help and important
feedbacks from them.

Carlo Curino (curino@apache.org) (Microsoft)
Subramaniam Krishnan (subru@apache.org) (Microsoft)
Arun Suresh (asuresh@apache.org) (Microsoft)
Konstantinos Karanasos (kkaranasos@apache.org) (Microsoft)
Jonathan Hung (jhung@apache.org) (LinkedIn)
Junping Du (junping_du@apache.org) (Tencent)
Jason Lowe (jlowe@apache.org) (Nvidia)

And DB Tsai is from Apache Spark community, getting Spark to integrate with
YuniKorn is very important task for us, and we need help from Spark
community. We received many valuable suggestions from DB about what the
project should go to better support batch workload like Spark.

Vinod Kumar Vavilapalli is helping the project from the beginning, from the
project execution, requirements and how to create a community,  we received
lots of suggestions and he volunteered to be the champion of the project.


>
> You mention the use of Slack, weekly catch up and open community meetings.
> How is the information discussed there passed on to other community members
> who are not on slack or not able to attend those meetings? Are any
> decisions made at those meetings or on Slack?
>

We made most of the decisions on Github issues, PRs and code review
publically. On the Slack or catch up meetings we mostly discuss issues we
talk about bugs or product issues.

For planning, we also keep
https://github.com/cloudera/yunikorn-core/blob/master/docs/roadmap.md up to
date, so other community members can get that easier.

To better improve, we will make use of mailing list, etc. in the future
like other Apache project once be part of the Apache incubator. Which can
get other contributors participated and we want to build an open community
instead of making closed-door decisions.


>
> The length of time to incubate seems a little on the short side,
> especially for a project that doesn’t seem to have a large community around
> it and most of the core set of developers working for a single company. I’d
> be happy to be proven wrong about that, but if the project ended up in
> incubation for longer would this be an issue?. Projects typically spend 1
> to 1 1/2 years in the incubator, some take less time some take more, some a
> lot more.
>

That makes sense, and I didn't see any issue if more incubation time
needed. We want to graduate once it is ready to graduate.


>
> There’s at least one section missing from your proposal [1], the
> information on the project name is important, can you answer that? Infra in
> particular is concerned about the amount of effort a name change takes.
>

We will add it to the proposal before a vote. We have done similar
practices like a PODLINGNAMESEARCH. We have done search of "YuniKorn" on
Github, nothing related to scheduler or distributed system. We also did a
search of YuniKorn as a trademark, no hit. And YuniKorn search on internet
also no relevant hit. Since the name is unique, easy to remember,
pronounce, and relevant to the project, we believe it is a good name.

I think we don't need to file PODLINGNAMESEARCH before the project approved
by incubator, please let me know if we need to file it before that.


>
> Your mentors need to be IPMC members [2][3] but as Dave notes, being ASF
> members they can ask to join the IPMC.
>

Make sense, and will ask ASF mentors to join the IPMC.

Hope this answered your questions, please let me know if I missed anything.

Best,
Wangda


> Thanks,
> Justin
>
> 1.
> https://wiki.apache.org/confluence/display/INCUBATOR/New+Podling+Proposal
> 2. https://incubator.apache.org/guides/participation.html#as_a_mentor
> 3.
> https://incubator.apache.org/guides/mentor.html#mentors_must_be_on_the_ipmc
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message