incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Atri Sharma <a...@apache.org>
Subject Re: [DISCUSS] Incubating Proposal of Kyuubi
Date Wed, 02 Jun 2021 14:56:36 GMT
I am genuinely worried about one contributor being the dominant of all.

This puts the stability of the project in jeopardy and unless this is
remedied, I feel the project should focus on developing a more diverse
community first.

On Wed, 2 Jun 2021, 20:23 Sheng Wu, <wu.sheng.841108@gmail.com> wrote:

> Hi Kent
>
> Thanks for replying, but to be honest, your mail format is chaos.
> Take a look at bellowing, which is what I saw in the gmail.
>
> This may be an issue of your mail client or some else.
>
> > The current #.2 - 5 top
> > contributors will stay as committers, and they will join the PPMC during
> > incubation.
>
> PPMC usually is the default for all initial committers, considering your
> community is still small and not very active.
> You should invite all people you trusted to join, rather than pushing them
> away.
> This is not just a too-high bar issue. Excluding 4 out of 5 is strange.
> I can't guess what is the root cause, why the other 4 PPMC members don't
> touch codes but are trusted completely. Meanwhile, people with easily
> measurable code contributions are just as committers?
> We(ASF) always remind the community to recognize no-code contribution, but
> in this case, it seems the opposite? Don't recognize code contributors
> enough?
> Please reach your mentors, this seems not a typical ASF style. If you have
> other people already involved in the ASF, asking them too.
>
> And from your description of the release process, I think you should talk
> more with your mentors.
> Such as what is a qualified ASF release, what should be checked.
> This is not a block, the point I mentioned this is, you will need more
> people.
>
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Kent Yao <yao@apache.org> 于2021年6月2日周三 下午10:37写道:
>
> > Hi Sheng,
> >
> > Thanks for your detailed check and valuable comments. They are
> > accommodating. I have addressed some of them directly in the proposal.
> > Please also check the table below for some Q&As.
> > QA
> > You listed only 5 core maintainers, but clearly only one has dominated
> > contributions. The distortion and skewness of the contributions are
> mainly
> > caused by the way how we merge PRs changed(mentioned in another email on
> > the list).
> > I agree with Atri, orphaned is measurable. It is better to adjust this
> > statement a little. Address the comment to notify the risk in the Kyuubi
> > proposal.
> > Also, with this fact, the core maintainers should notice whether this
> small
> > group of people to make sure your features iteration, bug fix, and
> release
> > process working by following ASF's requirements.This would not be an easy
> > job. Currently, we learn the release process from some Apache projects,
> > using major.minor.patch for versioning. As we can see, since Kyuubi 1.0,
> it
> > takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes
> until
> > EOL. And it takes us about two months to finish a minor release with 3 ~
> 5
> > big improvements and various small features/bug fixes. We also will start
> > with some release candidates for preview and test before the final. This
> is
> > how we develop and release Kyuubi for now, but we do have a long way to
> go
> > and learn to match the ASF requirement.
> > And your committer list is very strange... 1. Why some of them are PPMC
> and
> > others are committers? Where this hierarchy comes from? The current PPMC
> > candidates help us a lot build the Kyuubi project and community, not
> > directly to the codebase but as mentors(before Apache). Most of them are
> > from Apache mature TLPs, which have close relationships with Kyuubi. They
> > have sufficient Apache Way experience to guide us to become a more
> > Apache-like project before submitting this proposal in many ways. We will
> > continue to promote more contributors to the PPMC/committer based on
> their
> > code or non-code contributions during incubation.
> > 2. Your core maintainers have 5, which only 4 PPMC? The current #.2 - 5
> top
> > contributors will stay as committers, and they will join the PPMC during
> > incubation.
> > 3. I think only the top 4 have over 400 LOC, why you declared you have 5
> > core developers? I didn't notice that 400 LOC is a bar for core
> developer,
> > will change 5 to 4 core developers accordingly
> > I think you don't have enough no-salaried developers, am I missing
> > anything? Yes.
> > Changed in the proposal, most of the contributors are salaried, and some
> of
> > them are willing to help in the spare time. Thanks
> > This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA in
> > Proposal. All work here is approved by NetEase and can be forward.
> > At last, you are asking about github.com/apache/incubator-kyuubi-shaded
> > repo,
> > what does this represent? I can only see you have only one source repo
> and
> > a doc repo. Removed this requirement in the proposal, and we will apply
> it
> > from IPMC if we do need this in the future.
> > ------------------------------
> >
> > Bests
> >
> > Kent Yao
> >
> > Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午5:45写道:
> >
> > > Hi
> > >
> > > I read the contributor list and the proposal
> > > > The risk of the Kyuubi project being abandoned is minimal. Many
> > > organizations are using Kyuubi to build critical big data pipelines and
> > > willing to help develop Kyuubi's community if it becomes an ASF
> project.
> > >
> > > I think the team should evaluate the status again. You listed only 5
> core
> > > maintainers, but clearly only one has dominated contributions. I agree
> > with
> > > Atri, orphaned is measurable.
> > > It is better to adjust this statement a little. Also, with this fact,
> the
> > > core maintainers should notice whether this small group of people to
> make
> > > sure your features iteration, bug fix, and release process working by
> > > following ASF's requirements.
> > > This would not be an easy job.
> > >
> > > And your committer list is very strange
> > > ```
> > > Kent Yao (yao@apache.org) (PPMC)
> > > Wenchen Fan (wenchen@apache.org) (PPMC)
> > > Wanqiang Ji (jiwq@apache.org) (PPMC)
> > > Xun Liu (liuxun@apache.org) (PPMC)
> > > Fei Wang (cn.feiwang@gmail.com)
> > > Ulysses You (ulyssesyou18@gmail.com)
> > > Cheng Pan (pan3793@gmail.com)
> > > Hongxiang Jiang (jhx1008@gmail.com)
> > > Zen Jin(xinjingziranchan@gmail.com)
> > > ```
> > > 1. Why some of them are PPMC and others are committers? Where this
> > > hierarchy comes from?
> > > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy
> makes
> > > sense, most likely not)
> > > 3. I think only the top 4 have over 400 LOC, why you declared you have
> 5
> > > core developers?
> > >
> > > Another re-evaluation request is about `Reliance on Salaried
> Developers:`
> > > > They are all passionate about the project, and we are confident that
> > the
> > > project will continue even if no salaried developers contribute to the
> > > project.
> > >
> > > I think you don't have enough no-salaried developers, am I missing
> > > anything?
> > >
> > > About IP clearance, `Initial Source and Intellectual Property
> Submission
> > > Plan`
> > > > As soon as Kyuubi is approved to join Apache Incubator, our initial
> > > committers will submit iCLA(s) and CCLA(s). The codebase is already
> > > licensed under the Apache License 2.0.
> > >
> > > This project belongs to NetEase Inc. clearly, why NO SGA?
> > >
> > > At last, you are asking about `
> > github.com/apache/incubator-kyuubi-shaded`
> <http://github.com/apache/incubator-kyuubi-shaded>
> > <http://github.com/apache/incubator-kyuubi-shaded>
> > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > repo, what does this represent? I can only see you have only one source
> > > repo and a doc repo.
> > >
> > > Sheng Wu 吴晟
> > > Twitter, wusheng1108
> > >
> > >
> > > Atri Sharma <atri@apache.org> 于2021年6月2日周三 下午4:49写道:
> > >
> > > > Thanks.
> > > >
> > > > I looked at the link you advised on and if I am reading it correctly,
> > top
> > > > committer has 630ish commits and the next committer has 73 commits?
> > > >
> > > > IMO this is an indication of the community not being active enough
> and
> > > > hence the risk of project getting orphaned is measurable.
> > > >
> > > > Atri
> > > >
> > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.jiang@gmail.com>
> > wrote:
> > > >
> > > > > Hi Atri Sharma,
> > > > >
> > > > > It's easier to check out the recent contribution of kyuubi from
> this
> > > > > link[1].
> > > > > We already identified the one person contribution issue of kyuubi
> > when
> > > > > Kent came to us and we kept reminding him to put more effort into
> > > > > building the community.
> > > > > Last week I had a long F2F conversation with Kent, I think he
> already
> > > > > knows about it and there have been some improvements in recent
> months
> > > > > but there is still a long way to go if Kyuubi is accepted as a
> > podling
> > > > > in the Apache Incubator.
> > > > >
> > > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors
> > > > >
> > > > > Willem Jiang
> > > > >
> > > > > Twitter: willemjiang
> > > > > Weibo: 姜宁willem
> > > > >
> > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <atri@apache.org>
> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I took a look at commit log and seems like all the commits for
> > last 3
> > > > > > months have been done by a single person.
> > > > > >
> > > > > > How active are the remaining committers?
> > > > > >
> > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <yao@apache.org>
wrote:
> > > > > >
> > > > > > > Hi Alexander Alten,
> > > > > > >
> > > > > > > Thanks for your interest and +1 for Kyuubi.
> > > > > > >
> > > > > > > Not being very familiar with Dremel, I just took a quick
look
> at
> > > the
> > > > > > > profile of a very similar and exciting project, Apache
Drill.
> If
> > I
> > > > > > > understand correctly, Kyuubi is built on top of the compute
> layer
> > > > > > > (Apache Spark), while Apache Drill is built on top of the
> storage
> > > > > > > layer.
> > > > > > >
> > > > > > > Apache Spark has a large user base and usage scenarios
all over
> > the
> > > > > > > world. When users have some basic Spark programming and
tuning
> > > > skills,
> > > > > > > things are easy for them. But many times, most users do
not
> have
> > > all
> > > > > > > of these abilities when they use Spark directly or indirectly.
> We
> > > > want
> > > > > > > to build a more easy-to-use platform on top of it to help
these
> > > > users.
> > > > > > > On the one hand, we will work on the interface layer to
make it
> > > easy
> > > > > > > to access (of course, this effort will make our project
look
> very
> > > > > > > similar to other projects, such as HiveServer2). On the
other
> > hand,
> > > > we
> > > > > > > will work under Kyuubi's multi-tenant architecture to extend
> our
> > > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to make it
more
> > > stable
> > > > > > > and efficient in our scenario.
> > > > > > >
> > > > > > > What's more, we are going to add more on-prem Kyuubi engine
> > types,
> > > > e.g.
> > > > > > > Flink.
> > > > > > >
> > > > > > > Alexander Alten <alo@scalytics.io> 于2021年5月31日周一
下午2:35写道:
> > > > > > >
> > > > > > > >
> > > > > > > > Hey,
> > > > > > > >
> > > > > > > > Looks like an interesting project, competition to
Starburst.
> > How
> > > > > does it
> > > > > > > > compare to Apache Dremel?
> > > > > > > >
> > > > > > > >  +1 (non-binding) for incubation.
> > > > > > > >
> > > > > > > > Thanks and stay safe,
> > > > > > > > --alex
> > > > > > > >
> > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <yao@apache.org>
wrote:
> > > > > > > >
> > > > > > > > > Dear all,
> > > > > > > > >
> > > > > > > > > We want to propose Kyuubi as a new Apache Incubator
> project.
> > > > > > > > >
> > > > > > > > > Kyuubi is a distributed multi-tenant Thrift JDBC/ODBC
> server
> > > for
> > > > > > > > > large-scale data management, processing, and
analytics,
> built
> > > on
> > > > > top
> > > > > > > > > of Apache Spark and designed to support more
engines (i.e.,
> > > > Apache
> > > > > > > > > Flink). We are aiming to make Kyuubi an "out-of-the-box"
> tool
> > > for
> > > > > data
> > > > > > > > > warehouses and data lakes. Kyuubi concentrates
on improving
> > the
> > > > > > > > > experience for non-Spark users, making it easy
for them to
> > use
> > > > > Spark
> > > > > > > > > as a service and focus on business data of their
own, not
> > where
> > > > it
> > > > > > > > > stores, how it computes. NetEase has open-sourced
Kyuubi
> > since
> > > > > 2018,
> > > > > > > > > https://github.com/NetEase/kyuubi.
> > > > > > > > >
> > > > > > > > > Many thanks to Willem Jiang (Champion), Jeff
Zhang, Duo
> > Zhang,
> > > > > Akira
> > > > > > > > > Ajisaka for being the mentors of the Kyuubi project
and
> > helping
> > > > us
> > > > > set
> > > > > > > > > up this proposal. The proposal can be found at
> > > > > > > > >
> > > > >
> https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal
> > .
> > > > > > > > >
> > > > > > > > > Looking forward to your feedback and thank you
very much.
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > > Kent Yao
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail:
> > > general-unsubscribe@incubator.apache.org
> > > > > > > > > For additional commands, e-mail:
> > > > general-help@incubator.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > >  Kent Yao
> > > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase
> Corp.
> > > > > > > a spark enthusiast
> > > > > > > kyuubiis a unified multi-tenant JDBC interface for large-scale
> > data
> > > > > > > processing and analytics, built on top of Apache Spark.
> > > > > > >
> > > > > > > spark-authorizerA Spark SQL extension which provides SQL
> Standard
> > > > > > > Authorization for Apache Spark.
> > > > > > > spark-postgres A library for reading data from and transferring
> > > data
> > > > > > > to Postgres / Greenplum with Spark SQL and DataFrames,
10~100x
> > > > faster.
> > > > > > > itatchiA library that brings useful functions from various
> modern
> > > > > > > database management systems to Apache Spark.
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail:
> general-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail:
> > general-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > > > For additional commands, e-mail: general-help@incubator.apache.org
> > > > >
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> >  *Kent Yao *
> > @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> > a spark enthusiast
> > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant
> JDBC
> > interface for large-scale data processing and analytics, built on top of
> > Apache
> > Spark <http://spark.apache.org/>.
> > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL
> > extension which provides SQL Standard Authorization for Apache Spark
> > <http://spark.apache.org/>.
> > spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
> for
> > reading data from and transferring data to Postgres / Greenplum with
> Spark
> > SQL and DataFrames, 10~100x faster.
> > itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
> > brings useful functions from various modern database management
> > systems to Apache
> > Spark <http://spark.apache.org/>.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message