incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Wu <wu.sheng.841...@gmail.com>
Subject Re: [DISCUSS] Incubating Proposal of Kyuubi
Date Wed, 02 Jun 2021 14:52:34 GMT
Hi Kent

Thanks for replying, but to be honest, your mail format is chaos.
Take a look at bellowing, which is what I saw in the gmail.

This may be an issue of your mail client or some else.

> The current #.2 - 5 top
> contributors will stay as committers, and they will join the PPMC during
> incubation.

PPMC usually is the default for all initial committers, considering your
community is still small and not very active.
You should invite all people you trusted to join, rather than pushing them
away.
This is not just a too-high bar issue. Excluding 4 out of 5 is strange.
I can't guess what is the root cause, why the other 4 PPMC members don't
touch codes but are trusted completely. Meanwhile, people with easily
measurable code contributions are just as committers?
We(ASF) always remind the community to recognize no-code contribution, but
in this case, it seems the opposite? Don't recognize code contributors
enough?
Please reach your mentors, this seems not a typical ASF style. If you have
other people already involved in the ASF, asking them too.

And from your description of the release process, I think you should talk
more with your mentors.
Such as what is a qualified ASF release, what should be checked.
This is not a block, the point I mentioned this is, you will need more
people.


Sheng Wu 吴晟
Twitter, wusheng1108


Kent Yao <yao@apache.org> 于2021年6月2日周三 下午10:37写道:

> Hi Sheng,
>
> Thanks for your detailed check and valuable comments. They are
> accommodating. I have addressed some of them directly in the proposal.
> Please also check the table below for some Q&As.
> QA
> You listed only 5 core maintainers, but clearly only one has dominated
> contributions. The distortion and skewness of the contributions are mainly
> caused by the way how we merge PRs changed(mentioned in another email on
> the list).
> I agree with Atri, orphaned is measurable. It is better to adjust this
> statement a little. Address the comment to notify the risk in the Kyuubi
> proposal.
> Also, with this fact, the core maintainers should notice whether this small
> group of people to make sure your features iteration, bug fix, and release
> process working by following ASF's requirements.This would not be an easy
> job. Currently, we learn the release process from some Apache projects,
> using major.minor.patch for versioning. As we can see, since Kyuubi 1.0, it
> takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes until
> EOL. And it takes us about two months to finish a minor release with 3 ~ 5
> big improvements and various small features/bug fixes. We also will start
> with some release candidates for preview and test before the final. This is
> how we develop and release Kyuubi for now, but we do have a long way to go
> and learn to match the ASF requirement.
> And your committer list is very strange... 1. Why some of them are PPMC and
> others are committers? Where this hierarchy comes from? The current PPMC
> candidates help us a lot build the Kyuubi project and community, not
> directly to the codebase but as mentors(before Apache). Most of them are
> from Apache mature TLPs, which have close relationships with Kyuubi. They
> have sufficient Apache Way experience to guide us to become a more
> Apache-like project before submitting this proposal in many ways. We will
> continue to promote more contributors to the PPMC/committer based on their
> code or non-code contributions during incubation.
> 2. Your core maintainers have 5, which only 4 PPMC? The current #.2 - 5 top
> contributors will stay as committers, and they will join the PPMC during
> incubation.
> 3. I think only the top 4 have over 400 LOC, why you declared you have 5
> core developers? I didn't notice that 400 LOC is a bar for core developer,
> will change 5 to 4 core developers accordingly
> I think you don't have enough no-salaried developers, am I missing
> anything? Yes.
> Changed in the proposal, most of the contributors are salaried, and some of
> them are willing to help in the spare time. Thanks
> This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA in
> Proposal. All work here is approved by NetEase and can be forward.
> At last, you are asking about github.com/apache/incubator-kyuubi-shaded
> repo,
> what does this represent? I can only see you have only one source repo and
> a doc repo. Removed this requirement in the proposal, and we will apply it
> from IPMC if we do need this in the future.
> ------------------------------
>
> Bests
>
> Kent Yao
>
> Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午5:45写道:
>
> > Hi
> >
> > I read the contributor list and the proposal
> > > The risk of the Kyuubi project being abandoned is minimal. Many
> > organizations are using Kyuubi to build critical big data pipelines and
> > willing to help develop Kyuubi's community if it becomes an ASF project.
> >
> > I think the team should evaluate the status again. You listed only 5 core
> > maintainers, but clearly only one has dominated contributions. I agree
> with
> > Atri, orphaned is measurable.
> > It is better to adjust this statement a little. Also, with this fact, the
> > core maintainers should notice whether this small group of people to make
> > sure your features iteration, bug fix, and release process working by
> > following ASF's requirements.
> > This would not be an easy job.
> >
> > And your committer list is very strange
> > ```
> > Kent Yao (yao@apache.org) (PPMC)
> > Wenchen Fan (wenchen@apache.org) (PPMC)
> > Wanqiang Ji (jiwq@apache.org) (PPMC)
> > Xun Liu (liuxun@apache.org) (PPMC)
> > Fei Wang (cn.feiwang@gmail.com)
> > Ulysses You (ulyssesyou18@gmail.com)
> > Cheng Pan (pan3793@gmail.com)
> > Hongxiang Jiang (jhx1008@gmail.com)
> > Zen Jin(xinjingziranchan@gmail.com)
> > ```
> > 1. Why some of them are PPMC and others are committers? Where this
> > hierarchy comes from?
> > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy makes
> > sense, most likely not)
> > 3. I think only the top 4 have over 400 LOC, why you declared you have 5
> > core developers?
> >
> > Another re-evaluation request is about `Reliance on Salaried Developers:`
> > > They are all passionate about the project, and we are confident that
> the
> > project will continue even if no salaried developers contribute to the
> > project.
> >
> > I think you don't have enough no-salaried developers, am I missing
> > anything?
> >
> > About IP clearance, `Initial Source and Intellectual Property Submission
> > Plan`
> > > As soon as Kyuubi is approved to join Apache Incubator, our initial
> > committers will submit iCLA(s) and CCLA(s). The codebase is already
> > licensed under the Apache License 2.0.
> >
> > This project belongs to NetEase Inc. clearly, why NO SGA?
> >
> > At last, you are asking about `
> github.com/apache/incubator-kyuubi-shaded`
> <http://github.com/apache/incubator-kyuubi-shaded>
> > <http://github.com/apache/incubator-kyuubi-shaded>
> > repo, what does this represent? I can only see you have only one source
> > repo and a doc repo.
> >
> > Sheng Wu 吴晟
> > Twitter, wusheng1108
> >
> >
> > Atri Sharma <atri@apache.org> 于2021年6月2日周三 下午4:49写道:
> >
> > > Thanks.
> > >
> > > I looked at the link you advised on and if I am reading it correctly,
> top
> > > committer has 630ish commits and the next committer has 73 commits?
> > >
> > > IMO this is an indication of the community not being active enough and
> > > hence the risk of project getting orphaned is measurable.
> > >
> > > Atri
> > >
> > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.jiang@gmail.com>
> wrote:
> > >
> > > > Hi Atri Sharma,
> > > >
> > > > It's easier to check out the recent contribution of kyuubi from this
> > > > link[1].
> > > > We already identified the one person contribution issue of kyuubi
> when
> > > > Kent came to us and we kept reminding him to put more effort into
> > > > building the community.
> > > > Last week I had a long F2F conversation with Kent, I think he already
> > > > knows about it and there have been some improvements in recent months
> > > > but there is still a long way to go if Kyuubi is accepted as a
> podling
> > > > in the Apache Incubator.
> > > >
> > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors
> > > >
> > > > Willem Jiang
> > > >
> > > > Twitter: willemjiang
> > > > Weibo: 姜宁willem
> > > >
> > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <atri@apache.org> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I took a look at commit log and seems like all the commits for
> last 3
> > > > > months have been done by a single person.
> > > > >
> > > > > How active are the remaining committers?
> > > > >
> > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <yao@apache.org> wrote:
> > > > >
> > > > > > Hi Alexander Alten,
> > > > > >
> > > > > > Thanks for your interest and +1 for Kyuubi.
> > > > > >
> > > > > > Not being very familiar with Dremel, I just took a quick look
at
> > the
> > > > > > profile of a very similar and exciting project, Apache Drill.
If
> I
> > > > > > understand correctly, Kyuubi is built on top of the compute
layer
> > > > > > (Apache Spark), while Apache Drill is built on top of the storage
> > > > > > layer.
> > > > > >
> > > > > > Apache Spark has a large user base and usage scenarios all over
> the
> > > > > > world. When users have some basic Spark programming and tuning
> > > skills,
> > > > > > things are easy for them. But many times, most users do not
have
> > all
> > > > > > of these abilities when they use Spark directly or indirectly.
We
> > > want
> > > > > > to build a more easy-to-use platform on top of it to help these
> > > users.
> > > > > > On the one hand, we will work on the interface layer to make
it
> > easy
> > > > > > to access (of course, this effort will make our project look
very
> > > > > > similar to other projects, such as HiveServer2). On the other
> hand,
> > > we
> > > > > > will work under Kyuubi's multi-tenant architecture to extend
our
> > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to make it more
> > stable
> > > > > > and efficient in our scenario.
> > > > > >
> > > > > > What's more, we are going to add more on-prem Kyuubi engine
> types,
> > > e.g.
> > > > > > Flink.
> > > > > >
> > > > > > Alexander Alten <alo@scalytics.io> 于2021年5月31日周一
下午2:35写道:
> > > > > >
> > > > > > >
> > > > > > > Hey,
> > > > > > >
> > > > > > > Looks like an interesting project, competition to Starburst.
> How
> > > > does it
> > > > > > > compare to Apache Dremel?
> > > > > > >
> > > > > > >  +1 (non-binding) for incubation.
> > > > > > >
> > > > > > > Thanks and stay safe,
> > > > > > > --alex
> > > > > > >
> > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <yao@apache.org>
wrote:
> > > > > > >
> > > > > > > > Dear all,
> > > > > > > >
> > > > > > > > We want to propose Kyuubi as a new Apache Incubator
project.
> > > > > > > >
> > > > > > > > Kyuubi is a distributed multi-tenant Thrift JDBC/ODBC
server
> > for
> > > > > > > > large-scale data management, processing, and analytics,
built
> > on
> > > > top
> > > > > > > > of Apache Spark and designed to support more engines
(i.e.,
> > > Apache
> > > > > > > > Flink). We are aiming to make Kyuubi an "out-of-the-box"
tool
> > for
> > > > data
> > > > > > > > warehouses and data lakes. Kyuubi concentrates on
improving
> the
> > > > > > > > experience for non-Spark users, making it easy for
them to
> use
> > > > Spark
> > > > > > > > as a service and focus on business data of their own,
not
> where
> > > it
> > > > > > > > stores, how it computes. NetEase has open-sourced
Kyuubi
> since
> > > > 2018,
> > > > > > > > https://github.com/NetEase/kyuubi.
> > > > > > > >
> > > > > > > > Many thanks to Willem Jiang (Champion), Jeff Zhang,
Duo
> Zhang,
> > > > Akira
> > > > > > > > Ajisaka for being the mentors of the Kyuubi project
and
> helping
> > > us
> > > > set
> > > > > > > > up this proposal. The proposal can be found at
> > > > > > > >
> > > > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal
> .
> > > > > > > >
> > > > > > > > Looking forward to your feedback and thank you very
much.
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Kent Yao
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail:
> > general-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail:
> > > general-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > >  Kent Yao
> > > > > > @ Data Science Center, Hangzhou Research Institute, NetEase
Corp.
> > > > > > a spark enthusiast
> > > > > > kyuubiis a unified multi-tenant JDBC interface for large-scale
> data
> > > > > > processing and analytics, built on top of Apache Spark.
> > > > > >
> > > > > > spark-authorizerA Spark SQL extension which provides SQL Standard
> > > > > > Authorization for Apache Spark.
> > > > > > spark-postgres A library for reading data from and transferring
> > data
> > > > > > to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x
> > > faster.
> > > > > > itatchiA library that brings useful functions from various modern
> > > > > > database management systems to Apache Spark.
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail:
> general-help@incubator.apache.org
> > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > > For additional commands, e-mail: general-help@incubator.apache.org
> > > >
> > > >
> > >
> >
>
>
> --
>
>  *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> a spark enthusiast
> kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top of
> Apache
> Spark <http://spark.apache.org/>.
> spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark SQL
> extension which provides SQL Standard Authorization for Apache Spark
> <http://spark.apache.org/>.
> spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.
> itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
> brings useful functions from various modern database management
> systems to Apache
> Spark <http://spark.apache.org/>.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message