incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sheng Wu <wu.sheng.841...@gmail.com>
Subject Re: [DISCUSS] Incubating Proposal of Kyuubi
Date Thu, 03 Jun 2021 00:09:53 GMT
Kent

Thanks.
The proposal is generally better this time.

If your initial committers and your mentors agree, you are ready to embrace
the ASF culture, I am good with that.
Just from my experience, your community looks a little too young, like
Atri's concern, which is reasonable.
But I would like to trust you and your mentors' judgment and determination.

Sheng Wu 吴晟
Twitter, wusheng1108


Kent Yao <yao@apache.org> 于2021年6月3日周四 上午12:54写道:

> Hi Sheng,
>
> Truly sorry for the format issue.
>
> Please trust me I did not mean to hurt anybody and push any contributor
> away. We drafted this proposal together based on the New Podling Proposal.
> We never tried to use PPMC and committer to measure whose contribution is
> higher. This mistake has been fixed in the proposal.
>
> About releases, we will follow
> https://www.apache.org/legal/release-policy.html and the guide of our
> mentors. We will also take the initiative to learn other documents.
>
> > This is not a block, the point I mentioned this is, you will need more
> people.
>
> We will keep working on this.
>
> Bests,
>
> Kent Yao
>
>
> Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午10:53写道:
>
> > Hi Kent
> >
> > Thanks for replying, but to be honest, your mail format is chaos.
> > Take a look at bellowing, which is what I saw in the gmail.
> >
> > This may be an issue of your mail client or some else.
> >
> > > The current #.2 - 5 top
> > > contributors will stay as committers, and they will join the PPMC
> during
> > > incubation.
> >
> > PPMC usually is the default for all initial committers, considering your
> > community is still small and not very active.
> > You should invite all people you trusted to join, rather than pushing
> them
> > away.
> > This is not just a too-high bar issue. Excluding 4 out of 5 is strange.
> > I can't guess what is the root cause, why the other 4 PPMC members don't
> > touch codes but are trusted completely. Meanwhile, people with easily
> > measurable code contributions are just as committers?
> > We(ASF) always remind the community to recognize no-code contribution,
> but
> > in this case, it seems the opposite? Don't recognize code contributors
> > enough?
> > Please reach your mentors, this seems not a typical ASF style. If you
> have
> > other people already involved in the ASF, asking them too.
> >
> > And from your description of the release process, I think you should talk
> > more with your mentors.
> > Such as what is a qualified ASF release, what should be checked.
> > This is not a block, the point I mentioned this is, you will need more
> > people.
> >
> >
> > Sheng Wu 吴晟
> > Twitter, wusheng1108
> >
> >
> > Kent Yao <yao@apache.org> 于2021年6月2日周三 下午10:37写道:
> >
> > > Hi Sheng,
> > >
> > > Thanks for your detailed check and valuable comments. They are
> > > accommodating. I have addressed some of them directly in the proposal.
> > > Please also check the table below for some Q&As.
> > > QA
> > > You listed only 5 core maintainers, but clearly only one has dominated
> > > contributions. The distortion and skewness of the contributions are
> > mainly
> > > caused by the way how we merge PRs changed(mentioned in another email
> on
> > > the list).
> > > I agree with Atri, orphaned is measurable. It is better to adjust this
> > > statement a little. Address the comment to notify the risk in the
> Kyuubi
> > > proposal.
> > > Also, with this fact, the core maintainers should notice whether this
> > small
> > > group of people to make sure your features iteration, bug fix, and
> > release
> > > process working by following ASF's requirements.This would not be an
> easy
> > > job. Currently, we learn the release process from some Apache projects,
> > > using major.minor.patch for versioning. As we can see, since Kyuubi
> 1.0,
> > it
> > > takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes
> > until
> > > EOL. And it takes us about two months to finish a minor release with 3
> ~
> > 5
> > > big improvements and various small features/bug fixes. We also will
> start
> > > with some release candidates for preview and test before the final.
> This
> > is
> > > how we develop and release Kyuubi for now, but we do have a long way to
> > go
> > > and learn to match the ASF requirement.
> > > And your committer list is very strange... 1. Why some of them are PPMC
> > and
> > > others are committers? Where this hierarchy comes from? The current
> PPMC
> > > candidates help us a lot build the Kyuubi project and community, not
> > > directly to the codebase but as mentors(before Apache). Most of them
> are
> > > from Apache mature TLPs, which have close relationships with Kyuubi.
> They
> > > have sufficient Apache Way experience to guide us to become a more
> > > Apache-like project before submitting this proposal in many ways. We
> will
> > > continue to promote more contributors to the PPMC/committer based on
> > their
> > > code or non-code contributions during incubation.
> > > 2. Your core maintainers have 5, which only 4 PPMC? The current #.2 - 5
> > top
> > > contributors will stay as committers, and they will join the PPMC
> during
> > > incubation.
> > > 3. I think only the top 4 have over 400 LOC, why you declared you have
> 5
> > > core developers? I didn't notice that 400 LOC is a bar for core
> > developer,
> > > will change 5 to 4 core developers accordingly
> > > I think you don't have enough no-salaried developers, am I missing
> > > anything? Yes.
> > > Changed in the proposal, most of the contributors are salaried, and
> some
> > of
> > > them are willing to help in the spare time. Thanks
> > > This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA in
> > > Proposal. All work here is approved by NetEase and can be forward.
> > > At last, you are asking about
> github.com/apache/incubator-kyuubi-shaded
> > > repo,
> > > what does this represent? I can only see you have only one source repo
> > and
> > > a doc repo. Removed this requirement in the proposal, and we will apply
> > it
> > > from IPMC if we do need this in the future.
> > > ------------------------------
> > >
> > > Bests
> > >
> > > Kent Yao
> > >
> > > Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午5:45写道:
> > >
> > > > Hi
> > > >
> > > > I read the contributor list and the proposal
> > > > > The risk of the Kyuubi project being abandoned is minimal. Many
> > > > organizations are using Kyuubi to build critical big data pipelines
> and
> > > > willing to help develop Kyuubi's community if it becomes an ASF
> > project.
> > > >
> > > > I think the team should evaluate the status again. You listed only 5
> > core
> > > > maintainers, but clearly only one has dominated contributions. I
> agree
> > > with
> > > > Atri, orphaned is measurable.
> > > > It is better to adjust this statement a little. Also, with this fact,
> > the
> > > > core maintainers should notice whether this small group of people to
> > make
> > > > sure your features iteration, bug fix, and release process working by
> > > > following ASF's requirements.
> > > > This would not be an easy job.
> > > >
> > > > And your committer list is very strange
> > > > ```
> > > > Kent Yao (yao@apache.org) (PPMC)
> > > > Wenchen Fan (wenchen@apache.org) (PPMC)
> > > > Wanqiang Ji (jiwq@apache.org) (PPMC)
> > > > Xun Liu (liuxun@apache.org) (PPMC)
> > > > Fei Wang (cn.feiwang@gmail.com)
> > > > Ulysses You (ulyssesyou18@gmail.com)
> > > > Cheng Pan (pan3793@gmail.com)
> > > > Hongxiang Jiang (jhx1008@gmail.com)
> > > > Zen Jin(xinjingziranchan@gmail.com)
> > > > ```
> > > > 1. Why some of them are PPMC and others are committers? Where this
> > > > hierarchy comes from?
> > > > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy
> > makes
> > > > sense, most likely not)
> > > > 3. I think only the top 4 have over 400 LOC, why you declared you
> have
> > 5
> > > > core developers?
> > > >
> > > > Another re-evaluation request is about `Reliance on Salaried
> > Developers:`
> > > > > They are all passionate about the project, and we are confident
> that
> > > the
> > > > project will continue even if no salaried developers contribute to
> the
> > > > project.
> > > >
> > > > I think you don't have enough no-salaried developers, am I missing
> > > > anything?
> > > >
> > > > About IP clearance, `Initial Source and Intellectual Property
> > Submission
> > > > Plan`
> > > > > As soon as Kyuubi is approved to join Apache Incubator, our initial
> > > > committers will submit iCLA(s) and CCLA(s). The codebase is already
> > > > licensed under the Apache License 2.0.
> > > >
> > > > This project belongs to NetEase Inc. clearly, why NO SGA?
> > > >
> > > > At last, you are asking about `
> > > github.com/apache/incubator-kyuubi-shaded`
> <http://github.com/apache/incubator-kyuubi-shaded>
> > <http://github.com/apache/incubator-kyuubi-shaded>
> > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > > repo, what does this represent? I can only see you have only one
> source
> > > > repo and a doc repo.
> > > >
> > > > Sheng Wu 吴晟
> > > > Twitter, wusheng1108
> > > >
> > > >
> > > > Atri Sharma <atri@apache.org> 于2021年6月2日周三 下午4:49写道:
> > > >
> > > > > Thanks.
> > > > >
> > > > > I looked at the link you advised on and if I am reading it
> correctly,
> > > top
> > > > > committer has 630ish commits and the next committer has 73 commits?
> > > > >
> > > > > IMO this is an indication of the community not being active enough
> > and
> > > > > hence the risk of project getting orphaned is measurable.
> > > > >
> > > > > Atri
> > > > >
> > > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.jiang@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi Atri Sharma,
> > > > > >
> > > > > > It's easier to check out the recent contribution of kyuubi from
> > this
> > > > > > link[1].
> > > > > > We already identified the one person contribution issue of kyuubi
> > > when
> > > > > > Kent came to us and we kept reminding him to put more effort
into
> > > > > > building the community.
> > > > > > Last week I had a long F2F conversation with Kent, I think he
> > already
> > > > > > knows about it and there have been some improvements in recent
> > months
> > > > > > but there is still a long way to go if Kyuubi is accepted as
a
> > > podling
> > > > > > in the Apache Incubator.
> > > > > >
> > > > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors
> > > > > >
> > > > > > Willem Jiang
> > > > > >
> > > > > > Twitter: willemjiang
> > > > > > Weibo: 姜宁willem
> > > > > >
> > > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <atri@apache.org>
> > wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I took a look at commit log and seems like all the commits
for
> > > last 3
> > > > > > > months have been done by a single person.
> > > > > > >
> > > > > > > How active are the remaining committers?
> > > > > > >
> > > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <yao@apache.org>
wrote:
> > > > > > >
> > > > > > > > Hi Alexander Alten,
> > > > > > > >
> > > > > > > > Thanks for your interest and +1 for Kyuubi.
> > > > > > > >
> > > > > > > > Not being very familiar with Dremel, I just took a
quick look
> > at
> > > > the
> > > > > > > > profile of a very similar and exciting project, Apache
Drill.
> > If
> > > I
> > > > > > > > understand correctly, Kyuubi is built on top of the
compute
> > layer
> > > > > > > > (Apache Spark), while Apache Drill is built on top
of the
> > storage
> > > > > > > > layer.
> > > > > > > >
> > > > > > > > Apache Spark has a large user base and usage scenarios
all
> over
> > > the
> > > > > > > > world. When users have some basic Spark programming
and
> tuning
> > > > > skills,
> > > > > > > > things are easy for them. But many times, most users
do not
> > have
> > > > all
> > > > > > > > of these abilities when they use Spark directly or
> indirectly.
> > We
> > > > > want
> > > > > > > > to build a more easy-to-use platform on top of it
to help
> these
> > > > > users.
> > > > > > > > On the one hand, we will work on the interface layer
to make
> it
> > > > easy
> > > > > > > > to access (of course, this effort will make our project
look
> > very
> > > > > > > > similar to other projects, such as HiveServer2). On
the other
> > > hand,
> > > > > we
> > > > > > > > will work under Kyuubi's multi-tenant architecture
to extend
> > our
> > > > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine) to make
it more
> > > > stable
> > > > > > > > and efficient in our scenario.
> > > > > > > >
> > > > > > > > What's more, we are going to add more on-prem Kyuubi
engine
> > > types,
> > > > > e.g.
> > > > > > > > Flink.
> > > > > > > >
> > > > > > > > Alexander Alten <alo@scalytics.io> 于2021年5月31日周一
下午2:35写道:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Hey,
> > > > > > > > >
> > > > > > > > > Looks like an interesting project, competition
to
> Starburst.
> > > How
> > > > > > does it
> > > > > > > > > compare to Apache Dremel?
> > > > > > > > >
> > > > > > > > >  +1 (non-binding) for incubation.
> > > > > > > > >
> > > > > > > > > Thanks and stay safe,
> > > > > > > > > --alex
> > > > > > > > >
> > > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <yao@apache.org>
> wrote:
> > > > > > > > >
> > > > > > > > > > Dear all,
> > > > > > > > > >
> > > > > > > > > > We want to propose Kyuubi as a new Apache
Incubator
> > project.
> > > > > > > > > >
> > > > > > > > > > Kyuubi is a distributed multi-tenant Thrift
JDBC/ODBC
> > server
> > > > for
> > > > > > > > > > large-scale data management, processing,
and analytics,
> > built
> > > > on
> > > > > > top
> > > > > > > > > > of Apache Spark and designed to support
more engines
> (i.e.,
> > > > > Apache
> > > > > > > > > > Flink). We are aiming to make Kyuubi an
"out-of-the-box"
> > tool
> > > > for
> > > > > > data
> > > > > > > > > > warehouses and data lakes. Kyuubi concentrates
on
> improving
> > > the
> > > > > > > > > > experience for non-Spark users, making it
easy for them
> to
> > > use
> > > > > > Spark
> > > > > > > > > > as a service and focus on business data
of their own, not
> > > where
> > > > > it
> > > > > > > > > > stores, how it computes. NetEase has open-sourced
Kyuubi
> > > since
> > > > > > 2018,
> > > > > > > > > > https://github.com/NetEase/kyuubi.
> > > > > > > > > >
> > > > > > > > > > Many thanks to Willem Jiang (Champion),
Jeff Zhang, Duo
> > > Zhang,
> > > > > > Akira
> > > > > > > > > > Ajisaka for being the mentors of the Kyuubi
project and
> > > helping
> > > > > us
> > > > > > set
> > > > > > > > > > up this proposal. The proposal can be found
at
> > > > > > > > > >
> > > > > >
> > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal
> > > .
> > > > > > > > > >
> > > > > > > > > > Looking forward to your feedback and thank
you very much.
> > > > > > > > > >
> > > > > > > > > > Best regards,
> > > > > > > > > > Kent Yao
> > > > > > > > > >
> > > > > > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > > > > > To unsubscribe, e-mail:
> > > > general-unsubscribe@incubator.apache.org
> > > > > > > > > > For additional commands, e-mail:
> > > > > general-help@incubator.apache.org
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > >  Kent Yao
> > > > > > > > @ Data Science Center, Hangzhou Research Institute,
NetEase
> > Corp.
> > > > > > > > a spark enthusiast
> > > > > > > > kyuubiis a unified multi-tenant JDBC interface for
> large-scale
> > > data
> > > > > > > > processing and analytics, built on top of Apache Spark.
> > > > > > > >
> > > > > > > > spark-authorizerA Spark SQL extension which provides
SQL
> > Standard
> > > > > > > > Authorization for Apache Spark.
> > > > > > > > spark-postgres A library for reading data from and
> transferring
> > > > data
> > > > > > > > to Postgres / Greenplum with Spark SQL and DataFrames,
> 10~100x
> > > > > faster.
> > > > > > > > itatchiA library that brings useful functions from
various
> > modern
> > > > > > > > database management systems to Apache Spark.
> > > > > > > >
> > > > > > > >
> > > > ---------------------------------------------------------------------
> > > > > > > > To unsubscribe, e-mail:
> > general-unsubscribe@incubator.apache.org
> > > > > > > > For additional commands, e-mail:
> > > general-help@incubator.apache.org
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > > > > > For additional commands, e-mail:
> general-help@incubator.apache.org
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > >  *Kent Yao *
> > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> > > a spark enthusiast
> > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant
> > JDBC
> > > interface for large-scale data processing and analytics, built on top
> of
> > > Apache
> > > Spark <http://spark.apache.org/>.
> > > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> > SQL
> > > extension which provides SQL Standard Authorization for Apache Spark
> > > <http://spark.apache.org/>.
> > > spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
> > for
> > > reading data from and transferring data to Postgres / Greenplum with
> > Spark
> > > SQL and DataFrames, 10~100x faster.
> > > itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
> > > brings useful functions from various modern database management
> > > systems to Apache
> > > Spark <http://spark.apache.org/>.
> > >
> >
>
>
> --
>
>  *Kent Yao *
> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> a spark enthusiast
> kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
> interface for large-scale data processing and analytics, built on top of
> Apache
> Spark <http://spark.apache.org/>.
> spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark SQL
> extension which provides SQL Standard Authorization for Apache Spark
> <http://spark.apache.org/>.
> spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
> reading data from and transferring data to Postgres / Greenplum with Spark
> SQL and DataFrames, 10~100x faster.
> itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
> brings useful functions from various modern database management
> systems to​ Apache
> Spark <http://spark.apache.org/>.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message