incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kent Yao <...@apache.org>
Subject Re: [DISCUSS] Incubating Proposal of Kyuubi
Date Thu, 03 Jun 2021 02:55:52 GMT
Thanks very much, Sheng :)



Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月3日周四 上午8:10写道:

> Kent
>
> Thanks.
> The proposal is generally better this time.
>
> If your initial committers and your mentors agree, you are ready to embrace
> the ASF culture, I am good with that.
> Just from my experience, your community looks a little too young, like
> Atri's concern, which is reasonable.
> But I would like to trust you and your mentors' judgment and determination.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Kent Yao <yao@apache.org> 于2021年6月3日周四 上午12:54写道:
>
> > Hi Sheng,
> >
> > Truly sorry for the format issue.
> >
> > Please trust me I did not mean to hurt anybody and push any contributor
> > away. We drafted this proposal together based on the New Podling
> Proposal.
> > We never tried to use PPMC and committer to measure whose contribution is
> > higher. This mistake has been fixed in the proposal.
> >
> > About releases, we will follow
> > https://www.apache.org/legal/release-policy.html and the guide of our
> > mentors. We will also take the initiative to learn other documents.
> >
> > > This is not a block, the point I mentioned this is, you will need more
> > people.
> >
> > We will keep working on this.
> >
> > Bests,
> >
> > Kent Yao
> >
> >
> > Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午10:53写道:
> >
> > > Hi Kent
> > >
> > > Thanks for replying, but to be honest, your mail format is chaos.
> > > Take a look at bellowing, which is what I saw in the gmail.
> > >
> > > This may be an issue of your mail client or some else.
> > >
> > > > The current #.2 - 5 top
> > > > contributors will stay as committers, and they will join the PPMC
> > during
> > > > incubation.
> > >
> > > PPMC usually is the default for all initial committers, considering
> your
> > > community is still small and not very active.
> > > You should invite all people you trusted to join, rather than pushing
> > them
> > > away.
> > > This is not just a too-high bar issue. Excluding 4 out of 5 is strange.
> > > I can't guess what is the root cause, why the other 4 PPMC members
> don't
> > > touch codes but are trusted completely. Meanwhile, people with easily
> > > measurable code contributions are just as committers?
> > > We(ASF) always remind the community to recognize no-code contribution,
> > but
> > > in this case, it seems the opposite? Don't recognize code contributors
> > > enough?
> > > Please reach your mentors, this seems not a typical ASF style. If you
> > have
> > > other people already involved in the ASF, asking them too.
> > >
> > > And from your description of the release process, I think you should
> talk
> > > more with your mentors.
> > > Such as what is a qualified ASF release, what should be checked.
> > > This is not a block, the point I mentioned this is, you will need more
> > > people.
> > >
> > >
> > > Sheng Wu 吴晟
> > > Twitter, wusheng1108
> > >
> > >
> > > Kent Yao <yao@apache.org> 于2021年6月2日周三 下午10:37写道:
> > >
> > > > Hi Sheng,
> > > >
> > > > Thanks for your detailed check and valuable comments. They are
> > > > accommodating. I have addressed some of them directly in the
> proposal.
> > > > Please also check the table below for some Q&As.
> > > > QA
> > > > You listed only 5 core maintainers, but clearly only one has
> dominated
> > > > contributions. The distortion and skewness of the contributions are
> > > mainly
> > > > caused by the way how we merge PRs changed(mentioned in another email
> > on
> > > > the list).
> > > > I agree with Atri, orphaned is measurable. It is better to adjust
> this
> > > > statement a little. Address the comment to notify the risk in the
> > Kyuubi
> > > > proposal.
> > > > Also, with this fact, the core maintainers should notice whether this
> > > small
> > > > group of people to make sure your features iteration, bug fix, and
> > > release
> > > > process working by following ASF's requirements.This would not be an
> > easy
> > > > job. Currently, we learn the release process from some Apache
> projects,
> > > > using major.minor.patch for versioning. As we can see, since Kyuubi
> > 1.0,
> > > it
> > > > takes us 2 ~ 3 weeks for a patch release that contains 5-10 bug fixes
> > > until
> > > > EOL. And it takes us about two months to finish a minor release with
> 3
> > ~
> > > 5
> > > > big improvements and various small features/bug fixes. We also will
> > start
> > > > with some release candidates for preview and test before the final.
> > This
> > > is
> > > > how we develop and release Kyuubi for now, but we do have a long way
> to
> > > go
> > > > and learn to match the ASF requirement.
> > > > And your committer list is very strange... 1. Why some of them are
> PPMC
> > > and
> > > > others are committers? Where this hierarchy comes from? The current
> > PPMC
> > > > candidates help us a lot build the Kyuubi project and community, not
> > > > directly to the codebase but as mentors(before Apache). Most of them
> > are
> > > > from Apache mature TLPs, which have close relationships with Kyuubi.
> > They
> > > > have sufficient Apache Way experience to guide us to become a more
> > > > Apache-like project before submitting this proposal in many ways. We
> > will
> > > > continue to promote more contributors to the PPMC/committer based on
> > > their
> > > > code or non-code contributions during incubation.
> > > > 2. Your core maintainers have 5, which only 4 PPMC? The current #.2
> - 5
> > > top
> > > > contributors will stay as committers, and they will join the PPMC
> > during
> > > > incubation.
> > > > 3. I think only the top 4 have over 400 LOC, why you declared you
> have
> > 5
> > > > core developers? I didn't notice that 400 LOC is a bar for core
> > > developer,
> > > > will change 5 to 4 core developers accordingly
> > > > I think you don't have enough no-salaried developers, am I missing
> > > > anything? Yes.
> > > > Changed in the proposal, most of the contributors are salaried, and
> > some
> > > of
> > > > them are willing to help in the spare time. Thanks
> > > > This project belongs to NetEase Inc. clearly, why NO SGA? Added SGA
> in
> > > > Proposal. All work here is approved by NetEase and can be forward.
> > > > At last, you are asking about
> > github.com/apache/incubator-kyuubi-shaded
> > > > repo,
> > > > what does this represent? I can only see you have only one source
> repo
> > > and
> > > > a doc repo. Removed this requirement in the proposal, and we will
> apply
> > > it
> > > > from IPMC if we do need this in the future.
> > > > ------------------------------
> > > >
> > > > Bests
> > > >
> > > > Kent Yao
> > > >
> > > > Sheng Wu <wu.sheng.841108@gmail.com> 于2021年6月2日周三 下午5:45写道:
> > > >
> > > > > Hi
> > > > >
> > > > > I read the contributor list and the proposal
> > > > > > The risk of the Kyuubi project being abandoned is minimal. Many
> > > > > organizations are using Kyuubi to build critical big data pipelines
> > and
> > > > > willing to help develop Kyuubi's community if it becomes an ASF
> > > project.
> > > > >
> > > > > I think the team should evaluate the status again. You listed only
> 5
> > > core
> > > > > maintainers, but clearly only one has dominated contributions. I
> > agree
> > > > with
> > > > > Atri, orphaned is measurable.
> > > > > It is better to adjust this statement a little. Also, with this
> fact,
> > > the
> > > > > core maintainers should notice whether this small group of people
> to
> > > make
> > > > > sure your features iteration, bug fix, and release process working
> by
> > > > > following ASF's requirements.
> > > > > This would not be an easy job.
> > > > >
> > > > > And your committer list is very strange
> > > > > ```
> > > > > Kent Yao (yao@apache.org) (PPMC)
> > > > > Wenchen Fan (wenchen@apache.org) (PPMC)
> > > > > Wanqiang Ji (jiwq@apache.org) (PPMC)
> > > > > Xun Liu (liuxun@apache.org) (PPMC)
> > > > > Fei Wang (cn.feiwang@gmail.com)
> > > > > Ulysses You (ulyssesyou18@gmail.com)
> > > > > Cheng Pan (pan3793@gmail.com)
> > > > > Hongxiang Jiang (jhx1008@gmail.com)
> > > > > Zen Jin(xinjingziranchan@gmail.com)
> > > > > ```
> > > > > 1. Why some of them are PPMC and others are committers? Where this
> > > > > hierarchy comes from?
> > > > > 2. Your core maintainers have 5, which only 4 PPMC? (Even hierarchy
> > > makes
> > > > > sense, most likely not)
> > > > > 3. I think only the top 4 have over 400 LOC, why you declared you
> > have
> > > 5
> > > > > core developers?
> > > > >
> > > > > Another re-evaluation request is about `Reliance on Salaried
> > > Developers:`
> > > > > > They are all passionate about the project, and we are confident
> > that
> > > > the
> > > > > project will continue even if no salaried developers contribute to
> > the
> > > > > project.
> > > > >
> > > > > I think you don't have enough no-salaried developers, am I missing
> > > > > anything?
> > > > >
> > > > > About IP clearance, `Initial Source and Intellectual Property
> > > Submission
> > > > > Plan`
> > > > > > As soon as Kyuubi is approved to join Apache Incubator, our
> initial
> > > > > committers will submit iCLA(s) and CCLA(s). The codebase is already
> > > > > licensed under the Apache License 2.0.
> > > > >
> > > > > This project belongs to NetEase Inc. clearly, why NO SGA?
> > > > >
> > > > > At last, you are asking about `
> > > > github.com/apache/incubator-kyuubi-shaded`
> <http://github.com/apache/incubator-kyuubi-shaded>
> > <http://github.com/apache/incubator-kyuubi-shaded>
> > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > > > <http://github.com/apache/incubator-kyuubi-shaded>
> > > > > repo, what does this represent? I can only see you have only one
> > source
> > > > > repo and a doc repo.
> > > > >
> > > > > Sheng Wu 吴晟
> > > > > Twitter, wusheng1108
> > > > >
> > > > >
> > > > > Atri Sharma <atri@apache.org> 于2021年6月2日周三 下午4:49写道:
> > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > I looked at the link you advised on and if I am reading it
> > correctly,
> > > > top
> > > > > > committer has 630ish commits and the next committer has 73
> commits?
> > > > > >
> > > > > > IMO this is an indication of the community not being active
> enough
> > > and
> > > > > > hence the risk of project getting orphaned is measurable.
> > > > > >
> > > > > > Atri
> > > > > >
> > > > > > On Wed, 2 Jun 2021, 06:57 Willem Jiang, <willem.jiang@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi Atri Sharma,
> > > > > > >
> > > > > > > It's easier to check out the recent contribution of kyuubi
from
> > > this
> > > > > > > link[1].
> > > > > > > We already identified the one person contribution issue
of
> kyuubi
> > > > when
> > > > > > > Kent came to us and we kept reminding him to put more effort
> into
> > > > > > > building the community.
> > > > > > > Last week I had a long F2F conversation with Kent, I think
he
> > > already
> > > > > > > knows about it and there have been some improvements in
recent
> > > months
> > > > > > > but there is still a long way to go if Kyuubi is accepted
as a
> > > > podling
> > > > > > > in the Apache Incubator.
> > > > > > >
> > > > > > > [1]https://github.com/NetEase/kyuubi/graphs/contributors
> > > > > > >
> > > > > > > Willem Jiang
> > > > > > >
> > > > > > > Twitter: willemjiang
> > > > > > > Weibo: 姜宁willem
> > > > > > >
> > > > > > > On Tue, Jun 1, 2021 at 10:52 PM Atri Sharma <atri@apache.org>
> > > wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I took a look at commit log and seems like all the
commits
> for
> > > > last 3
> > > > > > > > months have been done by a single person.
> > > > > > > >
> > > > > > > > How active are the remaining committers?
> > > > > > > >
> > > > > > > > On Mon, 31 May 2021, 12:59 Kent Yao, <yao@apache.org>
wrote:
> > > > > > > >
> > > > > > > > > Hi Alexander Alten,
> > > > > > > > >
> > > > > > > > > Thanks for your interest and +1 for Kyuubi.
> > > > > > > > >
> > > > > > > > > Not being very familiar with Dremel, I just took
a quick
> look
> > > at
> > > > > the
> > > > > > > > > profile of a very similar and exciting project,
Apache
> Drill.
> > > If
> > > > I
> > > > > > > > > understand correctly, Kyuubi is built on top
of the compute
> > > layer
> > > > > > > > > (Apache Spark), while Apache Drill is built on
top of the
> > > storage
> > > > > > > > > layer.
> > > > > > > > >
> > > > > > > > > Apache Spark has a large user base and usage
scenarios all
> > over
> > > > the
> > > > > > > > > world. When users have some basic Spark programming
and
> > tuning
> > > > > > skills,
> > > > > > > > > things are easy for them. But many times, most
users do not
> > > have
> > > > > all
> > > > > > > > > of these abilities when they use Spark directly
or
> > indirectly.
> > > We
> > > > > > want
> > > > > > > > > to build a more easy-to-use platform on top of
it to help
> > these
> > > > > > users.
> > > > > > > > > On the one hand, we will work on the interface
layer to
> make
> > it
> > > > > easy
> > > > > > > > > to access (of course, this effort will make our
project
> look
> > > very
> > > > > > > > > similar to other projects, such as HiveServer2).
On the
> other
> > > > hand,
> > > > > > we
> > > > > > > > > will work under Kyuubi's multi-tenant architecture
to
> extend
> > > our
> > > > > > > > > on-prem Spark SQL engine(a.k.a Kyuubi engine)
to make it
> more
> > > > > stable
> > > > > > > > > and efficient in our scenario.
> > > > > > > > >
> > > > > > > > > What's more, we are going to add more on-prem
Kyuubi engine
> > > > types,
> > > > > > e.g.
> > > > > > > > > Flink.
> > > > > > > > >
> > > > > > > > > Alexander Alten <alo@scalytics.io> 于2021年5月31日周一
下午2:35写道:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hey,
> > > > > > > > > >
> > > > > > > > > > Looks like an interesting project, competition
to
> > Starburst.
> > > > How
> > > > > > > does it
> > > > > > > > > > compare to Apache Dremel?
> > > > > > > > > >
> > > > > > > > > >  +1 (non-binding) for incubation.
> > > > > > > > > >
> > > > > > > > > > Thanks and stay safe,
> > > > > > > > > > --alex
> > > > > > > > > >
> > > > > > > > > > On Mon, May 31, 2021, 08:21 Kent Yao <yao@apache.org>
> > wrote:
> > > > > > > > > >
> > > > > > > > > > > Dear all,
> > > > > > > > > > >
> > > > > > > > > > > We want to propose Kyuubi as a new
Apache Incubator
> > > project.
> > > > > > > > > > >
> > > > > > > > > > > Kyuubi is a distributed multi-tenant
Thrift JDBC/ODBC
> > > server
> > > > > for
> > > > > > > > > > > large-scale data management, processing,
and analytics,
> > > built
> > > > > on
> > > > > > > top
> > > > > > > > > > > of Apache Spark and designed to support
more engines
> > (i.e.,
> > > > > > Apache
> > > > > > > > > > > Flink). We are aiming to make Kyuubi
an
> "out-of-the-box"
> > > tool
> > > > > for
> > > > > > > data
> > > > > > > > > > > warehouses and data lakes. Kyuubi concentrates
on
> > improving
> > > > the
> > > > > > > > > > > experience for non-Spark users, making
it easy for them
> > to
> > > > use
> > > > > > > Spark
> > > > > > > > > > > as a service and focus on business
data of their own,
> not
> > > > where
> > > > > > it
> > > > > > > > > > > stores, how it computes. NetEase has
open-sourced
> Kyuubi
> > > > since
> > > > > > > 2018,
> > > > > > > > > > > https://github.com/NetEase/kyuubi.
> > > > > > > > > > >
> > > > > > > > > > > Many thanks to Willem Jiang (Champion),
Jeff Zhang, Duo
> > > > Zhang,
> > > > > > > Akira
> > > > > > > > > > > Ajisaka for being the mentors of the
Kyuubi project and
> > > > helping
> > > > > > us
> > > > > > > set
> > > > > > > > > > > up this proposal. The proposal can
be found at
> > > > > > > > > > >
> > > > > > >
> > > https://cwiki.apache.org/confluence/display/INCUBATOR/KyuubiProposal
> > > > .
> > > > > > > > > > >
> > > > > > > > > > > Looking forward to your feedback and
thank you very
> much.
> > > > > > > > > > >
> > > > > > > > > > > Best regards,
> > > > > > > > > > > Kent Yao
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > > > > > To unsubscribe, e-mail:
> > > > > general-unsubscribe@incubator.apache.org
> > > > > > > > > > > For additional commands, e-mail:
> > > > > > general-help@incubator.apache.org
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > >
> > > > > > > > >  Kent Yao
> > > > > > > > > @ Data Science Center, Hangzhou Research Institute,
NetEase
> > > Corp.
> > > > > > > > > a spark enthusiast
> > > > > > > > > kyuubiis a unified multi-tenant JDBC interface
for
> > large-scale
> > > > data
> > > > > > > > > processing and analytics, built on top of Apache
Spark.
> > > > > > > > >
> > > > > > > > > spark-authorizerA Spark SQL extension which provides
SQL
> > > Standard
> > > > > > > > > Authorization for Apache Spark.
> > > > > > > > > spark-postgres A library for reading data from
and
> > transferring
> > > > > data
> > > > > > > > > to Postgres / Greenplum with Spark SQL and DataFrames,
> > 10~100x
> > > > > > faster.
> > > > > > > > > itatchiA library that brings useful functions
from various
> > > modern
> > > > > > > > > database management systems to Apache Spark.
> > > > > > > > >
> > > > > > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > > > > > To unsubscribe, e-mail:
> > > general-unsubscribe@incubator.apache.org
> > > > > > > > > For additional commands, e-mail:
> > > > general-help@incubator.apache.org
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail:
> general-unsubscribe@incubator.apache.org
> > > > > > > For additional commands, e-mail:
> > general-help@incubator.apache.org
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > >  *Kent Yao *
> > > > @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> > > > a spark enthusiast
> > > > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant
> > > JDBC
> > > > interface for large-scale data processing and analytics, built on top
> > of
> > > > Apache
> > > > Spark <http://spark.apache.org/>.
> > > > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A
> Spark
> > > SQL
> > > > extension which provides SQL Standard Authorization for Apache Spark
> > > > <http://spark.apache.org/>.
> > > > spark-postgres <https://github.com/yaooqinn/spark-postgres> A
> library
> > > for
> > > > reading data from and transferring data to Postgres / Greenplum with
> > > Spark
> > > > SQL and DataFrames, 10~100x faster.
> > > > itatchi <https://github.com/yaooqinn/spark-func-extras>A library
> that
> > > > brings useful functions from various modern database management
> > > > systems to Apache
> > > > Spark <http://spark.apache.org/>.
> > > >
> > >
> >
> >
> > --
> >
> >  *Kent Yao *
> > @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
> > a spark enthusiast
> > kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant
> JDBC
> > interface for large-scale data processing and analytics, built on top of
> > Apache
> > Spark <http://spark.apache.org/>.
> > spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark
> SQL
> > extension which provides SQL Standard Authorization for Apache Spark
> > <http://spark.apache.org/>.
> > spark-postgres <https://github.com/yaooqinn/spark-postgres> A library
> for
> > reading data from and transferring data to Postgres / Greenplum with
> Spark
> > SQL and DataFrames, 10~100x faster.
> > itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
> > brings useful functions from various modern database management
> > systems to​ Apache
> > Spark <http://spark.apache.org/>.
> >
>


-- 

 *Kent Yao *
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
a spark enthusiast
kyuubi <https://github.com/yaooqinn/kyuubi>is a unified multi-tenant JDBC
interface for large-scale data processing and analytics, built on top of Apache
Spark <http://spark.apache.org/>.
spark-authorizer <https://github.com/yaooqinn/spark-authorizer>A Spark SQL
extension which provides SQL Standard Authorization for Apache Spark
<http://spark.apache.org/>.
spark-postgres <https://github.com/yaooqinn/spark-postgres> A library for
reading data from and transferring data to Postgres / Greenplum with Spark
SQL and DataFrames, 10~100x faster.
itatchi <https://github.com/yaooqinn/spark-func-extras>A library that
brings useful functions from various modern database management
systems to​ Apache
Spark <http://spark.apache.org/>.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message