incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <guosi...@gmail.com>
Subject Re: [VOTE] Pulsar into the Apache Incubator
Date Wed, 17 May 2017 04:58:12 GMT
+1 (non-binding)

Great to see Pulsar joining the ASF and more and more projects are built
over Apache BookKeeper!

- Sijie

On Tue, May 16, 2017 at 7:39 PM, Bryan Call <bcall@apache.org> wrote:

> Hi All,
>
> As the champion for Pulsar, I would like to start a VOTE to bring the
> project in as Apache incubator podling.
>
> The ASF voting rules are described:
>
>         https://www.apache.org/foundation/voting.html
>
> A vote for accepting a new Apache Incubator podling is a majority vote for
> which
> only Incubator PMC member votes are binding.
>
> This vote will run for at least 72 hours. Please VOTE as follows
> [] +1 Accept Pulsar into the Apache Incubator
> [] +0 Abstain.
> [] -1 Do not accept Pulsar into the Apache Incubator because ...
>
> The proposal is listed below, but you can also access it on the wiki:
>
>         https://wiki.apache.org/incubator/PulsarProposal
>
> -Bryan
>
>
> = Pulsar Proposal =
>
> == Abstract ==
>
> Pulsar is a highly scalable, low latency messaging platform running on
> commodity hardware. It provides simple pub-sub semantics over topics,
> guaranteed at-least-once delivery of messages, automatic cursor management
> for
> subscribers, and cross-datacenter replication.
>
> == Proposal ==
>
> Pub-sub messaging is a very common design pattern that is increasingly
> found
> in distributed systems powering Internet applications. These applications
> provide real-time services, and need publish-latencies of 5ms on average
> and
> no more than 15ms at the 99th percentile. At Internet scale, these
> applications require a messaging system with ordering, strong durability,
> and
> delivery guarantees. In order to handle the “five 9’s” durability
> requirements
> of a production environment, the messages have to be committed on multiple
> disks or nodes.
>
> Pulsar has been developed at Yahoo to address these specific requirements
> by
> providing a hosted service supporting millions of topics for multiple
> tenants.
> The current incarnation of Pulsar has been open-sourced under Apache
> license
> in September 2016 and it is the direct evolution of systems that were
> developed at Yahoo since 2011.
>
> We believe there is currently no other system that provides a multi-tenant
> hosted messaging platform capable of supporting a huge number of topics
> while
> maintaining strict guarantees for durability, ordering and low latency.
> Current solutions would require to run multiple individual clusters with
> additional operational work and capacity overhead.
>
> Since the open sourcing of Pulsar, the development has been done
> exclusively
> on the public Github repository and two major releases were shipped (1.15
> and
> 1.16), along with multiple minor ones. Several other companies have
> expressed
> interest in the project and its future direction.
>
> == Rationale ==
>
> Pulsar is a platform that is built on top of several other Apache
> projects. In
> particular, Apache BookKeeper is used to store the data and Apache
> ZooKeeper
> is used for coordination and metadata storage. Pulsar is also interoperable
> out of the box with Apache Storm, to provide an easy to use stream
> processing
> solution.
>
> We want to establish a community outside the scope of initial core
> developers
> at Yahoo and we believe that the Apache Foundation is a great fit and long-
> term home for Pulsar, as it provides an established process for community-
> driven development and decision making by consensus. This is exactly the
> model
> we want to adopt for future Pulsar development.
>
> == Initial Goals ==
>
> The initial goals will be to move the existing codebase to Apache and
> integrate with the Apache development process. Furthermore, we plan for
> incremental development, and releases along with the Apache guidelines.
>
> == Current Status ==
>
> Pulsar has been in service at large scale for more than 2 years at Yahoo.
> In
> this time around 60 different applications were integrated with Pulsar.
> Other
> companies are evaluating it as well and have been contributing code to the
> project.
>
> === Meritocracy ===
>
> We value meritocracy and we understand that it is the basis to form an open
> community that encourages multiple companies and individuals to contribute
> and
> get invested in the project future. We will encourage and monitor
> participation and make sure to extend privileges and responsibilities to
> all
> contributors.
>
> === Community ===
>
> We have validated, through the interest demonstrated by Pulsar users at
> Yahoo,
> that a reliable hosted pub-sub messaging platform represent a very
> important
> building block for web-scale distributed applications. We believe that many
> companies can benefit by applying the same model and that bringing Pulsar
> to
> Apache will get the community to grow stronger.
>
> === Core Developers ===
>
> Pulsar has been initially developed at Yahoo and received significant
> contributions from Yahoo Japan. After having open-sourced the project there
> have been contribution from developers from several external companies.
>
> === Alignment ===
>
> Pulsar builds upon other Apache projects such as ZooKeeper and BookKeeper,
> along with a number of other Apache libraries. We have already integrated
> with
> Storm and we envision to integrate with multiple other systems in the
> streaming and big data space.
>
> == Known Risks ==
>
> === Orphaned Products ===
>
> Yahoo has been doing most of the development and, given that many internal
> platforms depends on Pulsar, it is heavily invested in the long term
> success
> of the the project. Yahoo has a long history participating in open-source
> projects, and has been also a long time contributor to the Apache
> community.
>
> === Inexperience with Open Source ===
>
> Many Pulsar contributors are already familiar with the open source process
> and
> several of them are committers on other Apache projects. We will be
> actively
> working with experienced Apache community members to improve our project.
>
> === Homogenous Developers ===
>
> The initial committers are employed by large companies including Yahoo,
> Yahoo!
> Japan, Salesforce and MercadoLibre. We hope to grow the community and to
> include additional committers based on their contributions to the project.
>
> === Reliance on Salaried Developers ===
>
> It is expected that Pulsar development will occur on both salaried time
> and on
> volunteer time, after hours. The majority of initial committers are paid by
> their employer to contribute to this project. However, they are all
> passionate
> about the project, and we are confident that the project will continue
> even if
> no salaried developers contribute to the project.
>
> === Relationships with Other Apache Products ===
>
> As mentioned in the Rationale section, Pulsar is closely dependent and
> integrated with BookKeeper and ZooKeeper and Storm. There are ongoing to
> integrate with other projects such Apache Spark. We look forward to
> collaborating with those communities, as well as other Apache communities.
>
> === An Excessive Fascination with the Apache Brand ===
>
> We are applying to the Incubator process because we think it is the next
> logical step for the Pulsar project after open-sourcing the code in 2016.
> This
> proposal is not for the purpose of generating publicity. Rather, we want to
> make sure to create a very inclusive and meritocratic community, outside
> the
> umbrella of a single company. Yahoo has a long standing history of
> contributing to Apache projects and the Pulsar developers and contributors
> understand the implication of making it an Apache project.
>
> == Documentation ==
>  * Pulsar code base: https://github.com/yahoo/pulsar
>  * Pulsar documentation: https://github.com/yahoo/pulsar/blob
>         /master/docs/Documentation.md
>  * Blog post:  [[https://yahooeng.tumblr.com/post/150078336821/
>         open-sourcing-pulsar-pub-sub-messaging-at-scale|Open-sourcing
> Pulsar,
>         Pub-sub Messaging at Scale]]
>
> == Initial Source ==
>
> The Pulsar codebase is currently hosted on Github:
> https://github.com/yahoo/pulsar. This is the exact codebase that we would
> migrate to the Apache Software Foundation.
>
> == Source and Intellectual Property Submission Plan ==
>
> The Pulsar source code in Github is currently licensed under Apache License
> v2.0 and the copyright is assigned to Yahoo. All the contributions from
> external parties have been received under Apache style CLA. If Pulsar
> fulfills
> and passes the conditions for being an Incubator project in the ASF, Yahoo
> will transition the source code ownership to the Apache Software Foundation
> via the Software Grant Agreement.
>
> == External Dependencies ==
>
> To the best of our knowledge, all of Pulsar dependencies are distributed
> under
> Apache compatible licenses.
>
> === External dependencies licensed under Apache License 2.0: ===
>
> Athenz, JCommander, HPPC - High Performance Primitive Collections for Java,
> FasterXML Jackson, Caffeine Async Cache, GSon, Guava, Netty, DataSketches,
> Joda-time, Jna Java Native Access, Lz4-java, AsyncHttpClient, Jetty,
> SnakeYAML
>
> === ASF Projects: ===
>
> BookKeeper, ZooKeeper, Storm, Log4J, Commons (BeanUtils, CLI,  Codec,
> Collections, Configuration, Digester, IO, Lang, Lang3, Logging)
>
> === Others: ===
>  * Protobuf (3-clause BSD)
>  * JLine (BSD License)
>  * Jersey (CDDL - Version 1.1)
>  * HdrHistogram (BSD License)
>  * RocksDB-JNI (3-clause BSD)
>  * SLF4J API (MIT)
>
> == Required Resources ==
>
> === Mailing lists ===
>  * users@pulsar.incubator.apache.org
>  * dev@pulsar.incubator.apache.org
>  * commits@pulsar.incubator.apache.org
>  * private@pulsar.incubator.apache.org (with moderated subscriptions)
>
> === Git Repository ===
>  * https://git-wip-us.apache.org/repos/asf/incubator-pulsar
>
> === Issue Tracking ===
>  * JIRA Pulsar (PULSAR)
>
> == Initial Committers ==
>  * Matteo Merli - <mmerli@apache.org>
>  * Joe Francis - <joef@yahoo-inc.com>
>  * Rajan Dhabalia - <rdhabalia@yahoo-inc.com>
>  * Sahaya Andrews Albert - <sandrews@yahoo-inc.com>
>  * Maurice Barnum - <msb@yahoo-inc.com>
>  * Ludwig Pummer - <ludwig@yahoo-inc.com>
>  * Jai Asher - <jai1@yahoo-inc.com>
>  * Siddharth Boobna - <sboobna@apache.org>
>  * Nozomi Kurihara - <nkurihar@yahoo-corp.jp>
>  * Yuki Shiga - <yushiga@yahoo-corp.jp>
>  * Masakazu Kitajo - <maskit@apache.org>
>  * Sebastián Schepens - <sebastian.schepens@mercadolibre.com>
>  * Brad McMillen - <bradtm@yahoo-inc.com>
>  * Bobbey Reese - <breese@yahoo-inc.com>
>  * Masahiro Sakamoto <massakam@yahoo-corp.jp>
>  * Hiroyuki Sakai <hsakai@yahoo-corp.jp>
>
> == Affiliations ==
>  * Matteo Merli - Streamlio
>  * Joe Francis - Yahoo
>  * Rajan Dhabalia - Yahoo
>  * Sahaya Andrews Albert - Yahoo
>  * Maurice Barnum - Yahoo
>  * Ludwig Pummer - Yahoo
>  * Jai Asher - Yahoo
>  * Siddharth Boobna - Salesforce
>  * Nozomi Kurihara - Yahoo! Japan
>  * Yuki Shiga - Yahoo! Japan
>  * Masakazu Kitajo - Apple
>  * Sebastián Schepens - Mercado Libre
>  * Brad McMillen - Yahoo
>  * Bobbey Reese - Yahoo
>
> == Sponsors ==
>
> === Champion ===
>  * Bryan Call
>
> === Nominated Mentors ===
>  * Dave Fisher
>  * Jim Jagielski
>  * P. Taylor Goetz
>  * Francis Liu
>
> === Sponsoring Entity ===
>  * The Apache Incubator PMC
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message