incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <>
Subject [VOTE] Merge DistributedLog as the subproject of Apache BookKeeper
Date Fri, 09 Jun 2017 00:21:37 GMT
( /cc bookkeeper dev@ and incubator general@ for awareness )

Hi all,

There was a joint discussion between BookKeeper PMC and DistributedLog PPMC
about moving the development of DistributedLog as part of Apache
BookKeeper. The reasons behind it are:

First, DistributedLog is born as an extension to BookKeeper, to offer
continuous log streams as the service. The ledger API in bookkeeper is a
lower level API and has learning curves, while the log stream API in
distributedlog is a higher level API that simplifies the usage. The
combination of ledger API and stream API would offer a better
developer/user experience for applications.

Secondly, using ledgers to build continuous (re-openable) log stream is a
very common pattern for BookKeeper use cases. We did this for HDFS namenode
journal, for Hedwig, for DistributedLog, and for Pulsar. The same pattern
has been implemented again and again. Merge DistributedLog (also
ManagedLedger in Pulsar) with BookKeeper will consolidate all the
development efforts around this common 'log stream' pattern.

Thirdly, the 'log' stream abstraction is a good abstraction for both
messaging and streaming. Internally at BookKeeper, there are a few places
that can use such 'messaging' facility to improve bookkeeper itself. the
log stream in DistributedLog can be used internally at bookkeeper for
streaming changes as well.

We choose merging DistributedLog as subproject rather than modules. It is a
softer starting point to avoid disrupting the folks who are depending on
the ledger api alone. The BookKeeper PMC and DistributedLog PPMC has
achieved initial consensus on this merge. There is an official VOTE ongoing
in bookkeeper PMC. We'd like to bring this to the distributedlog community
for a community vote following the guidelines here

Please vote +1 if in favor of merging DistributedLog to BookKeeper, and -1
if not. The vote will be open until Tuesday 13rd June, 18:00 PST.

- Sijie

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message