incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: [VOTE] Move Lucy to the Incubator
Date Mon, 19 Jul 2010 18:44:49 GMT


On 07/17/2010 03:23 PM, Chris Hostetter wrote:
> I would like to call a vote for accepting "Apache Lucy" for incubation
> in the Apache Incubator. The full proposal is available below. We ask
> the Incubator PMC to sponsor it, with myself (hossman) as Champion, and
> mattmann, upayavira, mikemccand, and hossman volunteering to be Mentors.
> Please cast your vote:
> [ ] +1, bring Lucy into Incubator
> [ ] +0, I don't care either way,
> [ ] -1, do not bring Lucy into Incubator, because...
> This vote will be open for 72 hours and only votes from the Incubator
> PMC are binding.
> ------------------------------------
> Lucy is a sub-project which is being spun off from the Lucene TLP but is
> not yet ready for graduation. We propose to address certain needs of the
> project by transitioning to an Incubator Podling, and assimilating the
> KinoSearch codebase.
> Lucy will be a loose port of the Lucene search engine library, written in
> C and targeted at dynamic language users.
> Lucy has two aims. First, it will be a high-performance C search engine
> library. Second, it will maximize its usability and power when accessed
> via dynamic language bindings. To that end, it will present highly
> idiomatic, carefully tailored APIs for each of its "host" binding
> languages, including support for subclasses written entirely in the
> "host" language.
> Lucy, a "loose C" port of Java Lucene, began as an ambitious,
> from-scratch Lucene sub-project, with David Balmain (author of Ferret, a
> Ruby/C port of Lucene), Doug Cutting, and Marvin Humphrey (founder of
> KinoSearch, a Perl/C port) as committers. During an initial burst of
> activity, the overall architecture for Lucy was sketched out by Dave and
> Marvin. Unfortunately, Dave became unavailable soon after, and without a
> working codebase to release or any users, it proved difficult to replace
> him. Still, Marvin carried on their work throughout a period of
> seemingly low activity.
> In the last year, that work has come to fruition: major technical
> milestones have been achieved and Lucy's underpinnings have been
> completed. Additionally, other developers from the KinoSearch community
> have taken an interest in Lucy and have begun to ramp up their
> contributions. The next steps for Lucy were articulated by the Lucene
> PMC in a recent review: make releases, acquire users, grow community.
> To implement the Lucene PMC's recommendations and get to a release as
> quickly as possible, the Lucy community proposes to assimilate the
> KinoSearch codebase, which has been retrofitted to use Lucy's core. Lucy
> still lacks a number of important indexing and search classes; we wish to
> flesh these out via IP clearance work rather than software development.
> Because Lucene is working to move away from being an "umbrella project",
> a long term goal of the Lucy project is to graduate to an ASF TLP. With
> that in mind, it seems more appropriate for the KinoSearch software grant
> to take place within the context of the Incubator, and that a Lucy
> podling and PPMC be established which will ultimately take responsibility
> for the codebase.
> There is great hunger for a search engine library in the mode of Lucene
> which is accessible from various dynamic languages, and for one
> accessible from pure C. Individuals naturally wish to code in their
> language of choice. Organizations which do not have significant Java
> expertise may not want to support Java strictly for the sake of running a
> Lucene installation. Developers may want to take advantage of C's
> interoperability and fine-grained control. Lucy will meet all these
> demands.
> Apache is a natural home for our project given the way it has always
> operated: user-driven innovation, security as a requirement, lively and
> amiable mailing list discussions, strength through diversity, and so on.
> We feel comfortable here, and we believe that we will become exemplary
> Apache citizens.
> * Make a 1.0 stable release as quickly as possible.
> * Concentrate on community expansion.
> * Expose a public C API.
> Meritocracy
> Our initial committer list includes two individuals (Peter Karman and
> Nathan Kurz) who started off as KinoSearch users, demonstrated merit
> through constructive forum participation, adept negotiation, consensus
> building, and submission of high-quality contributions, and were invited
> to become committers. Peter now rolls most releases.
> We look forward to continuing to operate as a meritocracy under the
> established traditions and rules of the ASF.
> Community
> Lucy's most active participants of late have been drawn from the
> KinoSearch and Lucene communities. Having been focused on features and
> technical goals for a long time, we are considerably overdue for a stable
> release, and anticipate rapid growth in its wake.
> Core Developers
> * Marvin Humphrey is the project founder of KinoSearch, and co-founded
> the existing Lucy sub-project. He is presently employed by Eventful,
> Inc.
> * Peter Karman has contributed to several open source projects since
> 2001, including being a committer at (a search
> engine), (an ORM) and
> (web framework). He is employed by American
> Public Media.
> * Nathan Kurz is excited by the intersection of search and
> recommendations, and has been a KinoSearch committer since 2007. As
> the owner of Scream Sorbet (, he divides his
> time between code and fruit.
> Alignment
> One Apache value which is particularly cherished by the Lucy community is
> codebase transparency. We have developed institutions which enable us to
> measure and maximize usability (see
>, and we feel strongly that the
> bindings for Lucy must present APIs and documentation which are idiomatic
> to the host language culture so that end users can consume our work as
> easily as possible.
> The controlled competition of meritocratic community development is also
> very important to us. There has been substantial cross-pollination of
> ideas between Lucene and Lucy, yielding considerable benefits for both
> projects. The Lucy developers envision that our host-language
> sub-communities will approach using and extending the library in distinct
> ways; we hope to harness the creative tension between them to drive
> innovation, building productive relationships akin to the one that Lucene
> and Lucy have today.
> A third priority of ours is to be bound by existing Apache institutions,
> for the protection of all our stakeholders.
> Orphaned products
> All core developers have been associated with the project for several
> years across multiple jobs. However, at this time, the project would
> probably not survive the departure of Marvin Humphrey, so there is a risk
> of being orphaned. Marvin has no plans to leave, but we have been
> actively working to disperse his knowledge of the code base and
> administrative responsibilities in order to make him dispensable. Having
> staggered badly after Dave Balmain's departure, we are keenly aware of
> this vulnerability and highly motivated to eliminate it.
> Inexperience with Open Source
> The core developers all have significant experience with open source
> development, and include one present Apache committer. We recognize that
> we lack PMC experience and seek to address that deficiency by using the
> Incubator environment to educate ourselves and prepare for responsible
> self-governance.
> Homogenous Developers
> Our community is geographically dispersed, with members in San Diego,
> Oakland, and Minneapolis. We all work for different organizations.
> Reliance on Salaried Developers
> Marvin Humphrey has a great job at Eventful working primarily on this
> project and supporting applications that use it. Nevertheless, he is
> extremely dedicated to Lucy and is determined to see it through to the
> point where it becomes self-sustaining, regardless of work circumstances.
> Relationships with Other Apache Products
> Lucy's relationship with Lucene of cordial "coopetition" has produced
> benefits for Lucene users in terms of indexing speed, near-real-time
> search support, and more. We expect this dynamic to continue delivering
> improvements for all parties involved.
> An Excessive Fascination with the Apache Brand
> Our desire to maintain Lucy's affiliation with Apache has less to do with
> the brand and more to do with our conviction that developing the project
> The Apache Way under Apache institutions is in Lucy's best interests.
> However, we have to acknowledge that during its time as a Lucene
> subproject, Lucy has not always fulfilled certain key requirements for an
> Apache project. In particular, it has failed to "release early, release
> often", and it has made minimal progress in expanding its community.
> We attribute some of our difficulties to the what may have been excess
> ambition in the original Lucy plan, given the scope of the project and
> the size of the initial committer list:
> The basic requirements for incubation are:
> * a working codebase -- over the years and after several failures,
> the foundation came to understand that without an initial working
> codebase, it is generally hard to bootstrap a community.
> By rebooting the project with a working codebase, we expect to avoid the
> trap that ensnared Lucy's first incarnation: we will release early,
> release often, accumulate users, nurture contributors, and grow our
> community.
> * Current Lucy website:
> * Current Lucy Subversion repository:
> * Current Lucy mailing lists:
> * KinoSearch Subversion repository:
> * KinoSearch Perl API documentation:
> * KinoSearch Discussion list:
> The initial source will be a snapshot from the KinoSearch subversion
> repository.
> KinoSearch is currently under a GPL/Artistic license. There are five
> individuals who have made multiple significant contributions to the
> codebase and whose participation is either essential or would be very
> helpful: Marvin Humphrey, Peter Karman, Nathan Kurz, Chris Nandor, and
> Father Chrysostomos. All have been contacted and are amenable to
> re-licensing their work and contributing it to Apache. We will contact
> as many other contributors as possible; if there are any that we cannot
> obtain permission from, we will refactor to expunge their work.
> The Perl bindings for KinoSearch currently depend on a few CPAN modules
> which do not have Apache-compatible licenses. It will be possible to
> eliminate all such dependencies if necessary.
> Mailing lists
> * lucy-dev
> * lucy-private (with moderated subscriptions)
> * lucy-commits
> * lucy-users
> Lucy already has lucy-dev, lucy-users, and lucy-commits mailing lists
> under Perhaps these could be deactivated and the
> memberships migrated to the appropriate lists under,
> leaving the archives as read-only.
> Subversion Directory
> Lucy already has a Subversion directory at
> In keeping with naming
> conventions, it could be moved to
> Issue Tracking
> Lucy already has a JIRA tracker: Lucy (LUCY)
> Other Resources
> Lucy already has a MoinMoin wiki at It would be
> convenient to keep it, especially since its current location is also
> where it would end up upon TLP graduation, but we will defer to the
> wishes of the Incubator PMC if standard Incubator wiki placement is
> recommended.
> Marvin Humphrey marvin AT apache DOT org Eventful yes
> Peter Karman peter AT peknet DOT com American Public Media yes
> Nathan Kurz nate AT verse DOT com Scream Sorbet yes
> Simon Willnauer simonw AT apache DOT org yes
> Champion
> * Chris Hostetter (hossman AT apache DOT org)
> Nominated Mentors
> * Chris Mattmann (mattmann AT apache DOT org)
> * Upayavira (upayavira AT apache DOT org)
> * Mike McCandless (mikemccand AT apache DOT org) (pending IPMC membership)
> * Chris Hostetter (hossman AT apache DOT org) (pending IPMC membership)
> Sponsoring Entity
> Lucy is currently sponsored by Lucene as a sub-project. This proposal
> advocates changing Lucy's relationship with Apache from developing all
> new code as a Lucene sub-project, to instead assimilating existing code
> (KinoSearch) under the sponsorship of the Incubator.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message