incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <>
Subject Re: [VOTE] Accept Trafodion into Apache Incubator
Date Wed, 20 May 2015 05:02:59 GMT
+1 (binding)

--Chris Nauroth

On 5/19/15, 2:27 PM, "Stack" <> wrote:

>Following the discussion earlier in the thread [1], I would like to call a
>VOTE to accept Trafodion as a new Apache Incubator project.
>The proposal is available on the wiki at [2] and is also attached to this
>The VOTE is open for at least the next 72 hours:
> [ ] +1 accept Trafodion into the Apache Incubator
> [ ] ±0 Abstain
> [ ] -1 because...
>I am +1 (binding)
>Thank you,
>Trafodion Apache Incubator Proposal
>Trafodion is a webscale SQL-on-Hadoop solution enabling transactional or
>operational workloads on Hadoop.
>Apache Trafodion builds on the scalability, elasticity, and flexibility of
>Hadoop. Trafodion extends Hadoop to provide guaranteed transactional
>integrity, enabling new kinds of big data applications to run on Hadoop.
>features of Apache Trafodion include:
>* Full-functioned ANSI SQL language support
>* JDBC/ODBC connectivity for Linux/Windows clients
>* Distributed ACID transaction protection across multiple statements,
>tables and rows
>* Performance improvements for OLTP workloads with compile-time and
>run-time optimizations
>* Support for large data sets using a parallel-aware query optimizer
>* ANSI SQL security and data integrity constraints including referential
>Hewlett-Packard Company submits this proposal to donate its Apache
>Version 2.0 open source project known as Trafodion, its source code,
>documentation, and web site content to the Apache Software Foundation in
>order to build an open source community
>Trafodion is an open source project sponsored by HP, incubated at HP Labs
>and HP-IT, to develop an enterprise-class SQL-on-Hadoop solution targeting
>big data transactional or operational workloads. HP publically announced
>the open source project and uploaded the source code to GitHub in June
>The SQL compiler, optimizer and executor components of Trafodion have a
>rich heritage. Under development since 1993, they were released as
>commercial closed source software in various flavors such as HP NonStop
>SQL/MX and HP Neoview. NonStop SQL/MX was designed for online transaction
>processing on HP¹s NonStop (formerly Tandem) fault-tolerant servers and is
>known for its high availability, scalability, and performance. Hundreds of
>companies and thousands of servers are running mission-critical
>applications today on NonStop SQL/MX. In addition, much of these
>today are running internal to HP as the core of its Enterprise Data
>Warehouse (EDW), managing over a PB of data.
>Starting in 2013, the software was modified to run on HBase and a new
>distributed transaction manager was written to run as an HBase
>Unlike most NOSQL and other SQL-on-Hadoop open source projects, Trafodion
>provides comprehensive ANSI SQL language support including full-functioned
>data definition (DDL), data manipulation (DML), transaction control (TCL)
>and database utility support.
>Trafodion provides comprehensive and standard SQL data manipulation
>including SELECT, INSERT, UPDATE, DELETE, and UPSERT/MERGE syntax with
>language options including join variants, unions, where predicates,
>aggregations (group by and having), sort ordering, sampling, correlated
>nested sub-queries, cursors, and many SQL functions.
>Utilities are provided for updating table statistics used by the optimizer
>for costing (i.e. selectivity/cardinality estimates) plan alternatives,
>displaying the chosen SQL execution plan, plan shaping, backup and
>restoring the database, data loading and unloading, and a command line
>utility for interfacing with the database engine.
>Explicit control statements are provided to allow applications to define
>transaction boundaries and to abort transactions when warranted, including
>Trafodion supports ANSI¹s grant/revoke semantics to define user and role
>privileges in terms of managing and accessing the database objects.
>The name ³Trafodion² (the Welsh word for transactions, pronounced
>³Tra-vod-eee-on²) was chosen specifically to emphasize the differentiation
>that Trafodion provides in closing a critical gap in the Hadoop ecosystem.
>Trafodion builds on the scalability, elasticity, and flexibility of
>Trafodion extends Hadoop to provide guaranteed transactional integrity,
>enabling new kinds of big data applications to run on Hadoop.
>Current Status
>HP released the Trafodion code under the Apache License, Version 2, in
>of 2014. Since that time, we have had one major release in January 2015
>one minor release in April 2015. The focus of these releases has been in
>getting our base functionality, including security, working on top of
>Apache HBase, as well as improving performance, availability and
>scalability, and integrating better with HBase.
>We want to build a diverse developer community, based on the Apache Way,
>around Trafodion. To help developers become contributors, we have
>documentation on the wiki about the architecture, the source tree
>structure, and an example enhancement. We plan to publish our project
>backlog to the community, specifically highlighting areas where developers
>new to Trafodion may best start contributing, such as extending the
>database functionality with User Defined Routines (UDRs) and integrating
>with other Apache projects in the Hadoop ecosystem.
>We have already begun building a community but at this time the community
>consists only of Trafodion developers ­ all HP employees ­ and prospective
>users. We have participated in and hosted HBase Meetups and intend to ramp
>up our community building efforts.
>The Trafodion project has seen interest in China, where HP has conducted
>proof-of-concepts with multiple companies and expects to see some of its
>first commercial deployments. To help recruit contributors and users in
>China, members of the team are translating Trafodion wiki content into
>Core Developers
>The core developers are very experienced in database and transaction
>monitor technology, with many having spent more than 20 years working in
>this space.
>Apache Trafodion relies on Apache HBase as its storage engine. The
>development team has collaborated with and gained valuable advice from
>working with the Apache HBase core developers. Apache Trafodion has
>federation capabilities as well, and can query Trafodion tables stored in
>HBase, native HBase tables, and Apache Hive tables.
>Known Risks
>Orphaned Products
>HP Labs and HP-IT have been incubating Trafodion development for almost
>years. This is part of HP¹s strategy to leverage its investment in
>software and bring software to market as open source and is similar to
>efforts with OpenStack. Trafodion builds on HP¹s equity investment in the
>Hadoop ecosystem and its efforts to monetize Hadoop through hardware,
>software, and services. HP wants Trafodion to be successful, as HP will
>offer a commercially supported distribution of Trafodion.
>Inexperience with Open Source
>We have been working with open source software in building closed source
>software for well over two decades. To help transition to doing open
>development, the development team received guidance and best practices
>HP developers working on OpenStack open source projects, many of whom have
>experience working on Apache and other open source projects as well. Since
>releasing Trafodion as an open source project in June of 2014, the
>committers and contributors have moved forward using open source
>development processes and tools for bug tracking and design blueprints and
>Jenkins for continuous integration. As part of the incubation process, we
>recognize we may need to change some of our development processes/tools
>conduct our discussions using Apache email dlists.
>Homogenous Developers
>Since the initial development of Trafodion has been supported by HP, all
>the current developers are HP employees. Through the support of the Apache
>incubation project, we aim to expand the list of developers and gain
>contributors from related SQL-on-Hadoop projects and the Apache HBase
>project. Trafodion developers are experienced with distributed development
>processes, being primarily based in Palo Alto, CA; Austin, TX; and
>Shanghai, China. Trafodion is written in C++ and Java.
>Reliance on Salaried Developers
>Currently all of the developers working on the project are paid by their
>employer to work on the project. These developers will work on the open
>source project as well as work on the commercially supported distribution
>of Trafodion that HP will offer.
>Relationship with Other Apache Products
>Trafodion is built upon Apache HBase and extends it to support ACID
>transactions with HBase co-processors for distributed transaction
>management and recovery. Trafodion envisions future collaborations with
>Apache HBase project on performance optimizations, such as in the areas of
>mixed workload support, High Availability, etc. It also provides
>transactional support and querying from native HBase tables as well.
>Trafodion uses Apache Zookeeper to coordinate and manage the distribution
>of connection services across the cluster for load-balancing and high
>availability reconnection purposes in the event a Trafodion process should
>Trafodion also envisions working with the Apache Ambari project on
>better Trafodion manageability. While Ambari focuses on system and
>component level performance metrics, Trafodion manageability will focus in
>a complimentary way on database workload monitoring and performance
>analytics with capabilities more geared towards database administrators.
>There are alternative open source projects that are providing
>capabilities, such as Apache Hive, Apache Drill, and Apache Phoenix. These
>are more focused on reporting and analytics across data structures
>supported on HDFS. In comparison to all of these technologies Trafodion
>provides a very complete implementation of ANSI SQL, one of the most
>sophisticated optimizers for such workloads, a completely parallel data
>flow architecture that does not materialize intermediate results unless
>necessary, full ACID transactional support, ANSI GRANT/REVOKE security,
>other capabilities that would take decades to build in these products. On
>the other hand currently Trafodion is just focused on HBase and querying
>Hive, whereas Hive and Drill provide access to other data formats in HDFS.
>An Excessive Fascination with the Apache Brand
>We understand the reputation and value of the Apache brand, and no doubt
>believe that it will help us attract contributors and users. Our primary
>goal is to follow a proven, open source development and community building
>model that will make Trafodion successful and enable better collaboration
>with other Apache projects in the Hadoop ecosystem. We also understand the
>rules and guidelines about the use of the Apache brand and intend to
>Documentation and technical details on Trafodion can be found at:
>Initial Source
>The source is available today in a public github repository:
>Source and Intellectual Property Submission Plan
>The source code has already been released under the Apache License,
>2. The manuals have been released in Adobe PDF format. As part of the
>submission process, the source for the manuals will be converted from a
>proprietary DocBook XML format to AsciiDoc.
>External Dependencies
>Two dependencies do not have Apache compatible licenses and will be
>addressed as we enter incubation. One dependency is log4cpp, which is
>licensed under the LGPL. A compatible alternative might be Apache
>project log4cxx. The other dependency is unixodbc, which is used as the
>ODBC driver manager. We will look into how Apache Hive manages being able
>to use this incompatible software and do similar. All other dependencies
>have Apache compatible licenses, including Apache 2.0, MIT/X11, MIT, and
>Trafodion does not contain any cryptographic code. It does call
>cryptographic libraries: OpenSSL for C++ code and Java Cryptography
>Extension (JCE) for Java code.
>Required Resources
>Mailing Lists
>Git Repository
>Issue Tracking
>JIRA: JIRA Trafodion (Trafodion)
>Initial Committers and Affiliation
>Dave Birdsall, Hewlett-Packard Company, Dave.Birdsall<AT>hp<DOT>com
>Matt Brown, Hewlett-Packard Company, mattbrown<AT>hp<DOT>com
>Tharak Capirala, Hewlett-Packard Company, Tharak.Capirala<AT>hp<DOT>com
>Alice Chen, Hewlett-Packard Company, Alice.Chen<AT>hp<DOT>com
>John DeRoo, Hewlett-Packard Company, John.Deroo<AT>hp<DOT>com
>Roberta Marton, Hewlett-Packard Company, Roberta.Marton<AT>hp<DOT>com
>Amanda Moran, Hewlett-Packard Company, Amanda.Kay.Moran<AT>hp<DOT>com
>Suresh Subbiah, Hewlett-Packard Company, Suresh.Subbiah<AT>hp<DOT>com
>Sandyha Sundaresan, Hewlett-Packard Company,
>Michael Stack, Stack<AT>apache<DOT>org
>Nominated Mentors
>Andrew Purtell apurtell<AT>apache<DOT>org
>Devaraj Das, ddas<AT>apache<DOT>or
>Enis Söztutar, Enis<AT>apache<DOT>org
>Lars Hofhansl, larsh<AT>apache<DOT>org
>Michael Stack, Stack<AT>apache<DOT>org
>Roman Shaposhnik, rshaposhnik<AT>pivotal<DOT>io
>Sponsoring Entity
>Apache Incubator PMC

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message