incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenneth Knowles <k...@apache.org>
Subject Re: Write Access to Incubator Wiki
Date Fri, 01 Nov 2019 03:55:12 GMT
FWIW, though I am mentoring other podlings, I don't seem to have the
ability to grant write permissions to others. (I double checked the
workflow for the Beam wiki, so I know what it looks like)

Kenn

On Thu, Oct 31, 2019 at 4:04 PM Dave Fisher <wave@apache.org> wrote:

> Hi -
>
> I’ll point out that your champion and mentors do have access and they
> could easily do that for you.
>
> Regards,
> Dave
>
> > On Nov 1, 2019, at 12:48 AM, Dominik Riemer <riemer@fzi.de> wrote:
> >
> > Thank you very much Kenn, that's awesome!
> > I'll add you as a mentor once we got access to the wiki, afterwards I'll
> also start a separate DISCUSS thread to discuss our proposal.
> >
> > Dominik
> >
> > -----Original Message-----
> > From: Kenneth Knowles <kenn@apache.org>
> > Sent: Thursday, October 31, 2019 1:49 AM
> > To: general <general@incubator.apache.org>
> > Subject: Re: Write Access to Incubator Wiki
> >
> > Cool!
> >
> > I see you have mentors lined up. I would love to also be involved as
> mentor.
> >
> > The list of initial committers seems a little small. Browsing the commit
> history it does seem to be dominated by just a couple of contributors. I
> see this is acknowledged (at least implicitly) in the proposal. I expect a
> principal challenge will be attracting diverse contributors, committers,
> and PMC to ensure this project survives independent of any particular
> entity (individual or corporate).
> >
> > Kenn
> >
> > On Wed, Oct 30, 2019 at 1:16 PM Dominik Riemer <riemer@fzi.de> wrote:
> >
> >> Hi all,
> >>
> >>
> >>
> >> a short introduction to myself: My name is Dominik Riemer and I'm a
> >> co-initiator of StreamPipes, an open source self-service toolbox for
> >> analyzing IoT data streams. After presenting the tool at this year's
> >> ApacheCon NA, followed by very friendly and fruitful discussions with
> >> many people from the Apache community, we are sure that we'd like
> >> continue the development of StreamPipes as an Apache community
> >> project. Before we start a discussion process, I'd like to ask to get
> >> write access to the Incubator wiki (username: riemer). An initial draft
> of the proposal is attached below.
> >>
> >>
> >>
> >> Thanks for your help!
> >>
> >> Dominik
> >>
> >>
> >>
> >>
> >>
> >> --------------------------------
> >>
> >>
> >>
> >> StreamPipes - Apache Incubator Proposal, Proposal Draft
> >>
> >>
> >>
> >> == Abstract ==
> >>
> >> StreamPipes is a self-service (Industrial) IoT toolbox to enable
> >> non-technical users to connect, analyze and explore (Industrial) IoT
> >> data streams.
> >>
> >>
> >>
> >> = Proposal =
> >>
> >>
> >>
> >> The goal of StreamPipes
> >> (www.streampipes.org<http://www.streampipes.org>)
> >> is to provide an easy-to-use toolbox for non-technical users, e.g.,
> >> domain experts, to exploit data streams coming from (Industrial) IoT
> >> devices. Such users are provided with an intuitive graphical user
> >> interface with the Pipeline Editor at its core. Users are able to
> >> graphically model processing pipelines based on data sources (streams),
> data processors and data sinks.
> >> Data processors and sinks are self-contained microservices, which
> >> implement either stateful or stateless processing logic (e.g., a trend
> >> detection or image classifier). Their processing logic is implemented
> >> using one of several provided wrappers (we currently have wrappers for
> >> standalone/Edge-based processing, Apache Flink, Siddhi and working
> >> wrapper prototypes for Apache Kafka Streams and Spark, in the future
> >> we also plan to integrate with Apache Beam). An SDK allows to easily
> >> create new pipeline elements. Pipeline elements can be installed at
> >> runtime. To support users in creating pipelines, an underlying
> >> semantics-based data model enables pipeline elements to express
> >> requirements on incoming data streams that need to be fulfilled, thus
> reducing modeling errors.
> >>
> >> Data streams are integrated by using StreamPipes Connect, which allows
> >> to connect data sources (based on standard protocols, such as MQTT,
> >> Kafka, Pulsar, OPC-UA and further PLC4X-supported protocols) without
> >> further programming using a graphical wizard. Additional user-faced
> >> modules of StreamPipes are a Live dashboard to quickly explore IoT
> >> data streams and a wizard that generates code templates for new
> >> pipeline elements, a Pipeline Element Installer used to extend the
> algorithm feature set at runtime.
> >>
> >>
> >>
> >> === Background ===
> >>
> >> StreamPipes was started in 2014 by researchers from FZI Research
> >> Center for Information Technology in Karlsruhe, Germany. The original
> >> prototype was funded by an EU project centered around predictive
> >> analytics for the manufacturing domain. Since then, StreamPipes was
> >> constantly improved and extended by public funding mainly from federal
> >> German ministries. In early 2018, the source code was officially
> released under the Apache License 2.0.
> >> At the same time, while we focused on bringing the research prototype
> >> to a production-grade tool, the first companies started to use
> StreamPipes.
> >> Currently, the primary goal is to widen the user and developer base.
> >> At ApacheCon NA 2019, after having talked to many people from the
> >> Apache Community, we finally decided that we would like to bring
> >> StreamPipes to the Apache Incubator.
> >>
> >>
> >>
> >> === Rationale ===
> >>
> >> The (Industrial) IoT domain is a highly relevant and emerging sector.
> >> Currently, IoT platforms are offered by many vendors ranging from SMEs
> >> up to large enterprises. We believe that open source alternatives are
> >> an important cornerstone for manufacturing companies to easily adopt
> >> data-driven decision making. From our point of view, StreamPipes fits
> >> very well into the existing (I)IoT ecosystem within the ASF, with
> >> projects such as Apache PLC4X focusing on connecting machine data from
> >> PLCs, or other tools we are also using either in the core of
> >> StreamPipes or with integrations (Apache Kafka, Apache IoTDB, Apache
> >> Pulsar). StreamPipes itself focuses on enabling self-service IoT data
> >> analytics for non-technical users.
> >>
> >> The whole StreamPipes code is currently on Github. To get a rough
> >> estimate of the project size:
> >>
> >> * streampipes: Backend and core modules, ~3300 commits
> >>
> >> * streampipes-ui: User Interface, ~1300 commits
> >>
> >> * streampipes-pipeline-elements: ~100 Pipeline Elements (data
> >> processors/algorithms and sinks), ~500 Commits
> >>
> >> * streampipes-connect-adapters: ~20 Adapters to connect data, ~100
> >> commits To achieve our goal to further extend the code base with new
> >> features, new connectors and new algorithms and to grow both the user
> >> and developer community, we believe that a community-driven
> >> development process is the best way to further develop StreamPipes.
> >> Finally, after having talked to committers from various Apache
> >> IoT-related projects and participation in spontaneous hacking sessions
> >> and being impressed by the collaboration among individual projects, we
> >> decided that (from our point of view) the ASF is the ideal place to be
> the future home of StreamPipes.
> >>
> >>
> >>
> >> === Initial Goals ===
> >>
> >> * Move the existing codebase to Apache
> >>
> >> * Fully align with Apache development- and release processes
> >>
> >> * Perform name search and do a thorough review of existing licenses
> >>
> >> * First Apache release
> >>
> >>
> >>
> >> === Current Status ===
> >>
> >> ** Meritocracy **
> >>
> >> We are absolutely committed to strengthen StreamPipes as a real
> >> community-driven open source project. The existing committer base is
> >> highly motivated to foster the open source way in the industrial IoT
> >> sector and, together with existing Apache communities focused on this
> >> domain, provide open source tooling for Industrial IoT projects in the
> >> same way Apache offers in the Big Data space, for instance.
> >>
> >> The development philosophy behind StreamPipes has always followed the
> >> principles of meritocracy - although most committers are still active
> >> in the project, we managed to onboard new, committed developers
> >> regularly. 2 people, who are today core of the developer team, have
> >> joined during the past year. Therefore, we would aim to continuously
> >> expand the PMC and committer base based on merit.
> >>
> >>
> >>
> >> ** Community **
> >>
> >> Since being open-sourced in 2018, the public interest in StreamPipes
> >> has steadily grown. Several companies, mainly from the manufacturing
> >> domain, have tested StreamPipes in form of proof-of-concept projects.
> >> First companies have started to use StreamPipes in production. This
> >> was due to a high number of events from meetups, research conferences,
> >> demo sessions up to hackathons we participated or organized during the
> >> past two years. After having generated a general interest in
> >> StreamPipes, our next focus will be to find more committers to
> diversify the contributor base.
> >>
> >>
> >>
> >> ** Core Developers **
> >>
> >> The core developers of the system are Dominik Riemer, Philipp Zehnder,
> >> Patrick Wiener and Johannes Tex. All core developers are initial
> >> committers in the current proposal. Some former students who recently
> >> started to work at companies and who have also worked on the project
> >> with great commitment, will be asked to further contribute to the
> project.
> >>
> >>
> >>
> >> ** Alignment **
> >>
> >> StreamPipes has dependencies to a lot of existing Apache projects -
> >> this is one reason why we think that the ASF is the best future home
> >> for StreamPipes. The messaging layer is based on Apache Kafka (and
> >> also Apache Pulsar as a future option), and runtime wrappers exist for
> >> Apache Flink, Apache Spark and Apache Kafka Streams. StreamPipes
> >> Connect already includes adapters for several Apache projects. Most
> >> importantly, we integrate (and plan to deepen the integration) with
> >> IIoT-focused projects such as Apache PLC4X. Also, several data sinks
> >> exist to send messages to tools from other Apache projects (e.g.,
> Apache Kafka, Apache Pulsar, and Apache IoTDB).
> >> Together with these tools (and also after having talked to the core
> >> developers after this year's ApacheCon) we are absolutely convinced
> >> that a tight integration between these tools will strengthen the open
> >> source IoT ecosystem.
> >>
> >>
> >>
> >> === Known Risks ===
> >>
> >> ** Orphaned Products **
> >>
> >> We don't expect the risk of an orphaned product. The initial
> >> committers have worked on the project for years and are absolutely
> >> committed to making this open source tool a great success. All initial
> >> committers are committed to work on StreamPipes in their free time.
> >>
> >>
> >>
> >> ** Inexperience with Open Source **
> >>
> >> All initial committers have years of expertise related to open source
> >> development and understand what open source means. However, none of
> >> the initial committers are currently committers to Apache projects,
> >> although some have already contributed to some projects. From a
> >> variety of events and from intensively studying Apache mailing lists,
> >> we are sure that the Apache Way is the way we'd like the project to
> >> move into the future. We expect to benefit from the experiences from
> >> the ASF in building successful open source projects.
> >>
> >>
> >>
> >> ** Length of Incubation **
> >>
> >> We are aware that incubation is a process that is focused on building
> >> the community, learning the Apache Way and other important things such
> >> as learning the release process and handling licensing and trademark
> issues.
> >> We are also aware that, although there is a steadily increasing
> >> interest in StreamPipes, a major challenge we would need (and are
> >> willing) to work on during the incubation phase is widening the
> >> committer base. We look forward to that as a large developer base is
> exactly what we are striving for.
> >>
> >>
> >>
> >> ** Homogeneous Developers **
> >>
> >> Most current developers work for the same institution (FZI). The
> >> motivation of all developers goes beyond their commitment to work and
> >> all current committers work on StreamPipes in their free time.
> >> Recently, we have received first pull requests from external
> >> contributors and a growing interest from users and companies outside
> >> of FZI. First manufacturing companies have already evaluated and
> >> adopted StreamPipes. To attract external developers, we've created an
> >> extensive documentation, have a Slack channel to quickly answer
> >> questions, and provide help via mail. Therefore, we believe that
> >> making the developer community more heterogeneous is not only
> mandatory, but something that can be achieved during the next months.
> >>
> >>
> >>
> >> ** Reliance on salaried developers **
> >>
> >> Currently, StreamPipes receives support from salaried developers,
> >> mainly research scientists from FZI. However, all core developers
> >> substantially work on StreamPipes in their spare time. As this has
> >> been the case from the beginning in early 2014, it can be expected
> >> that a substantial portion of volunteers will continue to be working
> >> on the project and we aim at strengthening the base of non-paid
> >> committers and paid committers of other companies. At the same time,
> >> funding of the initial StreamPipes team is secured by public funding
> >> for the next few years, making sure that there will be also enough
> commitment from developers during their work time.
> >>
> >>
> >>
> >> ** Relationships with other Apache products ** StreamPipes is often
> >> compared to tools such as Node-Red and Apache Nifi. This is mainly
> >> based on a similar UI concept (dataflow approach). Despite some
> >> technological differences (e.g., the microservice analytics approach
> >> vs. single-host runtime of Node-Red, the wrapper architecture and the
> >> underlying semantics-based model), we believe the target audience
> >> differs. We aim to collaborate with the Apache Nifi community in terms
> >> of exchanging best practices and also integrating both projects (e.g.,
> by building connectors).
> >>
> >> As mentioned above, quite a few adapters and data sinks are already
> >> available that link to existing Apache projects.
> >>
> >>
> >>
> >> ** An excessive fascination with the Apache Brand ** Although we
> >> recognize the Apache brand as the most visible brand in the open
> >> source domain, the primary goal of this proposal is not to create
> >> publicity, but to widen the developer base. We believe that successful
> >> projects have broad and diverse communities. We expect that an Apache
> >> project, with a clear and proven way to develop open source software,
> >> helps in finding new committers. As the core development team has
> >> already worked on StreamPipes for the past few years and is fully
> >> committed to the software and its benefit for the industrial IoT
> >> domain, we would also continue development without being an Apache
> project.
> >>
> >>
> >>
> >> === Documentation ===
> >>
> >> Currently, we host a website at https://www.streampipes.org More
> >> technical info (user + developer guide) can be found in the
> documentation:
> >> https://docs.streampipes.org, where users can find tutorials and
> >> manuals on how to extend StreamPipes using the SDK.
> >>
> >>
> >>
> >> === Initial Source ===
> >>
> >> Currently, the following Github repositories exist, all licensed under
> >> the Apache Software License 2.0:
> >>
> >> * streampipes (https://www.github.com /streampipes/streampipes, the
> >> backend & pipeline management module)
> >>
> >> * streampipes-ui (https://www.github.com/streampipes/streampipes-ui,
> >> the UI module)
> >>
> >> * streampipes-pipeline-elements (
> >> https://www.github.com/streampipes/streampipes-pipeline-elements,
> >> library of data processors and sinks)
> >>
> >> * streampipes-connect-adapters (
> >> https://www.github.com/streampipes/streampipes-connect-adapters,
> >> StreamPipes connect adapters)
> >>
> >> * streampipes-docs
> >> (https://www.github.com/streampipes/streampipes-docs,
> >> the abovementioned documentation)
> >>
> >>
> >>
> >> === Source and intellectual property submission plan === All initial
> >> committers will sign a ICLA with the ASF. FZI, as the organizational
> >> body that has employed the main contributors of StreamPipes, will sign
> >> a CCLA and donate the codebase to the ASF (both subject to formal
> >> approval). All major contributors are still active in the project.
> >>
> >>
> >>
> >> === External Dependencies ===
> >>
> >> We did an initial review of all dependencies used in the various
> projects.
> >> No critical libraries that depend on category X licenses were found,
> >> some minor issues have already been resolved (e.g., removing
> >> dependencies to org.json libraries). Most external dependencies used
> >> by the Java-based (backend, pipeline-elements and connect) modules are
> >> licensed under the Apache License 2.0, whereas some licenses are Cat B
> >> (e.g., CDDL). Most external dependencies the UI requires on are
> licensed under the MIT license.
> >>
> >> Once we are moving to the Incubator, we would do a complete check of
> >> all transitive dependencies. We don't expect any surprises here.
> >>
> >>
> >>
> >> === Cryptography ===
> >>
> >> (not applicable)
> >>
> >>
> >>
> >> === Required Resources ===
> >>
> >> ** Mailing Lists **
> >>
> >> We plan to use the following mailing lists:
> >>
> >> * users@streampipes.incubator.apache.org<mailto:
> >> users@streampipes.incubator.apache.org>
> >>
> >> * dev@streampipes.incubator.apache.org<mailto:
> >> dev@streampipes.incubator.apache.org>
> >>
> >> * private@streampipes.incubator.apache.org<mailto:
> >> private@streampipes.incubator.apache.org>
> >>
> >> * commits@streampipes.incubator.apache.org<mailto:
> >> commits@streampipes.incubator.apache.org>
> >>
> >> As StreamPipes is targeted to a non-technical audience, we see a
> >> dedicated user mailing list as an important requirement to help users.
> >>
> >>
> >>
> >> ** Subversion directory **
> >>
> >> (not applicable)
> >>
> >>
> >>
> >> ** Git repositories **
> >>
> >> We would like to use Git for source code management and enable Github
> >> mirroring functionality.
> >>
> >>
> >>
> >> As we plan to merge some of the repos described above to simplify the
> >> release process we ask to create the following source repositories:
> >>
> >> * streampipes (containing backend + UI)
> >>
> >> * streampipes-extensions (containing modules that can be dynamically
> >> installed at runtime: pipeline elements and connect adapters)
> >>
> >> * streampipes-website (containing docs + website)
> >>
> >>
> >>
> >> ** Issue tracking **
> >>
> >> JIRA ID: StreamPipes
> >>
> >>
> >>
> >> === Initial Committers ===
> >>
> >> List of initial committers in alphabetical order:
> >>
> >> Christofer Dutz (christofer.dutz at c-ware dot de) Dominik Riemer
> >> (dominik dot riemer at gmail dot com) Johannes Tex (tex at fzi dot de)
> >> Patrick Wiener (wiener at fzi dot de) Philipp Zehnder (zehnder at fzi
> >> dot de)
> >>
> >>
> >>
> >> === Sponsors ===
> >>
> >> ** Champion **
> >>
> >> * Christofer Dutz (christofer.dutz at c-ware dot de)
> >>
> >>
> >>
> >> ** Mentors **
> >>
> >> * Christofer Dutz (christofer.dutz at c-ware dot de)
> >>
> >> * Julian Feinauer (Jfeinauer at apache dot org)
> >>
> >> * Justin Mclean (justin at classsoftware dot com)
> >>
> >>
> >>
> >> ** Sponsoring Entity **
> >>
> >> The Apache Incubator
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: general-help@incubator.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message