incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luciano Resende <luckbr1...@gmail.com>
Subject Re: [DISCUSS] Project proposal - Apache Rainbow
Date Tue, 24 Mar 2020 04:12:31 GMT
Could you speak a little more about the current status of the project?

>From the, now available, repository, it seems that it is in very early stages?

https://github.com/Natural-Intelligence/rainbow/graphs/contributors


On Mon, Feb 24, 2020 at 3:45 AM Aviem Zur <aviemzur@apache.org> wrote:
>
> Hi all,
>
> Thanks for the feedback.
>
> 1. Indeed, the codebase is still under a private repository. We intend to
> have it ready to share publicly later this March.
> 2. The project is built in Python and Java.This is due to the fact that we
> have deep integrations with open source projects written in these languages.
> We also considered the fact that it is used by both data scientists and
> data engineers and we believe a combination of Python/Java will promote
> collaboration and contribution.
> 3. Rainbow project intends to facilitate and simplify the composition of
> complex pipelines, which are based on other open source projects.
> As such it does not compete or overlap but rather complement these projects.
> 4. Re: DLAB project - as we see it this project focuses in the research
> phase, while Rainbow's focus is in the production phase.
> Seems the 2 projects complement each other and it would be very interesting
> for us to collaborate with the DLAB team.
> 5. We will adjust the proposal to provide more details on how other Apache
> projects are used in Rainbow.
> We currently mainly use Apache Airflow in order to run pipelines defined by
> users in our APIs (YAML, with plans of UI/REST), this reduces the
> engineering requirements for transitioning data science code into
> production. We also leverage Apache Spark and Apache Hive for data
> preparation features and there are plans to integrate with Apache Karaf as
> well.
>
> Thanks,
> Aviem
>
> On Sat, Feb 22, 2020 at 4:29 AM Paul King <paulk@asert.com.au> wrote:
>
> > Indeed, it does sound interesting.
> >
> > I would find it useful if the "existing Apache projects" bit of "Rainbow is
> > in development, leveraging existing Apache projects." could be expanded in
> > any way. I know there is a list of external dependencies later but  any
> > further description of how those technologies are used would be helpful.
> >
> > Also, I'd be interested in knowing how the proposal relates to DLAB:
> > https://dlab.apache.org/
> >
> > Nice work.
> >
> > Cheers, Paul.
> >
> >
> >
> > On Sat, Feb 22, 2020 at 2:34 AM larry mccay <lmccay@apache.org> wrote:
> >
> > > This seems like an interesting proposal.
> > >
> > > Couple points/questions:
> > >
> > > * The existing source is not available for viewing as it is still in
> > > private repos?
> > > * Is it a primarily java project?
> > > * It seems the intent of Rainbow is to not compete or overlap with the
> > > Hadoop ecosystem projects but rather to provide an efficient interface
> > > above them - correct?
> > >
> > >
> > > On Fri, Feb 21, 2020 at 8:51 AM Aviem Zur <aviemzur@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > We would like to propose Rainbow as an Apache incubator project.
> > Rainbow
> > > is
> > > > an end-to-end platform for data engineers & scientists, allowing them
> > to
> > > > build, train and deploy machine learning models in a robust and agile
> > > way.
> > > > The project's goal is to operationalize the machine learning process,
> > > > allowing data scientists to quickly transition from a successful
> > > experiment
> > > > to an automated pipeline in production.
> > > >
> > > > The proposal can be found here:
> > > > https://cwiki.apache.org/confluence/display/INCUBATOR/Apache+Rainbow
> > > >
> > > > We would appreciate your feedback and thoughts on the proposal.
> > > >
> > > > Thanks,
> > > > Aviem
> > > >
> > >
> >



-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message