flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <j.guilm...@accenture.com>
Subject RE: Flume HTTP sink for binary data
Date Fri, 28 Nov 2014 22:35:57 GMT
Hello all,

I’ve submitted a patch to the review board for the below feature.

https://issues.apache.org/jira/browse/FLUME-2524

(Review link is attached in the JIRA)

I’ve worked a lot on performances, which was not easy considering HTTPClient complexity.

Regards

From: j.guilmard@accenture.com [mailto:j.guilmard@accenture.com]
Sent: lundi 3 novembre 2014 11:59
To: user@flume.apache.org
Subject: RE: Flume HTTP sink for binary data

Hello,

I’ve proposed a patch in version v0, to be commented. I’m still working on improvements,
but it could be used as-is I think.

Link of the JIRA and patch: https://issues.apache.org/jira/browse/FLUME-2524

Your comments are more than welcome.
________________________________________________
Jean-François Guilmard

From: Behram Khan [mailto:behramk@gmail.com]
Sent: lundi 3 novembre 2014 11:25
To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Re: Flume HTTP sink for binary data

Hi
Can you please update. Can you please share the link to this project.
Regards


On Tue, Oct 28, 2014 at 1:36 PM, <j.guilmard@accenture.com<mailto:j.guilmard@accenture.com>>
wrote:
I’ve created a new JIRA, and will propose a patch. Due to your time constraints, I’m not
sure I will be able to commit a 100% final version. It will be more a good base of work, subject
to comments, optimizations and propositions ☺

Regards

Jeff

From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com<mailto:otis.gospodnetic@gmail.com>]
Sent: mardi 28 octobre 2014 12:47

To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Re: Flume HTTP sink for binary data

Hi Jean-François,

Great to hear!
There is no existing JIRA issu for this that I'm aware of.  I did see Hari created a new JIRA
issue yesterday about some HTTP Sink having some sort of issues with SSL, but I wasn't aware
of Flume containing any HTTP Sinks in the first place, so I'm not sure what that issue is
really about.
For example, I don't see any HTTP Sink here: https://git-wip-us.apache.org/repos/asf?p=flume.git;a=tree;f=flume-ng-sinks;h=ea7df0714f788e92ddbb403b762623832228a0fe;hb=trunk

Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


On Tue, Oct 28, 2014 at 7:31 AM, <j.guilmard@accenture.com<mailto:j.guilmard@accenture.com>>
wrote:
Otis,

I’m about to propose something (today). I’m waiting for a formal authorization from my
client: I’ve received an informal GO.

Is there any JIRA existing already ?

Best regards

From: Otis Gospodnetic [mailto:otis.gospodnetic@gmail.com<mailto:otis.gospodnetic@gmail.com>]
Sent: lundi 27 octobre 2014 17:28
To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Re: Flume HTTP sink for binary data

Hi,

Shouldn't something like that exist in Flume?

Jean-François, is this something you could contribute?  Maybe we can use yours or work on
it if you can put up a patch?  If not, I think we can take one of the existing implementations
you mentioned and try to improve that and contribute that...... but if your stuff is better,
somewhat ready, and contributable.....

Thanks,
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Oct 27, 2014 at 12:09 PM, <j.guilmard@accenture.com<mailto:j.guilmard@accenture.com>>
wrote:
Hi Pavel,

I have found 2 initiatives on this.

https://github.com/hammer/flume/blob/f140709502abf286d9e25a174caa24629a776448/plugins/http/src/java/com/cloudera/flume/handlers/http/HttpPostSink.java
https://github.com/josealvarezmuguerza/flume-http-sink/blob/master/src/main/java/org/apache/flume/sink/HttpSink.java

None of them appeared to me with the code quality/flexibility I expected, so I guess it might
be worth going further?

Good luck and let me know if you find other alternatives. (I’m currently developing my own
code, based on Apache HttpClient )

Best regards.
________________________________________________
Jean-François Guilmard


From: Pavel Zalunin [mailto:wr4bbit@gmail.com<mailto:wr4bbit@gmail.com>]
Sent: lundi 27 octobre 2014 16:44
To: user@flume.apache.org<mailto:user@flume.apache.org>
Subject: Flume HTTP sink for binary data

Hi,

We need to send binary data to our http endpoint, I looked at built-in sinks ( https://flume.apache.org/FlumeUserGuide.html#flume-sinks)
and can't find http sink. Are there opensourced implementations of such thing?

Pavel.

________________________________

This message is for the designated recipient only and may contain privileged, proprietary,
or otherwise confidential information. If you have received it in error, please notify the
sender immediately and delete the original. Any other use of the e-mail by you is prohibited.
Where allowed by local law, electronic communications with Accenture and its affiliates, including
e-mail and instant messaging (including content), may be scanned by our systems for the purposes
of information security and assessment of internal compliance with Accenture policy.
______________________________________________________________________________________

www.accenture.com<http://www.accenture.com>



Mime
View raw message