flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Flume HTTP sink for binary data
Date Mon, 27 Oct 2014 19:00:50 GMT
Hi,

Thans for input, Hari.

Jean-François - we are looking to do something either custom or as part of
Flume pretty soon (tomorrow), so there is anything you want to share that
we should work off of, please stick it in JIRA and we'll work on that
instead of something custom.  Sorry for the last minute email/push. :(

Thanks,
Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


On Mon, Oct 27, 2014 at 2:47 PM, <j.guilmard@accenture.com> wrote:

>  Hello Ari, Otis,
>
>
>
> Yes I could contribute on this: I’m currently developing it. I’ve followed
> the pattern described below, reusing existing Sink as a model.
>
>
>
> I’m working on generic options that appear to me as mandatory for a wider
> usage that mine only:
>
> -          Load balancing
>
> -          Authentication
>
> -          Caching of Http connexion instead of creating one per event.
>
> -          Adding HTTP headers from config.
>
>
>
> I will follow up on this.
>
>
>
> Regards
>
>
> *________________________________________________ **Jean-François
> Guilmard*
>
>
>
> *From:* Hari Shreedharan [mailto:hshreedharan@cloudera.com]
> *Sent:* lundi 27 octobre 2014 19:29
> *To:* user@flume.apache.org
> *Cc:* user@flume.apache.org
> *Subject:* Re: Flume HTTP sink for binary data
>
>
>
> You could build one using Apache HttpClient - just look at any other sink
> (like hbase sink or rolling file sink) and follow the same pattern.
>
>
> Thanks,
> Hari
>
>
>
> On Mon, Oct 27, 2014 at 9:29 AM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
>  Hi,
>
>
>
> Shouldn't something like that exist in Flume?
>
>
>
> Jean-François, is this something you could contribute?  Maybe we can use
> yours or work on it if you can put up a patch?  If not, I think we can take
> one of the existing implementations you mentioned and try to improve that
> and contribute that...... but if your stuff is better, somewhat ready, and
> contributable.....
>
>
>
> Thanks,
>
> Otis
> --
>
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
>
>
> On Mon, Oct 27, 2014 at 12:09 PM, <j.guilmard@accenture.com> wrote:
>
> Hi Pavel,
>
>
>
> I have found 2 initiatives on this.
>
>
>
>
> https://github.com/hammer/flume/blob/f140709502abf286d9e25a174caa24629a776448/plugins/http/src/java/com/cloudera/flume/handlers/http/HttpPostSink.java
>
>
> https://github.com/josealvarezmuguerza/flume-http-sink/blob/master/src/main/java/org/apache/flume/sink/HttpSink.java
>
>
>
> None of them appeared to me with the code quality/flexibility I expected,
> so I guess it might be worth going further?
>
>
>
> Good luck and let me know if you find other alternatives. (I’m currently
> developing my own code, based on Apache HttpClient )
>
>
>
> Best regards.
>
>
> *________________________________________________ **Jean-François
> Guilmard*
>
>
>
> *From:* Pavel Zalunin [mailto:wr4bbit@gmail.com]
> *Sent:* lundi 27 octobre 2014 16:44
> *To:* user@flume.apache.org
> *Subject:* Flume HTTP sink for binary data
>
>
>
> Hi,
>
>
>
> We need to send binary data to our http endpoint, I looked at built-in
> sinks ( https://flume.apache.org/FlumeUserGuide.html#flume-sinks) and
> can't find http sink. Are there opensourced implementations of such thing?
>
>
>
> Pavel.
>
>
>  ------------------------------
>
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
>
> ______________________________________________________________________________________
>
> www.accenture.com
>
>
>
>
>

Mime
View raw message