flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Meyer, Dennis" <dennis.me...@adtech.com>
Subject Re: Integrating Flume a Esper
Date Wed, 28 Mar 2012 12:02:27 GMT
Hi Som,

I'd be interested also in connection flume against Esper. Would you mind sharing your code
if it's meant to be open source?


Von: shekhar sharma <shekhar2581@gmail.com<mailto:shekhar2581@gmail.com>>
Antworten an: <flume-user@incubator.apache.org<mailto:flume-user@incubator.apache.org>>
Datum: Wed, 28 Mar 2012 15:45:35 +0530
An: <flume-user@incubator.apache.org<mailto:flume-user@incubator.apache.org>>
Betreff: Re: Integrating Flume a Esper

i have written a custom sink for Esper, can you please how would i make FLume to recognize
this is my new sink..Do i have to specify in configuration file. Can you please help me?


On Wed, Mar 28, 2012 at 1:56 PM, shekhar sharma <shekhar2581@gmail.com<mailto:shekhar2581@gmail.com>>
Thanks for a quick response..
But the sinks written for cloudera flume (i was going through hello world plugin) use cloudera
classes and all.. For FLume-728 the sinks  written (HDFSSink and IRCSink) is different from
 CLoudera flume requires cloudera's hadoop to be installed, if i am not wrong?

My idea is to collect the events from Flume agent and send to Esper, where i will query the
streams using EPL.
Can you please tell me what all things are required to write a custom sink. i mean what needs
to be done so that Flume node recognizes the sink.



On Wed, Mar 28, 2012 at 1:06 PM, alo alt <wget.null@googlemail.com<mailto:wget.null@googlemail.com>>

Cloudera uses in 3u3 Flume-798

Apache flume isn't different, you can build it among your installation and tweak the configs
to match them or use the binary distribution:

As you read, apache flume 1.1.0 was just released:

You have to write plugins for sinks, here you found some created from the community:

And the docs:

Q3 - I don't know, if you use serialized data whats with avro?

- Alex

Alexander Lorenz

On Mar 28, 2012, at 5:49 AM, shekhar sharma wrote:

> Hello every one,
> I am trying to integrate FLume and Esper by writing a custom sink for Esper. But as per
the documentation , i am unable to do so. i have tried the hello world plugin, but that one
also not working.
> I am using FLume-728.
> I have few question:
> Q1. Is Apache Flume is different from Cloudera's Flume?
>        If i am using Cloudera's FLume, then does it have any dependency that i have to
use CLoudera's distribution of Hadoop.
> Q2. What are the necessary steps for writing custom sink ?
> Q3.  I have implemented a POC in Esper, and it takes POJO (Plain old java objects), XML
or Map as events. Now if i try integrating both, how feasible it is?
> Regards,
> Som Shekhar Sharma

View raw message