flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jintao Guan <ilovexia...@gmail.com>
Subject Re: How to use Esper in Flume
Date Tue, 04 Jun 2013 18:49:28 GMT
Hi Shekhar,

Thank you for your email. The first solution is what I am looking for.
I really want to know the details of the custom sink.

1. The first thing I want to know is that how to install the custom sink ?
Compile it then put the ***.jar into $FLOME_HOME/lib ?
2. Is there any material about how to write custom sink ?

Thank you.

On Tue, Jun 4, 2013 at 2:41 PM, Shekhar Sharma <shekhar2581@gmail.com>wrote:

> You have two options :
> (1) You can write a custom flume sink for esper..Disadvantage would be you
> cant change the esper queries at run time
> (2) YOu can write esper service (Rest API) and the flume sink will send
> the java pojo to Esper service..The advantage is that you will have the
> ability to change the queries at run time..
> I have tried both the approaches..let me know if you would like to know
> more...
> The first approach is very simple, just need to extend the sink from flume
> Abstract sink and override the process method..
> In the constructor of the flume you need to initialze the esper engine, as
> well as esper statements and esper queries..
> Make sure your listeners are attached as well when constructing the
> object..
> While you receive the events in the process method, you need to convert
> the Flume SimpleEvent to POJO and call EsperEngine's sendevent() method...
> For the second method you can have your queries,listeners and statements
> defined in XML file and you parse them when the service starts and have a
> thread that looks for the change of this xml file..
> Hope that helps.
> Regards,
> Som Shekhar Sharma
> +91-8197243810
> On Tue, Jun 4, 2013 at 11:36 PM, Jintao Guan <ilovexiao77@gmail.com>wrote:
>> Hello guys,
>> I want to use Esper in Flume Agents, so that I can monitor all of the
>> messages and logs and trigger alerts when there are Error logs.
>>  Does it seem like a plugin ?  How do I install the custom interceptor ?
>> Does anyone have any idea ?

View raw message