flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jintao Guan <ilovexia...@gmail.com>
Subject Re: How to use Esper in Flume
Date Wed, 05 Jun 2013 15:02:27 GMT
Thank you guys.
Now I have written my custom sink of using Esper in Flume.


On Tue, Jun 4, 2013 at 2:49 PM, Jintao Guan <ilovexiao77@gmail.com> wrote:

> Hi Shekhar,
>
> Thank you for your email. The first solution is what I am looking for.
> I really want to know the details of the custom sink.
>
> 1. The first thing I want to know is that how to install the custom sink ?
> Compile it then put the ***.jar into $FLOME_HOME/lib ?
> 2. Is there any material about how to write custom sink ?
>
> Thank you.
>
>
> On Tue, Jun 4, 2013 at 2:41 PM, Shekhar Sharma <shekhar2581@gmail.com>wrote:
>
>> You have two options :
>>
>> (1) You can write a custom flume sink for esper..Disadvantage would be
>> you cant change the esper queries at run time
>>
>> (2) YOu can write esper service (Rest API) and the flume sink will send
>> the java pojo to Esper service..The advantage is that you will have the
>> ability to change the queries at run time..
>>
>> I have tried both the approaches..let me know if you would like to know
>> more...
>> The first approach is very simple, just need to extend the sink from
>> flume Abstract sink and override the process method..
>> In the constructor of the flume you need to initialze the esper engine,
>> as well as esper statements and esper queries..
>> Make sure your listeners are attached as well when constructing the
>> object..
>> While you receive the events in the process method, you need to convert
>> the Flume SimpleEvent to POJO and call EsperEngine's sendevent() method...
>>
>> For the second method you can have your queries,listeners and statements
>> defined in XML file and you parse them when the service starts and have a
>> thread that looks for the change of this xml file..
>>
>> Hope that helps.
>> Regards,
>> Som Shekhar Sharma
>> +91-8197243810
>>
>>
>> On Tue, Jun 4, 2013 at 11:36 PM, Jintao Guan <ilovexiao77@gmail.com>wrote:
>>
>>> Hello guys,
>>>
>>> I want to use Esper in Flume Agents, so that I can monitor all of the
>>> messages and logs and trigger alerts when there are Error logs.
>>>
>>>  Does it seem like a plugin ?  How do I install the custom interceptor ?
>>>
>>> Does anyone have any idea ?
>>>
>>
>>
>

Mime
View raw message