flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shekhar sharma <shekhar2...@gmail.com>
Subject Re: Understanding the data flow between Source and Sink in Flume
Date Fri, 30 Mar 2012 06:36:21 GMT
Thanks Will. that was very useful and interesting.

Regards,
Som

On Fri, Mar 30, 2012 at 11:19 AM, Will McQueen <will@cloudera.com> wrote:

> Hi Som,
>
> I noticed you mentioned you're using flume-728. That was the name of the
> original Flume NG branch, but Flume NG development has now moved to the
> trunk. Please see the Flume NG Getting Started guide for details:
>      https://cwiki.apache.org/FLUME/getting-started.html
>
> There's also some info about the architecture of NG if you're interested:
>      https://blogs.apache.org/flume/entry/flume_ng_architecture
>
> Cheers,
> Will
>
>
> On Thu, Mar 29, 2012 at 9:40 PM, shekhar sharma <shekhar2581@gmail.com>wrote:
>
>> HI Arvind,
>> Yeah i have done the same and able to integrate a custom sink with Flume.
>> I think there has to be some extender facility in Flume, so that i don't
>> have to make any code changes in Flume. i have tried the other way around
>> i.e. copying the jar file in the lib directory, but it didn't work.
>>
>>
>> Regards,
>> Som Shekhar
>>
>>
>> On Thu, Mar 29, 2012 at 10:04 PM, Arvind Prabhakar <arvind@apache.org>wrote:
>>
>>> Hi Som,
>>>
>>> Look at the implementation of NullSink to get an idea of how to code
>>> up the sink. Once it is ready, make sure that you have it packaged in
>>> the lib directory of the distribution and reference it by using the
>>> fully qualified class name of the implementation in  the type
>>> attribute.
>>>
>>> Thanks,
>>> Arvind
>>>
>>> On Thu, Mar 29, 2012 at 2:16 AM, shekhar sharma <shekhar2581@gmail.com>
>>> wrote:
>>> > Hi,
>>> > I am using Flume-728 (Flume NG) and i am trying to build a custom sink
>>> for
>>> > Esper to apply EPL queries on the events generated. I have already
>>> custom
>>> > sink, but it is not at all instantiating my sink class...I would like
>>> to
>>> > understand how the flow actually happens from source to sink.
>>> >
>>> > Lets say my source is avro who is collecting events from an avro client
>>> > (AvroCLI) and i am sending the events to custom sink, then wat all
>>> things i
>>> > need to do so that source is able to send the events to custom sink...
>>> > I have done some study on the code, and following things are mandatory
>>> >
>>> > custom sink should be derived from Abstract Sink and optionally
>>> implements
>>> > Configurable. The class should override process() method which does the
>>> > actual processing of events..
>>> > Can you please let me know what all things are required inorder to
>>> recognize
>>> > the custom sink by flume...
>>> >
>>> > Regards,
>>> > Som
>>>
>>
>>
>

Mime
View raw message