flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will McQueen <w...@cloudera.com>
Subject Re: Understanding the data flow between Source and Sink in Flume
Date Fri, 30 Mar 2012 05:49:22 GMT
Hi Som,

I noticed you mentioned you're using flume-728. That was the name of the
original Flume NG branch, but Flume NG development has now moved to the
trunk. Please see the Flume NG Getting Started guide for details:

There's also some info about the architecture of NG if you're interested:


On Thu, Mar 29, 2012 at 9:40 PM, shekhar sharma <shekhar2581@gmail.com>wrote:

> HI Arvind,
> Yeah i have done the same and able to integrate a custom sink with Flume.
> I think there has to be some extender facility in Flume, so that i don't
> have to make any code changes in Flume. i have tried the other way around
> i.e. copying the jar file in the lib directory, but it didn't work.
> Regards,
> Som Shekhar
> On Thu, Mar 29, 2012 at 10:04 PM, Arvind Prabhakar <arvind@apache.org>wrote:
>> Hi Som,
>> Look at the implementation of NullSink to get an idea of how to code
>> up the sink. Once it is ready, make sure that you have it packaged in
>> the lib directory of the distribution and reference it by using the
>> fully qualified class name of the implementation in  the type
>> attribute.
>> Thanks,
>> Arvind
>> On Thu, Mar 29, 2012 at 2:16 AM, shekhar sharma <shekhar2581@gmail.com>
>> wrote:
>> > Hi,
>> > I am using Flume-728 (Flume NG) and i am trying to build a custom sink
>> for
>> > Esper to apply EPL queries on the events generated. I have already
>> custom
>> > sink, but it is not at all instantiating my sink class...I would like to
>> > understand how the flow actually happens from source to sink.
>> >
>> > Lets say my source is avro who is collecting events from an avro client
>> > (AvroCLI) and i am sending the events to custom sink, then wat all
>> things i
>> > need to do so that source is able to send the events to custom sink...
>> > I have done some study on the code, and following things are mandatory
>> >
>> > custom sink should be derived from Abstract Sink and optionally
>> implements
>> > Configurable. The class should override process() method which does the
>> > actual processing of events..
>> > Can you please let me know what all things are required inorder to
>> recognize
>> > the custom sink by flume...
>> >
>> > Regards,
>> > Som

View raw message