From flume-user-return-862-apmail-incubator-flume-user-archive=incubator.apache.org@incubator.apache.org Fri Mar 30 04:41:31 2012 Return-Path: X-Original-To: apmail-incubator-flume-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-flume-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2D7C099AC for ; Fri, 30 Mar 2012 04:41:31 +0000 (UTC) Received: (qmail 67604 invoked by uid 500); 30 Mar 2012 04:41:30 -0000 Delivered-To: apmail-incubator-flume-user-archive@incubator.apache.org Received: (qmail 67449 invoked by uid 500); 30 Mar 2012 04:41:27 -0000 Mailing-List: contact flume-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: flume-user@incubator.apache.org Delivered-To: mailing list flume-user@incubator.apache.org Received: (qmail 67428 invoked by uid 99); 30 Mar 2012 04:41:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Mar 2012 04:41:27 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of shekhar2581@gmail.com designates 209.85.160.175 as permitted sender) Received: from [209.85.160.175] (HELO mail-gy0-f175.google.com) (209.85.160.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 Mar 2012 04:41:19 +0000 Received: by ghbz2 with SMTP id z2so93321ghb.6 for ; Thu, 29 Mar 2012 21:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=5xSwTQNBIh7SE7eLMEzesLdE4/ubqzoI3Od0MUmRitA=; b=iQbdd+rJ5FIxZQyOwD+LCuumapWMlCKApcN3hzMP496z+wbLvrlrRix3DnKbKGq4vp RhzHlQk1enxN0aXglYVSUu4QONkKq5wLbIOrV8t6fyOxUMJoCo4S2tQnNmnJmYp+QZ2H NlphoYW6qQYeXYcPDz77+Sc36feQBkKWOqRlh2mBmELOqypB3WnFINjuQ3mlHdxgNwG6 NL57i2ihyXeY6Uk0NI6bLZdJ5e/f4rmHNZqco+WjufdRBKsFIzn17MLkXeWi6eoMOhhB 64qxhW2L1J2t1VYYOfDNHUVJsTUqIAgn6boF5YuGyu5IE2YCRumJN8WKS90p+iFpMPjS rltQ== MIME-Version: 1.0 Received: by 10.236.145.34 with SMTP id o22mr685234yhj.7.1333082458633; Thu, 29 Mar 2012 21:40:58 -0700 (PDT) Received: by 10.146.138.36 with HTTP; Thu, 29 Mar 2012 21:40:58 -0700 (PDT) In-Reply-To: References: Date: Fri, 30 Mar 2012 10:10:58 +0530 Message-ID: Subject: Re: Understanding the data flow between Source and Sink in Flume From: shekhar sharma To: flume-user@incubator.apache.org Content-Type: multipart/alternative; boundary=20cf303b3cdf6736ec04bc6e725d --20cf303b3cdf6736ec04bc6e725d Content-Type: text/plain; charset=ISO-8859-1 HI Arvind, Yeah i have done the same and able to integrate a custom sink with Flume. I think there has to be some extender facility in Flume, so that i don't have to make any code changes in Flume. i have tried the other way around i.e. copying the jar file in the lib directory, but it didn't work. Regards, Som Shekhar On Thu, Mar 29, 2012 at 10:04 PM, Arvind Prabhakar wrote: > Hi Som, > > Look at the implementation of NullSink to get an idea of how to code > up the sink. Once it is ready, make sure that you have it packaged in > the lib directory of the distribution and reference it by using the > fully qualified class name of the implementation in the type > attribute. > > Thanks, > Arvind > > On Thu, Mar 29, 2012 at 2:16 AM, shekhar sharma > wrote: > > Hi, > > I am using Flume-728 (Flume NG) and i am trying to build a custom sink > for > > Esper to apply EPL queries on the events generated. I have already custom > > sink, but it is not at all instantiating my sink class...I would like to > > understand how the flow actually happens from source to sink. > > > > Lets say my source is avro who is collecting events from an avro client > > (AvroCLI) and i am sending the events to custom sink, then wat all > things i > > need to do so that source is able to send the events to custom sink... > > I have done some study on the code, and following things are mandatory > > > > custom sink should be derived from Abstract Sink and optionally > implements > > Configurable. The class should override process() method which does the > > actual processing of events.. > > Can you please let me know what all things are required inorder to > recognize > > the custom sink by flume... > > > > Regards, > > Som > --20cf303b3cdf6736ec04bc6e725d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable HI Arvind,
Yeah i have done the same and able to integrate a custom sink= with Flume. I think there has to be some extender facility in Flume, so th= at i don't have to make any code changes in Flume. i have tried the oth= er way around i.e. copying the jar file in the lib directory, but it didn&#= 39;t work.


Regards,
Som Shekhar

On Thu, M= ar 29, 2012 at 10:04 PM, Arvind Prabhakar <arvind@apache.org> wrote:
Hi Som,

Look at the implementation of NullSink to get an idea of how to code
up the sink. Once it is ready, make sure that you have it packaged in
the lib directory of the distribution and reference it by using the
fully qualified class name of the implementation in =A0the type
attribute.

Thanks,
Arvind

On Thu, Mar 29, 2012 at 2:16 AM, shekhar sharma <shekhar2581@gmail.com> wrote:
> Hi,
> I am using Flume-728 (Flume NG) and i am trying to build a custom sink= for
> Esper to apply EPL queries on the events generated. I have already cus= tom
> sink, but it is not at all instantiating my sink class...I would like = to
> understand how the flow actually happens from source to sink.
>
> Lets say my source is avro who is collecting events from an avro clien= t
> (AvroCLI) and i am sending the events to custom sink, then wat all thi= ngs i
> need to do so that source is able to send the events to custom sink...=
> I have done some study on the code, and following things are mandatory=
>
> custom sink should be derived from Abstract Sink and optionally implem= ents
> Configurable. The class should override process() method which does th= e
> actual processing of events..
> Can you please let me know what all things are required inorder to rec= ognize
> the custom sink by flume...
>
> Regards,
> Som

--20cf303b3cdf6736ec04bc6e725d--