From flume-user-return-337-apmail-incubator-flume-user-archive=incubator.apache.org@incubator.apache.org Mon Oct 10 17:35:16 2011 Return-Path: X-Original-To: apmail-incubator-flume-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-flume-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1C5177EC7 for ; Mon, 10 Oct 2011 17:35:16 +0000 (UTC) Received: (qmail 51926 invoked by uid 500); 10 Oct 2011 17:35:16 -0000 Delivered-To: apmail-incubator-flume-user-archive@incubator.apache.org Received: (qmail 51899 invoked by uid 500); 10 Oct 2011 17:35:15 -0000 Mailing-List: contact flume-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: flume-user@incubator.apache.org Delivered-To: mailing list flume-user@incubator.apache.org Received: (qmail 51891 invoked by uid 99); 10 Oct 2011 17:35:15 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Oct 2011 17:35:15 +0000 X-ASF-Spam-Status: No, hits=-0.6 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mjlai09@gmail.com designates 209.85.210.41 as permitted sender) Received: from [209.85.210.41] (HELO mail-pz0-f41.google.com) (209.85.210.41) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Oct 2011 17:35:09 +0000 Received: by pzk5 with SMTP id 5so18191360pzk.0 for ; Mon, 10 Oct 2011 10:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=u2cX5B1yWkTGV+oFdgiL2H90xiZ8XSgJ84kmvPxmz8g=; b=Kz3A6c7y47XGbNfLFjn1EVTcah7JhQ1NUIhs6+3yegZ6WCEE8IuBALAbz01ORZJdc0 gaGWE1wEeTo0aZrioR90kvPiTe491pismjOH3OlyeN/j2Uuw1t/+5FStsGrbbK4g1zkt XydFlnjbYlD4eNAAEZoDiMJ11awUyRvvCeR7w= Received: by 10.68.55.100 with SMTP id r4mr38648964pbp.69.1318268087777; Mon, 10 Oct 2011 10:34:47 -0700 (PDT) Received: from [172.16.31.99] (69-170-60-202.static-ip.telepacific.net. [69.170.60.202]) by mx.google.com with ESMTPS id h5sm69756162pbq.11.2011.10.10.10.34.45 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 10 Oct 2011 10:34:46 -0700 (PDT) Message-ID: <4E932CB5.2070603@googlemail.com> Date: Mon, 10 Oct 2011 10:34:45 -0700 From: Mingjie Lai User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110424 Thunderbird/3.1.10 MIME-Version: 1.0 To: flume-user@incubator.apache.org Subject: Re: Flume-HBase References: <1317932302.56544.YahooMailNeo@web130121.mail.mud.yahoo.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Yes. You need to place the plugin jar to both master and nodes. Master does validate a plugin configure by loading the classes. That's my observation, and it really makes sense. -mingjie On 10/10/2011 01:03 AM, Alex Baranau wrote: > I believe you need jar to be placed on node with > * collector<- for sure! > * master<- I think that master performs some validation when you use > it to configure flume nodes, hence it should be aware of the new sink. > Not that sure though. > > Alex Baranau > ---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBase > > On Thu, Oct 6, 2011 at 11:23 PM, Saritha Ravi wrote: >> Hi Otis, >> Thanks for your reply, >> I am following setup instructions from the below link, do I have to update >> flume-site.xml and other configs in master,collector as well as agent >> machines. >> Thanks, >> Saritha. >> From: Otis Gospodnetic >> Reply-To: "flume-user@incubator.apache.org" >> , Otis Gospodnetic >> >> Date: Thu, 6 Oct 2011 13:18:22 -0700 >> To: "flume-user@incubator.apache.org" >> Subject: Re: Flume-HBase >> >> Saritha, >> Have a look >> at http://blog.sematext.com/2011/07/28/flume-and-hbase-integration/ >> Otis >> ---- >> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch >> Lucene ecosystem search :: http://search-lucene.com/ >> >> ________________________________ >> From: Saritha Ravi >> To: "flume-user@incubator.apache.org" >> Cc: Flume Development >> Sent: Wednesday, October 5, 2011 10:48 PM >> Subject: Flume-HBase >> >> Hi All, >> I need to configure Flume with hbase in cloud. Could anyone help me with >> this. Is there a better documentation. >> Does flume collector and and hbase master should be in the same cluster. I >> was able to configure hbase(master ,Zookeeper,region server using WHIRR) and >> flume(Master , collector) from CDH3. Can anyone suggest me the basic steps >> how I can configure these two in ec2 cloud. >> Thanks, >> Saritha. >> >> >> >> >