flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mingjie Lai <mjla...@gmail.com>
Subject Re: Flume-HBase
Date Thu, 06 Oct 2011 06:37:49 GMT
 > Does flume collector and and hbase master should be in the same cluster.

In your case, the flume collector will be writing data to hbase as a 
regular hbase client. So it needs to access hbase thru either, 1) hbase 
java api, or 2) hbase rest/thrift gateway. If you want to use the 
default hbase sink (which uses java api), the collector need to have 
direct connection to all region servers and master.

On the other hand, you can also build your own new hbase REST/thrift 
sink. And in this case, the collector only needs to talk to the REST 
gateway.

 > Can anyone suggest me
 > the basic steps how I can configure these two in ec2 cloud.

I don't quite understand your question. Sounds like you've already had 
hbase, then you can just have some extra machines for flume nodes, 
master, etc.

-mingjie

On 10/05/2011 07:48 PM, Saritha Ravi wrote:
> Hi All,
>
> I need to configure Flume with hbase in cloud. Could anyone help me with
> this. Is there a better documentation.
> Does flume collector and and hbase master should be in the same cluster.
> I was able to configure hbase(master ,Zookeeper,region server using
> WHIRR) and flume(Master , collector) from CDH3. Can anyone suggest me
> the basic steps how I can configure these two in ec2 cloud.
>
> Thanks,
> Saritha.
>
> *
> *

Mime
View raw message