flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saritha Ravi <Saritha.R...@chacha.com>
Subject Re: Flume-HBase
Date Thu, 06 Oct 2011 13:34:37 GMT
Hi Mingjie,
Good Morning...
Thanks  for your response.
I would like use default hbase sink I downloaded from
https://github.com/cloudera/flume/tree/master/plugins/(placed the
hbase-sink.jar in flume-home/lib and updated the flume-site.xml)
You mentioned the collector needs to have direct connection to all region
servers and master..
Could you help me how I can do that.

I already have hbase and flume master as well as flume collector all are
different machine I need to link all these together but don't know how...

Thanks,
Saritha.

On 10/6/11 2:37 AM, "Mingjie Lai" <mjlai09@gmail.com> wrote:

> > Does flume collector and and hbase master should be in the same
>cluster.
>
>In your case, the flume collector will be writing data to hbase as a
>regular hbase client. So it needs to access hbase thru either, 1) hbase
>java api, or 2) hbase rest/thrift gateway. If you want to use the
>default hbase sink (which uses java api), the collector need to have
>direct connection to all region servers and master.
>
>On the other hand, you can also build your own new hbase REST/thrift
>sink. And in this case, the collector only needs to talk to the REST
>gateway.
>
> > Can anyone suggest me
> > the basic steps how I can configure these two in ec2 cloud.
>
>I don't quite understand your question. Sounds like you've already had
>hbase, then you can just have some extra machines for flume nodes,
>master, etc.
>
>-mingjie
>
>On 10/05/2011 07:48 PM, Saritha Ravi wrote:
>> Hi All,
>>
>> I need to configure Flume with hbase in cloud. Could anyone help me with
>> this. Is there a better documentation.
>> Does flume collector and and hbase master should be in the same cluster.
>> I was able to configure hbase(master ,Zookeeper,region server using
>> WHIRR) and flume(Master , collector) from CDH3. Can anyone suggest me
>> the basic steps how I can configure these two in ec2 cloud.
>>
>> Thanks,
>> Saritha.
>>
>> *
>> *
>

Mime
View raw message