flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saritha Ravi <Saritha.R...@chacha.com>
Subject FW: Need help: Flume-HBase Sink
Date Wed, 12 Oct 2011 18:11:43 GMT
Hi All,

I am working on setting up flume with hbase in EC2. I have different clusters for hbase-master,
flume-node,flume-master,flume-collector. I can see the hbasesink in flume-master GUI(http://<masterIP>:35871/masterext.jsp)
as well as flume-collector GUI(http://<CollectorIP>/extension.jsp).


1. I am trying to follow  the doc at https://github.com/apache/flume/tree/flume-728/plugins/flume-plugin-hbasesink/docs

Since I am newbie for both, I think I didn't understand the steps  properly. Could anyone
please help me with this how I can execute these commands…
I can create table in hbase, I can able to connect to flume-master via flume shell
But I can't run

attr2hbase("t1","f1","f2:event","attr-prefix","10","false") command…

Could some provide me steps how I can execute this. (or example for "exec config node 'source'
'sink'")


In real time Do I need to write a java client/script to do the sink in hbase. If so, please
help me or give me suggetsion how I can do this…


One more thing I observed , When I see the logs in flume collector it has the following line
at the bottom of the file….(I am not sure whether it is acceptible error or not )
2011-10-11 17:53:52,463 INFO com.cloudera.flume.agent.MultiMasterRPC: No active master RPC
connection
2011-10-11 17:53:52,526 INFO com.cloudera.flume.agent.ThriftMasterRPC: Connected to master
at ip-10-96-239-171.ec2.internal:35872
2011-10-11 17:53:52,544 WARN com.cloudera.flume.conf.SourceFactoryImpl: No source builders
found in com.cloudera.flume.hbase.HBaseSink
2011-10-11 17:53:52,546 WARN com.cloudera.flume.conf.SourceFactoryImpl: No source builders
found in com.cloudera.flume.hbase.Attr2HBaseEventSink
2011-10-11 17:53:52,675 INFO com.cloudera.flume.conf.SinkFactoryImpl: Found sink builder hbase
in com.cloudera.flume.hbase.HBaseSink
2011-10-11 17:53:52,675 WARN com.cloudera.flume.conf.SinkFactoryImpl: No sink decorators found
in com.cloudera.flume.hbase.HBaseSink
2011-10-11 17:53:52,676 INFO com.cloudera.flume.conf.SinkFactoryImpl: Found sink builder attr2hbase
in com.cloudera.flume.hbase.Attr2HBaseEventSink
2011-10-11 17:53:52,676 WARN com.cloudera.flume.conf.SinkFactoryImpl: No sink decorators found
in com.cloudera.flume.hbase.Attr2HBaseEventSink
2011-10-11 17:53:52,709 INFO com.cloudera.flume.agent.LogicalNodeManager: creating new logical
node domU-12-31-39-16-C4-92.compute-1.internal
2011-10-11 17:53:52,866 INFO com.cloudera.flume.agent.FlumeNode: Hadoop Security enabled:
false

I am using https://github.com/abaranau/flume-plugin-hbasesink-compiled/raw/master/flume-plugin-hbasesink-0.9.5-SNAPSHOT-20110627.jar
.

Thanks for you time..

Thanks,
Saritha.



Mime
View raw message