flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Khanna <khannap...@yahoo.com>
Subject HBase and Flume
Date Thu, 02 Feb 2012 18:54:49 GMT
Hi,
Am using flume Flume 0.9.4-cdh3u2 and wanted to use the HBase Sink. 

Entered this as the sink: hbase("testtable", "%{nanos}", "colfam1", "q1", "%{body}")

and got the following in the master logs (now I know that I am missing some configuration
steps - please can anybody point me to where I can get the Hbase related plugin and how to
configure that?):

2012-02-02 12:41:36,916 [exec-thread] INFO master.CommandManager: Executing command: config
[POC_Hadoop_Client_1.ssodev-bo.mtv1.vrsn.com, tail("/app/log-file-gen/out/server.log"), hbase("testtable",
"%{nanos}", "colfam1", "q1", "%{body}")]
2012-02-02 12:41:36,917 [exec-thread] ERROR master.CommandManager: Unexpected exception during
cmdid:0 config [POC_Hadoop_Client_1.ssodev-bo.mtv1.vrsn.com, tail("/app/log-file-gen/out/server.log"),
hbase("testtable", "%{nanos}", "colfam1", "q1", "%{body}")] : Attempted to write an invalid
sink/source: Invalid sink: hbase( "testtable", "%{nanos}", "colfam1", "q1", "%{body}" )
java.lang.IllegalArgumentException: Attempted to write an invalid sink/source: Invalid sink:
hbase( "testtable", "%{nanos}", "colfam1", "q1", "%{body}" )
        at com.cloudera.flume.master.ConfigManager.setConfig(ConfigManager.java:96)
        at com.cloudera.flume.master.flows.FlowConfigManager.setConfig(FlowConfigManager.java:285)
        at com.cloudera.flume.master.TranslatingConfigurationManager.setConfig(TranslatingConfigurationManager.java:101)
        at com.cloudera.flume.master.ConfigCommand$1.exec(ConfigCommand.java:87)
        at com.cloudera.flume.master.CommandManager.exec(CommandManager.java:266)
        at com.cloudera.flume.master.CommandManager.handleCommand(CommandManager.java:205)
        at com.cloudera.flume.master.CommandManager$ExecThread.run(CommandManager.java:236)
Caused by: com.cloudera.flume.conf.FlumeIdException: Invalid sink: hbase( "testtable", "%{nanos}",
"colfam1", "q1", "%{body}" )
        at com.cloudera.flume.conf.FlumeBuilder.buildEventSink(FlumeBuilder.java:503)
        at com.cloudera.flume.conf.FlumeBuilder.buildEventSink(FlumeBuilder.java:531)
        at com.cloudera.flume.conf.FlumeBuilder.buildSink(FlumeBuilder.java:302)
        at com.cloudera.flume.master.ConfigManager.setConfig(ConfigManager.java:93)
Mime
View raw message