flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kumar, Deepak8 " <deepak8.ku...@citi.com>
Subject RE: Moving from Flume OG to Flume NG
Date Fri, 05 Oct 2012 15:28:59 GMT
Brock,

Apologize miss spelled your name last time. Hbase.jar as well as hbase-site.xml is there in
/opt/sponge/fluming/flume/lib directory. Could you please guide me, how to provide hbase command
while starting flume?

Regards,

Deepak



-----Original Message-----
From: Brock Noland [mailto:brock@cloudera.com]
Sent: Friday, October 05, 2012 11:17 AM
To: user@flume.apache.org
Cc: adams@cloudera.com
Subject: Re: Moving from Flume OG to Flume NG



Does either of



'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'



contain the hbase jars? Generally you want the hbase command to be in

the path of the user starting flume. In the hbase conf folder would be

the hbase-site.xml which would get picked up by the flume agent.



Brock



On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:

> Broke, this is the command it prints.

> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf
-name hbase-agent

> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name
hbase-agent

>

> Regards,

> Deepak

>

> -----Original Message-----

> From: Brock Noland [mailto:brock@cloudera.com]

> Sent: Friday, October 05, 2012 10:35 AM

> To: user@flume.apache.org

> Cc: adams@cloudera.com

> Subject: Re: Moving from Flume OG to Flume NG

>

> Hi,

>

> When start the flume agent it prints out the command it is running to

> start the agent. Can you send that along?

>

> Brock

>

> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:

>> Hi Adam,

>>

>> I am just trying to use hbase sink with flume. It is a sort of Hello World.

>> My configuration file looks something like this. When I execute the node for

>> hbase-agent, the log shows following error:

>>

>>

>>

>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test

>> from HBase

>>

>>

>>

>> It seems, it looks for hbase at localhost, but the hbase is running on

>> another VM. How should I provide the hbase VM details here? For testing

>> purpose, I have created a hbase-site.xml at location flume/lib/ but it

>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is

>> 1.2.

>>

>>

>>

>>

>>

>> Configuration File:

>>

>>

>>

>> hbase-agent.sources = tail

>>

>> hbase-agent.sinks = sink1

>>

>> hbase-agent.channels = ch1

>>

>> hbase-agent.sources.tail.type = exec

>>

>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt

>>

>> hbase-agent.sources.tail.channels = ch1

>>

>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink

>>

>> hbase-agent.sinks.sink1.channel = ch1

>>

>> hbase-agent.sinks.sink1.table = test

>>

>> hbase-agent.sinks.sink1.columnFamily = f1

>>

>> hbase-agent.sinks.sink1.serializer =

>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer

>>

>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1

>>

>> hbase-agent.sinks.sink1.serializer.keyType = timestamp

>>

>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1

>>

>> hbase-agent.sinks.sink1.serializer.suffix = timestamp

>>

>> hbase-agent.channels.ch1.type=memory

>>

>>

>>

>> Regards,

>>

>> Deepak

>

>

>

> --

> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/







--

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message