flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Moving from Flume OG to Flume NG
Date Fri, 05 Oct 2012 15:17:00 GMT
Does either of

'/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'

contain the hbase jars? Generally you want the hbase command to be in
the path of the user starting flume. In the hbase conf folder would be
the hbase-site.xml which would get picked up by the flume agent.

Brock

On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:
> Broke, this is the command it prints.
> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f ./conf/deepakConf/flume.conf
-name hbase-agent
> + exec /opt/sponge/java//bin/java -Xmx20m -cp '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
-Djava.library.path= org.apache.flume.node.Application -f ./conf/deepakConf/flume.conf -name
hbase-agent
>
> Regards,
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:brock@cloudera.com]
> Sent: Friday, October 05, 2012 10:35 AM
> To: user@flume.apache.org
> Cc: adams@cloudera.com
> Subject: Re: Moving from Flume OG to Flume NG
>
> Hi,
>
> When start the flume agent it prints out the command it is running to
> start the agent. Can you send that along?
>
> Brock
>
> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:
>> Hi Adam,
>>
>> I am just trying to use hbase sink with flume. It is a sort of Hello World.
>> My configuration file looks something like this. When I execute the node for
>> hbase-agent, the log shows following error:
>>
>>
>>
>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>> from HBase
>>
>>
>>
>> It seems, it looks for hbase at localhost, but the hbase is running on
>> another VM. How should I provide the hbase VM details here? For testing
>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume is
>> 1.2.
>>
>>
>>
>>
>>
>> Configuration File:
>>
>>
>>
>> hbase-agent.sources = tail
>>
>> hbase-agent.sinks = sink1
>>
>> hbase-agent.channels = ch1
>>
>> hbase-agent.sources.tail.type = exec
>>
>> hbase-agent.sources.tail.command = tail -F /opt/sponge/flumeNG/jongkook.txt
>>
>> hbase-agent.sources.tail.channels = ch1
>>
>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>> hbase-agent.sinks.sink1.channel = ch1
>>
>> hbase-agent.sinks.sink1.table = test
>>
>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>> hbase-agent.sinks.sink1.serializer =
>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>> hbase-agent.channels.ch1.type=memory
>>
>>
>>
>> Regards,
>>
>> Deepak
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message