flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brock Noland <br...@cloudera.com>
Subject Re: Moving from Flume OG to Flume NG
Date Tue, 09 Oct 2012 14:51:39 GMT
Great to hear, that's interesting it did not work in the lib directory.

On Tue, Oct 9, 2012 at 1:03 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:
> Brock,
> I copied hbase-site.xml in conf & is working fine now.
>
> Thanks,
>
> Deepak
>
> -----Original Message-----
> From: Brock Noland [mailto:brock@cloudera.com]
> Sent: Monday, October 08, 2012 12:03 PM
> To: user@flume.apache.org
> Cc: adams@cloudera.com
> Subject: Re: Moving from Flume OG to Flume NG
>
> Deepak,
>
> What you have should work. Could you post your full log file on
> pastebin.com or send it to me privately? (List doesn't take it)
>
> Brock
>
> On Fri, Oct 5, 2012 at 10:28 AM, Kumar, Deepak8 <deepak8.kumar@citi.com> wrote:
>> Brock,
>>
>> Apologize miss spelled your name last time. Hbase.jar as well as
>> hbase-site.xml is there in /opt/sponge/fluming/flume/lib directory. Could
>> you please guide me, how to provide hbase command while starting flume?
>>
>> Regards,
>>
>> Deepak
>>
>>
>>
>> -----Original Message-----
>> From: Brock Noland [mailto:brock@cloudera.com]
>> Sent: Friday, October 05, 2012 11:17 AM
>> To: user@flume.apache.org
>> Cc: adams@cloudera.com
>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>
>>
>> Does either of
>>
>>
>>
>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>
>>
>>
>> contain the hbase jars? Generally you want the hbase command to be in
>>
>> the path of the user starting flume. In the hbase conf folder would be
>>
>> the hbase-site.xml which would get picked up by the flume agent.
>>
>>
>>
>> Brock
>>
>>
>>
>> On Fri, Oct 5, 2012 at 10:08 AM, Kumar, Deepak8 <deepak8.kumar@citi.com>
>> wrote:
>>
>>> Broke, this is the command it prints.
>>
>>> [root@vm-93ff-5f31 flume]# bin/flume-ng agent --conf ./conf/deepakConf/ -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>> + exec /opt/sponge/java//bin/java -Xmx20m -cp
>>> '/opt/sponge/flumeNG/flume/conf/deepakConf:/opt/sponge/flumeNG/flume/lib/*'
>>> -Djava.library.path= org.apache.flume.node.Application -f
>>> ./conf/deepakConf/flume.conf -name hbase-agent
>>
>>>
>>
>>> Regards,
>>
>>> Deepak
>>
>>>
>>
>>> -----Original Message-----
>>
>>> From: Brock Noland [mailto:brock@cloudera.com]
>>
>>> Sent: Friday, October 05, 2012 10:35 AM
>>
>>> To: user@flume.apache.org
>>
>>> Cc: adams@cloudera.com
>>
>>> Subject: Re: Moving from Flume OG to Flume NG
>>
>>>
>>
>>> Hi,
>>
>>>
>>
>>> When start the flume agent it prints out the command it is running to
>>
>>> start the agent. Can you send that along?
>>
>>>
>>
>>> Brock
>>
>>>
>>
>>> On Fri, Oct 5, 2012 at 6:19 AM, Kumar, Deepak8 <deepak8.kumar@citi.com>
>>> wrote:
>>
>>>> Hi Adam,
>>
>>>>
>>
>>>> I am just trying to use hbase sink with flume. It is a sort of Hello
>>>> World.
>>
>>>> My configuration file looks something like this. When I execute the node
>>>> for
>>
>>>> hbase-agent, the log shows following error:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> 2012-10-05 06:55:05,330 ERROR hbase.HBaseSink: Could not load table, test
>>
>>>> from HBase
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> It seems, it looks for hbase at localhost, but the hbase is running on
>>
>>>> another VM. How should I provide the hbase VM details here? For testing
>>
>>>> purpose, I have created a hbase-site.xml at location flume/lib/ but it
>>
>>>> doesn't makes any different. My Hbase version is 0.92.1-cdh4.0.1 & Flume
>>>> is
>>
>>>> 1.2.
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> Configuration File:
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> hbase-agent.sources = tail
>>
>>>>
>>
>>>> hbase-agent.sinks = sink1
>>
>>>>
>>
>>>> hbase-agent.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sources.tail.type = exec
>>
>>>>
>>
>>>> hbase-agent.sources.tail.command = tail -F
>>>> /opt/sponge/flumeNG/jongkook.txt
>>
>>>>
>>
>>>> hbase-agent.sources.tail.channels = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.channel = ch1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.table = test
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.columnFamily = f1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.serializer =
>>
>>>> org.apache.flume.sink.hbase.SimpleHbaseEventSerializer
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.serializer.payloadColumn = col1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.serializer.keyType = timestamp
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.serializer.rowPrefix = 1
>>
>>>>
>>
>>>> hbase-agent.sinks.sink1.serializer.suffix = timestamp
>>
>>>>
>>
>>>> hbase-agent.channels.ch1.type=memory
>>
>>>>
>>
>>>>
>>
>>>>
>>
>>>> Regards,
>>
>>>>
>>
>>>> Deepak
>>
>>>
>>
>>>
>>
>>>
>>
>>> --
>>
>>> Apache MRUnit - Unit testing MapReduce -
>>> http://incubator.apache.org/mrunit/
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/



-- 
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/

Mime
View raw message