flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zaenal rifai <togatta.f...@gmail.com>
Subject Re: Flume-agent conf on ambari
Date Mon, 16 Nov 2015 08:31:15 GMT
1.Yes i get error, the error is " Agent configuration for hdfs-agent does
not contain any valid channels. Marking it as invalid "
    here is for more about the error

    http://tempel.blankon.in/1527288

2.No, i don't have enable the kerberos

3.i'm not sure about the problem, i tried another method sink and it still
failed.

can you give me some simple example for agent configuration to collect log
and write on hdfs ?






On 16 November 2015 at 14:47, Gonzalo Herreros <gherreros@gmail.com> wrote:

> I looks fine to me. Do you get any errors?
> Do you have kerberos enabled? maybe is a security issue
> Are you sure the problem is hdfs and not some netcat error? Try with
> another sink to confirm that.
>
> Regards,
> Gonzalo
>
> On 16 November 2015 at 07:35, zaenal rifai <togatta.fudo@gmail.com> wrote:
>
>> Hello guys, i'm newbie on flume, i want to ask something about
>> configuration on flume-agent which can write event on HDFS.
>>
>> for example which i can find on internet is like this.
>>
>> hdfs-agent.sources= netcat-collect
>> hdfs-agent.sinks = hdfs-write
>> hdfs-agent.channels= memory-channel
>> hdfs-agent.sources.netcat-collect.type = netcat
>> hdfs-agent.sources.netcat-collect.bind = 127.0.0.1
>> hdfs-agent.sources.netcat-collect.port = 11111
>> hdfs-agent.sinks.hdfs-write.type = hdfs
>> hdfs-agent.sinks.hdfs-write.hdfs.path =/user/user/togatta/
>> hdfs-agent.sinks.hdfs-write.rollInterval = 10
>> hdfs-agent.sinks.hdfs-write.hdfs.writeFormat=Text
>> hdfs-agent.sinks.hdfs-write.hdfs.fileType=DataStream
>> hdfs-agent.channels.memoryChannel.type = memory
>> hdfs-agent.channels.memoryChannel.capacity=10000
>> hdfs-agent.sources.netcat-collect.channels=memoryChannel
>> hdfs-agent.sinks.hdfs-write.channel=memoryChannel
>>
>> this configuration will collect about log netcat event, but when i run
>> it, it cant write on path hdfs which i had to config. So any suggest from
>> all of you guys?
>>
>
>

Mime
View raw message