flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julien Ricard <jar...@gmail.com>
Subject Re: collector not writing to file/hdfs/...
Date Wed, 29 Feb 2012 19:46:19 GMT
ok sorry : my mistake, or rsyslog's mistake, rsyslog doesn't like the use
of "localhost" in his config file, so it did not have anything to do with
flume. Rsyslog was not forwarding anything to flume.

I should sleep more :)

Thank you for your help.


On Sat, Feb 25, 2012 at 7:54 PM, Jay Stricks <jay@wapolabs.com> wrote:

> I just went through a similar process, but using an S3 bucket as the hdfs
> sink. Checking the collector's log file would be my first suggestion.
>
> On Sat, Feb 25, 2012 at 1:37 PM, alo alt <wget.null@googlemail.com> wrote:
>
>> Hi,
>>
>> what happen when you start fume with a sink to STDOUT?
>>
>> start flume:
>> flume dump 'syslogTcp(5140)'
>>
>> netcat into it:
>> echo "<37>hello via syslog"  | nc localhost 5140
>>
>> - Alex
>>
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> On Feb 24, 2012, at 4:21 AM, Julien Ricard wrote:
>>
>> > Hello,
>> >
>> > I'm trying to understand why my collector doesn't write anything to the
>> sink, file or hdfs destination.
>> >
>> > I'm using the cloudera release on debian squeeze.
>> >
>> > My setup is really simple. "web7" is an agent, and it sends to
>> "collector1-flume" which should write to disk or hdfs. The collector is, at
>> this time, on the same machine than the flume-master node.
>> >
>> > At this time, I start the collector with :
>> > flume node -n collector1-flume &
>> >
>> > web7: syslogTcp(5140) | agentSink("collector2-flume",35853);
>> > collector1-flume : collectorSource(35853) |
>> collectorSink("file:///tmp","req");
>> >
>> > web7 listens on the port 5140 :
>> > tcp        0      0 0.0.0.0:5140            0.0.0.0:*
>> LISTEN
>> >
>> > collector1-flume listens on the port 35853 :
>> > tcp        0      0 0.0.0.0:35853           0.0.0.0:*
>> LISTEN
>> >
>> > The master says that the nodes are alive. The configurations of the
>> nodes seem to be correct when I read them on the :35862/flumeagent.jsp page.
>> >
>> > I tried to listen to the incoming packets with netcat, the syslog is
>> correctly sending to the agent, and the agent is sending to the collector,
>> I don't understand what the collector is not doing correctly.
>> >
>> > Any idea where to look ?
>> >
>> > Thanks !
>>
>>
>


-- 
-- cette signature ne contient aucun gif.

Mime
View raw message