flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Stricks <...@wapolabs.com>
Subject Re: collector not writing to file/hdfs/...
Date Sat, 25 Feb 2012 18:54:31 GMT
I just went through a similar process, but using an S3 bucket as the hdfs
sink. Checking the collector's log file would be my first suggestion.

On Sat, Feb 25, 2012 at 1:37 PM, alo alt <wget.null@googlemail.com> wrote:

> Hi,
>
> what happen when you start fume with a sink to STDOUT?
>
> start flume:
> flume dump 'syslogTcp(5140)'
>
> netcat into it:
> echo "<37>hello via syslog"  | nc localhost 5140
>
> - Alex
>
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Feb 24, 2012, at 4:21 AM, Julien Ricard wrote:
>
> > Hello,
> >
> > I'm trying to understand why my collector doesn't write anything to the
> sink, file or hdfs destination.
> >
> > I'm using the cloudera release on debian squeeze.
> >
> > My setup is really simple. "web7" is an agent, and it sends to
> "collector1-flume" which should write to disk or hdfs. The collector is, at
> this time, on the same machine than the flume-master node.
> >
> > At this time, I start the collector with :
> > flume node -n collector1-flume &
> >
> > web7: syslogTcp(5140) | agentSink("collector2-flume",35853);
> > collector1-flume : collectorSource(35853) |
> collectorSink("file:///tmp","req");
> >
> > web7 listens on the port 5140 :
> > tcp        0      0 0.0.0.0:5140            0.0.0.0:*
> LISTEN
> >
> > collector1-flume listens on the port 35853 :
> > tcp        0      0 0.0.0.0:35853           0.0.0.0:*
> LISTEN
> >
> > The master says that the nodes are alive. The configurations of the
> nodes seem to be correct when I read them on the :35862/flumeagent.jsp page.
> >
> > I tried to listen to the incoming packets with netcat, the syslog is
> correctly sending to the agent, and the agent is sending to the collector,
> I don't understand what the collector is not doing correctly.
> >
> > Any idea where to look ?
> >
> > Thanks !
>
>

Mime
View raw message