flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Layland <stephen.layl...@gmail.com>
Subject is collectorSink(dest, prefix, millis, format) broken or am i stupid?
Date Fri, 16 Sep 2011 23:16:49 GMT

Forgive the n00b question, but I'm trying to benchmark flume while building
out a hadoop based central log store and am coming across some weirdness.
 The flume-conf.xml has the default flume.collector.output.format set to
'avrojson'.  I had two simple configs:

test1: syslogTcp(5140) | collectorSink("hdfs://...", "test", 30000,
test2: syslogTcp(5140) | collectorSink("hdfs://...", "test", 30000, "raw")

I then mapped a test flume node to each of these logical nodes in turn (exec
map node1 test1; exec refreshAll) and tested it out but the actual dfs files
are all appear to be the same size and all appear to be avronjson?

Am I doing something wrong here?

Using flume version: 0.9.4-cdh3u1.



View raw message