flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: Data invisible
Date Thu, 21 Jul 2011 11:16:11 GMT
[Please subscribe to new flume-user@incubator.apache.org list, bcc
flume-user@cloudera.org, cc flume-user@incubator.apache.org]

Data in hdfs doesn't "arrive" until the file is closed or certain size
thresholds are met.  To see progress,  you can look at the status web page
on the collector node on port 35862.  (http://<collector>:35862).   If data
is arriving if the the
...Collector.GunzipDecorator.UnbatchingDecorator.AckChecksumChecker.InsistentAppend.appendSuccessses
field should be incrementing.

Jon.

On Sun, Jul 3, 2011 at 2:57 PM, TARIQ <dontariq@gmail.com> wrote:

> Could anyone tell me how to verify whether flume is aggregating the
> data to the specified directory or not???
> I am getting SUCCEEDED in the command history but the data is not
> visible when I browse the specified directory.
> I am using the following configuration -
>
> node1 : tail("/var/log/apache2/error.log") | agentSink("localhost",
> 35853);
> node2 : collectorSource(35853) | collectorSink("hdfs://namenode/user/
> flume","weblogs");
>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
View raw message