flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: FileChannel Replays consistently take a long time
Date Wed, 20 Aug 2014 04:02:03 GMT
How large is your channel (and how long does it take to replay?)

Gary Malouf wrote:
>
> For the record, we are using Flume 1.4.0 packaged with CDH5.0.2
>
>
> On Tue, Aug 19, 2014 at 11:55 PM, Gary Malouf <malouf.gary@gmail.com
> <mailto:malouf.gary@gmail.com>> wrote:
>
> We are repeatedly running into cases where the replays of from a
> file channel going to HDFS take an eternity.
>
> I've read this thread
> <http://mail-archives.apache.org/mod_mbox/flume-dev/201306.mbox/%3CCAHbPYVbmeD6pKZkDADmyaW_gC_p7cQDefPsycWKnkY72TFiAuQ@mail.gmail.com%3E>,
> but I just am not convinced that our checkpoints are constantly
> being corrupted.
>
> We are seeing messages such as:
>
> 20 Aug 2014 03:52:26,849 INFO [lifecycleSupervisor-1-2]
> (org.apache.flume.channel.file.EventQueueBackingStoreFileV3.<init>:57)
>
> - Reading checkpoint metadata from
> /opt/flume/brq/ch1/checkpoint/checkpoint.meta
>
>
> How can it be that this takes so long?
>

Mime
View raw message