flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: checkpoint lifecycle
Date Wed, 29 Jan 2014 17:22:52 GMT
The checkpoint file itself should be fixed size though other files in that
directory may vary in size. What version of flume are you using? Newer
versions should have more files in that directory.

On Wednesday, January 29, 2014, Umesh Telang <Umesh.Telang@bbc.co.uk> wrote:

>  Hello,
>  Under a file channels checkpoint directory, I see the following files:
> checkpoint
> checkpoint.meta
>  I wanted to know whether the size of the checkpoint file should reach a
> steady state if the amount and rate of input to the file chain remains the
> same.
>  My understanding is that the checkpoint file is associate with the write
> ahead log. Is this something that continues to grow indefinitely?
>  Or is there some lifecycle management that cleans out very old entries
> from the write ahead log?
>  If not, is there some strategy that we should employ to manage the size
> of the checkpoint file (in our case, it's currently over 1GB after 2 days'
> operation).
>  Thanks for any advice on this.
>  Kind regards,
> Umesh
> ----------------------------
> http://www.bbc.co.uk
> This e-mail (and any attachments) is confidential and may contain personal
> views which are not the views of the BBC unless specifically stated.
> If you have received it in error, please delete it from your system.
> Do not use, copy or disclose the information in any way nor act in
> reliance on it and notify the sender immediately.
> Please note that the BBC monitors e-mails sent or received.
> Further communication will signify your consent to this.
> ---------------------

View raw message