flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alten-Lorenz <wget.n...@gmail.com>
Subject Re: Flume Sink Cleanup
Date Thu, 20 Jun 2013 15:41:15 GMT
Hi Rajesh,

You have to write a cron job, a overwrite after some time isn't possible.

Regards,
 Alex

On Jun 20, 2013, at 4:59 PM, Rajesh Jain <rjain15@gmail.com> wrote:

> Hi 
> 
> This is a newbie question. 
> 
> I have configured flume sink with the Rolling File Sink
> 
> I have the following in my flume-conf.properties page
> 
> agent1.sinks.purepath.type = com.x.diagnostics.flume.RollingFileSink
> # once an hour 
> agent1.sinks.purepath.sink.rollInterval = 3600
> # Force cutoff at 100 MB
> agent1.sinks.purepath.sink.rollSize = 100
> agent1.sinks.purepath.sink.directory = /log/export
> agent1.sinks.purepath.batchSize = 1000
> 
> The directory fills up and is there a way to clean it up. Do I have to write an external
cron job to clean up the directory, or is there a way to overwrite the files (recycle) after
a specified period - say 6 months
> 
> Thanks, Rajesh
> 

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF


Mime
View raw message