flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabio Moreira <fabiomore...@neemu.com>
Subject File channel getting huge
Date Fri, 21 Aug 2015 03:04:28 GMT
When do the file channels get clean?

My file channel folder is getting really huge.

a1.channels.c2_processing.type = file
a1.channels.c2_processing.transactionCapacity = 10000
a1.channels.c2_processing.capacity = 10000000
a1.channels.c2_processing.checkpointDir =
/usr/local/bin/neemu/flume/processing/checkpoint
a1.channels.c2_processing.dataDirs =
/usr/local/bin/neemu/flume/processing/data

*224G    processing*/data

I looked at my log errors and HDFS files and everything seems to be OK. No
errors, no missing files.

The datadir contains files from Ago/17 until today Ago/20. Can I delete the
old ones manually? Will flume complaint about it? Can I reduce the number
of files on datadirs? OR ate least compress them? I know I can set
maxFileSize.

-- 
neemu
* | Fábio Moreira . * 92 3877.9010

Mime
View raw message