flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bhaskar V. Karambelkar" <bhaska...@gmail.com>
Subject hdfs.idleTimeout ,what's it used for ?
Date Thu, 17 Jan 2013 20:07:46 GMT
Say If I have

a1.sinks.k1.hdfs.path = /flume/events/%y-%m-%d/


Now, if there is a file
This file is not ready to be rolled over yet, i.e. 60 seconds are not
up and now it's past 12 midnight, i.e. new day
And events start to be written to

will the file 2013-01-17 never be rolled over, unless I have something
like hdfs.idleTimeout=60  ?
If so how do flume sinks keep track of files they need to rollover
after idealTimeout ?

In short what's the exact use of idealTimeout parameter ?

View raw message