You can use roundValue as 15 and roundUnit as Minute - so a new dir would get created every 15 mins.


Thanks,
Hari

On Tue, Sep 8, 2015 at 9:42 AM, Sutanu Das <sd2302@att.com> wrote:

Hi Community,

 

Is it possible to create /write HDFS dirs./folders every 15 mins?

 

Right now, we use /hdfs_sink_path/%Y/%m/%d/%H --- ie using every Hour, But, Is it possible to write every 15 mins? – not every minute…

 

If this is not possible out of the box config, can DEV team please advise which code/java-file do we need to customize so Flume creates dirs. Every 15 mins? Like /%Y/%m/%d/%H/%15M ?

 

Please help us, thanks.

 

Here is our present 1 hour HDFS dir/folder creation config:

 

 

airwave.sinks.sink1.type = hdfs

airwave.sinks.sink1.channel = channel1

airwave.sinks.sink1.hdfs.path = /prod/hadoop/smallsite/airwave_flume_ingest_hak_3_amp_1/%Y/%m/%d/%H

airwave.sinks.sink1.hdfs.fileType = DataStream

airwave.sinks.sink1.hdfs.writeFormat = Text

airwave.sinks.sink1.hdfs.filePrefix = Airwave_amp_1

airwave.sinks.sink1.hdfs.useLocalTimeStamp = true

airwave.sinks.sink1.hdfs.round = true

airwave.sinks.sink1.hdfs.roundValue = 1

airwave.sinks.sink1.hdfs.roundUnit = hour

airwave.sinks.sink1.hdfs.callTimeout = 10000000000