flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Sharma <saurabh.sha...@nviz.com>
Subject Flume spool directory source issue
Date Thu, 25 Feb 2016 07:30:03 GMT
Hi,

We are using Flume Version 1.6.0. We are facing the following issue  with SpoolingDirectory
source.
The scenario is when we place a file with name i.e. 'x' in spool directory then file is read
and renamed as 'x.COMPLETED'.
If we place the same file again in the spooling directory(without deleting the 'x.COMPLETED'),
the file is processed again.
But on placing the same file for a third time, the file is not processed.

We observed that the file is read for the second time  because after reading/processing the
file for the first time, the meta file created for the same in .flumespool directory gets
deleted.
But after the file is read for the second time, the metafile created for the same is not deleted
after processing, hence when the file is read for the third time does not get processed.

Is it a required behaviour. Is there a way to stop processing the same file again.

Thanks

Mime
View raw message