flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyotsna G <jyotsn...@gmail.com>
Subject Flume spooldir source- subdirectory issue
Date Thu, 19 Apr 2018 03:57:02 GMT
Hi All,

I am currently using Flume 1.7 . Configured a spooling directory source. I
have enabled recursiveDirectorySearch=true to look in to the sub
directories for files.


and under /tmp/test, subdirectories get created with data files
/tmp/test/data1/file.csv , /tmp/test/data2/file2.csv .

I want the exact sub directory structure to be created in the HDFS sink



When i use the %{file} for HDFS sink filepath, i get the complete absolute
path, and %{basename} gives me only the file name. I want to extract the
sub directory structure from the spooldir source path. Any way to achieve
Thanks ,

View raw message