flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Himanshu Patidar <himanshu.pati...@hotmail.com>
Subject Flume - Custom HDFS Sink does not write to hdfs unless killed
Date Mon, 03 Mar 2014 09:11:40 GMT
I have a custom HDFS Sink which takes events, parse them (convert binary data to .avro files)
and then writes these files to different directories in hdfs. Trying to do so I get a strange
error - Only the last avro file gets written and to hdfs and rest of the files show a size
of 0 kbs untill I kill my flume agent (Ctrl + C). As soon as I kill the script, I can see
the data in rest of the .avro files. I am using flume 1.4 cdh 4.0.0 with hdfs 2.0.0.Can anyone
please suggest some solution?
View raw message