flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gonzalo Herreros <gherre...@gmail.com>
Subject Re: Flume sink for writing Tar formatted files on HDFS
Date Tue, 17 Nov 2015 08:32:21 GMT
AFAIK, only the Hdfs sink supports that. So you are going to have to extend
the standard HDFSEventSink to build a tar using a library like Apache
Commons Compress.
Please note, the tar format is not really compressed but just appended, if
you want compression you need to add gzip on the tar so it becomes tar.gz


On 17 November 2015 at 00:42, R P <hadooper@outlook.com> wrote:

> Hello All,
> I am aware of flume support for writing data in  gzip, bzip2, lzo, lzop,
> snappy.
> Due to legacy code I need to write data in Tar format.  Have anybody used
> flume to write Tar compressed files?
> Any pointers will be greatly appreciated.
> Thanks,
> R

View raw message