flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish <paliwalash...@gmail.com>
Subject Re: Spooling director source with compressed (zip, gzip) text files
Date Mon, 10 Aug 2015 17:21:46 GMT
Related JIRA https://issues.apache.org/jira/browse/FLUME-2459

Patches are welcome :)

On Mon, Aug 10, 2015 at 4:16 AM, Phil Scala <Phil.Scala@globalrelay.net> wrote:
> I was starting to give some thought about how to spool a set of compressed
> files using the spooling directory source.  At the moment I unzip them in a
> staging script, but maybe it would be better do that in flume.   The files
> are compressed using zip, gzip etc and contain 1 or more text files within.
> Does anyone have any thoughts on if out of the box would be possible with
> flume or higher?  I am thinking that I could need to create a custom
> EventDeserializer on the spooling source that would do the decompression.
> Thoughts?
> Thanks in advance
>   Phil


Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal

View raw message