flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil Scala <Phil.Sc...@globalrelay.net>
Subject Spooling director source with compressed (zip, gzip) text files
Date Mon, 10 Aug 2015 11:16:41 GMT
I was starting to give some thought about how to spool a set of compressed files using the
spooling directory source.  At the moment I unzip them in a staging script, but maybe it would
be better do that in flume.   The files are compressed using zip, gzip etc and contain 1 or
more text files within.  Does anyone have any thoughts on if out of the box would be possible
with flume 1.5.0.1 or higher?  I am thinking that I could need to create a custom EventDeserializer
on the spooling source that would do the decompression.

Thoughts?

Thanks in advance
  Phil

Mime
View raw message