flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bart Verwilst <li...@verwilst.be>
Subject Using Python and Flume to store avro data
Date Thu, 08 Nov 2012 18:45:12 GMT

I've been spending quite a few hours trying to push avro data to Flume 
so i can store it on HDFS, this all with Python.
It seems like something that is impossible for now, since the only  way 
to push avro data to Flume is by the use of deprecated thrift binding 
that look pretty cumbersome to get working.
I would like to know what's the best way to import avro data into Flume 
with Python? Maybe Flume isnt the right tool and I should use something 
else? My goal is to have multiple python workers pushing data to HDFS 
which ( by means of Flume in this case ) consolidates this all in 1 file 

Any thoughts?



View raw message