flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Frech <m...@hooklogic.com>
Subject Need a flume-core.jar that will write to S3 and HDFS for CDH Update 3
Date Mon, 19 Mar 2012 16:43:06 GMT
I'm using running Cloudera's flume(0.9.4-cdh3u3) to write to S3 and HDFS (Hadoop 0.20.2-cdh3u3).
 I'm able to get one or the other to work based on the version of hadoop-core.jar I use.

In order to get rid of my "Unexpected response code 404, expected 200" errors when writing
to S3,  I'm trying to use emr-hadoop-core-0.20.jar file from an EC2 distribution of hadoop
as suggested in this article:

I used the hadoop-core.jar file linked to by the author here:

That fixes the S3 writes, but I suspect I need a newer version of that file because when I
try to write to hdfs with the updated jar file, I get this error in my hadoop NAMENODE log
file:  "WARN org.apache.hadoop.ipc.Server: Incorrect header or version mismatch from
got version 3 expected version 4".

My question is, am I on the right track to solving this?  And if so, what is the most reliable
way to get an updated version of flume-core.jar that will work with my version of flume/hadoop?

View raw message