flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Sammer <esam...@cloudera.com>
Subject Re: Problems writing to S3
Date Thu, 17 Nov 2011 20:36:59 GMT

I'm not an S3 ninja, but if I remember correctly, this can happen when an
S3 node falls behind another with respect to a copy of the data (i.e. you
can see an inconsistent picture with concurrent access). Is it possible
you're hitting different machines in S3? Does one even get that kind of
visibility into the system?

On Thu, Nov 17, 2011 at 11:23 AM, Mark Lewandowski <
mark.e.lewandowski@gmail.com> wrote:

> Has anyone had success getting flume-0.9.4 to write to S3?  Since I
> upgraded I've been having issues where I get 404s from S3 the majority of
> the time, but not all the time.  Here's my error:
> 2011-11-17 19:10:59,755 WARN
> org.jets3t.service.impl.rest.httpclient.RestS3Service: Response
> '/logs%2F2011-10-22%2F0400%2Fpath-web-20111117-190826507%2B0000.32131453853534114.00000039.tmp'
> - Unexpected response code 404, expected 200
> I've followed Eric Lubow's blog post about how to get it to work, but for
> some reason I'm still getting these errors, a lot.
> Thanks in advance,
> -Mark

Eric Sammer
twitter: esammer
data: www.cloudera.com

View raw message