flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prasad Mujumdar <pras...@cloudera.com>
Subject Re: fix for hanging collector node
Date Tue, 24 Jan 2012 18:25:50 GMT
    yes, looks like you are running into FLUME-798, so these two patches
and configured timeout property should resolve the problem.
If you don't want to build the patch, then you can use flume from the
upcoming CDH3 update3 which should be available by end of the month.

thanks
Prasad



On Tue, Jan 24, 2012 at 2:12 AM, Søren <sd@syntonetic.com> wrote:

> Dear flume community
>
> We have encountered a serious problem in our collector node (flume 0.9.4).
> After browsing the mail group it seems to be a well known issue:
> ..........
> 2012-01-23 15:26:31,611 INFO com.cloudera.flume.handlers.**debug.StubbornAppendSink:
> append Interrupted event '[eventcontent]' with error: Blocked append
> interrupted by rotation event
> 2012-01-23 15:26:31,612 INFO com.cloudera.flume.handlers.**rolling.RollSink:
> closing RollSink 'escapedCustomDfs("s3n://**flume-log/eventlog/","el-%{**rolltag}"
> )'
> 2012-01-23 15:26:31,612 ERROR com.cloudera.flume.core.**connector.DirectDriver:
> Closing down due to exception during append calls
> 2012-01-23 15:26:31,612 INFO com.cloudera.flume.core.**connector.DirectDriver:
> Connector logicalNode CollectorEventlog-22 exited with error: null
> .........
>
> We are using S3 sink which supports the theory that the the node is
> sensitive to timeout or similar from the sink.
> The bug should be solved with:
> + patch FLUME-762
> + patch FLUME-798
> + setting flume.collector.roll.timeout to 0
>
> Do I get it right so far?
>
> We have installed flume via apt, and haven't dealed with other than the
> binary build of flume until now.
> Are there updated binaries out there?
> If not, what is the easiest and safest way to apply those patches?
>
> Thanks for a great product anyway. We are looking forward to flume-ng with
> support for thrift source/S3 sink.
>
> Thanks in advance
> Søren
>
>

Mime
View raw message