flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AD <straightfl...@gmail.com>
Subject Re: Weird interrupted exception in DirectDriver during append
Date Mon, 17 Oct 2011 18:53:04 GMT
weird i have been seeing the same thing.  Do you have the node and collector
on different hosts?  Are you using hbase by chance?

On Mon, Oct 17, 2011 at 1:48 PM, Stephen Layland
<stephen.layland@gmail.com>wrote:

> Hi, after letting flume idle for the weekend listening in on a small stream
> of live data, we noticed several of our flume collector nodes failing with
> InterruptedException's being thrown.  Logs have errors that look something
> like below.  Any idea of what's going on here and how to fix it?
>
> 2011-10-17 08:02:02,107 INFO
>> com.cloudera.flume.handlers.debug.StubbornAppendSink: append Interrupted
>> event 'flume-node.lindenlab.com' [INFO Mon Oct 17 08:01:59 UTC 2011] {
>> syslogfacility : 16 } { syslogseverity : 6 } Oct 17 01:01:59 SOME MESSAGE'
>> with error: Blocked append interrupted by rotation event
>> 2011-10-17 08:02:02,107 INFO com.cloudera.flume.handlers.rolling.RollSink:
>> closing RollSink
>> 'escapedCustomDfs("hdfs://master-node:54310/logs/raw/%Y/%m/%d/%H00","test%{rolltag}"
>> )'
>> 2011-10-17 08:02:02,109 ERROR
>> com.cloudera.flume.core.connector.DirectDriver: Closing down due to
>> exception during append calls
>> java.lang.InterruptedException
>>         at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)
>>         at
>> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)
>>         at
>> com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)
>>         at
>> com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)
>>         at
>> com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)
>> 2011-10-17 08:02:02,109 INFO
>> com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode
>> node6-22 exited with error: nulljava.lang.InterruptedException
>>         at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)
>>
>>         at
>> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)
>>         at
>> com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)
>>         at
>> com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)
>>         at
>> com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)
>> 2011-10-17 08:02:02,109 INFO
>> com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode
>> node6-22 exited with error: null
>> java.lang.InterruptedException
>>         at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)
>>         at
>> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)
>>         at
>> com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
>>         at
>> com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)
>>         at
>> com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)
>>         at
>> com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)
>>         at
>> com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)
>>         at
>> com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)
>
>
> Many thanks,
>
> -Steve
>

Mime
View raw message