I have been experiencing a similar error and notice it only happens when I have a large number of files open to hdfs. I am running some tests removing the bucketing to see if I can send the same files. I will let you know if I come across anything. I have around 50 nodes writing to a single collector. This error consistently happens within 10 mins of starting my collector.<div>

<br></div><div>My exception.<br><div><br></div><div><div><div>2011-10-17 17:30:07,173 [Roll-TriggerThread-0] INFO  com.cloudera.flume.handlers.hdfs.CustomDfsSink - done writing raw file to hdfs</div><div>2011-10-17 17:30:07,189 [logicalNode collector0_log_dir-19] ERROR com.cloudera.flume.core.connector.DirectDriver - Closing down due to exception during append calls</div>

<div>2011-10-17 17:30:07,190 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.core.connector.DirectDriver - Connector logicalNode collector0_log_dir-19 exited with error: Blocked append interrupted by rotation event</div>

<div>java.lang.InterruptedException: Blocked append interrupted by rotation event</div><div>        at com.cloudera.flume.handlers.rolling.RollSink.append(RollSink.java:209)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div>

<div>        at com.cloudera.flume.core.MaskDecorator.append(MaskDecorator.java:43)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div><div>        at com.cloudera.flume.handlers.debug.InsistentOpenDecorator.append(InsistentOpenDecorator.java:169)</div>

<div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div><div>        at com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:71)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div>

<div>        at com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div>

<div>        at com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div><div>        at com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)</div>

<div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div><div>        at com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)</div><div>        at com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)</div>

<div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div><div>        at com.cloudera.flume.core.extractors.DateExtractor.append(DateExtractor.java:129)</div><div>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)</div>

<div>        at com.cloudera.flume.core.extractors.RegexExtractor.append(RegexExtractor.java:88)</div><div>        at com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:133)</div><div>2011-10-17 17:30:07,191 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.collector.CollectorSource - closed</div>

<div>2011-10-17 17:30:08,191 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.handlers.thrift.ThriftEventSource - Closed server on port 36892...</div><div>2011-10-17 17:30:08,191 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.handlers.thrift.ThriftEventSource - Queue still has 1000 elements ...</div>

<div>2011-10-17 17:30:18,200 [logicalNode collector0_log_dir-19] WARN  com.cloudera.flume.handlers.thrift.ThriftEventSource - Close timed out due to no progress.  Closing despite having 1000 values still enqueued</div><div>

2011-10-17 17:30:18,200 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.handlers.rolling.RollSink - closing RollSink &#39;escapedCustomDfs(&quot;hdfs://<a href="http://van-mang-perf-hadoop-namenode1.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00">van-mang-perf-hadoop-namenode1.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00</a>&quot;,&quot;raw-%{rolltag}&quot; )&#39;</div>

<div>2011-10-17 17:30:18,200 [logicalNode collector0_log_dir-19] INFO  com.cloudera.flume.handlers.rolling.RollSink - double close &#39;escapedCustomDfs(&quot;hdfs://<a href="http://van-mang-perf-hadoop-namenode1.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00">van-mang-perf-hadoop-namenode1.net:8020/rawLogs/%{dateyear}-%{datemonth}-%{dateday}/%{datehr}00</a>&quot;,&quot;raw-%{rolltag}&quot; )&#39;</div>

<div>2011-10-17 17:30:18,200 [logicalNode collector0_log_dir-19] ERROR com.cloudera.flume.core.connector.DirectDriver - Exiting driver logicalNode collector0_log_dir-19 in error state CollectorSource | RegexExtractor because Blocked append interrupted by rotation event</div>

</div><div><br></div><br><div class="gmail_quote">On Mon, Oct 17, 2011 at 12:47 PM, Stephen Layland <span dir="ltr">&lt;<a href="mailto:stephen.layland@gmail.com">stephen.layland@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">

Hi, we&#39;re not actually using flume nodes, but just the collector at the moment.  We&#39;re listening on a syslog port and dumping straight to HDFS for now.  After some digging, I&#39;m pretty sure it&#39;s related to this:<div>



<br></div><div><a href="https://issues.apache.org/jira/browse/FLUME-757" target="_blank">https://issues.apache.org/jira/browse/FLUME-757</a></div><div><br></div><div>-Steve</div><div><div></div><div class="h5"><div><br><br>

<div class="gmail_quote">On Mon, Oct 17, 2011 at 11:53 AM, AD <span dir="ltr">&lt;<a href="mailto:straightflush@gmail.com" target="_blank">straightflush@gmail.com</a>&gt;</span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">weird i have been seeing the same thing.  Do you have the node and collector on different hosts?  Are you using hbase by chance?<div>



<div></div><div><br><br><div class="gmail_quote">On Mon, Oct 17, 2011 at 1:48 PM, Stephen Layland <span dir="ltr">&lt;<a href="mailto:stephen.layland@gmail.com" target="_blank">stephen.layland@gmail.com</a>&gt;</span> wrote:<br>




<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi, after letting flume idle for the weekend listening in on a small stream of live data, we noticed several of our flume collector nodes failing with InterruptedException&#39;s being thrown.  Logs have errors that look something like below.  Any idea of what&#39;s going on here and how to fix it?<div>






<br></div><div><blockquote class="gmail_quote" style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0.8ex;border-left-width:1px;border-left-color:rgb(204, 204, 204);border-left-style:solid;padding-left:1ex">






2011-10-17 08:02:02,107 INFO com.cloudera.flume.handlers.debug.StubbornAppendSink: append Interrupted event &#39;<a href="http://flume-node.lindenlab.com" target="_blank">flume-node.lindenlab.com</a>&#39; [INFO Mon Oct 17 08:01:59 UTC 2011] { syslogfacility : 16 } { syslogseverity : 6 } Oct 17 01:01:59 SOME MESSAGE&#39; with error: Blocked append interrupted by rotation event<br>






2011-10-17 08:02:02,107 INFO com.cloudera.flume.handlers.rolling.RollSink: closing RollSink &#39;escapedCustomDfs(&quot;hdfs://master-node:54310/logs/raw/%Y/%m/%d/%H00&quot;,&quot;test%{rolltag}&quot; )&#39;<br>2011-10-17 08:02:02,109 ERROR com.cloudera.flume.core.connector.DirectDriver: Closing down due to exception during append calls<br>






java.lang.InterruptedException<br>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)<br>        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)<br>






        at com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>






        at com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>






        at com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60) <br>        at com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)<br>        at com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)<br>






        at com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)<br>2011-10-17 08:02:02,109 INFO com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode node6-22 exited with error: nulljava.lang.InterruptedException<br>






        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)        <br>        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)<br>






        at com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>






        at com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>






        at com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)<br>        at com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)<br>






        at com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)<br>2011-10-17 08:02:02,109 INFO com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode node6-22 exited with error: null<br>






java.lang.InterruptedException<br>        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1223)<br>        at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:976)<br>






        at com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>






        at com.cloudera.flume.handlers.debug.InsistentOpenDecorator.close(InsistentOpenDecorator.java:175)<br>        at com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)<br>        at com.cloudera.flume.handlers.debug.StubbornAppendSink.append(StubbornAppendSink.java:78)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.debug.InsistentAppendDecorator.append(InsistentAppendDecorator.java:110)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>






        at com.cloudera.flume.handlers.endtoend.AckChecksumChecker.append(AckChecksumChecker.java:113)<br>        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.batch.UnbatchingDecorator.append(UnbatchingDecorator.java:62)<br>






        at com.cloudera.flume.core.EventSinkDecorator.append(EventSinkDecorator.java:60)<br>        at com.cloudera.flume.handlers.batch.GunzipDecorator.append(GunzipDecorator.java:81)<br>        at com.cloudera.flume.collector.CollectorSink.append(CollectorSink.java:222)<br>






        at com.cloudera.flume.core.connector.DirectDriver$PumperThread.run(DirectDriver.java:110)</blockquote><div><div><br></div></div></div><div>Many thanks,</div><div><br></div><div>-Steve</div>
</blockquote></div><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Thanks<br><br>Cameron Gandevia<br>
</div></div>