flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cameron Gandevia <cgande...@gmail.com>
Subject Re: flume dying on InterruptException (nanos)
Date Wed, 19 Oct 2011 07:13:33 GMT
I think Prasad is correct, if you increase your file handles this error should go away. You
will still experience the Collector shutting down I have commented why in flume-798. 

The current solution we have taken is to roll back to the 0.9.4 branch until we can patch
the RollSink to fix our issue.

Let me know if you find any other fixes.

Sent from my iPad

On 2011-10-18, at 7:21 PM, AD <straightflush@gmail.com> wrote:

> were you able to work around it at all ?
> 
> On Tue, Oct 18, 2011 at 7:32 PM, Mingjie Lai <mjlai09@gmail.com> wrote:
> 
> May relate to these 2:
> https://issues.apache.org/jira/browse/FLUME-757
> https://issues.apache.org/jira/browse/FLUME-798
> 
> I saw it when I did some re-configures.
> 
> 
> 
> On 10/18/2011 01:55 PM, AD wrote:
> Hello,
> 
>  My collector keeps dying with the following error, is this a known
> issue? Any idea how to prevent or find out what is causing it ?  is
> format("%{nanos}" an issue ?
> 
> 2011-10-17 23:16:33,957 INFO
> com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode
> flume1-18 exited with error: null
> java.lang.InterruptedException
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(AbstractQueuedSynchronizer.java:1246)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1009)
> at com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296)
> at
> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
> at
> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java:67)
> 
> 
> source:  collectorSource("35853")
> sink:  regexAll("^([0-9.]+)\\s\\[([0-9a-zA-z\\/:
> -]+)\\]\\s([A-Z]+)\\s([a-zA-Z0-9.:]+)\\s\"([^\\s]+)\"\\s([0-9]+)\\s([0-9]+)\\s\"([^\\s]+)\"\\s\"([a-zA-Z0-9\\/()_
> -;]+)\"\\s(hit|miss)\\s([0-9.]+)","hbase_remote_host","hbase_request_date","hbase_request_method","hbase_request_host","hbase_request_url","hbase_response_status","hbase_response_bytes","hbase_referrer","hbase_user_agent","hbase_cache_hitmiss","hbase_origin_firstbyte")
> format("%{nanos}:") split(":", 0, "hbase_") format("%{node}:")
> split(":",0,"hbase_node") digest("MD5","hbase_md5") collector(10000) {
> attr2hbase("apache_logs","f1","","hbase_") }
> 

Mime
View raw message