From flume-user-return-376-apmail-incubator-flume-user-archive=incubator.apache.org@incubator.apache.org Wed Oct 19 02:14:12 2011 Return-Path: X-Original-To: apmail-incubator-flume-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-flume-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2981E7F5B for ; Wed, 19 Oct 2011 02:14:12 +0000 (UTC) Received: (qmail 50457 invoked by uid 500); 19 Oct 2011 02:14:12 -0000 Delivered-To: apmail-incubator-flume-user-archive@incubator.apache.org Received: (qmail 50435 invoked by uid 500); 19 Oct 2011 02:14:12 -0000 Mailing-List: contact flume-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: flume-user@incubator.apache.org Delivered-To: mailing list flume-user@incubator.apache.org Received: (qmail 50427 invoked by uid 99); 19 Oct 2011 02:14:12 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 02:14:12 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of straightflush@gmail.com designates 209.85.161.175 as permitted sender) Received: from [209.85.161.175] (HELO mail-gx0-f175.google.com) (209.85.161.175) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Oct 2011 02:14:03 +0000 Received: by ggnq1 with SMTP id q1so1216581ggn.6 for ; Tue, 18 Oct 2011 19:13:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=rarlfXziV9LLnnxE7r+xWnfBZDK7rsQnoFF9y2K/BFE=; b=Zl5OO7hMf2w8qyAj9C5XHg8vBHODdB9rbkdeismPv+yPL/z6s34r66IftRYbGNPbyT 4xoSc4KV1DjlS0ZO8SbabxyS+H/oYf8Bjcxml/iJk+pKBKw6WgJEba8yRmKBtRf7/F+5 2US2UoVFLEyeOBmrHV+yaF4ExbcjnLjIngTZ4= Received: by 10.236.190.193 with SMTP id e41mr7012026yhn.118.1318990423176; Tue, 18 Oct 2011 19:13:43 -0700 (PDT) Received: from mail-yw0-f47.google.com (mail-yw0-f47.google.com [209.85.213.47]) by mx.google.com with ESMTPS id 44sm6219769yhs.18.2011.10.18.19.13.41 (version=SSLv3 cipher=OTHER); Tue, 18 Oct 2011 19:13:42 -0700 (PDT) Received: by ywm39 with SMTP id 39so1205839ywm.6 for ; Tue, 18 Oct 2011 19:13:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.43.50.201 with SMTP id vf9mr3526755icb.10.1318990420789; Tue, 18 Oct 2011 19:13:40 -0700 (PDT) Received: by 10.182.197.70 with HTTP; Tue, 18 Oct 2011 19:13:40 -0700 (PDT) In-Reply-To: References: Date: Tue, 18 Oct 2011 22:13:40 -0400 Message-ID: Subject: Re: flume dying on InterruptException (nanos) From: AD To: flume-user@incubator.apache.org Content-Type: multipart/alternative; boundary=bcaec529a0a17e49c904af9d6378 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec529a0a17e49c904af9d6378 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable hmm not yet, will try though. On Tue, Oct 18, 2011 at 8:20 PM, Prasad Mujumdar wrot= e: > > Have you tried increasing the maximum number of open file on the > system ? > > thanks > Prasad > > > > On Tue, Oct 18, 2011 at 1:55 PM, AD wrote: > >> Hello, >> >> My collector keeps dying with the following error, is this a known issu= e? >> Any idea how to prevent or find out what is causing it ? is >> format("%{nanos}" an issue ? >> >> 2011-10-17 23:16:33,957 INFO >> com.cloudera.flume.core.connector.DirectDriver: Connector logicalNode >> flume1-18 exited with error: null >> java.lang.InterruptedException >> at >> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireNanos(Ab= stractQueuedSynchronizer.java:1246) >> at >> java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.tryLock(Reen= trantReadWriteLock.java:1009) >> at com.cloudera.flume.handlers.rolling.RollSink.close(RollSink.java:296) >> at >> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java= :67) >> at >> com.cloudera.flume.core.EventSinkDecorator.close(EventSinkDecorator.java= :67) >> >> >> source: collectorSource("35853") >> sink: regexAll("^([0-9.]+)\\s\\[([0-9a-zA-z\\/: >> -]+)\\]\\s([A-Z]+)\\s([a-zA-Z0-9.:]+)\\s\"([^\\s]+)\"\\s([0-9]+)\\s([0-9= ]+)\\s\"([^\\s]+)\"\\s\"([a-zA-Z0-9\\/()_ >> -;]+)\"\\s(hit|miss)\\s([0-9.]+)","hbase_remote_host","hbase_request_dat= e","hbase_request_method","hbase_request_host","hbase_request_url","hbase_r= esponse_status","hbase_response_bytes","hbase_referrer","hbase_user_agent",= "hbase_cache_hitmiss","hbase_origin_firstbyte") >> format("%{nanos}:") split(":", 0, "hbase_") format("%{node}:") >> split(":",0,"hbase_node") digest("MD5","hbase_md5") collector(10000) { >> attr2hbase("apache_logs","f1","","hbase_") } >> > > --bcaec529a0a17e49c904af9d6378 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable hmm not yet, will try though.

On Tue, Oct= 18, 2011 at 8:20 PM, Prasad Mujumdar <prasadm@cloudera.com> wrote:

=A0=A0=A0=A0=A0 Have you tried increasing the maximum number of open fi= le on the system ?

thanks
Prasad



On T= ue, Oct 18, 2011 at 1:55 PM, AD <straightflush@gmail.com> wrote:
Hello,

=A0My collector ke= eps dying with the following error, is this a known issue? Any idea how to = prevent or find out what is causing it ? =A0is format("%{nanos}" = an issue ?

2011-10-17 23:16:33,957 INFO com.cloudera.flume.core.connector.DirectD= river: Connector logicalNode flume1-18 exited with error: null
ja= va.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquir= eNanos(AbstractQueuedSynchronizer.java:1246)
at java.util.concurrent.l= ocks.ReentrantReadWriteLock$WriteLock.tryLock(ReentrantReadWriteLock.java:1= 009)
at com.cloudera= .flume.handlers.rolling.RollSink.close(RollSink.java:296)
at com.cloudera.flume.cor= e.EventSinkDecorator.close(EventSinkDecorator.java:67)
at com.cloudera.flume.core.EventSinkDeco= rator.close(EventSinkDecorator.java:67)


source: =A0collectorSource("3= 5853")
sink: =A0regexAll("^([0-9.]+)\\s\\[([0-9a-zA-z\\= /: -]+)\\]\\s([A-Z]+)\\s([a-zA-Z0-9.:]+)\\s\"([^\\s]+)\"\\s([0-9]= +)\\s([0-9]+)\\s\"([^\\s]+)\"\\s\"([a-zA-Z0-9\\/()_ -;]+)\&q= uot;\\s(hit|miss)\\s([0-9.]+)","hbase_remote_host","hba= se_request_date","hbase_request_method","hbase_request_= host","hbase_request_url","hbase_response_status",= "hbase_response_bytes","hbase_referrer","hbase_use= r_agent","hbase_cache_hitmiss","hbase_origin_firstbyte&= quot;) format("%{nanos}:") split(":", 0, "hbase_&q= uot;) format("%{node}:") split(":",0,"hbase_node&q= uot;) digest("MD5","hbase_md5") collector(10000) { attr= 2hbase("apache_logs","f1","","hbase_&quo= t;) }


--bcaec529a0a17e49c904af9d6378--